Electronic device, method, and non-transitory computer-readable storage medium for displaying 3D representation of object

Abstract

This electronic device comprises at least one processor including a processing circuit, a display, and a memory storing one or more programs configured to be individually or collectively executed by the at least one processor and including one or more storage media. The one or more programs may include instructions for instructing the electronic device to: display a 2D image including an object through the display; generate a 3D representation of the object by using the 2D image on the basis of a first input for generating the 3D representation; display a UI including a first UI object and the 3D representation through the display on the basis of a second input for displaying the 3D representation; determine the pose of the 3D representation by changing the pose of the 3D representation on the basis of a user input received using the first UI object; and display a second UI object and a 2D representation of the object having the determined pose in the UI through the display.

Description

Electronic device, method, and non-transient computer-readable storage medium for displaying a 3D representation of an object

[0001] The present disclosure relates to an electronic device, a method, and a non-transient computer-readable storage medium for displaying a 3D representation of an object.

[0002] Artificial intelligence is a technology for simulating human (or biological) neural activities such as perception and / or inference, and can be implemented as hardware, software, or a combination thereof designed to perform calculations for simulating neural activities.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] An electronic device is described. The electronic device may include at least one processor comprising a processing circuit, a display, and a memory comprising one or more storage media configured to store one or more programs configured to be executed individually or collectively by the at least one processor. The one or more programs may include instructions that cause the electronic device to display a 2D image containing an object through the display. The one or more programs may include instructions that cause the electronic device to receive a first input for generating a 3D representation of the object. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the 2D image containing the object based on the first input. The one or more programs may include instructions that cause the electronic device to receive a second input for displaying the 3D representation of the object. The above one or more programs may include instructions that cause the electronic device to display a UI including a first UI object and the 3D representation of the object through the display based on the second input. The first UI object may be available to change the pose of the 3D representation of the object. The above one or more programs may include instructions that cause the electronic device to determine the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object.The above one or more programs may include instructions that cause the electronic device to display a second UI object and a 2D representation of the object having the determined pose within the UI through the display. The second UI object may be available to change the position where the 2D representation of the object is displayed.

[0005] A method is described. The method may be performed within an electronic device including a display. The method may include an operation of displaying a 2D image containing an object through the display. The method may include an operation of receiving a first input for generating a 3D representation of the object. The method may include an operation of generating the 3D representation of the object using the 2D image containing the object based on the first input. The method may include an operation of receiving a second input for displaying the 3D representation of the object. The method may include an operation of displaying a UI including a first UI object and the 3D representation of the object through the display based on the second input. The first UI object may be available to change the pose of the 3D representation of the object. The method may include an operation of determining the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object. The above method may include an operation of displaying a second UI object and a 2D representation of the object having the determined pose within the UI through the display. The second UI object may be available to change the position where the 2D representation of the object is displayed.

[0006] A non-transient computer-readable storage medium is described. The non-transient computer-readable storage medium may store one or more programs. The one or more programs may include instructions that cause the electronic device to display a 2D image containing an object through the display when executed by the electronic device including a display. The one or more programs may include instructions that cause the electronic device to receive a first input for generating a 3D representation of the object when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the 2D image containing the object based on the first input when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to receive a second input for displaying the 3D representation of the object when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to display a UI including a first UI object and the 3D representation of the object through the display, based on the second input when executed by the electronic device. The first UI object may be available to change the pose of the 3D representation of the object. The above one or more programs may include instructions that cause the electronic device to determine the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object when executed by the electronic device.The above one or more programs may include instructions that cause the electronic device to display a second UI object and a 2D representation of the object having the determined pose within the UI through the display when executed by the electronic device. The second UI object may be available to change the position where the 2D representation of the object is displayed.

[0007] An electronic device is described. The electronic device may include at least one processor comprising a processing circuit, a display, and a memory comprising one or more storage media configured to store one or more programs configured to be executed individually or collectively by the at least one processor. The one or more programs may include instructions that cause the electronic device to display a 2D image containing an object viewed from a first direction through the display. The one or more programs may include instructions that cause the electronic device to receive an input for generating a 3D representation of the object using the 2D image containing the object. The one or more programs may include instructions that cause the electronic device to obtain a first 2D representation of the object viewed from the first direction from the 2D image based on the input. The one or more programs may include instructions that cause the electronic device to identify one or more images containing the object among images stored in the electronic device. The above one or more programs may include instructions that cause the electronic device to obtain a second 2D representation of the object viewed from at least a portion of the one or more images in a second direction different from the first direction. The above one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object.

[0008] A method is described. The method may be performed within an electronic device including a display. The method may include an operation of displaying a 2D image including an object viewed from a first direction through the display. The method may include an operation of receiving an input for generating a 3D representation of the object using the 2D image including the object. The method may include an operation of obtaining a first 2D representation of the object viewed from the first direction from the 2D image based on the input. The method may include an operation of identifying one or more images each including the object among images stored within the electronic device. The method may include an operation of obtaining a second 2D representation of the object viewed from a second direction different from the first direction from at least some of the one or more images. The method may include an operation of generating the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object.

[0009] A non-transient computer-readable storage medium is described. The non-transient computer-readable storage medium may store one or more programs. The one or more programs may include instructions that cause the electronic device to display a 2D image containing an object viewed from a first direction through the display when executed by the electronic device including a display. The one or more programs may include instructions that cause the electronic device to receive input to generate a 3D representation of the object using the 2D image containing the object when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to obtain a first 2D representation of the object viewed from the first direction from the 2D image based on the input when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to identify one or more images containing the object among images stored within the electronic device when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to obtain a second 2D representation of the object viewed from at least a portion of the one or more images in a second direction different from the first direction when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object when executed by the electronic device.

[0010] Figure 1 illustrates an example of generating another 2D representation of an object using a 2D image containing the object.

[0011] Figure 2 is a simplified block diagram of an exemplary electronic device.

[0012] FIG. 3 is a flowchart illustrating exemplary operations of an electronic device for generating a 3D representation of an object using a 2D image containing the object.

[0013] Figure 4 illustrates an example of input for generating a 3D representation of an object.

[0014] FIG. 5 illustrates an example of user input authorizing identification of whether images stored in an electronic device contain an object.

[0015] FIG. 6 illustrates an example of one or more images, each containing an object, among images stored in an electronic device.

[0016] Figure 7 illustrates an example of generating a 3D representation of an object using a trained model.

[0017] Figure 8 illustrates an example of a window indicating the creation of a 3D representation of an object.

[0018] FIG. 9 is a flowchart illustrating exemplary operations of an electronic device for determining an image to be used to generate a 3D representation of an object among a plurality of images.

[0019] FIG. 10 is a flowchart illustrating exemplary operations of an electronic device for generating different 3D representations of an object.

[0020] Figure 11 illustrates an example of an input for generating different 3D representations of an object.

[0021] FIG. 12 illustrates an example of a window to acquire an image containing an object viewed from a reference direction.

[0022] FIG. 13 is a flowchart illustrating exemplary operations of an electronic device for displaying a 3D representation of an object and a 2D representation of an object having a determined pose.

[0023] FIGS. 14a and FIGS. 14b illustrate examples of user input for changing the pose of a 3D representation of an object.

[0024] FIG. 15 illustrates an example of user input for changing a 3D representation of an object into a 2D representation of an object.

[0025] FIG. 16 is a flowchart illustrating exemplary operations of an electronic device for changing a 2D representation of an object into a 3D representation of an object.

[0026] FIG. 17 illustrates an example of user input for changing a 2D representation of an object into a 3D representation of an object.

[0027] FIG. 18 illustrates an example of user input for changing the position of a light source relative to an object.

[0028] FIG. 19 illustrates an example of user input for changing the shape of a 3D representation of an object.

[0029] FIG. 20 illustrates an example of compositing a 3D representation of an object onto an image.

[0030] FIG. 21 is a block diagram of an electronic device in a network environment according to various embodiments.

[0031] Figure 22 is a schematic diagram of an exemplary AI system.

[0032] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0033] Figure 1 illustrates an example of generating another 2D representation of an object using a 2D image containing the object.

[0034] Referring to FIG. 1, the electronic device (100) may be described as a device available for generating a 3D representation of an object. For example, the electronic device (100) may be one of various forms of mobile devices such as smartphones having various form factors (e.g., bar-type smartphones, foldable-type smartphones, or rollable-type smartphones), tablets, wearable devices, cellular phones, personal computers (PCs) (e.g., laptops and / or desktops), and / or other similar computing devices that include circuits (or circuitry) for generating a 3D representation of an object.

[0035] The electronic device (100) may include a display (110) (e.g., the display (230) of FIG. 2). In state (105), the electronic device (100) may display a 2D image (115) containing an object through the display (110). The object contained in the 2D image (115) may be viewed from a first direction. The electronic device (100) may obtain a 2D representation (120) of the object viewed from the first direction from the 2D image (115). The electronic device (100) may use a plurality of 2D representations of the object viewed from different directions (e.g., 2D representation of the object (120), 2D representation of the object (125), and / or 2D representation of the object (135)) to generate a 3D representation of the object. In order to generate a 3D representation of the object, it may be required to obtain a 2D representation of the object viewed from a direction different from the first direction.

[0036] The electronic device (100) can obtain a 2D representation (125) of an object viewed in a second direction different from the first direction by using a 2D representation (120) of an object viewed in a first direction. For example, since the 2D representation (120) of the object does not include a part of the object not viewed in the first direction (e.g., back of the head), a part of the object not viewed in the first direction can be arbitrarily generated within the 2D representation (125) of the object obtained by using the 2D representation (120) of the object. For example, since a part of the object not viewed in the first direction is arbitrarily generated within the 2D representation (125) of the object obtained by using the 2D representation (120) of the object, it can be represented differently within the 2D representation (125) of the object obtained by using the 2D representation (120). For example, as parts of an object that are not visible in the first direction are represented differently in the 2D representation (125) of an object obtained using the 2D representation (120) of an object, the 3D representation of an object created using the 2D representation (125) of an object may have errors.

[0037] In order to resolve errors in the 3D representation of an object generated using the 2D representation (125) of such an object, it may be required to use an image (130) containing an object viewed from a second direction, which is stored in the electronic device (100), as a replacement for the 2D representation (125) of the object. For example, the electronic device (100) may obtain a 2D representation (135) of an object viewed from a second direction from an image (130) containing an object viewed from a second direction. For example, the 2D representation (135) of the object may include a part of the object viewed from a second direction (e.g., the back of the head). For example, the electronic device (100) may identify an image (130) containing said object among the images stored in the electronic device (100) in order to obtain a 2D representation (135) of an object viewed from a second direction. The electronic device (100) may perform operations exemplified within the description of FIGS. 3 through 20 to obtain a 2D representation (135) of an object viewed from a second direction. The electronic device (100) may include components for performing said operations. said components may be exemplified within the description of FIG. 2.

[0038] Figure 2 is a simplified block diagram of an exemplary electronic device.

[0039] Referring to FIG. 2, the electronic device (200) may be one of various types of mobile devices, such as smartphones having various form factors (e.g., bar-type smartphones, foldable-type smartphones, or rollable-type smartphones), tablets, wearable devices, cellular phones, personal computers (PCs) (e.g., laptops and / or desktops), and / or other similar computing devices. For example, the electronic device (200) may include or correspond to the electronic device (100) of FIG. 1. For example, the electronic device (200) may include at least a part of the electronic device (2101) of FIG. 21 or correspond to at least a part of the electronic device (2101) of FIG. 21. For example, the electronic device (200) may include at least one processor (210), memory (220), and display (230).

[0040] At least one processor (210) may include a processing circuit. For example, at least one processor (210) may include a CPU (central processing unit) (e.g., including a processing circuit). For example, at least one processor (210) may include a GPU (graphic processing unit) (e.g., including a processing circuit) and / or an NPU (neural processing unit) (e.g., including a processing circuit). At least one processor (210) may include at least a part of the processor (2120) of FIG. 21 or correspond to at least a part of the processor (2120) of FIG. 21. For example, at least one processor (210) may be described as an application processor. For example, at least one processor (210) may be configured to control memory (220) and a display (230). At least one processor (210) may be configured to execute instructions stored in memory (220) individually or collectively to cause the electronic device (200) to perform at least some of the operations illustrated in the description of FIG. 1. At least one processor (210) may be configured to execute instructions stored in memory (220) to cause the electronic device (200) to perform at least some of the operations illustrated in the description of FIG. 3 through 20.

[0041] For example, the term “processor” as used herein, including in the claims, may include various processing circuits comprising at least one processor, and one or more of said at least one processor may be configured to perform the various functions described below in a distributed manner, individually and / or collectively. As used below, where “processor,” “at least one processor,” and “one or more processors” are described as being configured to perform various functions, these terms encompass, for example, but not limited to, situations where one processor performs some of the cited functions and another processor(s) perform other parts of the cited functions, and also situations where one processor can perform all of the cited functions. Additionally, said at least one processor may include a combination of processors that perform the enumerated / disclosed various functions, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform the various functions.

[0042] The memory (220) may include one or more storage media. For example, the memory (220) may store various data used by at least one component of the electronic device (200) (e.g., at least one processor (210) and / or display (230)). For example, the memory (220) may include at least a portion of the memory (2130) of FIG. 21 or correspond to at least a portion of the memory (2130) of FIG. 21. For example, the data may include input data or output data for software and related commands. The memory (220) may include volatile memory or non-volatile memory.

[0043] The display (230) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by the touch. For example, the display (230) may include at least a part of the display module (2160) of FIG. 21 or correspond to at least a part of the display module (2160) of FIG. 21. For example, the display (230) may be configured to display visual information (e.g., an image). For example, the display (230) may be configured to receive user input. For example, a display (230) that supports touch functions may be referred to as a touch screen.

[0044] The electronic device (200) illustrated in the description of FIG. 2 may perform at least some of the operations illustrated in the descriptions of FIG. 3 through 20. For example, the operations illustrated in the descriptions of FIG. 3 through 20 may be caused by (or within) the electronic device (200) under the control of at least one processor (210).

[0045] FIG. 3 is a flowchart illustrating exemplary operations of an electronic device for generating a 3D representation of an object using a 2D image containing the object.

[0046] Referring to FIG. 3, in operation 300, at least one processor (210) may display a 2D image (e.g., 2D image of FIG. 4 (405)) containing an object (e.g., object of FIG. 4 (410)) through a display (230). For example, the object contained in the 2D image may be viewed from a first direction. By example, without limitation, the object contained in the 2D image may include a subject, a virtual object, and / or a drawing object. By example, without limitation, the 2D image may be displayed in a media application (or gallery application) or in a webpage. However, it is not limited thereto.

[0047] In operation 310, at least one processor (210) may receive input for generating a 3D representation of an object contained within a 2D image using a 2D image. For example, the input for generating a 3D representation of an object may be received through an object contained within a 2D image. For example, the input for generating a 3D representation of an object may include an input for determining an object representing the 3D representation to be generated. The input for generating a 3D representation of an object is exemplified in the description of FIG. 4.

[0048] Figure 4 illustrates an example of input for generating a 3D representation of an object.

[0049] Referring to FIG. 4, the state (400) may be described as a state in which a 2D image (405) containing an object (410) is displayed. In the state (400), at least one processor (210) may receive an input (415) for an object (410) contained in the 2D image (405). For example, the input (415) may include an input for determining an object representing a 3D representation to be generated.

[0050] For example, the input (415) may include a touch input having a contact point on an object (410) within a 2D image (405). For example, the input (415) may include a long press input (or tap input) having a contact point on the object (410). For example, at least one processor (210) may identify the input (415) (e.g., a long press input) based on identifying that the touch input is maintained on the object (410) for a reference time. For example, at least one processor (210) may receive the input (415) through a display (230) (e.g., a touchscreen). For example, at least one processor (210) may determine, based on the input (415), the object (410) among the objects included in the 2D image (405) as the object of the 3D representation to be created. For example, at least one processor (210) can extract (or separate, or crop) an object (410) from a 2D image (405) based on an input (415). For example, a trained model may be used to extract (or separate, or crop) the object (410) from a 2D image. For example, at least one processor (210) can obtain a first 2D representation of the object (410) by extracting (or separating, or cropping) the object (410) from a 2D image (405).

[0051] The electronic device (200) can transition from state (400) to state (420) based on receiving input (415). In state (420), at least one processor (210) can display a window (or pop-up window) (425) through a display (230) based on input (415). For example, the window (425) may be displayed over a 2D image (405) (or together with the 2D image (405)). For example, the window (425) may be available to determine the type of representation of an object to be created using an object (410) determined by input (415). For example, the window (425) may include executable objects that indicate the type of representation to be created using the object (410) determined by the input (415) (e.g., an executable object (430) that indicates a 3D representation of the object (410)).

[0052] For example, at least one processor (210) may receive an input (435) for generating a 3D representation of an object (410) determined by an input (415) through (or using) a window (425). For example, the input (435) may include a touch input to an executable object (430) that indicates a 3D representation of an object (410) among executable objects that indicate the type of representation of an object (410) to be generated using the object (410) determined by the input (415) contained within the window (425). For example, the input (435) may have a contact point on the executable object (430) that indicates a 3D representation of an object (410). For example, the input (435) may be received through a display (230) (e.g., a touchscreen). For example, at least one processor (210) can perform the operation 320 of FIG. 3 based on the input (435).

[0053] For example, an input for generating a 3D representation of an object (410) may include a sequence of one or more inputs. For example, a sequence of one or more inputs may include an input (415) for determining the object (410) of the 3D representation to be generated and an input (435) for generating a 3D representation of the object (410) determined by the input (415). For example, at least one processor (210) may perform the operation 320 of FIG. 3 based on receiving the input (415) and receiving the input (435).

[0054] As an example without limitation, input (415) and input (435) may be received using a stylus pen or through a microphone. For example, input (415) and input (435) may include voice input (or speech input) received through a microphone, but are not limited thereto.

[0055] Referring again to FIG. 3, in operation 320, at least one processor (210) can identify one or more images containing each of the object among the images stored in the electronic device (200). For example, a trained model may be used to identify one or more images containing each of the object among the images stored in the electronic device (200). For example, the images stored in the electronic device (200) may include images in the gallery (or media gallery) of the electronic device (200). For example, the images stored in the electronic device (200) may include images stored in memory (220). For example, the object may include an object determined based on an input for generating a 3D representation of the object.

[0056] For example, at least one processor (210) can identify whether each of the images stored in the electronic device (200) contains said object in order to generate a 3D representation of the object. For example, identifying whether each of the images stored in the electronic device (200) contains said object without user authorization may cause inconvenience to the user. To resolve this inconvenience, at least one processor (210) may receive user input authorizing identification of whether each of the images stored in the electronic device (200) contains said object. User input authorizing identification of whether each of the images stored in the electronic device (200) contains said object is exemplified in the description of FIG. 5.

[0057] FIG. 5 illustrates an example of user input authorizing identification of whether images stored in an electronic device contain an object.

[0058] Referring to FIG. 5, the state (500) may be described as a state in which input for generating a 3D representation of an object (410) is received. In the state (500), at least one processor (210) may display a window (or pop-up window) (505) via a display (230) to inquire whether each of the images stored in the electronic device (200) contains the object (410) in order to generate a 3D representation of the object (410), based on the input for generating a 3D representation of the object (410). For example, the window (505) may contain text inquiring whether each of the images stored in the electronic device (200) contains the object (410) (e.g., “Should I use a different image to improve the quality of the 3D sticker?”). It may include text such as ”.

[0059] For example, at least one processor (210) may receive a user input (515) that authorizes identification of whether each of the images stored in the electronic device (200) contains an object (410) through a window (505) (or a user input that does not authorize identification of whether each of the images stored in the electronic device (200) contains an object (410). For example, at least one processor (210) may refrain from (or stop, or skip, or bypass, or not identify) identification of whether each of the images stored in the electronic device (200) contains an object (410) based on a user input that does not authorize identification of whether each of the images stored in the electronic device (200) contains an object (410).

[0060] For example, a user input (515) authorizing identification of whether each of the images stored in the electronic device (200) contains an object (410) may include a touch input to text (e.g., text such as “OK”) (510) on the window (505) that indicates authorizing identification of whether each of the images stored in the electronic device (200) contains an object (410). For example, the user input (515) may have a contact point on the text (510). For example, the user input (515) may be received through a display (230) (e.g., a touchscreen). For example, at least one processor (210) may identify whether each of the images stored in the electronic device (200) contains an object (410) based on the user input (515).

[0061] A state (520) can be described as a state in which input for settings regarding the creation of a 3D representation of an object is received. In the state (520), at least one processor (210) may display a UI (525) available for settings regarding the creation of a 3D representation of an object via a display (230) based on the input for settings regarding the creation of a 3D representation of an object. For example, the UI (525) may include a toggle button (530) (or a check box, or a radio button) that indicates whether to identify images stored in the electronic device (200) to create a 3D representation of an object. For example, at least one processor (210) may receive user input (535) for the toggle button (530). For example, the user input (535) may include a touch input (e.g., a tap input) having a contact point on the toggle button (530). For example, user input (535) can be received through a display (230) (e.g., a touchscreen).

[0062] For example, at least one processor (210) may change the shape of the toggle button (530) (e.g., ON shape and / or OFF shape) based on user input (535). For example, at least one processor (210) may switch, based on user input (535), a first state of the electronic device (200) that identifies whether each of the images stored in the electronic device (200) contains said object to create a 3D representation of the object to a second state of the electronic device (200) that refrains from identifying whether each of the images stored in the electronic device (200) contains said object to create a 3D representation of the object (or the second state of the electronic device (200) to the first state of the electronic device (200). For example, the first state of the electronic device (200) (or the second state of the electronic device (200)) may be represented by the shape (or color) of the toggle button (530).

[0063] For example, in a first state of the electronic device (200), at least one processor (210) can identify whether each of the images stored in the electronic device (200) contains the object (410) based on input for generating a 3D representation of the object (410). For example, in a first state of the electronic device (200), at least one processor (210) can identify whether each of the images stored in the electronic device (200) contains the object (410) by bypassing (or refraining from, or not receiving) receiving user input authorizing identification of whether each of the images stored in the electronic device (200) contains the object (410) for generating a 3D representation of the object (410). For example, in a second state of the electronic device (200), at least one processor (210) may refrain from (or stop, or skip, or bypass, or not identify) whether each of the images stored in the electronic device (200) contains the object (410) based on an input for generating a 3D representation of the object (410).

[0064] Referring again to FIG. 3, in operation 330, at least one processor (210) can obtain a second 2D representation of an object viewed in a second direction different from a first direction from at least a portion of one or more images each containing an object (e.g., object (410) of FIG. 4). For example, at least one processor (210) can identify the directions in which objects contained in one or more images are viewed by providing one or more images each containing said object to a trained model. For example, the trained model may include a machine learning model and / or a deep learning model. For example, the trained model may include a convolutional neural network (CNN) model.

[0065] For example, at least one processor (210) can identify a second direction different from the first direction among the directions in which objects included in each of one or more images are viewed. For example, the second direction may be defined as a direction orthogonal to the first direction. For example, the second direction may be defined as a direction in which a 2D representation of an object, to be used together with a first 2D representation of an object to generate a 3D representation of an object, is viewed. For example, at least one processor (210) can identify an image containing an object viewed from the second direction. For example, at least one processor (210) can obtain (or extract, or crop) a second 2D representation of an object from an image containing an object viewed from the second direction. Obtaining a second 2D representation of an object from an image containing an object viewed from the second direction is exemplified in the description of FIG. 6.

[0066] FIG. 6 illustrates an example of one or more images, each containing an object, among images stored in an electronic device.

[0067] Referring to FIG. 6, the state (600) can be described as a state in which one or more images (610-1, 610-2), each containing objects (615-1, 615-2) among the images (605) stored in the electronic device (200), are identified. For example, the objects (615-1, 615-2) can be described as objects viewed from a first direction and a direction different from the first. In state (600), at least one processor (210) can identify a second direction in which an object (615-1) (or object (615-2)) contained within an image (610-1) (or image (610-2)) is viewed, based on identifying an image (610-1) (or image (610-2)) containing an object (615-1) (or object (615-2)) among images (605) stored in an electronic device (200). For example, the second direction may be described as a direction orthogonal to the first direction. For example, at least one processor (210) may obtain a second 2D representation of the object (615-1) (or object (615-2)) seen in the second direction from the image (610-1) (or image (610-2)) based on identifying a second direction in which the object (615-1) (or object (615-2)) contained within the image (610-1) (or image (610-2)) is seen. For example, obtaining a second 2D representation of the object (615-1) (or object (615-2)) from the image (610-1) (or image (610-2)) may refer to obtaining a first 2D representation of the object (e.g., object (410) of FIG. 4) from the image (e.g., image (405) of FIG. 4).

[0068] According to another embodiment, in a state (600), at least one processor (210) may display a window (620) for inquiring whether to use the image (610-1) (or image (610-2)) to create a 3D model of an object through a display (230), based on identifying an image (610-1) (or image (610-2)) containing an object among the images (605) stored in the electronic device (200). For example, the window (620) may include the image (610-1) (or image (610-2)). For example, the image (610-1) (or image (610-2)) may be displayed in a reduced size within the window (620). For example, the image (610-1) (or image (610-2)) may have a reduced resolution value within the window (620).

[0069] For example, at least one processor (210) may receive user input (625) through a window (620) indicating that an image (610-1) (or image (610-2)) is used to create a 3D model of an object. For example, user input (625) may include a touch input (or tap input) having a contact point on the image (610-1) (or image (610-2)) to be used to create a 3D model of an object. For example, user input (625) may be received through a display (230) (e.g., a touchscreen). As an example, but not limited to, user input (625) may be received using a stylus pen or through a microphone. For example, user input (625) may include voice input (or speech input) received through a microphone, but is not limited thereto.

[0070] For example, user input (625) may include a sequence of one or more inputs. For example, user input (625) may include an input for selecting an image (e.g., image (610-1) and / or image (610-2)) to be used to create a 3D model of an object, and an input indicating that the selected images are used to create a 3D model of an object. For example, the input for selecting an image (e.g., image (610-1) and / or image (610-2)) to be used to create a 3D model of an object may include a touch input having a point of contact on the selected image (610-1) (or image (610-2)). For example, the input for using the selected image to create a 3D model of an object may include a touch input having a point of contact on text (e.g., text such as “OK”) indicating that the selected image is used to create a 3D model of an object. For example, a sequence of one or more inputs can be received through a display (230) (e.g., a touchscreen).

[0071] For example, at least one processor (210) can obtain (or extract, or crop) a second 2D representation of an object (615-1) (or object (615-2)) from an image (610-1) (or image (610-2)) based on user input (625). For example, at least one processor (210) can bypass (or refrain from, or do not identify) identifying a second direction in which an object (615-1) (or object (615-2)) contained within an image (610-1) (or image (610-2)) is viewed, and obtain (or extract, or crop) a second 2D representation of an object (615-1) (or object (615-2)) contained within an image (610-1) (or image (610-2)).

[0072] Referring again to FIG. 3, in operation 340, at least one processor (210) may generate a 3D representation of an object using a first 2D representation of an object (e.g., object (410) of FIG. 4) and a second 2D representation of an object (e.g., object (615-1) and / or object (615-2) of FIG. 6). For example, a trained model (e.g., trained model (705) of FIG. 7) may be used to generate a 3D representation of an object. For example, generating a 3D representation of an object using a trained model is illustrated in the description of FIG. 7.

[0073] Figure 7 illustrates an example of generating a 3D representation of an object using a trained model.

[0074] Referring to FIG. 7, the electronic device (200) may include a trained model (705). For example, the trained model (705) may include a machine learning model, a deep learning model, and / or a generative AI (artificial intelligence) model. For example, the trained model (705) may include the generative AI model (2230) of FIG. 22. For example, the trained model (705) may include an artificial neural network model comprising a plurality of layers and / or operations (or computations). As a non-limiting example, at least one trained model may include one of FNN (feedforward neural network), DNN (deep neural network), CNN (convolutional neural network), R-CNN (region with convolution neural network), RPN (region proposal network), RNN (recurrent neural network), S-DNN (stacking-based deep neural network), S-SDNN (state-space dynamic neural network), deconvolution network, RBM (restricted Boltzmann machine), DBN (deep belief network), BRDNN (bidirectional recurrent deep neural network), deep Q-networks, fully convolutional network, LSTM (long short-term memory) network, and classification network, or a combination of two or more of these, but is not limited thereto.For example, at least one trained model may be trained on input data and may obtain output data by performing computation (or inference) based on receiving input data. At least one trained model on the electronic device (200) may include a hardware structure or a software structure that is additional (or alternative) to the hardware structure.

[0075] For example, the trained model (705) may include a VFushion 3D model and / or a stable fast 3D (SF3D) model. For example, the trained model (705) may be composed of a density MLP (multi-layer perceptron), an offset MLP, an albedo MLP, and / or a normal MLP. For example, at least one processor (210) may require 2D representations of an object viewed from multiple (e.g., three) different directions (e.g., a first 2D representation of object (410), a second 2D representation of object (615-1), and / or a second 2D representation of object (615-2)) to generate a 3D representation (710) of an object using the trained model (705). For example, the different directions may be described as mutually orthogonal directions.

[0076] For example, at least one processor (210) may provide (or input) the first 2D representation of object (410) and the second 2D representation of object (615-1) (and / or object (615-2)) to the trained model (705) based on obtaining the second 2D representation of object (615-1) (and / or object (615-2)). For example, at least one processor (210) may obtain (or generate) the 3D representation (710) of the object from the trained model (705). For example, since the second 2D representation of object (615-1) (and / or object (615-2)) includes a portion of the object that is not visible from the first direction, the portion of the object that is not visible from the first direction may be represented in the 3D representation (710) of the object created using the second 2D representation of object (615-1) (and / or object (615-2)). For example, the 3D representation (710) of the object created using the second 2D representation of object (615-1) (and / or object (615-2)) may not have an error in which the portion of the object that is not visible from the first direction is arbitrarily created.

[0077] For example, at least one processor (210) may notify the user that the 3D representation (710) of the object is being generated while generating the 3D representation (710) of the object. For example, at least one processor (210) may notify the user that the 3D representation (710) of the object is being generated based on generating the 3D representation (710) of the object. Notifying the generation of the 3D representation (710) of the object is exemplified in the description of FIG. 8.

[0078] Figure 8 illustrates an example of a window indicating the creation of a 3D representation of an object.

[0079] Referring to FIG. 8, the state (800) can be described as a state in which a 3D representation of an object (e.g., a 3D representation of an object in FIG. 7 (710)) is being created. In the state (800), at least one processor (210) may display a window (or information) through a display (230) to indicate that a 3D representation of an object is being created, based on providing (or inputting) a first 2D representation of an object and a second 2D representation of an object to a trained model (e.g., a trained model in FIG. 7 (705)). For example, the window (or information) to indicate that a 3D representation of an object is being created may include an image (805) and / or text (810). For example, the image (805) may have the shape of the 3D representation of the object to be created (e.g., the shape of a blurred 3D representation of an object). For example, the text (810) may include text indicating that a 3D representation of an object is being created (e.g., text such as “Creating a 3D sticker using an image in the gallery.”). For example, at least one processor (210) may display a window (or information) indicating that a 3D representation of an object is being created while (or until) the 3D representation of an object is being created. For example, at least one processor (210) may inform the user that a 3D representation of an object is being created by displaying a window (or information) indicating that a 3D representation of an object is being created.

[0080] For example, based on the creation of a 3D representation of an object, the electronic device (200) may transition from state (800) to state (815). In state (815), at least one processor (210) may display a window (or information) through a display (230) to indicate that a 3D representation of an object has been created, based on the creation of a 3D representation of an object. For example, the window (or information) to indicate that a 3D representation of an object has been created may include an image (820) and text (825). For example, the image (820) may have the shape of the created 3D representation of the object. For example, at least one processor (210) may display a 3D representation of an object having multiple poses as the image (820) is rotated. For example, text (825) may include text to indicate that a 3D representation of an object has been created (e.g., text such as “A 3D sticker has been created using an image in the gallery.”). For example, at least one processor (210) may display a window (or information) to indicate that a 3D representation of an object has been created for a threshold time. For example, at least one processor (210) may display a window (or information) to indicate that a 3D representation of an object has been created until the threshold time has elapsed after displaying the window (or information) to indicate that a 3D representation of an object has been created. For example, at least one processor (210) may indicate to the user that a 3D representation of an object has been created by displaying a window (or information) to indicate that a 3D representation of an object has been created.

[0081] For example, at least one processor (210) can identify a plurality of images, each containing an object viewed from a second direction, among the images stored in the electronic device (200). For example, when identifying a plurality of images, each containing an object viewed from a second direction, it may be required to determine an image among the plurality of images to generate a 3D representation of the object. Determining an image among the plurality of images to generate a 3D representation of the object is exemplified in the description of FIG. 9.

[0082] FIG. 9 is a flowchart illustrating exemplary operations of an electronic device for determining an image to be used to generate a 3D representation of an object among a plurality of images.

[0083] Referring to FIG. 9, in operation 900, at least one processor (210) can identify one or more images, each containing an object, among the images stored in the electronic device (200). For example, at least one processor (210) can identify the direction in which the object contained in each of the one or more images is viewed. For example, at least one processor (210) can identify the direction in which the object contained in each of the one or more images is viewed using a trained model. For example, identifying the direction in which the object contained in each of the one or more images is viewed may refer to operation 330 of FIG. 3.

[0084] For example, at least one processor (210) can identify a plurality of images, each containing an object viewed from a second direction among one or more images. For example, the objects viewed from a second direction may have different appearances (e.g., the clothing of the object, the body shape of the object, and / or the hairstyle of the object) within the plurality of images. For example, a 3D representation of an object created using a first 2D representation of an object viewed from a first direction and a 2D representation of an object having an appearance different from the appearance of the object in the first 2D representation may have errors. For example, it may be required to determine an image to be used to create a 3D representation among a plurality of images containing each of objects viewed from a second direction and having different appearances.

[0085] In operation 910, at least one processor (210) can determine an image among the plurality of images to be used to generate a 3D representation of an object based on identifying a plurality of images each containing an object viewed from a second direction. For example, at least one processor (210) can determine an image among the plurality of images to be used to generate a 3D representation of an object based on the timepoint at which each of the plurality of images was acquired. For example, at least one processor (210) can identify a first timepoint at which a 2D image containing an object viewed from a first direction was acquired (or stored in the electronic device (200)). For example, at least one processor (210) can identify second timepoints at which a plurality of images each containing an object viewed from a second direction were acquired (or stored in the electronic device (200)). For example, at least one processor (210) can compare the first timepoint with the second timepoints. For example, at least one processor (210) can determine an image acquired at a time relatively close to the first time among the second time points.

[0086] For example, at least one processor (210) may determine an image acquired at a time relatively close to the first time point as an image to be used to generate a 3D representation of an object. For example, the shape of an object included in the image acquired at a time relatively close to the first time point may correspond (or substantially correspond) to the shape of an object included in the image acquired at the first time point. For example, at least one processor (210) may acquire (or extract, or crop) a second 2D representation of an object viewed from a second direction from the determined image. For example, at least one processor (210) may generate a 3D representation of an object by using the first 2D representation of an object viewed from the first direction and the second 2D representation of an object viewed from the second direction, which has a shape corresponding (or substantially corresponds) to the shape of the object viewed from the first direction. For example, the description of FIG. 7 may be referenced for generating a 3D representation of an object.

[0087] According to another embodiment, at least one processor (210) can determine an image to be used to generate a 3D representation of an object among a plurality of images based on the appearance of an object included in each of the plurality of images. For example, at least one processor (210) can identify the appearance of an object viewed from a first direction. For example, at least one processor (210) can identify the appearance of an object included in each of the plurality of images. For example, at least one processor (210) can compare the appearance of an object viewed from a first direction with the appearance of an object included in each of the plurality of images. For example, at least one processor (210) can determine an image containing an object having an appearance corresponding (or substantially corresponding) to the appearance of an object viewed from a first direction among the plurality of images.

[0088] For example, at least one processor (210) may obtain (or extract, or crop) a second 2D representation of an object viewed from a second direction from the determined image. For example, at least one processor (210) may generate a 3D representation of an object by using a first 2D representation of an object viewed from a first direction and a second 2D representation of an object viewed from a second direction having an appearance corresponding (or substantially corresponding) to the appearance of the object viewed from the first direction. For example, the description of FIG. 7 may be referenced for generating a 3D representation of an object.

[0089] According to another embodiment, the first 2D representation of an object and the second 2D representation of an object may not include a part of the object (e.g., the top of the head) that is viewed in a third direction different from the first and second directions. For example, since the first 2D representation of an object and the second 2D representation of an object do not include a part of the object that is viewed in a third direction, a part of the object that is viewed in a third direction may be arbitrarily generated within the 3D representation of the object created using the first 2D representation of the object and the second 2D representation of the object. For example, since the 3D representation of an object created using the first 2D representation of an object and the second 2D representation of an object may have an error regarding the part of the object that is viewed in a third direction, it may be required to create another 3D representation of the object by further using a third 2D representation of the object that is viewed in a third direction. For example, at least one processor (210) may obtain a third 2D representation of an object viewed from a third direction based on obtaining an image containing an object viewed from a third direction. For example, at least one processor (210) may further use the third 2D representation of the object to generate another 3D representation of the object. Generating another 3D representation of the object is exemplified in the description of FIG. 10.

[0090] FIG. 10 is a flowchart illustrating exemplary operations of an electronic device for generating different 3D representations of an object.

[0091] Referring to FIG. 10, in operation 1000, at least one processor (210) can generate a 3D representation of an object using a first 2D representation of an object and a second 2D representation of an object. For example, generating a 3D representation of an object may be described by operation 340 of FIG. 3 and FIG. 7. For example, after the 3D representation of an object is generated, at least one processor (210) can obtain one or more other images each containing the object.

[0092] For example, at least one processor (210) can identify whether the acquired image contains the object based on acquiring the image after the 3D representation of the object is generated. For example, at least one processor (210) can identify whether the acquired image contains the object within a first state of the electronic device (200) of FIG. 5. For example, at least one processor (210) can identify whether the images acquired after the 3D representation of the object is generated contain the object based on a period. For example, at least one processor (210) can use a trained model to identify whether the images acquired after the 3D representation of the object is generated contain the object.

[0093] In operation 1010, at least one processor (210) can identify an image containing an object that is viewed in a third direction different from the first and second directions among one or more other images each containing an object. For example, the third direction may be defined as a direction orthogonal to the first and second directions. For example, at least one processor (210) can identify an image containing an object that is viewed in a third direction by comparing the direction in which an object is viewed, each contained in one or more other images, with the first and second directions.

[0094] For example, at least one processor (210) may obtain (or extract, or crop) a third 2D representation of an object viewed from a third direction from an image containing an object viewed from a third direction, based on identifying an image containing an object viewed from a third direction. For example, at least one processor (210) may inquire whether to generate another 3D representation of an object using the third 2D representation, based on identifying an image containing an object viewed from a third direction. For example, inquiring whether to generate another 3D representation of an object using the third 2D representation will be exemplified in the description of FIG. 11.

[0095] In operation 1020, at least one processor (210) may provide (or input) a first 2D representation of an object, a second 2D representation of an object, and a third 2D representation of an object to a trained model (e.g., the trained model (705) of FIG. 7). For example, at least one processor (210) may generate another 3D representation of an object from the trained model. For example, generating another 3D representation of an object may be described in operation 340 of FIG. 3 and FIG. 7. For example, the other 3D representation of an object may include a portion represented by the third 2D representation of an object. For example, the other 3D representation of an object may represent (or include) a portion of an object that is not seen in the first and second directions (and / or is seen in the third direction). For example, at least one processor (210) can resolve errors in the 3D representation of an object by generating a different 3D representation of the object. Information indicating that a different 3D representation of the object is generated is exemplified in the description of FIG. 11.

[0096] Figure 11 illustrates an example of an input for generating different 3D representations of an object.

[0097] Referring to FIG. 11, the state (1100) may be described as a state in which, after a 3D representation of an object is generated, one or more different images each containing the object are acquired. In the state (1100), at least one processor (210) may identify an image (1110) containing an object viewed in a third direction different from the first and second directions among the one or more different images. For example, at least one processor (210) may display a window (1105) through a display (230) based on identifying an image (1110) containing an object viewed in a third direction. For example, the window (1105) may inquire whether to generate another 3D representation of the object using a third 2D representation of the object viewed in the third direction. For example, the window (1105) may display text (e.g., “A new image has been found. Shall we remake the 3D sticker?”). It may include text such as “”. For example, the window (1105) may include an image (1110) containing an object viewed from a third direction. For example, the image (1110) containing an object viewed from a third direction may be displayed in a reduced size within the window (1105). For example, the image (1110) containing an object viewed from a third direction may have a reduced resolution value within the window (1105).

[0098] For example, at least one processor (210) may receive, through the window (1105), a user input (1115) instructing to create another 3D representation of an object using a third 2D representation of an object viewed from a third direction (or a user input instructing to refrain from creating another 3D representation of an object using a third 2D representation of an object viewed from a third direction). For example, at least one processor (210) may refrain from obtaining a third 2D representation of an object viewed from a third direction from an image (1110) based on a user input instructing to refrain from creating another 3D representation of an object using a third 2D representation of an object viewed from a third direction.

[0099] For example, at least one processor (210) may obtain (or extract, or crop) a third 2D representation of an object viewed from a third direction from an image (1110) based on receiving a user input (1115) that instructs to generate another 3D representation of an object using a third 2D representation of an object viewed from a third direction. For example, the user input (1115) may include a touch input (or tap input) having a contact point on text (e.g., text such as “OK”) within the window (1105). For example, the user input (1115) may be received via a display (230) (e.g., a touchscreen). As an example without limitation, the user input (1115) may be received using a stylus pen or via a microphone. For example, user input (1115) may include voice input (or speech input) received through a microphone, but is not limited thereto.

[0100] For example, at least one processor (210) may provide (or input) a first 2D representation of an object, a second 2D representation of an object, and a third 3D representation of an object to a trained model (e.g., the trained model (705) of FIG. 7) based on obtaining a third 2D representation of an object seen from a third direction from an image (1110). For example, at least one processor (210) may generate another 3D representation of an object from the trained model. For example, the description of FIG. 7 may be used to generate another 3D representation of an object. For example, the other 3D representation of an object may include a portion represented by the third 2D representation of an object. For example, the other 3D representation of an object may represent (or include) a portion of an object that is not seen from the first and second directions (and / or is seen from the third direction).

[0101] Based on generating a different 3D representation of an object, the electronic device (200) can transition from state (1100) to state (1120). In state (1120), at least one processor (210) can display a window (or information) (1125) indicating that a different 3D representation of an object has been generated based on generating a different 3D representation of an object. For example, the window (or information) (1125) may include text indicating the generation of a different 3D representation of an object (e.g., text such as “3D sticker has been generated.”). For example, the window (or information) (1125) may include an image (1130) having the shape of the generated different 3D representation of the object. For example, at least one processor (210) can display a different 3D representation of an object having multiple poses by rotating the image (1130) having the shape of the generated different 3D representation of the object. For example, at least one processor (210) may display a window (or information) (1125) for a critical time. For example, at least one processor (210) may display the window (or information) (1125) after displaying the window (or information) (1125) until the critical time has elapsed. For example, at least one processor (210) may notify the user that a different 3D representation of the object is being created by displaying the window (or information) (1125).

[0102] According to another embodiment, at least one processor (210) may display a window (or information) requesting the acquisition of an image including an object viewed in a third direction different from the first and second directions in order to generate another 3D representation of an object based on generating a 3D representation of an object using a first 2D representation of an object and a second 2D representation of an object. A window (or information) requesting the acquisition of an image including an object viewed in a third direction is exemplified in the description of FIG. 12.

[0103] FIG. 12 illustrates an example of a window to acquire an image containing an object viewed from a reference direction.

[0104] Referring to FIG. 12, the state (1200) may be described as a state in which a 3D representation of an object is generated using a first 2D representation of an object and a second 2D representation of an object. In the state (1200), at least one processor (210) may identify a reference direction different from the first direction and the second direction based on generating a 3D representation of an object. For example, the reference direction may be orthogonal to the first direction and the second direction. For example, a part of the object viewed in the reference direction may not be included in the first 2D representation of the object viewed in the first direction and the second 2D representation of the object viewed in the second direction. For example, a 2D representation of the object viewed in the reference direction may be required to generate another 3D representation of the object that includes (or represents) a part of the object viewed in the reference direction.

[0105] For example, at least one processor (210) may display a window (or information) requesting the acquisition of an image containing an object viewed in a reference direction based on identifying a reference direction. For example, the window (or information) may include text (1205) and an image (1210). For example, the text (1205) may include text requesting the acquisition of an image containing an object viewed in a reference direction (e.g., text such as “3D sticker saved. Quality can be improved if there is a top image.”). For example, the image (1210) may have the shape of an object viewed in a reference direction. For example, at least one processor (210) may display a window (or information) requesting the acquisition of an image containing an object viewed in a reference direction for a threshold time. For example, at least one processor (210) may display a window (or information) requesting the acquisition of an image containing an object viewed in a reference direction until the threshold time has elapsed after displaying the information.

[0106] For example, at least one processor (210) may acquire an image containing an object viewed in a third direction corresponding to (or substantially corresponding to) the reference direction while displaying a window (or information) requesting the acquisition of an image containing an object viewed in the reference direction (or after displaying information requesting the acquisition of an image containing an object viewed in the reference direction). For example, at least one processor (210) may acquire a third 2D representation of an object viewed in the third direction from an image containing an object viewed in the third direction based on acquiring an image containing an object viewed in the third direction. For example, at least one processor (210) may provide (or input) a first 2D representation of an object, a second 2D representation of an object, and a third 2D representation of an object to a trained model (e.g., the trained model (705) of FIG. 7). For example, at least one processor (210) may generate another 3D representation of an object from the trained model. For example, the description of FIG. 7 may be referenced for generating another 3D representation of an object. For example, another 3D representation of an object may include a part represented by a third 2D representation of the object. For example, another 3D representation of an object may represent (or include) a part of the object that is not seen in the first and second directions (and / or is seen in the third direction).

[0107] According to another embodiment, at least one processor (210) may provide (or input) a first 2D representation of an object to a trained model based on failing to identify an image containing an object among the images stored in the electronic device (200), or failing to identify an image containing an object viewed from a second direction different from the first direction. For example, at least one processor (210) may obtain a fourth 2D representation of an object viewed from a second direction by providing (or inputting) the first 2D representation of an object to the trained model. For example, since the fourth 2D representation of an object does not include a part of the object not viewed from the first direction (e.g., back of the head), an arbitrarily generated part of the object not viewed from the first direction may be included in the fourth 2D representation of the object. For example, since the 3D representation of an object generated using the first 2D representation of the object and the fourth 2D representation of the object may have errors, it may be required to regenerate the 3D representation of the object using the second 2D representation of the object. For example, at least one processor (210) may regenerate the 3D representation of the object based on acquiring an image containing the second 2D representation of the object. For example, regenerating the 3D representation of the object based on acquiring an image containing the second 2D representation of the object may be described in FIGS. 10, FIGS. 11, and FIGS. 12.

[0108] For example, at least one processor (210) may display a 3D representation of the generated object (or another 3D representation of the object) through a display (230). For example, displaying a 3D representation of the object (or another 3D representation of the object) is illustrated in the description of FIG. 13.

[0109] FIG. 13 is a flowchart illustrating exemplary operations of an electronic device for displaying a 3D representation of an object and a 2D representation of an object having a determined pose.

[0110] Referring to FIG. 13, in operation 1300, at least one processor (210) may receive input for displaying a 3D representation of an object. For example, the input for displaying a 3D representation of an object may be received while displaying a screen and / or displaying another image through a display (230). For example, the input for displaying a 3D representation of an object may be described as an input for displaying a 3D representation of an object on a screen (or image, or other screen) being displayed. For example, the input for displaying a 3D representation of an object may be described as an input for retrieving a 3D representation of an object.

[0111] As an example without limitation, input for displaying a 3D representation of an object may be received using a part of the user's body, or received using a stylus pen, or received via a microphone. For example, input for displaying a 3D representation of an object may include a touch input (or tap input) having a point of contact on the location where the 3D representation of the object is to be displayed. For example, input for displaying a 3D representation of an object may include a touch input (or tap input) having a point of contact on an executable object representing the 3D representation of the object to be displayed. For example, input for displaying a 3D representation of an object that is a touch input (or tap input) may be received via a display (230) (e.g., a touchscreen). For example, input for displaying a 3D representation of an object may include voice input (or speech input) received via a microphone, but is not limited thereto.

[0112] In operation 1310, at least one processor (210) may display a 3D representation of an object through a display (230) based on an input for displaying a 3D representation of an object. For example, at least one processor (210) may display a first UI object (e.g., the first UI object (1405) of FIG. 14a and / or the first UI object (1440) of FIG. 14b) that is available to change the orientation of the 3D representation of an object together with (or on) the 3D representation of an object. For example, the first UI object may have a shape that surrounds the 3D representation of an object, or may be superimposed on at least a part of the 3D representation of an object. For example, at least one processor (210) may display the 3D representation of an object and the first UI object within a UI. For example, the UI may be described as a UI for editing (or changing) a screen (or image) being displayed.

[0113] In operation 1320, at least one processor (210) may receive user input that determines the pose of a 3D representation of an object using (or through) a first UI object. For example, the user input that determines the pose of a 3D representation of an object may include a sequence of user inputs. For example, the sequence of user inputs may include user input that changes the pose of a 3D representation of an object using (or through) a first UI object. For example, at least one processor (210) may display a 3D representation of an object having a changed pose by changing the pose of a 3D representation of an object. For example, the sequence of user inputs may include user input that determines the changed pose of a 3D representation of an object as the pose of a 3D representation of an object. For example, user input that determines the changed pose of a 3D representation of an object as the pose of a 3D representation of an object will be exemplified in the description of FIG. 15. For example, at least one processor (210) may receive user input to change the pose of a 3D representation of an object and receive user input to determine the changed pose of the 3D representation of the object as the pose of the 3D representation of the object. User input for changing the pose of the 3D representation of an object is exemplified in the description of FIGS. 14a and FIGS. 14b.

[0114] FIGS. 14a and FIGS. 14b illustrate examples of user input for changing the pose of a 3D representation of an object.

[0115] Referring to FIG. 14a, the state (1400) can be described as a state in which an input for displaying a 3D representation (710) of an object is received. In the state (1400), at least one processor (210) can display the 3D representation (710) of an object and a first UI object (1405) within the UI (1401) via a display (230) based on the input for displaying the 3D representation (710) of an object. For example, the first UI object (1405) may be displayed on the 3D representation (710) of the object or together with the 3D representation (710) of the object. For example, the first UI object (1405) may have the shape of a sphere surrounding the 3D representation (710) of the object. For example, the first UI object (1405) may be referred to as a bounding sphere. For example, the first UI object (1405) may include parts (e.g., circumferences of a sphere) indicating the direction in which the 3D representation (710) of the object is to be rotated (e.g., x-axis direction, y-axis direction, and / or z-axis direction). For example, at least one processor (210) may receive user input (1415) to change the orientation of the 3D representation (710) of the object through one of the parts (1410) of the first UI object (1405) indicating the direction in which the 3D representation (710) of the object is to be rotated.

[0116] For example, user input (1415) may include an input that drags from a part (1410) toward the direction (1420) in which the 3D representation (710) of the object is to be rotated. For example, user input (1415) may include a drag input in which, after receiving a touch input having a contact point on the part (1410), the contact point is moved toward the direction (1420) in which the 3D representation of the object is to be rotated while the touch input is maintained. For example, at least one processor (210) may display the 3D representation (710) of the object that is rotated according to the input that drags toward the direction (1420) in which the 3D representation of the object is to be rotated, through a display (230) in the UI (1401). For example, user input (1415) may include an input that releases the touch input after the drag input is received. For example, user input (1415) can be received through a display (230) (e.g., a touchscreen).

[0117] The electronic device (200) can transition from state (1400) to state (1425) based on user input (1415). In state (1425), at least one processor (210) can rotate the 3D representation (710) of an object based on user input (1415). For example, at least one processor (210) can change the orientation of the 3D representation (710) of an object by rotating the 3D representation (710) of an object. For example, at least one processor (210) can display the 3D representation (1430) of an object having the changed orientation in the UI (1401) based on user input (1415). For example, the orientation of the changed 3D representation (1430) of an object can be described as the orientation of the 3D representation of an object at the time when the touch input of user input (1415) is released. For example, at least one processor (210) may display a first UI object (1405) in the UI (1401) on (or together with) the 3D representation (1430) of an object having a changed pose. For example, at least one processor (210) may further change the pose of the 3D representation (1430) of an object having a changed pose using (or through) the first UI (1405).

[0118] Referring to FIG. 14b, the state (1435) can be described as a state in which an input for displaying a 3D representation (710) of an object is received. In the state (1435), at least one processor (210) can display the 3D representation (710) of an object and a first UI object (1440) within the UI (1401) via a display (230) based on the input for displaying the 3D representation (710) of an object. For example, the first UI object (1440) may be displayed on the 3D representation (710) of the object or together with the 3D representation (710) of the object. For example, the bounding box may have the shape of a cuboid surrounding the 3D representation (710) of the object. For example, the first UI (1440) object may be referred to as a bounding box. For example, the first UI object (1440) may include parts (e.g., each side of a hexahedron) indicating the direction in which the 3D representation (710) of the object is to be rotated (e.g., x-axis direction, y-axis direction, and / or z-axis direction). For example, at least one processor (210) may receive user input (1455) to change the orientation of the 3D representation (710) of the object through one of the parts (1445) of the first UI object (1440) indicating the direction in which the 3D representation (710) of the object is to be rotated.

[0119] For example, user input (1455) may include an input for dragging from a part (1445) toward the direction (1450) in which the 3D representation (710) of the object is to be rotated. For example, user input (1455) may include an input for dragging toward the direction (1450) in which the 3D representation of the object is to be rotated while the touch input is maintained, after a touch input having a contact point on the part (1445) is received. For example, at least one processor (210) may display the 3D representation (710) of the object being rotated according to the input for dragging toward the direction (1450) in which the 3D representation of the object is to be rotated, through a display (230) in the UI (1401). For example, user input (1455) may include an input for releasing the touch input after the drag input is received. For example, user input (1455) can be received through a display (230) (e.g., a touchscreen).

[0120] The electronic device (200) can transition from state (1435) to state (1460) based on user input (1455). In state (1460), at least one processor (210) can rotate the 3D representation (710) of an object based on user input (1455). For example, at least one processor (210) can change the orientation of the 3D representation (710) of an object by rotating the 3D representation (710) of an object. For example, at least one processor (210) can display the 3D representation (1430) of an object having the changed orientation in the UI (1401) based on user input (1455). For example, the orientation of the changed 3D representation (1430) of an object can be described as the orientation of the 3D representation of the object at the time when the touch input of user input (1455) is released. For example, at least one processor (210) may display a first UI object (1440) in the UI (1401) on (or together with) the 3D representation (1430) of an object having a changed pose. For example, at least one processor (210) may further change the pose of the 3D representation (1430) of an object having a changed pose using (or through) the first UI object (1440).

[0121] Referring again to FIG. 13, in operation 1330, at least one processor (210) may receive user input to determine the pose of a 3D representation of an object having a changed pose. For example, at least one processor (210) may obtain a fifth 2D representation of an object having a determined pose based on user input to determine the pose of a 3D representation of an object. For example, at least one processor (210) may obtain the fifth 2D representation of an object having a determined pose by cropping the 3D representation of an object having a determined pose. For example, at least one processor (210) may display the fifth 2D representation of an object having a determined pose within a UI through a display (230). For example, the fifth 2D representation of an object may have the pose of the 3D representation of an object at the time when user input to determine the pose of the 3D representation of an object is received. For example, the fifth 2D representation of an object may be displayed at the location where the 3D representation of the object is displayed at the time when user input for determining the pose of the 3D representation of the object is received.

[0122] For example, at least one processor (210) may not use the first UI object to change the location where the 3D representation of an object having a determined pose is displayed. For example, since using the first UI object to change the location where the 3D representation of an object is displayed is not intuitive, it may cause inconvenience to the user. For example, at least one processor (210) may use a relatively intuitive second UI object (e.g., the second UI object (1525) of FIG. 15) to change the location where the fifth 2D representation of an object is displayed.

[0123] For example, at least one processor (210) may display a second UI object available to change the location where the fifth 2D representation of the object is displayed, along with the fifth 2D representation of the object. For example, the second UI object may have a shape that surrounds the fifth 2D representation of the object. For example, at least one processor (210) may receive user input determining the location where the fifth 2D representation of the object is displayed using (or through) the second UI object. For example, at least one processor (210) may display the fifth 2D representation of the object and the second UI object within the UI.

[0124] For example, a user input determining the location where the fifth 2D representation of an object is to be displayed may include a sequence of user inputs. For example, the sequence of user inputs may include a user input that changes the location where the fifth 2D representation of an object is to be displayed using (or through) a second UI object. For example, at least one processor (210) may display the fifth 2D representation of an object at a changed location within the UI by changing the location where the fifth 2D representation of an object is to be displayed within the UI. For example, the sequence of user inputs may include a user input determining the location of the changed fifth 2D representation of an object as the location of the fifth 2D representation of an object. For example, at least one processor (210) may receive a user input changing the location where the fifth 2D representation of an object is to be displayed and receive a user input determining the location of the changed fifth 2D representation of an object as the location of the fifth 2D representation of an object.

[0125] In operation 1350, at least one processor (210) may display the fifth 2D representation of an object at a determined location within the UI via a display (230), based on user input determining the location where the fifth 2D representation is to be displayed. Displaying the fifth 2D representation of an object at a determined location within the UI based on user input is exemplified in the description of FIG. 15.

[0126] FIG. 15 illustrates an example of user input for changing a 3D representation of an object into a 2D representation of an object.

[0127] Referring to FIG. 15, the state (1500) can be described as a state in which the 3D representation (1430) of an object and the first UI object (1405) are displayed. For example, in the state (1500), at least one processor (210) may further display a third UI object (1505) on a portion adjacent to the portion in which the 3D representation (1430) of an object within the UI (1401) and the first UI object (1405) are displayed, via a display (230). For example, changing the position where the 3D representation (1430) of an object is displayed using the first UI object (1405) is not intuitive and may cause inconvenience to the user. For example, the third UI object (1505) may indicate that the 3D representation of an object is changed to a 2D representation of an object. For example, at least one processor (210) may receive user input (1510) for a third UI object (1505). For example, user input (1510) may include touch input (or tap input) having a contact point on the third UI object (1505). For example, user input (1510) may be received via a display (230) (e.g., a touchscreen). As an example, but not limited to, user input (1510) may be received using a stylus pen or via a microphone. For example, user input (1510) may include voice input (or speech input) received via a microphone, but is not limited thereto.

[0128] For example, at least one processor (210) can determine the pose of the 3D representation (1430) of an object at the time when the user input (1510) is received, based on the user input (1510). For example, the user input (1510) can be described as a user input that determines the pose of the changed 3D representation (1430) of the object. For example, at least one processor (210) can obtain a fifth 2D representation (1520) of the object having the determined pose. For example, at least one processor (210) can store the fifth 2D representation (1520) of the object having the determined pose in conjunction with the 3D representation (1430) of the object.

[0129] The electronic device (200) can transition from state (1500) to state (1515) based on user input (1510). In state (1515), at least one processor (210) can display a fifth 2D representation (1520) of an object and a second UI object (1525) in the UI (1401) via a display (230) based on user input (1510). For example, the fifth 2D representation (1520) of an object may be displayed at the position of the 3D representation (1430) of the object at the time when the user input (1510) in the UI (1401) is received. For example, the fifth 2D representation (1520) of an object may have the posture of the 3D representation (1430) of the object at the time when the user input (1510) is received.

[0130] For example, the second UI object (1525) may surround the fifth 2D representation (1520) of the object or be displayed together with the fifth 2D representation (1520) of the object. For example, the second UI object (1525) may have a rectangular shape. For example, at least one processor (210) may receive user input (1535) that changes the position where the fifth 2D representation (1520) of the object is displayed within the UI (1401) through a part (1530) (e.g., a side of the rectangle) on the second UI object (1525). For example, at least one processor (210) may change the position where the fifth 2D representation of the object is displayed within the UI (1401) using the relatively intuitive second UI object (1525).

[0131] For example, user input (1535) may include a sequence of user inputs. For example, user input (1535) may include a touch input (or tap input) having a contact point on a part (1530) on the second UI object (1525). For example, user input (1535) may include an input to drag in a direction (1540) that changes the location where the fifth 2D representation (1520) of the object is to be displayed from the part (1530) on the second UI object (1525). For example, at least one processor (210) may display the fifth 2D representation (1520) of the object that is moved according to the direction (1540) indicated by the drag input through the display (230) while the drag input is received. For example, user input (1535) may include an input to release the touch input on the location where the fifth 2D representation (1520) of the object is to be displayed. For example, an input that releases a touch input at a location where the fifth 2D representation (1520) of an object is to be displayed may be defined as an input that determines the location where the fifth 2D representation (1520) of an object is to be displayed. For example, user input (1535) may include a sequence of user inputs that receive a touch input (or tap input) having a contact point on a part (1530) on the second UI object (1525), receive an input that drags from the part (1530) on the second UI object (1525) to a location where the fifth 2D representation (1520) of an object is to be displayed, and receive an input that releases a touch input at a location where the fifth 2D representation (1520) of an object is to be displayed. For example, user input (1535) may be received through a display (230) (e.g., a touchscreen).

[0132] The electronic device (200) can transition from state (1515) to state (1545) based on user input (1535). In state (1545), at least one processor (210) can display a fifth 2D representation (1520) of an object at a determined location within the UI (1401) based on user input (1535). For example, the determined location may be described as the location of the fifth 2D representation (1520) of the object at the point where the contact point of the drag input of user input (1535) is released. For example, the second UI object (1525) may be moved as the fifth 2D representation (1520) of the object moves. For example, at least one processor (210) can display the second UI object (1525) together with the fifth 2D representation (1520) of the object at the determined location. For example, at least one processor (210) can additionally change the position of the fifth 2D representation (1520) of an object displayed at a changed position using (or through) the second UI object (1525).

[0133] For example, at least one processor (210) can change the fifth 2D representation (1520) of an object back into a 3D representation (1430) of an object. For example, at least one processor (210) can display the first UI object (1405) on the changed 3D representation (1430) of the object (or together with the changed 3D representation (1430) of the object) via a display (230). For example, at least one processor (210) can display the changed 3D representation (1430) of the object and the first UI object (1405) within a UI (1401). Changing the fifth 2D representation (1520) of an object back into a 3D representation (1430) of an object is exemplified in the description of FIG. 16.

[0134] FIG. 16 is a flowchart illustrating exemplary operations of an electronic device for changing a 2D representation of an object into a 3D representation of an object.

[0135] Referring to FIG. 16, in operation 1600, at least one processor (210) may store the fifth 2D representation of an object in association (or pair) with the 3D representation of an object based on user input (e.g., user input (1510) of FIG. 15) that changes the 3D representation of an object to the fifth 2D representation of an object. For example, at least one processor (210) may store the 3D representation of an object in association with the first 2D representation of an object based on generating the 3D representation of an object. For example, the first 2D representation of an object, the 3D representation of an object, and / or the fifth 2D representation of an object may be stored in association in memory (220).

[0136] In operation 1610, at least one processor (210) may receive user input to change the fifth 2D representation of an object into a 3D representation of an object. For example, it may be required to change the fifth 2D representation of an object into a 3D representation of an object in order to change the pose of the fifth 2D representation of an object. For example, at least one processor (210) may retrieve a 3D representation of an object stored in association with the fifth 2D representation of an object based on user input to change the fifth 2D representation of an object into a 3D representation of an object.

[0137] In operation 1620, at least one processor (210) may display the 3D representation of the object within the UI via the display (230) based on user input to change the fifth 2D representation of the object into the 3D representation of the object. For example, the 3D representation of the object may have the pose of the fifth 2D representation of the object. For example, the 3D representation of the object may be displayed at the location where the fifth 2D representation of the object is displayed within the UI. For example, at least one processor (210) may display the first UI object on (or together with) the 3D representation of the object within the UI. For example, at least one processor (210) may use the first UI object to change the pose of the 3D representation of the object again. Changing the 2D representation of the object into the 3D representation of the object is exemplified in the description of FIG. 17.

[0138] FIG. 17 illustrates an example of user input for changing a 2D representation of an object into a 3D representation of an object.

[0139] Referring to FIG. 17, the state (1700) can be described as a state in which a fifth 2D representation (1520) of an object and a second UI object (1525) are displayed within the UI (1401). For example, in the state (1700), at least one processor (210) may further display a fourth UI object (1705) on a portion adjacent to the portion where the fifth 2D representation (1520) of the object and the second UI object (1525) are displayed, via a display (230). For example, to change the orientation of the fifth 2D representation (1520) of the object, it may be required to change the fifth 2D representation (1520) of the object to a 3D representation (1430) of the object. For example, the fourth UI object (1705) may indicate that the 2D representation of the object is changed to a 3D representation of the object. For example, at least one processor (210) may receive user input (1710) for the fourth UI object (1705). For example, user input (1710) may include touch input (or tap input) having a contact point on the fourth UI object (1705). For example, user input (1710) may be received via a display (230) (e.g., a touchscreen). As an example, but not limited to, user input (1710) may be received using a stylus pen or via a microphone. For example, user input (1710) may include voice input (or speech input) received via a microphone, but is not limited thereto. For example, at least one processor (210) can retrieve a 3D representation (1430) of an object stored in conjunction with a fifth 2D representation (1520) of an object based on user input (1710).

[0140] The electronic device (200) can switch from state (1700) to state (1715) based on user input (1710). In state (1715), at least one processor (210) can display a 3D representation (1430) of an object and a first UI object (1405) through a display (230). For example, the 3D representation (1430) of an object may have the pose of a fifth 2D representation (1520) of an object. For example, the 3D representation (1430) of an object may be displayed at a location where the fifth 2D representation (1520) of an object is displayed within the UI (1401). For example, the first UI object (1405) may be displayed on the 3D representation (1430) of an object (or together with the 3D representation (1430)) of an object within the UI (1401). For example, at least one processor (210) can change the orientation of the 3D representation (1430) of the object by using (or through) the first UI object (1405) by displaying the 3D representation (1430) of the object and the first UI object (1405) in the UI (1401). The description of FIG. 14a and FIG. 14b may be used to change the orientation of the 3D representation (1430) of the object by using (or through) the first UI object (1405).

[0141] For example, at least one processor (210) can change the position of a light source for the 3D representation (1430) of an object. Changing the position of a light source for the 3D representation (1430) of an object is exemplified in the description of FIG. 18.

[0142] FIG. 18 illustrates an example of user input for changing the position of a light source relative to an object.

[0143] Referring to FIG. 18, a state (1800) can be described as a state in which a 3D representation (710) of an object and a fifth UI object (1805) are displayed within a UI (1401). In the state (1800), at least one processor (210) may display within the UI (1401) a fifth UI object (1805) available to change the position of a light source for the 3D representation (710) of an object via a display (230), based on user input for setting the position of a light source for the 3D representation (710) of an object. For example, at least one processor (210) may display the fifth UI object (1805) to surround (or together with) the 3D representation (710) of an object. For example, the fifth UI object (1805) may include an indicator (1810) that indicates the position of a light source for the 3D representation (710) of the object. For example, the 3D representation (710) of the object may have a shape that receives light from the position of the light source indicated by the indicator (1810). For example, at least one processor (210) may receive user input (1815) that changes the position of the light source for the 3D representation (710) of the object through the indicator (1810).

[0144] For example, user input (1815) may include a sequence of user inputs. For example, user input (1815) may include a touch input (or tap input) having a contact point on the indicator (1810). For example, user input (1815) may include an input to drag from the indicator (1810) in a direction (1820) that changes the position of the light source for the 3D representation (710) of the object. For example, at least one processor (210) may display the 3D representation (710) of the object through the display (230) while the drag input is received, in which the position of the light source is moved according to the direction (1820) indicated by the drag input. For example, user input (1815) may include an input to release the touch input at the location where the position of the light source for the 3D representation (710) of the object is to be changed. For example, an input that releases a touch input at a location where the position of a light source for a 3D representation (710) of an object is to be changed may be defined as an input that determines the position of a light source for a 3D representation (710) of an object. For example, user input (1815) may include a sequence of user inputs that receive a touch input (or tap input) having a contact point on an indicator (1810) of a fifth UI object (1805), receive an input that drags from the indicator (1810) in a direction (1820) that changes the position of a light source for a 3D representation (710) of an object, and receive an input that releases a touch input at a location where the position of a light source for a 3D representation (710) of an object is to be changed. For example, user input (1815) may be received through a display (230) (e.g., a touchscreen).

[0145] The electronic device (200) can switch from state (1800) to state (1830) based on user input (1815). In state (1830), at least one processor (210) can change the position of an indicator (1810) on a fifth UI object (1805) based on user input (1815). For example, the changed position of the indicator (1810) can be defined as a part where the contact point is released on the position where the position of the light source for the 3D representation (710) of the object is changed. For example, at least one processor (210) can display the 3D representation (710) of the object having a shape that receives light from the position of the light source indicated by the indicator (1810) having the changed position in the UI (1401) through the display (230). For example, at least one processor (210) can change the position of the light source for the 3D representation (710) of the object based on user input (1815). For example, at least one processor (210) can change the position of the light source for the 3D representation (710) of the object again through the indicator (1810).

[0146] According to another embodiment, in state (1800), at least one processor (210) may display a 3D representation (710) of an object on another image within the UI (1401). For example, the other image may be referred to as a background image. For example, at least one processor (210) may identify an object (1825) having the shape of a light source contained within the other image. For example, based on identifying the object (1825) having the shape of a light source, at least one processor (210) may determine the location of the light source relative to the 3D representation (710) of the object as the location of the object (1825) relative to the 3D representation (710) of the object. For example, at least one processor (210) may display a 3D representation (710) of an object having the shape of receiving light from the object (1825) within the UI (1401).

[0147] For example, at least one processor (210) can identify a skeleton (or bone) that constitutes the 3D representation (710) of an object. For example, at least one processor (210) can change the shape (or pose) of the 3D representation (710) of an object by changing the position of the skeleton that constitutes the 3D representation (710) of an object. Changing the shape (or pose) of the 3D representation (710) of an object by changing the position of the skeleton that constitutes the 3D representation (710) of an object is exemplified in the description of FIG. 19.

[0148] FIG. 19 illustrates an example of user input for changing the shape of a 3D representation of an object.

[0149] Referring to FIG. 19, a state (1900) can be described as a state in which a 3D representation (710) of an object and a skeleton (1905) of an object are displayed within a UI (1401). In the state (1900), at least one processor (210) may display the skeleton (1905) of an object via a display (230) based on user input for setting the shape (or pose) of the 3D representation (710) of the object. For example, the skeleton (1905) of an object may be displayed on the 3D representation (710) of the object. For example, each part of the skeleton (1905) of the object may be displayed at a location corresponding to each part of the 3D representation (710) of the object. For example, at least one processor (210) can receive user input (1910) that changes the shape (or pose) of the 3D representation (710) of the object through the skeleton (1905) of the object.

[0150] For example, user input (1910) may include a sequence of user inputs. For example, user input (1910) may include a touch input (or tap input) having a contact point on a part of the object's skeleton (1905) to be moved. For example, user input (1910) may include an input for dragging from a part of the object's skeleton (1905) toward the direction (1915) in which the part of the object's skeleton (1905) is to be moved. For example, user input (1910) may include an input for releasing the touch input at the location where the part of the object's skeleton (1905) is to be moved. For example, the input for releasing the touch input at the location where the part of the object's skeleton (1905) is to be moved may be defined as an input for determining the location of the part of the object's skeleton (1905) to be moved. For example, user input (1910) may include a sequence of user inputs, such as receiving a touch input (or tap input) having a contact point on a part of the object's skeleton (1905), receiving an input to drag from the part of the object's skeleton (1905) toward the direction (1915) in which the part of the object's skeleton (1905) is to be moved, and receiving an input to release the touch input at the position where the part of the object's skeleton (1905) is to be moved. For example, user input (1910) may be received through a display (230) (e.g., a touchscreen).

[0151] The electronic device (200) can transition from state (1900) to state (1920) based on user input (1910). In state (1920), at least one processor (210) can display the skeleton (1925) of an object in which the position of a part of the skeleton (1925) of the object has been changed based on user input (1910). For example, the part of the skeleton (1925) in which the position has been changed can be described as the part of the skeleton (1925) where a touch input has been received. For example, the changed position of the part of the skeleton (1925) can be described as the position where the touch input has been released. For example, at least one processor (210) can change the shape (or pose) of the 3D representation (1930) of the object according to the change in the part of the skeleton (1925) of the object. For example, a part corresponding to a part of the skeleton (1925) whose position of the 3D representation (1930) of the object has been changed may be moved to the changed position of the part of the skeleton (1925). For example, at least one processor (210) may display the 3D representation (1930) of the object having the changed shape (or pose) in the UI (1401). For example, at least one processor (210) may display the skeleton (1925) of the object on the 3D representation (1930) of the object having the changed shape (or pose) in the UI (1401). For example, at least one processor (210) may change the shape (or pose) of the 3D representation (1930) of the object having the changed shape (or pose) again through the skeleton (1925) of the object.

[0152] For example, at least one processor (210) can composite a 3D representation (710, 1930) of an object onto another image (e.g., a background image). For example, at least one processor (210) can composite a 3D representation (710, 1930) of an object onto another image (e.g., a background image) by modifying the 3D representation (710, 1930) of an object so that it has a shape corresponding to an object contained within another image. Composite a 3D representation (710, 1930) of an object onto another image (e.g., a background image) is illustrated in the description of FIG. 20.

[0153] FIG. 20 illustrates an example of compositing a 3D representation of an object onto an image.

[0154] Referring to FIG. 20, the state (2000) may be described as a state in which a 3D representation (2015) of an object and a fourth UI (2016) are displayed on another image (2005) within the UI (1401). For example, the other image (2005) may be referred to as a background image. For example, the other image (2005) may include an object (2010). For example, the 3D representation (2015) of the object may be described as a representation having a planar shape. For example, the fourth UI (2016) may be available to change the position where the 3D representation (2015) of the object is displayed, or to change the shape of the 3D representation (2015) of the object. In state (2000), at least one processor (210) may display a sixth UI object (2016) in the UI (1401) via a display (230) based on user input to change the location (or shape of the 3D representation (2015) of the object to be displayed. For example, the sixth UI object (2016) may be displayed on the 3D representation (2015) of the object or together with the 3D representation (2015) of the object. For example, the sixth UI object (2016) may have the shape of a sphere surrounding the 3D representation (2015) of the object. For example, the sixth UI object (2016) may be defined as a bounding sphere. For example, at least one processor (210) can receive user input (2020) that changes the position (or shape) of the 3D representation (2015) of an object through the 6th UI object (2016).

[0155] For example, user input (2020) may include a sequence of user inputs. For example, user input (2020) may include a touch input (or tap input) having a contact point on a part of the sixth UI object (2016). For example, user input (2020) may include an input for dragging from a part of the sixth UI object (2016) toward a direction (2025) in which the 3D representation (2015) of the object is to be moved. For example, at least one processor (210) may display the 3D representation (2015) of the object being moved according to the direction (2025) indicated by the drag input through a display (230) while the drag input is received. For example, user input (2020) may include an input for releasing the touch input at a location where the position of the 3D representation (2015) of the object is to be changed. For example, an input that releases a touch input on a location where the position of the 3D representation (2015) of an object is to be changed can be defined as an input that determines the location where the 3D representation (2015) of the object is to be displayed. For example, the user input (2020) may include a long press input having a contact point on an object (2010) having a shape where the 3D representation (2015) of the object is to be changed. For example, the long press input can be defined as an input that determines an object having a shape where the 3D representation (2015) of the object is to be changed. For example, the user input (2020) may include a sequence of user inputs, such as receiving a touch input (or tap input) having a contact point on a part of the sixth UI object (2016), receiving an input to drag from the part of the sixth UI object (2016) toward the direction (2025) in which the 3D representation (2015) of the object is to be moved, receiving an input to release the touch input at a position where the position of the 3D representation (2015) of the object is to be changed, and receiving a long press input having a contact point on an object (2010) having a shape in which the 3D representation (2015) of the object is to be changed.For example, user input (2020) can be received through a display (230) (e.g., a touchscreen).

[0156] The electronic device (200) can transition from state (2000) to state (2030) based on user input (2020). In state (2030), at least one processor (210) can identify, based on user input (2020), an object (2010) on a position where the touch input is released at a position where the position of the 3D representation (2015) of the object is to be changed. For example, at least one processor (210) can identify the shape of the object (2010). For example, at least one processor (210) can change the position of the 3D representation (2035) of the object and change the shape of the 3D representation (2035) of the object to correspond to the shape of the object (2010) based on user input (2020). For example, at least one processor (210) can change the shape of the 3D representation (2035) of an object by performing plane morphing. For example, the 3D representation (2035) of an object having the changed shape can be represented as a set of polygons that have undergone tessellation or as a NURBS (non-uniform rational B spline) surface. For example, at least one processor (210) can display the 3D representation (2035) of an object having a shape corresponding to the shape of the object (2010) on the object (2010) within the UI (1401).

[0157] FIG. 21 is a block diagram of an electronic device in a network environment according to various embodiments.

[0158] Referring to FIG. 21, in a network environment (2100), an electronic device (2101) may communicate with an electronic device (2102) through a first network (2198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (2104) or a server (2108) through a second network (2199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (2101) may communicate with the electronic device (2104) through a server (2108). According to one embodiment, the electronic device (2101) may include a processor (2120), memory (2130), input module (2150), sound output module (2155), display module (2160), audio module (2170), sensor module (2176), interface (2177), connection terminal (2178), haptic module (2179), camera module (2180), power management module (2188), battery (2189), communication module (2190), subscriber identification module (2196), or antenna module (2197). In some embodiments, at least one of these components (e.g., connection terminal (2178)) may be omitted from the electronic device (2101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (2176), camera module (2180), or antenna module (2197)) may be integrated into a single component (e.g., display module (2160)).

[0159] The processor (2120) can, for example, execute software (e.g., program (2140)) to control at least one other component (e.g., hardware or software component) of the electronic device (2101) connected to the processor (2120) and perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (2120) can store commands or data received from other components (e.g., sensor module (2176) or communication module (2190)) in volatile memory (2132), process the commands or data stored in volatile memory (2132), and store the resulting data in non-volatile memory (2134). According to one embodiment, the processor (2120) may include a main processor (2121) (e.g., a central processing unit or an application processor) or an auxiliary processor (2123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (2101) includes a main processor (2121) and an auxiliary processor (2123), the auxiliary processor (2123) may be configured to use less power than the main processor (2121) or to be specialized for a specified function. The auxiliary processor (2123) may be implemented separately from the main processor (2121) or as part thereof.

[0160] The auxiliary processor (2123) may control at least some of the functions or states associated with at least one component of the electronic device (2101) (e.g., display module (2160), sensor module (2176), or communication module (2190)) on behalf of the main processor (2121) while the main processor (2121) is in an inactive (e.g., sleep) state, or together with the main processor (2121) while the main processor (2121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (2123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (2180) or communication module (2190)). According to one embodiment, the auxiliary processor (2123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (2101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (2108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0161] The memory (2130) can store various data used by at least one component of the electronic device (2101) (e.g., processor (2120) or sensor module (2176)). The data may include, for example, software (e.g., program (2140)) and input or output data for related commands. The memory (2130) may include volatile memory (2132) or non-volatile memory (2134).

[0162] The program (2140) may be stored as software in memory (2130) and may include, for example, an operating system (2142), middleware (2144), or an application (2146).

[0163] The input module (2150) can receive commands or data to be used for a component of the electronic device (2101) (e.g., processor (2120)) from outside the electronic device (2101) (e.g., user). The input module (2150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0164] The sound output module (2155) can output a sound signal to the outside of the electronic device (2101). The sound output module (2155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0165] The display module (2160) can visually provide information to an external (e.g., user) of the electronic device (2101). The display module (2160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (2160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0166] The audio module (2170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (2170) can acquire sound through the input module (2150) or output sound through the sound output module (2155) or an external electronic device (e.g., electronic device (2102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (2101).

[0167] The sensor module (2176) can detect the operating state of the electronic device (2101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (2176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0168] The interface (2177) may support one or more specified protocols that can be used for the electronic device (2101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (2102)). According to one embodiment, the interface (2177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0169] The connection terminal (2178) may include a connector through which the electronic device (2101) can be physically connected to an external electronic device (e.g., electronic device (2102)). According to one embodiment, the connection terminal (2178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0170] The haptic module (2179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (2179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0171] The camera module (2180) can capture still images and video. According to one embodiment, the camera module (2180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0172] The power management module (2188) can manage the power supplied to the electronic device (2101). According to one embodiment, the power management module (2188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0173] The battery (2189) can supply power to at least one component of the electronic device (2101). According to one embodiment, the battery (2189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0174] The communication module (2190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (2101) and an external electronic device (e.g., electronic device (2102), electronic device (2104), or server (2108)), and the performance of communication through the established communication channel. The communication module (2190) may include one or more communication processors that operate independently of the processor (2120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (2190) may include a wireless communication module (2192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (2194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (2104) via a first network (2198) (e.g., a short-range communication network such as Bluetooth, Wi-Fi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (2199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (2192) can identify or authenticate the electronic device (2101) within a communication network such as the first network (2198) or the second network (2199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (2196).

[0175] The wireless communication module (2192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (2192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (2192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beamforming, or large-scale antenna. The wireless communication module (2192) can support various requirements specified in the electronic device (2101), external electronic device (e.g., electronic device (2104)), or network system (e.g., second network (2199)). According to one embodiment, the wireless communication module (2192) can support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0176] The antenna module (2197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (2197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (2197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (2198) or a second network (2199), may be selected from the plurality of antennas, for example, by a communication module (2190). The signal or power may be transmitted or received between the communication module (2190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (2197).

[0177] According to various embodiments, the antenna module (2197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0178] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0179] According to one embodiment, commands or data may be transmitted or received between the electronic device (2101) and an external electronic device (2104) through a server (2108) connected to a second network (2199). Each of the external electronic devices (2102, or 2104) may be the same or a different type of device as the electronic device (2101). According to one embodiment, all or part of the operations performed on the electronic device (2101) may be performed on one or more of the external electronic devices (2102, 2104, or 2108). For example, if the electronic device (2101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (2101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (2101). The electronic device (2101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (2101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (2104) may include an Internet of Things (IoT) device. The server (2108) may be an intelligent server using machine learning and / or neural networks.According to one embodiment, an external electronic device (2104) or server (2108) may be included within the second network (2199). The electronic device (2101) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0180] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0181] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0182] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0183] Various embodiments of the present document may be implemented as software (e.g., program (2140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (2136) or external memory (2138)) readable by a machine (e.g., electronic device (2101)). For example, a processor (e.g., processor (2120)) of the machine (e.g., electronic device (2101)) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-transient' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0184] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0185] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0186] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs.

[0187] Figure 22 is a schematic diagram of an exemplary AI system.

[0188] Referring to FIG. 22, the AI ​​system (2200) may include an input / output interface (2210), an AI framework (2220), a generative AI model (2230), and / or a knowledge repository (2290).

[0189] The input / output interface (2210) can receive input. The input may include user input and / or data acquired or generated by an electronic device (e.g., the electronic device (200) or electronic device (2101) described above). The data may include images, videos, and / or sensor data generated by at least one processor of the electronic device (e.g., at least one processor (210) or processor (2120)), such as illuminance data around the electronic device acquired from a sensor or sensor hub (e.g., auxiliary processor (2123), attitude data (or orientation data) of the electronic device, temperature inside the electronic device (e.g., display (230)), or temperature of at least one processor (210), size information of the display area of ​​the display (230), and / or images acquired through an image sensor of the electronic device (e.g., included in a camera module (2180)). The user input may include natural language, touch data obtained through a touch circuit included within the display panel (e.g., used to identify input from a finger and / or stylus), an image displayed (and / or to be displayed) on the display panel, and / or video. By example, without limitation, the user input may be received by an input / output interface (2210) along with context information. The context information may be described as additional information obtained in relation to the user input. The context information may be related to the state at the time the user input is received (e.g., the state of the electronic device and / or the state of the surroundings of the electronic device (e.g., user state)). For example, the context information may include information about one or more software applications executed within the electronic device at the time the user input is received.For example, the above situation information may include information regarding the location of the electronic device (or the location of the user of the electronic device) at the time the user input is received. For example, the user input may be integrated with the situation information. For example, the user input with the situation information integrated as the input may be received by the input / output interface (2210).

[0190] The input / output interface (2210) may transmit (or provide) an output. The output may include a result (or result information) generated or obtained by the AI ​​system (2200) based on at least part of the input. The format of the output may vary. For example, the output may include natural language. For example, the output may include content (e.g., media content and / or multimedia content). For example, the output may include an action related to the user of the electronic device. For example, the output may have a format according to the user settings of the electronic device.

[0191] The input / output interface (2210) can be described as a user question / response interface (2210).

[0192] The AI ​​framework (2220) can be used to obtain information (or data) about the input from the input / output interface (2210) and to control one or more components related to the AI ​​system (2200) using the obtained information.

[0193] For example, a prompt design component (2221) within an AI framework (2220) can generate or obtain prompts for a generative AI model (2230) (e.g., including a large language model (LLM) or a large multimodal model (LMM)) using the acquired information. For example, the prompt design component (2221) may be described as an AI component that uses a learning algorithm and / or a neural network to provide prompts that are enhanced over time. For example, the prompt design component (2221) can generate or obtain prompts by accessing a knowledge component (e.g., a knowledge repository (2290)) containing user preference data, a prompt library, and / or prompt examples using the acquired information. The generated prompts may be provided to the generative AI model (2230) (e.g., including an LLM or LMM).

[0194] For example, an API / plugin management component (2222) within the AI ​​framework (2220) may be used to support communication for additional information requested (or induced) in relation to the prompt provided (or to be provided) to the generative AI model (2230). For example, the API / plugin management component (2222) may be used to create or establish a channel for communication with various data sources (e.g., knowledge repository (2290)). For example, the API / plugin management component (2222) may support access to at least some of the data sources. For example, the API / plugin management component (2222) may be used to request another component (e.g., application / service component (2280)) that performs feedback (or response) according to the prompt. As a non-limiting example, information obtained (or generated) through the API / plugin management component (2222) may be provided to the prompt design component (2221) for generating a prompt. As a non-limiting example, information obtained (or generated) through the API / plugin management component (2222) may be provided to the generative AI model (2230).

[0195] For example, an improvement component (2223) within the AI ​​framework (2220) can at least partially tune (or adjust) (or change) the result (e.g., content) obtained (or output) from the generative AI model (2230). For example, the improvement component (2223) can determine or verify whether the content obtained from the generative AI model (2230) is related to the input. For example, the improvement component (2223) can determine or verify whether the content obtained from the generative AI model (2230) contains biased content. For example, the improvement component (2223) can determine or verify whether the content obtained from the generative AI model (2230) contains harmful content. For example, the improvement component (2223) can support or assist in performing additional processing to improve the content obtained from the generative AI model (2230). For example, the improvement component (2223) may support providing a hint to the user to improve the content.

[0196] A generative AI model (2230) can be described as an artificial intelligence neural network that generates feedback in response to a prompt. For example, the feedback may include additional data and / or information relative to the prompt, but relative to the prompt. For example, the feedback may include new content relative to the prompt. For example, the generative AI model (2230) may include a model that generates images and / or a model that generates language. For example, the model that generates images may include a generative adversarial network (GAN) and / or a variational autoencoder (VAE). For example, the model that generates images may include a diffusion-based generative model (e.g., a transformer VAE). For example, the model that generates language may include CHAT-GPT 3 and / or CHAT-GPT 4. For example, the generative AI model (2230) may include an LMM that generates the feedback by recognizing text, images, and / or voice.

[0197] As an example without limitation, the AI ​​framework (2220) and / or generative AI model (2230) may be included within an AI module (e.g., including a processing circuit) within the electronic device (200). For example, the AI ​​module may be operatively coupled with at least one processor of the electronic device (200) (e.g., at least one processor (210) or processor (2120)). For example, the AI ​​module may be operatively coupled with a display driving circuit of the electronic device. For example, the AI ​​module may be operatively coupled with a sensor hub of the electronic device for one or more sensors within the electronic device.

[0198] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs.

[0199] The electronic device described above (e.g., the electronic device (200) of FIG. 2) may include at least one processor (e.g., at least one processor (210) of FIG. 2) comprising a processing circuit, a display (e.g., the display (230) of FIG. 2), and a memory (e.g., the memory (220) of FIG. 2) comprising one or more storage media configured to store one or more programs configured to be executed individually or collectively by the at least one processor. The one or more programs may include instructions that cause the electronic device to display a 2D image containing an object through the display. The one or more programs may include instructions that cause the electronic device to receive a first input for generating a 3D representation of the object. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the 2D image containing the object based on the first input. The above one or more programs may include instructions that cause the electronic device to receive a second input for displaying the 3D representation of the object. The above one or more programs may include instructions that cause the electronic device to display a UI including a first UI object and the 3D representation of the object through the display based on the second input. The first UI object may be available to change the pose of the 3D representation of the object. The above one or more programs may include instructions that cause the electronic device to determine the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object.The above one or more programs may include instructions that cause the electronic device to display a second UI object and a 2D representation of the object having the determined pose within the UI through the display. The second UI object may be available to change the position where the 2D representation of the object is displayed.

[0200] For example, the one or more programs may include instructions that cause the electronic device to determine the location where the 2D representation of the object is to be displayed within the UI based on other user input received using the second UI object. The one or more programs may include instructions that cause the electronic device to display the 2D representation of the object at the determined location within the UI through the display.

[0201] For example, the one or more programs may include instructions that cause the electronic device to store the 2D representation of the object in conjunction with the 3D representation of the object. The one or more programs may include instructions that cause the electronic device to receive other user input to change the 2D representation of the object to the 3D representation of the object while displaying the 2D representation of the object within the UI. The one or more programs may include instructions that cause the electronic device to display the 3D representation of the object having the first UI object and the determined pose within the UI through the display based on the other user input.

[0202] For example, the user input may include an input that drags the 3D representation of the object in a direction to rotate it using the first UI object. The one or more programs may include instructions that cause the electronic device to change the orientation of the 3D representation of the object by rotating the 3D representation of the object in the direction within the UI based on the user input received using the first UI object. The one or more programs may include instructions that cause the electronic device to determine the orientation of the 3D representation of the object based on the changed orientation of the 3D representation of the object.

[0203] For example, the other user input may include an input that drags the 2D representation of the object in the direction to be moved using the second UI object. The one or more programs may include instructions that cause the electronic device to determine the position where the 2D representation of the object is to be displayed within the UI by moving the 2D representation of the object in the direction within the UI based on the other user input received using the second UI object.

[0204] For example, the 2D representation of the object may be a first 2D representation of the object. The one or more programs may include instructions that cause the electronic device to obtain a second 2D representation of the object viewed in a first direction from the 2D image containing the object viewed in a first direction, based on the first input. The one or more programs may include instructions that cause the electronic device to identify one or more images, each containing the object, among the images stored in the electronic device. The one or more programs may include instructions that cause the electronic device to obtain a third 2D representation of the object viewed in a second direction different from the first direction from at least some of the one or more images. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0205] For example, the one or more programs may include instructions that cause the electronic device to identify whether a first shape of the object included in at least a portion of the one or more images corresponds to a second shape of the object included in the 2D image. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object, based on identifying that the first shape of the object corresponds to the second shape of the object.

[0206] For example, the one or more programs may include instructions that cause the electronic device to determine an image among the plurality of images to be used to generate the 3D representation of the object by identifying a plurality of images each containing the object viewed from the second direction among the images stored in the electronic device, and comparing the shape of each of the objects viewed from the second direction included in each of the plurality of images with the shape of the second 2D representation of the object. The one or more programs may include instructions that cause the electronic device to obtain the third 2D representation of the object from the determined image. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0207] For example, the one or more programs may include instructions that cause the electronic device to determine an image among the plurality of images to be used to generate the 3D representation of the object by comparing the time at which each of the plurality of images was acquired with the time at which the 2D image was acquired, based on identifying a plurality of images among the images stored in the electronic device that each include the object viewed from the second direction. The one or more programs may include instructions that cause the electronic device to obtain the third 2D representation of the object from the determined image. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0208] For example, the 2D representation of the object may be a first 2D representation of the object. The one or more programs may include instructions that cause the electronic device to obtain a second 2D representation of the object viewed in a first direction from the 2D image containing the object viewed in a first direction, based on the first input. The one or more programs may include instructions that cause the electronic device to generate a 3D representation of the object using the second 2D representation of the object. The one or more programs may include instructions that cause the electronic device to obtain one or more images each containing the object after the 3D representation of the object is generated. The one or more programs may include instructions that cause the electronic device to obtain a third 2D representation of the object viewed in a second direction different from the first direction from at least some of the one or more images. The above one or more programs may include instructions that cause the electronic device to generate another 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0209] For example, the 2D representation of the object may be a first 2D representation of the object. The one or more programs may include instructions that cause the electronic device to obtain a second 2D representation of the object viewed in a first direction from the 2D image containing the object viewed in a first direction, based on the first input. The one or more programs may include instructions that cause the electronic device to identify a reference direction in which the 2D representation of the object is viewed, to be used together with the second 2D representation of the object to generate the 3D representation of the object. The one or more programs may include instructions that cause the electronic device to identify whether an image containing the object viewed in the reference direction is stored in the electronic device. The above one or more programs may include instructions that cause the electronic device to display information to acquire an image including the object shown in the reference direction through the display, based on identifying that an image including the object shown in the reference direction is not stored in the electronic device.

[0210] For example, the one or more programs may include instructions that cause the electronic device to obtain a third 2D representation of the object seen in the reference direction from the other image based on obtaining another image containing the object seen in the reference direction. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0211] The above-described method may be performed within an electronic device including a display. The method may include an operation of displaying a 2D image containing an object through the display. The method may include an operation of receiving a first input for generating a 3D representation of the object. The method may include an operation of generating the 3D representation of the object using the 2D image containing the object based on the first input. The method may include an operation of receiving a second input for displaying the 3D representation of the object. The method may include an operation of displaying a UI including a first UI object and the 3D representation of the object through the display based on the second input. The first UI object may be available to change the pose of the 3D representation of the object. The method may include an operation of determining the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object. The above method may include an operation of displaying a second UI object and a 2D representation of the object having the determined pose within the UI through the display. The second UI object may be available to change the position where the 2D representation of the object is displayed.

[0212] For example, the above method may include an operation to determine a location where the 2D representation of the object is to be displayed within the UI based on other user input received using the second UI object. The above method may include an operation to display the 2D representation of the object at the determined location within the UI through the display.

[0213] For example, the above method may include an operation of storing the 2D representation of the object in conjunction with the 3D representation of the object. The above method may include an operation of receiving another user input to change the 2D representation of the object to the 3D representation of the object while displaying the 2D representation of the object within the UI. The above method may include an operation of displaying the 3D representation of the object having the first UI object and the determined pose within the UI through the display based on the other user input.

[0214] For example, the user input may include an input that drags the 3D representation of the object in a direction to rotate it using the first UI object. The method may include an operation to change the orientation of the 3D representation of the object by rotating the 3D representation of the object in the direction within the UI based on the user input received using the first UI object. The method may include an operation to determine the orientation of the 3D representation of the object based on the changed orientation of the 3D representation of the object.

[0215] For example, the 2D representation of the object may be a first 2D representation of the object. The method may include an operation of obtaining a second 2D representation of the object viewed in a first direction from a 2D image containing the object viewed in a first direction, based on the first input. The method may include an operation of identifying one or more images, each having the object, among images stored in the electronic device. The method may include an operation of obtaining a third 2D representation of the object viewed in a second direction different from the first direction from at least some of the one or more images. The method may include an operation of generating the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0216] For example, the above method may include an operation of identifying whether a first shape of the object included in at least a portion of the one or more images corresponds to a second shape of the object included in the 2D image. Based on identifying that the first shape of the object corresponds to the second shape of the object, the above method may include an operation of generating the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0217] For example, the above method may include an operation of determining an image among the plurality of images to be used to generate the 3D representation of the object by identifying a plurality of images each containing the object viewed from the second direction among the images stored in the electronic device, and comparing the shape of each of the objects viewed from the second direction included in each of the plurality of images with the shape of the second 2D representation of the object. The above method may include an operation of obtaining the third 2D representation of the object from the determined image. The above method may include an operation of generating the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0218] For example, the above method may include an operation of determining an image among the plurality of images to be used to generate the 3D representation of the object by comparing the time at which each of the plurality of images was acquired with the time at which the 2D image was acquired, based on identifying a plurality of images among the images stored in the electronic device, each including the object viewed from the second direction. The above method may include an operation of acquiring the third 2D representation of the object from the determined image. The above method may include an operation of generating the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0219] For example, the 2D representation of the object may be a first 2D representation of the object. The method may include an operation of obtaining a second 2D representation of the object viewed in a first direction from a 2D image containing the object viewed in a first direction, based on the first input. The method may include an operation of generating a 3D representation of the object using the second 2D representation of the object. The method may include an operation of obtaining one or more images each containing the object after the 3D representation of the object is generated. The method may include an operation of obtaining a third 2D representation of the object viewed in a second direction different from the first direction from at least some of the one or more images. The method may include an operation of generating another 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0220] For example, the 2D representation of the object may be a first 2D representation of the object. The method may include an operation of obtaining a second 2D representation of the object viewed in a first direction from a 2D image containing the object viewed in a first direction, based on the first input. The method may include an operation of identifying a reference direction in which the 2D representation of the object is viewed, to be used together with the second 2D representation of the object to generate the 3D representation of the object. The method may include an operation of identifying whether an image containing the object viewed in the reference direction is stored in the electronic device. Based on identifying that an image containing the object viewed in the reference direction is not stored in the electronic device, the method may include an operation of displaying information to obtain an image containing the object viewed in the reference direction through the display.

[0221] For example, the above method may include an operation of obtaining a third 2D representation of the object viewed in the reference direction from the other image based on obtaining another image containing the object viewed in the reference direction. The above method may include an operation of generating the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object.

[0222] The above-described non-transient computer-readable storage medium may store one or more programs. The one or more programs may include instructions that cause the electronic device to display a 2D image containing an object through the display when executed by the electronic device including a display. The one or more programs may include instructions that cause the electronic device to receive a first input for generating a 3D representation of the object when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the 2D image containing the object based on the first input when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to receive a second input for displaying the 3D representation of the object when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to display a UI including a first UI object and the 3D representation of the object through the display, based on the second input when executed by the electronic device. The first UI object may be available to change the pose of the 3D representation of the object. The above one or more programs may include instructions that cause the electronic device to determine the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object when executed by the electronic device.The above one or more programs may include instructions that cause the electronic device to display a second UI object and a 2D representation of the object having the determined pose within the UI through the display when executed by the electronic device. The second UI object may be available to change the position where the 2D representation of the object is displayed.

[0223] For example, the one or more programs may include instructions that cause the electronic device to store the 2D representation of the object in conjunction with the 3D representation of the object when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to receive other user input to change the 2D representation of the object to the 3D representation of the object while the 2D representation of the object is displayed in the UI when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to display the 3D representation of the object having the first UI object and the determined pose in the UI through the display, based on the other user input when executed by the electronic device.

[0224] For example, the user input may include an input that drags the 3D representation of the object in a direction to rotate it using the first UI object. The one or more programs may include instructions that cause the electronic device to change the orientation of the 3D representation of the object by rotating the 3D representation of the object in the direction within the UI, based on the user input received using the first UI object when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to determine the orientation of the 3D representation of the object based on the changed orientation of the 3D representation of the object when executed by the electronic device.

[0225] For example, the other user input may include an input that drags the 2D representation of the object in the direction to be moved using the second UI object. The one or more programs may include instructions that cause the electronic device to determine the position where the 2D representation of the object is displayed within the UI by moving the 2D representation of the object in the direction within the UI, based on the other user input received using the second UI object when executed by the electronic device.

[0226] For example, the 2D representation of the object may be a first 2D representation of the object. The one or more programs may include instructions that cause the electronic device to obtain a second 2D representation of the object shown in the first direction from the 2D image containing the object shown in the first direction, based on the first input, when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to identify one or more images containing the object among the images stored in the electronic device when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to obtain a third 2D representation of the object shown in a second direction different from the first direction from at least some of the one or more images when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object when executed by the electronic device.

[0227] For example, the one or more programs may include instructions that cause the electronic device to identify whether, when executed by the electronic device, a first shape of the object included in at least a portion of the one or more images corresponds to a second shape of the object included in the 2D image. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object, based on identifying that, when executed by the electronic device, the first shape of the object corresponds to the second shape of the object.

[0228] For example, the above one or more programs may include instructions that cause the electronic device to determine, when executed by the electronic device, to determine an image among the plurality of images to be used to generate the 3D representation of the object by identifying a plurality of images each containing the object shown in the second direction among the images stored in the electronic device, and comparing the shape of each of the objects shown in the second direction included in each of the plurality of images with the shape of the second 2D representation of the object. The above one or more programs may include instructions that cause the electronic device to obtain the third 2D representation of the object from the determined image when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object when executed by the electronic device.

[0229] For example, the above one or more programs may include instructions that cause the electronic device to determine, when executed by the electronic device, an image among the plurality of images to be used to generate the 3D representation of the object by comparing the time at which each of the plurality of images was acquired with the time at which the 2D image was acquired, based on identifying a plurality of images among the images stored in the electronic device that each include the object viewed from the second direction. The above one or more programs may include instructions that cause the electronic device to obtain the third 2D representation of the object from the determined image when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object when executed by the electronic device.

[0230] For example, the 2D representation of the object may be a first 2D representation of the object. The one or more programs may include instructions that cause the electronic device to obtain a second 2D representation of the object viewed in a first direction from the 2D image containing the object viewed in a first direction, based on the first input, when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to generate a 3D representation of the object using the second 2D representation of the object when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to obtain one or more images each containing the object after the 3D representation of the object is generated when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to obtain a third 2D representation of the object viewed from at least a portion of the one or more images in a second direction different from the first direction when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to generate another 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object when executed by the electronic device.

[0231] For example, the 2D representation of the object may be a first 2D representation of the object. The one or more programs may include instructions that cause the electronic device to obtain a second 2D representation of the object seen in a first direction from the 2D image containing the object seen in a first direction, based on the first input, when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to identify a reference direction in which the 2D representation of the object is viewed, to be used together with the second 2D representation of the object to generate the 3D representation of the object, when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to identify whether an image containing the object seen in the reference direction is stored in the electronic device when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to display information to acquire an image including the object shown in the reference direction through the display, based on identifying that, when executed by the electronic device, an image including the object shown in the reference direction is not stored in the electronic device.

[0232] For example, the one or more programs may include instructions that cause the electronic device to obtain a third 2D representation of the object seen in the reference direction from the other image, based on obtaining the other image containing the object seen in the reference direction when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object when executed by the electronic device.

[0233] The electronic device described above may include at least one processor comprising a processing circuit, a display, and a memory comprising one or more storage media configured to store one or more programs configured to be executed individually or collectively by the at least one processor. The one or more programs may include instructions that cause the electronic device to display a 2D image containing an object viewed from a first direction through the display. The one or more programs may include instructions that cause the electronic device to receive an input for generating a 3D representation of the object using the 2D image containing the object. The one or more programs may include instructions that cause the electronic device to obtain a first 2D representation of the object viewed from the first direction from the 2D image based on the input. The one or more programs may include instructions that cause the electronic device to identify one or more images containing the object among the images stored in the electronic device. The above one or more programs may include instructions that cause the electronic device to obtain a second 2D representation of the object viewed from at least a portion of the one or more images in a second direction different from the first direction. The above one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object.

[0234] For example, the one or more programs may include instructions that cause the electronic device to receive other input for displaying the 3D representation of the object. The one or more programs may include instructions that cause the electronic device to display a UI including a first UI object and the 3D representation of the object through the display based on the other input. The first UI object may be available to change the pose of the 3D representation of the object. The one or more programs may include instructions that cause the electronic device to determine the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object. The one or more programs may include instructions that cause the electronic device to display a second UI object and a third 2D representation of the object having the determined pose within the UI through the display. The above second UI object may be available to change the position where the above third 2D representation of the object is displayed.

[0235] For example, the one or more programs may include instructions that cause the electronic device to determine the location where the third 2D representation of the object is to be displayed within the UI based on other user input received using the second UI object. The one or more programs may include instructions that cause the electronic device to display the third 2D representation of the object at the determined location within the UI through the display.

[0236] For example, the one or more programs may include instructions that cause the electronic device to store the third 2D representation of the object in association with the 3D representation of the object. The one or more programs may include instructions that cause the electronic device to receive other user input to change the third 2D representation of the object to the 3D representation of the object while displaying the third 2D representation of the object within the UI. The one or more programs may include instructions that cause the electronic device to display the first UI object and the 3D representation of the object having the determined pose within the UI through the display, based on the other user input.

[0237] For example, the one or more programs may include instructions that cause the electronic device to identify whether a first shape of the object included in at least a portion of the one or more images corresponds to a second shape of the object included in the 2D image. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object, based on identifying that the first shape of the object corresponds to the second shape of the object.

[0238] For example, the above one or more programs may include instructions that cause the electronic device to determine an image among the plurality of images to be used to generate the 3D representation of the object by identifying a plurality of images each containing the object viewed from the second direction among the images stored in the electronic device, and comparing the shape of each of the objects viewed from the second direction included in each of the plurality of images with the shape of the first 2D representation of the object. The above one or more programs may include instructions that cause the electronic device to obtain the second 2D representation of the object from the determined image. The above one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object.

[0239] For example, the one or more programs may include instructions that cause the electronic device to determine an image among the plurality of images to be used to generate the 3D representation of the object by comparing the time at which each of the plurality of images is acquired with the time at which the 2D image is acquired, based on identifying a plurality of images among the images stored in the electronic device that each include the object viewed from the second direction. The one or more programs may include instructions that cause the electronic device to obtain the second 2D representation of the object from the determined image. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object.

[0240] For example, the one or more programs may include instructions that cause the electronic device to acquire one or more other images each containing the object after the 3D representation of the object is generated. The one or more programs may include instructions that cause the electronic device to acquire a third 2D representation of the object seen from at least some of the one or more other images in a third direction different from the first direction and the second direction. The one or more programs may include instructions that cause the electronic device to generate another 3D representation of the object using the first 2D representation of the object, the second 2D representation of the object, and the third 2D representation of the object.

[0241] The above-described method may be performed within an electronic device including a display. The method may include an operation of displaying a 2D image containing an object viewed from a first direction through the display. The method may include an operation of receiving an input for generating a 3D representation of the object using the 2D image containing the object. The method may include an operation of obtaining a first 2D representation of the object viewed from the first direction from the 2D image based on the input. The method may include an operation of identifying one or more images, each containing the object, among images stored within the electronic device. The method may include an operation of obtaining a second 2D representation of the object viewed from a second direction different from the first direction from at least some of the one or more images. The method may include an operation of generating the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object.

[0242] For example, the above method may include an operation of receiving another input for displaying the 3D representation of the object. The above method may include an operation of displaying a UI including a first UI object and the 3D representation of the object through the display based on the other input. The first UI object may be available to change the pose of the 3D representation of the object. The above method may include an operation of determining the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object. The above method may include an operation of displaying a second UI object and a third 2D representation of the object having the determined pose within the UI through the display. The second UI object may be available to change the position where the third 2D representation of the object is displayed.

[0243] For example, the above method may include an operation to determine a location where the third 2D representation of the object is to be displayed within the UI based on other user input received using the second UI object. The above method may include an operation to display the third 2D representation of the object at the determined location within the UI through the display.

[0244] For example, the above method may include an operation of storing the third 2D representation of the object in conjunction with the 3D representation of the object. The above method may include an operation of receiving another user input to change the third 2D representation of the object to the 3D representation of the object while displaying the third 2D representation of the object within the UI. The above method may include an operation of displaying the first UI object and the 3D representation of the object having the determined pose within the UI through the display based on the other user input.

[0245] For example, the above method may include an operation of identifying whether a first shape of the object included in at least a portion of the one or more images corresponds to a second shape of the object included in the 2D image. Based on identifying that the first shape of the object corresponds to the second shape of the object, the above method may include an operation of generating the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object.

[0246] For example, the above method may include an operation of determining an image among the plurality of images to be used to generate the 3D representation of the object by identifying a plurality of images each containing the object viewed from the second direction among the images stored in the electronic device, and comparing the shape of each of the objects viewed from the second direction included in each of the plurality of images with the shape of the first 2D representation of the object. The above method may include an operation of obtaining the second 2D representation of the object from the determined image. The above method may include an operation of generating the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object.

[0247] For example, the above method may include an operation of determining an image among the plurality of images to be used to generate the 3D representation of the object by comparing the time at which each of the plurality of images is acquired with the time at which the 2D image is acquired, based on identifying a plurality of images among the images stored in the electronic device, each including the object viewed from the second direction. The above method may include an operation of acquiring the second 2D representation of the object from the determined image. The above method may include an operation of generating the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object.

[0248] For example, the above method may include the operation of acquiring one or more other images each containing the object after the 3D representation of the object is generated. The above method may include the operation of acquiring a third 2D representation of the object seen from a third direction different from the first direction and the second direction from at least some of the one or more other images. The above method may include the operation of generating another 3D representation of the object using the first 2D representation of the object, the second 2D representation of the object, and the third 2D representation of the object.

[0249] The above-described non-transient computer-readable storage medium may store one or more programs. The one or more programs may include instructions that cause the electronic device to display a 2D image containing an object viewed from a first direction through the display when executed by the electronic device including a display. The one or more programs may include instructions that cause the electronic device to receive input for generating a 3D representation of the object using the 2D image containing the object when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to obtain a first 2D representation of the object viewed from the first direction from the 2D image based on the input when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to identify one or more images containing the object among the images stored within the electronic device when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to obtain a second 2D representation of the object viewed from at least a portion of the one or more images in a second direction different from the first direction when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object when executed by the electronic device.

[0250] For example, the one or more programs may include instructions that cause the electronic device to receive other inputs for displaying the 3D representation of the object when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to display a UI including a first UI object and the 3D representation of the object through the display based on the other inputs when executed by the electronic device. The first UI object may be available to change the pose of the 3D representation of the object. The one or more programs may include instructions that cause the electronic device to determine the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to display, through the display, a second UI object and a third 2D representation of the object having the determined pose within the UI when executed by the electronic device. The second UI object may be available to change the position where the third 2D representation of the object is displayed.

[0251] For example, the one or more programs may include instructions that cause the electronic device to determine the location where the third 2D representation of the object is to be displayed within the UI based on other user input received using the second UI object when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to display the third 2D representation of the object at the determined location within the UI through the display when executed by the electronic device.

[0252] For example, the one or more programs may include instructions that cause the electronic device to store the third 2D representation of the object in association with the 3D representation of the object when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to receive other user input to change the third 2D representation of the object to the 3D representation of the object while displaying the third 2D representation of the object within the UI when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to display the first UI object and the 3D representation of the object having the determined pose within the UI through the display, based on the other user input when executed by the electronic device.

[0253] For example, the one or more programs may include instructions that cause the electronic device to identify whether, when executed by the electronic device, a first shape of the object included in at least a portion of the one or more images corresponds to a second shape of the object included in the 2D image. The one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object, based on identifying that, when executed by the electronic device, the first shape of the object corresponds to the second shape of the object.

[0254] For example, the above one or more programs may include instructions that cause the electronic device to determine, when executed by the electronic device, to determine an image among the plurality of images to be used to generate the 3D representation of the object by identifying a plurality of images each containing the object shown in the second direction among the images stored in the electronic device, and comparing the shape of each of the objects shown in the second direction included in each of the plurality of images with the shape of the first 2D representation of the object. The above one or more programs may include instructions that cause the electronic device to obtain the second 2D representation of the object from the determined image when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object when executed by the electronic device.

[0255] For example, the above one or more programs may include instructions that cause the electronic device to determine, when executed by the electronic device, to determine an image among the plurality of images to be used to generate the 3D representation of the object by comparing the time at which each of the plurality of images is acquired with the time at which the 2D image is acquired, based on identifying a plurality of images among the images stored in the electronic device that each include the object viewed from the second direction. The above one or more programs may include instructions that cause the electronic device to obtain the second 2D representation of the object from the determined image when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object when executed by the electronic device.

[0256] For example, the one or more programs may include instructions that cause the electronic device to obtain one or more different images each containing the object after the 3D representation of the object is generated when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to obtain a third 2D representation of the object seen in a third direction different from the first direction and the second direction from at least some of the one or more different images when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to generate another 3D representation of the object using the first 2D representation of the object, the second 2D representation of the object, and the third 2D representation of the object when executed by the electronic device.

[0257] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs.

Claims

1. In an electronic device, At least one processor including a processing circuit; Display; and Memory comprising one or more programs configured to be executed individually or collectively by at least one processor, and including one or more storage media, The above one or more programs are: Through the above display, a 2D (two-dimensional) image containing an object is displayed; Receiving a first input for generating a 3D (three-dimensional) representation of the above object; Based on the first input above, a 3D representation of the object is generated using the 2D image including the object; Receiving a second input for displaying the 3D representation of the above object; Based on the second input above, a UI including a first UI (user interface) object and a 3D representation of the object is displayed through the display, and the first UI object is available to change the pose of the 3D representation of the object; Determining the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object; and To display a 2D representation of the second UI object and the object having the determined pose within the UI through the above display, Includes instructions that cause the above electronic device, The second UI object above is available to change the position where the 2D representation of the object is displayed, Electronic device.

2. In Claim 1, The above one or more programs are: Based on other user input received using the second UI object, determine the position where the 2D representation of the object is to be displayed within the UI; and To display the 2D representation of the object at the determined location within the UI through the above display, Instructions including those that cause the above electronic device Electronic device.

3. In Claim 1, The above one or more programs are: The 2D representation of the above object is stored in conjunction with the 3D representation of the above object; While displaying the 2D representation of the object within the UI, receiving other user input to change the 2D representation of the object to the 3D representation of the object; and Based on the other user input above, to display the 3D representation of the first UI object and the object having the determined pose within the UI through the display, Instructions including those that cause the above electronic device Electronic device.

4. In Claim 1, The above user input is, Using the first UI object, the input includes dragging in the direction to rotate the 3D representation of the object, and The above one or more programs are: Based on the user input received using the first UI object, the 3D representation of the object is rotated in the direction within the UI to change the orientation of the 3D representation of the object; and To determine the pose of the 3D representation of the object based on the changed pose of the 3D representation of the object, Instructions including those that cause the above electronic device Electronic device.

5. In Claim 2, The other user input mentioned above is, It includes an input for dragging the 2D representation of the object in the direction to be moved using the second UI object, and The above one or more programs are: Based on the other user input received using the second UI object, by moving the 2D representation of the object in the direction within the UI, the position where the 2D representation of the object is to be displayed within the UI is determined. Instructions including those that cause the above electronic device Electronic device.

6. In Claim 1, The 2D representation of the above object is, It is a first 2D representation of the above object, and The above one or more programs are: Based on the first input, a second 2D representation of the object viewed in the first direction is obtained from the 2D image including the object viewed in the first direction; Identifying one or more images among the images stored in the electronic device that each contain the object; Obtaining a third 2D representation of the object viewed from a second direction different from the first direction from at least a portion of the one or more images; and To generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object, Instructions including those that cause the above electronic device Electronic device.

7. In Claim 6, The above one or more programs are: Identifying whether a first shape of the object included in at least a portion of the one or more images corresponds to a second shape of the object included in the 2D image; and Based on identifying that the first shape of the object corresponds to the second shape of the object, the method for generating the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object. Instructions including those that cause the above electronic device Electronic device.

8. In Claim 6, The above one or more programs are: Based on identifying a plurality of images each containing an object viewed from the second direction among the images stored in the electronic device, and by comparing the shapes of each of the objects viewed from the second direction included in the plurality of images with the shapes of the second 2D representation of the object, an image among the plurality of images to be used to generate the 3D representation of the object; Obtaining the third 2D representation of the object from the determined image above; and To generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object, Instructions including those that cause the above electronic device Electronic device.

9. In Claim 6, The above one or more programs are: Based on identifying a plurality of images among the images stored in the electronic device, each including the object viewed from the second direction, and determining an image among the plurality of images to be used to generate the 3D representation of the object by comparing the time points at which each of the plurality of images was acquired with the time point at which the 2D image was acquired; Obtaining the third 2D representation of the object from the determined image above; and To generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object, Instructions including those that cause the above electronic device Electronic device.

10. In Claim 1, The 2D representation of the above object is, It is a first 2D representation of the above object, and The above one or more programs are: Based on the first input, a second 2D representation of the object viewed in the first direction is obtained from the 2D image including the object viewed in the first direction; Generating the 3D representation of the object using the second 2D representation of the object; After the 3D representation of the object is generated, one or more images each containing the object are obtained; Obtaining a third 2D representation of the object viewed from a second direction different from the first direction from at least a portion of the one or more images; and To generate another 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object, Instructions including those that cause the above electronic device Electronic device.

11. In Claim 1, The 2D representation of the above object is, It is a first 2D representation of the above object, and The above one or more programs are: Based on the first input, a second 2D representation of the object viewed in the first direction is obtained from the 2D image including the object viewed in the first direction; Identifying a reference direction in which the 2D representation of the object, to be used together with the second 2D representation of the object to generate the 3D representation of the object, is displayed; Identifying whether an image including the object viewed from the above reference direction is stored in the electronic device; and Based on identifying that an image including the object viewed in the reference direction is not stored in the electronic device, information to acquire an image including the object viewed in the reference direction through the display, Instructions including those that cause the above electronic device Electronic device.

12. In Claim 11, The above one or more programs are: Based on obtaining another image including the object viewed from the reference direction, obtaining a third 2D representation of the object viewed from the reference direction from the other image; and To generate the 3D representation of the object using the second 2D representation of the object and the third 2D representation of the object, Instructions including those that cause the above electronic device Electronic device.

13. In electronic devices, At least one processor including a processing circuit; Display; and Memory comprising one or more programs configured to be executed individually or collectively by at least one processor, and including one or more storage media, The above one or more programs are: Through the above display, a 2D image including an object viewed from a first direction is displayed; Receiving input for generating a 3D (three-dimensional) representation of an object using the 2D image including the object; Based on the above input, obtain a first 2D representation of the object viewed from the first direction from the 2D image; Identifying one or more images among the images stored in the electronic device that each contain the object; Obtaining a second 2D representation of the object viewed in a second direction different from the first direction from at least a portion of the one or more images; and To generate the 3D representation of the object using the first 2D representation of the object and the second 2D representation of the object, Instructions including those that cause the above electronic device Electronic device.

14. In Claim 13, The above one or more programs are: Receiving other inputs for displaying the 3D representation of the above object, and Based on the other inputs mentioned above, a UI including a first UI object and a 3D representation of the object is displayed through the display, and the first UI object is available to change the pose of the 3D representation of the object; Determining the pose of the 3D representation of the object by changing the pose of the 3D representation of the object based on user input received using the first UI object; and To display, through the above display, a second UI object and a third 2D representation of the object having the determined pose within the UI; Includes instructions that cause the above electronic device, The above second UI object is available to change the position where the above third 2D representation of the object is displayed, Electronic device.

15. In Claim 14, The above one or more programs are: Based on other user input received using the second UI object, determine the position where the third 2D representation of the object is to be displayed within the UI; and To display the third 2D representation of the object at the determined location within the UI through the above display, Instructions including those that cause the above electronic device Electronic device.

Images