Electronic device and method for generating artificial intelligence image by using user input

Abstract

The present disclosure relates to an electronic device and method for generating an artificial intelligence image by using a user input, the electronic device and method being capable of: displaying a preview image of a camera on a display unit; generating at least one artificial intelligence graphic object on the basis of at least a part of the user input when the user input for the preview image is received; and displaying the at least one artificial intelligence graphic object on the preview image.

Description

Electronic device and method for generating artificial intelligence images using user input

[0001] The following embodiments relate to an electronic device that generates an artificial intelligence image using user input.

[0002] With the advancement of artificial intelligence technology, revolutionary changes are also taking place in the field of image processing. AI-powered images, in particular, are a growing field in the field of computer vision. AI-powered images are created through the computer's ability to interpret and understand images using machine learning and deep learning algorithms.

[0003] By configuring a prompt using text and entering it, the AI ​​model can generate an image that matches the prompt. Furthermore, by specifying the area the user wishes to create and then entering text as a prompt, the AI ​​model can insert the generated image into the designated area.

[0004] To edit a photo using a generative AI model, a user first writes a text prompt expressing their desired intent, which is then used to generate an image. However, if the AI ​​model fails to interpret the prompt as intended, the resulting image may differ from the intended intent. If the AI ​​model generates the image based solely on the input prompt, without considering surrounding objects, it may produce an awkward image that doesn't blend in with the surrounding objects.

[0005] An electronic device and method for generating an image using user input according to one embodiment are proposed.

[0006] An electronic device according to one embodiment includes an image sensor; a display unit; and a processor, wherein the processor displays a preview image acquired from the image sensor through the display unit, receives a user input for the preview image through the display unit, generates at least one artificial intelligence graphic object based on at least a part of the user input, displays the at least one artificial intelligence graphic object on the preview image, and generates an artificial intelligence image based on at least a part of the preview image and the at least one artificial intelligence graphic object in response to another user input.

[0007] A method for generating an artificial intelligence image according to one embodiment may include: displaying a preview image of a camera on a display unit; receiving a user input for the preview image; generating at least one artificial intelligence graphic object based on at least a portion of the user input; and displaying the at least one artificial intelligence graphic object on the preview image.

[0008] According to one embodiment, an artificial intelligence image generation method utilizes a generative artificial intelligence model more easily through interaction (drawing) on ​​a shooting screen when taking a photo, so that a user can easily obtain a desired photo without going through a separate editing process.

[0009] FIG. 1 is a schematic diagram illustrating a configuration of an electronic device according to one embodiment.

[0010] Figure 2 is a flowchart illustrating a flow for generating an artificial intelligence image according to one embodiment.

[0011] FIG. 3 is a drawing illustrating an example of generating an artificial intelligence image through drawing according to one embodiment.

[0012] FIG. 4 is a drawing illustrating an example of generating an artificial intelligence image by selecting and changing an object added as a drawing from among samples according to one embodiment.

[0013] FIG. 5 is a drawing illustrating an example of generating an artificial intelligence image by changing an object added as a drawing by entering text according to one embodiment.

[0014] FIG. 6 is a diagram illustrating an example of generating an artificial intelligence image using a stored template according to one embodiment.

[0015] FIG. 7 is a drawing illustrating an example of generating an artificial intelligence image by specifying an object to be drawn according to one embodiment.

[0016] FIG. 8 is a diagram illustrating an example of retrieving a stored artificial intelligence image and modifying the artificial intelligence image according to one embodiment.

[0017] FIG. 9 is a drawing illustrating an example of fixing a preview and generating an artificial intelligence image through drawing according to one embodiment.

[0018] FIG. 10 is a schematic diagram illustrating a configuration of an electronic device that generates an artificial intelligence image according to one embodiment.

[0019] FIG. 11 is a block diagram of an electronic device within a network environment according to one embodiment.

[0020] Hereinafter, embodiments are described in detail with reference to the attached drawings. However, the embodiments may be modified in various ways, and the scope of the patent application is not limited or restricted by these embodiments. It should be understood that all modifications, equivalents, or alternatives to the embodiments are included within the scope of the patent application.

[0021] The terms used in the examples are for illustrative purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood to not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0022] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the embodiments pertain. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0023] In addition, when describing with reference to the attached drawings, identical components will be assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted. When describing embodiments, if a detailed description of a related known technology is judged to unnecessarily obscure the gist of the embodiment, the detailed description will be omitted.

[0024] Additionally, terms such as first, second, A, B, (a), (b), etc. may be used to describe components of the embodiments. These terms are only intended to distinguish the components from other components, and the nature, order, or sequence of the components are not limited by the terms. When a component is described as being "connected," "coupled," or "connected" to another component, it should be understood that the component may be directly connected or connected to the other component, but another component may also be "connected," "coupled," or "connected" between each component.

[0025] Components included in one embodiment and components with common functions will be described using the same names in other embodiments. Unless otherwise stated, the descriptions given in one embodiment may also apply to other embodiments, and detailed descriptions will be omitted to the extent of overlap.

[0026] Hereinafter, an artificial intelligence image generation device and method utilizing user input according to an embodiment of the present disclosure will be described in detail with reference to the attached FIGS. 1 to 11.

[0027] FIG. 1 is a schematic diagram illustrating a configuration of an electronic device according to one embodiment.

[0028] Referring to FIG. 1, an electronic device (100) may be configured to include a processor (110), a communication unit (120), a display unit (130), a memory (140), and an image sensor (150).

[0029] The communication unit (120) is a communication interface device including a receiver and a transmitter, and transmits and receives data wired or wirelessly. The communication unit (120) can communicate with an artificial intelligence server (160) that generates artificial intelligence images. In this case, the communication unit (120) may have a configuration corresponding to the communication module (1190) of FIG. 11, and the artificial intelligence server (160) may have a configuration corresponding to the server (1108) of FIG. 11.

[0030] The display unit (130) displays status information (or indicators), limited numbers and characters, moving pictures, and still pictures generated during the operation of the electronic device (100). In addition, the present disclosure may display a preview image, at least one artificial intelligence graphic object, and an artificial intelligence image.

[0031] The display unit (130) of the present disclosure may be a touch screen capable of receiving a touch input. The touch screen includes a display unit that performs a screen output function and a touch sensor that performs a touch input function. Such a touch screen may have a structure in which the touch sensor is arranged on the entire surface of the display unit. The display unit may be formed of a liquid crystal display (LCD), an organic light emitting diode (OLED), or the like. The touch sensor performs a function of receiving a touch input, i.e., a touch event, a double touch event, a touch movement event, and a touch release event. That is, the touch sensor may generate a touch event when an object, for example, a user's finger, touches the touch sensor, and transmit the generated touch event to the processor (110). In addition, when a finger moves in a certain direction on the touch sensor while maintaining a touched state, the touch sensor may generate a touch movement event and transmit the event to the processor (110). The touch movement event may be divided into a flick event having a movement speed greater than a preset threshold and a drag event having a movement speed less than the threshold. In addition, after a touch event or touch movement event is generated, if the user's finger moves away from the touch sensor, the touch sensor can generate a touch release event and transmit it to the processor (110). Such a touch sensor can be formed by a pressure-sensitive method, an infrared method, a capacitive method, etc. Hereinafter, for the convenience of explanation, the display unit (130) will be described in parallel with the touch screen. At this time, the display unit (130) may have a configuration corresponding to the display module (1160) of FIG. 11.

[0032] The memory (140) stores an operating system, application programs, and storage data (compressed image files, videos, etc.) for controlling the overall operation of the electronic device (100). Furthermore, the memory (140) may store preset data, artificial intelligence images, and metadata of artificial intelligence images according to various embodiments of the present disclosure. In this case, the memory (140) may have a configuration corresponding to the memory (1130) of FIG. 11 .

[0033] The image sensor (150) can capture still images and moving images. Furthermore, to capture an image, it can provide a preview image corresponding to the image to be captured to the processor (110). At this time, the image sensor (150) may be a camera or a component included in a camera. At this time, the image sensor (150) may be a component corresponding to the camera module (1180) of FIG. 11.

[0034] The processor (110) may be configured to include a drawing control unit (111), a layer control unit (112), an object analysis unit (112), a prompt generation unit (114), an image generation unit (115), a depth analysis unit (116), an image synthesis unit (117), and an image storage unit (118).

[0035] At this time, the drawing control unit (111), layer control unit (112), object analysis unit (112), prompt generation unit (114), image generation unit (115), depth analysis unit (116), image synthesis unit (117), and image storage unit (118) may be stored in the form of instructions in the memory (1400) as a program (1140).

[0036] The drawing control unit (111) can generate user input in the form of a drawing. At this time, the drawing input by the user can be input into a layer, and the user can specify drawing elements such as color and thickness. In addition, the drawing control unit (111) can change or delete the drawing form input by the user, and can store the drawing form input by the user.

[0037] Additionally, the drawing control unit (111) can load a drawing form (template) saved in advance by the user.

[0038] The layer control unit (112) controls a layer for receiving input in the form of a drawing.

[0039] The layer control unit (112) can fix the drawing input by the user to the layer even if the shooting screen is moved. Accordingly, the layer control unit (112) can move the screen and shoot using a drawing drawn by the user in advance.

[0040] The object analysis unit (112) can analyze objects displayed on the captured screen to distinguish the type and area of ​​each object. Furthermore, the object analysis unit (112) can analyze objects designated by the user to distinguish their type and area. Furthermore, the object analysis unit (112) can distinguish the type and area of ​​surrounding objects based on the user's drawing data.

[0041] The prompt generation unit (114) can generate an input prompt for use in a generative artificial intelligence model by using the user's drawing input data and object data obtained through the object analysis unit (112). At this time, the prompt generation unit (114) utilizes an artificial intelligence model trained to convert text input data into a prompt together with the drawing input data, and the artificial intelligence model can utilize a deep learning model such as Diffusion or Transformer. At this time, the drawing input data corresponding to the user input is converted into an image-type input and used, and can be converted into at least one prompt together with the text input data. For example, if there is a drawing of a cherry blossom shape and text data such as "cherry blossoms in full bloom," these two pieces of data can be mixed and converted into a single prompt. The prompt generated at this time may be in the form of text or image.

[0042] The image generation unit (115) can generate an image using a generative artificial intelligence model as input using a prompt obtained through the prompt generation unit (114). At this time, the generative artificial intelligence model can utilize a deep learning model such as Diffusion or GAN.

[0043] The depth analysis unit (116) can estimate the depth of the area to be synthesized by analyzing the depth with surrounding objects based on object classification and area recognition data and user drawing data.

[0044] The image synthesis unit (117) can generate an artificial intelligence image by synthesizing the image generated by the image generation unit (115) and the depth data estimated by the depth analysis unit (116) with a preview.

[0045] The image synthesis unit (117) may request the creation of an artificial intelligence image from the artificial intelligence server (160) through the communication unit (120), and may also receive the artificial intelligence image from the artificial intelligence server (160). At this time, the received artificial intelligence image may have a higher resolution than the artificial intelligence image created through the image synthesis unit (117).

[0046] The image storage unit (118) stores the generated artificial intelligence image. Drawing input information (drawing shape, color, etc.), text data (data used as auxiliary input), analyzed object information, used prompts, etc. used in generating the artificial intelligence image can be stored together with the artificial intelligence image in the form of metadata. For example, when an image created by drawing a cherry blossom tree and a bird on it on an empty branch is saved, the metadata of the image can include the drawing of the shape of the drawn cherry blossom tree, the drawing shape of the bird, the prompt created by the drawing, and analyzed surrounding object information (empty branch, tree, etc.).

[0047] The processor (110) can control the overall operation of the electronic device (100). In addition, the processor (110) can perform the functions of a drawing control unit (111), a layer control unit (112), an object analysis unit (112), a prompt generation unit (114), an image generation unit (115), a depth analysis unit (116), an image synthesis unit (117), and an image storage unit (118). The image analysis unit (111), the keyword analysis unit (112), the prompt generation unit (113), the image generation unit (114), the area management unit (116), the image synthesis unit (117), and the image storage unit (118) are illustrated separately to explain each function separately. Accordingly, the processor (110) may include at least one processor configured to perform each function of the drawing control unit (111), the layer control unit (112), the object analysis unit (112), the prompt generation unit (114), the image generation unit (115), the depth analysis unit (116), the image synthesis unit (117), and the image storage unit (118). In addition, the processor (110) may include at least one processor configured to perform some of the functions of the drawing control unit (111), the layer control unit (112), the object analysis unit (112), the prompt generation unit (114), the image generation unit (115), the depth analysis unit (116), the image synthesis unit (117), and the image storage unit (118). At this time, the processor (110) may have a configuration corresponding to the processor (1180) of FIG. 11.

[0048]

[0049] Hereinafter, the method according to the present disclosure configured as above will be described with reference to the drawings below.

[0050] Figure 2 is a flowchart illustrating a flow for generating an artificial intelligence image according to one embodiment.

[0051] Referring to FIG. 2, in operation 200, the electronic device (100) can display a preview image of the image sensor (150) on the display unit (130).

[0052] In operation 210, the electronic device (100) can detect whether the artificial intelligence mode is selected.

[0053] If the artificial intelligence mode is selected as a result of the confirmation of operation 210, the electronic device (100) can confirm whether a user input for the preview image is received in operation 220.

[0054] If no user input is received on the preview image as a result of the confirmation of operation 220, the electronic device (100) can return to operation 210 and repeat a series of operations.

[0055] If a user input for a preview image is received as a result of the confirmation of operation 220, the electronic device (100) can confirm whether the user input is completed in operation 230. At this time, in operation 230, the electronic device (100) can determine that the user input is completed in at least one of the following cases: when no user input is detected for a preset period of time (for example, when there is no user input (for example, user touch input, pen input) for 5 seconds or more), when an input notifying that the user input is completed is detected, and when a request is made to create an artificial intelligence graphic object.

[0056] The 220 action is a user input. When a user's drawing input or text input is received, the drawing input or text input can be overlaid on the preview image and displayed. At this time, the 220 action is a user input. When the drawing mode is activated, a transparent layer is added where the drawing is input, and the input tool required for drawing can be activated. In other words, when the user touches the screen using a hand or a pen, the drawing can be input into the transparent layer considering the set color, thickness value, etc.

[0057] In addition, when operation 220 is requested to call a stored template (e.g., multiple graphic objects), it provides a list of stored templates (e.g., multiple graphic objects), and processes the selected template (graphic object) as a user input through another user input (e.g., an input for selecting one of the stored templates), and displays the selected template by overlaying it on a preview image. In this case, the other user input may represent an input for selecting one of the stored templates rather than the user's drawing input or the user's text input as another user input.

[0058] Additionally, the 220 action can display user input on a transparent layer over the preview image, rather than fixing the preview image, depending on the user's choice.

[0059] Additionally, the 220 operation can fix a preview image according to the user's selection and display user input on the preview image. That is, the 220 operation can capture a preview image, display a graphic object corresponding to the user input on the captured preview image, and display the user input.

[0060] Additionally, the 220 operation can display user input on the selected object included in the preview image when an object is selected based on another user input. In this case, the other user input can represent an input for selecting an object in the displayed preview image, rather than the user's drawing input or the user's text input.

[0061] That is, the user's drawing is fixed to a transparent layer, allowing the user to adjust the composition to their desired level even when moving the shooting screen. Alternatively, the user can temporarily capture a preview image using the "Freeze Screen" command and then draw on the captured preview image to adjust the composition to their desired level. Alternatively, the user can designate an object and draw on it, thereby fixing the drawing to the object.

[0062] And, in operation 240, the electronic device (100) can generate at least one artificial intelligence graphic object based on at least a part of the user input and display at least one artificial intelligence graphic object in the preview image.

[0063] In addition, in operation 250, the electronic device (100) can determine whether a modification input for at least one artificial intelligence graphic object is received. At this time, the modification input may be an input requesting a change to at least one artificial intelligence graphic object or an input for moving at least one artificial intelligence graphic object.

[0064] If a modification input of at least one artificial intelligence graphic object is received as a result of the confirmation of operation 250, the electronic device (100) can return to operation 230 and repeat a series of operations.

[0065] If, as a result of the confirmation of operation 250, a modification input of at least one artificial intelligence graphic object is not received, in operation 260, the electronic device (100) can confirm whether a preview image and a shooting input (input requesting generation of an artificial intelligence image) corresponding to at least one artificial intelligence graphic object are received.

[0066] If a photographing input is received as a result of the confirmation of operation 260, in operation 270, the electronic device (100) may generate an artificial intelligence image using an image captured using a camera based on the photographing input and at least one artificial intelligence graphic object using the user's input, and display the generated artificial intelligence image. At this time, the electronic device (100) may generate the artificial intelligence image on its own, or may transmit the image captured using a camera and at least one artificial intelligence graphic object using the user's input to an external artificial intelligence server (160), and receive and display the artificial intelligence image generated from the artificial intelligence server (160).

[0067] And, in operation 280, the electronic device (100) can store the generated artificial intelligence image together with metadata of the artificial intelligence image. At this time, the metadata of the artificial intelligence image can include at least one of an image captured using a camera, at least one artificial intelligence graphic object using a user's input, a user input used to generate the artificial intelligence image, object information included in the artificial intelligence image, and a prompt used to generate the artificial intelligence image.

[0068] Meanwhile, when the electronic device (100) receives a request to call a stored artificial intelligence image, it can load the artificial intelligence image and metadata of the artificial intelligence image, use the captured image and at least one artificial intelligence graphic object included in the metadata of the artificial intelligence image to overlay and display at least one artificial intelligence graphic object on a preview image, modify at least one artificial intelligence graphic object based on a user's modification input, and overlay and display at least one modified artificial intelligence graphic object on the preview image.

[0069]

[0070] FIG. 3 is a drawing illustrating an example of generating an artificial intelligence image through drawing according to one embodiment.

[0071] Referring to FIG. 3, the first drawing (310) shows an example of displaying a user's input (e.g., drawing a tree in a specific color, drawing a bird) by overlaying it on a preview input through an image sensor (150) of an electronic device (100).

[0072] The second figure (320) shows an example in which at least one artificial intelligence graphic object corresponding to the user's input is displayed by overlaying it on the corresponding object in the preview.

[0073] The third figure (330) shows an example of selecting at least one AI graphic object of a bird and changing it to at least one AI graphic object of a different type of bird.

[0074] The user can turn on the image sensor (150) of the electronic device (100) to capture a subject in the surroundings. At this time, the subject in the surroundings can be previewed through the display unit (130) of the electronic device (100). The user can add an AI graphic object through a menu selection and select an AI mode for generating an AI image, and can create a desired AI graphic object by drawing on the preview before capturing the image as in the first drawing (310), and can also input text. At this time, the input can be done, for example, by placing the electronic device (100) on a flat surface, drawing in advance, and then turning on the image sensor (150) to overlay and display the entered content on the preview image, or by turning on the image sensor (150) to compose and draw on the preview image.

[0075] For example, when a user wants to take a picture of a tree branch in winter, he can compose it as in the first picture (310) and draw a flower and a bird in the corresponding position of the displayed tree by overlaying it on the preview image.

[0076] When a user draws a picture and a certain amount of time (e.g., 5 seconds) has passed or the user issues a command to end the drawing input, the electronic device (100) may display at least one artificial intelligence graphic object corresponding to a part of the tree where flowers are in bloom by overlaying it on the subject, as shown in the second drawing (320), and may display an artificial intelligence graphic object of a bird at the location of the bird drawn by the user. At this time, the overlaid artificial intelligence graphic object may follow the object and be displayed by overlaying it even when the user moves the composition. In other words, the artificial intelligence graphic object may correspond to the object.

[0077] For example, a user can change the position or color of an AI graphic object through drag input, etc. For example, the electronic device (100) can change or delete an AI graphic object of a bird to another AI graphic object of a bird through an input of selecting the AI ​​graphic object of the bird if the user does not like the new image, as shown in the third figure (330).

[0078] The image confirmed by the user in the preview may be an image generated using the processor (110) within the electronic device (100). When the image is captured, each displayed AI graphic object can be generated as a single AI image. The AI ​​image may be generated using the processor (110) of the electronic device (100), but a high-quality AI image may also be generated using an external generation type AI server (160) and displayed on the electronic device (100).

[0079]

[0080] FIG. 4 is a drawing illustrating an example of generating an artificial intelligence image by selecting and changing an object added as a drawing from among samples according to one embodiment.

[0081] Referring to FIG. 4, the first drawing (410) shows an example of displaying a user input (e.g., drawing a tree in a specific color, drawing a bird) overlaid on a preview image input through an image sensor (150) of an electronic device (100).

[0082] The second picture (420) displays an artificial intelligence graphic object corresponding to a part of a tree in bloom by overlaying it on the subject, and shows an example of an artificial intelligence graphic object of a bird through a selection window (421).

[0083] The third picture (430) shows an example in which an artificial intelligence graphic object corresponding to a part of a tree in bloom is overlaid on the subject, and an artificial intelligence graphic object of a bird is overlaid on the preview image.

[0084] That is, the electronic device (100) can provide the user with candidate artificial intelligence graphic objects that can be generated after a bird drawing input, as shown in the second figure (420), through a user interface such as a selection window (421). Thereafter, when the electronic device (100) receives a command to take a picture, it can generate an artificial intelligence image with an appearance applied, as shown in the third figure (430).

[0085]

[0086] FIG. 5 is a drawing illustrating an example of generating an artificial intelligence image by changing an object added as a drawing by entering text according to one embodiment.

[0087] Referring to FIG. 5, the first drawing (510) shows an example of displaying a user's input (e.g., drawing a tree in a specific color, drawing a bird) by overlaying it on a preview image input through an image sensor (150) of an electronic device (100).

[0088] The second figure (520) shows an example of overlaying an artificial intelligence graphic object corresponding to a part of a tree where flowers are in bloom on the subject, and adding a specific shape of the bird as text before the artificial intelligence graphic object for the bird is created after drawing the bird.

[0089] The third picture (530) shows an example of an artificial intelligence graphic object corresponding to a part of a tree where flowers are in bloom being overlaid on the subject, and an artificial intelligence graphic object of a bird being overlaid on the subject.

[0090] That is, if the user determines that the information is insufficient with only the drawing input, as in the second figure (520), the user can input additional text information along with the drawing to specify the drawing object and generate an artificial intelligence image that specifically reflects the user's intention.

[0091] Accordingly, the electronic device (100) can provide the user with a user interface for entering text. When text is entered, the electronic device (100) can utilize the text data along with drawing information to generate a prompt. In this case, the user can directly input text through a key input user interface displayed on the electronic device (100), or can input text through voice using a technology such as Speech-to-Text.

[0092]

[0093] FIG. 6 is a diagram illustrating an example of generating an artificial intelligence image using a stored template according to one embodiment.

[0094] Referring to FIG. 6, the first drawing (610) shows an example of outputting presets (611, 612, 613, 614, 615, 616) stored in advance corresponding to graphic objects on a preview image input through an image sensor (150) of an electronic device (100).

[0095] The second figure (620) shows an example in which a preset (611) is selected from among the saved presets (611, 612, 613, 614, 615, 616) and displayed as an overlay on a preview input through the image sensor (150) of the electronic device (100).

[0096] The third figure (630) shows an example of overlaying the selected preset (611) on the preview and overlaying the bird drawn by the user on the preview.

[0097] The fourth figure (640) shows an example of displaying a preset (611) corresponding to a user's input and at least one artificial intelligence graphic object for a drawn bird by overlaying it on the corresponding object in the preview.

[0098] That is, the electronic device (100) can store various presets in the memory (140). At this time, the presets may be provided by the manufacturer of the electronic device (100), supplied by a separate business operator that supplies presets, or may be created and stored in advance by the user of the electronic device (100).

[0099] The user can additionally draw or modify a picture within the preset. For example, a new shape can be additionally drawn within the preset (611) corresponding to the loaded cherry blossom picture, as shown in the third picture (630). When the user completes the drawing and issues a command to take a picture, the electronic device (100) generates a prompt based on the preset and the drawing, uses the prompt to create at least one AI graphic object, and uses the analyzed depth information to create an AI image, as shown in the fourth picture (640).

[0100]

[0101] FIG. 7 is a drawing illustrating an example of generating an artificial intelligence image by specifying an object to be drawn according to one embodiment.

[0102] Referring to FIG. 7, the first drawing (710) shows an example of specifying a tree (711), which is an object to which drawing is to be applied, in a preview input through an image sensor (150) of an electronic device (100).

[0103] The second figure (720) shows an example of overlaying fruits and leaves drawn on top of an object in a preview image. In this case, the fruits and leaves drawn can be positioned in correspondence with the selected object, which is a tree.

[0104] The third picture (730) is an example showing that even if the composition of the preview image changes, the drawing of the fruit and leaves is positioned in response to the object because it was drawn by specifying the object.

[0105] The fourth figure (740) shows an example where the drawing of the fruit and leaves is positioned relative to the designated objects even though the composition of the preview image is restored to its original state.

[0106] The fifth figure (750) shows an example of displaying artificial intelligence graphic objects corresponding to drawings of fruits and leaves by overlaying them on the corresponding objects in the preview image.

[0107] In FIG. 7, in a situation where a user is photographing a person with a tree that has not yet borne fruit, if the user composes the image and issues a command to designate a tree (711) displayed in the preview image, as shown in the first figure (710), the object can be indicated as having been designated (e.g., a thick outline indicating that it has been designated is displayed). The user can draw fruit and leaves on the designated tree object, as shown in the second figure (720). At this time, the overlaid drawing corresponding to the designated object can be displayed by following the designated object even when the user moves the composition, as shown in the third figure (730). Based on the drawing information drawn by the user, a related prompt can be automatically generated using information on the designated object. In addition, depth information based on the designated object can be analyzed and utilized to synthesize the image at an appropriate depth. When the user completes the drawing and issues a command to take a picture, an AI image is generated using the designated object information and the drawing information input by the user, and through synthesis, an AI image can be generated, as shown in the fifth figure (750). In addition, if the user incorrectly designates an object, the object designation can be canceled. When an object is de-assigned, the drawing can be deleted or kept, and the drawing can be displayed without following the de-assigned object.

[0108]

[0109] FIG. 8 is a diagram illustrating an example of retrieving a stored artificial intelligence image and modifying the artificial intelligence image according to one embodiment.

[0110] Referring to FIG. 8, the first drawing (810) is an example of loading an artificial intelligence image stored in the memory (140) of an electronic device (100), and is an artificial intelligence image with a part of a tree in bloom and a bird added.

[0111] The second figure (820) shows an example in which, when loading an artificial intelligence image stored in the memory (140) of an electronic device (100), the user's drawing information (821, 822) is displayed by overlaying it on top of the artificial intelligence image from the corresponding metadata stored together with the artificial intelligence image.

[0112] The third figure (830) shows an example of changing drawing information (821, 822) by selecting drawing information (831, 832). For example, a user can change the shape of a bird by selecting drawing information (831), and change the color of a leaf by changing the color of drawing information (832).

[0113] The fourth figure (840) shows an example of an artificial intelligence graphic object that has changed according to changes in drawing information (821, 822).

[0114] That is, when a user issues a command to load an AI image generated and saved by a method according to various embodiments of the present disclosure, as shown in the first drawing (810), the metadata stored together with the AI ​​image can be loaded. At this time, since the metadata stores drawing data, text data, analyzed object information, generated prompts, etc. input by the user, the drawing drawn by the user and the specified object can be overlaid and displayed on the AI ​​image generated using the metadata, as shown in the second drawing (820) above. If the user wants to edit the generated image for the loaded image, the user can edit each generated image and drawing. For example, if the user wants to edit the image of a bird generated by drawing, the user can change the generated bird to a different new image, as shown in the third drawing (830), through user input, or delete it through a delete command, as shown in the second drawing (820) above. In addition, if the user wants to change the drawing form of the generated image to a new AI image, the user can change the color of the input drawing, as shown in the third drawing (830). Additionally, you can change the shape of your drawing or delete an existing drawing and input a new one.

[0115] When the user issues a command to complete the modification, a modified AI image, as shown in the fourth figure (840), can be generated using the new information. When the user issues a command to complete the modification, the modified image and metadata are saved.

[0116]

[0117] FIG. 9 is a drawing illustrating an example of fixing a preview and generating an artificial intelligence image through drawing according to one embodiment.

[0118] Referring to FIG. 9, the first drawing (910) shows an example of fixing a preview image using a button (e.g., a pause button (911)) before applying drawing in a preview input through an image sensor (150) of an electronic device (100).

[0119] The second picture (920) shows an example of capturing a preview image using the pause button (911) and drawing tree fruits and leaves on the captured preview image.

[0120] The third picture (930) shows an example in which, when the pause button (911) is released to release the fixation of the preview image, the image drawn in the fixed state is fixed to the object and remains fixed to the object even if the composition of the preview image changes.

[0121] That is, the user can generate an AI image by temporarily pausing the screen shown in the preview image (so that the screen does not move and is fixed when the terminal is moved) and inputting a drawing. For example, in a situation where the user is taking a picture of a person with a tree that has not yet borne fruit, if the user composes the picture and inputs a pause button (911) to fix the preview screen, as in the first picture (910), the preview screen can be fixed, as in the second picture (920), and the preview screen can be indicated as being fixed. Using this, in a situation where the user has difficulty drawing due to shaking, the user can temporarily fix the preview screen, finish drawing, and then input a command to release the fixation of the shooting preview, so that the user can return to the moving preview screen, as in the third picture (630). At this time, if an object is designated and drawn as in the previous embodiment, the drawing moves along the object, as in the fourth picture above, and if an object is not designated and drawn, the input drawing can move along the preview screen.

[0122]

[0123] FIG. 10 is a schematic diagram illustrating a configuration of an electronic device that generates an artificial intelligence image according to one embodiment.

[0124] Referring to FIG. 10, an electronic device (1000) may be configured to include a processor (1010) and a memory (1020).

[0125] The memory (1020) stores an operating system, application programs, and storage data for controlling the overall operation of the electronic device (1000). In addition, the memory (1020) may store preset data, artificial intelligence images, and metadata of artificial intelligence images according to various embodiments of the present disclosure.

[0126] The processor (1010) may correspond to the processor (110) of the electronic device (100) of FIG. 1. That is, the processor (1210) may include the configuration of the processor (110) of FIG. 1.

[0127] The processor (1010) can display a preview generated through a camera on a display unit, and when receiving a user input for the preview image, can generate at least one artificial intelligence graphic object based on the user input, and overlay the at least one artificial intelligence graphic object on the preview image and display it on the display unit.

[0128] When the processor (1010) receives a shooting input corresponding to a preview image and at least one artificial intelligence graphic object, the processor (1010) can generate an artificial intelligence image using the preview image and at least one artificial intelligence graphic object, and store the artificial intelligence image together with metadata of the artificial intelligence image in the memory (1020).

[0129] At this time, the metadata of the artificial intelligence image may include at least one of a preview image, at least one artificial intelligence graphic object, user input used to generate the artificial intelligence image, object information included in the artificial intelligence image, and a prompt used to generate the artificial intelligence image.

[0130] The processor (1010) can display a graphic object corresponding to user input. At this time, the user input can include drawing input and text input.

[0131] When the processor (1010) receives a selection of a template (a plurality of graphic objects) stored in the memory (1020) as another user input, the processor (1010) can display the selected template (graphic object) on a preview image. More specifically, the processor (1010) can display a plurality of graphic objects for generating at least one artificial intelligence graphic object, and when receiving another user input for selecting a graphic object from among the plurality of graphic objects, the processor (1010) can display the selected graphic object on a preview image.

[0132] When receiving a user input, the processor (1010) can fix a preview image and display the user input on the preview image on the display unit. More specifically, the processor (1010) can capture a preview image and display a graphic object corresponding to the user input on the captured preview image.

[0133] When receiving a user input, the processor (1010) can display the user input on a transparent layer existing over the preview image on the display unit without fixing the preview image.

[0134] When receiving a user input, the processor (1010) may receive another user input for selecting an object in a preview image and display the user input for the object selected in the preview image.

[0135] The processor (1010) may determine that the user input is complete in at least one of the following cases: when no user input is detected for a preset period of time, when an input indicating that the user input is complete is detected, and when a request is made to create at least one artificial intelligence graphic object.

[0136] When the processor (1010) displays at least one artificial intelligence graphic object on the display unit by overlaying it on the preview image, the processor can change or move at least one artificial intelligence graphic object based on other user input.

[0137] When generating an artificial intelligence image, the processor (1010) can transmit a preview image and at least one artificial intelligence graphic object to an external artificial intelligence server, and receive the artificial intelligence image from the external artificial intelligence server and display it on a display unit.

[0138] When a request is made to call an artificial intelligence image stored in a memory, the processor (1010) may load an artificial intelligence image and metadata of the artificial intelligence image from the memory, use a preview image and at least one artificial intelligence graphic object included in the metadata of the artificial intelligence image to overlay at least one artificial intelligence graphic object on the preview image and display it on the display unit, modify at least one artificial intelligence graphic object based on a user's modification input, and overlay at least one modified artificial intelligence graphic object on the preview image and display it on the display unit.

[0139]

[0140] FIG. 11 is a block diagram of an electronic device within a network environment according to one embodiment.

[0141] Referring to FIG. 11, in a network environment (1100), an electronic device (1101) may communicate with an electronic device (1102) via a first network (1198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (1104) or a server (1108) via a second network (1199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (1101) may communicate with the electronic device (1104) via the server (1108). According to one embodiment, the electronic device (1101) may include a processor (1120), a memory (1130), an input module (1150), an audio output module (1155), a display module (1160), an audio module (1170), a sensor module (1176), an interface (1177), a connection terminal (1178), a haptic module (1179), a camera module (1180), a power management module (1188), a battery (1189), a communication module (1190), a subscriber identification module (1196), or an antenna module (1197). In one embodiment, the electronic device (1101) may omit at least one of these components (e.g., the connection terminal (1178)), or may have one or more other components added. In one embodiment, some of these components (e.g., sensor module (1176), camera module (1180), or antenna module (1197)) may be integrated into one component (e.g., display module (1160)).

[0142] The processor (1120) may control at least one other component (e.g., hardware or software component) of the electronic device (1101) connected to the processor (1120) by executing, for example, software (e.g., program (1140)), and may perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (1120) may store commands or data received from other components (e.g., sensor module (1176) or communication module (1190)) in the volatile memory (1132), process the commands or data stored in the volatile memory (1132), and store result data in the non-volatile memory (1134).

[0143] Meanwhile, the processor (1120) can perform the operation of the processor (110) of FIG. 1 or the operation of the processor (1010) of FIG. 10.

[0144] According to one embodiment, the processor (1120) may include a main processor (1121) (e.g., a central processing unit or an application processor) or an auxiliary processor (1123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (1121). For example, when the electronic device (1101) includes the main processor (1121) and the auxiliary processor (1123), the auxiliary processor (1123) may be configured to use less power than the main processor (1121) or to be specialized for a given function. The auxiliary processor (1123) may be implemented separately from the main processor (1121) or as a part thereof.

[0145] The auxiliary processor (1123) may control at least a part of functions or states associated with at least one component (e.g., the display module (1160), the sensor module (1176), or the communication module (1190)) of the electronic device (1101), for example, on behalf of the main processor (1121) while the main processor (1121) is in an inactive (e.g., sleep) state, or together with the main processor (1121) while the main processor (1121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (1123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (1180) or a communication module (1190)). In one embodiment, the auxiliary processor (1123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0146] The memory (1130) can store various data used by at least one component (e.g., the processor (1120) or the sensor module (1176)) of the electronic device (1101). The data can include, for example, software (e.g., the program (1140)) and input data or output data for commands related thereto. The memory (1130) can include a volatile memory (1132) or a non-volatile memory (1134).

[0147] Meanwhile, the memory (1130) can perform the role of the memory (140) of FIG. 1 or the memory (1020) of FIG. 10.

[0148] The program (1140) may be stored as software in memory (1130) and may include, for example, an operating system (1142), middleware (1144), or an application (1146).

[0149] The input module (1150) can receive commands or data to be used in a component of the electronic device (1101) (e.g., a processor (1120)) from an external source (e.g., a user) of the electronic device (1101). The input module (1150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0150] The audio output module (1155) can output audio signals to the outside of the electronic device (1101). The audio output module (1155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0151] The display module (1160) can visually provide information to an external party (e.g., a user) of the electronic device (1101). The display module (1160) may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module (1160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch. In this case, the display module (1160) may perform the role of the display unit (130) of FIG. 1.

[0152] The audio module (1170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (1170) can acquire sound through the input module (1150), output sound through the sound output module (1155), or an external electronic device (e.g., electronic device (1102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (1101).

[0153] The sensor module (1176) can detect the operating status (e.g., power or temperature) of the electronic device (1101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (1176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0154] The interface (1177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (1101) with an external electronic device (e.g., the electronic device (1102)). In one embodiment, the interface (1177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0155] The connection terminal (1178) may include a connector through which the electronic device (1101) may be physically connected to an external electronic device (e.g., the electronic device (1102)). According to one embodiment, the connection terminal (1178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0156] The haptic module (1179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (1179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0157] The camera module (1180) can capture still images and videos. According to one embodiment, the camera module (1180) may include one or more lenses, image sensors, image signal processors, or flashes. In this case, the camera module (1180) may perform the role of the image sensor (150) of FIG. 1.

[0158] The power management module (1188) can manage power supplied to the electronic device (1101). According to one embodiment, the power management module (1188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0159] A battery (1189) may power at least one component of the electronic device (1101). In one embodiment, the battery (1189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0160] The communication module (1190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (1101) and an external electronic device (e.g., electronic device (1102), electronic device (1104), or server (1108)), and the performance of communication through the established communication channel. The communication module (1190) may operate independently from the processor (1120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1190) may include a wireless communication module (1192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (1194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, a corresponding communication module can communicate with an external electronic device (1104) via a first network (1198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1192) can verify or authenticate the electronic device (1101) within a communication network such as the first network (1198) or the second network (1199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (1196). The communication module (1190) can perform the role of the communication unit (120) of FIG. 1.

[0161] The wireless communication module (1192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (1192) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1192) may support various requirements specified in the electronic device (1101), an external electronic device (e.g., the electronic device (1104)), or a network system (e.g., the second network (1199)). According to one embodiment, the wireless communication module (1192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0162] The antenna module (1197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module (1197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (1197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (1198) or the second network (1199), may be selected from the plurality of antennas, for example, by the communication module (1190). A signal or power may be transmitted or received between the communication module (1190) and an external electronic device via the selected at least one antenna. According to one embodiment, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (1197).

[0163] In one embodiment, the antenna module (1197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0164] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0165] According to one embodiment, commands or data may be transmitted or received between the electronic device (1101) and an external electronic device (1104) via a server (1108) connected to a second network (1199). Each of the external electronic devices (1102 or 1104) may be the same or a different type of device as the electronic device (1101). According to one embodiment, all or part of the operations executed in the electronic device (1101) may be executed in one or more of the external electronic devices (1102, 1104, or 1108). For example, when the electronic device (1101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1101). The electronic device (1101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (1101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (1104) may include an Internet of Things (IoT) device. The server (1108) may be an intelligent server utilizing machine learning and / or a neural network.According to one embodiment, an external electronic device (1104) or server (1108) may be included in the second network (1199). The electronic device (1101) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0166] Electronic devices according to embodiments of the present disclosure may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of the present disclosure are not limited to the aforementioned devices.

[0167] The embodiments of the present disclosure and the terminology used herein are not intended to limit the technical features described in the present disclosure to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In the present disclosure, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among the phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0168] The term "module" used in one embodiment of the present disclosure may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0169] An embodiment of the present disclosure may be implemented as software (e.g., a program (1140)) including one or more instructions stored in a storage medium (e.g., an internal memory (1136) or an external memory (1138)) readable by a machine (e.g., an electronic device (1101)). For example, a processor (e.g., a processor (1120)) of the machine (e.g., an electronic device (1101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0170] According to one embodiment, a method according to one embodiment of the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0171] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0172]

[0173] According to one embodiment, an electronic device (100; 1000; 1100) includes an image sensor (150; 1180); a display unit (130; 1160); and a processor (110; 1010; 1120), wherein the processor (110; 1010; 1120) displays a preview image obtained from the image sensor (150; 1180) through the display unit (130; 1160), receives a user input for the preview image through the display unit (130; 1160), generates at least one artificial intelligence graphic object based on at least a part of the user input, displays the at least one artificial intelligence graphic object on the preview image, and generates an artificial intelligence image based on at least a part of a captured image and the at least one artificial intelligence graphic object in response to another user input.

[0174] According to one embodiment, the processor (110; 1010; 1120) stores the artificial intelligence image together with metadata of the artificial intelligence image, and the metadata of the artificial intelligence image may include at least one of the captured image, the at least one artificial intelligence graphic object, the user input used to generate the at least one artificial intelligence graphic object, object information included in the artificial intelligence image, and a prompt used to generate the artificial intelligence image.

[0175] According to one embodiment, when the processor (110; 1010; 1120) receives the user input for the preview image, it displays a graphic object corresponding to the user input, and the user input may include a drawing input or a text input.

[0176] According to one embodiment, the processor (110; 1010; 1120) may, upon receiving the user input, display a plurality of graphic objects for generating the at least one artificial intelligence graphic object, and receive another user input for selecting a graphic object from among the plurality of graphic objects.

[0177] According to one embodiment, the processor (110; 1010; 1120) may capture the preview image when receiving the user input and display a graphic object corresponding to the user input on the captured preview image.

[0178] According to one embodiment, the processor (110; 1010; 1120), when receiving the user input, may receive another user input for selecting an object in the preview image and display the user input for the object in the preview image.

[0179] According to one embodiment, the processor (110; 1010; 1120) may determine that the user input is complete in at least one of the following cases: when the user input is not detected for a preset period of time when receiving the user input; when an input indicating that the user input is complete is detected; and when generation of at least one artificial intelligence graphic object is requested.

[0180] According to one embodiment, the processor (110; 1010; 1120) may change or move the at least one artificial intelligence graphic object based on another user input when displaying the at least one artificial intelligence graphic object in the preview image.

[0181] According to one embodiment, the processor (110; 1010; 1120) may, when generating the artificial intelligence image, transmit the captured image and at least one artificial intelligence graphic object to an external artificial intelligence server, receive the artificial intelligence image from the external artificial intelligence server, and display the artificial intelligence image.

[0182] According to one embodiment, a method for generating an artificial intelligence image may include: displaying a preview image of a camera (150; 1180) on a display unit (130; 1160); receiving a user input for the preview image; generating at least one artificial intelligence graphic object based on at least a portion of the user input; and displaying the at least one artificial intelligence graphic object on the preview image.

[0183] According to one embodiment, the artificial intelligence image generation method may further include, when receiving a photographing input for the preview image, generating an artificial intelligence image using an image captured based on the photographing input and the at least one artificial intelligence graphic object.

[0184] According to one embodiment, the method for generating an artificial intelligence image further includes an operation of storing the generated artificial intelligence image together with metadata of the artificial intelligence image, wherein the metadata of the artificial intelligence image may include at least one of the captured image, the at least one artificial intelligence graphic object, the user input used to generate the at least one artificial intelligence graphic object, object information included in the artificial intelligence image, and a prompt used to generate the artificial intelligence image.

[0185] In one embodiment, the act of receiving the user input for the preview image includes the act of displaying a graphic object corresponding to the user input, wherein the user input may include a drawing input or a text input.

[0186] According to one embodiment, the act of receiving the user input for the preview image may include the act of displaying a plurality of graphic objects for generating the at least one artificial intelligence graphic object; and the act of receiving another user input for selecting a graphic object from among the plurality of graphic objects.

[0187] According to one embodiment, the act of receiving the user input for the preview image may include the act of capturing the preview image and displaying a graphic object corresponding to the user input on the captured preview image.

[0188] In one embodiment, the act of receiving the user input for the preview image may include receiving another user input for selecting an object in the preview image, and displaying the user input for the object in the preview image.

[0189] According to one embodiment, the operation of receiving the user input for the preview image may include an operation of determining that the user input is complete in at least one of the following cases: when no user input is detected for a preset period of time; when an input indicating that the user input is complete is detected; and when a request is made to create at least one artificial intelligence graphic object.

[0190] In one embodiment, the act of displaying the at least one artificial intelligence graphic object in the preview image may include an act of changing or moving the at least one artificial intelligence graphic object based on another user input.

[0191] According to one embodiment, the operation of generating the artificial intelligence image using the photographed image and the at least one artificial intelligence graphic object may include the operation of transmitting the photographed image and the at least one artificial intelligence graphic object to an external artificial intelligence server; the operation of receiving the artificial intelligence image from the external artificial intelligence server; and the operation of displaying the artificial intelligence image.

[0192]

[0193] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program commands, data files, data structures, etc., singly or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

[0194] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to perform a desired operation or, independently or collectively, command the processing device. The software and / or data may be stored on any type of machine, component, physical device, virtual equipment, computer storage medium, or device, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems, and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0195] Although the embodiments described above have been described with limited drawings, those skilled in the art will appreciate that various technical modifications and variations can be applied based on the above. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0196] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In electronic devices, image sensor; Display section; and Contains a processor, The above processor, A preview image obtained from the image sensor is displayed through the display unit, Receiving user input for the preview image through the above display unit, Generating at least one artificial intelligence graphic object based on at least a portion of said user input; Displaying at least one artificial intelligence graphic object in the above preview image, Generating an AI image based at least in part on an image captured in response to another user input and at least one AI graphic object. Electronic devices.

2. In paragraph 1, The above processor, Store the above artificial intelligence image together with metadata of the above artificial intelligence image, The metadata of the above artificial intelligence image is: The captured image, the at least one AI graphic object, the user input used to generate the at least one AI graphic object, object information included in the AI image, and the prompt used to generate the AI image. An electronic device comprising at least one of:

3. In any one of paragraphs 1 and 2, The above processor, When receiving the user input for the above preview image, display a graphic object corresponding to the user input, The above user input is, Including drawing input or text input Electronic devices.

4. In any one of paragraphs 1 to 3, The above processor, When receiving the above user input, displaying a plurality of graphic objects for generating the at least one artificial intelligence graphic object, and receiving another user input for selecting a graphic object among the plurality of graphic objects. Electronic devices.

5. In any one of paragraphs 1 to 4, The above processor, When receiving the above user input, capturing the preview image and displaying a graphic object corresponding to the user input on the captured preview image. Electronic devices.

6. In any one of paragraphs 1 to 5, The above processor, When receiving the above user input, receiving another user input for selecting an object in the preview image, and displaying the user input for the object in the preview image. Electronic devices.

7. In any one of paragraphs 1 to 6, The above processor, When receiving the above user input, If no user input is detected for a preset period of time, When an input is detected indicating that the above user input has been completed, When requested to create at least one artificial intelligence graphic object In at least one case, the user input is judged to be complete. Electronic devices.

8. In any one of paragraphs 1 to 7, The above processor, When displaying at least one artificial intelligence graphic object in the above preview image, changing or moving the at least one artificial intelligence graphic object based on another user input. Electronic devices.

9. In any one of paragraphs 1 to 8, The above processor, When generating the above artificial intelligence image, the captured image and at least one artificial intelligence graphic object are transmitted to an external artificial intelligence server, the artificial intelligence image is received from the external artificial intelligence server, and the artificial intelligence image is displayed. Electronic devices.

10. An action to display a preview image of the camera on the display section; An action for receiving user input for the above preview image; An operation of generating at least one artificial intelligence graphical object based on at least a portion of said user input; and An action of displaying at least one artificial intelligence graphic object in the above preview image. An artificial intelligence image generation method comprising:

11. In paragraph 10, When receiving a shooting input for the above preview image, an operation of generating an artificial intelligence image using an image shot based on the shooting input and the at least one artificial intelligence graphic object An artificial intelligence image generation method further comprising:

12. In any one of paragraphs 10 to 11, An action of saving the generated artificial intelligence image together with metadata of the artificial intelligence image. Including more, The metadata of the above artificial intelligence image is: The captured image, the at least one AI graphic object, the user input used to generate the at least one AI graphic object, object information included in the AI image, and the prompt used to generate the AI image. An artificial intelligence image generation method comprising at least one of:

13. In any one of paragraphs 10 to 12, The action of receiving the user input for the above preview image is: An action to display a graphic object corresponding to the above user input. Including, The above user input is, Including drawing input or text input How to generate artificial intelligence images.

14. In any one of paragraphs 10 to 13, The action of receiving the user input for the above preview image is: An operation of displaying a plurality of graphic objects for generating at least one artificial intelligence graphic object; and An action of receiving another user input for selecting a graphic object from among the plurality of graphic objects. An artificial intelligence image generation method comprising:

15. In any one of paragraphs 10 to 14, The action of receiving the user input for the above preview image is: An action of capturing the above preview image and displaying a graphic object corresponding to the user input on the captured preview image. An artificial intelligence image generation method comprising:

Images