Electronic device for changing object in image and control method thereof

Abstract

A display device is disclosed. The display device according to the present disclosure may comprise: a display; a memory for storing instructions; and at least one processor including processing circuitry, wherein, when the instructions are individually or collectively executed, the at least one processor: when a touch input for a drawing drawn on an image displayed on the display is received, displays, on the display, a UI element indicating selection of the drawing and a list of a plurality of animations; and when a touch input for an animation among the plurality of animations is received, applies the animation to the drawing selected by the UI element.

Description

Electronic device for changing objects within an image and method for controlling the same

[0001] The present disclosure relates to an electronic device for changing an object within an image and a method for controlling the same.

[0002] When users want to decorate their own images on smart electronic devices, there has been an increasing demand to apply animation effects to their drawings, going beyond simply drawing on the image. Additionally, users may need to modify the components within a photo, going beyond merely extracting objects from the image.

[0003] In the past, there was inconvenience in use because users had to directly change the components of a photo or apply animation effects to specific objects through a specialized program.

[0004] Consequently, the need arose for technology that automatically changes components or applies animation effects to objects within a photo through user input.

[0005] A display device according to the present disclosure comprises at least one processor including a display, a memory for storing instructions, and processing circuitry, wherein when the instructions are executed individually or collectively, the at least one processor receives a touch input for a drawing drawn on an image displayed on the display, displays a UI element indicating the selection of the drawing and a list of a plurality of animations on the display, and when a touch input for an animation among the plurality of animations is received, the animation can be applied to the drawing selected by the UI element.

[0006] In addition, at least one processor can obtain motion information applied to the skeleton of the drawing based on an animation filter corresponding to the animation, and apply the motion information to the skeleton of the drawing to apply the animation to the drawing.

[0007] In addition, at least one processor identifies a drawing area containing the drawing in the image based on the skeleton of the drawing, and when a touch input for the drawing is received, the UI element can be displayed in the drawing area.

[0008] Additionally, at least one processor may display a UI element on the display for selecting another drawing area related to the drawing area when a touch input for the drawing is received, and may display a plurality of UI elements on the display indicating the selection of a plurality of drawings drawn on the image when a touch input for the UI element is received.

[0009] In addition, at least one processor can apply the animation to a plurality of drawings selected by the plurality of UI elements when a touch input for the animation is received among the plurality of animations.

[0010] Additionally, when at least one processor receives a touch input to deselect some drawings among the plurality of drawings, it stops displaying some UI elements corresponding to some drawings among the plurality of UI elements, and when a touch input for an animation among the plurality of animations is received, it can apply the animation to the drawing selected by the remaining UI elements.

[0011] Additionally, a method for controlling a display device may include the steps of: displaying a list of a UI element indicating the selection of a drawing and a plurality of animations when a touch input for a drawing drawn on an image is received; and applying the animation to the drawing selected by the UI element when a touch input for an animation among the plurality of animations is received.

[0012] In addition, in a non-transient computer-readable recording medium that stores one or more instructions executed by a processor of an electronic device to perform an operation of a display device, the operation may include the step of displaying a UI element indicating selection of a drawing and a list of a plurality of animations when a touch input for a drawing drawn on an image is received, and the step of applying the animation to the drawing selected by the UI element when a touch input for an animation among the plurality of animations is received.

[0013] FIG. 1 is a drawing for explaining the operation of an electronic device according to an embodiment of the present disclosure.

[0014] FIG. 2 is a drawing for explaining the configuration of an electronic device according to an embodiment of the present disclosure.

[0015] FIG. 3 is a flowchart for explaining the operation of an electronic device according to an embodiment of the present disclosure.

[0016] FIG. 4 is a drawing for showing a user drawing on a photograph according to an embodiment of the present disclosure.

[0017] FIG. 5 is a flowchart for explaining the operation of an electronic device according to an embodiment of the present disclosure.

[0018] FIG. 6 is a drawing for explaining a drawing area according to an embodiment of the present disclosure.

[0019] FIG. 7 is a drawing for explaining a method of displaying a menu UI according to an embodiment of the present disclosure.

[0020] FIG. 8 is a drawing showing a menu UI having a function to apply animation to a drawing according to an embodiment of the present disclosure.

[0021] FIG. 9 is a drawing for explaining the function of selecting all picture areas according to an embodiment of the present disclosure.

[0022] FIGS. 10a and 10b are drawings for explaining a method of applying an animation effect according to an embodiment of the present disclosure.

[0023] FIG. 11 is a drawing illustrating a method for an electronic device according to an embodiment of the present disclosure to change a component in an image based on a user command.

[0024] FIG. 12 is a flowchart illustrating a method for an electronic device according to an embodiment of the present disclosure to change a component in an image based on a user command.

[0025] FIGS. 13 to 17 are drawings for explaining a method in which an electronic device changes components of an image according to a user command according to an embodiment of the present disclosure.

[0026] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments.

[0027] In relation to the description of the drawings, similar reference numerals may be used for similar or related components.

[0028] The singular form of the noun corresponding to an item may include one or plural items, unless the relevant context clearly indicates otherwise.

[0029] In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0030] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another component and do not limit the components in other aspects (e.g., importance or order).

[0031] Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that the component may be connected to the other component directly (e.g., via a wire), wirelessly, or through a third component.

[0032] Terms such as “include” or “have” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this document, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0033] When it is said that one component is “connected,” “combined,” “supported,” or “in contact” with another component, this includes not only cases where the components are directly connected, combined, supported, or in contact, but also cases where they are indirectly connected, combined, supported, or in contact through a third component.

[0034] When it is said that a component is located “on” another component, this includes not only cases where one component is in contact with the other, but also cases where another component exists between the two components.

[0035] The term “and / or” includes a combination of multiple related described components or any of the multiple related described components.

[0036] The operating principle and embodiments of the present invention will be described below with reference to the attached drawings.

[0037]

[0038] An embodiment of the present disclosure will be described in more detail below with reference to the attached drawings.

[0039] FIG. 1 is a drawing for explaining the operation of an electronic device according to an embodiment of the present disclosure.

[0040] The display device (100) according to the present disclosure may be a smartphone, tablet PC, laptop PC, smart TV, etc.

[0041] The display device (100) can display an image on the display (110). A drawing based on user input can be displayed on the image. The display device (100) can apply animation effects to the drawing displayed on the image. For example, the display device (100) can apply an animation effect selected based on user input to the drawing among a plurality of animation effects. Accordingly, the drawing drawn by the user can be moved based on the animation effect set by the user.

[0042] For example, referring to FIG. 1, the image may be a photograph of a child holding an object. The user (1) can freely draw pictures such as plants on the image.

[0043] The user (1) can command that a specific animation effect be applied to a drawing they have drawn. The display device (100) can apply an animation effect to the drawing based on user input so that the drawing moves according to the user's intention.

[0044] FIG. 2 is a drawing for explaining the configuration of a display device (100) according to an embodiment of the present disclosure.

[0045] Referring to FIG. 2, a display device (100) may include a display (110), at least one memory (120) (hereinafter referred to as memory (120)), and at least one processor (130) (hereinafter referred to as processor (130)).

[0046] The display (110) is controlled by the processor (130) to output visualized information to the user. The visualized information may include visual objects displayed on the display (110). For example, the visual objects may include screens, images, icons, GUI, UI elements, etc.

[0047] For example, the display (110) can be implemented as a flat panel display (FPD), a curved display, or a flexible display. For example, the display (110) can be implemented as various types of displays such as an LCD (Liquid Crystal Display), an AMOLED (Active Matrix Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, etc.

[0048] The display (110) may be implemented as a touch screen. For example, the display (110) may include a touch detection circuit configured to detect touch. The touch detection circuit may acquire user input that touches (or selects) a visual object displayed on the display (110). For example, the touch detection circuit may detect an input (e.g., touch input or hovering input) on a specific location by measuring a change in a signal (e.g., voltage, light intensity, resistance, or charge) for a specific location on the display (110), and provide information regarding the detected input to the processor (130).

[0049] The memory (120) may store at least one instruction regarding the display device (100). Additionally, the memory (120) may store an operating system (O / S) for driving the display device (100). Furthermore, the memory (120) may store various software programs or applications for operating the display device (100) according to various embodiments of the present disclosure. Additionally, the memory (120) may include semiconductor memory such as flash memory or magnetic storage media such as a hard disk.

[0050] Specifically, various software modules for operating a display device (100) according to various embodiments of the present disclosure may be stored in the memory (120), and at least one processor (130) may control the operation of the display device (100) by executing the various software modules stored in the memory (120). That is, the memory (120) is accessed by at least one processor (130), and reading / writing / modifying / deleting / updating of data by at least one processor (130) may be performed.

[0051] Meanwhile, in the present disclosure, the term memory (120) may be used to include memory (120), ROM, RAM, or a memory card (e.g., micro SD card, memory stick) mounted in a display device (100) within at least one processor (130).

[0052] In addition, various information necessary within the scope of achieving the purpose of the present disclosure may be stored in the memory (120), and the information stored in the memory (120) may be updated as it is received from an external device or input by a user.

[0053] The processor (130) can control the overall operation of the display device (100). For example, the processor (130) can cause other components of the electronic device (100) to perform various operations by executing instructions stored in memory (120). For example, the processor (130) can control the operation of the display device (100) by being electrically connected to the display (110) and memory (120). The processor (130) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. The processor (130) can control one or any combination of other components of the display device (100) and can perform operations related to communication or data processing. A processor (130) can execute one or more programs or instructions stored in memory (120). For example, at least one processor (130) can perform a method according to one embodiment of the present disclosure by executing one or more instructions stored in memory (120).

[0054] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor) and the third operation may be performed by a second processor (e.g., an artificial intelligence dedicated processor).

[0055] The processor (130) may be implemented as a single-core processor including one core, or as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When the processor (130) is implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one embodiment of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one embodiment of the present disclosure.

[0056] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one of the plurality of cores included in a multi-core processor, or may be performed by a plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in a multi-core processor, or the first operation and the second operation may be performed by a first core included in a multi-core processor and the third operation may be performed by a second core included in a multi-core processor.

[0057] In the embodiments of the present disclosure, a processor may mean a system-on-chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, GPU, APU, MIC, DSP, NPU, hardware accelerator, or machine learning accelerator, but the embodiments of the present disclosure are not limited thereto.

[0058] FIG. 3 is a flowchart illustrating the operation of an electronic device according to an embodiment of the present disclosure. A processor (130) may perform at least one of the operations of FIG. 3. When instructions stored in memory (120) are executed individually or collectively, the processor (130) may perform the operations of FIG. 3.

[0059] In operation 310, the processor (130) can display an image on the display (110). For example, the processor (130) can display an image on the display (110) when user input is received to select an image from among the images stored in the display device (100). The image may be an image taken directly by the user using the display device (100) or an image received from an external electronic device (e.g., the user's smartphone, server, etc.).

[0060] In operation 320, when the processor (130) receives user input for the display (110), it can display a drawing on the display (110) based on the user input.

[0061] For example, user input may include touch input to the display (110). Referring to 401 in FIG. 4, the user may touch the display (110) using a digital pen or finger to draw on the image. Referring to 402 in FIG. 4, the processor (130) may display the drawing (420) on the image by displaying the drawing on the display (110) along the trajectory of the touch input.

[0062] In one embodiment, the processor (130) can create a drawing layer on an image and display a drawing through the drawing layer. The drawing layer may be a virtual window that can be edited independently of the original image. When the processor (130) receives a touch input for the display (110), it can display a drawing (420) on the drawing layer according to the touch input. Additionally, when the processor (130) saves the drawing, it can store the drawing layer with the displayed drawing in memory (120).

[0063] FIG. 5 is a flowchart for explaining the operation of an electronic device according to an embodiment of the present disclosure. A processor (130) may perform at least one of the operations of FIG. 5. When instructions stored in memory (120) are executed individually or collectively, the processor (130) may perform the operations of FIG. 5.

[0064] In operation 510, the processor (130) can identify whether a touch input has been received for a drawing drawn on an image displayed on the display (110).

[0065] For example, the processor (130) can identify whether a touch input for a drawing has been received based on the location of the drawing drawn on the image and the location of the touch input.

[0066] In operation 510-N, the processor (130) may not perform subsequent operations if no touch input for the drawing is received. For example, if no touch input for the drawing is received, the processor (130) may not identify the skeleton of the drawing.

[0067] In operation 510-Y, 520, when the processor (130) receives a touch input for a drawing drawn on an image displayed on the display (110), it can identify the skeleton of the drawing.

[0068] A skeleton refers to the skeletal structure of a drawing. A skeleton can consist of bones and joints. A joint can be the point where one bone meets another.

[0069] For example, the processor (130) can detect the outline of the drawing, identify the center points of the outline, and identify the skeleton connecting the center points. The skeleton may include a torso skeleton and branch skeletons. The torso skeleton may be a part that forms the central axis of the skeleton. Additionally, the branch skeleton may be a skeleton branching out from the torso skeleton. In this case, the joint may be a point where the torso skeleton and the branch skeleton meet. However, the present disclosure is not limited to this example, and the skeleton may be detected in various ways.

[0070] In operation 530, the processor (130) can identify a drawing area based on the skeleton of the drawing.

[0071] A drawing area may refer to an area occupied by a drawing in an image. For example, a drawing area may be a rectangular area of ​​the smallest size that contains the drawing in the image. The processor (130) may identify a rectangle of the smallest area containing the framework of the drawing as the drawing area. If multiple drawings exist on the image, the processor (130) may identify a drawing area for each of the multiple drawings. Meanwhile, although the drawing area was described as being rectangular in the example above, the present disclosure is not limited to this example. For example, the drawing area may be created along the outline of the drawing and may include various shapes such as a circle or a polygon.

[0072] In operation 540, the processor (130) can display a list of UI elements and a plurality of animations representing the selection of a drawing on the display (110).

[0073] In one embodiment, the processor (130) may display UI elements indicating a selection of a drawing in the drawing area. As in the example described above, the drawing area may be a rectangular area containing the drawing. The processor (130) may display UI elements indicating a selection of a drawing by displaying straight-line graphic elements on the sides of the rectangular area.

[0074] For example, referring to 601 in FIG. 6, the processor (130) can receive user input for a drawing. In this disclosure, user input is described as touch input, but the user input method is not limited thereto and may include various input methods.

[0075] The processor (130) can control whether to display a UI element (60) indicating a selection of a drawing based on the duration of user input to the drawing.

[0076] Specifically, referring to 602 in FIG. 6, the processor (130) may display a UI element (60) indicating a drawing selection if the duration of the user (1)'s touch input is less than or equal to a preset time. Additionally, the processor (130) may stop displaying the UI element (60) indicating a drawing selection if it receives user input outside the area where the UI element (60) indicating a drawing selection is displayed while the UI element (60) indicating a drawing selection is displayed.

[0077] For example, assume that the preset time is 1 second. If the duration of the touch input for the drawing is 1 second or less, the processor (130) may display a UI (60) indicating the selection of the drawing on the display (110). Additionally, if the duration of the touch input for the drawing is 1 second or less while the UI element (60) indicating the selection of the drawing is displayed, the processor (130) may stop displaying the UI element (60) indicating the selection of the drawing.

[0078] Additionally, the UI element (60) indicating the selection of the drawing may include a delete button (X button) at the top right. When the processor (130) receives user input for the delete button, it may stop displaying the UI element (60) indicating the selection of the drawing on the display (110).

[0079] Referring to FIG. 7, the processor (130) can display a menu UI (70) including multiple functions on the display (110) if user input persists for longer than a preset time while a UI element (60) indicating a selection of drawing is displayed on the display (110).

[0080] Referring to FIG. 8, the menu UI (70) may include a button (71) for applying animation effects to a drawing, a button (72) for selecting all drawings within an image, and a button (73) for deleting a drawing.

[0081] Referring to 901 in FIG. 9, the processor (130) can receive user input for a button (73) that can select all drawings within the image. Referring to 902 in FIG. 9, the processor (130) can display all drawing areas within the image on the display (110).

[0082] Even when all drawing areas within an image are displayed on the display (110), the processor (130) may stop displaying the drawing area and drawing containing the delete button on the display (110) when the “delete” button (marked with an X) located at the top right of the drawing area or the delete button (74) of the menu window (70) is touched. Accordingly, the user may display only the drawing and drawing area desired by the user on the display (110).

[0083] In the example described above, the processor (130) is described as displaying a menu UI (70) containing multiple functions on the display (110) if user input persists for longer than a preset time while a UI element (60) indicating a selection of a drawing is displayed on the display (110), but this is not limited thereto. For example, the processor (130) may display a menu UI (70) containing multiple functions on the display (110) even if the UI element (60) indicating a selection of a drawing is not displayed, if the duration of user input for any part of the image exceeds a preset time.

[0084] In the menu UI (70) of FIG. 8, when user input for an animation button (71) is received, the processor (130) may display an animation list on the display (110). The animation list may include the name of at least one animation that can be applied to a drawing.

[0085] The processor (130) can display a list of animations corresponding to the type of drawing. Here, the drawing may include a drawing selected by a UI element.

[0086] For example, the processor (130) can identify the type of drawing. The type of drawing may include people and non-human objects. Non-human objects may include, for example, plants, objects, etc.

[0087] For example, the processor (130) can identify the shape of the drawing and identify the type of the drawing based on the identified shape.

[0088] For example, if the processor (130) analyzes the skeleton of the drawing and identifies that the skeleton consists of a head (e.g., a closed curve at the top of the skeleton) and a torso (e.g., a closed curve at the bottom of the skeleton), and that there are two protruding parts (e.g., arms, legs) on the left and right sides of the top and bottom of the torso, respectively, then the drawing can be identified as a person. And, if the processor (130) analyzes the skeleton of the drawing and identifies that the drawing is not a person, then the drawing can be identified as an object other than a person. In this case, the drawing may consist of a torso skeleton and a branch skeleton.

[0089] The processor (130) can display a list of animations corresponding to the type of identified drawing.

[0090] For example, the memory (120) may include information about an animation corresponding to a person and information about an animation corresponding to an object other than a person. The information about the animation may include the name of the animation that can be applied to the drawing and an animation filter corresponding to the animation. The animation filter may include movement information about how each joint should move when the animation is applied to the drawing. For example, the animation filter applied to a person may include movement information for each of the arms, legs, and torso. Additionally, the animation filter applied to an object other than a person may include movement information for each of the torso bones and branch bones.

[0091] For example, if the processor (130) identifies that the type of drawing is a person, it may display a list of animations including the names of animations that can be applied to the person based on information about animations corresponding to the person. If the processor (130) identifies that the type of drawing is an object that is not a person, it may display a list of animations including the names of animations that can be applied to the object based on information about animations corresponding to the object. As an example, the list of animations for an object that is not a person may include “Fluttering” and “Spiky.” “Fluttering” is an animation to make the drawing sway smoothly from side to side, and “Spiky” is an animation to make the drawing move up and down and left and right at a fast speed.

[0092] In operation 550, when the processor (130) receives a touch input for an animation among a plurality of animations, it can apply the animation to a drawing selected by a UI element.

[0093] When the processor (130) receives user input touching an animation in the animation list, it can apply an animation filter corresponding to the selected animation to the drawing.

[0094] For example, the processor (130) can apply animation to the drawing by applying motion information corresponding to an animation filter corresponding to the animation to the skeleton of the drawing. As described above, the motion information may include information on how each joint should move when applying animation effects to the drawing.

[0095] For example, an animation filter can store a standard skeleton. The standard skeleton may be a skeleton in a standard format for applying animation. The aforementioned motion information may be motion information generated based on the standard skeleton.

[0096] For example, the processor (130) can map the skeleton of the drawing to the standard skeleton through skeleton mapping. Accordingly, the joints of the skeleton of the drawing can be mapped to the joints of the standard skeleton. The processor (130) can adjust the standard skeleton based on the difference in length between the skeleton of the drawing and the standard skeleton. For example, if the length of the skeleton of the drawing is shorter than the standard skeleton, the processor (130) can reduce the length of the standard skeleton by the difference in length between the skeleton of the drawing and the standard skeleton. As another example, if the length of the skeleton of the drawing is longer than the standard skeleton, the processor (130) can increase the length of the standard skeleton by the difference in length between the skeleton of the drawing and the standard skeleton. And, the processor (130) can obtain movement information to apply to the skeleton of the drawing from the movement information of the standard skeleton. For example, the processor (130) can obtain movement information by using a transformation matrix to transform the coordinates of the standard skeleton and the skeleton of the drawing so that the joints of the skeleton of the drawing can move in the same way as the joints of the reference skeleton. Then, the processor (130) can apply the obtained movement information to the joints of the skeleton of the drawing to apply animation to the drawing. However, this is just an example, and the processor (130) may apply animation filters to the drawing using various methods.

[0097] FIGS. 10a and FIGS. 10b are drawings for explaining the application of animation to a figure according to an embodiment of the present disclosure.

[0098] Referring to FIG. 10a, the processor (130) can display the animation effect applied to the drawing (420) on the display (110).

[0099] For example, the selected animation effect may be to repeat the shaking of the body frame of the drawing (420) from side to side and the shaking of the branch frame up and down at preset intervals. In FIG. 10a, the body frame may correspond to the stem of the drawing (420), and the branch frame may correspond to the leaf portion of the drawing (420).

[0100] When the processor (130) receives user input for a selected animation effect, it can apply the animation effect to the stem and leaf portions of the drawing (420) as in FIG. 10a. Accordingly, the stem of the drawing (420) can shake left and right and the leaf portion can shake up and down. The processor (130) can display the result of applying the animation effect to the drawing (420) on the display (110).

[0101] Additionally, the user may customize the animation filter. The user may touch the animation custom button through a menu UI (70) that displays multiple functions. The processor (130) may display a UI that provides a customization function on the display (110). The customization function may be a function in which the user adds movement information to the movement information stored in the animation filter. Through the customization function, the user may set the speed of the animation, the direction of movement, the repetition period of the movement, etc.

[0102] The processor (130) can apply animation effects to the drawing based on the user's customized input values.

[0103] For example, referring to FIG. 10b, the processor (130) can repeat the movement of the drawing (420) every 5 seconds if the user inputs that the drawing (420) should be repeated every 5 seconds. Additionally, the processor (130) can make the drawing move up and down every 10 seconds if the user (1) inputs that the drawing (420) should be moved up and down every 10 seconds.

[0104] Additionally, the user (1) may directly adjust the skeleton of the drawing (420). For example, the user (1) may adjust the skeleton if they do not like the result of applying animation to the drawing or feel it looks unnatural. When the processor (130) receives input from the user (1) in the menu UI (70) that they wish to adjust the skeleton, it may display a skeleton drawing representing the skeleton of the drawing (420) on the display (110). At this time, the skeleton drawing may be displayed overlaid on the drawing (420). The processor (130) may readjust the skeleton of the drawing based on the user input received regarding the skeleton drawing.

[0105] Additionally, when the processor (130) receives user input for an animation among multiple animations, it can apply the animation to the drawing selected by the remaining UI elements. When multiple drawings are selected as in FIG. 9, input may be provided to apply the animation filter to all selected drawings. At this time, the processor (130) may also apply the selected animation filter to the multiple selected drawings in bulk.

[0106] The processor (130) can not only apply animation to the drawing, but also replace the image contained in the area selected by the user in the image with another image.

[0107] FIG. 11 is a drawing illustrating a method for an electronic device according to an embodiment of the present disclosure to change a component included in an image based on a user command.

[0108] Referring to FIG. 11, the processor (130) can change a part of the photo (20) through user input.

[0109] Specifically, the user (1) selects the image change function from the menu bar (10), and when the input window (30) for receiving user commands is displayed, the user can enter a command to change the components for one area (20) of the photo.

[0110] For example, as shown in Fig. 1, the user (1) can input a command to change the object held by the child in the photo into a magic wand.

[0111] The display device (100) can generate multiple images (40) that have been modified based on a user's command. The user (1) can select one of the multiple images to change an object in the photograph. The method by which the processor (130) changes an object in the photograph is explained based on FIG. 12.

[0112] FIG. 12 is a flowchart for explaining the operation of a display device (100) according to an embodiment of the present disclosure. When instructions are executed individually or collectively, the processor (140) can perform the operations of FIG. 9.

[0113] In operation 1210, the processor (130) may receive an input that commands a portion of a photograph to be selected and a component of an object included in the portion of the photograph to be changed.

[0114] Referring to FIG. 13, the processor (130) can display a user's photo on the display (110). Even in this case, the user can operate the display device (100) through a touchscreen. The processor (140) can display a menu bar (10) on the right side of the display (110) so that the user can select a menu.

[0115] The menu bar (10) can be displayed in various locations as well as in the right area of ​​the display (110). Additionally, the user may operate the display device (100) through another menu window that appears when the display (110) is touched for a preset amount of time.

[0116] The user can select the desired function from the menu bar (10). Specifically, multiple icons may be located on the menu bar (10). Each icon may display a function that the user can use on the photo. For example, the icons located on the menu bar (10) may include an area selection function, a drawing function, an erase function, a sharing function, a back function, a forward function, etc.

[0117] The user can select one of the multiple icons on the menu bar (10) via a touchscreen. In addition to selecting via a touchscreen, the user can select one of the multiple icons through various methods, such as voice recognition.

[0118] In addition, the processor (130) may display a separate UI that allows a user to select a function menu when the user touches any area of ​​the display (110) for a preset time or longer, in addition to the menu bar (10).

[0119] Referring to Fig. 14, the user can select the area selection function.

[0120] The processor (130) can generate a dotted line along the path the user touches when the user draws a picture by touching the display (110).

[0121] The user can select an area by drawing a closed curve over a specific area of ​​the photo.

[0122] The processor (130) can display the selected area (20) on the display (110) when the user selects an area.

[0123]

[0124] * Referring to FIG. 15, the processor (130) can receive an input that commands a component of an object included in the selection area (20) to be changed.

[0125] Specifically, the processor (130) can display a UI (30) on the display (110) that can receive commands from the user when the user selects an area. The user may input commands through voice recognition, or through a touch pen or a soft keyboard. For example, after selecting an area, the user may input a command to change an object into a magic wand. In addition, the user may input a command to change the background into a different background. The user may not only simply command the object to be changed, but also give detailed instructions regarding the picture to be created. For example, the user may input detailed commands into the display device (100), such as “Change the object into a magic wand” as well as “Change the object into a cartoon-style magic wand.”

[0126] In operation 1220, the processor (130) can analyze the object included in the selected area.

[0127] Specifically, when a user command is input, the processor (130) can analyze objects included in the selected area using an object analyzer. Analyzing objects may be a process of identifying the type, color, and size of objects in an image.

[0128] The object analyzer can be stored in memory (120). The object analyzer can be, for example, a Fast-R-CNN or YOLO model.

[0129] The processor (130) can input an image and a selected area of ​​the image into an object analyzer. Through this, the processor (130) can extract features such as edges, colors, and textures of objects included in the selected area.

[0130] The processor (130) can generate a feature map in which the main features of the image are condensed based on the acquired features. The feature map may be generated by condensing the existing image. Accordingly, each pixel of the feature map may contain feature information for multiple pixels included in the original image. Accordingly, the processor (130) can identify the features of a specific region of the original image through the pixels of the feature map.

[0131] The processor (130) can generate multiple bounding boxes in a feature map. The bounding boxes are boxes of various sizes, and each pixel of the feature map can be positioned at the center of the box. The processor (130) can calculate the probability that the box contains an object in the original image. If the calculated probability is less than a preset value, the processor (130) can identify that there is no object. The processor (130) can identify boxes in the feature map where the probability of an object existing is greater than or equal to a preset value. Additionally, the processor (130) can identify the area occupied by each identified box as a candidate area. If multiple boxes overlap and the candidate areas overlap, the processor (130) can identify the area occupied by the box with a higher probability of an object existing as a candidate area.

[0132] The processor (130) can analyze candidate regions using an object analyzer through a non-maximum suppression (NMS) technique and finally identify one candidate region where an object exists. The processor (130) can analyze the final candidate region through the object analyzer to identify the type, size, color, etc. of the object included in the region selected by the user.

[0133] In operation 1230, the processor (130) can use an artificial intelligence model to change an object included in the selected area and display a photo including the changed image on the display (110).

[0134] Referring to FIG. 16, the processor (130) can generate an image of an object modified based on a user command using an artificial intelligence model. The artificial intelligence model may be a generative AI. Additionally, the artificial intelligence model for generating an image based on a user command may include Adobe Firefly or Midjourney.

[0135] The processor (130) can input a photo, information analyzing a selected area in the photo, and user commands into the artificial intelligence model. The artificial intelligence model can generate a picture desired by the user using a generator and a discriminator. The generator generates an image according to the user's command, and the discriminator evaluates the harmony between the generated image and the original image and can play a role in naturally transforming the image.

[0136] The processor (130) can generate a plurality of images (40) that have been modified from an object and display them on the display (110). The user can select a desired image from the plurality of images by touch input or voice recognition.

[0137] For example, the user can command the object held by the child to be changed into a magic wand. In this case, the processor (130) can generate multiple images of the child holding the magic wand. The user can select one of the multiple images.

[0138]

[0139] * Referring to FIG. 17, when one of the multiple images is selected, the processor (130) can change the original image into an image with the object changed and display it on the display (110).

[0140] For example, the original image shows a child holding a cylindrical object, but the processor (130) can change the object the child is holding into a magic wand based on the user's command and display it on the display (110).

[0141] Meanwhile, various embodiments of the present disclosure may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented as the processor itself. According to software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0142] Meanwhile, computer instructions for performing processing operations of a display device (100) according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When computer instructions stored in such a non-transitory computer-readable medium are executed by a processor of a specific device, they cause the specific device to perform processing operations in the display device (100) according to various embodiments described above.

[0143] Non-transient computer-readable media refers to media that store data semi-permanently and can be read by a device, unlike media that store data for a short period of time such as registers, caches, and memory. Specific examples of non-transient computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0144] Although the present invention has been described above with reference to the attached drawings, the scope of the present invention is determined by the claims set forth below and should not be interpreted as being limited to the aforementioned embodiments and / or drawings. Furthermore, it should be clearly understood that improvements, changes, and modifications to the invention described in the claims that are obvious to those skilled in the art are also included within the scope of the present invention.

Claims

1. In a display device, display; Memory for storing instructions; and at least one processor including processing circuitry; and The above at least one processor, when the instructions are executed individually or collectively, When a touch input is received for a drawing drawn on an image displayed on the above display, a list of UI elements indicating the selection of the drawing and a plurality of animations is displayed on the above display, and A display device that applies the animation to the drawing selected by the UI element when a touch input for the animation is received among the plurality of animations above.

2. In Paragraph 1, The above at least one processor, when the instructions are executed individually or collectively, Based on an animation filter corresponding to the animation above, motion information applied to the skeleton of the drawing is obtained, and A display device that applies the above movement information to the skeleton of the drawing and applies the above animation to the drawing.

3. In Paragraph 1, The above at least one processor, when the instructions are executed individually or collectively, Identify a drawing region containing the drawing in the image based on the skeleton of the drawing, and A display device that displays the UI element in the drawing area when a touch input for the above drawing is received.

4. In Paragraph 1, The above at least one processor, when the instructions are executed individually or collectively, When a touch input for the above drawing is received, a UI element for selecting another drawing area related to the above drawing area is displayed on the display, and A display device that, when a touch input for the above UI element is received, displays a plurality of UI elements on the display indicating a selection of a plurality of drawings drawn on the image.

5. In Paragraph 4, The above at least one processor, when the instructions are executed individually or collectively, When touch input for an animation is received among the above plurality of animations, A display device that applies the animation to a plurality of drawings selected by the plurality of UI elements.

6. In Paragraph 4, The above at least one processor, when the instructions are executed individually or collectively, When a touch input is received to deselect some drawings among the plurality of drawings, the display of some UI elements corresponding to some drawings among the plurality of UI elements is stopped, and A display device that, when a touch input for an animation is received among the plurality of animations above, applies the animation to a drawing selected by the remaining UI elements.

7. In a method for controlling a display device, When a touch input is received for a drawing on an image, a step of displaying a list of UI elements indicating the selection of the drawing and a plurality of animations; and A control method comprising the step of applying the animation to the drawing selected by the UI element when a touch input for the animation is received among the plurality of animations.

8. In Paragraph 7, The step of applying the above animation to the drawing is, A step of obtaining motion information applied to the skeleton of the drawing based on an animation filter corresponding to the animation; and A control method comprising the step of applying the above movement information to the skeleton of the drawing to apply the above animation to the drawing.

9. In Paragraph 7, A step of identifying a drawing region containing a drawing in an image based on the skeleton of the drawing; and A control method further comprising the step of displaying the UI element in the drawing area when a touch input for the drawing is received.

10. In Paragraph 7, When a touch input for the above drawing is received, a step of displaying a UI element for selecting another drawing area related to the above drawing area; and A control method further comprising the step of displaying a plurality of UI elements indicating a selection of a plurality of drawings drawn on the image when a touch input to the above UI element is received.

11. In Paragraph 10, When touch input for an animation is received among the above plurality of animations, A control method further comprising the step of applying the animation to a plurality of drawings selected by the plurality of UI elements.

12. In Paragraph 10, When a touch input is received to deselect some drawings among the plurality of drawings, a step of stopping the display of some UI elements corresponding to some drawings among the plurality of UI elements; and A control method further comprising the step of applying the animation to a drawing selected by the remaining UI elements when a touch input for the animation is received among the plurality of animations.

13. A non-transient computer-readable recording medium that stores one or more instructions executed by a processor of an electronic device to enable a display device to perform an operation, The above operation includes the step of displaying a list of UI elements indicating the selection of the drawing and a plurality of animations when a touch input for a drawing drawn on an image is received; and A recording medium comprising the step of applying the animation to the drawing selected by the UI element when a touch input for the animation is received among the plurality of animations above.

Images