Apparatus and method for displaying sticker including at least one prompt

Abstract

The present disclosure relates to an apparatus and a method for displaying a sticker including at least one prompt, the method comprising: displaying a first image in a first area of a display; displaying one or more stickers in a second area of the display; when at least one of the one or more stickers is selected through an input of a user, displaying the selected sticker in association with the first image; generating second feature information on the basis of at least a portion of first feature information stored in association with the selected sticker, the first image, and metadata stored in association with the first image; and using the second feature information so as to modify the first image, thereby enabling a second image to be generated.

Description

Device and method for displaying a sticker including at least one prompt

[0001] The present embodiments relate to a device and method for displaying a sticker including at least one prompt.

[0002] Generative AI (generative AI) has recently attracted significant attention due to the rapid advancements in deep learning and artificial intelligence technologies. Generative AI is a model capable of generating various formats, such as text, images, and audio, and has shown particularly outstanding performance in image generation. Representative examples include generative adversarial networks (GANs) and variational autoencoders (VAEs).

[0003] Technologies that generate artificial intelligence images have the following problems:

[0004] Existing technologies for generating artificial intelligence images require very complex settings and adjustments to create the images users want, making them difficult for non-experts to use.

[0005] Additionally, prompts need to be written to generate specific types of AI images, but writing these prompts can be challenging, and the quality of the results can vary depending on the prompts written.

[0006] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0007] A method for displaying a sticker according to one embodiment may include: displaying a first image in a first area of ​​a display; displaying at least one sticker in a second area of ​​the display; selecting at least one of the at least one sticker through a user input; displaying the selected sticker in association with the first image; generating second characteristic information based at least in part on first characteristic information stored in association with the selected sticker and the first image and metadata stored in association with the first image; and generating a second image by modifying the first image using the second characteristic information.

[0008] According to one embodiment, an electronic device includes a display for displaying images and videos; one or more processors; and a memory for storing commands, wherein the commands, when executed by the one or more processors, cause the electronic device to perform the following operations: displaying a first image in a first area of ​​the display; displaying at least one sticker in a second area of ​​the display; selecting at least one of the at least one sticker through a user input; displaying the selected sticker in association with the first image; generating second characteristic information based at least in part on first characteristic information stored in association with the selected sticker and the first image and metadata stored in association with the first image; and generating a second image by modifying the first image using the second characteristic information.

[0009] In one embodiment, a computer-readable recording medium stores commands, and when the commands are executed by the one or more processors, the commands may cause the electronic device to perform the following operations: displaying a first image in a first area of ​​the display; displaying at least one sticker in a second area of ​​the display; selecting at least one of the at least one sticker through a user input; displaying the selected sticker in association with the first image; generating second characteristic information based at least in part on first characteristic information stored in association with the selected sticker and the first image and metadata stored in association with the first image; and generating a second image by modifying the first image using the second characteristic information.

[0010] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0011] FIG. 1 is a block diagram of an electronic device within a network environment, according to one embodiment.

[0012] FIG. 2 is a block diagram of a system for generating content using generative artificial intelligence according to one embodiment.

[0013] FIG. 3 is a diagram illustrating a configuration of an electronic device displaying a sticker including at least one prompt according to one embodiment.

[0014] FIG. 4 is a flowchart illustrating a flow of applying a sticker to an image in an electronic device according to one embodiment.

[0015] FIG. 5 is a flowchart illustrating a flow for displaying an image including a sticker in an electronic device according to one embodiment.

[0016] FIG. 6 is a flowchart illustrating a flow for editing an image including a sticker in an electronic device according to one embodiment.

[0017] FIG. 7 is a drawing illustrating an example of applying a seasonal sticker to an image according to one embodiment.

[0018] FIG. 8 is a drawing illustrating an example of displaying or not displaying a sticker applied to an image according to one embodiment.

[0019] FIG. 9 is a drawing illustrating an example in which multiple stickers are applied to an image according to one embodiment.

[0020] FIG. 10 is a drawing illustrating an example of applying stickers repeatedly to an image according to one embodiment.

[0021] FIG. 11 is a drawing showing an example of applying a sticker by adjusting its size to an image according to one embodiment.

[0022] FIG. 12 is a drawing illustrating an example of changing a sticker applied to an image according to one embodiment.

[0023] FIG. 13 is a diagram illustrating an example of displaying an auxiliary sticker of a selected sticker when a sticker to be applied to an image is selected according to one embodiment.

[0024] FIG. 14 is a drawing showing an example in which an auxiliary sticker is selected and applied in FIG. 13 according to one embodiment.

[0025] Hereinafter, embodiments are described in detail with reference to the attached drawings. However, the embodiments may be modified in various ways, and the scope of the patent application is not limited or restricted by these embodiments. It should be understood that all modifications, equivalents, or alternatives to the embodiments are included within the scope of the patent application.

[0026] FIG. 1 is a block diagram of an electronic device within a network environment, according to one embodiment.

[0027] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0028] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0029] The processor (120) can control the operations of the electronic device (101) of FIG. 1 by executing commands stored in the memory (130).

[0030] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0031] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0032] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0033] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0034] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0035] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0036] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0037] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0038] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0039] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0040] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0041] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0042] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0043] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0044] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0045] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0046] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0047] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0048] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0049] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0050]

[0051] FIG. 2 is a block diagram of a system (200) that generates content using generative artificial intelligence according to one embodiment.

[0052] Referring to FIG. 2, a user query / response interface (210) can receive user input. The user input may be in the form of natural language, text, audio, images, and / or video. Furthermore, the user query / response interface (210) can also transmit context information when transmitting the user input. The context information may include various additional information at the time of user input. For example, the additional information included in the context information may include information on the application currently being used by the user or information on the user's location. Furthermore, the user input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Furthermore, the user input may also be in a non-natural language form, such as selecting a menu. The user query / response interface (210) can output the results of the generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of an action requested by the user. The user query / response interface (210) can output results from a generative artificial intelligence system to the user. The output can be in natural language or specific content format, and can also be provided in the form of an action requested by the user.

[0053] The AI ​​(artificial intelligence) framework (220) can receive user input and coordinate and control each component necessary to perform the user's intention based on the user's query.

[0054] The AI ​​framework (220) may include a prompt design component (221), an API / plug-in management component (222), and an output modification component (223).

[0055] The prompt design component (221) can receive user input through the user query / response interface (210).

[0056] The prompt design component (221) can be used to generate prompts suitable for inputting user input into a large language model (LLM) or a large multimodal model (LMM). The prompt design component (221) can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The prompt design component (221) can access knowledge repositories (240) containing user preference data, a prompt library, and prompt examples based on user input to generate prompts and pass the generated prompts to the LLM or LMM.

[0057] The API / plugin management component (222) can communicate with external information when there is a request for additional information when passing user input as input to the generative model. The API / plugin management component (222) can establish a channel for communicating with the outside of the AI ​​framework (220) through the API, and can enable access to various data sources (e.g., knowledge repositories (240)) through the established channel. In addition, the API / plugin management component (222) can request the application / service component (250) through the API to perform an action that ultimately performs the user input, rather than an intermediate result, when the application or service needs to perform the action. Information obtained from the outside can be used to generate a prompt in the prompt design component (221) together with the user input, or can be passed as input to the generative model.

[0058] The output modification component (223) can fine-tune the output from the generative model. For example, the output modification component (223) can verify whether the content generated through the LLM and / or LMM is irrelevant, biased, or harmful. In addition, the output modification component (223) can determine whether the content matches the user's desired result to some extent and, if necessary, can perform additional processes. The output modification component (223) can additionally configure and provide the user with hints to avoid undesired output.

[0059] A generative AI model (230) may generally refer to an artificial intelligence neural network that creates new types of data based on user input information. The generative AI model (230) may include a model that generates images and / or a model that generates language. Representative models that generate images include a generative adversarial network (GAN) and a variational autoencoder (VAE), and examples include a diffusion-based generative model that uses a VAE and a transformer structure. A model that generates language is a model that is trained to statistically output the most appropriate output based on input values, and representative examples include models such as the conversational humanoid AI transformer-generative pre-trained transform (CHAT-GPT) 3 and CHAT-GPT 4. In addition, there is also an LMM that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.

[0060]

[0061] FIG. 3 is a diagram illustrating a configuration of an electronic device displaying a sticker including at least one prompt according to one embodiment.

[0062] Referring to FIG. 3, the electronic device (300) may be configured to include a processor (310), a display unit (320), a memory (330), and an interface unit (340).

[0063] According to various embodiments, the electronic device (300) of FIG. 3 may be at least partially similar to the electronic device (101) of FIG. 1 or may include other embodiments of the electronic device.

[0064] According to one embodiment, the display unit (320) can display status information (or indicators), limited numbers and characters, moving pictures, and still pictures generated during the operation of the electronic device (300). In addition, according to the present disclosure, an image can be displayed in a first area of ​​the display unit (320), and at least one sticker can be displayed in a second area of ​​the display unit (320). The display unit (320) can have a configuration corresponding to the display module (160) of FIG. 1.

[0065] According to one embodiment, the memory (330) may store an operating system, application programs, and storage data (e.g., phone numbers, SMS messages, compressed image files, and videos) for controlling the overall operation of the electronic device (300). Furthermore, the memory (330) may store stickers containing at least one prompt according to the present disclosure. The memory (330) may have a configuration corresponding to the memory (130) of FIG. 1 .

[0066] According to one embodiment, the interface unit (340) is a device that provides an interface with a user of the electronic device (300) and receives user input. In addition, the interface unit (340) may receive a sticker selected by the user according to the present disclosure as the user input. The interface unit (340) may have a configuration corresponding to the interface (177) of FIG. 1.

[0067] According to one embodiment, the processor (310) may include a generative editing unit (311). The processor (310) may have a configuration corresponding to the processor (120) of FIG. 1.

[0068] According to one embodiment, when the generative editing unit (311) enters the editing mode through an image viewer application (e.g., gallery), it may display a first image selected by the user in a first area of ​​the display unit (320) and display at least one sticker in a second area of ​​the display unit (320). At this time, the second area may be an area separate from the first area located at the bottom of the first area, or an area located at the bottom of the first area and partially overlapping with the first area.

[0069] According to one embodiment, when displaying at least one sticker in the second area of ​​the display unit (320), the generative editing unit (311) may analyze the first image and display at least one sticker, excluding stickers that cannot be applied to the first image, in the second area of ​​the display unit (320).

[0070] According to one embodiment, when displaying at least one sticker in the second area of ​​the display unit (320), the generative editing unit (311) may analyze the first image, classify stickers that are suitable for application to the first image as recommended stickers, and display at least one recommended sticker in the second area of ​​the display unit (320).

[0071]

[0072] FIG. 7 is a drawing illustrating an example of applying a seasonal sticker to an image according to one embodiment.

[0073] Referring to FIG. 7, the generative editing unit (311) can output a first image in a first area (720) and display various stickers including at least one prompt in a second area (730) in an example (710) of applying a seasonal sticker (740).

[0074] When the generative editing unit (311) detects an input for selecting a seasonal sticker (740) having at least one prompt for applying the current season to the background of the image among various stickers displayed in the second area (730), the seasonal sticker (740) can be displayed on the first image. At this time, the method for selecting the seasonal sticker (740) may be for the user to drag the seasonal sticker (740) and drop it on the first image.

[0075] According to one embodiment, when a season sticker (740) is applied to the first image, the generative editing unit (311) can output an image (750) with the background and clothing changed by applying the summer season to the first image if the current season is summer, and can output an image (760) with the background and clothing changed by applying the fall season to the first image if the current season is fall.

[0076] In one embodiment, the summer image (750) and the fall image (760) may display a seasonal sticker (740) and a visual object (e.g., an icon) (e.g., four stars) (770) indicating that the image has been changed by artificial intelligence.

[0077] That is, when a user enters the editor through an image viewer (gallery) on an electronic device (300) (e.g., a mobile phone), the generation editing unit (311) of the electronic device (300) can display an image (first image) that the user wants to edit in the first area (720) as in the example (710) of FIG. 7, and display stickers that the user can select in the second area (730).

[0078] When a user selects a seasonal sticker (740) from among various stickers in the second area (730) and drags it to place it on the first image in the first area (720), the generative editing unit (311) can apply an effect corresponding to the current season to the first image as described above.

[0079]

[0080] Returning to the description of FIG. 3, according to one embodiment, the generative editing unit (311) confirms a user's input for selecting at least one sticker from among at least one sticker through the interface unit (340), and when selection of at least one sticker is confirmed, the selected sticker can be displayed in association with the first image. In this case, displaying the selected sticker in association with the first image may mean displaying the image of the selected sticker at a preset location on the first image or at a location selected by the user, or displaying the image of the selected sticker around the first image.

[0081] According to one embodiment, if a secondary sticker exists for the selected sticker, the generative editing unit (311) may provide the user with a secondary sticker corresponding to the selected sticker and receive the user's selection. At this time, the secondary sticker may be a sticker that is suitable for use with the selected sticker and may include at least one prompt related to at least one prompt included in the selected sticker. For example, if the user selects Sticker A, stickers that allow selection of the application range and the application shape within the image of Sticker A may be displayed as secondary stickers. In addition, if the user selects a sticker with a rain effect, stickers that adjust the intensity of the rain effect sticker may be displayed as secondary stickers.

[0082] According to one embodiment, the generative editing unit (311) may select a specific object by following the outline of the specific object when the user drags and places the selected sticker over a specific object in the first image, and finally, when the user places the selected sticker on the specific object, at least one prompt included in the selected sticker may be applied to the object.

[0083] In one embodiment, the generative editing unit (311) may provide information about a limited area (e.g., a selected object) along with the first image and the sticker to a generative engine (e.g., a large vision model (LVM) (360)) such that only the limited area (e.g., a selected object) in which at least one prompt of the sticker is reflected is applied.

[0084]

[0085] FIG. 8 is a drawing illustrating an example of displaying or not displaying a sticker applied to an image according to one embodiment.

[0086] Referring to FIG. 8, when displaying an image (810, 820) to which a seasonal sticker (840) is applied, the generative editing unit (311) may prevent the applied seasonal sticker (840) from being displayed, as in the example of 810, or may cause the applied seasonal sticker (840) to be displayed, as in the example of 820.

[0087] According to one embodiment, whether or not to display the applied seasonal sticker (840) can be determined by touching the image to select it, by touching a visual object (e.g., an icon) (830) indicating that the image has been transformed by artificial intelligence, or by selecting various other options.

[0088]

[0089] FIG. 9 is a drawing illustrating an example in which multiple stickers are applied to an image according to one embodiment.

[0090] Referring to FIG. 9, the generative editing unit (311) can generate an image (910) transformed by artificial intelligence by applying a plurality of stickers (930, 940).

[0091] In one embodiment, the transformed image (910) has a rain sticker (930) applied that includes a prompt to make it rain in the background, a season sticker (940) applied that includes a prompt to make the current season (e.g., spring) applied to the background, and a visual object (e.g., an icon) (920) indicating that the image has been transformed by artificial intelligence.

[0092] According to one embodiment, the generative editing unit (311) may sequentially apply a plurality of stickers (930, 940) to the first image. For example, if the second selected sticker is a subordinate sticker located below the first selected sticker, the generative editing unit (311) may generate a first result image by applying at least one prompt of the first sticker to the first image, and may generate a second result image by applying at least one prompt of the second selected sticker, which is a subordinate sticker, to the first result image, thereby generating a final result image.

[0093] According to one embodiment, the generative editing unit (311) can merge a plurality of stickers (930, 940) into one sticker, and can merge at least one prompt included in the plurality of stickers (930, 940) to create one prompt and store it in the merged sticker.

[0094]

[0095] Returning to the description of FIG. 3, according to one embodiment, the generative editing unit (311) may display the selected sticker by at least partially overlapping the first image or may display the selected sticker around the first image. At this time, the generative editing unit (311) may remove the display of the selected sticker displayed in the second area or update the display so that the selected sticker is displayed in the second area differently from other stickers that are not selected.

[0096] According to one embodiment, the generative editing unit (311) can change at least some of the display position, size, shape, and angle of the selected sticker in response to additional input from the user for the selected sticker displayed on the first image.

[0097] According to one embodiment, the generative editing unit (311) may generate second characteristic information based at least in part on first characteristic information stored in association with the selected sticker and metadata stored in association with the first image. At this time, the first characteristic information and the second characteristic information may each include prompt information. In addition, the first characteristic information may include information including a description for performing at least some change and manipulation operations among size, color, brightness, and saturation for at least some area of ​​the target image. The second characteristic information may include information regarding a range or object to which a sticker selected from among the first images selected by the user is applied.

[0098] According to one embodiment, the first characteristic information may be information stored in the EXIF ​​(exchangeable image file format) of the selected sticker, information managed and stored separately from the selected sticker in the storage of the electronic device, or information stored on an external server of the electronic device.

[0099] In one embodiment, the metadata stored in association with the first image may be image-related information stored in an EXIF ​​file of the first image.

[0100] According to one embodiment, the image-related information may be at least one of at least one keyword information extracted through object recognition of the first image, at least one image-related keyword information stored through a user's input, and at least one character information extracted and stored from text information in the first image.

[0101] According to one embodiment, before generating a second image, which is an AI image, the generative editing unit (311) may first overlay an image layer related to the effect of at least one prompt included in the selected sticker onto the first image and display it as a preview. This allows the user to check in advance the intent of the effect applied to the first image by the selected sticker.

[0102] The method of displaying stickers of the present disclosure can also be applied to the camera preview.

[0103] According to one embodiment, when a user selects a sticker in the camera preview state and takes a picture, or selects a sticker after taking a picture, the generative editing unit (311) may, before generating an artificial intelligence image, overlay an image layer related to the effect of at least one prompt included in the selected sticker on the taken image and show it as a preview first.

[0104] According to one embodiment, the generative editing unit (311) can generate a second image by transforming a first image using second characteristic information, and store the second characteristic information in conjunction with the second image.

[0105] According to one embodiment, the generative editing unit (311) may generate the second image on its own, but may also generate the second image through a large vision model (LVM) (360). The generative editing unit (311) may input at least one prompt included in the first image and the selected sticker into the large vision model (LVM) (360), and receive the second image, which is an artificial intelligence image generated through the large vision model (360).

[0106] According to one embodiment, the generative editing unit (311) may store the second characteristic information by including the second characteristic information in the second image, by matching the second image and the second characteristic information using a database, or by matching the second characteristic information corresponding to the second image and storing it through an external server, in order to store the second characteristic information in conjunction with the second image. In this case, the second characteristic information may include information about the generative engine (artificial intelligence engine) used to apply the selected sticker, and may include at least one prompt modified while applying the selected sticker.

[0107] According to one embodiment, the generative editing unit (311) can display the second image on the display unit (320).

[0108] According to one embodiment, the generative editing unit (311) may display a second image to replace the first image displayed in the first area of ​​the display unit (320). At this time, the generative editing unit (311) may move the display of the selected sticker displayed at the first position in the first area to the second position in the first area and display it. Here, the first position in the first area may be the position where the user initially placed the selected sticker, and the second position in the first area may be the placement position of a sticker defined in advance.

[0109] According to one embodiment, when the generative editing unit (311) displays a second image to replace the first image displayed in the first area of ​​the display unit (320), it may remove the display of the selected sticker displayed in the first location of the first area and display that it is a transformed image in the second location of the first area. That is, when displaying the second image, the generative editing unit (311) may display only a visual object (e.g., an icon, four stars) indicating that it is an image transformed by artificial intelligence without displaying a sticker.

[0110] According to one embodiment, when the generative editing unit (311) detects a user's information request input regarding information related to a selected sticker through the interface unit (340), the information related to the selected sticker can be displayed on the display unit (320). At this time, the user's information request input can be an input of touching a second image or an input of touching a visual object (e.g., an icon, four stars) indicating that it is a modified image. In addition, the information related to the selected sticker can include at least one of information related to at least one prompt included in the selected sticker and description information about the selected sticker.

[0111] According to one embodiment, when a request is made to display an image, the generative editing unit (311) may execute an image viewer, check whether the image includes a sticker, display the sticker included in the image if the image includes a sticker, apply a sticker not applied to the image to the image to transform the image, and display the image, in which all stickers included in the image are applied and the transformation is completed, as a final image on the display unit (320).

[0112] According to one embodiment, if the image does not include a sticker, the generative editing unit (311) can display the image requested by the user as a final image on the display unit (320).

[0113] In one embodiment, the generative editing unit (311) may execute an image viewer at the user's request and execute an editing mode for stickers applied to an image. In the editing mode, a new sticker may be added, an existing sticker may be changed to a different sticker, or the range or intensity to which at least one prompt of the sticker is applied may be changed.

[0114] According to one embodiment, the generative editing unit (311) can apply stickers repeatedly and adjust the intensity of the effect applied by at least one prompt included in the sticker.

[0115] FIG. 10 is a drawing illustrating an example of applying stickers repeatedly to an image according to one embodiment.

[0116] Referring to FIG. 10, the generative editing unit (311) can generate an image (1010) with a background of heavy rain applied by repeatedly applying a rain sticker (1030) including a prompt to make it rain in the background, and display a visual object (e.g., an icon) (1020) indicating that the image has been transformed by artificial intelligence.

[0117]

[0118] Returning to the description of FIG. 3, the generative editing unit (311) can adjust the intensity of the effect applied by at least one prompt included in the sticker by adjusting the size of the sticker.

[0119] FIG. 11 is a drawing showing an example of applying a sticker by adjusting its size to an image according to one embodiment.

[0120] Referring to FIG. 11, the generative editing unit (311) can generate an image (1110) with a small amount of rain background applied when the size of the rain sticker (1140) including a prompt to make it rain in the background is small, and can generate an image (1120) with a large amount of rain background applied when the size of the rain sticker (1150) is large.

[0121] According to one embodiment, the generative editing unit (311) may display a visual object (e.g., an icon) (1130) indicating that the image (1110, 1120) has been transformed by artificial intelligence.

[0122]

[0123] Returning to the description of FIG. 3, according to one embodiment, when the generative editing unit (311) detects selection of a sticker to be applied to an image through the interface unit (340) in the editing mode, it checks whether the selected sticker conflicts with an existing sticker already applied to the image, and if the selected sticker does not conflict with an existing sticker already applied to the image, it can add the selected sticker to the image.

[0124] According to one embodiment, if the selected sticker conflicts with an existing sticker already applied to the image, the generative editing unit (311) may display on the display unit (320) that the selected sticker conflicts along with information about the conflicting sticker.

[0125] Alternatively, if the selected sticker conflicts with an existing sticker already applied to the image, the generative editing unit (311) may display the already applied sticker that conflicts with the selected sticker to the user, and may notify (display a warning message) that adding the selected sticker will remove the conflicting already applied sticker, and if the user still requests to add the selected sticker, the existing conflicting sticker may be removed from the image and the selected sticker may be applied to the image.

[0126]

[0127] FIG. 12 is a drawing illustrating an example of changing a sticker applied to an image according to one embodiment.

[0128] Referring to FIG. 12, the generative editing unit (311) displays an image (1210) to which a rain sticker (1240) including a prompt to make it rain in the background and a season sticker (1250) having a prompt to apply the current season (spring) to the background of the image are applied, and when a user request is received to change the rain sticker (1240) to a cloud sticker (1260) including a prompt to make clouds appear in the background and to change the season sticker (1250) to a calendar sticker (1270) having a prompt to display the current month and day, the generative editing unit (311) can remove the rain sticker (1240) and the season sticker (1250) and display an image (1220) to which the cloud sticker (1260) and the calendar sticker (1270) are applied.

[0129] According to one embodiment, the generative editing unit (311) may display a visual object (e.g., an icon) (1230) indicating that the image (1210, 1220) has been transformed by artificial intelligence.

[0130] Even though the season sticker (1250) is removed from the transformed image (1220) in Fig. 12, the season does not change because the season sticker (1250) is not reflected when the original image is taken and the image is displayed in spring.

[0131]

[0132] Returning to the description of FIG. 3, according to one embodiment, if the selected sticker has additional options, the generative editing unit (311) may provide the user with a window for selecting additional options, allowing the user to select the additional options. The additional options may include an object (target) to which at least one prompt included in the selected sticker is applied, a range, or the intensity of the effect of at least one prompt. An example of providing a window for selecting an auxiliary sticker corresponding to an additional option is described below with reference to FIGS. 13 and 14.

[0133] FIG. 13 is a diagram illustrating an example of displaying an auxiliary sticker of a selected sticker when a sticker to be applied to an image is selected according to one embodiment.

[0134] Referring to FIG. 13, when a user enters an editor through an image viewer (gallery) on an electronic device (300) (e.g., a mobile phone), the generative editing unit (311) of the electronic device (300) can display an image (first image) that the user wants to edit in a first area (1320) as in the example (1301) of FIG. 13, and display stickers that the user can select in a second area (1330).

[0135] According to one embodiment, when a user selects and drags a rain sticker (1340) that can apply a rain effect from among various stickers in the second area (1330) and places it on a first image in the first area (1320), the generative editing unit (311) can generate an artificial intelligence image (second image) by applying the rain effect to the first image as described above.

[0136] According to one embodiment, when displaying an artificial intelligence image (second image), if a secondary sticker exists in the non-sticker (1340) that is the selected sticker, the generative editing unit (311) may display information about the secondary sticker as in example 1302 or example 1303.

[0137] According to one embodiment, in the example (1302) of FIG. 13, the generative editing unit (311) may display an artificial intelligence image (second image) to which the effect of a non-sticker (1340) is applied in a first area (1320), and display auxiliary stickers (1351, 1352, 1353) as a pop-up window (1350) below the non-sticker (1340) displayed in the first area (1320).

[0138] According to one embodiment, in the example (1303) of FIG. 13, the generative editing unit (311) can display an artificial intelligence image (second image) to which the effect of a non-sticker (1340) is applied in the first area (1320), and display auxiliary stickers (1351, 1352, 1353) of the non-sticker (1340) in the second area (1330).

[0139] At this time, the first auxiliary sticker (1351) is a sticker that applies a drizzle effect indicating a very small amount of rain, the second auxiliary sticker (1352) is a sticker that applies a heavy rain effect indicating a large amount of rain, and the third auxiliary sticker (1353) is a sticker that applies a hail effect indicating rain including hail.

[0140] FIG. 14 is a drawing showing an example in which an auxiliary sticker is selected and applied in FIG. 13 according to one embodiment.

[0141] Referring to FIG. 14, in the example (1410) of FIG. 14, the generative editing unit (311) may display an AI image (second image) to which a rain effect is applied to a first image after a rain sticker (1340) including a prompt to make it rain in the background is selected by the user in FIG. 13, and then, when a first auxiliary sticker (1351) is selected by the user, the generative editing unit may apply the drizzle effect included in the first auxiliary sticker (1351) to the AI ​​image (second image) to generate a modified AI image. At this time, the generative editing unit (311) may display a rain sticker (1340), a first auxiliary sticker (1351), and a visual object (e.g., an icon) (1440) indicating that the modified AI image is an image modified by AI.

[0142] According to one embodiment, in the example (1420) of FIG. 14, the generative editing unit (311) may display an AI image (second image) in which a rain effect is applied to a first image when a rain sticker (1340) including a prompt to make it rain in the background is selected by the user in FIG. 13, and then, when a second auxiliary sticker (1352) is selected by the user, the generative editing unit (311) may apply the heavy rain effect included in the second auxiliary sticker (1352) to the AI ​​image (second image) to generate a modified AI image. At this time, the generative editing unit (311) may display the rain sticker (1340), the second auxiliary sticker (1352), and a visual object (e.g., an icon) (1440) indicating that the image is modified by AI on the modified AI image.

[0143] According to one embodiment, in the example (1430) of FIG. 14, the generative editing unit (311) may display an AI image (second image) in which a rain effect is applied to a first image when a rain sticker (1340) including a prompt to make it rain in the background is selected by the user in FIG. 13, and then, when a third auxiliary sticker (1353) is selected by the user, the generative editing unit (311) may apply a hail effect included in the third auxiliary sticker (1353) to the AI ​​image (second image) to generate a modified AI image. At this time, the generative editing unit (311) may display a rain sticker (1340), a third auxiliary sticker (1353), and a visual object (e.g., an icon) (1440) indicating that the image has been modified by AI on the modified AI image.

[0144]

[0145] Returning to the description of FIG. 3, according to one embodiment, the generative editing unit (311) may also generate a sticker according to a user's request. When a user requests generation of a sticker including at least one prompt (for example, when the user presses a sticker generation button), the generative editing unit (311) may output an input window for inputting a prompt or a list of preset recommended prompts, receive a prompt corresponding to an action or effect to be applied to the sticker from the user, receive an appropriate image or character to be used in the generated sticker, and generate and save a sticker including at least one prompt.

[0146] According to one embodiment, the processor (310) can control the overall operation of the electronic device (300). In addition, the processor (310) can perform the function of the generative editing unit (311). The generative editing unit (311) is illustrated separately to explain each function separately. Therefore, the processor (310) can include at least one processor configured to perform the function of the generative editing unit (311). In addition, the processor (310) can include at least one processor configured to perform some of the functions of the generative editing unit (311).

[0147] According to one embodiment, the processor (310) can control the operations of the electronic device (300) of FIG. 3 by executing instructions stored in the memory (330).

[0148] Each "processor (310)" in this disclosure may include a processing circuit or may include multiple processors. For example, as used in this disclosure, including in the claims, the term "processor (310)" may include various processing circuits including at least one processor, wherein one or more of the at least one processor may be configured to perform various functions described in this disclosure, individually and / or collectively, in a distributed manner. When "processor," "at least one processor," and "one or more processors" are described in this disclosure as being configured to perform multiple functions, these terms include, but are not limited to, situations where one processor performs some of the recited functions and another processor performs other of the recited functions, and situations where a single processor may perform all of the recited functions. Furthermore, the at least one processor may include a combination of processors that perform various recited / disclosed functions, for example, in a distributed manner. The at least one processor may execute program instructions to achieve or perform various functions.

[0149] According to one embodiment, the electronic device (300) of FIG. 3 may be an augmented reality (AR) device. In this case, the electronic device (300) may generate and display a converted AR object by reflecting a sticker containing at least one prompt on a real object.

[0150] According to one embodiment, although the description of FIG. 3 describes a case where a sticker including at least one prompt is applied to an image, an artificial intelligence sound file can also be created by applying a sticker including at least one prompt when editing a sound file in addition to an image.

[0151] For example, when entering the editing mode through the sound editing application of the generative editing unit (311), the waveform image of the selected sound file selected by the user may be displayed in the first area of ​​the display unit (320), and at least one sticker may be displayed in the second area of ​​the display unit (320). At this time, at least one prompt included in the sticker may be a prompt for adjusting the sound file.

[0152] According to one embodiment, the generative editing unit (311) confirms a user's input for selecting at least one sticker from among at least one sticker through the interface unit (340), and when selection of at least one sticker is confirmed, the selected sticker can be displayed in association with a waveform image.

[0153] According to one embodiment, the generative editing unit (311) may generate second characteristic information based at least in part on first characteristic information stored in association with the selected sticker and metadata stored in association with the selected sound file. In this case, the first characteristic information and the second characteristic information may each include at least one prompt information. In addition, the first characteristic information may include information including a description for performing an action of changing the waveform of at least a portion of the sound, adjusting the volume of the sound, or selecting an instrument.

[0154] According to one embodiment, the generative editing unit (311) can generate an artificial intelligence sound file by modifying a selected sound file using second characteristic information, and can store the second characteristic information in conjunction with the artificial intelligence sound file.

[0155] According to one embodiment, the generative editing unit (311) can display a waveform image of an artificial intelligence sound file on a display unit (320) and output it through a speaker (not shown).

[0156] Meanwhile, the AI ​​editing method using stickers that include at least one prompt can also be applied to text editing in addition to images and sounds.

[0157] According to one embodiment, when an electronic device (300) checks an image with a sticker attached and posted on a social network service (SNS), the electronic device (300) may apply and display an image that reflects at least one prompt included in the sticker and some parameters of the electronic device (300) (e.g., region, weather, temperature, humidity, health information). That is, the present disclosure may generate and provide an artificial intelligence image based on a viewer's device that views an image with a sticker attached.

[0158]

[0159] FIG. 4 is a flowchart illustrating a flow of applying a sticker to an image in an electronic device according to one embodiment.

[0160] A method of applying a sticker to an image in an electronic device (300) according to one embodiment of the present disclosure may be performed, for example, according to a flowchart illustrated in FIG. 4. The flowchart illustrated in FIG. 4 is merely a flowchart according to one embodiment of the operation of the electronic device (300), and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 410 to 428 may be performed in at least one processor (310) of the electronic device (300).

[0161] Referring to FIG. 4, according to one embodiment, in operation 410, the electronic device (300) can display one image in a first area of ​​the display.

[0162] According to one embodiment, in operation 412, the electronic device (300) may display at least one sticker in a second area of ​​the display. In this case, the second area may be an area separate from the first area located at the bottom of the first area, or an area located at the bottom of the first area and partially overlapping with the first area.

[0163] According to one embodiment, in operation 412, the electronic device (300) may analyze the first image and display at least one sticker, excluding stickers that cannot be applied to the first image, in a second area of ​​the display. For example, if the first image is a landscape image without people, the electronic device (300) may prevent a smiley sticker with a prompt to "edit with a smiley face" that is applicable to people from being displayed in the second area because it cannot be applied to the first image.

[0164] According to one embodiment, in operation 414, the electronic device (300) can confirm selection of at least one of at least one sticker through a user input.

[0165] According to one embodiment, if selection of at least one sticker is confirmed as a result of operation 414, the electronic device (300) can display the selected sticker in association with the first image in operation 416.

[0166] According to one embodiment, in operation 416, the electronic device (300) may display the selected sticker by at least partially overlapping the first image or display the selected sticker around the first image. At this time, the electronic device (300) may remove the display of the selected sticker displayed in the second area or update the display so that the selected sticker is displayed in the second area differently from other stickers that are not selected.

[0167] Meanwhile, in operation 416, the electronic device (300) can change at least some of the display position, size, shape, and angle of the selected sticker in response to the user's additional input for the selected sticker displayed on the first image.

[0168] According to one embodiment, in operation 418, the electronic device (300) may generate second characteristic information based at least in part on first characteristic information stored in association with the selected sticker and metadata stored in association with the first image and the first image. At this time, the first characteristic information and the second characteristic information may each include at least one prompt information. In addition, the first characteristic information may include information including a description for performing at least some change and manipulation operations among size, color, brightness, and saturation for at least some area of ​​the target image.

[0169] According to one embodiment, the first characteristic information may be information stored in EXIF ​​of the selected sticker, information managed and stored separately from the selected sticker in the storage of the electronic device, or information stored on an external server of the electronic device.

[0170] In one embodiment, the metadata stored in association with the first image may be image-related information stored in an EXIF ​​file of the first image.

[0171] According to one embodiment, the image-related information may be at least one of at least one keyword information extracted through object recognition of the first image, at least one image-related keyword information stored through a user's input, and at least one character information extracted and stored from text information in the first image.

[0172] According to one embodiment, in operation 420, the electronic device (300) can generate a second image by transforming the first image using the second characteristic information.

[0173] According to one embodiment, in operation 422, the electronic device (300) can store the second characteristic information in conjunction with the second image.

[0174] According to one embodiment, in operation 424, the electronic device (300) can display a second image on the display.

[0175] According to one embodiment, in operation 424, the electronic device (300) may display a second image to replace the first image displayed in the first area of ​​the display. At this time, the electronic device (300) may move the display of the selected sticker displayed at the first location in the first area to the second location in the first area and display it. Here, the first location in the first area may be the location where the user initially placed the selected sticker, and the second location in the first area may be the placement location of the sticker defined in advance.

[0176] According to one embodiment, in operation 424, when the electronic device (300) displays a second image to replace the first image displayed in the first area of ​​the display, the electronic device may remove the display of the selected sticker displayed in the first location of the first area and display that it is a transformed image in the second location of the first area. That is, when the electronic device (300) displays the second image, it may display only a visual object (e.g., an icon, four stars) indicating that it is an image transformed by artificial intelligence without displaying a sticker.

[0177] According to one embodiment, in operation 426, the electronic device (300) may detect a user's information request input regarding information related to the selected sticker. In this case, the user's information request input may be an input of touching a second image or an input of touching a visual object (e.g., an icon, four stars) indicating that it is a modified image.

[0178] According to one embodiment, when the user's information request input detection in operation 426 is confirmed, the electronic device (300) may display information related to the selected sticker on the display in operation 428. At this time, the information related to the selected sticker may include at least one of information related to at least one prompt included in the selected sticker and descriptive information about the selected sticker.

[0179] According to one embodiment, if the selection of at least one sticker is not confirmed as a result of the confirmation of operation 414, or if the detection of the user's information request input is not confirmed as a result of operation 426, the present algorithm may be terminated.

[0180]

[0181] FIG. 5 is a flowchart illustrating a flow for displaying an image including a sticker in an electronic device according to one embodiment.

[0182] An operation method for displaying an image including a sticker in an electronic device (300) according to one embodiment of the present disclosure may be performed, for example, according to the flowchart illustrated in FIG. 5. The flowchart illustrated in FIG. 5 is merely a flowchart according to one embodiment of the operation of the electronic device (300), and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 510 to 560 may be performed in at least one processor (310) of the electronic device (300).

[0183] Referring to FIG. 5, according to one embodiment, in operation 510, the electronic device (300) can execute an image viewer according to a user's request.

[0184] According to one embodiment, in operation 520, the electronic device (300) can determine whether the image contains a sticker.

[0185] According to one embodiment, if the image confirmed in operation 520 does not contain a sticker, in operation 560, the electronic device (300) may display the image requested by the user as the final image on the display.

[0186] According to one embodiment, if the image includes a sticker as a result of the confirmation in operation 520, the electronic device (300) may display the sticker included in the image in operation 530.

[0187] According to one embodiment, in operation 540, the electronic device (300) can check whether there is a sticker to be applied that has not been applied to the image.

[0188] According to one embodiment, if there is a sticker to be applied that has not been applied to the image as a result of the confirmation in operation 540, the electronic device (300) may apply the unapplied sticker to the image in operation 550 to transform the image. At this time, if there are multiple stickers to be applied to the electronic device (300), they may be applied to the image sequentially one by one.

[0189] According to one embodiment, if there are no applicable stickers that have not been applied to the image as a result of the confirmation in operation 540, in operation 560, the electronic device (300) can display an image in which all stickers included in the image have been applied and the transformation has been completed as a final image on the display.

[0190]

[0191] FIG. 6 is a flowchart illustrating a flow for editing an image including a sticker in an electronic device according to one embodiment.

[0192] A method of editing an image including a sticker in an electronic device (300) according to one embodiment of the present disclosure may be performed, for example, according to a flowchart illustrated in FIG. 6. The flowchart illustrated in FIG. 6 is merely a flowchart according to one embodiment of the operation of the electronic device (300), and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 610 to 650 may be performed in at least one processor (310) of the electronic device (300).

[0193] Referring to FIG. 6, according to one embodiment, in operation 610, the electronic device (300) can execute an image viewer according to a user's request.

[0194] According to one embodiment, in operation 620, the electronic device (300) can execute an editing mode for a sticker applied to an image at the user's request.

[0195] According to one embodiment, in operation 630, the electronic device (300) can detect a selection of a sticker to be applied to an image in an edit mode.

[0196] According to one embodiment, if the selection of a sticker to be applied is detected as a result of the confirmation in operation 630, the electronic device (300) can check in operation 640 whether the selected sticker conflicts with an existing sticker already applied to the image. For example, a case where the selected sticker conflicts with an existing sticker applied may be when the selected sticker is a sticker indicating "apply summer weather" but a sticker indicating "apply winter weather" has already been applied.

[0197] According to one embodiment, if the selected sticker does not conflict with an existing sticker already applied to the image as a result of the verification in operation 640, the electronic device (300) may add the selected sticker to the image in operation 650.

[0198] According to one embodiment, if a sticker to be edited is not selected as a result of the confirmation of operation 630, the electronic device (300) may stop the editing mode and terminate the present algorithm.

[0199] According to one embodiment, if the selected sticker conflicts with an existing sticker already applied to the image as a result of the verification of operation 640, the electronic device (300) may terminate the present algorithm without adding the selected sticker to the image.

[0200] Meanwhile, in FIG. 6, if the selected sticker conflicts with an existing sticker already applied to the image as a result of the confirmation of operation 640, the selected sticker is not added to the image. However, as an example different from the case of FIG. 6, the electronic device (300) may remove the conflicting existing sticker already applied to the image from the image and apply the selected sticker to the image.

[0201]

[0202] According to one embodiment, a method for displaying a sticker may include: displaying a first image in a first area of ​​a display; displaying at least one sticker in a second area of ​​the display; selecting at least one of the at least one sticker through a user input; displaying the selected sticker in association with the first image; generating second characteristic information based at least in part on first characteristic information stored in association with the selected sticker and the first image and metadata stored in association with the first image; and generating a second image by modifying the first image using the second characteristic information.

[0203] According to one embodiment, the second region may be a region separate from the first region located at the bottom of the first region, or a region located at the bottom of the first region and partially overlapping with the first region.

[0204] According to one embodiment, the operation of displaying the selected sticker in association with the first image may display the selected sticker by at least partially overlapping the first image or displaying the selected sticker around the first image.

[0205] According to one embodiment, the first characteristic information may be at least one prompt information.

[0206] According to one embodiment, the first characteristic information may include information including a description for performing a change and manipulation operation of at least some of size, color, brightness, and saturation for at least some area of ​​the target image.

[0207] According to one embodiment, the first characteristic information may be information stored in EXIF ​​of the selected sticker, information managed and stored separately from the selected sticker in a storage device of the electronic device, or information stored in an external server of the electronic device.

[0208] According to one embodiment, the metadata stored in conjunction with the first image is image-related information stored in an EXIF ​​file of the first image, and the image-related information may be at least one of at least one keyword information extracted through object recognition of the first image, at least one image-related keyword information stored through input from the user, and at least one character information extracted and stored from text information in the first image.

[0209] According to one embodiment, the method may further include an operation of changing at least some of the display position, size, shape, and angle of the selected sticker in response to an additional input from the user for the selected sticker displayed on the first image.

[0210] In one embodiment, the operation of displaying the selected sticker in association with the first image may include an operation of removing the display of the selected sticker displayed in the second area or updating the display so that the selected sticker is displayed in the second area differently from other stickers that are not selected.

[0211] According to one embodiment, the operation of displaying the at least one sticker in the second area of ​​the display may include the operation of analyzing the first image; and the operation of displaying the at least one sticker, excluding stickers that cannot be applied to the first image, in the second area of ​​the display.

[0212] According to one embodiment, the method may further include an operation of displaying the second image on the display.

[0213] In one embodiment, the act of displaying the second image on the display may include an act of displaying the second image as a replacement for the first image displayed in the first area of ​​the display.

[0214] According to one embodiment, the operation of displaying the second image as a replacement for the first image displayed in the first area of ​​the display may further include an operation of removing display of the selected image displayed at the first location in the first area.

[0215] According to one embodiment, the operation of displaying the second image as a replacement for the first image displayed in the first area of ​​the display may further include an operation of moving the display of the selected sticker displayed at a first location in the first area to a second location in the first area and displaying it.

[0216] According to one embodiment, the operation of displaying the second image as a replacement for the first image displayed in the first area of ​​the display may further include an operation of removing the display of the selected sticker displayed at the first location of the first area and displaying the transformed image at the second location of the first area.

[0217] According to one embodiment, the method may further include displaying information related to the selected sticker on the display in response to a user input action indicating that the image is a modified image.

[0218] According to one embodiment, the information related to the selected sticker may include information related to at least one prompt included in the selected sticker and descriptive information about the selected sticker.

[0219] According to one embodiment, the second characteristic information may include at least one prompt information.

[0220] According to one embodiment, the method may further include an operation of storing the second characteristic information in conjunction with the second image.

[0221] According to one embodiment, an electronic device includes a display for displaying images and videos; one or more processors; and a memory for storing commands, wherein the commands, when executed by the one or more processors, cause the electronic device to perform the following operations: displaying a first image in a first area of ​​the display; displaying at least one sticker in a second area of ​​the display; selecting at least one of the at least one sticker through a user input; displaying the selected sticker in association with the first image; generating second characteristic information based at least in part on first characteristic information stored in association with the selected sticker and the first image and metadata stored in association with the first image; and generating a second image by modifying the first image using the second characteristic information.

[0222]

[0223] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0224] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0225] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0226] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0227] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0228] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0229] The embodiments of the present invention disclosed in this specification and drawings are merely specific examples presented to easily explain the technical contents according to the embodiments of the present invention and to help understand the embodiments of the present invention, and are not intended to limit the scope of the embodiments of the present invention. Therefore, the scope of the various embodiments of the present invention should be interpreted as including all changes or modified forms derived based on the technical ideas of the various embodiments of the present invention in addition to the embodiments disclosed herein.

Claims

1. In electronic devices, A display that displays images and videos; one or more processors; and Memory that stores commands Including, The above instructions, when individually or collectively executed by the one or more processors, cause the electronic device to: An action of displaying a first image in a first area of ​​the display; An action of displaying at least one sticker in a second area of ​​the display; An action of selecting at least one of the above at least one sticker through user input; An action of displaying the selected sticker in association with the first image; An operation of generating second characteristic information based at least in part on first characteristic information stored in association with the selected sticker and metadata stored in association with the first image and the first image; and An operation of generating a second image by transforming the first image using the second characteristic information. An electronic device that performs a task.

2. In paragraph 1, The second area above is, or an area separated from the first area located at the bottom of the first area; or An area located at the bottom of the first area and partially overlapping with the first area Electronic devices.

3. In any one of paragraphs 1 and 2, The above instructions, when individually or collectively executed by the one or more processors, cause the electronic device to: When performing an action to display the selected sticker in association with the first image, or displaying the selected sticker at least partially overlapping the first image; or To display the selected sticker around the first image Electronic devices.

4. In any one of paragraphs 1 to 3, The above first characteristic information is, At least one prompt information Including, The above second characteristic information is, At least one prompt information Electronic devices containing.

5. In any one of paragraphs 1 to 4, The above first characteristic information is, Information containing a description for performing at least some of the following operations: changing size, color, brightness, and saturation for at least some area of ​​the target image. Including, The above first characteristic information is, Information stored in EXIF ​​of the selected sticker above, or Information stored and managed separately from the selected sticker in the storage device of the electronic device, or Information stored on the external server of the above electronic device Electronic devices.

6. In any one of paragraphs 1 to 5, The metadata stored in conjunction with the above first image is: Image-related information stored in the EXIF ​​file of the above first image, Information related to the above image is: At least one keyword information extracted through object recognition of the first image, At least one image-related keyword information stored through the user's input, At least one character information extracted and stored in the text information in the first image above At least one of Electronic devices.

7. In any one of paragraphs 1 to 6, The above instructions, when individually or collectively executed by the one or more processors, cause the electronic device to: An action of changing at least some of the display position, size, shape, and angle of the selected sticker in response to the user's additional input for the selected sticker displayed on the first image. How to include more.

8. In any one of paragraphs 1 to 7, The above instructions, when individually or collectively executed by the one or more processors, cause the electronic device to: The action of displaying the selected sticker in association with the first image is: An action to remove the display of the selected sticker displayed in the second area or to update the display so that the selected sticker is displayed in the second area differently from other stickers that are not selected. How to include.

9. In any one of paragraphs 1 to 8, The above instructions, when individually or collectively executed by the one or more processors, cause the electronic device to: The action of displaying at least one sticker in the second area of ​​the display is: An operation of analyzing the first image; and An action of displaying at least one sticker, excluding stickers that cannot be applied to the first image, in the second area of ​​the display. How to include 10. In any one of paragraphs 1 to 9, The above instructions, when individually or collectively executed by the one or more processors, cause the electronic device to: An action of displaying the second image on the display How to include more.

11. In any one of paragraphs 1 to 10, The above instructions, when individually or collectively executed by the one or more processors, cause the electronic device to: The action of displaying the second image on the display is: An action of displaying the second image as a replacement for the first image displayed in the first area of ​​the display. How to include.

12. In any one of paragraphs 1 to 11, An action of displaying information related to the selected sticker on the display in response to a user input action indicating that the image is a modified image. How to include more.

13. In any one of paragraphs 1 to 12, Information related to the selected sticker above, Information related to at least one prompt contained in the above selected sticker; and Description information for the selected sticker above How to include.

14. In any one of paragraphs 1 to 13, An operation of storing the second characteristic information in conjunction with the second image. How to include more.

15. In the method of displaying stickers, An action of displaying a first image in a first area of ​​a display; An action of displaying at least one sticker in a second area of ​​the display; An action of selecting at least one of the above at least one sticker through user input; An action of displaying the selected sticker in association with the first image; An operation of generating second characteristic information based at least in part on first characteristic information stored in association with the selected sticker and metadata stored in association with the first image and the first image; and An operation of generating a second image by transforming the first image using the second characteristic information. How to include.

Images