Electronic apparatus and method for providing content thereof

Abstract

Disclosed are an electronic apparatus and a method for providing content. An operation method for an electronic apparatus according to an embodiment may comprise an operation in which a message is displayed on a first portion through a touch screen display. The operation method may comprise an operation in which, on the basis of at least a part of a determination that the message includes information associated with an image stored in an electronic apparatus (101), a plurality of images stored in the electronic apparatus (101) are identified on the basis of at least a part of the information. The operation method may comprise an operation in which the plurality of images are displayed on a second portion different from the first portion through the touch screen display. The operation method may comprise an operation in which a user input for selecting at least one image from among the plurality of images is received through the touch screen display. The operation method may comprise an operation in which the at least one image is displayed on the first portion through the touch screen display. Various other embodiments are possible.

Description

Electronic devices and methods for providing content thereon

[0001] Embodiments of the present invention relate to an electronic device and a method for providing content therefor.

[0002] Users can interact with other users through applications. For example, a user might converse with another user through a chat application. When attempting to send content (such as images or videos) to another user during a chat conversation, the user may need to manually locate the content within their device. Furthermore, navigating the content requires switching between applications and screens, potentially reducing usability.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0004] An electronic device according to one embodiment may include a touchscreen display. The electronic device may include communication circuitry. The electronic device may include at least one processor including processing circuitry. The electronic device may include a memory storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display a message on a first portion via the touchscreen display. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify a plurality of images stored in the memory based at least in part on a determination that the message includes information associated with an image stored in the memory. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the plurality of images on a second portion different from the first portion through the touchscreen display. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to receive, through the touchscreen display, a user input for selecting at least one image from the plurality of images.The above instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the at least one image on the first portion through the touchscreen display.

[0005] A method of operating an electronic device according to one embodiment may include an operation of displaying a message on a first portion via a touchscreen display. The method may include an operation of identifying a plurality of images stored in the electronic device (101) based on at least a portion of a determination that the message includes information associated with an image stored in the electronic device (101). The method may include an operation of displaying the plurality of images on a second portion different from the first portion via the touchscreen display. The method may include an operation of receiving a user input for selecting at least one image from the plurality of images via the touchscreen display. The method may include an operation of displaying the at least one image on the first portion via the touchscreen display.

[0006] According to one embodiment, a computer-readable recording medium storing one or more computer programs may include instructions for performing the method in a processor.

[0007] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0008] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0009] FIG. 2 is a diagram for explaining a generative artificial intelligence system according to one embodiment.

[0010] FIGS. 3A to 3D are drawings for explaining a content providing method according to one embodiment.

[0011] FIG. 4a and FIG. 4b are drawings for explaining a method for providing generated content according to one embodiment.

[0012] FIG. 5 is a diagram for explaining an operation of processing content according to one embodiment.

[0013] FIG. 6 illustrates a system for providing content according to one embodiment.

[0014] Figure 7 is a flowchart of a method of operating an electronic device according to one embodiment.

[0015] Figure 8 is a flowchart of a method of operating an electronic device according to one embodiment.

[0016] Figure 9 is a flowchart of a method of operating an electronic device according to one embodiment.

[0017] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0018]

[0019] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100), according to one embodiment.

[0020] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0021] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134).

[0022] According to one embodiment, the processor (120) may be implemented as a circuit (e.g., a processing circuit) such as a system on chip (SoC) or an integrated circuit (IC). The processor (120) may include one or more processors. For example, the processor (120) may include a combination of one or more processors such as a CPU, a GPU, an MPU, an AP, and a CP. The instructions stored in the memory (130) may be executed by one processor to cause the electronic device (101) to perform and / or control the operations of the electronic device (101) to be described with reference to FIGS. 2 to 9. The instructions stored in the memory (130) may be executed by multiple processors to cause the electronic device (101) to perform and / or control the operations of the electronic device (101) to be described with reference to FIGS.

[0023] According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0024] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0025] The memory (130) can store various data used by at least one component (e.g., the processor (120) or the sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., the program (140)) and input data or output data for commands related thereto. The memory (130) can include a volatile memory (132) or a non-volatile memory (134). According to one embodiment, the memory (130) can include one or more memories. The instructions stored in the memory (130) can be stored in one memory. The instructions stored in the memory (130) can be divided and stored in a plurality of memories. The instructions stored in the memory (130) can be executed by the processor (120) to cause the electronic device (101) to perform and / or control the operations of the electronic device (101) to be described with reference to FIGS. 2 to 9.

[0026] According to one embodiment, the instructions stored in the memory (130) may cause the electronic device (101) to perform one or more operations when individually or collectively executed by at least one processor (e.g., the main processor (121) and / or the auxiliary processor (123)). For example, the instructions stored in the memory (130) may be executed by one processor (e.g., the main processor (121) or an auxiliary processor (123) such as a communication processor) or by a plurality of processors operating cooperatively (e.g., the main processor (121) and the auxiliary processor (123)).

[0027] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0028] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0029] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0030] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0031] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0032] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0033] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0034] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0035] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0036] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0037] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0038] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0039] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0040] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0041] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0042] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0043] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0044] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0045]

[0046] FIG. 2 is a diagram for explaining a generative artificial intelligence system according to one embodiment.

[0047] Referring to FIG. 2, according to one embodiment, the generative artificial intelligence system (200) may be a program (e.g., a software module) implemented on an electronic device (e.g., an electronic device (101) of FIG. 1) and / or a server (e.g., a server (108) of FIG. 1).

[0048] According to one embodiment, a User Query / Response Interface (210) can receive user input. The user input can be input in a type (or modality) such as natural language, image, audio, and / or video. Additionally, context information can be transmitted together when the user input is transmitted. The context information can include various side information related to the time when the user input is input to the artificial intelligence system (200). For example, the context information may include information about the application currently being used by the user or information about the user's location. Additionally, the user input can be a mixed type of input of the above-described natural language, image, audio, video, and / or context information. Additionally, the user input can include non-natural language input, such as selecting a menu.

[0049] In one embodiment, the user query / response interface (210) may provide the user with output from the generative artificial intelligence system. The output may include a natural language-based response and / or specific content. The output may also include an action requested by the user.

[0050] In one embodiment, an AI framework (220) may receive user input. Based on the user input (e.g., a user's query), the AI ​​framework (220) may coordinate and / or control one or more components necessary to perform an action corresponding to the user's intent.

[0051] According to one embodiment, user input received from the user query / response interface (210) may be transmitted to a prompt design component (221). The prompt design component (221) may be used to generate a prompt suitable as input to a generative model (e.g., a large language model (LLM) and / or a large multimodal model (LMM)) based on the user input.

[0052] In one embodiment, the prompt design component (221) may be an AI component that utilizes a machine learning algorithm or a neural network. The prompt design component (221) may generate improved prompts over time through learning. The prompt design component (221) may access a knowledge repository (230) to generate prompts based on user input. The knowledge repository (230) may include user preference data, a prompt library, and / or prompt examples. The prompt design component (223) may provide the generated prompts to a generative model (e.g., an LLM and / or an LMM).

[0053] According to one embodiment, the APIs / Plugins management component (223) can communicate with an external information source based on a request for additional information when user input is transmitted to the generative model.

[0054] According to one embodiment, the APIs / Plugins management component (223) can establish a communication channel for communication with the outside of the system (200) via the API. The APIs / Plugins management component (223) can enable access to various data sources via the communication channel. The acquired information can be used to generate prompts by the prompt design component (221) together with user input, or can be used as input for the input of the generative model (250).

[0055] According to one embodiment, the APIs / Plugins management component (223) may request a final action via an API when the final action corresponding to user input, rather than an intermediate action, must be performed by an application or service.

[0056] In one embodiment, the refiner component (225) can fine-tune the output of the generative model (250). For example, the refiner component (225) can determine the relevance (e.g., a score) between the output (e.g., content) of the generative model and the user input. For example, the refiner component (225) can determine whether the output contains biased information (e.g., selective information). For example, the refiner component (225) can determine whether the output contains harmful information (e.g., violent content or profanity).

[0057] In one embodiment, the refinement component (225) may determine the degree of matching (e.g., a score) between the output of the generative model (250) and the user input (e.g., the intent of the user input). If the refinement component (225) determines that the output of the generative model (250) does not correspond to the user input, the refinement component (225) may modify the output to correspond to the user input.

[0058] In one embodiment, the refinement component (225) may provide hints to the user (e.g., hints for prompt generation) to enable the user to obtain information that matches the user's intent from the generative model (250).

[0059] According to one embodiment, a generative model (250) may refer to an artificial intelligence neural network that generates new data (e.g., text, images, audio, or video) based on user input (e.g., user utterance). The generative model (250) may include an image generation model and / or a language generation model.

[0060] In one embodiment, the image generation model may include a generative adversarial network (GAN) and / or a variational autoencoder (VAE). An example of an image generation model is a diffusion-based generative model having the structure of a VAE and a transformer.

[0061] In one embodiment, a language generation model (e.g., ChatGPT) may be a model trained to generate statistically most appropriate output based on input. The language generation model may include an LMM. The LMM can identify various types of input, such as text, images, audio (e.g., speech), and / or video, and generate new data corresponding to the input.

[0062]

[0063] FIGS. 3A to 3D are drawings for explaining a content providing method according to one embodiment.

[0064] Referring to FIGS. 3A to 3D , according to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1 ) may display a screen (301) to a screen (303) through a display module (e.g., the display module (160) of FIG. 1 ). The display module (e.g., the display module (160) of FIG. 1 ) may include a touchscreen display. The screens (301) to (303) may include screens for applications running on the electronic device (101). For example, the screens (301) to (303) may include screens for applications for a user (e.g., a user of the electronic device (101)) to interact (e.g., have a conversation) with another user. Interaction refers to an action in which two or more objects influence each other, and may include a conversation. The application may include an application that allows interaction between multiple users. For example, the application may include a chat application (e.g., chat applications such as Samsung Messages, KakaoTalk, Line, and Telegram), a social networking service (SNS) application (e.g., SNS applications such as Instagram, X, and Facebook), and a collaboration tool application. The application may include an application that allows a user to write or perform a task on their own. For example, the application may include a shopping application, a blog application, a web browser, and a diary application. Note that the types of applications running on the electronic device (101) are not limited thereto.

[0065] According to one embodiment, the electronic device (101) can display a message (e.g., message (320)) through a touchscreen display. As at least a part of displaying the message (e.g., message (320)), the electronic device (101) can display an input area (e.g., a virtual keyboard) on a screen (e.g., screen (301) to screen (303)). For example, when a user wishes to input characters into the electronic device (101), the electronic device (101) can display a virtual keyboard (350) on the screen (e.g., screen (301) to screen (303)). The virtual keyboard (350) can include a tab area (310). The tab area (310) can include at least one icon. Each icon can correspond to a function that the electronic device (101) can provide. For example, the tab area (310) may include an icon (311) associated with an AI (artificial intelligence) model, a gallery icon (312), an icon corresponding to an emoticon function, an icon corresponding to a translation function, an icon corresponding to a clipboard function, and an icon corresponding to a keyboard settings function. The icon (311) associated with the AI ​​model may be associated with providing a selectable function using the AI ​​model, or may be associated with generated content (e.g., generated content such as generated images, videos, and texts) generated using the AI ​​model. Note that the types of icons that may be included in the tab area (310) may vary depending on the user's settings and are not limited to the above examples. The user may input text into the electronic device (101) using the virtual keyboard (350). The user may select an icon (e.g., an icon such as the icon (311) associated with the AI ​​model, the gallery icon (312)) included in the tab area (310) to execute a corresponding function.For example, the electronic device (101) may execute a function associated with a gallery application in response to a user input to the gallery icon (312).

[0066] According to one embodiment, the electronic device (101) can determine whether a message (e.g., message (320)) includes information associated with an image (e.g., an image stored in a gallery, an image stored in the memory (130) of FIG. 1), such as a video. The message (e.g., message (320)) is a message received and / or sent through the electronic device (101) and may include text displayed through a display (e.g., a touchscreen display). The electronic device (101) can obtain the text from a running application. For example, the electronic device (101) can recognize text (e.g., message (320)) included in the screen (301) through optical character recognition (OCR) technology. For example, the electronic device (101) can receive the text directly from an application running on the electronic device (101). For example, the electronic device (101) can obtain text by converting a voice conversation between users (e.g., a voice conversation included in a voice call or a video call) into text. It should be noted that the operation of the electronic device (101) to obtain text is not limited thereto. The electronic device (101) can obtain text from a screen (e.g., screen (301)) on which a running application is displayed. For example, the electronic device (101) can obtain a message (320) including the text "Send me the pictures of the garden at the beach when we went to Da Nang" included in the screen (301) on which a chat application is displayed. The electronic device (101) can determine whether the text (e.g., message (320)) includes information associated with an image (e.g., a gallery image) stored in a memory (e.g., memory (130) of FIG. 1). For example, the electronic device (101) can determine whether the message (320) includes at least one of context information, location information, and object information associated with an image stored in the memory (130).For example, the electronic device (101) may determine whether the message (320) includes information associated with the transmission of content including an image stored in the memory (130). The message associated with the transmission of content may include a message from a user including an intention to transmit content and / or a message from a counterpart including an intention to request the user to transmit content. The electronic device (101) may utilize an artificial intelligence model (e.g., a context analysis model) to detect a message associated with the transmission of content. The artificial intelligence model may include a model trained for context analysis of a message (e.g., a large language model (LLM)). For example, the context analysis model may be trained to analyze the structure and context of language using a dataset including large-scale text data such as conversation logs and web text for natural language processing (NLP) and natural language understanding (NLU). The context analysis model may extract intents and / or entities from a message (e.g., the message (320)). An intent may refer to the purpose and intention of a speaker's utterance (e.g., writing a text) according to the context of a message (e.g., message (320)). An entity may refer to a word and / or phrase representing information included in a speaker's utterance (e.g., message (320)). The electronic device (101) may extract intents and / or entities from a message (e.g., message (320)) using a context analysis model, thereby determining whether the message (e.g., message (320)) includes information associated with an image stored in a memory (e.g., memory (130) of FIG. 1).For example, the electronic device (101) can determine that the message (320) includes information related to an image stored in the memory (130) by extracting an intent corresponding to the intention of the speaker of the message (320) that says, "Send me a picture of a garden taken at the sea in Da Nang" and entities such as "Da Nang," "at the sea," and "garden" included in the message (320). The electronic device (101) can analyze text (e.g., message (320)) displayed on a screen (e.g., screen (301) to screen (303)) in real time or at set intervals (e.g., n seconds) using an artificial intelligence model (e.g., context analysis model) to determine whether the message (e.g., message (320)) includes information related to an image stored in the memory (e.g., memory (130) of FIG. 1). The electronic device (101) can determine, based on established criteria, whether a message (e.g., message (320)) included in a screen (e.g., screen (301) to screen (303)) includes information associated with an image stored in a memory (e.g., memory (130) of FIG. 1). For example, the electronic device (101) can detect a message associated with the transmission of content by analyzing the context (e.g., intent, entity) of a previous conversation message of the message (320), even if the message is not displayed on the screen (301).

[0067] According to one embodiment, a message associated with the transmission of content (e.g., message (320)) may include a message containing an intent to transmit content, a message requesting the transmission of content, and a message associated with the creation of content. The electronic device (101) may detect whether words and / or phrases in a given context (e.g., a context regarding the transmission of content) are present in the message (e.g., message (320)). A message containing an intent to transmit content may include a message with the purpose of transmitting content (e.g., content such as an image, video, text, or document file) to another user and / or server. For example, a message containing an intent to transmit content may include a message that is determined to contain an intent to transmit content based on the context of the conversation, such as "I'll send you what I captured" or "Would you like to see this picture?" A message requesting the transmission of content may be a message requesting content from a counterparty (e.g., another user, a server). For example, a message requesting content transmission may include a message requesting content transmission from the other party based on the context of the conversation, such as "Send me a picture" or "Can you send me a file?". A message associated with content creation may include a message that the creation and / or transmission of the content is deemed to be helpful to the conversation between users. For example, a message associated with content creation may include a message that the electronic device (101) generates content associated with text, such as "I think this outfit would look good on you, what do you think?", and if the content is transmitted, it is deemed to be helpful to the conversation.

[0068] According to one embodiment, the electronic device (101) may display at least one graphical object or effect in the message (e.g., message (320)) based at least in part on a determination that the message (e.g., message (320)) includes information associated with an image stored in a memory (e.g., memory (130) of FIG. 1 ). The at least one graphical object or effect may be intended to emphasize the message (e.g., message (320)). An effect may be a graphical element used to draw a user's attention to or emphasize the message (e.g., message (320)) itself or a feature associated with the message (e.g., message (320)). For example, an effect may include graphical elements such as zoom in and out effects, blinking effects, shaking effects, animation effects, and coloring effects. At least one graphical object or effect may be for emphasizing the message itself (e.g., message (320)) or a function associated with the message (e.g., message (320)). The first graphical object may include at least one graphical object and / or effect for emphasizing the message itself. For example, the first graphical object may include graphical objects (330), (335), and (340) that are displayed to emphasize the message (320) itself. The second graphical object may include at least one graphical object and / or effect for emphasizing a function associated with the message. For example, the second graphical object may include a coloring effect (332) that is used to emphasize graphical objects (331) and a gallery icon (312) that are displayed on an icon (311) associated with an AI model corresponding to a function associated with the message (320). The coloring effect (332) may be an example of an effect in which the color of the gallery icon (312) changes.

[0069] According to one embodiment, the electronic device (101) may notify a user of the existence of an AI function associated with the message (e.g., message (320)) or of the existence of content (e.g., image) associated with the message by displaying at least one graphical object or effect in the message. The electronic device (101) may change the text color and / or text size of the message (e.g., message (320)), add a highlight effect, or display an animation effect in the message. The electronic device (101) may display at least one graphical object or effect in whole, part (e.g., keyword), and / or around the message (e.g., message (320)). For example, the electronic device (101) may notify a user of the existence of an image associated with the message (320) by displaying graphical objects (330), (335), and (340) in the message (320). For example, the electronic device (101) may display a graphic object (330) that is a highlight mark together with the message (320). For example, the electronic device (101) may display a graphic object (340) on the border of the message (320). The electronic device (101) may inform the user that there are functions (e.g., functions associated with an AI model, functions such as a gallery function) that the user can use with respect to the message (e.g., the message (320)). For example, the electronic device (101) may display a graphic object (335) together with the message (320) to inform the user that a function using an AI model is available with respect to the message (320). It should be noted that at least one of the graphic object (330), the graphic object (335), and the graphic object (340) that the electronic device (101) may display to highlight the message (e.g., the message (320)) may be displayed together, but is not limited thereto. For example, graphic objects (330), graphic objects (335), and graphic objects (340) may not be displayed together or may be displayed sequentially by toggling.It should be noted that the types and / or number of graphic objects or effects that the electronic device (101) can display in a message (e.g., message (320)) are not limited to the examples above. For example, the electronic device (101) can further display additional graphic objects or effects in a message (e.g., message (320)).

[0070] According to one embodiment, the electronic device (101) may display a second graphical object (e.g., a graphical object (331), an effect (332)) for emphasizing a function associated with a message (e.g., a message (320)) together with an icon (e.g., an icon (311) associated with an AI model, a gallery icon (312)) corresponding to the function associated with the message (e.g., a message (320)). The second graphical object may include at least one graphical object and / or effect for emphasizing a function associated with the message (e.g., a message (320)). For example, the electronic device (101) may display a graphical object (331) for emphasizing an icon (311) associated with an AI model together with an icon (311) associated with the AI ​​model. For example, the electronic device (101) may display a coloring effect (332) on the gallery icon (312) to highlight it, thereby informing the user that there is a function (e.g., a gallery application) that can be used in association with the message (320). A first graphical object displayed to emphasize the message (e.g., message (320)) itself and a second graphical object displayed in an icon included in the tab area (310) to emphasize a function associated with the message (e.g., message (320)) may be displayed together or separately. For example, the electronic device (101) may display a first graphical object (e.g., graphical object (330), graphical object (335), graphical object (340)) together with the message based at least in part on a determination that the message (320) includes information associated with an image stored in the memory (130), and in response to content (e.g., an image) to be transmitted being prepared, display a second graphical object (e.g., graphical object (331), effect (332)). For example, the electronic device (101) can display a second graphic object (e.g., graphic object (331), effect (332)) together with a first graphic object (e.g., graphic object (330), graphic object (335), graphic object (340)).For example, the electronic device (101) may display only the second graphic object (e.g., graphic object (331), effect (332)) without displaying the first graphic object (e.g., graphic object (330), graphic object (335), graphic object (340)), and may display the first graphic object (e.g., graphic object (330), graphic object (335), graphic object (340)) in response to the user selecting the second graphic object (e.g., graphic object (331), effect (332)). It should be noted that the above description is an example of an operation in which the first graphic object and the second graphic object are displayed, and is not limited to the described examples.

[0071] According to one embodiment, the electronic device (101) may display at least one graphic object or effect in the application itself, or may add a layer to the screen (e.g., screen (302), screen (303)) and display at least one graphic object or effect on the added layer, in order to display at least one graphic object or effect on the screen (e.g., screen (302), screen (303)) together with a message (e.g., message (320)). By adding a layer to the screen (e.g., screen (302), screen (303)) and displaying at least one graphic object and / or effect on the added layer, the electronic device (101) may freely display graphic objects or effects even in third-party applications.

[0072] According to one embodiment, the electronic device (101) may generate a query to search for content related to the message (e.g., message 320) among content (e.g., gallery content) stored in the electronic device (101) based on the message (e.g., message 320). The electronic device (101) may generate a query to identify an image related to the message (e.g., message 320) among a plurality of images stored in a memory (e.g., memory 130 of FIG. 1). As at least a part of identifying the plurality of images, the electronic device (101) may generate a keyword based on at least a portion of information (e.g., information related to the image) included in the message (e.g., message 320). The electronic device (101) may perform identification of the plurality of images based on the keyword. The electronic device (101) may generate a query to identify an image related to the message (e.g., message 320) among the plurality of images based on the keyword. The electronic device (101) can extract keywords for generating a query from a message (e.g., message (320)). For example, the keywords can include keywords corresponding to at least one of context information, location information, and object information. For example, the electronic device (101) can extract context information requesting to send a photo, location information of 'Da Nang, the sea', and information about an object called 'Jeong-i' from the message (320) "Send me the photos of Jeong-i taken at the sea when we went to Da Nang." The context information can include information about the flow of an interaction (e.g., a conversation) that includes the message (e.g., message (320)). The context information can include entities, which are words and / or phrases representing information included in an interaction (e.g., a conversation) that includes the message (e.g., message (320)).Object information may include personalized object information for the user. For example, the object information may include not only information classifying the object included in the message (320) (e.g., category), but also information about a specific person related to the user, such as 'Mom', 'Son', or 'Jeong-i-won' (e.g., information such as name or nickname). The electronic device (101) may generate a query by reflecting the personalized information for the user. For example, the electronic device (101) may recognize that the object information 'Jeong-i-won' is the name of a person related to the user and reflect this in the query generation. The object information may be associated with tag information included in content stored in the electronic device (101). The object information may be associated with a facial image stored in the electronic device (101). For example, the object information may include facial information extracted from a person image stored in the electronic device (101) (e.g., information such as face location information within the image and facial feature information). The electronic device (101) can reflect object information associated with a face image stored in the electronic device (101) in query generation.

[0073] According to one embodiment, the electronic device (101) can generate a query using keywords (e.g., 'Da Nang', 'sea', 'garden') extracted from a message (e.g., message (320)). For example, the electronic device (101) can generate a query such as 'A garden at the sea in Da Nang' from the message (320). The electronic device (101) can generate a query by extracting keywords included in the conversation content before and after the message (e.g., message (320)) associated with the transmission of the content. The electronic device (101) can generate a query based on the context of the entire conversation including the message (e.g., message (320)) associated with the transmission of the content. For example, the electronic device (101) can generate a query based on context information obtained by analyzing the content of the entire conversation.

[0074] According to one embodiment, the electronic device (101) may generate a query to perform a search for content (e.g., an image) stored in the electronic device (101) in response to detecting words and / or phrases in a given context (e.g., a context regarding the transmission of content) from a message (e.g., a message (320)). For example, the electronic device (101) may generate a query to perform a search for an image in response to detecting words and / or phrases in a context regarding the transmission of content, such as “Send me ~,” “Give me a picture / video of ~,” or “Do you have a picture / video of ~?” in a message (e.g., a message (320)). The electronic device (101) may generate a query to perform a search for content (e.g., an image) stored in the electronic device (101) in response to detecting the receipt and / or transmission of content (e.g., an image, a video) in a conversation between users. It should be noted that the trigger conditions for the electronic device (101) to generate a query for searching for content (e.g., images) stored in the electronic device (101) are not limited to the examples above. The electronic device (101) may detect words and / or phrases in a given context (e.g., context regarding the transmission of content) or, in response to detecting the reception and / or transmission of content (e.g., images, videos) in a conversation between users, collect messages received and / or transmitted immediately before and use them as inputs for a model (e.g., large language model (LLM)). The electronic device (101) may then use the content (e.g., images, videos) received and / or transmitted in the conversation between users as inputs for a model (e.g., a generative model) that generates the content.

[0075] According to one embodiment, the electronic device (101) may obtain first content associated with a message (e.g., message (320)) based on a query. The first content may include an image associated with the message (e.g., message (320)) among a plurality of images stored in a memory (e.g., memory (130) of FIG. 1). The content stored in the electronic device (101) may be converted into a vector format and stored in a database (DB) (e.g., DB (615) of FIG. 6). The electronic device (101) may convert the content converted into a vector format into a language vector (e.g., LangChain) format and store the content in the database (e.g., DB (615) of FIG. 6). The content converted into a language vector format and stored may be input into an artificial intelligence model (e.g., LLM) and searched by a query. For example, the electronic device (101) can retrieve the first content from the DB (615) in a retrieval-augmented generation (RAG) manner that retrieves related information using a query. The electronic device (101) can retrieve the first content associated with the message from among the contents stored in the database (e.g., the database (615) of FIG. 6) using the query. The electronic device (101) can determine the content whose properties correspond to the keyword from among the contents stored in the electronic device (101) as the first content. The properties of the content can be determined based on metadata (e.g., metadata such as EXIF ​​(exchangeable image file format), XMP (extensible metadata platform from Adobe), IPTC (international press telecommunications council)) and / or tag information (e.g., tag information such as person tag information, place tag information, keyword tag information, and event tag information) for each content stored in the electronic device (101).For example, the electronic device (101) may determine, as first content, content corresponding to keywords (e.g., keywords such as 'Da Nang', 'sea') corresponding to place information included in the query based on place tag information (e.g., global positioning system (GPS) coordinates, altitude, location information on a map) included in the content and / or information identifying an object (e.g., landscape, landmark) included in the content. For example, the electronic device (101) may determine, as first content, content corresponding to keywords (e.g., 'garden') corresponding to object information included in the query based on person tag information (e.g., name, nickname, object information such as relationship with the user) included in the content. The person tag information may include information classified by the electronic device (101) by recognizing the face of a person in the content. The electronic device (101) may acquire, by considering all information included in the query, first content associated with a message (e.g., message (320)) among the stored contents. For example, the electronic device (101) may determine the photo (370) as the first content associated with the message (320) by considering both the location information (e.g., 'Da Nang', 'sea') and the object information (e.g., 'garden') included in the query. The electronic device (101) may extract keywords (e.g., keywords such as 'Da Nang', 'sea', 'garden') from sentences (e.g., sentences such as 'When we went to Da Nang, send me the pictures of the garden taken at the sea') included in the message (e.g., message (320)) and use them as inputs for a search engine (e.g., search engine (605) of FIG. 6). The search engine (e.g., search engine (605) of FIG. 6) may be configured to search for a specific image from a plurality of images stored in a memory (e.g., memory (130) of FIG. 1).The electronic device (101) can determine an image (e.g., a photo (370)) determined through a search engine (e.g., a search engine (605) of FIG. 6) as the first content associated with a message (e.g., a message (320)).

[0076] According to one embodiment, the electronic device (101) may determine second content to be transmitted from the application based on the similarity between the message and the first content. The second content may be the first content or content generated based on the message and the first content. In response to the similarity between the message and the first content being lower than a predetermined threshold, the electronic device (101) may generate content and determine the generated content as the second content. The generated content will be described in detail later with reference to FIG. 4. The electronic device (101) may determine the first content as the second content in response to the similarity between the message (e.g., the message associated with the transmission of the content) and the first content being higher than a predetermined threshold. The electronic device (101) may determine the second content based on the similarity between the query generated based on the message (e.g., the message associated with the transmission of the content) and the first content. For example, the electronic device (101) may determine the photo (370) as the second content in response to the fact that the similarity between the acquired first content, the photo (370), and the query generated based on the message (320) (e.g., 'Garden by the sea in Da Nang') is higher than a predetermined threshold. If the first content has a high similarity to the message, the first content may be determined as content that matches the user's intention and may be determined as the second content to be transmitted. The second content may include content such as photos, videos, and documents to be transmitted to the other party and / or the server. The electronic device (101) may display the second content. The electronic device (101) may display the second content on a screen (e.g., screen (303)) through a display module (e.g., display module (160) of FIG. 1). The electronic device (101) may display the second content in an area where the virtual keyboard (350) was previously displayed.The electronic device (101) can display second content in response to a user input for a graphic object (e.g., graphic object (330), graphic object (335), graphic object (340)). For example, the electronic device (101) can display a photo (370), which is second content, in an area where a virtual keyboard (350) was displayed, in response to a user selecting a graphic object (330). The electronic device (101) can transmit the photo (370), in response to a user input to send the photo (370) to the other party. The electronic device (101) can transmit the photos selected by the user in response to a user input to transmit at least some of the photos (370), which are second content, to the other party.

[0077] According to one embodiment, the electronic device (101) may include an electronic device that provides a large screen (e.g., an electronic device such as a foldable electronic device or a rollable electronic device). The foldable electronic device may include an electronic device that includes a foldable housing and a foldable display disposed within a space formed by the foldable housing. The electronic device (101) may display a message (e.g., a message (320)) on the large screen using a multi-window function. The multi-window function may be a function that allows multiple applications to be displayed and operated on the screen simultaneously. For example, the multi-window function may be a function that displays multiple applications on the screen simultaneously in a manner such as a split screen or a pop-up window. The electronic device (101) may display an application that supports image search on a different portion of the screen from the portion where the message (e.g., the message (320)) is displayed. The different portion may be a different portion from the portion where a virtual keyboard is displayed. For example, the electronic device (101) may execute and display an application that supports image search, such as a gallery application, an SNS application, or a web browser. For example, the electronic device (101) may display a screen including a message (320) on the left side and a gallery application on the right side in a multi-window screen divided into two. The electronic device (101) may identify (e.g., search) an image associated with a message (e.g., message (320)) among a plurality of images stored in a memory (e.g., memory (130) of FIG. 1) through an application that supports searching for images displayed on the screen.The electronic device (101) may, in response to detecting words and / or phrases in a given context (e.g., context regarding transmission of content) from a message (e.g., message (320)), identify (e.g., search for) an image associated with the message (e.g., message (320)) from among a plurality of images stored in a memory (e.g., memory (130) of FIG. 1). For example, the electronic device (101) may, in response to detecting words and / or phrases in a context regarding transmission of content, such as “Send me ~,” “Give me a picture / video of ~,” or “Do you have a picture / video of ~?” from the message (e.g., message (320)), identify (e.g., search for) an image associated with the message (320) from among a plurality of images stored in the memory (130). The electronic device (101) may, in response to detecting the reception and / or transmission of content (e.g., images, videos) in a conversation between users, identify (e.g., search) an image associated with a message (e.g., message 320) among a plurality of images stored in a memory (e.g., memory 130 of FIG. 1). The electronic device (101) may generate a query to identify (e.g., search) an image associated with the message (e.g., message 320) among a plurality of images stored in the memory (e.g., memory 130 of FIG. 1). The electronic device (101) may display at least one image associated with the message (e.g., message 320) on a portion of the screen where an application supporting image search is displayed. For example, the electronic device (101) may display at least one image searched for using a query generated based on keywords included in or associated with the message (320). The portion of the screen where at least one of the searched images is displayed may be a different portion from the portion of the screen where a message (e.g., message (320)) is displayed. The portion of the screen where at least one of the searched images is displayed may be a different portion from the portion where a virtual keyboard is displayed. The electronic device (101) may receive a user input for selecting at least one image.The electronic device (101) can transmit at least one image corresponding to a user input through an application in which a message (e.g., message (320)) is displayed. The electronic device (101) can display a plurality of images by arranging them according to the user's preference using an artificial intelligence model. For example, the electronic device (101) can display a plurality of images by arranging them according to priority based on user history data for selecting at least one image from among the plurality of images and / or the user's preference determined using the artificial intelligence model. When the electronic device (101) provides a large screen, the electronic device (101) can display an application that supports image search in a pop-up form in response to the user's selection of a message (e.g., message (320)). The electronic device (101) can display at least one image in a pop-up form in response to the user's selection input for the message (e.g., message (320)) and allow the user to confirm and / or edit the at least one image.

[0078]

[0079] Referring to FIG. 3B, according to one embodiment, an electronic device (101) may provide content (e.g., images) based on interactions (e.g., conversations) between users in an application that supports video calls. The electronic device (101) may analyze the user's voice during the video call and detect messages associated with images stored in a memory (e.g., memory (130) of FIG. 1). For example, the electronic device (101) may detect whether the user's voice includes words and / or phrases within a given context (e.g., context regarding content transmission). The electronic device (101) may display keywords extracted from the user's voice on a screen (e.g., screen (304)). For example, the electronic device (101) can extract keywords such as 'Europe', 'travel', and 'photo' from a conversation between users about photos taken during a trip to Europe, and display each keyword (e.g., keyword (361) to keyword (363)) on the screen (304). Note that the form of the interface on which the keywords (e.g., keyword (361) to keyword (363)) are displayed is not limited to the example illustrated in FIG. 3B. The electronic device (101) can receive a user's input for the keywords (e.g., keyword (361) to keyword (363)). The electronic device (101) can display content (e.g., image) on the screen (e.g., screen (305)) based on the user input. The electronic device (101) can display content (e.g., image) on the screen during a video call or after the video call is terminated. For example, in response to a user input for a keyword (361), the electronic device (101) may search for an image (e.g., image (365)) related to 'Europe' from among a plurality of images stored in the memory (130) and display the image on the screen (305) during a video call. The electronic device (101) may also generate content (e.g., image) based on a user input for a keyword (e.g., keyword (361)).The electronic device (101) may generate content (e.g., an image) in response to a search result in which no content (e.g., an image) corresponding to a keyword (e.g., a keyword (361) or a keyword (363)) is found, or when it is determined that generating and providing content (e.g., an image) would assist in understanding a conversation between users. For example, the electronic device (101) may generate an image related to 'Europe' based on a user input for the keyword (361) and display the image on the screen (305). For example, the electronic device (101) may generate a prompt based on a keyword (e.g., a keyword (361)) corresponding to a user's selection input, input the generated prompt into a generative model (e.g., a generative model (250) of FIG. 2), and display the generated image on the screen (305). The electronic device (101) may, in response to receiving a user input for selecting an image (e.g., image (365)) displayed on a screen (e.g., screen (305)), transmit the selected image (e.g., image (365)) to a terminal (e.g., smartphone) of a counterpart (e.g., a counterpart of a video call). For example, the electronic device (101) may, in response to a user input for an image (e.g., image (365)) corresponding to a keyword (e.g., at least one keyword among keywords (361) to (363)), transmit the selected image (e.g., image (365)) to the counterpart. The image (e.g., image (365)) may be displayed on a video call screen through a video call application on the counterpart's terminal, or may be transmitted to the counterpart's terminal through an application such as a message application or a chat application.

[0080]

[0081] Referring to FIGS. 3C and 3D , according to one embodiment, the electronic device (101) can detect a message associated with an image stored in a memory (e.g., memory (130) of FIG. 1 ) from an application (e.g., an application such as a web browser, Paint, or Note) that allows a user to write or perform a task alone. The electronic device (101) can extract an image displayed on a screen (e.g., screen (306)) and search for an image associated with the extracted image among a plurality of images stored in the memory (e.g., memory (130) of FIG. 1 ). For example, the electronic device (101) can extract and analyze text and / or images included in a screen (306) on which content regarding writing a photo review of a product is displayed, and determine whether a search for images and / or videos stored in the electronic device (101) is necessary. For example, the electronic device (101) can extract a photo of a T-shirt and / or a product name from a screen (306) displaying information about writing a photo review of a product, and can search for a photo related to the T-shirt photo and / or the product name among a plurality of images stored in the memory (130) to identify the photo (385). The electronic device (101) can display the searched photo (e.g., the photo (385)) in an input area (e.g., the virtual keyboard area (380)) so that the user can immediately transmit (e.g., attach) the photo (e.g., the photo (385)). For example, the electronic device (101) can display the photo (385) in the virtual keyboard area (380) and attach it to a photo review in response to a user input (e.g., a user input such as a drag-and-drop input or a touch input).The electronic device (101) may, in response to a user input regarding a photo (e.g., photo (385)) displayed in a virtual keyboard area (e.g., virtual keyboard area (380)), link the photo (e.g., photo (385)) to an image attachment function for writing a photo review provided in an application running on the electronic device (101) to attach the photo (e.g., photo (385)) to the photo review. The electronic device (101) may also enable a search engine (e.g., search engine (605) of FIG. 6) to be utilized in an external application and / or web browser by providing content (e.g., image) to an input area (e.g., virtual keyboard area (380)) included in a screen (e.g., screen (306)).

[0082] According to one embodiment, the electronic device (101) can determine whether text input by a user using an input device (e.g., an input device such as an electronic pen, a keyboard, or a microphone) includes information associated with a memory (e.g., a memory (130) of FIG. 1). For example, the electronic device (101) can detect text (e.g., text (391)) associated with at least one image among a plurality of images stored in the memory (130) from text (390) input by the user on a screen (307) on which a note application is running. The electronic device (101) can display a detected portion of the text displayed on the screen (e.g., screen (307)) separately from other portions. The electronic device (101) can search for content (e.g., an image) using the detected portion (e.g., text (391)) among the text (e.g., text (390)). For example, the electronic device (101) can search for an image related to 'glass shoes that sparkle like stars' among a plurality of images stored in the memory (130) using text (391). The electronic device (101) can search for the image using a search engine (e.g., search engine (605) of FIG. 6). The electronic device (101) can detect a portion that can be used to generate content (e.g., an image) from text (e.g., text (390)) input by a user using an input device (e.g., an input device such as an electronic pen, a keyboard, or a microphone). For example, the electronic device (101) can detect a portion (e.g., text (391)) that is determined to be able to be used to generate an image, but does not include information related to an image stored in the memory (130), from the text (390). The electronic device (101) can extract keywords for generating content (e.g., an image) from the detected text (e.g., text (391)). The electronic device (101) can input a keyword into a model (e.g., a generative model) to generate an image (e.g., image (395)).The image (395) illustrated in FIG. 3D may be generated based on text (e.g., text (391)) detected by the electronic device (101), or may include a searched image. The electronic device (101) may display the image (e.g., image (395)) on a screen (e.g., screen (307)). For example, the electronic device (101) may provide at least one image in the form of an inserted image by displaying the image (395) together with text (390) on the screen (307). The user may select at least one image (e.g., image (395)) provided on the screen (e.g., screen (307)) to change the display position or select whether to display it. The electronic device (101) may display keywords (e.g., keywords such as 'glass slippers') used to search and / or generate the image (e.g., image (395)) together with the searched image (e.g., image (395)).

[0083]

[0084] FIG. 4a and FIG. 4b are drawings for explaining a content providing method according to one embodiment.

[0085] Referring to FIG. 4A, according to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may display a screen (401) to a screen (403) through a display (e.g., the display module (160) of FIG. 1). The display module (e.g., the display module (160) of FIG. 1) may include a touchscreen display. The screens (401) to (403) may include screens for applications running on the electronic device (101). For example, the screens (401) to (403) may include screens for applications for a user (e.g., a user of the electronic device (101)) to interact (e.g., have a conversation) with another user. The electronic device (101) may detect a message associated with the transmission of content from a text obtained from an application running on the electronic device (101). The electronic device (101) can obtain text from a screen (e.g., screen (401)) on which a running application is displayed. For example, the electronic device (101) can obtain a message (420) that is text included in the screen (401) on which a chat application is displayed, such as "I saw this on SNS earlier. Don't you think it would suit Cookie?" The electronic device (101) can determine whether the text (e.g., message (420)) includes information associated with an image (e.g., a gallery image) stored in a memory (e.g., memory (130) of FIG. 1). For example, the electronic device (101) can determine whether the message (420) includes context information, location information, and object information associated with the image stored in the memory (130). For example, the electronic device (101) can determine whether the message (420) includes information associated with the transmission of content including the image stored in the memory (130). The electronic device (101) can detect a message associated with the transmission of content from text (e.g., message (420)).A message associated with the transmission of content may include a message from a user containing an intention to transmit content and / or a message from a counterpart containing an intention to request the user to transmit content. The electronic device (101) may utilize an artificial intelligence model (e.g., a context analysis model) to detect a message associated with the transmission of content. The artificial intelligence model may include a model (e.g., a large language model (LLM)) trained to analyze the context of a message. For example, the context analysis model may be trained to analyze the structure and context of language using a dataset containing large-scale text data such as conversation logs and web texts for natural language processing (NLP) and natural language understanding (NLU). The context analysis model is the same as that described with reference to FIGS. 3A to 3D , and any redundant description thereof will be omitted. The electronic device (101) can detect messages related to the transmission of content by analyzing text displayed on a screen (e.g., screen (401) to screen (403)) in real time or at set intervals (e.g., n seconds) using an artificial intelligence model (e.g., a context model). The electronic device (101) can detect messages related to the transmission of content from text included in the screen (e.g., screen (401) to screen (403)) based on set criteria. For example, the electronic device (101) can detect messages related to the transmission of content by analyzing the context of a previous conversation message of a message (420), even if the message is not displayed on the screen (401). The electronic device (101) can detect messages related to the transmission of content from text. The electronic device (101) can detect messages related to the creation and / or transmission of content based on the analysis of the context of the message.For example, the electronic device (101) may determine that it will be helpful for understanding the conversation between users by generating and providing content showing 'Cookie', a puppy related to the user, wearing the 'flower-patterned clothes' included in the photo (440) by receiving a message (420) and a photo (440) of a puppy related to the user together with the photo (440) regarding puppy clothes, and may determine that the message (420) is a message related to the transmission of the content.

[0086] According to one embodiment, the electronic device (101) may display at least one graphical object or effect in the message (e.g., message (420)) based at least in part on a determination that the message (e.g., message (420)) includes information associated with an image stored in a memory (e.g., memory (130) of FIG. 1 ). The at least one graphical object or effect may be intended to emphasize the message (e.g., message (420)) itself or to emphasize a feature associated with the message (e.g., message (420)). The electronic device (101) may display the graphical object in the application itself to display the graphical object together with the message on the screen (e.g., screen (402), screen (403)), or may add a layer to the screen (e.g., screen (402), screen (403)) and display the graphical object on the added layer. The electronic device (101) can freely display graphic objects in a third-party application by adding a layer to a screen (e.g., screen (402), screen (403)) and displaying graphic objects on the added layer.

[0087] According to one embodiment, the electronic device (101) may notify the user of the existence of content that can be provided in relation to a message by displaying at least one graphical object or effect together with the message (e.g., message (420)). The electronic device (101) may display a first graphical object on the screen to emphasize the message itself (e.g., message (420)) and / or a second graphical object to emphasize a function associated with the message (e.g., message (420)). The electronic device (101) may emphasize the message itself by displaying a first graphical object (e.g., graphical object (430), graphical object (433), graphical object (435)) around the entire message, a portion (e.g., keyword), and / or the message associated with transmission of the content. The electronic device (101) can display a first graphic object (e.g., graphic object (430), graphic object (433), graphic object (435)) by changing the text color and / or text size of the message (e.g., message (420)), adding a highlight effect, or adding an animation effect to the message. For example, in response to detecting a message (420) that is a message associated with transmission of content, the electronic device (101) can display a graphic object (430) that is a highlight mark together with the message (420). For example, the electronic device (101) can highlight the message by displaying the graphic object (433) on the border of the message (420). The electronic device (101) can inform the user that there are functions (e.g., functions associated with an AI model, functions such as a gallery function) that the user can use with respect to the message associated with transmission of content. For example, the electronic device (101) may display a graphical object (435) along with the message (420) to inform the user that a function utilizing an AI model is available for the message (420). The electronic device (101) may display a second graphical object along with a corresponding icon to highlight the function associated with the message.For example, the electronic device (101) may display a second graphic object (e.g., a coloring effect (431)) on the icon (411) to emphasize the icon (411) associated with the AI ​​model. The coloring effect (431) may be an example of an effect that changes the color of the icon (411) associated with the AI ​​model.

[0088] According to one embodiment, the electronic device (101) may display at least one keyword extracted from a message (e.g., message (420)) together with the message (e.g., message (420)). The electronic device (101) may extract keywords that can be used to search for and / or generate content (e.g., images) from the message (e.g., message (420)) and display the extracted keywords together with the message (e.g., message (420)). For example, the electronic device (101) may highlight the word 'cookie' included in the message (420). For example, keywords such as 'puppy', 'eating cookie', and 'cookie image' may be displayed from the keyword 'cookie' extracted from the message (420). The electronic device (101) may search for and / or generate content (e.g., images) in response to a user input for a keyword. For example, in response to a user's selection input for two or more keywords, the electronic device (101) may search for or generate an image based on the selected plurality of keywords. The electronic device (101) may perform image generation or search upon the user's selection. If a message (e.g., message (420)) includes multiple words, the electronic device (101) may display multiple associated keywords corresponding to each word. In response to a user's input for multiple keywords, the electronic device (101) may search for and / or generate content (e.g., an image) using a combination of the selected plurality of keywords.

[0089] According to one embodiment, the electronic device (101) may generate a query to search for content related to the message (e.g., message 420) among content (e.g., gallery content) stored in the electronic device (101) based on a message (e.g., message 420). The electronic device (101) may extract a keyword for generating a query from the message (e.g., message 420). For example, the electronic device (101) may extract the keyword 'cookie' from the message (420). The keyword may include a keyword corresponding to at least one of context information, location information, and object information. The object information may include object information personalized to the user. The electronic device (101) may generate a query by reflecting the personalized information to the user. For example, the electronic device (101) may recognize that the object information 'cookie' is the name of a dog related to the user and reflect it in generating a query. Object information may be associated with tag information included in content stored in the electronic device (101). The electronic device (101) may extract keywords included in conversations before and after a message (e.g., message (420)) associated with the transmission of the content, and generate a query. The electronic device (101) may generate a query based on the context of the entire conversation including the message (e.g., message (420)) associated with the transmission of the content. For example, the electronic device (101) may generate a query based on context information obtained by analyzing the content of the entire conversation together with a photo (440) received prior to the message (420).

[0090] According to one embodiment, the electronic device (101) may obtain first content associated with a message (e.g., message (420)) based on a query. The content stored in the electronic device (101) may be converted into a vector format and stored in a database (e.g., DB (615) of FIG. 6). The electronic device (101) may convert the content converted into a vector format into a language vector (e.g., LangChain) format and store the converted content in the database (e.g., DB (615) of FIG. 6). The content converted and stored into a language vector format may be input into an artificial intelligence model (e.g., LLM) and searched by a query. The electronic device (101) may search for the first content associated with the message among the contents stored in the database (e.g., database (615) of FIG. 6) using the query. The electronic device (101) can determine, as first content, a content whose properties correspond to a keyword among the contents stored in the electronic device (101). The properties of the content can be determined based on metadata (e.g., metadata such as EXIF ​​(exchangeable image file format), XMP (extensible metadata platform from Adobe), and IPTC (international press telecommunications council)) and / or tag information (e.g., tag information such as person tag information, place tag information, keyword tag information, and event tag information) for each content stored in the electronic device (101). The electronic device (101) can obtain first content associated with a message (e.g., message (420)) based on a query generated based on the message (e.g., message (420)). For example, the electronic device (101) may generate a query (e.g., cookie wearing floral clothes) based on a message (420) and a photo (440) received prior to the message (420), and use the query to retrieve first content associated with the message (420).For example, the electronic device (101) may perform a capture on a screen (e.g., screen (401)), obtain an image (e.g., photo (440)) included in the screen (401), and generate a query based on the image. The electronic device (101) may determine, as the first content, a content whose properties correspond to a keyword among the contents stored in the electronic device (101). For example, the electronic device (101) may determine, as the first content, a photo (460) that corresponds to the keyword 'cookie' among the contents stored in the electronic device (101). For example, the electronic device (101) may determine, as the first content, the photo (460) as having the highest similarity to the query (e.g., a cookie wearing a floral dress, a cookie in a shape similar to the image 440). For example, since a picture of a cookie wearing a floral dress is not actually stored in the electronic device (101), the electronic device (101) may determine the picture (460), which is a picture of a cookie, as the first content. The electronic device (101) may determine the most suitable content for creating new content among the stored contents as the first content. For example, the electronic device (101) may determine the most suitable picture for creating a picture of a 'cookie wearing a floral dress' among the stored pictures of cookies (e.g., a picture of a cookie in a pose suitable for dressing the cookie) as the first content. The electronic device (101) may determine the similarity between a message (e.g., the message (420)) and the first content (e.g., the picture (460)) to determine whether to create an image by utilizing an artificial intelligence model (e.g., the generative model (250) of FIG. 2). The electronic device (101) may generate a prompt to generate content in response to a similarity between a message (e.g., message (420)) and a first content (e.g., photo (460)) being lower than a predetermined threshold.For example, the electronic device (101) may generate a prompt in response to the search not finding content corresponding to the query 'cookies wearing flower patterns' generated based on the message (420) (e.g., having a similarity higher than a predetermined threshold). The electronic device (101) may generate the content based on the prompt, which includes a first prompt portion corresponding to the message (e.g., message (420)) and a second prompt portion corresponding to the first content (e.g., photo (460)). The message corresponding to the first prompt portion may include text and / or content (e.g., content such as a photo, a video, a file, or a web link). For example, at least one image data received from a conversation partner (e.g., photo (440)), the first content (e.g., photo (460)), and / or at least one additional image acquired from an external source (e.g., a web browser) may be used to generate the first prompt portion. For example, a photo (440) of a puppy outfit received from a conversation partner can be used to generate a first prompt portion together with a message (420). The electronic device (101) can input the prompt into a generative model (e.g., the generative model (250) of FIG. 2) to generate content. For example, the electronic device (101) can input a prompt including a first prompt portion corresponding to the message (420), the photo (440), and a second prompt portion corresponding to the photo (460) into the generative model (250) to generate a photo (470). The photo (470) can be a photo of the user's puppy, 'Cookie', wearing the 'flowery outfit' included in the photo (440).

[0091] According to one embodiment, the electronic device (101) may input a prompt into a generative model (e.g., the generative model (250) of FIG. 2) to generate a plurality of contents. For example, the electronic device (101) may generate a plurality of pictures of the user's dog, 'Cookie', wearing a 'flower-patterned outfit' included in the picture (440) and striking different poses. The electronic device (101) may display the generated plurality of contents on a screen (e.g., the screen (403)). For example, the electronic device (101) may display a plurality of pictures of 'Cookie wearing a flower-patterned outfit' in an area of ​​the screen (403) where a virtual keyboard is located. The electronic device (101) may determine a content corresponding to the user's selection input among the plurality of contents as the second content to be transmitted to the other party and / or the server. The electronic device (101) may determine the picture (470) as the second content to be transmitted to the other party and / or the server. The electronic device (101) can display second content (e.g., a photo (470)) on a screen (e.g., a screen (403)). For example, the electronic device (101) can display the photo (470) in an area where a virtual keyboard is located. The electronic device (101) can display a graphic object (e.g., a graphic object (475)) indicating that the second content (e.g., the photo (470)) is content generated using a generative model, together with the second content (e.g., the photo (470)).

[0092]

[0093] Referring to FIG. 4B, according to one embodiment, the electronic device (101) can display a screen (404) on a display (e.g., the display module (160) of FIG. 1). The screen (404) can include second content (e.g., a photo (470)) to be transmitted to a counterpart and / or a server. The second content (e.g., the photo (470)) can be content generated by an artificial intelligence model (e.g., the generative model (250) of FIG. 2). The electronic device (101) can display on the screen (404) an indication of what the second content (e.g., the photo (470)) was generated on together with the second content (e.g., the photo (470)). For example, the electronic device (101) can generate an interface (465) that includes keywords and / or images (e.g., picture of floral clothes, picture of cookies, 'cookies', 'dog clothes') used to generate the picture (470), and display them together with the picture (470). In response to a user input, the electronic device (101) can transmit the second content (e.g., picture (470)) to the other party and / or server. The electronic device (101) can display the second content (e.g., picture (470)) in a portion where a message (e.g., message (420)) is displayed. The user can additionally modify and / or edit the second content (e.g., picture (470)).

[0094]

[0095] FIG. 5 is a diagram for explaining an operation of processing content according to one embodiment.

[0096] Referring to FIG. 5, according to one embodiment, the electronic device (101) can display screens (501) to (503) through a display module (e.g., the display module (160) of FIG. 1). The screens (501) to (503) can include screens for applications running on the electronic device (101). For example, the screens (501) to (503) can include screens for applications for a user (e.g., a user of the electronic device (101)) to interact (e.g., conduct a conversation) with another user. The electronic device (101) can detect a message associated with the transmission of content from text obtained from an application running on the electronic device (101). The electronic device (101) can obtain text from the running application. For example, the electronic device (101) can recognize text (e.g., message (520)) included in the screen (501) through optical character recognition (OCR) technology. For example, the electronic device (101) can receive text directly from an application running on the electronic device (101). For example, the electronic device (101) can obtain text by converting a voice conversation (e.g., voice call) between users into text. The electronic device (101) can obtain text from a screen (e.g., screen (501)) on which a running application is displayed. For example, the electronic device (101) can obtain the text message (520) “I’m trying to book a flight, please send me your passport information” included in the screen (501) on which a chat application is displayed. The electronic device (101) can detect a message related to the transmission of content from the text (e.g., message (520)). Messages associated with the transmission of content may include a message from a user containing an intent to transmit content and / or a message from a counterparty containing an intent to request transmission of content to the user.The electronic device (101) can utilize an artificial intelligence model (e.g., a context analysis model) to detect messages associated with content transmission. The operation of the electronic device (101) using the artificial intelligence model to detect messages associated with content transmission from text is substantially the same as the operation described with reference to FIGS. 1 to 4B , and thus, redundant descriptions thereof are omitted. The electronic device (101) can detect messages associated with content transmission from text. For example, the electronic device (101) can detect the other party's intention to request "passport information" from the message (520) "I'm trying to book a flight, so please send me my passport information," and can detect the message (520) as a message associated with content transmission.

[0097] According to one embodiment, the electronic device (101) may, in response to detecting a message associated with the transmission of content from text, display a graphic object together with the message to emphasize the message. For example, the electronic device (101) may display a graphic object (530) together with the message (520) to emphasize the keyword 'passport information' included in the message (520). To display the graphic object together with the message (e.g., the message (520)) on a screen (e.g., the screen (502)), the electronic device (101) may display the graphic object in the application itself, or may add a layer to the screen (e.g., the screen (502)) and display the graphic object on the added layer. By adding a layer to the screen (e.g., the screen (502)) and displaying the graphic object on the added layer, the electronic device (101) may freely display the graphic object even in a third-party application.

[0098] According to one embodiment, the electronic device (101) may generate a query to search for content related to the message (e.g., message 520) among content (e.g., gallery content) stored in the electronic device (101) based on a message (e.g., message 520). The electronic device (101) may extract a keyword for generating the query from the message (e.g., message 520). The keyword may include a keyword corresponding to at least one of context information, location information, and object information. For example, the electronic device (101) may extract context information requesting to send 'passport information', which is object information, from the message (520). The electronic device (101) may generate a query based on the message (520). The electronic device (101) may obtain first content related to the message (520) based on the query. Content stored in the electronic device (101) can be converted into a vector format and stored in a database (DB) (e.g., DB (615) of FIG. 6). The electronic device (101) can convert the content converted into a vector format into a language vector (e.g., LangChain) format and store the converted content in the database (e.g., DB (615) of FIG. 6). The content converted into a language vector format and stored can be input into an artificial intelligence model (e.g., LLM) and searched by a query. The electronic device (101) can search for first content associated with a message (e.g., message (520)) among the contents stored in the database (e.g., database (615) of FIG. 6) using the query. The operation of the electronic device (101) searching for first content associated with a message (e.g., message associated with transmission of content) among the contents stored in the electronic device (101) based on the query is substantially the same as that described with reference to FIGS. 3 to 4B, and any redundant description thereof will be omitted.The electronic device (101) may search for content (e.g., a photo (560)) associated with a message (520) among the content stored in the electronic device (101) and determine the content as the first content. For example, the electronic device (101) may determine a photo (560) that is a photo taken of a user's passport as the first content associated with a message (520) regarding 'passport information'. The electronic device (101) may determine the second content to be transmitted from the application based on the similarity between the message (e.g., the message (520)) and the first content (e.g., the photo (560)). The second content may be the first content or content generated and / or processed based on the message and the first content.

[0099] According to one embodiment, the electronic device (101) may modify a portion corresponding to personal information within an image (e.g., a photograph (560)) based on at least a portion of a determination that an image (e.g., a photograph (560)) from among a plurality of images stored in a memory (e.g., a memory (130) of FIG. 1) contains personal information. For example, the electronic device (101) may mask and obscure (575) the personal information (565) in response to the determination that the photo (560) determined as the first content contains personal information (565) corresponding to a resident registration number. Personal information may include sensitive information that may cause damage to the user if leaked externally. For example, personal information may include sensitive information such as a resident registration number, a credit card number, or a passport number. The electronic device (101) may use an artificial intelligence model to detect a portion corresponding to personal information (e.g., the personal information (565)) within the first content (e.g., the photograph (560)) and perform processing on the portion. For example, the electronic device (101) can detect a portion of personal information (565) that requires processing from a photo (560) using an artificial intelligence model. The electronic device (101) can use a message (e.g., message (520)) and a first content (e.g., photo (560)) as inputs to the artificial intelligence model to detect a portion of personal information within the first content (e.g., photo (560)). For example, not only the resident registration number included in the photo (560) but also the passport number can be classified as personal information that requires protection, but the artificial intelligence model can detect only the resident registration number as personal information (e.g., personal information (565)) that requires processing, excluding the 'passport information' that the other party requires, based on the content of the message (520). The electronic device (101) can determine a different level of processing to be performed on a portion corresponding to personal information (e.g., personal information (565)) based on the message (e.g., message (520)).For example, the electronic device (101) may determine the level of processing (e.g., degree of masking) to be performed on a portion corresponding to personal information (565) based on a conversation between users including a message (520). The electronic device (101) may display content (e.g., photo (570)) in which personal information (e.g., personal information (565)) has been obscured (e.g., obscured (575)) in a second content (e.g., photo (560)) as second content to be transmitted to the other party and / or a server. For example, the electronic device (101) may display a photo (570) in which sensitive information, such as a resident registration number (e.g., personal information (565)), has been obscured (575) in a photo (560) including passport information as second content to be transmitted to the other party. It should be noted that the obscuring that the electronic device (101) may perform to protect personal information is not limited to masking. For example, the electronic device (101) may perform processing such as mosaic processing or cropping on personal information to obscure the personal information. The electronic device (101) may display content (570) on the screen (503).

[0100]

[0101] FIG. 6 illustrates a system for providing content according to one embodiment.

[0102] Referring to FIG. 6, according to one embodiment, a content providing system (600) may include one or more components (601 to 615). The components (601 to 615) may be software modules implemented on an electronic device (e.g., the electronic device (101) of FIG. 1) or a server (e.g., the server (108) of FIG. 1). The components (601 to 615) are illustrated as an example for describing the content providing system (600). Accordingly, the content providing system (600) may include various variations of the components (601 to 615) as long as the operations of the content providing system (600) described in the present disclosure can be implemented. For example, two or more components may be combined, or one or more components may be added or omitted. Alternatively, the content provision system (600) may further include one or more components (e.g., components (210 to 250) of FIG. 2) of a generative artificial intelligence system (e.g., generative artificial intelligence system (200) of FIG. 2).

[0103] According to one embodiment, the content providing system (600) may include a text analyzing module (601), a query generating module (603), a search engine (605), a content analyzing module (607), a content generating module (609), a content processing module (611), and a database (615). The content providing system (600) may receive text (620) and, based on the text (620), output second content (670). The text (620) may be obtained from an application running on the electronic device (101). For example, the text (620) may include text recognized from a screen displayed on the electronic device (101) through OCR technology, text received directly from an application running on the electronic device (101), or text obtained by converting voice into text.

[0104] According to one embodiment, the text analysis module (601) can analyze the text (620) to detect a message associated with the transmission of content. The message associated with the transmission of content may include a message from a user containing an intention to transmit content and / or a message from a counterpart containing an intention to request the transmission of content to the user. The text analysis module (601) can analyze the text (620) in real time or analyze the text (620) at a set interval (e.g., n seconds) to detect a message associated with the transmission of content. The text analysis module (601) can analyze the text (620) based on text and / or content (e.g., content such as photos, videos, and files) received before and after the text (620). For example, the text analysis module (601) can analyze not only the text (620) but also content transmitted and / or received before and after the text (620) to detect a message associated with the transmission of content. The text analysis module (601) may utilize an artificial intelligence model (e.g., a context analysis model) to detect messages associated with the transmission of content from text (620). The artificial intelligence model may include a model trained to analyze the context of messages (e.g., a large language model (LLM)). For example, the context analysis model may be trained to analyze the structure and context of language using a dataset containing large-scale text data, such as conversation logs and web texts, for natural language processing (NLP) and natural language understanding (NLU). The text analysis module (601) may transmit messages associated with the transmission of content to the query generation module (603).

[0105] According to one embodiment, the query generation module (603) may generate a query to search for content related to the message (e.g., the message related to the transmission of content) among the content stored in the electronic device (101) based on the message (e.g., the message related to the transmission of content). The query generation module (603) may generate a query based on the message (e.g., the message related to the transmission of content) to search for content (e.g., content such as a photo, a video, or a file) stored in the electronic device (101). The query generation module (603) may extract a keyword from the message (e.g., the message related to the transmission of content). The keyword may include a keyword corresponding to at least one of context information, location information, and object information. The context information may include information regarding the flow of an interaction (e.g., a conversation) that includes the message (e.g., the message related to the transmission of content). For example, the context information may include information such as a topic or purpose of the conversation. For example, context information may include keywords included in interactions (e.g., conversations) before and after a message associated with the transmission of content. Object information may include object information personalized to the user. For example, object information may include information (e.g., categories) that classify objects included in a message (e.g., messages associated with the transmission of content), as well as information (e.g., names, nicknames, etc.) about objects that are specific people related to the user. The query generation module (603) may generate a query based on keywords. The query generation module (603) may generate a query based on keywords extracted from messages associated with the transmission of content. The query generation module (603) may generate a query based on keywords extracted from conversations before and after a message associated with the transmission of content. The query generation module (603) may generate a query based on the context of a conversation that includes a message associated with the transmission of content.For example, the query generation module (603) may generate a query in the form of a phrase that includes contextual information analyzed from the conversation content. The query generation module (603) may transmit the query to a search engine (605). The query may be used by the search engine (605) to search for content stored in the electronic device (101).

[0106] According to one embodiment, the search engine (605) may search for first content associated with a message (e.g., a message related to the transmission of content) among the content stored in the electronic device (101) based on a query. The content stored in the electronic device (101) may be converted into a vector format and stored in a database (DB) (615). The electronic device (101) may convert the content converted into a vector format into a language vector (e.g., LangChain) format and store the converted content in the DB (615). The content converted into a language vector format and stored may be input into an artificial intelligence model (e.g., LLM) and searched by a query. For example, the search engine (605) may search for the first content from the DB (615) using a retrieval-augmented generation (RAG) method that searches for related information using a query. The search engine (605) may determine the content, in which an attribute of the content corresponds to a keyword (e.g., a keyword included in the query), as the first content. The properties of the content can be determined based on metadata (e.g., metadata such as EXIF ​​(exchangeable image file format), XMP (extensible metadata platform from Adobe), IPTC (international press telecommunications council)) and / or tag information (e.g., tag information such as person tag information, place tag information, keyword tag information, and event tag information) for each piece of content. For example, the properties regarding the location of the content can be determined based on place tag information (e.g., global positioning system (GPS) coordinates, altitude, location information on a map) included in the content and / or information identifying objects (e.g., landscape, landmark) included in the content.For example, attributes of an object included in the content may be determined based on person tag information (e.g., object information such as a name, nickname, and relationship with a user) included in the content. The person tag information may include information classified by the electronic device (101) by recognizing the face of a person within the content. The search engine (605) may search for and acquire first content associated with a message (e.g., a message associated with the transmission of the content) among the stored content by considering all information included in the query. The search engine (605) may transmit the first content to the content analysis module (607).

[0107] According to one embodiment, the content analysis module (607) may determine whether to generate new content based on the similarity between a message (e.g., a message associated with the transmission of content) and first content. The content analysis module (607) may determine whether to generate new content based on contextual information of the message (e.g., a message associated with the transmission of content). For example, the content analysis module (607) may determine not to generate new content for a message that requires transmission of the stored content itself, such as "Send me the picture you took last time." If the similarity between the first content searched for based on a query among the contents stored in the electronic device (101) and the message (e.g., a message associated with the transmission of content) is lower than a predetermined threshold, the first content may not match the user's intention (e.g., content that the user wishes to transmit), and the content analysis module (607) may determine that new content needs to be generated. If the similarity between the searched first content and the message (e.g., a message related to the transmission of the content) is higher than a predetermined threshold, the content analysis module (607) may determine the first content as the second content to be transmitted, as transmitting the first content is in line with the user's intention. In response to determining that there is a need to generate content, the content analysis module (607) may transmit the first content to the content generation module (609). If the content analysis module (607) determines the first content as the second content (e.g., determines not to generate new content), the content analysis module (607) may transmit the first content to the content processing module (611).

[0108] According to one embodiment, the content generation module (609) can generate content based on a message (e.g., a message associated with the transmission of content) and first content. The content generation module (609) can generate content using a generative model (e.g., the generative model (250) of FIG. 2) and determine the generated content as second content to be transmitted. The content generation module (609) can generate a prompt including a first prompt portion corresponding to the message (e.g., a message associated with the transmission of content) and a second prompt portion corresponding to the first content. The message corresponding to the first prompt portion can include text and / or content (e.g., content such as a photo, a video, a file, or a web link). For example, the content generation module (609) can use a photo included in a conversation before or after a message to generate the prompt. For example, the content generation module (609) can generate the prompt using at least one image data received from a conversation partner, the first content, and / or at least one additional image acquired from an external source (e.g., a web browser). The content generation module (609) can generate content using an externally generated prompt or can generate content by directly generating a prompt. The generative model that the content generation module (609) can use to generate content can be located inside and / or outside the electronic device (600). The content generation module (609) can generate content by inputting a prompt into the generative model (e.g., the generative model (250) of FIG. 2). The content generation module (609) can transmit the generated content to the content processing module (611).

[0109] According to one embodiment, the content processing module (611) may process content received from the content analysis module (607) (e.g., first content associated with a message among the contents stored in the electronic device (101)) or generated content received from the content generation module (609) to output second content (670) to be transmitted. The content processing module (611) may mask personal information in response to the content determined as the second content including personal information. The content processing module (611) may utilize an artificial intelligence model to perform processing on the personal information. The artificial intelligence model that may be used to perform processing on the personal information may be located within the electronic device (600). The content processing module (611) may output (e.g., display) the second content (670) on which processing (e.g., masking processing, obscuring processing) has been performed on the personal information. The operation performed by the content processing module (611) may be substantially the same as the operation of processing content by the electronic device (101) described with reference to FIG. 5.

[0110]

[0111] Figure 7 is a flowchart of a method of operating an electronic device according to one embodiment.

[0112] Referring to FIG. 7, according to one embodiment, operations 710 to 790 may be substantially identical to operations performed by the electronic device (e.g., the electronic device (101) of FIG. 1) described with reference to FIGS. 1 to 6. Therefore, redundant descriptions will be omitted.

[0113] According to one embodiment, operations 710 to 790 may be understood to be performed in a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0114] In operation 710, the electronic device (101) may detect a message associated with the transmission of content from text obtained from an application running on the electronic device (101). The electronic device (101) may obtain the text by recognizing text included on the screen, receiving text directly from an application running on the electronic device (101), or converting a voice conversation between users into text. It should be noted that the operation of the electronic device (101) obtaining the text is not limited thereto. The electronic device (101) may utilize an artificial intelligence model (e.g., a context analysis model) to detect a message associated with the transmission of content from the text. The artificial intelligence model may include a model (e.g., a large language model (LLM)) trained for context analysis of a message.

[0115] In operation 730, the electronic device (101) may generate a query for searching for content related to the message among content stored in the electronic device (101) based on the message. The electronic device (101) may extract a keyword for generating the query from the message (e.g., a message related to transmission of the content) and generate the query based on the keyword. For example, the electronic device (101) may extract a keyword corresponding to at least one of context information, location information, and object information from the message related to transmission of the content and generate the query.

[0116] In operation 750, the electronic device (101) may obtain first content associated with the message based on the query. The electronic device (101) may obtain first content associated with the message from among content stored in the electronic device (101) based on the query. For example, the electronic device (101) may perform a search using the query on content converted and stored in a vector format, thereby obtaining the first content.

[0117] In operation 770, the electronic device (101) may determine second content to be transmitted from the application based on the similarity between the message and the first content. The electronic device (101) may determine whether the first content acquired from among the contents stored in the electronic device (101) has a similarity exceeding a predetermined threshold with the message (e.g., a message associated with the transmission of the content). In response to determining that the similarity between the message and the first content exceeds the predetermined threshold, the electronic device (101) may determine the first content as the second content to be transmitted. In response to determining that the similarity between the message and the first content does not exceed the predetermined threshold, the electronic device (101) may generate new content and determine the generated content as the second content.

[0118] In operation 790, the electronic device (101) may display second content. The electronic device (101) may display the second content through a display (e.g., the display module (160) of FIG. 1). For example, the electronic device (101) may display the second content in an area where a virtual keyboard (e.g., the virtual keyboard (350) of FIG. 3) was displayed, and provide the second content to the user.

[0119] In one embodiment, operations 710 through 790 may be performed sequentially, but are not limited thereto. For example, two or more operations may be performed in parallel.

[0120]

[0121] Figure 8 is a flowchart of a method of operating an electronic device according to one embodiment.

[0122] Referring to FIG. 8, according to one embodiment, operations 810 to 890 may be substantially identical to operations performed by the electronic device (e.g., the electronic device (101) of FIG. 1) described with reference to FIGS. 1 to 6. Therefore, redundant descriptions will be omitted.

[0123] According to one embodiment, operations 810 to 890 may be understood to be performed in a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0124] In operation 810, the electronic device (101) can detect a message associated with the transmission of content from text obtained from an application running on the electronic device (101). The electronic device (101) can detect a message associated with the transmission of content by analyzing the context of an interaction (e.g., a conversation) that includes the text obtained from the application running on the electronic device (101).

[0125] In operation 820, the electronic device (101) may generate a query to search for content related to the message among the content stored in the electronic device (101) based on the message. The electronic device (101) may extract keywords for generating the query from the message (e.g., a message related to the transmission of the content). For example, the electronic device (101) may generate the query by extracting keywords included in conversations before and after the message related to the transmission of the content. The keywords may correspond to at least one of context information, location information, and object information.

[0126] In operation 830, the electronic device (101) may obtain first content associated with the message based on the query. The electronic device (101) may obtain first content associated with the message from among content stored in the electronic device (101) based on the query. For example, the electronic device (101) may perform a search using the query on content converted and stored in a vector format, thereby obtaining the first content.

[0127] In operation 840, the electronic device (101) may determine whether the similarity between the message and the first content exceeds a predetermined threshold. The electronic device (101) may determine whether there is a need to generate new content by determining whether the similarity between the message and the first content exceeds the predetermined threshold. In response to determining that the similarity between the message and the first content exceeds the predetermined threshold, the electronic device (101) may perform operation 880. In response to determining that the similarity between the message and the first content does not exceed the predetermined threshold, the electronic device (101) may perform operation 850.

[0128] In operation 850, the electronic device (101) may generate a prompt. The electronic device (101) may generate the prompt including a first prompt portion corresponding to a message (e.g., a message associated with transmission of content) and a second prompt portion corresponding to the first content. The message corresponding to the first prompt portion may include text and / or content (e.g., content such as a photo, a video, a file, or a web link). For example, at least one image data received from a conversation partner, an image searched and / or generated in connection with the message, and / or at least one additional image acquired from an external source (e.g., a web browser) may be used to generate the first prompt portion.

[0129] In operation 860, the electronic device (101) can generate content based on the prompt. The electronic device (101) can generate content by inputting the prompt into a generative model (e.g., the generative model (250) of FIG. 2).

[0130] In operation 870, the electronic device (101) may determine the generated content as second content. The electronic device (101) may determine the generated content using a generative model (e.g., the generative model (250) of FIG. 2) as second content to be transmitted from the application.

[0131] In operation 880, the electronic device (101) may determine the first content as the second content. The electronic device (101) may determine the first content, which is found to be associated with a message among the contents stored in the electronic device (101), as the second content to be transmitted from the application.

[0132] In operation 890, the electronic device (101) may display second content. The electronic device (101) may display the second content through a display (e.g., the display module (160) of FIG. 1). For example, the electronic device (101) may display the second content in an area where a virtual keyboard (e.g., the virtual keyboard (350) of FIG. 3) is displayed, and provide the second content to the user.

[0133] In one embodiment, operations 810 through 890 may be performed sequentially, but are not limited thereto. For example, two or more operations may be performed in parallel.

[0134]

[0135] Figure 9 is a flowchart of a method of operating an electronic device according to one embodiment.

[0136] Referring to FIG. 9, according to one embodiment, operations 910 to 990 may be substantially identical to operations performed by the electronic device (e.g., the electronic device (101) of FIG. 1) described with reference to FIGS. 1 to 8. Therefore, any redundant description will be omitted.

[0137] According to one embodiment, operations 910 to 990 may be understood to be performed in a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0138] In operation 910, the electronic device (101) may display a message (e.g., a message (320) of FIG. 3A) on a first portion via a touchscreen display (e.g., a display module (160) of FIG. 1). The message (e.g., a message (320) of FIG. 3A) may be included in an interaction (e.g., a conversation) between multiple users.

[0139] At operation 930, the electronic device (101) may identify a plurality of images stored in the electronic device (101) based at least in part on a determination that a message (e.g., message (320) of FIG. 3A) includes information associated with an image stored in the electronic device (101). The electronic device (101) may identify a plurality of images stored in a memory (e.g., memory (130) of FIG. 1) based at least in part on a determination that the message (e.g., message (320) of FIG. 3A) includes information corresponding to at least one of context information, location information, and object information associated with an image stored in a memory (e.g., memory (130) of FIG. 1) of the electronic device (101). The electronic device (101) may identify (e.g., search for) at least one image associated with the message (e.g., message (320) of FIG. 3A) among the plurality of images stored in the memory (e.g., memory (130) of FIG. 1).

[0140] In operation 950, the electronic device (101) can display a plurality of images on a second portion different from a first portion through a touchscreen display (e.g., a display module (160) of FIG. 1). The electronic device (101) can display a plurality of images (e.g., a photo (370) of FIG. 3A) on a second portion different from a first portion where a message (e.g., a message (320) of FIG. 3A) is displayed (e.g., a portion where a virtual keyboard (350) of FIG. 3A is located).

[0141] In operation 970, the electronic device (101) may receive a user input to select at least one image from a plurality of images (e.g., a photograph (370) of FIG. 3A) via a touchscreen display (e.g., a display module (160) of FIG. 1).

[0142] In operation 990, the electronic device (101) may display at least one image on a first portion via a touchscreen display (e.g., display module (160) of FIG. 1). The electronic device (101) may display at least one image corresponding to a user input on the first portion where a message (e.g., message (320) of FIG. 3A) is being displayed.

[0143] In one embodiment, operations 910 through 990 may be performed sequentially, but are not limited thereto. For example, two or more operations may be performed in parallel.

[0144]

[0145] An electronic device (101) according to one embodiment may include at least one processor (120) including a processing circuit. The electronic device (101) may include a memory (130) that stores instructions. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to detect a message associated with the transmission of content from a text obtained from an application running on the electronic device (101). The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to generate a query for searching for content associated with the message among content stored on the electronic device (101) based on the message. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to obtain first content associated with the message based on the query. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine second content to be transmitted from the application based on a similarity between the message and the first content. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the second content.

[0146] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to extract a keyword for generating the query from the message. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine, as the first content, a content stored in the electronic device (101), the content of which properties correspond to the keyword. The keyword may include a keyword corresponding to at least one of context information, location information, and object information.

[0147] In one embodiment, the second content may be content generated based on a prompt that includes a first prompt portion corresponding to the first content or the message and a second prompt portion corresponding to the first content.

[0148] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to generate content based on the prompt in response to a similarity between the message and the first content being lower than a predetermined threshold. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine the generated content as the second content.

[0149] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine the first content as the second content in response to a similarity between the message and the first content being higher than a predetermined threshold value.

[0150] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the second content by masking the privacy information in response to the second content including the privacy information.

[0151] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to, in response to detecting the message, display a graphical object for emphasizing the message together with the message.

[0152] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a user interface associated with the second content in response to determining the second content.

[0153] In one embodiment, the message may include a message containing an intent to transmit content, a message requesting transmission of content, and a message associated with the creation of content.

[0154] A method of operating an electronic device (101) according to one embodiment may include an operation of detecting a message associated with transmission of content from a text obtained from an application running on the electronic device (101). The method may include an operation of generating a query for searching for content associated with the message among contents stored in the electronic device (101) based on the message. The method may include an operation of obtaining first content associated with the message based on the query. The method may include an operation of determining second content to be transmitted from the application based on a similarity between the message and the first content. The method may include an operation of displaying the second content.

[0155] According to one embodiment, the operating method may include an operation of extracting a keyword for generating the query from the message. The operating method may include an operation of determining, as the first content, a content whose properties correspond to the keyword among the content stored in the electronic device (101). The keyword may include a keyword corresponding to at least one of context information, location information, and object information.

[0156] In one embodiment, the second content may be content generated based on a prompt that includes a first prompt portion corresponding to the first content or the message and a second prompt portion corresponding to the first content.

[0157] In one embodiment, the method may include generating content based on the prompt in response to a similarity between the message and the first content being lower than a predetermined threshold. The method may further include determining the generated content as the second content.

[0158] According to one embodiment, the method may include an operation of determining the first content as the second content in response to a similarity between the message and the first content being higher than a predetermined threshold value.

[0159] According to one embodiment, the method of operation may include, in response to the second content including privacy information, an operation of masking and displaying the privacy information in the second content.

[0160] According to one embodiment, the method may include, in response to detecting the message, displaying a graphical object together with the message to emphasize the message.

[0161] According to one embodiment, the method of operation may include, in response to determining the second content, displaying a user interface associated with the second content.

[0162] In one embodiment, the message may include a message containing an intent to transmit content, a message requesting transmission of content, and a message associated with the creation of content.

[0163] According to one embodiment, a computer-readable recording medium storing one or more computer programs may include instructions for performing the method in a processor.

[0164] An electronic device (101) according to one embodiment may include a touchscreen display (160). The electronic device (101) may include a communication circuit (190). The electronic device (101) may include at least one processor (120) including a processing circuit. The electronic device (101) may include a memory (130) that stores instructions. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a message on a first portion through the touchscreen display. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to identify a plurality of images stored in the memory (130) based on at least a portion of the information, based on at least a portion of a determination that the message includes information associated with an image stored in the memory (130). The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the plurality of images on a second portion, different from the first portion, via the touchscreen display (160). The above instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to receive, through the touchscreen display (160), a user input for selecting at least one image from the plurality of images.The above instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the at least one image on the first portion through the touchscreen display (160).

[0165] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a graphical object or effect in the message based on at least a portion of the determination.

[0166] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to receive, through the touchscreen display (160), another user input regarding the message. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to perform identification of the plurality of images based on at least a portion of the another user input.

[0167] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a virtual keyboard in the second portion as at least a part of displaying the message. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to perform display of the plurality of images after refraining from displaying at least a part of the virtual keyboard in the second portion.

[0168] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to identify the plurality of images and then display another graphical object on at least a portion of the virtual keyboard. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to receive another user input regarding the another graphical object through the touchscreen display (160). The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the plurality of images through the touchscreen display (160) based on at least a portion of the another user input.

[0169] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to generate a keyword based on at least a portion of the information as at least a part of identifying the plurality of images. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to perform identification of the plurality of images based on the keyword. The keyword may include other information corresponding to at least one of context information, location information, and object information.

[0170] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display an image different from the message on the first portion, at least as a part of displaying the message. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to generate an artificial intelligence image using a machine learning model based on the at least one image and at least a part of the other image. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the artificial intelligence image on the second portion, via the touchscreen display (160). The machine learning model may be trained to generate an image based on at least one of text input, image input, or drawing input.

[0171] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to modify a portion of the image corresponding to the personal information based on at least a portion of what is determined to be an image from the plurality of images to include the personal information. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the modified image on the second portion via the touchscreen display (160).

[0172] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to generate a prompt based on the at least one image and the other image, at least as part of generating the artificial intelligence image. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to perform generation of the artificial intelligence image based on the prompt, the at least one image, and the other image. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display, in the second portion, at least a portion of the prompt, at least a portion of the at least one image, and at least a portion of the other image together with the artificial intelligence image, as at least a part of displaying the artificial intelligence image.

[0173] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to receive the message from an external electronic device via the communication circuitry, at least as part of displaying the message. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to transmit the at least one image to the external electronic device via the communication circuitry (190), at least as part of displaying the image.

[0174] A method of operating an electronic device (101) according to one embodiment may include an operation of displaying a message on a first portion via a touchscreen display (160). The method may include an operation of identifying a plurality of images stored in the electronic device (101) based on at least a portion of a determination that the message includes information associated with an image stored in the electronic device (101). The method may include an operation of displaying the plurality of images on a second portion different from the first portion via the touchscreen display (160). The method may include an operation of receiving a user input for selecting at least one image from the plurality of images via the touchscreen display (160). The method may include an operation of displaying the at least one image on the first portion via the touchscreen display (160).

[0175] According to one embodiment, the method may further include displaying a graphical object or effect in the message based at least in part on the determination.

[0176] In one embodiment, the method may further include receiving another user input regarding the message via the touchscreen display (160). The method may further include performing an identification of the plurality of images based on at least a portion of the other user input.

[0177] According to one embodiment, the method of operation may further include, as at least a part of the operation of displaying the message, an operation of displaying a virtual keyboard in the second portion. The method of operation may further include, after refraining from displaying at least a part of the virtual keyboard in the second portion, an operation of displaying the plurality of images.

[0178] According to one embodiment, the operating method may further include an operation of displaying another graphic object on at least a portion of the virtual keyboard after identifying the plurality of images. The operating method may further include an operation of receiving another user input regarding the another graphic object through the touchscreen display (160). The operating method may further include an operation of performing a display of the plurality of images through the touchscreen display (160) based on at least a portion of the another user input.

[0179] According to one embodiment, the method may further include, as at least part of identifying the plurality of images, generating a keyword based on at least a portion of the information. The method may further include performing identification of the plurality of images based on the keyword. The keyword may include other information corresponding to at least one of context information, location information, and object information.

[0180] According to one embodiment, the method of operation may further include, as at least a part of displaying the message, an operation of displaying an image different from the message in the first portion. The method of operation may further include an operation of generating an artificial intelligence image using a machine learning model based on the at least one image and at least a part of the other image. The method of operation may further include an operation of displaying the artificial intelligence image in the second portion through the touchscreen display (160). The machine learning model may be trained to generate an image based on at least one of a text input, an image input, or a drawing input.

[0181] In one embodiment, the method may further include modifying a portion of the image corresponding to the personal information based on at least a portion of the image determined to contain the personal information from the plurality of images. The method may further include displaying the modified image on the second portion via the touchscreen display (160).

[0182] In one embodiment, the method may further comprise, as at least a part of generating the artificial intelligence image, an action of generating a prompt based on the at least one image and the other image. The method may further comprise, as at least a part of displaying the artificial intelligence image, an action of performing generation of the artificial intelligence image based on the prompt, the at least one image, and the other image. The method may further comprise, as at least a part of displaying the artificial intelligence image, an action of displaying, on the second portion, at least a portion of the prompt, at least a portion of the at least one image, and at least a portion of the other image together with the artificial intelligence image.

[0183] According to one embodiment, the method of operation may further include, as at least a part of displaying the message, receiving the message from an external electronic device via the communication circuit (190). The method of operation may further include, as at least a part of displaying the image, transmitting the at least one image to the external electronic device via the communication circuit (190).

[0184] According to one embodiment, a computer-readable recording medium storing one or more computer programs may include instructions for performing the method in a processor.

[0185] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0186] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0187] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0188] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101) of FIG. 1). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0189] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0190] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101), Touchscreen display (160); Communication circuit (190); At least one processor (120) comprising processing circuitry; and Memory for storing instructions (130) Including, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Through the above touch screen display (160), a message is displayed in the first part, Based at least in part on a determination that the message includes information associated with an image stored in the memory (130), identifying a plurality of images stored in the memory based on at least in part on the information; Through the above touch screen display (160), the plurality of images are displayed in a second part different from the first part, Through the above touch screen display (160), a user input for selecting at least one image from the plurality of images is received, To display at least one image on the first part through the touch screen display (160), Electronic device (101).

2. In paragraph 1, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Based at least in part on the above decision, causing a graphic object or effect to be displayed in the message; Electronic device (101).

3. In any one of paragraphs 1 and 2, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Through the above touch screen display (160), other user input regarding the above message is received, To perform identification of said plurality of images based on at least a portion of said other user input, Electronic device (101).

4. In any one of paragraphs 1 to 3, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: As at least part of displaying the above message, displaying a virtual keyboard in the second portion; After refraining from displaying at least a part of the virtual keyboard in the second section, displaying the plurality of images is performed. Electronic device (101).

5. In any one of paragraphs 1 to 4, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: After identifying the plurality of images, displaying another graphic object on at least a part of the virtual keyboard, Through the above touch screen display (160), other user inputs regarding the other graphic objects are received, To cause the display of the plurality of images through the touchscreen display (160) based on at least a portion of the other user input. Electronic device (101).

6. In any one of paragraphs 1 to 5, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: As at least part of identifying said plurality of images, generating keywords based on at least part of said information, Based on the above keywords, identification of the plurality of images is performed, The above keywords are, Contains other information corresponding to at least one of context information, location information, and object information, Electronic device (101).

7. In any one of paragraphs 1 to 6, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: As at least part of displaying the above message, displaying an image different from the above message in the first part, Generating an artificial intelligence image using a machine learning model based on at least one image and at least a part of the other image, Through the above touch screen display (160), the artificial intelligence image is displayed on the second part, The above machine learning model is, Trained to generate an image based on at least one of text input, image input, or drawing input, Electronic device (101).

8. In any one of paragraphs 1 to 7, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Based on at least a portion of an image from the plurality of images determined to contain personal information, modifying a portion of the image corresponding to the personal information; To display the modified image on the second part through the above touch screen display (160), Electronic device (101).

9. In any one of paragraphs 1 to 8, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: As at least part of generating the artificial intelligence image, generating a prompt based on the at least one image and the other image, Based on the above prompt, the at least one image, and the other image, generating the artificial intelligence image, As at least part of displaying the artificial intelligence image, displaying at least a part of the prompt, at least a part of the at least one image, and at least a part of the other image in the second portion together with the artificial intelligence image. Electronic device (101).

10. In any one of paragraphs 1 to 9, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: As at least part of displaying the above message, receiving the message from an external electronic device through the communication circuit (190), As at least part of displaying said image, transmitting said at least one image to said external electronic device via said communication circuit (190). Electronic device (101).

11. In the operating method of an electronic device (101), An action of displaying a message in the first part via a touch screen display (160); An operation of identifying a plurality of images stored in the electronic device (101) based on at least a portion of a determination that the message includes information associated with an image stored in the electronic device (101); An operation of displaying the plurality of images in a second part different from the first part through the touch screen display (160); An operation of receiving a user input for selecting at least one image from the plurality of images through the touch screen display (160); and An operation of displaying at least one image on the first part through the touch screen display (160) including, How it works.

12. In paragraph 11, An action to display a graphic object or effect in the message based at least in part on the above decision. including more, How it works.

13. In any one of paragraphs 11 to 12, An operation of receiving another user input regarding the message through the touch screen display (160); and An operation of performing identification of the plurality of images based on at least a portion of the other user input. including more, How it works.

14. In any one of paragraphs 11 to 13, As at least part of the action of displaying the above message, the action of displaying a virtual keyboard in the second part; and An operation of displaying the plurality of images after refraining from displaying at least a part of the virtual keyboard in the second section. including more, How it works.

15. In any one of paragraphs 11 to 14, After identifying the plurality of images, an action of displaying another graphic object on at least a part of the virtual keyboard; An operation of receiving another user input regarding the other graphic object through the above touch screen display (160); and An operation of displaying the plurality of images through the touchscreen display based on at least a portion of the other user input. including more, How it works.

Images