Electronic apparatus, method, and storage medium for plug-in management of generative task

Abstract

According to an embodiment, an electronic apparatus may comprise: a display configured to display a screen; at least one processor including a control circuit; and a memory storing instructions. The instructions, when executed individually or collectively by the at least one processor, may cause the electronic apparatus to display a first screen corresponding to a first application. The instructions, when executed individually or collectively by the at least one processor, may cause the electronic apparatus to identify first information of the first application on the basis of a call to an artificial intelligence (AI) application while the first screen is being displayed. The instructions, when executed individually or collectively by the at least one processor, may cause the electronic apparatus to display a screen of the AI application in which a first plug-in corresponding to the first information is configured. The AI application may include a large language model (LLM) application and / or an AI assistant application of the electronic apparatus interworking with the LLM application. Various other embodiments are also possible.

Description

Electronic device, method, and storage medium for managing plug-ins of generative work

[0001] The present disclosure relates to an electronic device, method, and storage medium for managing plug-ins of a generative task.

[0002] Analytical artificial intelligence (AAI) models can perform data analysis and / or pattern recognition. In contrast, generative AI models can generate data or content corresponding to user input and provide the generated content. As the development of deep learning models used as generative AI models advances, the quality of data or content provided by generative AI models is also improving. The types of generation tasks provided by generative AI models (e.g., large multimodal model (LMM), large language model (LLM), large vision model (LVM)) may include, for example, text generation, image generation, code generation, speech generation, and / or video generation. Multiple plug-ins may be provided to supplement generative AI models. For example, the generation task by a generative AI model may be performed based on at least one of the multiple plug-ins.

[0003] The above information may be provided as background information to aid in understanding this document. None of the above is claimed to be prior art related to this document or can be used to determine prior art.

[0004] According to one embodiment, an electronic device may include a display configured to display a screen, at least one processor including a control circuit, and a memory storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display a first screen corresponding to a first application. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to check first information of the first application based on a call to an AI (artificial intelligence) application while the first screen is displayed. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display a screen of the AI ​​application in which a first plug-in corresponding to the first information is set. The AI ​​application may include a large language model (LLM) application and / or an AI assistant application of the electronic device that is linked with the LLM application.

[0005] According to one embodiment, a method of operating an electronic device may include an operation of displaying a first screen corresponding to a first application. The method may include an operation of checking first information of the first application based on a call to an AI (artificial intelligence) application while the first screen is displayed. The method may include an operation of displaying a screen of the AI ​​application in which a first plug-in corresponding to the first information is set. The AI ​​application may include an LLM (large language model) application and / or an AI assistant application of the electronic device that is linked with the LLM application.

[0006] According to one embodiment, a non-transitory computer-readable storage medium storing instructions may cause the instructions, when individually or collectively executed by at least one processor of an electronic device, to cause the electronic device to perform at least one operation. The at least one operation may include displaying a first screen corresponding to a first application. The at least one operation may include checking first information of the first application based on a call to an artificial intelligence (AI) application while the first screen is displayed. The at least one operation may include displaying a screen of the AI ​​application in which a first plug-in corresponding to the first information is set. The AI ​​application may include a large language model (LLM) application and / or an AI assistant application of the electronic device that is linked with the LLM application.

[0007] FIG. 1 is a block diagram of an electronic device within a network environment, according to one embodiment.

[0008] FIG. 2 is a drawing illustrating components including a framework of an electronic device according to one embodiment.

[0009] FIG. 3 is a block diagram of a system for generating content using a generative artificial intelligence model, according to one embodiment.

[0010] FIG. 4 is a diagram illustrating a plug-in of an electronic device according to one embodiment.

[0011] FIG. 5 is a diagram illustrating mapping of an application and a plug-in of an electronic device according to one embodiment.

[0012] FIG. 6 is a flowchart of a method of operating an electronic device according to one embodiment.

[0013] FIG. 7 is a diagram illustrating a call to an AI application while a first application is running, according to one embodiment.

[0014] FIG. 8 is a flowchart of a method of operating an electronic device according to one embodiment.

[0015] FIG. 9 is a diagram illustrating a plug-in corresponding to a function activated in an application according to one embodiment.

[0016] FIG. 10 is a flowchart of a method of operating an electronic device according to one embodiment.

[0017] FIG. 11 is a diagram illustrating a plug-in corresponding to context information of an application according to one embodiment.

[0018] FIG. 12 is a flowchart of a method of operating an electronic device according to one embodiment.

[0019] FIG. 13 is a diagram illustrating a plug-in corresponding to multiple applications according to one embodiment.

[0020] FIG. 14 is a flowchart of a method of operating an electronic device according to one embodiment.

[0021] FIG. 15 is a diagram illustrating a plug-in corresponding to a selected application according to one embodiment.

[0022] FIG. 16 is a flowchart of a method of operating an electronic device according to one embodiment.

[0023] Figure 17 is a diagram illustrating a recommendation of a plug-in according to one embodiment.

[0024] FIG. 18 is a diagram illustrating an activation operation of an AI application according to a state of an electronic device, according to one embodiment.

[0025] FIG. 19 is a flowchart of a method of operating an electronic device according to one embodiment.

[0026] FIG. 20A is a diagram illustrating a plug-in corresponding to a first application according to one embodiment.

[0027] FIG. 20b is a diagram illustrating a plug-in corresponding to a second application according to one embodiment.

[0028] FIG. 21A is a drawing illustrating a screen of an AI application corresponding to a first application according to one embodiment.

[0029] FIG. 21b is a drawing illustrating a screen of an AI application corresponding to a second application according to one embodiment.

[0030] FIG. 22 is a drawing illustrating an embodiment of a foldable electronic device according to one embodiment.

[0031] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0032] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to one embodiment. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with the electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0033] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0034] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where artificial intelligence is performed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0035] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0036] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0037] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0038] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0039] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0040] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0041] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0042] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0043] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0044] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0045] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0046] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0047] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0048] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0049] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0050] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0051] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0052] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0053] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0054] FIG. 2 is a drawing illustrating components including a framework of an electronic device according to one embodiment.

[0055] In one embodiment, the electronic device (108) may provide AI services in conjunction with a server (108). In one embodiment, the electronic device (108) may also provide AI services without being associated with a server (108).

[0056] According to one embodiment, the server (108) can transmit and / or receive data with the electronic device (101). The electronic device (101) is not limited to any device capable of transmitting and receiving data with the server (108), such as, for example, a computer or a mobile electronic device (e.g., a smart phone or a tablet PC). The server (108) may include and / or execute a frontend module, a prompt processing module, an AI model selection module, a UI generation module, and / or at least one AI model repository. At least one module may be executed by, or included as at least a part of, at least one processor included in the server (108), for example. At least some of the operations performed by at least one module in the present disclosure may be understood to be performed by, for example, at least one processor included in the server (108) and / or another entity under the control of the processor. For example, the electronic device (101) may provide the execution result of the AI ​​model in conjunction with the server (108), and this may be referred to as a non-standalone type. Meanwhile, depending on the implementation, the electronic device (101) alone may provide the execution result of the AI ​​model, and this may be referred to as a standalone type or an on-device type. The server (108) may include a cloud server, an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server, and may be implemented as one or more entities, and there is no limitation on the form of implementation.

[0057] The front-end module of the server (108) may perform at least one operation for data exchange with the electronic device (101), for example. For example, the front-end module of the server (108) may provide data for configuring a UI that can input user input in the electronic device (101). For example, the front-end module of the server (108) may be implemented as a web server, but is not limited thereto. The electronic device (101) may access the front-end module, for example, based on a URL entered in a web browsing application, or by executing an application, but the access method is not limited thereto. The front-end module may provide processing for a user request to at least one other module. The at least one other module may be referred to as a back-end module. The back-end module may provide a generated result corresponding to an input language-based command to the front-end module. The front-end module may provide the generated result corresponding to the language-based command to the electronic device (101).

[0058] For example, the prompt processing module of the server (108) may process a language-based command received from the electronic device (101) and provide the processing result to the AI ​​model selection module. The AI ​​model selection module may select an AI model among multiple AI models to perform a generation task corresponding to the language-based command. Meanwhile, those skilled in the art will understand that the prompt processing module and the AI ​​model selection module may be implemented as a single, integrated unit.

[0059] In one example, the prompt processing module and / or the AI ​​model selection module can identify a type (or category) of a generation task corresponding to a language-based command based on a rule-based approach, such as comparing the parsing result of the language-based command to keywords. The type of generation task can include, but is not limited to, text generation, image generation, code generation, speech generation, and / or video generation. The generative AI model can include, for example, a text generation AI model. The text generation AI model can provide various text-based outputs in response to a user's prompt, such as an essay on a specific topic, a summary of an input paragraph, or an answer to an inquiry. The generative AI model can include, for example, an image generation AI model. The image generation AI model can generate and provide an image in response to a user-entered prompt. The generative AI model can include, for example, a code generation AI model. The code generation AI model can generate and provide executable code in response to a user-entered prompt. Generative AI models may include, for example, a voice generation AI model. The voice generation AI model may generate and provide music in response to a user-entered prompt, or generate and provide voices that mimic human speech in response to a user-entered prompt. Generative AI models may include, for example, a video generation AI model. The video generation AI model may generate and provide videos in response to a user-entered prompt.

[0060] The prompt processing module and / or the AI ​​model selection module of the server (108) may select at least one AI model corresponding to the type of the identified generation task. For example, the prompt processing module and / or the AI ​​model selection module may determine that “draw” corresponds to a keyword of the category of the image generation task in the parsing result of the language-based command “draw a picture of a boy with a lonely face walking down a dark night street in a cartoon style.” For example, the prompt processing module and / or the AI ​​model selection module may select any one of the image generation AI models stored in the storage corresponding to the category of the image generation task. For example, the prompt processing module and / or the AI ​​model selection module may select the image generation AI model based on the analysis result of the language-based command. For example, the prompt processing module and / or the AI ​​model selection module may select an AI model that is identified as having relatively good cartoon image generation performance based on the keyword “cartoon style.” Meanwhile, the above-described keyword-based category and / or AI model identification method is merely exemplary, and those skilled in the art will understand that this may also be performed based on NLU (natural language understanding) (or natural language processing, NLP). The server (108) may also identify the category and / or AI model based on the execution result of, for example, an LLM (large language model) for AI model selection, for example, a sLLM (small LLM). Meanwhile, those skilled in the art will understand that, depending on the implementation, the prompt processing module and / or the AI ​​model selection module of the server (108) may identify an AI model to perform the creation task directly without performing the category identification process. Meanwhile, there is no limitation on the management and / or number of repositories.

[0061] The UI generation module of the server (108) can generate a UI based on the selected AI model. For example, the UI generation module can generate a UI for receiving input values ​​of the selected AI model. Alternatively, the UI generation module can generate a UI that selects and / or reflects at least some of the input values ​​of the selected AI model. For example, the UI generation module can identify parameters that constitute the UI and their corresponding values ​​based on the analysis results of language-based commands. The UI generation module can also generate a UI based on the identified information.

[0062] In one embodiment, the server (108) may not store and / or manage AI models. The server (108) may transmit and / or receive data with an external AI model provider. As described above, the server (108) may receive language-based commands from the electronic device (101). Based on the analysis results of the language-based commands, the server (108) may identify a category of generation tasks corresponding to the language-based commands and / or an AI model that will perform the generation tasks.

[0063] For example, the server (108) may identify an image generation AI model as an AI model to perform a generation task. In this case, the server (108) may request the image generation AI model provider to perform a generation task. The request for performing a generation task may include at least one piece of information identified based on the UI of the image generation AI model, but is not limited thereto. The image generation AI model provider may generate an image in response to the received request and provide the image to the server (108). The server (108) may provide the received image to the electronic device (101). Alternatively, the image generation AI model provider may directly transmit the generated image to the electronic device (101).

[0064] Alternatively, the server (108) may provide the electronic device (101) with information indicating that an image generation AI model is selected, information for executing an application corresponding to the image generation AI model, and / or connection information corresponding to the image generation AI model provider (121B). The electronic device (101) may directly connect to the image generation AI model provider based on the information received from the server (108). In this case, the electronic device (101) may request the image generation AI model provider to perform a generation task. The image generation AI model provider may generate an image and provide it to the electronic device (101) in response to the received request.

[0065] According to one embodiment, the electronic device (101) may perform AI model selection without being connected to the server (108). For example, the electronic device (101) may include a prompt processing module, an AI model selection module, and / or a UI generation module. As described above, at least one module may identify an AI model to perform a generation task based on a language-based command. The electronic device (101) may not store and / or manage the AI ​​model. The electronic device (101) may transmit and / or receive data with an external AI model provider. For example, the electronic device (101) may identify an image generation AI model as an AI model to perform a generation task. In this case, the electronic device (101) may request the image generation AI model provider to perform the generation task. The generation task request may include, but is not limited to, at least one piece of information identified based on the UI of the image generation AI model. The image generation AI model provider may, in response to the received request, generate an image and provide the image to the electronic device (101). Alternatively, the image generation AI model provider may generate an image in response to a received request and provide the image to the server (108), and the server (108) may provide the received image to the electronic device (101).

[0066] In one embodiment, the electronic device (101) can perform AI model selection without being connected to a server (108). For example, the electronic device (101) may include a prompt processing module, an AI model selection module, and / or a UI generation module. The electronic device (101) may store and / or manage multiple AI models. As described above, at least one module may identify an AI model to perform a generation task based on a language-based command. The electronic device (101) may, for example, execute an image generation AI model and provide a result of the generation task.

[0067] Referring to FIG. 2, according to one embodiment, the electronic device (101) may include a communication circuit (290) (e.g., the communication circuit of the communication module (190) of FIG. 1), a display (260) (e.g., the display of the display module (160) of FIG. 1), a speaker (270) (e.g., the speaker of the audio output module (155) of FIG. 1), and / or an input device (e.g., an input device of the input module (150) of FIG. 1) (e.g., a microphone, a mouse, a keyboard, keys (e.g., buttons), or a digital pen (e.g., a stylus pen)). As described above in FIG. 1, the electronic device (101) may include a processor (120) and a memory (130). As described above, at least some of at least one operation performed by the server (108) described in the present disclosure may also be performed by the electronic device (101). The electronic device (101) may include a hard disk, ROM, RAM, cache memory, and / or registers, and the implementation thereof is not limited thereto. Some of the above-described entities (e.g., registers, but not limited thereto) may be implemented as part of the electronic device (101), and the implementation form thereof is not limited thereto. The electronic device (101) may also store at least one generative AI model.

[0068] According to one embodiment, the operation of the electronic device (101) may be understood as the operation of at least one component of the electronic device (101). The operation of the electronic device (101) may be understood as the operation of at least one processor (e.g., 120). The electronic device (101) performing a specific operation may mean that at least one processor (e.g., 120) such as various hardware included in the electronic device (101), for example, a micro controlling unit (MCU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a microprocessor, a neural-network processing unit (NPU), a graphics processor (GP), an application processor (AP), or a communication processor (CP) performs the specific operation. For example, at least one processor (e.g., 120) may control at least one component of the electronic device (101). According to one embodiment, a non-transitory computer-readable storage medium storing instructions may be provided. The instructions stored in the storage medium (e.g., memory (130)) may, when individually or collectively executed by at least one processor (e.g., 120) of the electronic device (101), cause the electronic device (101) to perform at least one operation. The electronic device (101) performing a specific operation may also mean that the processor (e.g., 120) or other hardware performs the specific operation as instructions for performing the specific operation stored in a storage circuit (e.g., memory (130)) of the electronic device (101) are executed.Even when a plurality of processors (e.g., 120) are implemented, for convenience of explanation, the description may be referred to as “operation of the electronic device (101)”, “operation of the processor (120)”, or “operation of at least one processor (120)”.

[0069] The communication circuit (290) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and the server (108), and the performance of communication through the established communication channel. The communication circuit (290) can include, but is not limited to, one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication circuit (290) can include an Ethernet-based wired communication module (e.g., a local area network (LAN) communication module, or a power line communication module). The communication circuit (290) can also include, but is not limited to, a wireless communication module. At least some of the above components can be connected to each other through a communication method between peripheral devices (e.g., a bus, a general purpose input and output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and exchange signals (e.g., commands or data) with each other.

[0070] The display (260) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display (260) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. The input device (e.g., 150) can receive user input. According to one embodiment, the display (260) and the input device (e.g., 150) may be implemented as a touch display. Meanwhile, although not shown, an input / output device based on voice input / output, such as a microphone / speaker, may be further included in the electronic device (101).

[0071] Referring to FIG. 2, according to one embodiment, an electronic device (101) (e.g., a processor (120)) may provide an AI service based on a framework (220) related to an AI service, which includes a large language model (LLM) manager (224) and a plug-in mapper (PLUGIN mapper) (223). A plug-in may be a configuration that is set up or installed to add a specific function to an existing service. A low-rank adaptation of large language models (LORA) may be a model or a technique thereof in which only a portion of the parameters are updated to adapt a learned model to a specific task. A retrieval-augmented generation (RAG) may be a specific database or a technique thereof that is combined with a generative model. In the present disclosure, a plug-in, LORA, and RAG may be collectively referred to as a “plug-in.” According to one embodiment, the electronic device (101) may store information about applications of the electronic device (101) in APPs (221) and store information about plug-ins related to AI services in PLUGINs (222). According to one embodiment, the electronic device (101) (e.g., the processor (120)) may install a plug-in in the electronic device (101) based on the LLM manager (224). When the plug-in installation operation is performed, the electronic device (101) may manage information about applications related to the plug-in through the plug-in mapper (223). The plug-in mapper (223) may perform an operation of mapping information about applications related to the installed plug-in. For example, the installation of the plug-in may mean that a database and an operation engine related to the plug-in are downloaded in the form of sLLM within the electronic device (101). For example, installation of a plug-in may mean that activation information of the plug-in in the LLM service provided through the server (108) is provided to the electronic device (101).When a plug-in is installed, the electronic device (101) may store data related to the plug-in and information about an application related to the plug-in in the memory (130) (e.g., 221, 222) of the electronic device (101) through the LLM manager (224). According to one embodiment, the electronic device (101) may transmit installation information of the plug-in to an external server (108) or an AI model provider.

[0072] According to one embodiment, an AI service (e.g., an LLM service) of the electronic device (101) may be provided by an AI application of the electronic device (101). The "AI application" of the electronic device (101) may include an LLM application and / or an AI assistant application. The "LLM application" may be an application that provides an AI service. For example, the LLM application may be an application that is linked with the server (108). For example, the LLM application may be an application provided by the server (108) that provides the AI ​​service or an AI model provider and used in the electronic device (101). For example, the electronic device (101) may provide an AI service by linking with the server (108) by executing the LLM application. According to one embodiment, the LLM application may be an on-device type. For example, the LLM application may be an application that is linked with an LLM (large language model) included in the device itself. "AI assistant application" is an application (e.g. Bixby) provided on an electronic device (101) that is linked with a server (108) or LLM application that provides AI service. TM) may be. For example, the electronic device (101) may provide an AI service by linking with a server (108) or an LLM application through an AI assistant application. The AI ​​assistant application may use an on-device type AI model of the electronic device (101). For example, the electronic device (101) may provide an AI service by using an on-device type AI model of the electronic device (101) through an AI assistant application. According to one embodiment, the electronic device (101) may transmit information about an application installed in the electronic device (101) associated with a plug-in specialized for a specific task to the AI ​​application (e.g., the LLM application and / or the AI ​​assistant application) through the LLM manager (224). In one embodiment, an AI application (e.g., an LLM application and / or an AI assistant application) may be executed prior to an application (e.g., a gallery application, a messenger application, a mail application, a music application, a web browser application, etc.) being executed through a specific user input on the electronic device (101), or may be executed simultaneously on a single screen. Additionally, an AI application (e.g., an LLM application and / or an AI assistant application) may be activated through a user input through (within) a window of an application being executed.

[0073] FIG. 3 is a block diagram of a system for generating content using a generative artificial intelligence model, according to one embodiment.

[0074] According to one embodiment, the electronic device (101) may receive user input through the user query / response interface (310). The user input may be in the form of natural language, images, and / or videos. When transmitting the user input, additional information may also be transmitted. The additional information may include various additional information at the time of the user input. For example, the additional information may include information about the application currently being used by the user or information about the user's location. The user input may be in the form of a mixture of natural language, images, sounds, and additional information. The user input may also be in the form of non-natural language, such as selecting a menu. The electronic device (101) may output the results of the generative artificial intelligence system to the user through the user query / response interface (310). The results may be in the form of natural language or specific content. The results may also be provided in the form of an action requested by the user.

[0075] According to one embodiment, the electronic device (101) may receive user input based on the AI ​​framework (340) and coordinate and control each component (e.g., 341, 342, 343) necessary to perform the user's intention based on the user's query. The user input received at the user query / response interface (310) may be transmitted to the prompt design component (341). The electronic device (101) may use the prompt design component (341) to generate a prompt suitable for input into an LLM or LMM (large multimodal model) based on the user input. The prompt design component (341) may be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The prompt design component (341) may access a data store (320) containing user preference data, a prompt library, and prompt examples based on the user input to generate the prompt. The electronic device (101) can communicate with the outside world when there is a request for additional information when transmitting user input as input of a generative model using the API / plug-in management component (342). The electronic device (101) can establish a channel for communicating with the outside world through an application programming interface (API) using the API / plug-in management component (342), and can enable access to various data (e.g., data storage (320)) through the established channel. The electronic device (101) can request the application / service component (330) through the API to perform an action that performs the user input as a final result rather than an intermediate result in an application or service using the API / plug-in management component (342).Information obtained from an external source can be used to generate a prompt through the prompt design component (341) together with user input or can be passed as an input to the generative model. The electronic device (101) can fine-tune the output result from the generative model based on the output adjustment component (343). For example, the electronic device (101) can verify whether the content generated through the LLM and / or LMM is irrelevant, does not contain biased content, or does not contain harmful content based on the output adjustment component (343). The electronic device (101) can determine to what extent the content matches the result desired by the user based on the output adjustment component (343) and perform additional processing if necessary. The electronic device (101) can also configure hints to avoid additional unwanted output and provide them to the user based on the output adjustment component (343).

[0076] A generative AI model (360) may generally refer to an artificial intelligence neural network that creates new types of data based on user input information. The generative AI model (360) may be provided by a server (108) or may be provided on-device by an electronic device (101). The generative AI model (360) may include a model that generates images and / or a model that generates language. The model that generates images may typically include a generative adversarial network (GAN) and a variational autoencoder (VAE), and an example is a diffusion-based generative model that uses a VAE and a transformer structure. The generative AI model (360) may include an LMM that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding thereto.

[0077] FIG. 4 is a diagram illustrating a plug-in of an electronic device according to one embodiment.

[0078] Referring to FIG. 4, information about the settings (e.g., installation) of a plugin and the applications mapped to the plugin can be described. FIG. 4 is a diagram illustrating a screen in which a user searches for a specific plugin and determines an application associated with the specific plugin in a store where plugins related to AI services can be searched and downloaded. A specific plugin is searched based on the user's input, and the user can decide to install the plugin from the search results. When installing the plugin based on the user's decision, information about at least one application of the electronic device (101) is displayed, and installation can be performed with or without selecting at least one application based on the user's input. For example, in FIG. 4 (a), the electronic device (101) may display a screen (410) for searching for a plugin related to the AI ​​service. In FIG. 4 (b), the electronic device (101) may display a screen (420) for installing the plugin. In (c) of FIG. 4, the electronic device (101) may display a screen (430) including information about an application to be mapped with a plug-in. According to one embodiment, the mapping operation of the plug-in and the application in FIG. 4 may be automatically recommended or filtered and provided to the user based on plug-in information, application information installed in the electronic device (101), or the user's usage history, without user input. Alternatively, the plug-in may be automatically set based on the above information (e.g., plug-in information, application information, usage history) when the plug-in is installed.

[0079] FIG. 5 is a diagram illustrating mapping of an application and a plug-in of an electronic device according to one embodiment.

[0080] As described in FIG. 4, when installing a plug-in, mapping information between the installed plug-in and the application installed on the electronic device (101) can be determined. The electronic device (101) can store mapping information between at least one application (e.g., a gallery application, a messenger application, a mail application, a music application, a web browser application, etc.) and at least one plug-in. For example, one application can be mapped to one plug-in. For example, multiple applications can be mapped to one plug-in. For example, one application can be mapped to multiple plug-ins. For example, multiple applications can be mapped to multiple plug-ins. The mapping information can include mapping information between an application and a plug-in, mapping information between a specific function of the application and a plug-in, and / or mapping information between a context associated with an object displayed on the screen of the application and a plug-in, and there is no limitation on the type of mapping information. For example, if a plugin exists that can improve the quality of photos, the plugin can be associated with a messenger application and a camera application. When an LLM application or an AI assistant application is activated while the messenger application or camera application is running on the electronic device, the plugin can be determined as the plugin activated in the application. For example, in the case of a messenger application, a message or photo can be sent to the other party, and the file viewer function may be activated to send the photo. In this case, the plugin activated for the message sending function and the plugin activated when selecting an attachment may have different information. Therefore, the plugin can be determined based on information not only about the application being run, but also about the functions activated within each application.As shown in FIG. 5, the electronic device (101) can execute an AI application (e.g., 530) in which a specific plugin (e.g., at least one of 521, 522, 523, 524) corresponding to a specific application (e.g., at least one of 511, 512, 513, 514, 515, 516, 517, 518) is set based on mapping information. For example, in FIG. 5, the first application (511), the sixth application (516), and the eighth application (518) can correspond to the first plugin (521) and the second plugin (522), the third application (513) can correspond to the third plugin (523), and the fourth application (514) can correspond to the fourth plugin (524). However, this is merely an example, and specific embodiments will be described later.

[0081] The operations of the electronic device (101) can be described in detail with reference to the embodiments described above (e.g., the embodiments of FIGS. 1 to 5) and the embodiments described below (e.g., the embodiments of FIGS. 6 to 22). Although each embodiment is disclosed in a separate drawing and a separate paragraph, this is merely for convenience of explanation, and at least some of the embodiments described above and at least some of the embodiments described below can be applied together. At least some of the embodiments described above and at least some of the embodiments described below may be omitted.

[0082] FIG. 6 is a flowchart illustrating an operating method of an electronic device (101) according to one embodiment. FIG. 7 is a diagram illustrating a call to an AI application while a first application is running according to one embodiment. Some of the operations in FIG. 6 may be omitted or the order of the operations may be changed.

[0083] According to one embodiment, after the association with a specific application is stored when a plugin is installed, the electronic device (101) may, when an AI application (e.g., an LLM application or an AI assistant application) is activated, provide an AI application (e.g., an LLM application or an AI assistant application) in which at least one LLM plugin is set based on information associated with the currently running application. For example, as shown in FIG. 7, when the “SONG WRITER” plugin and the gallery application are mapped, the electronic device (101) may store mapping information between the gallery application and the “SONG WRITER” plugin, and when the AI ​​application (e.g., an LLM application or an AI assistant application) is activated while the gallery application is running, the electronic device (101) may execute the AI ​​application (e.g., an LLM application or an AI assistant application) in which the “SONG WRITER” plugin is set. Specifically, the following is provided.

[0084] In operation 601, according to one embodiment, the electronic device (101) may display a first screen corresponding to the first application. For example, in (a) of FIG. 7, the electronic device (101) may display a screen (710) of the gallery application.

[0085] In operation 603, according to one embodiment, the electronic device (101) can confirm a call to an AI application. The electronic device (101) can confirm a call to the AI ​​application while a first screen corresponding to the first application is displayed. The call to the AI ​​application may be a request to execute the AI ​​application or a request to display a screen of the AI ​​application that is currently running. The electronic device (101) can confirm a call to the AI ​​application based on a user input, and there is no limitation on the user input for calling the AI ​​application. For example, in FIG. 7, the electronic device (101) can confirm a call to the AI ​​application while a screen (710) of the gallery application is displayed.

[0086] In operation 605, according to one embodiment, the electronic device (101) can check the first information of the first application. The electronic device (101) can check the first information of the first application based on confirming a call to the AI ​​application while the first screen corresponding to the first application is displayed. For example, the first information of the first application may include basic information about the first application that is currently running. The basic information may include the name of the application, its type, its execution method, information about the included data, or additional information, and there is no limitation on the type of information as long as the basic information is information indicating that the application is the same. As described below, the information (e.g., the first information) of an application (e.g., the first application) may include basic information, functional information, and / or context information, which will be described below.

[0087] In operation 607, according to one embodiment, the electronic device (101) may display a screen of an AI application in which a first plugin corresponding to the first information of the first application is set. For example, as shown in FIG. 7, the electronic device (101) may display a screen (730) of an AI application in which a plugin (e.g., a “Song writer” plugin) corresponding to information about the gallery application is set based on a confirmation of a call to the AI ​​application while the screen (710) of the gallery application is displayed. As shown in FIG. 7, the screen (730) of the AI ​​application may be displayed to overlap a portion of the first screen (710) of the first application. However, this is merely an example, and the screen of the AI ​​application may be displayed without overlapping the screen of the currently running application, the screen of the AI ​​application may be displayed within the screen of the currently running application, or an object corresponding to the AI ​​application may be included within the screen of the currently running application. Referring to FIG. 7, according to one embodiment, a screen (730) of an AI application may include a prompt input window (731). The electronic device (101) may receive user input through the prompt input window (731). According to one embodiment, the screen (730) of the AI ​​application may include information (732) about a currently set plug-in. For example, the name and icon of the currently set plug-in may be included in the screen (730) of the AI ​​application.

[0088] FIG. 8 is a flowchart illustrating a method of operating an electronic device according to one embodiment. FIG. 9 is a diagram illustrating a plug-in corresponding to a function activated in an application according to one embodiment. Some of the operations in FIG. 8 may be omitted or the order of operations may be changed.

[0089] In one embodiment, the plugin may be configured based on the features enabled in the currently running application.

[0090] In operation 801, according to one embodiment, the electronic device (101) may display a first screen corresponding to the first application. For example, in (a) of FIG. 9, the electronic device (101) may display a screen (910) of a messenger application.

[0091] In operation 803, according to one embodiment, the electronic device (101) may confirm a request for activation of the first function of the first application. For example, in (a) of FIG. 9, the screen (910) of the messenger application may include an object (911) corresponding to the photo attachment function. The electronic device (101) may confirm a request for activation of the photo attachment function based on a user input for the object (911) corresponding to the photo attachment function on the screen (910) of the messenger application. Based on the request for activation of the photo attachment function, the electronic device (101) may activate the photo attachment function and display objects (e.g., 921, 922) for attaching a photo, as in (b) of FIG.

[0092] In operation 805, according to one embodiment, the electronic device (101) can confirm a call to the AI ​​application. The electronic device (101) can confirm a call to the AI ​​application while the first function of the first application is activated. For example, while the screen of FIG. 9 (b) is displayed, the electronic device (101) can confirm a call to the AI ​​application.

[0093] In operation 807, according to one embodiment, the electronic device (101) can check first information of the first application including basic information and function information. The electronic device (101) can check the first information of the first application including basic information and function information based on confirming a call to the AI ​​application while the first function of the first application is activated. The function information may be information about a function activated in the application. For example, based on confirming a call to the AI ​​application while the screen of FIG. 9 (b) is displayed, the electronic device (101) can check information of the messenger application including basic information of the messenger application and function information related to a photo attachment function activated in the messenger application.

[0094] In operation 809, according to one embodiment, the electronic device (101) may display a screen of an AI application in which a plug-in corresponding to the first information of the first application is set. The electronic device (101) may display a screen of an AI application in which a plug-in corresponding to the first information of the first application, which includes basic information and functional information, is set. For example, in (c) of FIG. 9, the electronic device (101) may display a screen (930) of an AI application in which a "Photo Doctor" plug-in is set based on basic information of a messenger application and functional information corresponding to a photo attachment function. The screen (930) of the AI ​​application may include a prompt input window (931). The screen (930) of the AI ​​application may include information (932) (e.g., a name and / or an icon) about the currently set plug-in.

[0095] FIG. 10 is a flowchart illustrating a method of operating an electronic device according to one embodiment. FIG. 11 is a diagram illustrating a plug-in corresponding to context information of an application according to one embodiment. Some of the operations in FIG. 10 may be omitted or the order of operations may be changed.

[0096] In one embodiment, a plugin may be set that corresponds to a context associated with an object displayed on the screen.

[0097] In operation 1001, according to one embodiment, the electronic device (101) may display a first screen corresponding to a first application. For example, in (a) of FIG. 11, the electronic device (101) may display a screen (1110) of a messenger application.

[0098] In operation 1003, according to one embodiment, the electronic device (101) can confirm a call to the AI ​​application. The electronic device (101) can confirm a call to the AI ​​application while the first screen of the first application is displayed. For example, while the screen of (a) of FIG. 11 is displayed, the electronic device (101) can confirm a call to the AI ​​application.

[0099] In operation 1005, according to one embodiment, the electronic device (101) may check the context (e.g., context information) associated with an object displayed on the first screen of the first application. For example, in (a) of FIG. 10, an object corresponding to a message (e.g., "What I want to eat") may be included on the screen (1110) of the messenger application. The electronic device (101) may check the context (e.g., "food") associated with the speech bubble corresponding to "What I want to eat" displayed on the screen (1110) of the messenger application.

[0100] In operation 1007, according to one embodiment, the electronic device (101) may verify first information of the first application, including basic information and context information. For example, based on confirming a call to the AI ​​application while the screen of FIG. 11 (a) is displayed, the electronic device (101) may verify information of the messenger application, including basic information of the messenger application and context information (e.g., “create food photo”) associated with an object displayed on the screen of the messenger application. According to one embodiment, in operation 1007, the electronic device (101) may also verify only the context information.

[0101] In operation 1009, according to one embodiment, the electronic device (101) may display a screen of an AI application in which a plug-in corresponding to the first information of the first application is set. The electronic device (101) may display a screen of an AI application in which a plug-in corresponding to the first information of the first application, which includes basic information and context information, is set. For example, in (b) of FIG. 11, the electronic device (101) may display a screen (1130) of an AI application in which a "Photo Grapher" plug-in is set based on the basic information of a messenger application and context information corresponding to "create food photos." The screen (1130) of the AI ​​application may include a prompt input window (1131). The screen (1130) of the AI ​​application may include information (1132) (e.g., a name and / or an icon) about the currently set plug-in. According to one embodiment, the electronic device (101) may acquire only context information in operation 1007, and display a screen of an AI application in which a plug-in corresponding to the context information is set in operation 1009. The plug-in corresponding to the context information may be the same as or different from the plug-in corresponding to the first information of the first application including basic information and context information.

[0102] Figure 12 is a flowchart illustrating a method of operating an electronic device according to one embodiment. Figure 13 is a diagram illustrating a plug-in corresponding to multiple applications according to one embodiment. Some of the operations in Figure 12 may be omitted or the order of operations may be changed.

[0103] According to one embodiment, at least one plug-in corresponding to multiple applications may be configured.

[0104] In operation 1201, according to one embodiment, the electronic device (101) can execute multiple applications. For example, as shown in FIG. 13, the electronic device (101) can execute a first application, a second application, and a third application. The electronic device (101) can execute the third application in the foreground, and execute the first application and the second application in the background. The foreground may be a screen (e.g., 1330 of FIG. 13) of the corresponding application (e.g., the third application of FIG. 13) among multiple screens (or windows) in which the screen (e.g., 1330 of FIG. 13) is activated. The background may be a state other than the foreground. For example, as shown in FIG. 13, if 1330 is activated and displayed at the front among 1310, 1320, and 1330, the third application can be said to be executed in the foreground, and the first and second applications can be said to be executed in the background. Alternatively, it can be said to run in the background even if the screen (or window) of the first application or the screen (or window) of the second application is minimized.

[0105] In operation 1203, according to one embodiment, the electronic device (101) may confirm a call to an AI application. The electronic device (101) may confirm a call to an AI application while multiple applications are running. For example, the electronic device (101) may confirm a call to an AI application while a third application is running in the foreground and the first and second applications are running in the background.

[0106] In operation 1205, according to one embodiment, the electronic device (101) can check multiple pieces of information corresponding to multiple applications currently running. The "information" may include basic information, functional information, and / or contextual information, and there is no limitation on the type of information.

[0107] In operation 1207, according to one embodiment, the electronic device (101) may display a screen of an AI application in which a plurality of plugins corresponding to a plurality of pieces of information of a plurality of applications are set. For example, as illustrated in FIG. 13, plugins corresponding to a first application, a second application, and a third application may be identified as a first plugin and a third plugin. In this case, the electronic device (101) may display a screen of an AI application in which the first plugin and the third plugin are set, and those skilled in the art will understand that the number of corresponding plugins is merely an example.

[0108] Figure 14 is a flowchart illustrating a method of operating an electronic device according to one embodiment. Figure 15 is a diagram illustrating a plug-in corresponding to a selected application according to one embodiment. Some of the operations in Figure 14 may be omitted or the order of operations may be changed.

[0109] According to one embodiment, at least one plug-in corresponding to the selected application may be set.

[0110] In operation 1401, according to one embodiment, the electronic device (101) can execute multiple applications. For example, the electronic device (101) can execute a first application and a second application.

[0111] In operation 1403, according to one embodiment, the electronic device (101) may confirm a call to an AI application. The electronic device (101) may confirm a call to an AI application while multiple applications (e.g., a first application, a second application) are running.

[0112] In operation 1405, according to one embodiment, the electronic device (101) may display a screen for selecting at least one of a plurality of currently running applications. For example, in FIG. 15 , the electronic device (101) may display a screen (1500) for selecting at least one of a first application and a second application currently running. The electronic device (101) may determine an application to be selected from a plurality of currently running applications based on user inputs for the first object (1510) and the second object (1520).

[0113] In operation 1407, according to one embodiment, the electronic device (101) can check information corresponding to the selected application. The “information” may include basic information, functional information, and / or context information, and there is no limitation on the type of information.

[0114] In operation 1409, according to one embodiment, the electronic device (101) may display a screen of an AI application in which a plug-in corresponding to information of a selected application is set. For example, in FIG. 15 , if the plug-ins corresponding to information of a first application are a first plug-in and a second plug-in, the electronic device (101) may display a screen (1530) of an AI application in which the first plug-in and the second plug-in corresponding to information of the first application are set when the first application is selected. The screen (1530) of the AI ​​application may include a prompt input window (1533). The screen (1530) of the AI ​​application may include information (e.g., 1531, 1532) (e.g., a name and / or an icon) about the currently set plug-in.

[0115] FIG. 16 is a flowchart illustrating a method of operating an electronic device according to one embodiment. FIG. 17 is a diagram illustrating a plug-in recommendation according to one embodiment. Some of the operations in FIG. 16 may be omitted or the order of the operations may be changed.

[0116] In one embodiment, plugins may be recommended based on user input.

[0117] In operation 1601, according to one embodiment, the electronic device (101) may confirm a user input. For example, the electronic device (101) may confirm the user input through a prompt input window (e.g., 1721 of FIG. 17 ). For example, in FIG. 17 , the user may input "I NEED SOME ADVICES FOR TAKING PHOTO SPECIALLY FOR MY BABY" (e.g., 1722) through a prompt input window (1721) included in a screen (1720) of the AI ​​application.

[0118] In operation 1603, according to one embodiment, the electronic device (101) may identify a plug-in related to a user input (e.g., "I NEED SOME ADVICES FOR TAKING PHOTO SPECIALLY FOR MY BABY"). For example, the electronic device (101) may identify a plug-in related to photo creation or photo modification based on "TAKING PHOTO" among the user inputs (e.g., "I NEED SOME ADVICES FOR TAKING PHOTO SPECIALLY FOR MY BABY").

[0119] In operation 1605, according to one embodiment, the electronic device (101) may display objects (e.g., 1711, 1712) for recommending installation of the identified plug-in. For example, in FIG. 17 , the electronic device (101) may display a screen (1710) including a first object (1711) for installing the "Photo Grapher" plug-in and a second object (1712) for installing the "Photo Doctor" plug-in.

[0120] FIG. 18 is a diagram illustrating an activation operation of an AI application according to a state of an electronic device, according to one embodiment.

[0121] In one embodiment, it can be described whether an AI application is activated while the home screen is displayed or whether an AI application is activated while a specific application is running.

[0122] In (a) of FIG. 18, when an AI application is activated while the home screen (1810) is displayed, the electronic device (101) may display a screen of an AI application for which a basic plugin is set or a plugin is not set, or an object (1811) corresponding to the AI ​​application. In (b) of FIG. 18, when an AI application is activated while a screen (1820) of a specific application (e.g., a car application) is displayed, the electronic device (101) may display a screen (or object) (e.g., a screen including 1821, 1822, 1823, or an object corresponding to 1821, 1822, 1823) of an AI application for which a plugin corresponding to information (e.g., basic information, function information, and / or context information) of the specific application (e.g., a car application) is set. As in FIG. 18, a plugin may be set depending on the state of the electronic device (101) (e.g., an application displayed on the screen).

[0123] FIG. 19 is a flowchart illustrating a method of operating an electronic device according to one embodiment. FIG. 20a is a diagram illustrating a plug-in corresponding to a first application according to one embodiment. FIG. 20b is a diagram illustrating a plug-in corresponding to a second application according to one embodiment. Some of the operations in FIG. 19 may be omitted or the order of operations may be changed.

[0124] In operation 1901, according to one embodiment, the electronic device (101) may display a first screen corresponding to a first application.

[0125] In operation 1903, according to one embodiment, the electronic device (101) may confirm a call to the AI ​​application while a first screen corresponding to the first application is displayed.

[0126] In operation 1905, according to one embodiment, the electronic device (101) can confirm first information of the first application based on confirming a call to the AI ​​application while a first screen corresponding to the first application is displayed.

[0127] In operation 1907, according to one embodiment, the electronic device (101) may display a screen of an AI application in which a first plug-in corresponding to first information of a first application is set.

[0128] In operation 1909, according to one embodiment, the electronic device (101) may display a second screen corresponding to the second application based on confirming a call (e.g., a request for execution or a request for display of a running screen) to the second application while the screen of the AI ​​application is displayed.

[0129] In operation 1911, according to one embodiment, the electronic device (101) may confirm a call to the AI ​​application while a second screen corresponding to the second application is displayed.

[0130] In operation 1913, according to one embodiment, the electronic device (101) can confirm second information of the second application based on confirming a call to the AI ​​application while a second screen corresponding to the second application is displayed.

[0131] In operation 1915, according to one embodiment, the electronic device (101) may display a screen of an AI application in which a second plug-in corresponding to second information of a second application is set.

[0132] For example, referring to FIGS. 20A and 20B, in FIG. 20A, the electronic device (101) may display a screen (2030) of an AI application in which a first plug-in corresponding to the first information of the first application is set based on confirming a call to the AI ​​application while the first screen (2010) of the first application is displayed (or activated), and at this time, the screen (2030) of the AI ​​application may include a prompt input window (2031) and information about the first plug-in (e.g., 2032, 2033). Thereafter, in FIG. 20b, the electronic device (101) may display a screen (2030) of the AI ​​application in which a second plug-in corresponding to the second information of the second application is set based on confirming a call to the AI ​​application while the second screen (2020) of the second application is displayed (or activated), and at this time, the screen (2030) of the AI ​​application may include a prompt input window (2031) and information about the second plug-in (e.g., 2042, 2043).

[0133] FIG. 21A is a diagram illustrating a screen of an AI application corresponding to a first application according to one embodiment. FIG. 21B is a diagram illustrating a screen of an AI application corresponding to a second application according to one embodiment.

[0134] According to one embodiment, as shown in FIGS. 21A and 21B , the screen of an AI application may be displayed within the screen of a currently running application. For example, in FIG. 21A , the electronic device (101) may display a screen (2110) of a first application and a screen (2120) of a second application. When the first application is activated and the AI ​​application is called, the electronic device (101) may display a screen (2131) of an AI application in which a plug-in corresponding to information of the first application is set within the screen (2110) of the first application. At this time, the screen (2131) of the AI ​​application may include information (e.g., a name and / or an icon) about the plug-in corresponding to information of the first application. For example, in FIG. 21b, when the second application is activated and the AI ​​application is called, the electronic device (101) may display, within the screen (2120) of the second application, a screen (2132) of the AI ​​application in which a plug-in corresponding to information of the second application is set. At this time, the screen (2132) of the AI ​​application may include information (e.g., a name and / or an icon) about the plug-in corresponding to information of the second application.

[0135] FIG. 22 is a drawing illustrating an embodiment of a foldable electronic device according to one embodiment.

[0136] According to one embodiment, the electronic device (101) may be a foldable device. For example, the first housing (2201) and the second housing (2202) of the electronic device (101) may be foldable based on a hinge structure. According to one embodiment, the display (260) of the electronic device (101) may include a cover display (2210) and a foldable display (2220). For example, the cover display (2210) may be disposed in the first housing and may face outward when the electronic device (101) is folded. For example, the foldable display (2220) may be disposed in the first housing and the second housing and may be folded when the electronic device (101) is folded. The embodiment of FIG. 22 is merely an example, and there is no limitation on the folding method of the electronic device (101), and the electronic device (101) may also be a multi-foldable device.

[0137] Referring to FIG. 22, according to one embodiment, the electronic device (101) may display a first screen (2211) of a first application on the cover display (2210) while the foldable display (2220) is folded (e.g., FIG. 22 (a)). Thereafter, based on the unfolding of the foldable display (2220), the electronic device (101) may display a screen (2221) of the first application corresponding to the first screen (2211) of the first application, and a screen (2222) of an AI application in which a plug-in corresponding to information of the first application is set, on the foldable display (2220). The screen (2222) of the AI ​​application may include information about the plug-in corresponding to information of the first application. In the embodiment of FIG. 22, the unfolding operation of the foldable display (2220) may correspond to a call of the AI ​​application. Alternatively, the embodiment of FIG. 22 may be performed as the foldable display (2220) unfolds after the call of a separate AI application is confirmed. In this document, “information” may include basic information, functional information, and / or contextual information, and there is no limitation on the type of information.

[0138] Those skilled in the art will appreciate that the embodiments described herein may be applied interchangeably, within the scope of their applicability. For example, those skilled in the art will appreciate that at least some operations of one embodiment described herein may be omitted and applied, or at least some operations of one embodiment may be applied in conjunction.

[0139] The technical tasks to be achieved in this document are not limited to the technical tasks mentioned above, and other technical tasks not mentioned will be clearly understood by those with ordinary skill in the technical field to which this document pertains from the description below.

[0140] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

[0141] According to one embodiment, an electronic device (101) may include a display (260) configured to display a screen, at least one processor (120) including a control circuit, and a memory (130) storing instructions. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a first screen corresponding to a first application. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to check first information of the first application based on a call to an AI (artificial intelligence) application while the first screen is displayed. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a screen of the AI ​​application in which a first plug-in corresponding to the first information is set. The above AI application may include an LLM (large language model) application and / or an AI assistant application of the electronic device (101) that is linked with the LLM application.

[0142] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a second screen corresponding to a second application based on a request for display of the second screen corresponding to the second application while the screen of the AI ​​application is displayed. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to check second information of the second application based on a request for display of the screen of the AI ​​application while the second screen is displayed. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a screen of the AI ​​application in which a second plug-in corresponding to the second information is set.

[0143] According to one embodiment, the screen of the AI ​​application may include information corresponding to the first plug-in set in the AI ​​application.

[0144] According to one embodiment, the first information of the first application may include first basic information of the first application and first function information about a first function activated in the first application. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to confirm a request for activation of the first function of the first application. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to confirm the first information of the first application including the first basic information and the first function information based on a call to the AI ​​application while the first screen is displayed. The above instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a screen of the AI ​​application in which a third plug-in corresponding to the first basic information and the first function information is set.

[0145] According to one embodiment, the first information of the first application may include first basic information of the first application and first context information associated with an object displayed on the first screen of the first application. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to identify the first information of the first application, including the first basic information and the first context information, based on a call to the AI ​​application while the first screen is displayed. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a screen of the AI ​​application in which a fourth plug-in corresponding to the first basic information and the first context information is set.

[0146] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the first screen of the first application while the first application is running in the foreground and the third application is running in the background. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to check, based on a call to the AI ​​application while the first screen is displayed, the first information of the first application running in the foreground and the third information of the third application running in the background. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the screen of the AI ​​application in which a fifth plug-in corresponding to the first information and the third information is set.

[0147] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a screen for selecting the first application and / or the fourth application based on a call to the AI ​​application while the first application and the fourth application are running. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to verify the first information of the first application and / or the fourth information of the fourth application based on a first user input for selecting the first application and / or the fourth application. The above instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display a screen of the AI ​​application in which the sixth plug-in corresponding to the first information and / or the fourth information corresponding to the selected first application and / or fourth application is set.

[0148] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to identify a seventh plug-in related to a second user input through a prompt input window of the screen of the AI ​​application based on the second user input. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display an object on the screen of the AI ​​application for recommending installation of the seventh plug-in.

[0149] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the screen of the AI ​​application in which the first plug-in is set within the first screen corresponding to the first application.

[0150] According to one embodiment, the display (260) may include a cover display (2210) and a foldable display (2220). The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the first screen of the first application on the cover display (2210) while the foldable display (2220) is folded. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to display the screen of the first application corresponding to the first screen and the screen of the AI ​​application on the foldable display (2220) based on the foldable display (2220) being unfolded.

[0151] According to one embodiment, a method of operating an electronic device (101) may include an operation of displaying a first screen corresponding to a first application. The method may include an operation of confirming first information of the first application based on a call to an AI (artificial intelligence) application while the first screen is displayed. The method may include an operation of displaying a screen of the AI ​​application in which a first plug-in corresponding to the first information is set. The AI ​​application may include an LLM (large language model) application and / or an AI assistant application of the electronic device (101) that is linked with the LLM application.

[0152] In one embodiment, the method may include an operation of displaying a second screen corresponding to the second application based on a request for display of the second screen corresponding to the second application while the screen of the AI ​​application is displayed. The method may include an operation of confirming second information of the second application based on a request for display of the screen of the AI ​​application while the second screen is displayed. The method may include an operation of displaying a screen of the AI ​​application in which a second plug-in corresponding to the second information is set.

[0153] According to one embodiment, in the method, the screen of the AI ​​application may include information corresponding to the first plug-in set in the AI ​​application.

[0154] According to one embodiment, in the method, the first information of the first application may include first basic information of the first application and first function information about a first function activated in the first application. The method may include an operation of confirming a request for activation of the first function of the first application. The method may include an operation of confirming the first information of the first application including the first basic information and the first function information based on a call to the AI ​​application while the first screen is displayed. The method may include an operation of displaying a screen of the AI ​​application in which a third plug-in corresponding to the first basic information and the first function information is set.

[0155] According to one embodiment, in the method, the first information of the first application may include first basic information of the first application and first context information associated with an object displayed on the first screen of the first application. The method may include an operation of confirming the first information of the first application including the first basic information and the first context information based on a call to the AI ​​application while the first screen is displayed. The method may include an operation of displaying a screen of the AI ​​application in which a fourth plug-in corresponding to the first basic information and the first context information is set.

[0156] In one embodiment, the method may include an operation of displaying the first screen of the first application while the first application is running in the foreground and the third application is running in the background. The method may include an operation of checking the first information of the first application running in the foreground and the third information of the third application running in the background based on a call to the AI ​​application while the first screen is displayed. The method may include an operation of displaying a screen of the AI ​​application in which a fifth plug-in corresponding to the first information and the third information is set.

[0157] According to one embodiment, the method may include an action of displaying a screen for selecting the first application and / or the fourth application based on a call to the AI ​​application while the first application and the fourth application are running. The method may include an action of confirming the first information of the first application and / or the fourth information of the fourth application based on a first user input for selecting the first application and / or the fourth application. The method may include an action of displaying a screen of the AI ​​application in which a sixth plug-in corresponding to the first information and / or the fourth information corresponding to the selected first application and / or the fourth application is set.

[0158] In one embodiment, the method may include an operation of identifying a seventh plugin related to a second user input via a prompt input window on the screen of the AI ​​application based on the second user input. The method may include an operation of displaying an object on the screen of the AI ​​application to recommend installation of the seventh plugin.

[0159] According to one embodiment, the method may include an operation of displaying the screen of the AI ​​application in which the first plug-in is set within the first screen corresponding to the first application.

[0160] According to one embodiment, in the method, the display (260) of the electronic device (101) may include a cover display (2210) and a foldable display (2220). The method may include an operation of displaying the first screen of the first application on the cover display (2210) while the foldable display (2220) is folded. The method may include an operation of displaying the screen of the first application corresponding to the first screen and the screen of the AI ​​application on the foldable display (2220) based on the foldable display (2220) being unfolded.

[0161] According to one embodiment, a non-transitory computer-readable recording medium storing instructions may cause the instructions, when individually or collectively executed by at least one processor (120) of an electronic device (101), to cause the electronic device (101) to perform at least one operation. The at least one operation may include an operation of displaying a first screen corresponding to a first application. The at least one operation may include an operation of confirming first information of the first application based on a call to an artificial intelligence (AI) application while the first screen is displayed. The at least one operation may include an operation of displaying a screen of the AI ​​application in which a first plug-in corresponding to the first information is set. The AI ​​application may include an LLM (large language model) application and / or an AI assistant application of the electronic device (101) that is linked with the LLM application.

[0162] In one embodiment, the at least one operation may include an operation of displaying a second screen corresponding to the second application based on a request for display of the second screen corresponding to the second application while the screen of the AI ​​application is displayed. The at least one operation may include an operation of checking second information of the second application based on a request for display of the screen of the AI ​​application while the second screen is displayed. The at least one operation may include an operation of displaying a screen of the AI ​​application in which a second plug-in corresponding to the second information is set.

[0163] According to one embodiment, in the storage medium, the screen of the AI ​​application may include information corresponding to the first plug-in set in the AI ​​application.

[0164] According to one embodiment, in the storage medium, the first information of the first application may include first basic information of the first application and first function information about a first function activated in the first application. The at least one operation may include an operation of confirming a request for activation of the first function of the first application. The at least one operation may include an operation of confirming the first information of the first application including the first basic information and the first function information based on a call to the AI ​​application while the first screen is displayed. The at least one operation may include an operation of displaying a screen of the AI ​​application in which a third plug-in corresponding to the first basic information and the first function information is set.

[0165] According to one embodiment, in the storage medium, the first information of the first application may include first basic information of the first application and first context information associated with an object displayed on the first screen of the first application. The at least one operation may include an operation of confirming the first information of the first application including the first basic information and the first context information based on a call to the AI ​​application while the first screen is displayed. The at least one operation may include an operation of displaying a screen of the AI ​​application in which a fourth plug-in corresponding to the first basic information and the first context information is set.

[0166] In one embodiment, the at least one operation may include an operation of displaying the first screen of the first application while the first application is running in the foreground and the third application is running in the background. The at least one operation may include an operation of checking the first information of the first application running in the foreground and the third information of the third application running in the background based on a call to the AI ​​application while the first screen is displayed. The at least one operation may include an operation of displaying a screen of the AI ​​application in which a fifth plug-in corresponding to the first information and the third information is set.

[0167] In one embodiment, the at least one operation may include an operation of displaying a screen for selecting the first application and / or the fourth application based on a call to the AI ​​application while the first application and the fourth application are running. The at least one operation may include an operation of confirming the first information of the first application and / or the fourth information of the fourth application based on a first user input for selecting the first application and / or the fourth application. The at least one operation may include an operation of displaying a screen of the AI ​​application in which a sixth plug-in corresponding to the first information and / or the fourth information corresponding to the selected first application and / or fourth application is set.

[0168] In one embodiment, the at least one action may include an action of identifying a seventh plug-in related to a second user input through a prompt input window on the screen of the AI ​​application based on the second user input. The at least one action may include an action of displaying an object on the screen of the AI ​​application to recommend installation of the seventh plug-in.

[0169] In one embodiment, the at least one action may include an action of displaying the screen of the AI ​​application in which the first plug-in is set within the first screen corresponding to the first application.

[0170] According to one embodiment, in the storage medium, the display (260) of the electronic device (101) may include a cover display (2210) and a foldable display (2220). The at least one operation may include an operation of displaying the first screen of the first application on the cover display (2210) while the foldable display (2220) is folded. The at least one operation may include an operation of displaying the screen of the first application corresponding to the first screen and the screen of the AI ​​application on the foldable display (2220) based on the foldable display (2220) being unfolded.

[0171] Devices according to the various embodiments disclosed in this document may take various forms. The devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Devices according to the embodiments of this document are not limited to the aforementioned devices.

[0172] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0173] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0174] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored on a storage medium that can be read by a machine (e.g., an electronic device). For example, a processor (e.g., a controller) of the machine may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0175] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0176] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101), A display (260) configured to display a screen; At least one processor (120) including a control circuit; and Includes a memory (130) for storing instructions, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Display the first screen corresponding to the first application, While the first screen is displayed, based on a call to an AI (artificial intelligence) application, the first information of the first application is checked, Causes the screen of the AI ​​application to be displayed, in which the first plug-in corresponding to the first information is set, The above AI application, Including an LLM (large language model) application and / or an AI assistant application of the electronic device (101) linked with the LLM application, Electronic device (101).

2. In paragraph 1, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: While the screen of the AI ​​application is displayed, based on a request for display of a second screen corresponding to the second application, the second screen corresponding to the second application is displayed, While the second screen is displayed, the second information of the second application is checked based on the request for display of the screen of the AI ​​application, Causing the screen of the AI ​​application to be displayed, in which the second plug-in corresponding to the second information is set, Electronic device (101).

3. In paragraph 1 or 2, The above screen of the above AI application is, Including information corresponding to the first plug-in set in the AI ​​application, Electronic device (101).

4. In any one of paragraphs 1 to 3, The first information of the first application includes first basic information of the first application and first function information about a first function activated in the first application, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Confirm the request for activation of the first function of the first application, While the first screen is displayed, based on a call to the AI ​​application, the first information of the first application including the first basic information and the first function information is confirmed, Causing the screen of the AI ​​application to be displayed, in which the third plug-in corresponding to the first basic information and the first function information is set, Electronic device (101).

5. In any one of paragraphs 1 to 4, The first information of the first application includes first basic information of the first application and first context information associated with an object displayed on the first screen of the first application, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: While the first screen is displayed, based on a call to the AI ​​application, the first information of the first application including the first basic information and the first context information is confirmed, Causing the screen of the AI ​​application to be displayed, in which the fourth plug-in corresponding to the first basic information and the first context information is set, Electronic device (101).

6. In any one of paragraphs 1 to 5, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: The first application is running in the foreground, and the third application is running in the background, while displaying the first screen of the first application; While the first screen is displayed, based on a call to the AI ​​application, the first information of the first application running in the foreground and the third information of the third application running in the background are checked, Causing the screen of the AI ​​application to be displayed, in which the fifth plug-in corresponding to the first information and the third information is set, Electronic device (101).

7. In any one of paragraphs 1 to 6, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: While the first application and the fourth application are running, based on a call to the AI ​​application, a screen for selecting the first application and / or the fourth application is displayed, Based on a first user input selecting the first application and / or the fourth application, the first information of the first application and / or the fourth information of the fourth application are verified, Causing the screen of the AI ​​application to be displayed, in which the first information corresponding to the selected first application and / or the fourth application and the sixth plug-in corresponding to the fourth information are set, Electronic device (101).

8. In any one of paragraphs 1 to 7, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Based on the second user input through the prompt input window of the above screen of the above AI application, the seventh plug-in related to the second user input is identified, Causing an object to be displayed on the screen of the AI ​​application to recommend installation of the above 7th plug-in; Electronic device (101).

9. In any one of paragraphs 1 to 8, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Causing the screen of the AI ​​application in which the first plug-in is set to be displayed within the first screen corresponding to the first application; Electronic device (101).

10. In any one of paragraphs 1 to 9, The above display (260) includes a cover display (2210) and a foldable display (2220). The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: While the above foldable display (2220) is folded, the first screen of the first application is displayed on the cover display (2210), Based on the unfolding of the above foldable display (2220), causing the screen of the first application corresponding to the first screen and the screen of the AI ​​application to be displayed on the foldable display (2220). Electronic device (101).

11. In the operating method of an electronic device (101), An action to display the first screen corresponding to the first application, and An action of checking first information of the first application based on a call to an AI (artificial intelligence) application while the first screen is displayed; Including an action of displaying a screen of the AI ​​application in which a first plug-in corresponding to the first information is set, The above AI application, Including an LLM (large language model) application and / or an AI assistant application of the electronic device (101) linked with the LLM application, method.

12. In paragraph 11, An operation of displaying the second screen corresponding to the second application based on a request for display of the second screen corresponding to the second application while the screen of the AI ​​application is displayed; An operation of checking second information of the second application based on a request for display of the screen of the AI ​​application while the second screen is displayed; Including an action of displaying a screen of the AI ​​application in which a second plug-in corresponding to the second information is set, method.

13. In paragraph 11 or 12, The above screen of the above AI application is, Including information corresponding to the first plug-in set in the AI ​​application, method.

14. In any one of paragraphs 11 to 13, The first information of the first application includes first basic information of the first application and first function information about a first function activated in the first application, The above method, An action for confirming a request for activation of the first function of the first application; An operation of confirming the first information of the first application including the first basic information and the first function information based on a call to the AI ​​application while the first screen is displayed; Including an action of displaying a screen of the AI ​​application in which a third plug-in corresponding to the first basic information and the first function information is set. method.

15. In a non-transitory computer-readable recording medium storing instructions, the instructions, when individually or collectively executed by at least one processor (120) of an electronic device (101), cause the electronic device (101) to perform at least one operation, At least one of the above actions: An action to display the first screen corresponding to the first application, and An action of checking first information of the first application based on a call to an AI (artificial intelligence) application while the first screen is displayed; Including an action of displaying a screen of the AI ​​application in which a first plug-in corresponding to the first information is set, The above AI application, An LLM (large language model) application that is linked to an external server and / or an AI assistant application of the electronic device (101) that is linked to the LLM application, Storage media.

Images