Electronic device supporting search function and operating method thereof

Abstract

This electronic device may comprise: a display; a memory including one or more storage media storing instructions; and at least one processor (210) including a processing circuit, wherein when the instructions are individually or collectively executed by the at least one processor (210), the electronic device performs operations of: obtaining a user query; obtaining personal usage data by accessing at least one application associated with the user query; obtaining at least one search word associated with the user query from the obtained personal usage data; generating at least one prompt by tuning the user query in an AI model on the basis of the obtained at least one search word; and displaying, via the display, a user interface screen in which the user query, the generated at least one prompt, and the obtained at least one search word are arranged according to a layout.

Description

Electronic device supporting search function and method of operation thereof

[0001] The present disclosure relates to an electronic device for generating search results based on artificial intelligence and a method of operating the same.

[0002] An artificial neural network (ANN) is a computational architecture modeled after the biological brain. Technologies such as deep learning and machine learning can be implemented based on ANNs. As an example of an ANN, a deep neural network (DNN) or deep learning can have a multilayer structure containing multiple layers.

[0003] AI models are being used in a variety of ways to analyze visual and audio data. To ensure effective operation of AI models on mobile devices, active research and development is underway on hardware technologies related to AI models. For example, research is being conducted on improving hardware architectures that take AI models into account, aiming to optimize the MAC (multiply-accumulate) operations performed in deep learning AI models.

[0004] The above information is provided as background information to aid in understanding this document. None of the above is claimed to be prior art related to this document or can be used to determine prior art.

[0005] According to one aspect of the present disclosure, an electronic device includes a display; a memory including one or more storage media storing instructions; and at least one processor including a processing circuit, wherein the electronic device can perform operations such as: obtaining a user query when the instructions are individually or collectively executed by the at least one processor; accessing at least one application associated with the user query to obtain personal usage data; obtaining at least one search term associated with the user query from the obtained personal usage data; tuning the user query in an AI model based on the at least one search term to generate at least one prompt; and displaying a user interface screen through the display in which the user query, the at least one generated prompt, and the at least one search term obtained are arranged in a layout.

[0006] According to one aspect of the present disclosure, at least one computer-readable instruction stored in a non-transitory computer-readable recording medium, when executed by at least one processor of an electronic device, causes the electronic device to perform the following operations: obtaining a user query; accessing at least one application associated with the user query to obtain personal usage data; obtaining at least one search term associated with the user query from the obtained personal usage data; tuning the user query in an AI model based on the at least one search term to generate at least one prompt; and displaying a user interface screen in which the user query, the at least one generated prompt, and the at least one obtained search term are arranged according to a specific layout.

[0007] According to one aspect of the present disclosure, a method of operating an electronic device (200) may include: obtaining a user query; accessing at least one application associated with the user query to obtain personal usage data; obtaining at least one search term associated with the user query from the obtained personal usage data; tuning the user query in an AI model based on the at least one search term to generate at least one prompt; and displaying a user interface screen in which the user query, the at least one generated prompt, and the at least one obtained search term are arranged according to a layout.

[0008] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0009] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0010] FIG. 2 is a block diagram of an exemplary electronic device capable of performing the operations described in this document.

[0011] FIG. 3 is an exemplary block diagram for providing generative artificial intelligence (AI) functionality in an electronic device according to one embodiment.

[0012] FIG. 4 is a block diagram of an exemplary AI system capable of performing the operations described in this document.

[0013] FIG. 5 is a block diagram of a prompt generation / processing system in an electronic device according to one embodiment.

[0014] FIG. 6A is a control flow diagram for generating a prompt in response to a user query in an electronic device, according to one embodiment.

[0015] FIG. 6b is a control flowchart for performing a response of an AI model in response to a user query in an electronic device according to one embodiment.

[0016] FIGS. 7A through 7E are exemplary diagrams of a user interface for generating a prompt in an electronic device according to one embodiment.

[0017] FIG. 8A or FIG. 8B is an example diagram of a user interface for generating a prompt in an electronic device according to one embodiment.

[0018] FIGS. 9A and 9B are exemplary diagrams of a user interface for generating a prompt in an electronic device according to one embodiment.

[0019] FIG. 10A or FIG. 10B is an example diagram of a user interface displaying a search word for editing a prompt in an electronic device according to one embodiment.

[0020] FIG. 11A or FIG. 11B is a diagram illustrating sequential access of applications to generate a prompt in an electronic device according to one embodiment.

[0021] FIG. 11c is a diagram illustrating generating a prompt by a sequential application in an electronic device according to one embodiment.

[0022] FIG. 11d is a diagram illustrating sequential processing of a user query or prompt using multiple LLMs in an electronic device according to one embodiment.

[0023] FIGS. 12A to 12C are exemplary diagrams showing a prompt editing screen displayed using an extended display in an electronic device.

[0024] FIG. 13 is an example diagram of a prompt editing screen in an electronic device according to one embodiment.

[0025] FIG. 14 is an exemplary diagram of a user interface for supporting the use of customized search results in an electronic device, according to one embodiment.

[0026] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0027] In various embodiments of the present disclosure, an electronic device and an operating method thereof can be provided that can output search results based on a prompt reflecting a user's intention based on AI.

[0028] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments.

[0029] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0030] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the non-volatile memory (134) may include a built-in memory (136) and an external memory (138). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0031] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the foregoing examples, but is not limited thereto. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may also include a software structure.

[0032] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0033] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0034] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0035] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0036] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by a touch.

[0037] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0038] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0039] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0040] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0041] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0042] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0043] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0044] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0045] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0046] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0047] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0048] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC positioned on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) positioned on or adjacent a second side (e.g., a top side or a side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0049] At least some of the components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)).

[0050] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that have received the request may execute at least a portion of the requested function or service, or additional functions or services related to the request, and transmit the results of the execution to the electronic device (101). The electronic device (101) may process the results as is or additionally and provide them as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0051] FIG. 2 is a block diagram of an exemplary electronic device (200) (e.g., the electronic device (101) of FIG. 1) capable of performing the operations described in this document.

[0052] Referring to FIG. 2, the electronic device (200) may be one of various forms of electronic devices, such as a notebook (290), smartphones (291) having various form factors (e.g., a bar-type smartphone (291-1), a foldable-type smartphone (291-2), or a sliderable (or rollable) type smartphone (291-3)), a tablet (292), a cellular phone (not shown), and other similar computing devices (not shown). The components, their relationships, and their functions illustrated in FIG. 2 are exemplary only and do not limit the implementations described or claimed in this document. The electronic device (200) may be referred to as a mobile device, a user device, a multi-function device, a portable device, or a server.

[0053] The electronic device (200) may include components including at least one processor (210) (hereinafter, referred to as 'processor (210)') (e.g., processor (120) of FIG. 1), at least one memory (220) (hereinafter, referred to as 'memory (220)') (e.g., memory (130) of FIG. 1), at least one display (240) (hereinafter, referred to as 'display (240)'), at least one image sensor (250) (hereinafter, referred to as 'image sensor (250)'), at least one communication circuit (260) (hereinafter, referred to as 'communication circuit (260)') (e.g., communication module (190) of FIG. 1), and / or at least one sensor (270) (hereinafter, referred to as 'sensor (270)') (e.g., sensor module (176) of FIG. 1). The components are merely exemplary. For example, the electronic device (200) may include other components (e.g., power management integrated circuitry (PMIC), audio processing circuitry, an antenna, a rechargeable battery, or input / output interfaces). For example, some components may be omitted from the electronic device (200). For example, some components may be integrated into a single component.

[0054] The processor (210) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing. The processor (210) may include at least one electrical circuit and may individually or collectively perform distributed processing of instructions (or programs, data, etc.) stored in the memory (220). The processor (210) may include a processor assembly including one or more processing circuits. The processor (210) may include any processing circuit operative to control the performance and operations of one or more components (e.g., the memory (220), the display (240), the image sensor (250), the communication circuit (260), and / or the sensor (270)) of the electronic device (200). For example, the processor (210) (e.g., the application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or a chipset). For example, the processor (210) may be implemented with multiple cores (or at least one core circuit), multiple chips, or multiple chipsets. For example, the processor (210) may include one or more processing circuits. For example, the processor (210) may include one or more processing circuits configured to individually and / or collectively perform various functions of the present disclosure. As a non-limiting example, at least a portion of the processor (210) may be included in a first chip of the electronic device (200), and at least another portion of the processor (210) may be included in a second chip of the electronic device (200) that is different from the first chip of the electronic device (200).

[0055] For example, the processor (210) may include a central processing unit (CPU) (211), a graphics processing unit (GPU) (212), a neural processing unit (NPU) (213), an image signal processor (ISP) (214), a display controller (215), a memory controller (216), a storage controller (217), a communication processor (CP) (218), and / or a sensor interface (219). These components of the processor (210) are merely exemplary. For example, the processor (210) may further include other components. For example, some components of the processor (210) may be omitted from the processor (210). For example, some components of the processor (210) may be included as separate components of the electronic device (200) outside the processor (210). For example, some components of the processor (210) (e.g., memory controller (216)) may be included within other components (e.g., at least a portion of memory (220), an interface (e.g., available for connection to at least one component of the electronic device (200)), a display (240) and / or an image sensor (250)).

[0056] The processor (210) may cause other components of the electronic device (200) to perform various operations by executing instructions stored in the memory (220). The CPU (211) (or central processing circuit) may be configured to control components of the processor (210) based on the execution of instructions stored in the memory (220) (e.g., volatile memory (221) (e.g., volatile memory (132) of FIG. 1) and / or non-volatile memory (222) (e.g., non-volatile memory (134) of FIG. 1)). The GPU (212) (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (213) (or neural processing circuit, or artificial intelligence (AI) chip) may be configured to execute operations for an AI model (e.g., convolution computation). The ISP (214) (or image signal processing circuit) may be configured to process a raw image acquired through the image sensor (250) into a format suitable for a component within the electronic device (200) or a component of the processor (210). The display controller (215) (or display control circuit, or display processing unit (DPU)) may be configured to process an image acquired from the CPU (211), the GPU (212), the ISP (214), or the memory (220) (e.g., the volatile memory (221)) into a format suitable for the display (240). The memory controller (216) (or memory control circuit) may be configured to control reading data from the volatile memory (221) and writing data to the volatile memory (221). The storage controller (217) (or storage control circuit) may be configured to control reading data from the nonvolatile memory (222) and writing data to the nonvolatile memory (222).The CP (218) (communication processing circuit) may be configured to process data obtained from a component of the processor (210) into a format suitable for transmission to another electronic device via the communication circuit (260), or to process data obtained from another electronic device via the communication circuit (260) into a format suitable for processing by the component of the processor (210). For example, the communication circuit (260) may include one or more communication circuits. The sensor interface (219) (or sensing data processing circuit, sensor hub) may be configured to process data on the state of the electronic device (200) and / or the state of the surroundings of the electronic device (200), obtained via the sensor (270), into a format suitable for the component of the processor (210).

[0057] The memory (220) may include one or more storage media (or one or more storage devices). For example, the memory (220) may include a memory assembly including one or more storage media. For example, the one or more storage media may include permanent memory (e.g., non-volatile memory (122)) such as a hard drive, flash memory, read-only memory (ROM), semi-permanent memory (e.g., volatile memory (221)) such as random access memory (RAM), any other suitable type of storage (or storage assembly), or any combination thereof. The memory (220) may include cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (200). As a non-limiting example, the cache memory may be included within the processor (210). The memory (220) may be fixedly embedded within the electronic device (200) or incorporated into one or more suitable types of components (e.g., a subscriber identity module (SIM) card and / or a secure digital (SD) card) that may be repeatedly inserted into and removed from the electronic device (200).

[0058] For example, the memory (220) may store one or more software applications, such as an operating system (or system) software application, a firmware software application, a driver software application, a plug-in (e.g., add-in, add-on, and / or applet) software application, and / or any other suitable software applications. For example, the one or more software applications may include instructions executable by the processor (210). For example, the memory (220) may store instructions callable by an application programming interface (API). For example, the memory (220) may store instructions within a library.

[0059] According to an example, the electronic device (200) can execute at least one instance of an AI model. An instance may be an object corresponding to a program (or application), such as an AI model, for example. An instance may be named a replica, a pod, a container, or a virtual machine, but there is no limitation on the name. The number of instances may correspond to the size of a resource (e.g., a GPU (212) or an NPU (213)), and accordingly, the number of instances may be used interchangeably with the size of the resource, or the instances may be used interchangeably with the resource.

[0060] As an example, multiple user requests may be input to the electronic device (200). The user requests may be associated with a service. The user request may be processed by a first instance of a first AI model, and a first processing result may be provided from the first instance of the first AI model. The first processing result may be processed by a first instance of a second AI model, and accordingly, a second processing result may be provided by the first instance of the second AI model. By serial processing of the processing results, the first instance of the M-th AI model may receive and process the N-1-th processing result. The first instance of the M-th AI model may provide the N-th processing result as a response. Accordingly, a response corresponding to the user request may be provided.

[0061] Based on the above-described process, responses corresponding to each of a plurality of user requests may be provided. Meanwhile, since processing must be performed by an instance, the time required to provide responses corresponding to each of a plurality of user requests (hereinafter referred to as “response time”) may take a relatively long time. The response time may affect the latency of the instance. To reduce the response time, the electronic device (200) may increase the number of instances of at least one AI model, which may be referred to as scaling out. However, there may be limitations in increasing the number of instances due to hardware and / or software constraints of the electronic device (200) and / or parameter restrictions of the AI ​​model (e.g., large language model (LLM)).

[0062] FIG. 3 is an exemplary block diagram for providing a generative artificial intelligence (AI) function in an electronic device (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2) (hereinafter referred to as 'electronic device (200)') according to one embodiment.

[0063] Referring to FIG. 3, the electronic device (200) may include a processor (210) (e.g., the processor (210) of FIG. 2), a memory (220) (e.g., the memory (220) of FIG. 2), and / or an interface (IF) (220). The electronic device (200) may be a device for providing a service linked to at least one AI system (320) (hereinafter, referred to as 'AI system (320)').

[0064] The AI ​​system (320) may include at least one AI model (hereinafter referred to as "AI model"). For example, the AI ​​system (320) may analyze received messages to generate a summary message. For example, the summary message may include one or more pieces of content (hereinafter referred to as "content") that have been reprocessed from received messages so that the user can easily recognize the content of the received messages. The content may be, for example, at least one of text, images, audio, or video.

[0065] The AI ​​system (320) may be based on natural language processing (NLP). NLP is, for example, a technology that allows an electronic device (200) to understand or process natural language input (hereinafter referred to as a "prompt (330)") that can be expressed in voice and / or text. The electronic device (200) can understand natural language through NLP and, based on this, understand human intention or convey information in a language that humans can understand. To understand human language, NLP can learn the order of words or tokens and predict the probability of the next word or token in a given text. A token is a basic unit for processing or understanding a prompt (330) in an AI model. Major technologies of NLP include tokenization, part-of-speech tagging, syntax analysis, named entity recognition, or sentiment analysis for a prompt (330) corresponding to a user's input.

[0066] The I / F (310) can input a prompt (330) and transmit the input prompt (330) to the processor (210). The prompt (330) can be a medium that guides the AI ​​system (320) to perform a task or generate a result in a desired direction. The prompt (330) can be the only window through which the user can communicate with the AI ​​system (320). The prompt (330) needs to be clear and specific in order to obtain an answer close to the desired result from the AI ​​system (320). According to an example, the I / F (310) can receive a response result (340) (e.g., a summary message and / or a response message) processed by the AI ​​system (320) based on the prompt (330) (e.g., received messages), and output the response result (340) converted into a natural language form (e.g., text, image, audio, or video) that can be recognized by humans. I / F (310) can input or output natural language in the form of voice and / or text, for example, with at least one component such as a keyboard, a touch panel, a display, and / or a speaker.

[0067] The processor (210) may execute software (e.g., a program) to control at least one other component (e.g., a hardware or software component) of the electronic device (200) that is electrically connected thereto. The processor (210) may perform various data processing or operations. As at least a part of the data processing or operations, the processor (210) may store commands or data received from other components (e.g., the I / F (310)) in the memory (220) (e.g., a volatile memory). As at least a part of the data processing or operations, the processor (210) may process commands or data stored in the memory (220) (e.g., a volatile memory). As at least a part of the data processing or operations, the processor (210) may store data resulting from processing commands or data in the memory (230) (e.g., a non-volatile memory).

[0068] The memory (220) may store various data used by at least one component (e.g., processor (210) and / or I / F (310)) of the electronic device (200). The data may include, for example, input data or output data for software (e.g., program) and commands related thereto. The memory (220) may also store at least one AI model (e.g., LLM, LVM (large vision models), LMM (large multimodal models)) for instance execution.

[0069] The memory (220) can store at least one instruction. The processor (210) can execute at least one instruction stored in the memory (220). When the at least one instruction is executed by the processor (210), the at least one instruction can cause the electronic device (200) to perform at least one operation. For example, as the at least one instruction is executed by the processor (210), at least one other component can be controlled, and / or various data processing or calculations can be performed. When an operation is performed by the processor (210), it can mean that the operation is performed by (or under the control of) one entity (e.g., a main processor) included in the processor (210), for example. The performance of one operation can mean that a specific operation is performed by (or under the control of) multiple entities (e.g., multiple processors), for example. The fact that multiple operations are performed may mean, for example, that all of the multiple operations are performed by (or under the control of) one entity (e.g., the main processor (121) of FIG. 1). When multiple operations are performed, this may mean, for example, that some of the multiple operations are performed by at least one entity, and some of the remaining operations are performed by at least one other entity. At least one instruction that causes the performance of one or more operations may be stored, for example, in one memory, or may be stored in a distributed manner in each of a plurality of memories.

[0070] In the electronic device (200), the AI ​​system (320) may share resources (e.g., data processing or computational capabilities) corresponding to part or all of at least one processor included in the processor (210) and / or resources (e.g., data recording area) corresponding to part or all of the memory (220). For example, the AI ​​system (320) may be operated by at least one of the CPU (211), the GPU (212), or the NPU (213). The AI ​​system (320) may be performed solely by the CPU (211), for example, by being allocated at least a portion of the memory (220). The AI ​​system (320) may be performed solely by the GPU (212), for example, by being allocated at least a portion of the memory (230). The AI ​​system (320) may be performed solely by the NPU (213), for example, by being allocated at least a portion of the memory (220). The AI ​​system (320) can be performed by, for example, allocating at least a portion of the memory (230) and allowing the CPU (211) and the GPU (212) to cooperate (e.g., together) in order to perform the operation. The AI ​​system (320) can be performed by, for example, allocating at least a portion of the memory (230) and allowing the CPU (211) and the NPU (213) to cooperate (e.g., together) in order to perform the operation. The AI ​​system (320) can be performed by, for example, allocating at least a portion of the memory (230) and allowing the GPU (212) and the NPU (213) to cooperate (e.g., together) in order to perform the operation. The AI ​​system (320) can be performed by, for example, allocating at least a portion of the memory (230) and allowing the CPU (211), the GPU (212), and the NPU (213) to cooperate (e.g., together) in order to perform the operation. The various embodiments described later in this disclosure are not limited to the combination of components for performing the AI ​​system (320), and may be implemented and / or applied based on any combination.

[0071] FIG. 4 is a block diagram of an exemplary AI system (e.g., AI system (320) of FIG. 3) capable of performing the operations described in this document. The AI ​​system (320) may be a generative AI system, but will be referred to as the “AI system (320)” hereinafter.

[0072] Referring to FIG. 4, the AI ​​system (320) may include an interface (User Query / Response Interface) (410) (e.g., I / F (310) of FIG. 3) (hereinafter referred to as 'I / F (410)'), an AI framework (420), a generative AI model (430) (hereinafter referred to as 'AI model'), a database (440), or an application / service component (Application / Service Component) (450).

[0073] The I / F (410) can receive input (e.g., user input or data acquired or generated by the terminal). The data acquired or generated by the terminal may include image or video data generated using a processor, values ​​transmitted through a sensor or sensor hub (e.g., external illumination, angle of the terminal, temperature of the display or terminal, display size or expansion / reduction information, captured images from an image sensor, etc.). The user input may be in the form of natural language, touch coordinates or stylus coordinates acquired through a touch panel or digitizer included in the display, images, and / or videos. In addition, context information may also be transmitted when the user input is transmitted. The context information may include various additional information at the time of the user input. For example, information on the application currently being used by the user or information on the user's location. In addition, the user input may also be in a mixed form of the above-described natural language, images, sounds, and context information. In addition, the user input may also be in a non-natural language form, such as selecting a menu. The I / F (410) can output the results of analyzing the output and / or input of the AI ​​system (320) to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of an action requested by the user. The output can also be provided in the form of a specific value specified by the user. The I / F (410) can output the results of the AI ​​system (320) to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of an action requested by the user.

[0074] The AI ​​framework (420) can receive user input and coordinate and control each component necessary to carry out the user's intention based on the user's query. For example, the AI ​​framework (420) can include a prompt design component (421), a management component (API / Plug-in management component) (423), or an output modification component (or refiner component) (425).

[0075] User input received from I / F (410) can be transmitted to prompt design component (421). Prompt design component (421) can be used to generate a prompt (e.g., prompt (330) of FIG. 3) suitable for inputting user input into AI model (430) (e.g., LLM, LVM or LMM). Prompt design component (421) can be an AI component that uses machine learning algorithm or neural network to develop better prompt (330) over time. Although not shown, prompt design component (421) can access user preference data, a prompt library, and a knowledge component including prompt examples based on user input to generate prompt (330) and transmit the generated prompt (330) to AI model (430).

[0076] The management component (423) may communicate with external information when there is a request for additional information when passing user input as input to the generative model. The management component (423) establishes a channel for communicating with the outside of the AI ​​interface through an API, and may access various data sources (e.g., knowledge repositories (445)) through the established channel. When an action must be performed based on the user's last input rather than an intermediate result in an application / service, the management component (423) may request the action to the application / service component (450) through the API. Information obtained from the outside may be used to generate a prompt (330) in the prompt design component (421) together with the user input, or may be passed as an input to the AI ​​model (430).

[0077] The output processing component (425) can fine-tune or reprocess the output from the AI ​​model (430). For example, the output processing component (425) can verify whether the content generated through the AI ​​model (430) is irrelevant, does not contain biased content, or does not contain harmful content. The output processing component (425) can also determine whether the content matches the user's desired result to some extent and, if necessary, perform additional processing. The output processing component (425) can additionally configure and provide the user with hints to avoid unwanted output.

[0078] An AI model (430) may generally refer to an AI neural network that generates new types of data based on user input information. The AI ​​model (430) may include a model that generates images and / or a model that generates language. The model that generates images may include, for example, a network or a variational autoencoder. The model that generates images may be, for example, a diffusion-based AI model that uses a variational autoencoder and a transformer structure. The model that generates language may be a model trained to statistically output the most appropriate output value based on input values. Representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. In addition, there is also an LMM as an AI model (430) that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding thereto.

[0079] FIG. 5 is a block diagram of a prompt generation / processing system (500) in an electronic device (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2) (hereinafter referred to as 'electronic device (200)') according to one embodiment.

[0080] Referring to FIG. 5, the prompt generation / processing system (500) can obtain input data (e.g., user query) (560). The prompt generation / processing system (500) can receive input data (560) in a natural language such as text or voice, for example. According to one example, the prompt generation / processing system (500) can receive a user query that is input as voice through an AI platform (e.g., Bixby) or as text through an input window (Search & Finder window).

[0081] The prompt generation / processing system (500) can analyze input data (560) and, based on the analysis results, obtain personal use data and / or log data within the scope of permitted access. To this end, the prompt generation / processing system (500) can determine (or identify) one or more target objects (e.g., an application, an account system, or a setting system) to be accessed to obtain personal use data and / or log data based on the analysis results of the input data (560). The prompt generation / processing system (500) can obtain personal use data and / or log data from, for example, an internal installation system (540). The prompt generation / processing system (500) can obtain personal use data and / or log data from, for example, another electronic device within the network environment (100). The prompt generation / processing system (500) can obtain personal use data and / or log data from, for example, a 3rd party application. Personal usage data may be, for example, personal data managed by a user through a specific application. For example, personal usage data may include personal schedule information managed by a user through an application that provides a schedule management function (hereinafter referred to as a "schedule management application"). For example, personal usage data may include weather information for a specific date checked by a user through an application that provides a weather guidance function (hereinafter referred to as a "weather guidance application"). For example, personal usage data may include data (e.g., photos or impressions) related to personal experiences registered by a user in an application that provides a social network service (SNS) function (hereinafter referred to as a "SNS application"). Log data may be, for example, data related to a user's usage history of an electronic device (e.g., the electronic device (200) of FIG. 2).Log data may, for example, be data related to a user's use of a specific application. For example, log data may include information about travel destinations searched for by the user using the electronic device (200). For convenience of explanation, personal usage data and / or log data may be collectively referred to as "personal usage data" or "related data" or "reference data" below.

[0082] In one example, the prompt generation / processing system (500) can determine (e.g., identify) the application with the highest priority among a plurality of applications as the central application. Depending on which application among the plurality of applications the prompt generation / processing system (500) selects as the central application, the scenario for generating the prompt may vary. For example, the prompt generation / processing system (500) can prioritize running applications so that they can be selected as the central application first. For example, the prompt generation / processing system (500) can give a relatively high priority to related applications based on recent usage history (e.g., capture). The prompt generation / processing system (500) can analyze the attributes of an additionally installed application (e.g., a map application) in response to input data (e.g., a user query) (560) and determine (e.g., identify) whether to consider the application associated with the input data based on the analyzed results.

[0083] For example, the prompt generation / processing system (500) can determine (e.g., identify) the order in which to access multiple applications to obtain relevant data based on input data (560). The prompt generation / processing system (500) can sequentially access the corresponding applications based on the determined access order to obtain relevant data. For example, the prompt generation / processing system (500) can access an application that manages a 'schedule' to obtain schedule data, and access an application that provides 'weather' to obtain weather data.

[0084] For example, the prompt generation / processing system (500) can analyze the acquired data to determine (e.g., identify) the next application to access (e.g., including a third-party application). The prompt generation / processing system (500) can access the determined application, select a candidate query, and then access additional related applications to obtain additional data. That is, the prompt generation / processing system (500) can additionally select or access necessary applications step by step. For example, the prompt generation / processing system (500) can obtain weather data from an application that provides weather information after referencing schedule data obtained from an application that manages schedules.

[0085] For example, when the prompt generation / processing system (500) obtains input data such as 'What should I eat for dinner tonight?', it can determine at least one related application in response to the input data. The prompt generation / processing system (500) can analyze data (hereinafter, referred to as 'application data') managed by the determined at least one related application. For example, the prompt generation / processing system (500) can select a schedule management application and / or a weather information application as related applications based on the keyword (or complex entry) 'tonight' included in the input data. The prompt generation / processing system (500) can access the schedule management application within the range permitted by the user to access and obtain schedule data related to 'tonight'. The prompt generation / processing system (500) can analyze the obtained schedule data and analyze application data such as attendees or times related to the schedule. For example, the prompt generation / processing system (500) can select a schedule management application, a function providing location information (e.g., GPS function), and / or a map guidance application as related applications based on the keyword 'where' included in the input data.

[0086] According to one example, the prompt generation / processing system (500) can determine (e.g., identify) one or more search terms (e.g., words or tokens) that reflect a semantic distance based on the analysis results of personal usage data and / or log data. Here, the search terms may include one or more applied search terms (hereinafter referred to as “applied search terms”) and / or one or more recommended search terms (hereinafter referred to as “recommended search terms”). The applied search terms may be, for example, search terms to be used for generating a prompt in the prompt generation / processing system (500). The recommended search terms may be, for example, search terms that can be used or selected for reprocessing a prompt in the prompt generation / processing system (500). In the following description, the applied search terms may be used to mean one or more applied search terms, and the recommended search terms may be used to mean one or more recommended search terms. Here, "semantic distance" may include measuring the conceptual difference between two or more entities within a given context. For example, semantic distance is an indicator that quantifies the degree of dissimilarity or similarity between various concepts and can be utilized to identify semantic relationships. For example, by mathematically representing the semantic distance between numerous words in a multidimensional vector space, an NLP can be developed that can acquire words based on their dissimilarity or similarity to a specific word. For example, the NLP can predict the next word of a specific word based on the semantic distance between concepts or words in a text or two or more texts.

[0087] The prompt generation / processing system (500) can determine one or more recommended search terms in addition to applicable search terms based on the analysis results of personal usage data and / or log data.

[0088] According to one example, the prompt generation / processing system (500) may not limit the data to be analyzed to data related to the application, but may additionally consider related information provided from other electronic devices (e.g., electronic devices (102, 104) of FIG. 1) within the network environment (e.g., the network environment (100) of FIG. 1). The prompt generation / processing system (500) may analyze related information transmitted from other electronic devices (102, 104) and determine one or more search terms (e.g., words or tokens) reflecting a semantic distance based on the analysis result of the related information. The network environment (100) may be provided, for example, to support a multi-device experience (MDE). The MDE may provide an environment in which a differentiated experience can be provided by grafting AI and / or IoT onto multiple devices. In this case, the application data to be referenced by the prompt generation / processing system (500) may include, for example, functions and / or information related to IoT devices (or related applications). Data may be added. For example, the prompt generation / processing system (500) may obtain (e.g., receive) temperature information from another electronic device within a network environment (100), such as an air conditioner, in response to input data (e.g., a user query) (560) such as "I wish it were cool." In this case, the prompt generation / processing system (500) may consider (e.g., receive) the obtained temperature information to determine (e.g., identify) a search term.

[0089] In one example, the prompt generation / processing system (500) can generate a prompt using determined applicable search terms. The prompt generation / processing system (500) can transmit the generated prompt, one or more applicable search terms, or one or more recommended search terms as output data (570). The prompt generation / processing system (500) can generate response data for the generated prompt and provide the generated response data as output data (570) or display it through a display (550).

[0090] In one example, the prompt generation / processing system (500) may reconfigure the prompt to reflect the removal of at least one applicable search term (hereinafter referred to as a “removed search term”) selected for removal from one or more applicable search terms based on input data (560). The prompt generation / processing system (500) may reconfigure the prompt to reflect the addition of at least one recommended search term (hereinafter referred to as an “additional search term”) selected for addition from one or more recommended search terms. The prompt generation / processing system (500) may reconfigure the prompt by removing at least one removed search term selected from one or more applicable search terms and adding at least one additional search term selected from one or more recommended search terms. The prompt generation / processing system (500) may also analyze personal usage data and / or log data to which access is permitted when reconfiguring the prompt, and reconfigure the prompt by taking the analysis results into consideration. The prompt generation / processing system (500) can provide a reconfigured prompt, one or more applicable search terms, or one or more recommended search terms as output data (570). The prompt generation / processing system (500) can generate response data for the reconfigured prompt and provide the generated response data as output data (570) or display it through a display (550).

[0091] According to one example, the prompt generation / processing system (500) may include an AI framework (510) (e.g., the AI ​​framework (420) of FIG. 4), a personal usage database (520), a generative AI model (530) (e.g., the generative AI model (430) of FIG. 4), or an installation system (Device Installed System) (540). The installation system (Device Installed System) (540) may include a basic system (e.g., an account system (543), a setting system (545)) that is basically pre-installed for use of the electronic device (200) and / or a plurality of applications (541) (e.g., Application#1 (541-1) to Application#n (541-n)) that the user selectively installs as needed. The basic systems (543, 545) may not be deletable. Applications (541) included in the installation system may be selectively installed or deleted by the user.

[0092] For example, the personal use database (520) may include storage space for managing personal use data and / or log data generated by a user accessing and using the installation system (540). The personal use database (520) may provide user use data and / or log data within a permitted access range. The permitted access range of the personal use database (520) may be set by the user, for example.

[0093] For example, the AI ​​framework (510) may access the personal usage database (520) based on the content of natural language (e.g., text or voice) included in the input data (560) to obtain personal usage data and / or log data within an allowed scope. The personal usage data and / or log data may include information related to the results of a user's use, recording, or search for a specific application (e.g., a calendar application, a weather application, a health application, etc.). The AI ​​framework (510) may perform the function of a prompt assistant manager. The function of the prompt assistant manager may include, for example, a function of determining at least one application to access based on semantic distance using personal usage data, and extracting and combining application data from at least one application in consideration of priority to generate a prompt. For example, the AI ​​framework (510) may consider the correlation between applications when reading application data from the personal usage database (520). The AI ​​framework (510) can sequentially select applications (including third-party applications) to access personal usage data based on priority. For example, if "Please make a restaurant reservation" is entered as input data (560), the AI ​​framework (510) can reference personal user data from a schedule management application (e.g., family dinner at 7 p.m. this weekend), access a weather forecast application to retrieve weather information for the corresponding schedule, or retrieve personal user data from a social networking service application (e.g., information on restaurants visited).

[0094] For example, a generative AI model (530) can analyze a user's query in response to a prompt provided by an AI framework (510) and generate meaningful content as a response based on the analysis results. The generative AI model (530) can output the generated response results through a display (550).

[0095] For example, an interface (e.g., I / F (310) of FIG. 3) may exist between the AI ​​framework (510) and the user. The I / F may provide the user with prioritized application data based on the user's input data (560) (e.g., Text Input content) and the semantic distance from the personal usage data. The prioritized application data may include applicable search terms used to generate the prompt and / or recommended search terms that were not used to generate the prompt but may be considered when reprocessing the prompt. The I / F may receive information related to search terms added and / or removed by the user (e.g., information about added search terms and / or removed search terms) as input and transmit the information to the AI ​​framework (510).

[0096] For example, among the configurations, the application may include all applications that are installed directly on the cloud app or the electronic device (200) and store information on the electronic device (200). In addition, the LLM may also include cases where it is operated on-device or on a server. In addition, in the case of a 3rd party application, it may be possible to provide data in the application to the LLM based on data provided in the form of an API from the LLM system and operate it. As an example, in an LLM system where multiple LLM models are operated, the LLM models may be sequentially used based on the data to be processed. For example, the LLM system may operate a first LLM for processing personal data and a second LLM for processing public data separately. In this case, the LLM system may operate the first LLM to obtain a primary result using the personal data, and may operate the second LLM to obtain a secondary result using the obtained primary result.

[0097] As described above, when at least one of the data of the installed application (541) is selected, the electronic device (200) including the prompt generation / processing system (500) can generate a first prompt based on the user query and the data of the at least one selected authorized application through the AI ​​framework (510). The first prompt generated in the electronic device (200) can be transmitted to the generation AI module (530). The electronic device (200) can output the generated first prompt and the data used for generating the prompt from among the data of the at least one authorized application. At this time, the result according to the first prompt can also be received through the generation AI module (530) and displayed. The electronic device (200) supports the user to delete and / or add related content to the first prompt through the displayed information. In response to the user input, the electronic device (200) can generate and display a processed second prompt by adding or excluding the data of the authorized application from the first prompt. More specific examples will be discussed in detail below.

[0098] FIG. 6A is a control flowchart for generating a prompt in response to a user query in an electronic device (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2) (hereinafter referred to as 'electronic device (200)') according to one embodiment.

[0099] Referring to FIG. 6A, the electronic device (200) (e.g., AI framework (510)) may, in operation 611, receive data corresponding to a user query (e.g., input data (560) of FIG. 5). The input data may include, for example, data requesting a search for desired information. The electronic device (200) may receive the input data (560) in a natural language, such as text or voice. For example, the electronic device (200) may receive a user query input as voice through an AI platform (e.g., Bixby) or as text through an input window (Search & Finder window).

[0100] According to one embodiment, the electronic device (200) (e.g., the AI ​​framework (510)) may analyze a user query at operation 613. For example, the electronic device (200) (e.g., the generative AI model (530)) may utilize an NLP function to convert input data (560) which is a natural language (e.g., text or voice) into machine language, and recognize the content of the user's desired query through the converted machine language. The electronic device (200) (e.g., the AI ​​framework (510)) may, at operation 613, select one or more applications to access in order to prepare a response to the user query based on the analysis result. For example, if the user query is analyzed as 'What should I eat for dinner tonight?', the electronic device (200) (e.g., the AI ​​framework (510)) may determine (e.g., identify) at least one related application in response to the analysis result. For example, the electronic device (200) (e.g., AI framework (510)) may determine (e.g., identify) a schedule management application and / or a weather guidance application as related applications based on the keyword (or compound index word) “this evening” according to the analysis results. For example, the electronic device (200) (e.g., AI framework (510)) may select a schedule management application, a function providing location information (e.g., GPS function), and / or a map guidance application as related applications based on the keyword “where” according to the analysis results.

[0101] According to one embodiment, the electronic device (200) (e.g., the AI ​​framework (510)) may, at operation 615, access one or more applications (e.g., a schedule management application and / or a weather guidance application) selected based on the analysis results of the user query, and obtain application data corresponding to the data related to the user query within a range to which the user has permitted access. As an example, the electronic device (200) (e.g., the AI ​​framework (510)) may access the schedule management application within a range to which the user has permitted access, and obtain schedule data, which is application data related to 'this evening'. As an example, the electronic device (200) (e.g., the AI ​​framework (510)) may access a function providing location information (e.g., a GPS function) and / or a map guidance application within a range to which the user has permitted access, and obtain data related to a location and / or a restaurant, which is application data related to 'a place to eat dinner', based on the analysis results of 'where'.

[0102] According to one embodiment, the electronic device (200) (e.g., the AI ​​framework (510)) may generate a first prompt based on the analyzed user query and the acquired application data, which is the associated data, in operation 617. For example, the electronic device (200) (e.g., the AI ​​framework (510)) may analyze the associated data to which access is permitted. The electronic device (200) (e.g., the AI ​​framework (510)) may extract applicable search terms or recommended search terms that reflect a semantic distance based on the analysis result of the associated data. As an example, the electronic device (200) (e.g., the AI ​​framework (510)) may generate the first prompt based on the search terms acquired based on the user query (e.g., “What should I eat for dinner tonight?”) and the associated data. For example, if schedule data such as 'family gathering at 7 PM' and SNS data of visiting a 'steakhouse in Yangjae' are acquired as related data, the electronic device (200) (e.g., AI framework (510)) can acquire search terms such as '7 PM', 'family', 'Yangjae', and 'steakhouse'. The electronic device (200) (e.g., AI framework (510)) can determine some or all of the acquired search terms as applicable search terms. For example, if all of the acquired search terms are determined as applicable search terms, the electronic device (200) (e.g., AI framework (510)) can generate a first prompt such as 'Make a reservation at a steakhouse in Yangjae for our family gathering at 7 PM tonight'. In this case, the applicable search terms can be determined as, for example, '7 PM', 'family', 'gathering', 'Yangjae', and 'steakhouse'. For example, if some of the acquired search terms are determined to be applicable search terms, the electronic device (200) (e.g., AI framework (510)) may generate a first prompt such as 'Make a reservation at a steakhouse in Yangjae for 7 PM tonight.'In this case, applicable search terms might be determined as, for example, "7 p.m.," "Yangjae," and "steakhouse," while recommended search terms might be determined as, for example, "family" and "gathering." Examples of the first prompt will be discussed in detail below.

[0103] According to one embodiment, the electronic device (200) (e.g., the AI ​​framework (510)) may output a user query, associated data (e.g., applicable search terms and / or recommended search terms), and a first prompt at operation 619. The electronic device (200) (e.g., the AI ​​framework (510)) may, for example, display the user query, associated data (e.g., applicable search terms and / or recommended search terms), and the first prompt as visual information on a display. The electronic device (200) (e.g., the AI ​​framework (510)) may, for example, output the user query, associated data (e.g., applicable search terms and / or recommended search terms), and the first prompt as auditory information through an audio output means (e.g., a speaker).

[0104] For example, the electronic device (200) (e.g., the generative AI model (530)) can display the processing result of the AI ​​model for the user query, i.e., the search result, through a display (e.g., the display (550) of FIG. 5) or output it as an audible signal (e.g., the output data (570)) through an audio output means. The overall operation for obtaining the processing result corresponding to the user query can be performed by the generative AI model (530). For example, the electronic device (200) (e.g., the generative AI model (530)) can display the processing result of the AI ​​model for the first prompt, i.e., the search result, through a display (e.g., the display (550) of FIG. 5) or output it as an audible signal (e.g., the output data (570)) through an audio output means. The overall operation for obtaining the processing result corresponding to the first prompt can be performed by the generative AI model (530).

[0105] FIG. 6b is a control flowchart for performing a response of an AI model in response to a user query in an electronic device (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2) (hereinafter referred to as 'electronic device (200)') according to one embodiment.

[0106] According to an example, the operation of the electronic device (200) generating the first prompt in response to a user query in operations 621 to 625 of FIG. 6b is substantially the same as operations 611 to 619 described with reference to FIG. 6a, and therefore, a detailed description thereof is omitted.

[0107] According to an example, the electronic device (200) may, in operation 626, receive user adjustment information. The user adjustment information may be information that may be considered when reprocessing the first prompt. For example, the user adjustment information may be information about one or more applied (or applied) search terms to be excluded from among applied search terms when reprocessing the prompt. The user adjustment information may be information about one or more recommended search terms to be added from among recommended search terms when reprocessing the prompt. For example, the user adjustment information may be information about one or more applied (or applied) search terms to be excluded from among applied search terms when reprocessing the prompt and one or more recommended search terms to be added from among recommended search terms when reprocessing the prompt. For example, the electronic device (200) may suggest an application to be additionally considered for reprocessing the first prompt. For example, the electronic device (200) may allow the user to specify an application to be additionally considered for reprocessing the first prompt. For example, the electronic device (200) may suggest that application data of a specific application be additionally considered for reprocessing the first prompt. For example, the electronic device (200) may request that the user additionally consider application data that can be obtained by accessing the specific application for reprocessing the first prompt.

[0108] In one example, the electronic device (200) may, at operation 627, reprocess the first prompt to reflect (e.g., consider or apply) user-adjusted information and generate a second prompt. The second prompt may be reprocessed to more accurately reflect the user's intent. In one example, the reprocessing of the first prompt may be performed in response to removing one or more applicable search terms (e.g., rainy day, family, four people, or lunch) from among the applicable search terms that have been reflected (e.g., considered or applied) to generate the current prompt (e.g., recommend a good restaurant near Yangjae Station for a family of four to go to for lunch on a rainy day). For example, reprocessing of the first prompt may be performed in response to adding one or more suggested search terms (e.g., "cold day," "friends," "two people," "snacks," or "near Seocho-dong") that were not reflected in the generation of the current prompt (e.g., "Recommend a good restaurant near Yangjae Station for a family of four to go to for lunch on a rainy day") but that could be considered to replace or newly reflect the applicable search terms. For example, reprocessing of the first prompt may be performed in response to removing one or more applicable search terms and adding one or more suggested search terms. Examples of reprocessing prompts to reflect user-customized information are discussed in more detail below.

[0109] For example, the electronic device (200) may, at operation 628, acquire a processing result of an AI model based on a second prompt and output the acquired processing result. The electronic device (200) may output the second prompt, applicable search terms used to generate the second prompt, and / or recommended search terms that were not used when generating the second prompt but can be selected for further processing. If user-adjusted information is input in response to the output second prompt, applicable search terms, and / or recommended search terms, the electronic device (200) may perform reprocessing for the second prompt.

[0110] FIGS. 7A to 7E are exemplary diagrams of a user interface (UI) (hereinafter, referred to as a “prompt generation UI”) for generating a prompt (e.g., a prompt (330) of FIG. 3) in an electronic device (e.g., an electronic device (101) of FIG. 1 or an electronic device (200) of FIG. 2) (hereinafter, referred to as an “electronic device (200)”) according to one embodiment.

[0111] The prompt creation UI may include a prompt creation screen activated to create a prompt or a prompt editing screen activated to edit a prompt. The prompt creation screen or prompt editing screen may have different screen configurations or items used or their placements may be modified depending on the situation. The prompt creation UI may include a search results display screen that displays search results generated based on analysis of an AI model (e.g., the generative AI model (430) of FIG. 4) in response to a user query or prompt.

[0112] Referring to FIG. 7a, the electronic device (200) can output a first prompt editing screen (700b) in response to a user query (e.g., “Recommend a good restaurant near Yangjae Station”) entered in the prompt generation screen (700a).

[0113] For example, the prompt generation screen (700a) may include an input window (710) for entering a user query or a search term. The prompt generation screen (700a) may further include a window for suggesting search terms that can be combined with information (e.g., the user query) entered into the input window (710), a window for recommending applications to use for searching, or a window for displaying one or more entered search terms. For example, the user may enter a query (e.g., “Recommend a good restaurant near Yangjae Station”) into the input window (710) and then select an indicator (e.g., a search icon) requesting a response. The input window (710a) may also allow for input of user queries or search terms by voice rather than text. The input window (710) may also allow for input of user queries or search terms by voice rather than text based on an AI platform (e.g., Bixby). In the input window (710), a user may directly input a user query or search term, but it may also be input remotely by an external electronic device (e.g., electronic device (102, 104) of FIG. 1).

[0114] For example, the electronic device (200) may generate a first prompt (721) (e.g., "Recommend a good restaurant near Yangjae Station for a family of four to go to for lunch on a rainy day") in response to a user query. The electronic device (200) may not be able to generate the first prompt (721) with only the information that can be obtained by analyzing the input user query. In this case, the electronic device (200) may determine (e.g., identify) one or more applications to obtain additional information based on the results of analyzing the user query. The electronic device (200) may obtain (e.g., receive) related data (or application data) necessary to generate the first prompt (721) from one or more applications determined based on the results of analyzing the user query. In this case, the related data that can be obtained from one or more applications may be data of applications that the user has permitted to use.

[0115] In one example, the electronic device (200) may obtain first search terms (723, 725) that may be considered for generating a first prompt (721) corresponding to a user query based on relevant data. The first search terms (723, 725) may include, for example, first applicable search terms (723) and / or first recommended search terms (725). To obtain the search terms (723, 725), the electronic device (200) may analyze the user query and, based on the analysis results, determine (e.g., identify) one or more applications for obtaining relevant data. For example, the electronic device (200) may obtain (e.g., receive) the first applicable search terms (723) (e.g., rainy day, family, 4 people, or lunch) used for generating the first prompt (721). For example, the electronic device (200) may obtain first recommended search terms (725) (e.g., cold day, friend, two people, snack, near Seocho-dong) that were not used to generate the first prompt (721), but could be considered for reconstructing the first prompt (721). The electronic device (200) may determine (e.g., identify) the first search terms (e.g., first applied search terms (723) and / or first recommended search terms (725)) based on information obtained by analyzing a user query and / or information obtained from one or more applications.

[0116] As an example, an AI model (e.g., a generative AI model (430) of FIG. 4) may analyze a user query and / or information obtained from one or more applications to obtain first applicable search terms and / or first recommended search terms. For example, the AI ​​model (430) may analyze data of an application to which access is permitted within the electronic device (200) to obtain information related to the user query. For example, after obtaining access permission from the user, the AI ​​model (430) may access data of an application to which access is not permitted within the electronic device (200), analyze the relevant data, and obtain additional information related to the user query based on the analysis results. The electronic device (200) may generate search results corresponding to the user query using the AI ​​model (430). The electronic device (200) may output a first prompt editing screen (700b) using search results, a first prompt (721), first applicable search terms (723), or first recommended search terms (725) in response to a user query.

[0117] In one example, the first prompt editing screen (700b) may include a first prompt editing window (720). The first prompt editing window (720) may include items for editing the first prompt (721). For example, the first prompt editing window (720) may display a first prompt (721) (e.g., recommend a good restaurant near Yangjae Station for a family of four to go to for lunch on a rainy day), first applicable search terms (723) (e.g., rainy day, family, four people, or lunch), or first recommended search terms (725) (e.g., cold day, friend, two people, snack, or near Seocho-dong). Near each of the first applicable search terms (723) (e.g., rainy day, family, four people, or lunch), a removal identifier (e.g., an icon X) may be displayed to exclude the applicable search term when reprocessing the prompt. Near each of the first recommended search terms (725) (e.g., cold day, friend, two people, snack, or near Seocho-dong), an additional identifier (e.g., icon O) may be displayed for adding the additional search term when reprocessing the prompt. The first prompt editing window (720) may include an instruction (727) (e.g., a 'search' icon) for requesting a search for the first prompt (721). The first prompt editing screen (700b) may include a window (e.g., search result display #1) for displaying search results for the user query. For example, the user may select the search instruction (727) to request a search for the first prompt (721) displayed in the first prompt editing window (720).

[0118] Referring to FIG. 7b, when an instruction (727) requesting a search is input in the first prompt editing screen (700b), the electronic device (200) may output a second prompt editing screen (700c) including a search result for the first prompt (721) (see the left drawing of FIG. 7b). For example, the second prompt editing screen (700c) may include an input window (710) or a first prompt editing window (720). The first prompt editing window (720) may display the first prompt (721) (e.g., recommend a good restaurant near Yangjae Station for a family of four to go to for lunch on a rainy day), first applicable search terms (723) (e.g., rainy day, family, four people, or lunch), or first recommended search terms (725) (e.g., cold day, friend, two people, snack, or near Seocho-dong).

[0119] According to one example, when at least one additional search term and / or at least one removed search term is selected in the first prompt editing window (720), the electronic device (200) may reprocess the first prompt (721) to reflect the selected at least one additional search term and / or at least one removed search term. The electronic device (200) may output a prompt editing screen including search results for the reprocessed prompt.

[0120] For example, if at least one of the first applicable search terms (723) is selected as a removal search term (e.g., family (729)) in the first prompt editing window (720), the electronic device (200) may reprocess the first prompt (721) into a second prompt (731) (e.g., recommend a good restaurant near Yangjae Station for 4 people to go to for lunch on a rainy day) by reflecting (e.g., considering or applying) the selected removal search term. The electronic device (200) may output a third prompt editing screen (700d) including a second prompt editing window (730) (see the right drawing of FIG. 7b). The second prompt editing window (730) may include the reprocessed second prompt (731), the second applicable search terms (e.g., rainy day, 4 people, or lunch), or the second recommended search terms (725) (e.g., cold day, friend, 2 people, snack, or near Seocho-dong). Near each of the second recommended search terms (725), an additional identifier (e.g., an icon O) may be displayed to add the recommended search term when reprocessing the prompt. The user may, for example, select a search term to be added from among the second recommended search terms (725). The user may, for example, select 'friend (733)', which is one of the second recommended search terms (725) (e.g., cold day, friend, two people, snack, or near Seocho-dong). In this case, the electronic device (200) may output the seventh prompt editing screen (700h) of FIG. 7e.

[0121] Referring to FIG. 7c, when an instruction (727) requesting a search is input in the first prompt editing screen (700b), the electronic device (200) may output a fourth prompt editing screen (700e) including a search result for the first prompt (721) (see the left drawing of FIG. 7c). For example, the fourth prompt editing screen (700e) may include an input window (710) or a third prompt editing window (740). The third prompt editing window (740) may include a third prompt (741) that is substantially the same as the first prompt (721) (e.g., recommend a good restaurant near Yangjae Station for a family of four to go to for lunch on a rainy day), third applicable search terms (743) (e.g., rainy day, family, four people, or lunch), and / or an instruction (745) requesting the addition of a search term (e.g., a '+' icon). In the instruction (745) requesting addition of a search term, addition of a search term may be requested using hardware such as a microphone or camera in addition to the '+' icon. For example, when the '+' icon requesting addition of a search term is pressed, the electronic device (200) displays an identifier corresponding to the microphone and / or camera button, and when the identifier is selected by the user, the electronic device can receive input of a search term to be added through voice or image recognition.

[0122] In one example, when an instruction (745) (e.g., a '+' icon) requesting addition of a search word is input in the fourth prompt editing screen (700e), the electronic device (200) may output a fifth prompt editing screen (700f) including a search word addition window (750) (see the right drawing of FIG. 7c). In one example, the fifth prompt editing screen (700f) may include a search word addition window (750) and / or an instruction (983) (e.g., an 'add' icon) that may request addition. In the search word addition window (750), third recommended search words for addition (e.g., cold day, friend, two people, snack, or near Seocho-dong) may be displayed. An additional identifier (e.g., an icon O) may be displayed near each of the third recommended search words for adding the corresponding recommended search word when reprocessing the prompt. For example, a user can select a search term (e.g., "friend") to add from the search term addition window (750). The user can then select one of the third suggested search terms (e.g., "cold day," "friend," "two people," "snack," or "near Seocho-dong") (e.g., "friend") and then select an indicator (753) (e.g., "add" icon) to request addition. In this case, the electronic device (200) can display the seventh prompt editing screen (700h) of FIG. 7e.

[0123] Referring to FIG. 7d, when an instruction (727) requesting a search is input in the first prompt editing screen (700b), the electronic device (200) can output a sixth prompt editing screen (700g) including search results for the first prompt (721) (see the left drawing of FIG. 7d). For example, the sixth prompt editing screen (700g) can include an input window (760) or a fourth prompt editing window (770). The fourth prompt editing window (770) may include a fourth prompt (771) that is substantially the same as the first prompt (721) (e.g., recommend a good restaurant near Yangjae Station for a family of four to go to for lunch on a rainy day), fourth applicable search terms (773) (e.g., rainy day, family, four people, or lunch), fourth recommended search terms (775) (e.g., cold day, two people, snack, or near Seocho-dong), and / or an instruction (777) for requesting a search (e.g., a 'search' icon).

[0124] For example, the electronic device (200) can directly input a search term to be added to the input window (760) in the sixth prompt editing screen (700g). For example, the user can input 'friend' as a search term to be added to the input window (760) and then select an indicator (777) (e.g., a 'search' icon) that can request a search. In this case, the electronic device (200) can output the seventh prompt editing screen (700h) of FIG. 7e.

[0125] Referring to FIG. 7e, when a request is made to add a recommended search word in the third prompt editing screen (700d) or the fifth prompt editing screen (700f), the electronic device (200) may output the seventh prompt editing screen (700h) (see the left drawing of FIG. 7e). For example, the seventh prompt editing screen (700h) may include an input window (710) or a fifth prompt editing window (780). The fifth prompt editing window (780) may display a fifth prompt (781) (e.g., recommend a good restaurant near Yangjae Station for four friends to go to for lunch on a rainy day), fifth applicable search words (783) (e.g., rainy day, four people, lunch, or friends), or fifth recommended search words (785) (e.g., cold day, two people, snack, or near Seocho-dong).

[0126] According to one example, when at least one additional search term and / or at least one removed search term is selected in the fifth prompt editing window (780), the electronic device (200) can reprocess the fifth prompt (781) to reflect the selected at least one additional search term and / or at least one removed search term.

[0127] According to an example, when a search instruction (787) included in a fifth prompt editing window (780) is selected by a user, the electronic device (200) may generate a search result corresponding to the fifth prompt (781) using the AI ​​model (430). In response to the fifth prompt (781), the electronic device (200) may output an eighth prompt editing screen (700i) using a search result, a sixth prompt (751) substantially identical to the fifth prompt (781), sixth applicable search terms (783), or sixth recommended search terms (785).

[0128] In one example, the eighth prompt editing screen (700i) may include a sixth prompt editing window (780). The sixth prompt editing window (780) may include items for editing the sixth prompt (781). For example, the sixth prompt editing window (780) may display a sixth prompt (781) (e.g., recommend a good restaurant near Yangjae Station for four friends to go to for lunch on a rainy day), sixth applicable search terms (783) (e.g., rainy day, four people, lunch, or friends), or sixth recommended search terms (785) (e.g., cold day, two people, snack, or near Seocho-dong). Near each of the sixth applicable search terms (783), a removal identifier (e.g., an icon X) may be displayed to exclude the applicable search term when reprocessing the prompt. Near each of the 6th recommended search terms (785), an additional identifier (e.g., icon O) may be displayed to add the additional search term when reprocessing the prompt.

[0129] FIG. 8A or FIG. 8B is an example diagram of a user interface (UI) (hereinafter referred to as a “prompt generation UI”) for generating a prompt (e.g., a prompt (330) of FIG. 3) in an electronic device (e.g., an electronic device (101) of FIG. 1 or an electronic device (200) of FIG. 2) (hereinafter referred to as “electronic device (200)”) according to one embodiment.

[0130] The prompt creation UI may include a prompt creation screen activated to create a prompt or a prompt editing screen activated to edit a prompt. The prompt creation screen or prompt editing screen may have different screen configurations or items used or their placements may be modified depending on the situation. The prompt UI may include a search results display screen that displays search results generated based on analysis of an AI model (e.g., the generative AI model (430) of FIG. 4) in response to a user query or prompt.

[0131] Referring to FIG. 8a or FIG. 8b, the prompt generation screen (810a) may include an input window (811a) for entering a user query or a search term. The prompt generation screen (810a) may further include a window for displaying information entered into the input window (811a) and an identifier of one or more applications associated with the entered information, or a window for displaying one or more search terms entered.

[0132] According to one example, the electronic device (200) may analyze a user query and, based on the analysis results, select one or more applications to access in order to provide a response to the user query. For example, the applications to be accessed by the electronic device (200) may be differentiated according to the user query. For example, if the user query is analyzed (e.g., identified) as "What should I eat for dinner tonight?", the electronic device (200) may, in response to the analysis results, select a schedule management application and / or a weather guidance application as related applications based on the keyword (or compound index word) "tonight" according to the analysis results. In addition, the electronic device (200) may, in response to the analysis results, select a schedule management application, a function providing location information (e.g., a GPS function), and / or a map guidance application as related applications based on the keyword "where" according to the analysis results. For example, the electronic device (200) may select multiple applications at once to obtain related data in response to the user query. For example, the electronic device (200) may sequentially select multiple applications to obtain related data in response to a user query. For example, there may be multiple schedules obtained from a schedule management application based on the keyword (or compound index word) "this evening" according to the analysis results for the user query. In this case, the electronic device (200) needs to select one schedule among the multiple schedules. For example, the electronic device (200) may select a schedule with a higher priority among the multiple schedules. For example, the electronic device (200) may give a relatively higher priority to a pre-arranged appointment (e.g., a schedule that was registered first in terms of timing) among the multiple schedules. For example, the electronic device (200) may also select one schedule by considering the priority given to each appointment target (e.g., family, friends, or acquaintances).

[0133] In one example, the electronic device (200) may analyze a user query (811b, 841b, 871b) and select one or more related applications based on the analysis results. The electronic device (200) may access the selected application and select at least one of the information stored internally in association with the user query (811b, 841b, 871b) (e.g., app usage log data permitted by the user, data of installed applications).

[0134] When the electronic device (200) selects internal information related to a query, the parameters of the selected information may be information to which the user has permitted access. This may be determined based on the user's settings, application installation settings, etc. Alternatively, the electronic device (200) may request the user's permission to access data based on the query analysis results. For example, the electronic device (200) may select, in response to a user query, a specific application (e.g., a calendar app or a weather app) to which the user has permitted access and information stored in the app (e.g., schedule information stored in the Calendar application) among the applications installed in connection with the application or information stored in the application.

[0135] In one example, the electronic device (200) can check a query (811b, 841b, 871b) input by a user in a prompt generation screen (810b, 840b, 870b), and check or select an application (813b, 843b, 873b) and / or information (820b, 850b, 880b) stored in the application associated with the checked user query (811b, 841b, 871b). For example, when receiving a user query (811b, 841b, 871b) such as "How is the weather this weekend?", the electronic device (200) can analyze the user query (811b, 841b, 871b) and select an associated application, such as a calendar application and / or a weather application. The electronic device (200) can check the application data (821b, 823b or 851b, 853b, 855b or 881b, 883b, 885b, 887b) related to the information (820b, 850b, 880b) stored in response to the application (813b, 843b, 873b).

[0136] For example, the electronic device (200) may analyze a user query (811b) of 'How is the weather this weekend?' entered in a prompt generation screen (810b), select a Callender app and a Weather app as related applications, and obtain first personal data (821b) (e.g., 'Jeju Island trip this weekend, September') and second personal data (823b) (e.g., '30 degrees, clear weather on Jeju Island trip date') as personal data (820b) corresponding to the Callender app and the Weather app, respectively. The electronic device (200) may analyze the first personal data (821b) and the second personal data (823b) to add search terms (837b) (e.g., 'Jeju Island,' 'This weekend's weather') to the user query (831b) (e.g., 'How is the weather this weekend?') to reconstruct the prompt (835b). The electronic device (200) may output a prompt editing screen (830b) including a user query (831b) and a prompt editing window (833b). The prompt editing window (833b) may include a reconfigured prompt (835b) (e.g., “What is the weather in Jeju Island this weekend?”) and search terms (837b) (e.g., “Jeju Island,” “Weather this weekend”).

[0137] According to one example, the electronic device (200) analyzes a user query (841b) of 'Recommend me some trendy clothes these days' entered in the prompt generation screen (840b), selects a Callender app, a Weather app, and an account system as related applications, and obtains first personal data (851b) (e.g., arrival date of Saipan in August), second personal data (853b) (e.g., weather on the date of Saipan travel 34 degrees, sunny), and third personal data (855b) (e.g., female in her 30s) as personal data (850b) corresponding to the Callender app, the Weather app, and the account system, respectively. The electronic device (200) can analyze the first personal data (851b), the second personal data (853b), and the third personal data (855b) to reconfigure the prompt (865b) by adding search terms (867b) (e.g., 34 degrees, clear day, August, female, Saipan, recommend trendy clothes) to the user query (841b) (e.g., recommend trendy clothes these days). The electronic device (200) can output a prompt editing screen (860b) including the user query (841b) and a prompt editing window (863b). The prompt editing window (863b) can include the reconfigured prompt (865b) (e.g., recommend trendy clothes for a female in her 30s to wear in Saipan on a 34 degree, clear day in August) and search terms (867b) (e.g., 34 degrees, clear day, August, female, Saipan, recommend trendy clothes).

[0138] According to one example, the electronic device (200) analyzes a user query (871b) of 'Tell me about Osaka tourist attractions' entered in a prompt generation screen (870b), selects a Callender app, a Health app, a Weather app, and an account system as related applications, and obtains first personal data (851b) (e.g., Osaka backpacking trip, date), second personal data (883b) (e.g., average steps of 10,000 steps), third personal data (885b) (e.g., Osaka travel date, weather 28 degrees, sunny), and fourth personal data (887b) (e.g., female in her 30s) as personal data (880b) corresponding to the Callender app, the Health app, the Weather app, and the account system, respectively. The electronic device (200) can analyze the first personal data (881b), the second personal data (883b), the third personal data (885b), and the fourth personal data (887b) to add search terms (897b) (e.g., 28 degrees, clear day, 30s, female, warm-weather backpacking, walking less than 10,000 steps, Osaka tourist attractions) to the user query (871b) (e.g., tell me about Osaka tourist attractions) to reconstruct the prompt (895b). The electronic device (200) can output a prompt editing screen (890b) including the user query (871b) and a prompt editing window (893b). The prompt editing window (893b) can include reconfigured prompts (895b) (e.g., Tell me about Osaka tourist attractions within 10,000 steps that a woman in her 30s can go on a backpacking trip alone on a 28-degree, clear day), and search terms (897b) (e.g., 28 degrees, clear day, 30s, female, warm weather backpacking trip, walking within 10,000 steps, Osaka tourist attractions).

[0139] For example, the number of applications selected by the electronic device (200) and the stored data may be limited to a specified number. For example, applications associated with the query "Tell me about tourist attractions in Osaka" may include Calendar, Health, Weather, and Account applications. However, if the specified number is two, the Calendar and Weather applications may be selected. In this case, the electronic device (200) may determine the applications based on priority. Here, the priority may be determined based on various information, such as relevance to the user query and user preferences.

[0140] FIGS. 9A and 9B are exemplary diagrams of a user interface (UI) (hereinafter referred to as a “prompt generation UI”) for generating a prompt (e.g., a prompt (330) of FIG. 3) in an electronic device (e.g., an electronic device (101) of FIG. 1 or an electronic device (200) of FIG. 2) (hereinafter referred to as “electronic device (200)”) according to one embodiment.

[0141] Referring to FIG. 9a or 9b, the prompt creation UI may include a prompt creation screen (900a) that is activated to create a prompt, or a prompt editing screen (900b, 900c, 900d) that is activated to edit a prompt. The prompt creation screen (900a) or the prompt editing screen (900b, 900c, 900d) may have different screen configurations or items used or their placement positions changed depending on the situation. The prompt UI may include a search result display screen (930b, 930d) that displays search results generated based on analysis of an AI model (e.g., the generative AI model (430) of FIG. 4) in response to a user query or prompt.

[0142] For example, the prompt generation screen (900a) may include an input window (910a) for entering a user query or a search term. The prompt generation screen (900a) may further include a window for suggesting search terms that can be combined with the information entered in the input window (910a), a window for recommending applications to use for searching, or a window for displaying one or more entered search terms. For example, the user may enter a query (e.g., “How about high-bandwidth turtle intestines?”) in the input window (910a) and then select an indicator (920a) requesting a response (e.g., a “search” icon). The input window (910a) may also allow the user to enter a query or search term using voice (e.g., an artificial intelligence platform) rather than text. In the input window (910a), the user can directly input a user query or search term, or input it remotely using an external electronic device (e.g., the electronic device (102, 104) of FIG. 1).

[0143] In one example, the electronic device (200) may generate a plurality of recommendation prompts (921b, 923b, 925b) in response to a user query being input. The electronic device (200) may output a prompt editing screen (900b) including the plurality of generated recommendation prompts (1021b, 1023b, 1025b). For example, the plurality of recommendation prompts (921b, 923b, 925b) may include, ‘① How about turtle tripe at Gyodae Station for lunch at work?’, ‘② How about turtle tripe at Gyodae Station for dinner with friends?’, or ‘③ How about turtle tripe at Gyodae Station for dinner with family?’. As an example, the user may select one (921b) among the plurality of recommendation prompts (921b, 923b, 925b) (1027b). For example, the electronic device (200) can analyze the user query, 'How about turtle tripe at Gyodae Station?', and select one or more applications (e.g., a schedule management application, a weather application, a map guidance application, an SNS application) related to the user query based on the analysis result. The electronic device (200) can, for example, simultaneously select multiple applications related to the user query. The electronic device (200) can, for example, sequentially select multiple applications related to the user query. The electronic device (200) can obtain application data to which the user is permitted to access from one or more applications. The electronic device (200) can determine search terms by analyzing the obtained application data. The electronic device (200) can additionally reflect at least one applicable search term selected from among the determined search terms in response to the user query to generate various recommendation prompts (1021b, 1023b, 1025b). For example, the electronic device (200) can select various reference applications from among the selected applications and generate various recommendation prompts (1021b, 1023b, 1025b) through this.

[0144] In one example, the electronic device (200) can obtain applicable search terms (e.g., company, lunch, Gyodae Station, turtle intestines) used in the selected recommendation prompt (921b) (e.g., “How about turtle intestines at Gyodae Station for company lunch?”). The electronic device (200) can obtain recommended search terms (e.g., dinner, near company, good restaurant) that may be considered for reconstructing the selected recommendation prompt, although they were not used in the selected recommendation prompt (921b). The electronic device (200) can generate search results corresponding to the selected recommendation prompt (921b) using the AI ​​model (430). The electronic device (200) can output a first prompt editing screen (900c) using the search results, the selected recommendation prompt (921c), applicable search terms (923c), or recommended search terms (925c) corresponding to the selected recommendation prompt (921b).

[0145] For example, the first prompt editing screen (1000c) may include a first input window (910c) or a first prompt editing window (920c). The first prompt editing window (920c) may include items for editing the first prompt (921c). For example, the first prompt editing window (920c) may display the first prompt (921c) (e.g., “How about turtle tripe at Gyodae Station for lunch?”), first applicable search terms (923c) (e.g., company, lunch, Gyodae Station, turtle tripe), or first recommended search terms (925c) (e.g., dinner, near the company, good restaurant). Near each of the first applicable search terms (923c) (e.g., company, lunch, Gyodae Station, turtle tripe), a removal identifier (e.g., an icon X) may be displayed to exclude the applicable search term when reprocessing the prompt. Near each of the first recommended search terms (925c) (e.g., dinner, near the company, good restaurant), an additional identifier (e.g., icon O) may be displayed to add the additional search term when reprocessing the prompt. The first prompt editing window (920c) may include an instruction (927) (e.g., a 'search' icon) for requesting a search for the first prompt (921c). The first prompt editing screen (900c) may include a window (e.g., search result display #2) for displaying search results for the first prompt (921c). For example, a user may select an instruction (927c) to request a search for the first prompt (921c) displayed in the first prompt editing window (920c).

[0146] When an instruction (927c) requesting a search is input in the first prompt editing screen (900c), the electronic device (200) may output a second prompt editing screen (910d) including a search result for the first prompt (921c). In one example, the second prompt editing screen (900d) may include a second prompt editing window (920d). The second prompt editing window (920d) may include items for editing the second prompt (921d). For example, the second prompt editing window (920d) may display the second prompt (921d) (e.g., “How about turtle tripe at Gyodae Station for lunch at work?”), second applicable search terms (923d) (e.g., company, lunch, Gyodae Station, turtle tripe), or second recommended search terms (925d) (e.g., dinner, near the company, good restaurant). Near each of the second applied search terms (923d), a removal identifier (e.g., icon X) may be displayed to exclude the applied search term when reprocessing the prompt. Near each of the second recommended search terms (925d), an additional identifier (e.g., icon O) may be displayed to add the recommended search term when reprocessing the prompt.

[0147] FIG. 10A or FIG. 10B is an example diagram of a user interface for displaying a search word for editing a prompt (e.g., prompt (330) of FIG. 3) in an electronic device (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2) (hereinafter referred to as 'electronic device (200)'), according to one embodiment.

[0148] Referring to FIG. 10a, the prompt editing screen (1000a) may include an input window (1010a) or a prompt editing window (1020a) capable of editing a prompt. For example, the input window (1010a) may be an interface for entering a user query (e.g., “What’s the weather like this weekend?”). For example, the input window (1010a) may be an interface for entering a search term to be added for creating or reconfiguring a prompt. The prompt editing screen (1000a) may be a user interface screen that provides editing of a prompt (1021a) (e.g., prompt (330) of FIG. 3). The prompt editing window (1020a) may display a prompt (1021a) (e.g., “What’s the weather like in Jeju Island this weekend?”) or applicable search terms for the prompt (1021a) (e.g., “Jeju Island” (1023a), “This weekend’s weather” (1025a)). The prompt (1021a) may be generated by an AI model (e.g., the generative AI model (430) of FIG. 4) in response to a user question entered in the input window (1010a). The prompt editing window (1020a) may display identifiers (1027a, 1029a) indicating applications related to the applicable search terms (1023a, 1025a). For example, the identifiers (1027a, 1029a) may be displayed near the applicable search terms (1023a, 1025a). The identifiers (1027a, 1029a) may indicate the source of the applicable search terms (1023a, 1025a).

[0149] For example, in response to a user selecting a specific identifier (1027a, 1029a) displayed in a prompt editing window (1020a), the electronic device (200) may execute an application corresponding to the selected specific identifier to output corresponding information (e.g., Jeju Island weather information or Jeju travel itinerary information).

[0150] Referring to FIG. 10b, the prompt editing screen (1000b) may include an input window (1010b) or a prompt editing window (1020b) for editing a prompt. For example, the input window (1010a) may be an interface for entering a user query (e.g., "What's the weather this weekend?"). For example, the input window (1010a) may be an interface for entering a search term to be added for creating or reconfiguring a prompt. The prompt editing screen (1000b) may be a user interface screen that provides editing of a prompt (1021b) (e.g., prompt (330) of FIG. 3). The prompt editing window (1020b) may display a prompt (1021b) (e.g., “What’s the weather like in Jeju Island this weekend?”), applicable search terms for the prompt (1021b) (e.g., “Jeju Island,” “Weather this weekend”), or recommended search terms (1023b) for restructuring the prompt (1021b) (e.g., “Phu Quoc” (1027b)). The prompt (1021a) may be generated by an AI model (e.g., the generative AI model (430) of FIG. 4) in response to a user question entered in the input window (1010a). The prompt editing window (1020b) may display an identifier (1025b) indicating an application related to the recommended search term (1027b). For example, the identifier (1025b) may be displayed near the recommended search term (1027b). The identifier (1025b) may indicate the source of the recommended search term (1027b).

[0151] For example, in response to a user selecting a specific identifier (1027a, 1029a) displayed in a prompt editing window (1020a), the electronic device (200) may execute an application corresponding to the selected specific identifier to output corresponding information (e.g., Jeju Island weather information or Jeju travel itinerary information).

[0152] As described above, the prompt editing window (1020b) is displayed to display application information related to information with low priority that was not used in prompt generation. For example, the schedule information associated with the user query "What's the weather like this weekend?" includes information about a Jeju Island travel itinerary (e.g., family) and a Phu Quoc travel itinerary (e.g., parents). In this case, "Jeju Island itinerary information," which has a relatively high priority, can be applied when generating the prompt. "Phu Quoc travel itinerary information," which has a relatively low priority, was not applied when generating the prompt, but can be suggested as a recommended search term for addition.

[0153] FIG. 11A or FIG. 11B is a diagram for explaining sequential access of an application to generate a prompt in an electronic device (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2) (hereinafter referred to as 'electronic device (200)') according to one embodiment.

[0154] The electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) can determine a target object (e.g., an application, an account system, or a setting system) to access to obtain related data based on the analysis result of the user query (560). The electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) can, for example, analyze the user query to assign target objects and priorities corresponding to the target objects. The electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) can assign priorities to target objects, for example, for the purpose of generating a prompt. For example, a relatively high priority can be assigned to a running application. For example, the electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) can assign the next priority to an application that is highly related to an application assigned a priority. The electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) can give the next priority to an application (e.g., a map guidance application) after installation when an application with a high correlation with the prioritized application is not installed.

[0155] In one example, the electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) may determine an application with the highest priority among multiple applications as a central application. Depending on which application among the multiple applications the electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) selects as a central application, the scenarios for generating a prompt may differ. For example, the electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) may prioritize running applications so that they can be selected as central applications. For example, the electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) may give a relatively high priority to a related application based on recent usage history (e.g., capture). An electronic device (200) (e.g., an AI framework (510) of FIG. 5) can analyze the properties of an additionally installed application (e.g., a map application) in response to input data (e.g., a user query) (560) and determine whether to consider the application associated with the input data based on the analyzed result.

[0156] Referring to FIG. 11A, the electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) may determine the reference application to be accessed initially as application #5 (1150) in response to a user query. The reference application may be determined by the AI ​​framework (510). The electronic device (200) may determine (e.g., identify) an application to be accessed by considering the user query and the reference application, application #5 (1150). For example, after the electronic device (200) obtains (e.g., receives) application data from the reference application, application #5 (1150), the electronic device may determine (e.g., identify) the order of applications to be accessed as 'application #3 (1130) → application #2 (1120) → application #4 (1140) → application #1 (1110)'. The access order of the applications may be determined by the AI ​​framework (510). In this case, the electronic device (200) can obtain the first application data from the application #5 (1150) to be accessed first. The electronic device (200) can obtain the second application data from the application #3 (1130) to be accessed second (①). The electronic device (200) can obtain the third application data from the application #2 (1120) to be accessed third (②). The electronic device (200) can obtain the fourth application data from the application #4 (1140) to be accessed fourth (③). The electronic device (200) can obtain the fifth application data from the application #1 (1110) to be accessed fifth (④). According to an example, the electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) can obtain (e.g., receive) application data from the corresponding application to be accessed, and analyze the obtained application data to determine (e.g., identify) the next application to be accessed.For example, an electronic device (200) (e.g., an AI framework (510)) accesses the application to obtain and analyze schedule data, and if the analysis result determines that the schedule data is a 'family dinner appointment', the electronic device (200) may determine an application (e.g., an SNS application) that can obtain information on restaurants frequently used by the family as the next application to access. An example of determining the order in which the electronic device (200) accesses a plurality of selected applications for generating a prompt has been described above, and reference may be made thereto.

[0157] Referring to FIG. 11b, the electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) may determine (e.g., identify) the reference application to be accessed first as Application #2 (1120) in response to a user query. The reference application may be determined by the AI ​​framework (510). The electronic device (200) may determine (e.g., identify) the application to be accessed by considering the user query and the reference application, Application #2 (1120). For example, after the electronic device (200) obtains (e.g., receives) application data from the reference application, Application #2 (1120), the electronic device may determine (e.g., identify) the order of applications to be accessed as 'Application #6 (1160) → Application #1 (1110) → Application #4 (1140) → Application #5 (1150)'. The access order of the applications may be determined by the AI ​​framework (510). In this case, the electronic device (200) can obtain the first application data from the application #2 (1120) to be accessed first. The electronic device (200) can obtain the second application data from the application #6 (1160) to be accessed second (①). The electronic device (200) can obtain the third application data from the application #1 (1110) to be accessed third (②). The electronic device (200) can obtain the fourth application data from the application #4 (1140) to be accessed fourth (③). The electronic device (200) can obtain the fifth application data from the application #5 (1150) to be accessed fifth (④). According to an example, the electronic device (200) (e.g., the AI ​​framework (510) of FIG. 5) can obtain application data from the corresponding application to be accessed and analyze the obtained application data to determine the next application to be accessed.For example, an electronic device (200) (e.g., AI framework (510)) accesses the application to obtain and analyze weather data, and if the analysis result determines that 'there is a high probability of precipitation', the electronic device (200) can then access an application capable of obtaining schedule data to analyze whether there is a schedule that requires a change due to rain.

[0158] For example, the electronic device (200) (e.g., AI framework (510)) may analyze application data corresponding to each application accessed in a predetermined order and obtain (e.g., receive) related data based on the semantic distance from the user query. The electronic device (200) (e.g., AI framework (510)) may determine search terms (e.g., applicable search terms and / or recommended search terms) by referring to the related data.

[0159] In one example, the electronic device (200) (e.g., AI framework (510)) may generate a prompt (e.g., prompt #a (1170) or prompt #b (1180)) that can be predicted (e.g., identified) from a user query based on a determined search term. For example, the user query may be 'Recommend a good restaurant near Yangjae Station', and prompt #a (1170) generated by reflecting the applicable search term included in the search term may be 'Recommend a good restaurant near Yangjae Station that a family of four can go to for lunch on a rainy day'.

[0160] According to one example, the electronic device (200) (e.g., the generative AI model (530)) can output a user query, a prompt (e.g., prompt #a (1170) or prompt #b (1180)), and a search term (e.g., an applicable search term and / or a recommended search term) through the display (550). The electronic device (200) (e.g., the generative AI model (530)) can also output a processing result for the user query through the display (550). Detailed examples thereof will be described later with reference to FIGS. 7A to 7E .

[0161] For example, if a user requests a search for a prompt (e.g., prompt #a (1170) or prompt #b (1180)) output through the display (550), the electronic device (200) (e.g., generative AI model (530)) may output a processing result for the prompt (e.g., prompt #a (1170) or prompt #b (1180)) through the display (550). Detailed examples thereof will be described later with reference to FIGS. 7A to 7E.

[0162] For example, if a user requests removal of an applicable search term output through the display (550), the electronic device (200) (e.g., AI framework (510)) may reconfigure a prompt (e.g., prompt #a (1170) or prompt #b (1180)) excluding the removed search term. The electronic device (200) (e.g., generative AI model (530)) may output the user query, the reconfigured prompt, the applicable search term without the removed search term, and / or the recommended search term through the display (550). The electronic device (200) (e.g., generative AI model (530)) may also output the processing result for the reconfigured prompt through the display (550). Detailed examples thereof will be described later with reference to FIGS. 7A to 7E.

[0163] For example, if a user requests addition of a recommended search term output through the display (550), the electronic device (200) (e.g., AI framework (510)) may reconfigure a prompt (e.g., prompt #a (1170) or prompt #b (1180)) by applying the additional search term. The electronic device (200) (e.g., generative AI model (530)) may output the user query, the reconfigured prompt, the applied search term including the additional search term, and / or the recommended search term without the additional search term through the display (550). The electronic device (200) (e.g., generative AI model (530)) may also output the processing result for the reconfigured prompt through the display (550). Detailed examples thereof will be described later with reference to FIGS. 7A to 7E.

[0164] As described above, and as can be seen in FIGS. 11A and 11B , the electronic device (200) can generate different prompts depending on the reference application and the determination (e.g., identification) of the access order. An example of determining (e.g., identification) the order in which the electronic device (200) accesses a plurality of applications selected for prompt generation has been previously described, and reference may be made thereto.

[0165] FIG. 11C is a diagram for explaining generating a prompt by a sequential application in an electronic device (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2) (hereinafter referred to as 'electronic device (200)') according to one embodiment.

[0166] Referring to FIG. 11c, the electronic device (200) (e.g., AI framework (510)) may select a first application as a reference application to obtain related data based on input data (560) which is a user query (1181). The electronic device (200) (e.g., AI framework (510)) may access the first application to obtain the first related data and analyze the obtained first related data (1182). The electronic device (200) (e.g., AI framework (510)) may determine (e.g., identify) an additional application to access next based on the analysis result (1183).

[0167] When a second application to be accessed is selected, the electronic device (200) (e.g., AI framework (510)) can access the second application, acquire second-related data, and analyze the acquired second-related data (1184, 1185). If there is no additional application to be accessed based on the analysis results, the electronic device (200) (e.g., AI framework (510)) can generate a prompt based on the first-related data and the second-related data (1186).

[0168] FIG. 11d is a diagram for explaining the sequential processing of a user query or prompt using multiple LLMs in an electronic device (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2) (hereinafter referred to as 'electronic device (200)') according to one embodiment.

[0169] Referring to FIG. 11d, the electronic device (200) (e.g., generative AI model (530)) can operate multiple LLM models (1191, 1192, 1193). The multiple LLM models (1191, 1192, 1193) can be operated on-device, on a server, or distributedly operated by the on-device and the server. In one example, the electronic device (200) (e.g., generative AI model (530)) can sequentially use the LLM models (1191, 1192, 1193) based on data to be processed. In one example, the electronic device (200) (e.g., generative AI model (530)) can be arranged to process data by distinguishing data by LLM model (1191, 1192, 1193). For example, LLM#1 (1191) and LLM#2 (1192) can be arranged to separately process personal data, and LLM#3 (1193) can be arranged to process public data. In this case, the electronic device (200) (e.g., generative AI model (530)) can sequentially process personal data and public data in the processing order by LLM#1 (1191), LLM#2 (1192), and LLM#3 (1193).

[0170] FIGS. 12A to 12C are exemplary diagrams showing a prompt editing screen displayed using an extended display in an electronic device (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2) (hereinafter referred to as “electronic device (200)”). The prompt editing screen may be a user interface screen that provides editing of a prompt (e.g., the prompt (330) of FIG. 3).

[0171] Referring to FIG. 12a, an electronic device (200) of a flexible display type (e.g., fold, slide, etc.) that can be unfolded in a horizontal (or landscape) direction to expand the display area of ​​the display can display a prompt editing screen in a basic display area (e.g., left display area) (1210a) regardless of whether it is expanded. The electronic device (200) can display a search result output screen in an expanded display area (e.g., right display area) (1220a) that can be activated when the flexible display is in an expanded state (e.g., unfolded state).

[0172] Referring to FIG. 12b, a flip-type electronic device (200), which is an example of a flexible (flip-able) display that can be unfolded in the axial (or vertical) direction to expand the display area of ​​the display, may have a display format and / or layout that can change depending on the flip / unflip (e.g., unexpanded / expanded) or foldable state. This may take into account that the display layout and / or shape changes depending on the flip / unflip or foldable state.

[0173] A prompt editing screen can be displayed in a first display area (e.g., upper display area) (1211b) in an expanded state (e.g., un-flipped state). A flip-type electronic device (200) can display a search result output screen in a second display area (e.g., lower display area) (1213b) in an expanded state (e.g., un-flipped state) of the first expanded display area (1210b).

[0174] A flip-type electronic device (200) can switch the screen layout through interaction with a user. In one example, the flip-type electronic device (200) can display a prompt editing screen that was being displayed in a first display area (e.g., upper display area) (1211b) in an expanded state (e.g., un-flipped state) by moving it to a second display area (e.g., lower display area) (1223b). The flip-type electronic device (200) can display a search result output screen that was being displayed in a second display area (e.g., lower display area) (1213b) in an expanded state (e.g., un-flipped state) by moving it to a first display area (e.g., upper display area) (1221b) of a second expanded display area (1220b).

[0175] Referring to FIG. 12c, a rollable type electronic device (200) that can expand a display area of ​​a display by sliding in an axial (or vertical) direction can display a first prompt editing screen (1211c) including some items for prompt editing (e.g., a search word input window) in a first display area (e.g., a basic display area) (1210c) before expansion. The rollable type electronic device (200) can display a second prompt editing screen (1221c, 1223c) including all items for prompt editing (e.g., a search word input window, a prompt, an applied search word, a recommended search word) in a second display area (1220c) after expansion.

[0176] FIG. 13 is an example diagram of a prompt editing screen in an electronic device (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2) (hereinafter referred to as 'electronic device (200)') according to one embodiment.

[0177] Referring to FIG. 13, the electronic device (200) may include a display having a size that can be used by dividing the display area, such as an expandable display or tablet. In this case, the display may provide a display area (1300) with different layouts depending on whether it is expanded or the screen is divided. For example, a fold-type electronic device (200) that can expand the display area of ​​the display by unfolding in a horizontal (or horizontal) direction may provide a first display area (1310) in an unexpanded state (e.g., a folded state). The first display area (1310) may also display, for example, information for prompt editing (e.g., a user query, a prompt, an applicable search term, or a recommended search term) and identification information of applications that have provided application data analyzed to obtain the corresponding information.

[0178] For example, a fold type electronic device (200) may provide a first display area (1310) and a second display area (1320) in an expanded state (e.g., an unfolded state). For example, the electronic device (200) may output a prompt editing screen (1311) through an expanded display. The prompt editing screen (1311) may be a user interface screen that provides editing of a prompt (e.g., the prompt (330) of FIG. 3). Since the exemplary prompt editing screen (1311) has been sufficiently described above, the arrangement for displaying detailed items of the prompt editing screen (1311) on the expanded display will be described below.

[0179] According to one example, the electronic device (200) may output a first screen (1311) in a first display area (e.g., a left display area) (1310) included in an extended display area (1300) that may be selected to remove or add search terms (e.g., applicable search terms and / or recommended search terms) for reconfiguring a prompt (e.g., prompt (330) of FIG. 3).

[0180] According to one example, the electronic device (200) may output at least one second screen (1321, 1323) that can utilize an application (e.g., a weather providing application and / or a schedule management application) related to search words (e.g., applicable search words and / or recommended search words) for reconfiguring the prompt (330) in another second display area (e.g., a right display area) (1320) included in the extended display area (1300).

[0181] According to one example, the electronic device (200) may output a second screen (1321, 1323), which is an execution screen of an application (e.g., a weather providing application and / or a schedule management application), to an expanded display area (e.g., a right display area) (1320) in response to a change of the display from a non-expanded state (e.g., a folded state) to an expanded state (e.g., an unfolded state). The application for which the execution screen is output may be related to search words (e.g., applicable search words and / or recommended search words) for reconfiguring the prompt (330), for example. Regardless of whether the display is expanded, the first screen (1311) may continue to be output to a basic display area (e.g., a left display area (1310)) capable of screen output.

[0182] In one example, the electronic device (200) may no longer output the second screen (1321, 1323) displayed in the expanded display area (e.g., right display area) (1320) in response to the display changing from an expanded state (e.g., unfolded state) to an unexpanded state (e.g., folded state). However, the first screen (1311) may continue to be output in the basic display area (e.g., left display area (1310)) where screen output is possible regardless of whether the display is expanded.

[0183] According to one example, if the electronic device (200) has a display size sufficient to arrange the first display area (1310) and the second display area (1320), implementation according to the proposed screen arrangement may be possible regardless of whether the display is expanded.

[0184] FIG. 14 is an exemplary diagram of a user interface for supporting the use of customized search results in an electronic device (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2) (hereinafter referred to as 'electronic device (200)') according to one embodiment.

[0185] Referring to FIG. 14, the electronic device (200) may display search results generated by an AI model (e.g., the generative AI model (430) of FIG. 4) based on a customized search prompt generated based on a customized recommendation search in a search result display area (1430) allocated as part of a display area (1400) of the display. The customized search prompt corresponds to a prompt that is reprocessed from an initial prompt generated in response to a user query by the search word editing method proposed in this document.

[0186] According to an example, the electronic device (200) may display one or more identifiers (e.g., icons) (1441, 1443) for using the search results for other purposes in a partial area (1440) allocated to the display area (1400). The one or more identifiers (1441, 1443) may include, for example, an identifier (1441) that may utilize a specific application (e.g., contacts) that provides a function to share the search results with colleagues. The one or more identifiers (1441, 1443) may include, for example, an identifier (1443) that may utilize a specific application (e.g., calendar) that provides a function to save the search results. In this case, it may be convenient for the user to utilize the search results for other purposes by using a predetermined application. The one or more identifiers may also include an identifier for indicating information about another device (e.g., my account device) that is connected to the communication.

[0187] According to an example, an electronic device (200) may include a display (240). The electronic device (200) may include a memory (220) including one or more storage media for storing instructions. The electronic device (200) may include at least one processor (210) including a processing circuit. When the instructions are individually or collectively executed by the at least one processor (210), the instructions may cause the electronic device (200) to perform at least one operation. The at least one operation may include an operation of obtaining a user query (operation 611). The at least one operation may include an operation of accessing at least one application associated with the user query to obtain personal use data. The at least one operation may include an operation of obtaining at least one search term associated with the user query from the obtained personal use data (operation 615). The at least one operation may include an operation (operation 617) of tuning the user query in a specific AI model (e.g., LLM) based on the at least one acquired search term to generate at least one prompt. The at least one operation may include an operation of displaying a user interface screen in which the user query, the at least one generated prompt, and the at least one acquired search term are arranged according to a specific layout.

[0188] In one example, at least one action may include analyzing personal usage data and / or log data to which access is permitted in response to a user query.

[0189] For example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation such that the user interface screen includes data (e.g., user query data) resulting from processing the user query by the specific AI model.

[0190] In one example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation of analyzing the user query.

[0191] In one example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation of determining the at least one application from among the installed applications based on the analysis result of the user query.

[0192] In one example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to access the at least one application and perform an operation to obtain the personal usage data based on a semantic distance from the user query.

[0193] For example, when the instructions are individually or collectively executed by the at least one processor (210), the electronic device (200) may be caused to perform an operation of accessing the applications in consideration of priorities of the applications in order to sequentially access the applications when a plurality of applications are determined (e.g., based on a plurality of applications that are determined or identified) based on the analysis results of the user query.

[0194] In one example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation of analyzing first personal usage data acquired by accessing a first application included in the at least one application to determine a second application.

[0195] In one example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation of accessing the determined second application and obtaining second personal usage data.

[0196] In one example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation such that the user interface screen includes data (e.g., prompt data) resulting from processing the at least one prompt generated by the specific AI model.

[0197] According to an example, the at least one search term obtained may include at least one applicable search term applied to the at least one generated prompt or at least one recommended search term not applied to the at least one generated prompt.

[0198] In one example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation of determining at least one additional search term from among the one or more recommended search terms.

[0199] In one example, the instructions, when individually or collectively executed by the at least one processor (210), may cause the electronic device (200) to perform an operation of tuning the at least one generated prompt based on the determined at least one additional search term.

[0200] In one example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation of determining at least one removal search term from among the one or more applicable search terms.

[0201] In one example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation of tuning the at least one generated prompt based on the at least one determined removal search term.

[0202] In one example, the acquired personal use data may be data that the user has permitted to be used for at least one application.

[0203] For example, when the instructions are individually or collectively executed by the at least one processor (210), the electronic device (200) may perform an operation of outputting, through the display (240), a user interface for selecting at least one prompt from among the plurality of prompts, when a plurality of prompts are generated.

[0204] For example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation of outputting information about the at least one application through the display (240).

[0205] In one example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation of determining the number of applications to access to obtain the personal usage data, taking into account priorities assigned to the applications based on the user's query.

[0206] For example, when the instructions are individually or collectively executed by the at least one processor (210), they may cause the electronic device (200) to perform an operation of outputting information about another device through the display (240) when data used to obtain the at least one search term is provided from the other device.

[0207] According to one example, a storage medium storing computer-readable instructions may be provided. The instructions, when executed by at least a portion of at least one processor of an electronic device, may cause the electronic device to perform at least one operation. The at least one operation may include obtaining a user query (operation 611). The at least one operation may include accessing at least one application associated with the user query to obtain personal use data. The at least one operation may include obtaining at least one search term associated with the user query from the obtained personal use data (operation 615). The at least one operation may include tuning the user query in a specific AI model (e.g., LLM) based on the at least one obtained search term to generate at least one prompt (operation 617). The at least one action may include displaying a user interface screen in which the user query, the at least one generated prompt, and the at least one obtained search term are arranged according to a specific layout.

[0208] In one example, the action of displaying the user interface screen may include an action of causing the user interface screen to include data resulting from processing the user query by the specific AI model.

[0209] In one example, the act of obtaining the personal use data may include the act of analyzing the user query.

[0210] In one example, the act of obtaining the personal use data may include an act of determining at least one application from among installed applications based on the analysis results of the user query.

[0211] In one example, the act of obtaining the personal use data may include accessing the at least one application and obtaining the personal use data based on a semantic distance from the user query.

[0212] In one example, the operation of obtaining the personal use data may include an operation of accessing the application in consideration of the priority of the multiple applications in order to sequentially access the multiple applications when the multiple applications are determined based on the analysis results of the user query.

[0213] In one example, the act of obtaining the personal use data may include an act of accessing a first application included in the at least one application, analyzing the obtained first personal use data, and determining a second application.

[0214] In one example, the act of obtaining the personal use data may include an act of accessing the determined second application to obtain second personal use data.

[0215] In one example, the act of displaying the user interface screen may include an act of causing the user interface screen to include data resulting from processing at least one prompt generated by the specific AI model.

[0216] According to an example, the at least one search term obtained may include at least one applicable search term applied to the at least one generated prompt or at least one recommended search term not applied to the at least one generated prompt.

[0217] In one example, the at least one action may include determining at least one additional search term from among the one or more recommended search terms.

[0218] In one example, the at least one action may include tuning the at least one generated prompt based on the at least one determined additional search term.

[0219] In one example, the act of generating the at least one prompt may include the act of determining at least one removal search term from among the one or more applied search terms.

[0220] In one example, the act of generating the at least one prompt may include the act of tuning the at least one generated prompt based on the determined at least one removed search term.

[0221] In one example, the acquired personal use data may be data that the user has permitted to be used for at least one application.

[0222] In one example, the act of displaying the user interface screen may include, when a plurality of prompts are generated, an act of outputting a user interface that allows selection of at least one prompt from among the plurality of prompts through the display (240).

[0223] In one example, the act of displaying the user interface screen may include an act of outputting information about the at least one application through the display (240).

[0224] In one example, the at least one operation may include determining the number of applications to access to obtain the personal usage data, taking into account priorities assigned to the applications based on the user's query.

[0225] In one example, the act of displaying the user interface screen may include an act of outputting information about another device through the display (240) when data used to obtain the at least one search term is provided from the other device.

[0226] According to an example, a method of operating an electronic device (200) may include an operation of obtaining a user query (operation 611). The method may include an operation of accessing at least one application associated with the user query to obtain personal use data. The method may include an operation of obtaining at least one search term associated with the user query from the obtained personal use data (operation 615). The method may include an operation of tuning the user query in a specific AI model (e.g., LLM) based on the at least one obtained search term to generate at least one prompt (operation 617). The method may include an operation of displaying a user interface screen in which the user query, the at least one generated prompt, and the at least one obtained search term are arranged according to a specific layout.

[0227] In one example, the method of operation may include an operation of displaying the user interface screen, wherein the user interface screen includes data resulting from processing the user query by the specific AI model.

[0228] In one example, the act of obtaining the personal use data may include the act of analyzing the user query.

[0229] In one example, the act of obtaining the personal use data may include an act of determining at least one application from among installed applications based on the analysis results of the user query.

[0230] In one example, the act of obtaining the personal use data may include accessing the at least one application and obtaining the personal use data based on a semantic distance from the user query.

[0231] In one example, the operation of obtaining the personal use data may include an operation of accessing the application in consideration of the priority of the multiple applications in order to sequentially access the multiple applications when the multiple applications are determined based on the analysis results of the user query.

[0232] In one example, the act of obtaining the personal use data may include an act of accessing a first application included in the at least one application, analyzing the obtained first personal use data, and determining a second application.

[0233] In one example, the act of obtaining the personal use data may include an act of accessing the determined second application to obtain second personal use data.

[0234] In one example, the act of displaying the user interface screen may include an act of causing the user interface screen to include data resulting from processing at least one prompt generated by the specific AI model.

[0235] According to an example, the at least one search term obtained may include at least one applicable search term applied to the at least one generated prompt or at least one recommended search term not applied to the at least one generated prompt.

[0236] In one example, the method may include determining at least one additional search term from among the one or more recommended search terms.

[0237] In one example, the method may include tuning the generated at least one prompt based on the determined at least one additional search term.

[0238] In one example, the act of generating the at least one prompt may include the act of determining at least one removal search term from among the one or more applied search terms.

[0239] In one example, the act of generating the at least one prompt may include the act of tuning the at least one generated prompt based on the determined at least one removed search term.

[0240] In one example, the acquired personal use data may be data that the user has permitted to be used for at least one application.

[0241] In one example, the act of displaying the user interface screen may include, when a plurality of prompts are generated, an act of outputting a user interface that allows selection of at least one prompt from among the plurality of prompts through the display (240).

[0242] In one example, the act of displaying the user interface screen may include an act of outputting information about the at least one application through the display (240).

[0243] In one example, the method may include determining the number of applications to access to obtain the personal usage data, taking into account priorities assigned to the applications based on the user's query.

[0244] In one example, the act of displaying the user interface screen may include an act of outputting information about another device through the display (240) when data used to obtain the at least one search term is provided from the other device.

[0245] Electronic devices according to one or more embodiments disclosed herein may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments disclosed herein are not limited to the aforementioned devices.

[0246] The various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to a specific embodiment, but should be understood to include various modifications, equivalents, or substitutes of the embodiment. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0247] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or portion of a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0248] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., a memory (220)) readable by a machine (e.g., an electronic device (200)). For example, a processor (e.g., a processor (210)) of a machine (e.g., an electronic device (200)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not include a signal (e.g., an electromagnetic wave), and this term is used when data is semi-permanently stored in the storage medium. It does not distinguish between cases where it is temporarily stored.

[0249] According to one embodiment, the methods according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0250] According to one or more embodiments, each component (e.g., a module or a program) of the described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (200), display (240); A memory (220) including one or more storage media for storing instructions; and At least one processor (210) comprising a processing circuit, When the above instructions are executed individually or collectively by at least one processor (210), Obtaining a user query (action 610); Accessing at least one application associated with the user query to obtain personal use data; Obtaining at least one search term associated with the user query from the acquired personal usage data (operation 620); Generating at least one prompt by tuning the user query in an artificial intelligence (AI) model based on at least one search term obtained above (operation 630); A user interface screen in which the user query, at least one generated prompt, and at least one acquired search word are arranged in a layout is displayed through the display (240). An electronic device (200) that performs an action.

2. In paragraph 1, When the above instructions are executed individually or collectively by at least one processor (210), Displaying user query data generated by processing the user query in the AI ​​model or prompt data generated by processing at least one generated prompt on the user interface screen. An electronic device (200) that performs an action.

3. In paragraph 1 or 2, When the above instructions are executed individually or collectively by at least one processor (210), Analyze the above user queries; Identifying at least one application among the installed applications based on the analysis results of the above user query; Accessing at least one application identified above and obtaining the personal usage data based on the semantic distance from the user query. Performing the action, The personal usage data obtained above is data that the user has permitted to use for at least one application, an electronic device (200).

4. In paragraph 1 or paragraph 3, When the above instructions are executed individually or collectively by at least one processor (210), In order to sequentially access a plurality of applications based on the plurality of applications identified based on the analysis results of the user query, access to the application is made by considering the priority of the plurality of applications. An electronic device (200) that performs an action.

5. In any one of paragraphs 1 to 4, When the above instructions are executed individually or collectively by at least one processor (210), Identifying a second application by analyzing first personal usage data obtained by accessing a first application included in at least one of the above applications; Access the second application identified above to obtain second personal usage data. An electronic device (200) that performs an action.

6. In any one of paragraphs 1 to 5, When the above instructions are executed individually or collectively by at least one processor (210), A user interface for selecting at least one prompt from among the plurality of prompts is output through the display (240) based on the generation of multiple prompts; Outputting information about at least one application through the display (240) An electronic device (200) that performs an action.

7. In any one of paragraphs 1 to 6, When the above instructions are individually or collectively executed by the at least one processor (210), Identify the number of applications to access to obtain the personal usage data, taking into account the priority assigned to the applications based on the user's query. An electronic device (200) that performs an action.

8. In any one of paragraphs 1 to 7, When the above instructions are individually or collectively executed by the at least one processor (210), Outputting information about the other device through the display (240) based on the data used to obtain at least one search term provided from the other device Performing the action, An electronic device (200), wherein at least one search word obtained above includes at least one application search word applied to at least one generated prompt.

9. In any one of paragraphs 1 to 7, When the above instructions are individually or collectively executed by the at least one processor (210), Outputting information about the other device through the display (240) based on the data used to obtain at least one search term provided from the other device Performing the action, An electronic device (200), wherein at least one of the acquired search words includes at least one recommendation search word that is not applied to at least one of the generated prompts.

10. In paragraph 8 or 9, When the above instructions are executed individually or collectively by at least one processor (210), Identifying at least one additional search term among the at least one recommended search term; Tuning at least one generated prompt based on at least one additional search term identified above An electronic device (200) that performs an action.

11. In paragraph 8 or 9, When the above instructions are executed individually or collectively by at least one processor (210), Identifying at least one removal search term among the at least one applicable search term; Tuning at least one generated prompt based on at least one removed search term identified above An electronic device (200) that performs an action.

12. In a non-transitory computer-readable recording medium storing at least one computer-readable instruction, The at least one computer-readable instruction, when executed by at least one processor (210) of the electronic device (200), causes the electronic device (200) to perform at least one operation; At least one of the above actions: Action to obtain a user query (action 610); An action of accessing at least one application associated with the user query to obtain personal use data; An action of obtaining at least one search term associated with the user query from the acquired personal usage data (action 620); An action of tuning the user query in an artificial intelligence (AI) model based on at least one search term obtained above to generate at least one prompt (action 630); and An action of displaying a user interface screen in which the above user query, at least one generated prompt, and at least one obtained search term are arranged in a layout. A recording medium including:

13. In paragraph 12, A recording medium, wherein said at least one computer-readable instruction, when executed by at least one processor (210) of said electronic device (200), causes said electronic device to perform one or more operations described in at least one of claims 3 to 11.

14. In the operating method of the electronic device (200), Action to obtain a user query (action 610); An action of accessing at least one application associated with the user query to obtain personal use data; An action of obtaining at least one search term associated with the user query from the acquired personal usage data (action 620); An action of tuning the user query in an artificial intelligence (AI) model based on at least one search term obtained above to generate at least one prompt (action 630); and An action of displaying a user interface screen in which the above user query, at least one generated prompt, and at least one obtained search term are arranged in a layout. A method comprising:

15. In paragraph 14, A method comprising one or more operations performed by the electronic device (200) described in at least one of claims 2 to 11.

Images