Electronic device, method, and non-transitory computer-readable recording medium for providing recommended task

Abstract

Provided are an electronic device for providing a recommended task and a control method therefor. The electronic device according to an embodiment of the present disclosure may be configured to store instructions for: executing intelligent assistance on the basis of a first user input for executing the intelligent assistance; providing, through the display, at least one recommended task determined on the basis of context information related to the electronic device in response to the execution of the intelligent assistance; acquiring, through the display, a second user input for selecting any one recommended task from among the at least one recommended task; executing, through the intelligent application, a task corresponding to the recommended task in the background on the basis of the acquisition of the second user input; and providing, through the display, a notification message indicating that the task has been completed on the basis of the completion of the task executed in the background.

Description

Electronic device, method, and non-transitory computer-readable recording medium for providing a recommended task

[0001] The present disclosure relates to an electronic device, a method, and a non-transitory computer-readable recording medium for providing a recommendation task.

[0002] The variety of services and additional features offered through electronic devices, such as smartphones, is steadily increasing. To enhance the utility of these devices and satisfy the diverse needs of users, telecommunications service providers and electronic device manufacturers are competitively developing electronic devices to offer a variety of features and differentiate themselves from competitors. Consequently, the various functions offered through electronic devices are also becoming increasingly sophisticated.

[0003] The above information may be provided as background information to aid in understanding the present disclosure. None of the above is claimed to be prior art related to the present disclosure or can be used to determine prior art related to the present disclosure.

[0004] Conventional techniques have required that an AI assistant implemented in an electronic device, such as a voice assistant service, receive all required parameters from the user in order for the electronic device to execute a specified task through the AI ​​assistant. Furthermore, conventional techniques may prevent the user from performing other tasks in parallel while the requested task is being performed. Furthermore, since all processes / operations performed by the AI ​​assistant are provided through the mobile device's screen (e.g., full screen), the task being performed by the mobile device may be interrupted, and new tasks may not be performed until all processes / operations performed by the AI ​​assistant are completed.

[0005] According to one embodiment of the present disclosure, an electronic device may be provided that provides a user with task candidates that are likely to be performed at the current time through context-based inference, and performs actions in the background to perform a task selected by the user from among the provided task candidates.

[0006] According to one embodiment of the present disclosure, a method for controlling an electronic device may be provided that provides a user with task candidates that are highly likely to be performed at the current time through context-based inference, and performs actions in the background to perform a task selected by the user from among the provided task candidates.

[0007] An electronic device according to one embodiment of the present disclosure includes a display; at least one hardware processor; and a memory, wherein the memory may be configured to store instructions that, when executed by the at least one hardware processor, cause the electronic device to: execute an intelligent assistance based on a first user input for executing the intelligent assistance; provide, through the display, at least one recommended task determined based on context information related to the electronic device in response to the execution of the intelligent assistance; obtain, through the display, a second user input for selecting a recommended task from among the at least one recommended task; execute, in the background, a task corresponding to the recommended task based on the acquisition of the second user input; and provide, through the display, a guidance message indicating that the task has been completed based on completion of the task executed in the background.

[0008] An electronic device according to one embodiment of the present disclosure includes at least one hardware processor and a memory, wherein the memory is configured to store instructions that, when executed by the at least one hardware processor, cause the electronic device to present a first user interface corresponding to a first application, detect an input specified through the first user interface, obtain at least one recommended task based on the specified input, identify the at least one recommended task based on context information including the first user interface, use a trained machine learning model to generate a series of operations for executing each of the at least one recommended task, provide a list of tasks including the at least one recommended task through the first user interface, and, in response to selection of the at least one recommended task, perform at least some of the series of operations in the background.

[0009] A method for controlling an electronic device according to one embodiment of the present disclosure may include providing a first user interface corresponding to a first application, detecting a specified input through the first user interface, obtaining at least one task based on the specified input, each of the at least one task being executed by an operation sequence generated in the electronic device based on context information including the first user interface, executing a selected task among the at least one task, and providing a second user interface corresponding to a second application during at least a portion of the execution of the selected task, and displaying a notification indicating an execution status of the selected task on the second user interface.

[0010] According to one embodiment of the present disclosure, at least one recommended task is provided based on contextual information related to the electronic device, so that an electronic device can be provided that can perform a specified task through an artificial intelligence model even when essential parameters for performing the specified task are not obtained from a user.

[0011] According to one embodiment of the present disclosure, an electronic device may be provided in which a task specified by a user is performed in the background, so that the electronic device can perform another task specified by the user even while performing the specified task.

[0012] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0013] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments of the present disclosure.

[0014] FIG. 2 is an exemplary diagram illustrating a function or operation of an electronic device according to one embodiment of the present disclosure, which provides at least one recommended task based on contextual information related to the electronic device and executes a recommended task selected by a user.

[0015] FIG. 3 is an exemplary block diagram of an electronic device according to one embodiment of the present disclosure, wherein the electronic device is configured to perform a function or operation of providing at least one recommended task and executing a recommended task selected by a user.

[0016] FIG. 4 is an exemplary diagram illustrating a function or operation of an electronic device according to one embodiment of the present disclosure to generate a prompt based on contextual information and provide at least one recommended task based on the generated prompt.

[0017] FIG. 5 is an exemplary drawing for explaining a function or operation of an electronic device according to one embodiment of the present disclosure to perform task planning according to a user's selection input for at least one recommended task.

[0018] FIG. 6 is an exemplary diagram illustrating a function or operation of an electronic device according to one embodiment of the present disclosure to determine whether at least one required parameter is provided and to execute a designated task in the background based on the determination.

[0019] FIG. 7 is an exemplary drawing for explaining, from a user interface perspective, a function or operation of an electronic device according to one embodiment of the present disclosure transmitting information about an article provided through an Internet browser to a counterpart device.

[0020] FIG. 8 is an exemplary drawing for explaining a function or operation of adding a schedule from a user interface perspective after a user of an electronic device according to one embodiment of the present disclosure ends a call with a user of a counterpart device.

[0021] FIG. 9 is an exemplary drawing for explaining a function or operation of an electronic device according to one embodiment of the present disclosure to perform remittance to a user of a counterpart device from a user interface perspective.

[0022] FIG. 10A, FIG. 10B, and FIG. 10C are exemplary drawings for explaining a function or operation of sharing reservation information with a counterpart device and adding a schedule to the electronic device from a user interface perspective, according to an embodiment of the present disclosure.

[0023] FIG. 11 is an exemplary drawing for explaining, from a user interface perspective, a function or operation of an intelligent assistant continuously used during execution of a specified application by an electronic device according to an embodiment of the present disclosure.

[0024] FIG. 12 is an exemplary drawing for explaining a function or operation of providing at least one recommended task and summarizing a new message from a user interface perspective when a message application is executed through a notification of the message application, in an electronic device according to one embodiment of the present disclosure.

[0025] FIG. 13 is an exemplary drawing for explaining, from a user interface perspective, a function or operation of an intelligent assistant continuously used during execution of a specified application by an electronic device according to one embodiment of the present disclosure.

[0026] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0027] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments.

[0028] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0029] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0030] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0031] The number of processors (120) may be one or more. For example, the processor (120) may have a multi-core processor structure such as a dual core, quad core, or hexa core.

[0032] The processor (120) can control the operations of the electronic device (101) by executing instructions stored in the memory (130). For example, the processor (120) can correspond to a plurality of processors that collectively perform a plurality of operations by dividing them among the processors.

[0033] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0034] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0035] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0036] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0037] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0038] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0039] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0040] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0041] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0042] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0043] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0044] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0045] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0046] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0047] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0048] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0049] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0050] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0051] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0052] FIG. 2 is an exemplary drawing for explaining a function or operation of an electronic device (101) according to one embodiment of the present disclosure, providing at least one recommended task (e.g., at least one recommended task (720) of FIG. 7) based on contextual information related to the electronic device (101), and executing a recommended task selected by a user.

[0053] The flowchart illustrated in FIG. 2 is merely a flowchart according to one embodiment of the operation of the electronic device (101), and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 210 to 250 may be performed by at least one processor (120) of the electronic device (101).

[0054] Referring to FIG. 2, an electronic device (101) according to an embodiment of the present disclosure may, in operation 210, execute an intelligent assistant (e.g., Samsung® Bixby™) based on a first user input for executing the intelligent assistant. Although the present disclosure describes that the intelligent assistant is executed for functions or operations related to a recommended task, this is for convenience of explanation, and at least some of the functions or operations related to identification, suggestion, and / or execution of tasks according to various embodiments may be performed regardless of the execution of the intelligent assistant. An intelligent assistant according to an embodiment of the present disclosure may include hardware and / or software modules capable of two-way conversation with a user of the electronic device based on a learning model (e.g., artificial intelligence).

[0055] An electronic device (101) according to one embodiment of the present disclosure can obtain a first user input regardless of a current state of the electronic device (101) (e.g., whether the electronic device (101) is currently displaying a screen). The first user input according to one embodiment of the present disclosure can include, for example, at least one of a touch gesture for the electronic device (101) (e.g., a double tap gesture for a touchscreen display, a long press and / or an edge sliding gesture for a designated area), a voice command (e.g., “Hi Bixby!”), and an input for a hardware button (e.g., a volume control button) provided on the electronic device (101).

[0056] An electronic device (101) according to an embodiment of the present disclosure may, in response to the execution of an intelligent assistant, provide at least one recommended task (e.g., recommended task (720) of FIG. 7) determined based on context information related to the electronic device (101) in operation 220. An electronic device (101) according to an embodiment of the present disclosure may obtain context information related to the electronic device (101) in order to provide at least one recommended task (e.g., recommended task (720) of FIG. 7).Context information according to one embodiment of the present disclosure includes information about at least one application running in the foreground (e.g., a name of at least one application running in the foreground, a category to which at least one application running in the foreground belongs, and / or a description of at least one application running in the foreground), screen context information (e.g., information about an input UI object included in a screen currently being displayed through the electronic device (101), information about a text included in a screen currently being displayed through the electronic device (101), information about an image included in a screen currently being displayed through the electronic device (101), information about an available function included in a screen currently being displayed through the electronic device (101), activity information performed by the user in the past (e.g., in the past month, after the electronic device (101) was switched from an off state to an on state, or after the electronic device (101) was connected to a specific network) based on the current point in time (e.g., information about the time at which a specified application (e.g., an application that was run in the past) was run, information about a location at which a specified application was run, information about a specified Information about functions that have been executed in the past in relation to an application, information about applications that have been executed, and / or information about texts input by a user), and / or information about operable actions of at least one or more applications installed on the electronic device (101) (e.g., information about a schedule registration action, information about a notification registration action, information about whether a camera can be executed, information about whether a message can be sent, information about a battery status, and / or information about whether a wireless communication (e.g., Wi-Fi®) is connected). "Some of the at least one or more applications may include applications that have been installed without user intervention.At least one application according to one embodiment of the present disclosure may be determined based on current context information (e.g., screen context information, user activity information, device status information).

[0057] At least one recommended task (720) according to one embodiment of the present disclosure may include at least one action that performs at least one application function. An "action" according to one embodiment of the present disclosure may include at least one "function of an application." At least one recommended task (720) according to one embodiment of the present disclosure may mean a set of at least one action that may be performed simultaneously or sequentially on the electronic device (101). According to one embodiment, the term "action" may refer to a function provided by at least one application, and the term "operation" may refer to each step constituting a procedure for performing a task. Each step (e.g., operation) constituting a procedure for performing a task may include one or more actions. According to one embodiment, one operation may correspond to one action. According to one embodiment, "action" and "operation" may be used interchangeably / substitutely.

[0058] An electronic device (101) according to one embodiment of the present disclosure may generate a prompt for a machine learning model (or a deep learning model or an artificial intelligence model) to generate and / or determine at least one recommended task based on acquired context information. For example, the prompt may include text, voice, audio, still images, and / or video. The machine learning model according to one embodiment of the present disclosure may include a large language model (LLM) and / or a large vision model (LVM). A “prompt” as referred to in the present disclosure may be a software element input to an artificial intelligence model (e.g., LLM) and may include a series of information for obtaining a specified response and / or for directing or guiding the performance of a specified task.

[0059] An electronic device (101) according to one embodiment of the present disclosure may generate at least one prompt (e.g., a first prompt) to receive at least one recommended task (720) from a machine learning model (340). According to one embodiment of the present disclosure, a first prompt includes a first command for a machine learning model (340) to suggest at least one recommended task (720) using context information, a first prompt element including a description of a role as an intelligent assistant and / or a response as an intelligent assistant, a description of a result generation rule, a second prompt element including at least a portion of context information (e.g., activity history information), a third prompt element including other context information except for at least a portion of context information (e.g., information about a situation in which a user exchanged messages with “ooo” just before, information about a situation in which the currently used application is “Internet” and a user is reading an article, information about text and images of articles recognized on the screen, information about URLs, information about a function list of all applications installed on the user’s electronic device (101), information about a wireless communication network currently being used by the electronic device), and / or a description of output data (e.g., “share this article,” “summarize article,” “highlight,” “conversation” (among recommended tasks suggested in natural language format). A fourth prompt element may include a first prompt element that includes a list of functions corresponding to each recommended task, provided and in the background, taking into account priorities. According to one embodiment of the present disclosure, for example, the first to fourth prompt elements may also be referred to as terms such as “instruction,” “context,” “input data,” and “output description,” respectively.According to one embodiment of the present disclosure, a prompt (e.g., a first prompt) to be input into a machine learning model may utilize a data platform (360) that includes data (e.g., context information) and a function repository that stores a list of functions and function-related information of all applications controllable on an electronic device (101), a user's preference information (e.g., activity history), and a usage pattern. For example, by using information on an application currently running on an electronic device (101), a user's activity history, and a user's usage pattern of the electronic device (101) as input values, candidates for tasks suitable for subsequent execution may be determined by the machine learning model. A task may include at least one action that performs one or more application functions, and may be performed concurrently or sequentially. An example of the first prompt is expressed in detail in Table 1 below.

[0060] A list of applications and operable actions that can be provided from at least one external electronic device (e.g., a wearable device and / or an IoT device) connected to an electronic device (101) on which an intelligent assistant according to one embodiment of the present disclosure is executed, context information of at least one external electronic device (e.g., information about a currently running application, status of the external electronic device (e.g., power on / off status, wireless communication connection status), activity history) may also be collected and included in the prompt.

[0061] For example, an electronic device (101) according to an embodiment of the present disclosure may identify the type of a currently connected wireless communication network (e.g., a cellular network as a first network). Based on the identified network type, the electronic device (101) according to an embodiment of the present disclosure may generate a prompt so that tasks that are only executed when connected to a second network (e.g., Wi-Fi) that is different from the identified network type are not output as recommended tasks (e.g., a list of restricted applications is included as context in the prompt and transmitted to the learning model when connected via the first network). In addition, the electronic device (101) according to an embodiment of the present disclosure may provide a notification that the task can be performed when connected to another wireless communication network (e.g., Wi-Fi) when a task that includes restricted operations is suggested and selected in the currently connected wireless communication network (e.g., a cellular network). The electronic device (101) according to an embodiment of the present disclosure may also identify the current wireless communication connection state of the electronic device (101) and provide a recommended task based on the identified wireless communication connection state. For example, an electronic device (101) according to one embodiment of the present disclosure may provide a task called “flash turn-on” to a user as a recommended task when the electronic device (101) is set to airplane mode.

[0062] A machine learning model according to one embodiment of the present disclosure may determine a recommended task based on a task or function subsequently performed after the electronic device (101) provides a screen (e.g., content including a soccer video) that has a similarity level above a certain level with the current screen. For example, when an intelligent assistant is called while the electronic device (101) according to one embodiment of the present disclosure outputs content including a soccer video, the electronic device (101) may identify context information related to the soccer video, for example, a function or operation for executing a designated application (e.g., a food delivery application) and / or checking the rankings of teams in the game, and may propose a task for executing the designated application (e.g., a food delivery application) and / or checking the rankings of teams in the game as a recommended task. Alternatively, the machine learning model according to one embodiment of the present disclosure may determine a recommended task based on the subsequently performed task and / or function (e.g., a task and / or function provided after the call application is executed), for example, when the electronic device (101) provides a service through an execution order of applications used in the past (e.g., after executing a message application, then executing a web browser application, then executing a call application), an order with a certain degree of similarity to the menu usage order provided by the application (e.g., after executing a message application, then executing a weather application, then executing a call application). The electronic device (101) according to one embodiment of the present disclosure may, when an intelligent application is called, display edge lighting to intuitively guide the user that the intelligent application has been executed. The electronic device (101) according to one embodiment of the present disclosure may display at least one recommended task (e.g., display it in a row / sequentially at the bottom of the display (160)) after displaying the edge lighting or simultaneously with the edge lighting.The examples described above can be included in the prompt as contextual information or as separate reference information, or passed to the machine learning model along with the prompt.

[0063] An electronic device (101) according to an embodiment of the present disclosure may determine and / or output at least one recommended task (720) based on an input prompt. For example, if the electronic device (101) (e.g., the machine learning model (340)) according to an embodiment of the present disclosure identifies that a user has a history of using a message application immediately before executing an Internet application, and that the user's activity history frequently uses the message application when sharing web pages and uses a method of sharing a URL (uniform resource locator), the electronic device (101) may determine and / or generate a "share" task as a recommended task. The "share" task according to an embodiment of the present disclosure may be a task including a first action of obtaining URL information of a web page currently provided through an Internet application and a second action of transmitting the URL information to a counterpart of the message application. In addition, if the electronic device (101) (e.g., the machine learning model (340)) according to one embodiment of the present disclosure identifies that the electronic device (101) is displaying a specified article through an Internet application, the electronic device (101) may determine and / or generate a “summary” task as a recommended task. The “summary” task according to one embodiment of the present disclosure may be a task that includes a first action of summarizing an Internet article currently being displayed through the electronic device (101) and a second action of providing summarized information through an intelligent assistant. Alternatively, for example, if the electronic device (101) (e.g., the machine learning model (340)) according to one embodiment of the present disclosure identifies that the user has a history of calling the other party through a call application and performing the call recording function of the call application (e.g., exceeding a specified number of times), and managing the schedule through a calendar application, the electronic device (101) may determine and / or generate a “call summary” task and / or a “call wrap-up” task as a recommended task.A "call summary" task according to one embodiment of the present disclosure may be a task including a first action of summarizing recorded call content and a second action of providing the summarized call content via an electronic device (101). For example, the electronic device (101) according to one embodiment of the present disclosure may convert recorded call content into text, summarize the converted text through a machine learning model, and output the summarized text on a screen and / or convert the summarized text into voice and output it.

[0064] A "call wrap-up" task according to one embodiment of the present disclosure may be a task including a first action of summarizing the recorded call content and a second action of adding or modifying a schedule through a calendar application based on the summarized call content. Alternatively, for example, an electronic device (101) (e.g., a machine learning model (340)) according to one embodiment of the present disclosure may determine and / or generate a "transfer" task as a recommended task if it is identified that a user has a history of sending and receiving messages with the other party through a messaging application and that the user's preference is to frequently use a designated application when sending money to the other party. A "transfer" task according to one embodiment of the present disclosure may include a first action of identifying information currently displayed on the screen, identifying an account number, amount, and depositor included on the screen, and providing a user confirmation message, a second action of performing a transfer through an application for money transfer frequently used for money transfers after the user's confirmation, and a third action of providing a message indicating the completion of the money transfer after the transfer is completed. Alternatively, for example, if the electronic device (101) (e.g., the machine learning model (340)) according to one embodiment of the present disclosure identifies that a user sends and receives messages with a counterpart through a messaging application, that the user's preference is to manage schedules through a calendar application, and that the user primarily uses a designated reservation application when making a reservation, the electronic device (101) may determine and / or generate a "transmit reservation information" task and / or a "save to calendar" task as recommended tasks. The "transmit reservation information" task according to one embodiment of the present disclosure may include a first action of obtaining reservation information provided through a reservation application frequently used by the user, and a second action of transmitting the reservation information to the counterpart of the messaging application through the messaging application.A “save to calendar” task according to one embodiment of the present disclosure may include a first action of obtaining reservation information provided through a reservation application frequently used by a user and a second action of saving the reservation information in a calendar application.

[0065] An electronic device (101) according to an embodiment of the present disclosure may provide an option for a user to designate at least one recommended task (e.g., at least one recommended task (720) of FIG. 7). For example, in order for the electronic device (101) according to an embodiment of the present disclosure to provide at least one recommended task (720), at least one application to be used for performing the task may be designated in advance or in real time by the user based on location, time, and / or conversation partner. According to an embodiment, a machine learning model may use information about a user-designated recommended task and / or application together with context information to determine and / or generate a recommended task by receiving information about the user-designated recommended task and / or application as part of a prompt.

[0066] According to one embodiment, at least one recommended task (720) may have classification information according to style (e.g., category). For example, the style (e.g., category) of at least one recommended task (720) to be suggested may be determined by the machine learning model (340) or may be specified by the user. The determined or specified style may be reflected in the design of the recommended task provided and / or the style of the automatically generated response. For example, if the style (e.g., category) of at least one recommended task (720) to be suggested is a style that falls within the "regular" category, the at least one recommended task (720) provided may be provided in alphabetical order or in an expected order in which the execution and / or selection of the at least one recommended task (720) is expected. For example, if the style (e.g., category) of at least one recommended task (720) to be suggested is a style that falls within the "secure" category, the at least one recommended task (720) provided through the electronic device (101) may be provided so that the user's personal information is not used. Additionally, for example, if the style (e.g., category) of at least one recommended task (720) to be suggested via the electronic device (101) is a style that falls within the “work” or “business” category, the style or tone of the text generated via the at least one recommended task (720) provided may reflect a “professional” attribute rather than a “casual” attribute.

[0067] A function or operation of providing at least one recommended task (720) according to one embodiment of the present disclosure may be provided according to priority. The electronic device (101) according to one embodiment of the present disclosure may determine and provide (e.g., display at least one recommended task (720)) at least one recommended task (720) from among a plurality of recommended tasks, for example, based on a user's past usage pattern (e.g., frequency of application use, frequency of application use for a specific counterpart device, and / or frequently used functions by time zone), a user profile (e.g., permission granted for each application), and status information history of the electronic device (101). According to one embodiment of the present disclosure, the operation of determining at least one recommended task (720) from among a plurality of recommended tasks and / or the operation of providing (e.g., displaying at least one recommended task (720)) at least one recommended task (720) may be individually and / or collectively performed by the processor (120) or the machine learning model of the electronic device (101).

[0068] An electronic device (101) according to one embodiment of the present disclosure may obtain a list of functions (e.g., deep links, shortcuts, intents, APIs, functions) of all applications installed in the electronic device (101) and / or a user's usage pattern and user preference information in order to generate a first prompt. An example of a first prompt element according to one embodiment of the present disclosure may include the following prompt.

[0069] "인스트럭션(instruction)"- AI Agent has system permission and the ability to recognize and process information related to application interfaces, display content, system configurations, pre-installed app functions, as well as user activity records and past interactions.- AI Agent requires user's explicit permission grant on such actions when spending charges, or can expose personal status or notifying user's status.- Now, the situation is that the user has called out to AI agent because they want something done automatically.- AI agent has to suggest the most suitable task candidates.- User is reading a news article through the Internet browser and calls out to you.

[0070] 본 개시의 일 실시예에 따른 제2 프롬프트 요소의 예시는 아래와 같은 프롬프트를 포함할 수 있다.

[0071] "Context"- The user is having a conversation about Legoland with their friend C (name : Catherine, tel_num : 1-24-1512521, contact id : friend / 25) through a messaging app.- The application currently in use is 'Internet'- The smartphone is connected through wifi (MYHOME-WIFI)

[0072] An example of a third prompt element according to one embodiment of the present disclosure may include the following prompt:

[0073] "입력 데이터(input data)"Application : InternetUsage : The user had been reading 10 articles over the course of 20 minutes.URL : https: / www.news.com / news / read / 1515815The contents of the news article are as follows in triple backticks```This library concert is a monthly program held since March, and 2,200 citizens have participated so far.Previously, Legoland Korea Resort (hereinafter referred to as Legoland), a global theme park, visited Kangwon National University Children's Hospital on the 19th and conducted a 'visiting Lego Santa Claus' for 50 children admitted to the pediatric ward.There is an image in news article containing the following content:Hospital, a hospital bed where a young patient is sitting, food on the bed, a Santa-shaped human-sized Lego doll clapping in front of the bed.

[0074] A machine learning model (340) according to one embodiment of the present disclosure may determine or generate at least one recommended task (720) based on the first prompt as described above. An electronic device (101) according to one embodiment of the present disclosure may provide at least one generated recommended task (720). For example, the electronic device (101) may provide a list of at least one recommended task (720) through a user interface (e.g., displaying it on a screen or outputting it through a speaker). In operation 230, the electronic device (101) according to one embodiment of the present disclosure may obtain a second user input for selecting one recommended task (e.g., sharing) from among the provided at least one recommended task (720) (e.g., sharing, summarizing, highlighting). For example, the electronic device (101) according to one embodiment of the present disclosure may obtain a touch gesture for touching one task from among the at least one recommended task (720).

[0075] An electronic device (101) according to one embodiment of the present disclosure may also receive an input of a task (e.g., saving an article currently being displayed) that is not included in at least one recommended task (720). For example, the electronic device (101) may provide at least one recommended task (720) obtained using a machine learning model (340) and an additional item (e.g., a “chat” menu) together on a user interface. For example, the electronic device (101) according to one embodiment of the present disclosure may obtain a command specifying a task other than at least one recommended task (720) by obtaining and receiving text, voice, and / or visual elements from a user through a “chat” function. Referring to FIG. 7, at least one recommended task (720) according to one embodiment of the present disclosure may include, for example, a first recommended task (720a) (e.g., “share”), a second recommended task (720b) ​​(e.g., “summary”), and a third recommended task (720c) (e.g., “highlight”). Additionally, a "Chat" item may be provided as an additional item (720d). The first recommended task (720a) (e.g., "Share") according to one embodiment of the present disclosure may be a menu for performing a task of transmitting the currently displayed Internet article to the other party's device. The second recommended task (720b) ​​(e.g., "Summary") according to one embodiment of the present disclosure may be a menu for summarizing the currently displayed Internet article and providing it to the user. The third recommended task (720b) ​​(e.g., "Highlight") according to one embodiment of the present disclosure may be a menu for highlighting (e.g., displaying a specified color) at least a portion of the currently displayed Internet article. The additional item (720d) (e.g., "Chat") according to one embodiment of the present disclosure may be a menu for performing a two-way conversation with an intelligent assistant.

[0076] According to one embodiment of the present disclosure, the electronic device (101) may, in operation 240, execute a task corresponding to a recommended task in the background through an intelligent assistant based on acquisition of a second user input. According to one embodiment of the present disclosure, some of the tasks among at least one recommended task may have some functions or operations pre-executed in the background before the user selects the task. Such pre-execution of tasks according to one embodiment of the present disclosure may be executed according to the type or priority of the task to reduce task execution time, or may be executed for specific tasks to check in advance whether user approval is required during task execution.

[0077] According to one embodiment of the present disclosure, the electronic device (101) may perform task planning when a second user input is acquired. For example, task planning may refer to a function of identifying or generating a series of detailed operations for executing a selected task. According to one embodiment, a series of operations or a sequence of operations identified or generated for executing a task may include operations or orders not provided by the manufacturer of the electronic device (101) or the manufacturer and / or producer of the application. According to one embodiment of the present disclosure, the electronic device (101) may select applications and / or functions related to the task from a function repository, and schedule functions for performing the task. According to one embodiment of the present disclosure, at least one recommended task (720) may include at least one action for performing at least one application function. According to one embodiment of the present disclosure, an “action” may include at least one or more “application functions.” At least one recommended task (720) according to one embodiment of the present disclosure may mean a set of at least one action that can be performed simultaneously or sequentially on the electronic device (101). The electronic device (101) according to one embodiment of the present disclosure may identify at least one application and / or function, determine a priority for at least one application and / or function, and generate a plan that can be performed sequentially or in parallel.For example, if the task acquired from the user consists of actions such as translating the currently displayed article, sharing it with a friend, and adding a related schedule, the electronic device (101) may translate the article content acquired through the Internet application into another language using a machine learning model, share the article with the other party's device through the sharing function and communication module (190), and schedule a new event to be added through a calendar application. The task planning function and / or operation according to one embodiment of the present disclosure may also be performed in operation 220. For example, the task planning function and / or operation may be performed through the machine learning model (340).

[0078] According to one embodiment of the present disclosure, the function of identifying a task to be provided or the function of generating a series of detailed operations for executing each task may include a function of determining what, when, to whom, where, and how to generate and provide. To this end, if the electronic device (101) according to one embodiment of the present disclosure determines that at least one parameter essential for determining what, when, to whom, where, and how to generate and provide for performing task planning is insufficient, the electronic device (101) may acquire additional insufficient information from the data platform (360) or request input of at least one required parameter from the user.

[0079] An electronic device (101) according to an embodiment of the present disclosure can perform a task based on the result of task planning. The electronic device (101) according to an embodiment of the present disclosure can determine whether essential parameters are missing for performing the task. According to an embodiment of the present disclosure, in order to perform a task such as a screenshot sharing function, information about a screen to be screenshotted and a counterpart device with which the captured screen will be shared is essentially requested, and thus, such information may be preset as essential parameters. If essential parameters are missing, the electronic device (101) according to an embodiment of the present disclosure can obtain the essential parameters from the data platform (360). The electronic device (101) according to an embodiment of the present disclosure can obtain at least one of information about the most frequently used application, functions of frequently used applications (e.g., screenshot sharing function, URL sharing function, text sharing function), and information input in relation to the application (e.g., contact information, schedule information, location information). An electronic device (101) according to an embodiment of the present disclosure may execute a corresponding task in the background if all essential parameters are acquired. Alternatively, if a non-essential parameter is missing, the electronic device (101) according to an embodiment of the present disclosure may supplement the missing / missing parameter by using information identified based on functions or operations frequently used by the user based on the user's activity history. For example, when sharing a photo with a partner after taking a photo, information about the partner's device is an essential parameter, but the mode in which the photo was taken may not be an essential parameter. Even in this case, the electronic device (101) according to an embodiment of the present disclosure may identify that the user frequently took photos in selfie mode based on the user's activity history.Accordingly, an electronic device (101) according to one embodiment of the present disclosure can execute a camera application based on a selfie mode.

[0080] During the execution of a task according to one embodiment of the present disclosure, content required for task execution may be generated. The content required for task execution according to one embodiment of the present disclosure may be generated by a machine learning model (340). The content required for task execution according to one embodiment of the present disclosure may include text, voice, audio, still images, and / or video.

[0081] According to one embodiment of the present disclosure, the electronic device (101) may perform at least a part of the generation, task planning, or content generation operations of a recommendation task (720) (e.g., a first recommendation task (720a) (e.g., “share”), a second recommendation task (720b) ​​(e.g., “summary”), a third recommendation task (720c) (e.g., “highlight”)) using an artificial intelligence model (e.g., a machine learning model) stored within the electronic device (101) and / or a remote artificial intelligence model. According to one embodiment, the machine learning model (340) may correspond to an artificial intelligence model stored within the electronic device (101). For example, the electronic device (101) may use an artificial intelligence model stored within the electronic device (101) to generate the recommendation task (720) and a remote artificial intelligence model to generate the content. According to one embodiment, depending on the type of content required for task execution, an artificial intelligence model stored in the electronic device (101) or a remote artificial intelligence model may be selectively utilized. For example, if it is determined that creation or modification of text content is necessary in connection with task execution, the electronic device (101) may create or modify the text content using an artificial intelligence model (e.g., a machine learning model (340)) stored in the electronic device (101). For example, if it is determined that creation or modification of multimedia content is necessary in connection with task execution, the electronic device (101) may be connected to a remote LLM server. According to one embodiment of the present disclosure, the electronic device (101) may transmit a prompt for creation / modification of the corresponding content to an external LLM server and receive the created / modified content from the external LLM server.

[0082] An electronic device (101) according to one embodiment of the present disclosure may generate a second prompt to perform task planning through an artificial intelligence model (e.g., a machine learning model (340)). According to one embodiment of the present disclosure, the second prompt may include at least one of a fifth prompt element including a second command as basic instruction information for performing a task selected by the user (e.g., "Find an appropriate recipient and share this article"), a sixth prompt element including information that must be confirmed in order to perform or complete the task (e.g., whether user confirmation is required to complete the task), a seventh prompt element including context information considered in creating the task and / or context information collected at the time the task is performed (e.g., the user has just exchanged messages with "ooo", the application currently being used is "Internet" and is reading the article, the application mainly used when sharing, user pattern information on sharing methods (e.g., screenshot, text copy, URL), a list of functions of all applications installed on the user's mobile phone and function-related information), or an eighth prompt element including a sequence of functions of at least one application for performing the task (e.g., a function sequence including execution information so that the selected task can be performed).

[0083] The first prompt and / or the second prompt according to one embodiment of the present disclosure may provide the machine learning model (340) with information about what function the user of the electronic device (101) executed before the current screen was displayed, what the user did in a similar situation in the past (e.g., a situation in which specific applications were used consecutively), what application the current screen is, what category the application belongs to, what menu has been entered, and what text / image is being displayed.

[0084] An electronic device (101) according to one embodiment of the present disclosure may provide a function list of all applications that may be provided through the electronic device (101) to a machine learning model (340), or may provide a function list for applications related to a task selected by a user to the machine learning model (340). For example, the electronic device (101) may also transmit a function list of applications that may be provided from an external device (e.g., the electronic device (102), the electronic device (104), or the server (108)) connected to the electronic device (101) via a network (e.g., the first network (198), the second network (199)) to the machine learning model (340). According to one embodiment, when at least a part of a task is performed using an external device, a guidance message related to the performance of the task may be displayed on the corresponding external device.

[0085] According to one embodiment, the machine learning model (340) may select an application and / or function for performing at least one task (720) from a list of applications and / or functions transmitted via the second prompt, and generate a series of operations for performing the at least one task (720). The process of selecting an application and / or function for performing the at least one task (720) may take into consideration at least one of information related to user preferences, information related to usage history, the user's current location, and the type of connectable network.

[0086] A second prompt according to one embodiment of the present disclosure may include the following prompt:

[0087] 제5 프롬프트 요소- AI Agent showed provided tasks to user.- and user decided to do task "공유하기".- now AI Agent have to create a task with previously provided functions.제6 프롬프트 요소- from the provided task and situation, fill up function uri and parameter and execution order.- also AI Agent can use information from Data Platform.- Data Platform contains "Activity History", "Task History", "Function History used for Tasks", "Frequently used parameters", "User preference", "Activity History" can provide user activity consists of "Application usage time" of "Activity name" and "Package Name" based on datetime.-"Task History" can provide user frequently selected Task.-"Frequently used parameters" can provide parameters history for that Task.-"User preference" has information about user confirmation about following system prompt.- Remember that AI Agent requires user's explicit permission grant on such actions when spending charges, or can expose personal status or notifying user's status.Prompt Element 7 - The user had shared 3 articles about travel to 'Catherine' yesterday. - The user had shared 2 summarized article to 'Catherine' yesterday.

[0088] An electronic device (101) according to one embodiment of the present disclosure may identify a series of operations or a sequence of operations for performing at least one selected task (720) based on a response to a second prompt output from a machine learning model (340), and perform the task by executing the series of operations. According to one embodiment, the electronic device (101) may perform the task in the background. For example, at least some of the series of operations may be executed in the background while a user interface in which a second user input is received or another user interface provided after the user interface is provided to the user. According to one embodiment, the electronic device (101) may notify that the task is being performed in the background or provide a guidance message indicating what operation or step is being performed in the background.

[0089] An electronic device (101) according to one embodiment of the present disclosure may, in operation 250, provide a guidance message indicating that a task has been completed based on the completion of a task executed in the background. An electronic device (101) according to one embodiment of the present disclosure may provide a guidance message indicating that a task has been completed through an intelligent assistant when the background execution of the task is terminated.

[0090] An electronic device (101) according to an embodiment of the present disclosure may generate content required in an intermediate stage of task execution or generate an answer to be provided in the final stage of task execution through a machine learning model (340). For example, when a user requests to transmit a message along with an article, the electronic device (101) according to an embodiment of the present disclosure may generate and provide an answer such as “I have completed sharing the article” through the machine learning model (340). The electronic device (101) according to an embodiment of the present disclosure may transmit a generated message such as “Please read this article, I think you might be interested!” along with an article or a summary of an article obtained by performing a task to another electronic device (e.g., the electronic device (104)). The generated message (e.g., “Please read this article, I think you might be interested!”) according to an embodiment of the present disclosure may be a phrase previously used by a user of the electronic device (101) or a message reflecting a phrase style (e.g., writing style, tone) frequently used by the user.

[0091] According to one embodiment of the present disclosure, when the completion of a task is identified, the electronic device (101) may generate a third prompt to provide a guidance message indicating that the performance of the task has been completed (e.g., "Sharing has been completed!"). Alternatively, the electronic device (101) according to one embodiment of the present disclosure may generate a third prompt to provide a guidance message related to the performance of the task (e.g., "An error occurred", "Permission to execute is required") during the performance of the task. The third prompt according to one embodiment of the present disclosure may be generated during the process of creating a task, during the process of creating a series of operations for performing the task, during the performance of the task, or after the performance of the task has been completed. The third prompt according to one embodiment of the present disclosure may include the following prompts.

[0092] - Task is carried out by AI Agent.- Not AI Agent's responsibility to generate a text response for the user.- AI Agent adheres to the following stylistic guidelines.1. Prefers active voice over passive voice whenever possible.2. Leans towards positive wording over negative.3. Keep statements concise.4. Avoids not using technical or robotic words.

[0093] FIG. 3 is an exemplary block diagram of an electronic device (101) according to one embodiment of the present disclosure, wherein the electronic device (101) is configured to perform a function or operation of providing at least one recommendation task (720) and executing a recommendation task selected by a user. The electronic device (101) according to one embodiment of the present disclosure may include at least one of an interface (310), an application / framework module (320), an artificial intelligence inference module (330), a machine learning model (340) (e.g., LLM), a function repository (350), a data platform (360), and / or a screen information management module (370).

[0094] An interface module (310) according to one embodiment of the present disclosure may include a module for managing interaction between a user and an electronic device (101). For example, with respect to at least some of the embodiments of the present disclosure, the electronic device (101) and the user may interact with each other through an intelligent assistant. The interface module (310) according to one embodiment of the present disclosure may perform a call, a request, a suggestion, and / or a response. The interface module (310) according to one embodiment of the present disclosure may provide a user interface that is used when a user calls an artificial intelligence module (e.g., when at least one of a voice, text input, or a specific gesture such as a double touch is input) or transmits a request, when the intelligent assistant suggests a task list or receives a request through a conversation function, when a prompt is received during an intermediate step of execution, and / or when a task performance result is provided.

[0095] A framework module (320) according to one embodiment of the present disclosure may include a module for managing and executing applications and service packages (e.g., capsules, domains) installed in an electronic device (101). A framework module (320) according to one embodiment of the present disclosure may manage functions (e.g., deep links, shortcuts, intents) provided by the electronic device (101) and / or each application. A deep link according to one embodiment of the present disclosure may refer to an object that moves to a designated application and displays a screen desired by the user or induces a user action. A shortcut according to one embodiment of the present disclosure may refer to an object that provides a shortcut function or operation for a designated application. An intent according to one embodiment of the present disclosure may refer to a software module for broadcasting an event generated in an operating system of the electronic device (101) to an application, or an operation or function thereof. The framework module (320) according to one embodiment of the present disclosure may be updated whenever an application is installed / updated / deleted. The framework module (320) according to one embodiment of the present disclosure may store function-related information (e.g., definitions, usage instructions, and function descriptions for each function).

[0096] An artificial intelligence inference module (330) according to one embodiment of the present disclosure may include an execution manager module (332), a conversation manager module (334), a function repository (350), and / or a machine learning model (340). The artificial intelligence inference module (330) according to one embodiment of the present disclosure may infer a task to be proposed to a user, plan the task by composing it into sub-actions / operations, and generate content necessary to complete the task performance using generative artificial intelligence (e.g., a machine learning model (340)). The function repository (350) and / or the machine learning model (340) according to one embodiment of the present disclosure may be stored in the data platform (360) or may be stored in another component. The execution manager module (332) according to one embodiment of the present disclosure may obtain the result of task planning. The execution manager module (332) according to one embodiment of the present disclosure may transmit a function for performing a task selected by a user to each application and system to perform the corresponding task. The execution manager module (332) according to one embodiment of the present disclosure can complete the execution of a task by acquiring essential parameters necessary for performing the task from the data platform (360). The conversation manager module (334) according to one embodiment of the present disclosure can receive a response generated by the machine learning module (340) from the machine learning module (340). The conversation manager module (334) according to one embodiment of the present disclosure can provide the response provided from the machine learning module (340) to the user through the electronic device (101).

[0097] A data platform (360) according to one embodiment of the present disclosure may include a module that manages a history of a user of an electronic device (101) using the electronic device (101) and / or information about the electronic device (101). The data platform (360) according to one embodiment of the present disclosure may manage an activity history performed prior to the current point in time, an application and the function of the application used when the user performs a specific task, frequently used parameters, and the user's preference information. In addition, the data platform (360) according to one embodiment of the present disclosure may manage (e.g., store) usage patterns, such as whether to skip a step requiring user confirmation, and contents generated during the application use process.

[0098] According to one embodiment of the present disclosure, the screen information management module (370) may perform a function of reading the screen currently being displayed on the electronic device (101) and extracting meaningful information, such as text and images, when a user of the electronic device (101) calls an intelligent assistant. The electronic device (101) according to one embodiment of the present disclosure may extract meaningful information using OCR, image recognition, and / or scene recognition algorithms. The function or operation of the electronic device (101) according to one embodiment of the present disclosure to extract meaningful information may be performed based on at least a part of the contents disclosed in, for example, U.S. application US 2023 / 0223008. According to one embodiment of the present disclosure, even if not visible to the user, information constituting the current user interface (e.g., UI structure, accessibility information, DOM (document object model)) may also be included in the extraction target of the screen information management module (370).

[0099] According to one embodiment of the present disclosure, the system of the present disclosure including the modules described above (e.g., the artificial intelligence inference module (330)) may include an AI hub (e.g., an AI platform) structure in which at least some of the modules or some of the subcomponents of the modules are driven by an always-on low-power AI processor (or engine). According to one embodiment of the present disclosure, the AI ​​hub structure can intelligently connect various applications, functions, input / output interfaces, and power elements of the electronic device (101), as well as wearable devices (e.g., smart watches, smart rings, XR (extended reality), IoT devices (e.g., home appliances, access points, speakers), cloud servers, and automobiles) that are operatively connected to the electronic device (101), thereby providing the artificial intelligence service of the present disclosure that proposes a context-based recommendation task. For example, at least one recommendation task (720) can include a series of actions / operations that can be collectively performed by various devices (e.g., electronic device (102), electronic device (104), server (108)) connected to the AI ​​hub structure described above. According to one embodiment, the function storage (350) can manage applications and / or functions that can be provided by various devices connected to the AI ​​hub structure. The present disclosure can include software or hardware driven by an application processor or a dedicated processor.

[0100] FIG. 4 is an exemplary drawing for explaining a function or operation of an electronic device (101) according to one embodiment of the present disclosure, which generates a prompt based on contextual information and provides at least one recommended task (720) based on the generated prompt.

[0101] The flowchart illustrated in FIG. 4 is merely a flowchart according to one embodiment of the operation of the electronic device (101), and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 410 to 450 may be performed by at least one processor (120) of the electronic device (101).

[0102] Referring to FIG. 4, an electronic device (101) according to one embodiment of the present disclosure may obtain context information related to the electronic device (101) in operation 410. Context information according to one embodiment of the present disclosure includes information about at least one application running in the foreground (e.g., a name of at least one application running in the foreground, a category to which at least one application running in the foreground belongs, and / or a description of at least one application running in the foreground), screen context information (e.g., information about an input UI object included in a screen currently being displayed through the electronic device (101), information about a text included in a screen currently being displayed through the electronic device (101), information about an image included in a screen currently being displayed through the electronic device (101), information about an available function included in a screen currently being displayed through the electronic device (101), information about an activity performed by a user in the past based on a current point in time (e.g., information about a time at which a specified application was run, information about a location at which a specified application was run, information about a function that was run in the past in relation to a specified application, information about applications that were run, and / or information about text that was input by a user), and / or information about all applications installed in the electronic device (101). The application may include at least one of information about the actionable actions of the application (e.g., information about a schedule registration action, information about a notification registration action, information about whether the camera can be launched, information about whether a message can be sent, information about the battery status, and / or information about whether a wireless communication (e.g., Wi-Fi®) connection is available).

[0103] An electronic device (101) according to one embodiment of the present disclosure may generate at least one prompt based on acquired context information in operation 420. The prompt according to one embodiment of the present disclosure may include at least one of a first prompt for receiving at least one recommended task (720) from a machine learning model (340), a second prompt for task planning, and a third prompt for providing a task execution result. For example, at least some of the first, second, and third prompts may be included in one prompt. According to one embodiment of the present disclosure, operation 420 may be performed by an artificial intelligence inference module (330).

[0104] An electronic device (101) according to one embodiment of the present disclosure may, at operation 430, provide at least one prompt generated according to operation 420 to a machine learning model (340).

[0105] An electronic device (101) according to one embodiment of the present disclosure may, at operation 440, obtain, from a machine learning model (340), a specification of at least one recommendation task (720) and / or a series of actions / operations for performing the recommendation task based on the provision of at least one prompt. Operations 430 and 440 according to one embodiment of the present disclosure may be performed by the artificial intelligence inference model (330).

[0106] An electronic device (101) according to one embodiment of the present disclosure may, in operation 450, provide at least one recommended task obtained according to operation 440 through the electronic device (101). Operation 450 according to one embodiment of the present disclosure may be performed by an interface module (310).

[0107] FIG. 5 is an exemplary drawing for explaining a function or operation of an electronic device (101) according to one embodiment of the present disclosure to perform task planning according to a user's selection input for at least one recommended task (720).

[0108] The flowchart illustrated in FIG. 5 is merely a flowchart according to one embodiment of the operation of the electronic device (101), and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 510 to 530 may be performed by at least one processor (120) of the electronic device (101).

[0109] An electronic device (101) according to an embodiment of the present disclosure may, at operation 510, identify a function associated with at least one application for performing a task selected by a user (e.g., a task selected at operation 230 of FIG. 2 ). For example, an electronic device (101) according to an embodiment of the present disclosure may identify at least one application and / or function, determine a priority for at least one application and / or function, and generate a plan that can be performed sequentially or in parallel.

[0110] An electronic device (101) according to one embodiment of the present disclosure may, in operation 520, generate at least one prompt (e.g., a second prompt) based on the identified function. According to one embodiment of the present disclosure, the second prompt may include at least one of a fifth prompt element including a second command to perform the task (e.g., "Find an appropriate recipient and share this article"), a sixth prompt element including information on whether user confirmation is required to complete the task (e.g., whether user confirmation is required to complete the task), a seventh prompt element including context information (e.g., the user recently exchanged messages with "ooo", the currently used application is "Internet" and the user is reading an article, the application mainly used when sharing, sharing method (e.g., screenshot, text copy, URL) user pattern information, a function list of all applications installed on the user's electronic device (101) and function-related information), or an eighth prompt element including a sequence of functions of at least one application to perform the task (e.g., a function sequence including execution information so that the selected task can be performed). Operation 520 according to one embodiment of the present disclosure may be performed by the artificial intelligence inference model (330).

[0111] An electronic device (101) according to an embodiment of the present disclosure may provide the generated prompt to a machine learning model (340) in operation 530. An electronic device (101) according to an embodiment of the present disclosure may execute a task in the background based on a response to the second prompt output from the machine learning model (340). When an omission of a required parameter is identified during the process of performing a task, the electronic device (101) according to an embodiment of the present disclosure may obtain information about the required parameter from the data platform (360) or obtain information about the required parameter from the user. Execution of a task according to an embodiment of the present disclosure may be performed by an execution manager module (332).

[0112] FIG. 6 is an exemplary diagram for explaining a function or operation of an electronic device (101) according to one embodiment of the present disclosure to determine whether at least one required parameter is provided and to execute a designated task in the background based on the determination.

[0113] The flowchart illustrated in FIG. 6 is merely a flowchart according to one embodiment of the operation of the electronic device (101), and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 610 to 640 may be performed by at least one processor (120) of the electronic device (101).

[0114] Referring to FIG. 6, an electronic device (101) according to an embodiment of the present disclosure may determine, in operation 610, whether at least one parameter required to perform a task selected by a user has been provided. To this end, the electronic device (101) may check a function sequence for performing the task based on a response generated through operation 530 of FIG. 5 and may determine whether parameters included in the function sequence have been determined. The electronic device (101) according to an embodiment of the present disclosure may determine, in operation 620, whether all required parameters have been provided. Operations 610 and 620 according to an embodiment of the present disclosure may be executed by the execution manager module (332).

[0115] According to one embodiment of the present disclosure, the electronic device (101) may execute a task in the background if it is determined that all required parameters have been provided (e.g., operation 620 - Yes) at operation 630. According to one embodiment of the present disclosure, the electronic device (101) may request the user to provide the required required parameters if it is determined that all required parameters have not been provided (e.g., operation 620 - No) at operation 640. Operation 640 according to one embodiment of the present disclosure may be performed by the conversation manager module (334).

[0116] FIG. 7 is an exemplary drawing for explaining, from a user interface perspective, a function or operation of an electronic device (101) according to one embodiment of the present disclosure to transmit information about an article provided through an Internet browser to a counterpart device.

[0117] Referring to FIG. 7, an electronic device (101) according to an embodiment of the present disclosure may obtain a user input (e.g., a first user input (710)) for calling an intelligent assistant from a user while a designated application (e.g., an Internet application) is being executed. The electronic device (101) according to an embodiment of the present disclosure may provide at least one recommended task (720) (e.g., share, summarize, highlight, chat) based on the first user input (710). “Chat” according to an embodiment of the present disclosure may be an option for calling an interactive artificial intelligence assistant. Through the “chat” function of the present disclosure, after receiving a text or voice command from the user, a user’s request may be performed based on at least one of information from the data platform (360) (e.g., user preference, device status), information from the screen information management module (370), and information from the framework (320), and not only the performance result but also suggestions for recommended tasks related to the performance result may be provided to the user together.

[0118] At least one recommended task (720) according to one embodiment of the present disclosure may be provided without user input (e.g., first user input). For example, according to one embodiment of the present disclosure, when a user checks a specific application for a specified period of time or longer (e.g., runs it in the foreground for a specified period of time or longer), selects a specified object (e.g., text and / or menu included in an application execution screen), performs a specified function (e.g., captures a screen, takes a picture of a specified subject), downloads a specified file, or transmits a specified message to a counterpart device, the electronic device (101) may generate a prompt and / or execute an intelligent assistant in the background based on this. Alternatively, the electronic device (101) according to one embodiment of the present disclosure may provide specified content to the user and generate a prompt and / or execute an intelligent assistant in the background based on the user's response to the provided content.

[0119] An electronic device (101) according to one embodiment of the present disclosure may provide at least one recommended task (720) based on a generated prompt and / or an executed intelligent assistant. For example, if a user checks travel information for a specific region for a certain period of time, the electronic device (101) according to one embodiment of the present disclosure may suggest at least one task that the user is expected to want (e.g., sending a summary of the content included in the current screen to a person with whom he or she recently shared travel information for the region, or updating the content included in the current screen in a document or app recording a travel itinerary for the region) through the electronic device (101) without a user request.

[0120] An electronic device (101) according to one embodiment of the present disclosure may perform at least one task (e.g., "sharing") selected by a user in the background and provide a guidance message (730) indicating the result of the performance while a designated application (e.g., an Internet application) or another application is running. According to one embodiment, the electronic device (101) may provide the guidance message (730) through an intelligent assistant.

[0121] FIG. 8 is an exemplary drawing for explaining, from a user interface perspective, a function or operation of adding a schedule after a user of an electronic device (101) according to one embodiment of the present disclosure ends a call with a user of the other party's device.

[0122] Referring to FIG. 8, an electronic device (101) according to an embodiment of the present disclosure may obtain a user input (e.g., a first user input (810)) for calling an intelligent assistant from a user after execution of a call application is terminated. An electronic device (101) according to an embodiment of the present disclosure may provide at least one recommended task (820). An electronic device (101) according to an embodiment of the present disclosure may obtain a user input for selecting one task (e.g., “ending a call”) from among the at least one recommended task (820). An electronic device (101) according to an embodiment of the present disclosure may convert a call recording into text based on a user input, summarize the call recording, and extract main keywords (e.g., meal appointment, recipient, time, place) based on the converted text and / or summarized content. An electronic device (101) according to one embodiment of the present disclosure can add or modify a schedule through a calendar application based on the extracted information, and provide a guidance message (830) indicating the performance result through a functional assistant.

[0123] FIG. 9 is an exemplary drawing for explaining a function or operation of an electronic device according to one embodiment of the present disclosure to perform remittance to a user of a counterpart device from a user interface perspective.

[0124] Referring to FIG. 9, an electronic device (101) according to an embodiment of the present disclosure may obtain a user input (e.g., a first user input (910)) for calling an intelligent assistant while a screen including information for an account transfer is displayed in a messenger application. The electronic device (101) according to an embodiment of the present disclosure may provide at least one recommended task (e.g., “transfer,” “share,” “chat”). The electronic device (101) according to an embodiment of the present disclosure may obtain a user input for selecting any one task (e.g., “transfer”) from among the at least one recommended task. The electronic device (101) according to an embodiment of the present disclosure may check whether any required parameters (e.g., account number, amount, depositor) for performing the “transfer” task are missing. According to an embodiment of the present disclosure, if it is determined that essential parameters (e.g., account number, amount, depositor) for performing a "transfer" task are missing, the electronic device (101) may extract information included in the screen to confirm information about the account number, amount, and depositor. According to an embodiment of the present disclosure, if information included in the sensitive information category (e.g., account number, resident registration number, password, user account, address, phone number, amount) is extracted, the electronic device (101) may output a guidance message (920) to obtain confirmation from the user. According to an embodiment of the present disclosure, based on a user input selecting one task (e.g., "transfer") among at least one recommended task (e.g., "transfer," "share," "chat"), if the user's confirmation for the transfer is obtained, the electronic device (101) may perform the transfer through a remittance application most frequently used by the user of the electronic device (101).An electronic device (101) according to one embodiment of the present disclosure may perform at least one task (e.g., “transfer”) selected by a user in the background and provide a guidance message (930) indicating the result of the performance through an intelligent assistant.

[0125] Recommended tasks according to one embodiment of the present disclosure may be classified into tasks that require user approval for the execution of the task or at least some of the operations among a series of operations for the execution of the task, and tasks that do not require user approval. According to one embodiment of the present disclosure, tasks corresponding to each classification may be managed as first type tasks and second type tasks, respectively. According to one embodiment of the present disclosure, different types of tasks may be displayed using different colors, fonts, sizes, and / or icons in the user interface of the recommended task list (e.g., at least one recommended task (720)). According to one embodiment of the present disclosure, when a task requiring user approval is selected, an additional interface requesting user authentication (e.g., biometric authentication) may be provided at the time of task selection, at the time of task start, or during task execution. The biometric authentication at this step may include the same biometric authentication required to unlock a mobile device. The biometric authentication at this step may correspond to biometric authentication pre-registered for an application required for task execution. An electronic device (101) according to one embodiment of the present disclosure may generate a message to be input into a messenger application as part of a task action. The electronic device (101) according to one embodiment of the present disclosure may display the generated message (e.g., "I completed the deposit!") on a user input screen, or display a user interface that prompts the user to confirm whether the generated message has been input.

[0126] FIGS. 10A to 10C are exemplary drawings for explaining a function or operation of an electronic device (101) according to one embodiment of the present disclosure to share reservation information with a counterpart device and add a schedule to the electronic device from a user interface perspective.

[0127] Referring to FIGS. 10A to 10C , an electronic device (101) according to an embodiment of the present disclosure may obtain a user input (e.g., a first user input (1015)) for calling an intelligent assistant on an execution screen (1020) of a reservation application, which includes information on a reserved store and reservation time, while having a conversation with the other party through an execution screen (1010) of a message application. The electronic device (101) according to an embodiment of the present disclosure may provide at least one recommended task (720) on the execution screen (1020) of the reservation application. The electronic device (101) according to an embodiment of the present disclosure may obtain a selection input from a user for selecting one task (e.g., “do both”) from among the at least one recommended task (720). The electronic device (101) according to an embodiment of the present disclosure may create a new schedule (1032) in a calendar application using information included in the execution screen (1020) of the reservation application. An electronic device (101) according to one embodiment of the present disclosure may display a new schedule (1032) on the execution screen (1030) of a calendar application. An electronic device (101) according to one embodiment of the present disclosure may transmit reservation information for a "wine bar" to a counterpart device. In this case, a designated application (e.g., a message application) may be used. An electronic device (101) according to one embodiment of the present disclosure may transmit at least one message (1012) generated by a machine learning model (340) to a counterpart device. An electronic device (101) according to one embodiment of the present disclosure may perform at least one task (e.g., "do both") selected by a user in the background and provide a guidance message (730) indicating the performance result through an intelligent assistant.

[0128] FIG. 11 is an exemplary diagram illustrating, from a user interface perspective, a function or operation of an intelligent assistant continuously used during execution of a specified application by an electronic device (101) according to one embodiment of the present disclosure. In FIG. 11 , "in" may refer to information input to a machine learning model (340), and "out" may refer to information output by the machine learning model (340).

[0129] Referring to FIG. 11, an electronic device (101) according to an embodiment of the present disclosure may obtain a first user input on a message application execution screen to check whether a schedule is saved for a specific date while a user is exchanging messages with a counterpart through the message application. The electronic device (101) according to an embodiment of the present disclosure may provide at least one recommended task (720) based on the first user input (710). The electronic device (101) according to an embodiment of the present disclosure may obtain a user input for selecting at least one task (e.g., “check schedule”). The electronic device (101) according to an embodiment of the present disclosure may perform a task (e.g., check whether there is a schedule for a specific date) based on the obtained user input. The electronic device (101) according to an embodiment of the present disclosure may provide a task execution result (e.g., schedule check result). An electronic device (101) according to an embodiment of the present disclosure may obtain a first user input for recalling an intelligent assistant on an execution screen of a message application. Based on the acquisition of the first user input for recalling the intelligent assistant, the electronic device (101) according to an embodiment of the present disclosure may provide at least one recommended task (720). The electronic device (101) according to an embodiment of the present disclosure may obtain a user input for selecting at least one task (e.g., “add schedule”). The electronic device (101) according to an embodiment of the present disclosure may perform a task (e.g., create and save a new schedule) based on the acquired user input. The electronic device (101) according to an embodiment of the present disclosure may provide a result of performing the task (e.g., create and save a new schedule).An electronic device (101) according to one embodiment of the present disclosure may perform at least one task (e.g., “finish appointment”) selected by a user in the background (e.g., deleting from reminders, collecting and automatically settling expenses, or posting a photo taken during the schedule on social media) and provide a guidance message (730) indicating the result of the performance through an intelligent assistant. An electronic device (101) according to one embodiment of the present disclosure may transmit at least one message generated by a machine learning model (340) to a counterpart device. In FIG. 11, an embodiment in which a “conversation” item is provided as a recommended task is illustrated, but this is exemplary, and the “conversation” item may not be included in the recommended task (720).

[0130] FIG. 12 is an exemplary drawing for explaining, from a user interface perspective, a function or operation of an electronic device (101) according to one embodiment of the present disclosure, providing at least one recommended task (720) and summarizing and providing a new message when a message application is executed through a notification of the message application.

[0131] Referring to FIG. 12, an electronic device (101) according to an embodiment of the present disclosure may select a notification corresponding to the message application in the notification display area of ​​the quick panel to enter the message application (e.g., display the execution screen of the message application). The electronic device (101) according to an embodiment of the present disclosure may obtain a first user input (1210) from a user while entering the message application. In response to the first user input (1210), the electronic device (101) according to an embodiment of the present disclosure may provide a list including a message catch-up task, a read processing task, a notification clearing task, and / or a conversation item. At least one of the tasks or items displayed in the list may be generated by a machine learning model (340) based on context information. A "message catch-up" according to an embodiment of the present disclosure may be a function or operation that summarizes and provides the contents of a plurality of messages that the user has not checked. According to one embodiment of the present disclosure, the provision of a summary of messages may include a summary that summarizes unconfirmed content in all chat rooms at once, separate summaries may be provided for each chat room, separate summaries may be provided for each conversation partner, or separate summaries may be provided for each conversation topic. "Clearing notifications" according to one embodiment of the present disclosure may be a function or operation of deleting at least one notification from a notification window (e.g., a notification window in which notifications from various applications are collectively provided). According to one embodiment of the present disclosure, the "clearing notifications" task may include an operation of deleting a notification from the notification window and registering it in a reminder application to provide a reminder requesting confirmation to the user if the user does not continuously check it for a specified period of time. "Read processing" according to one embodiment of the present disclosure may be a function or operation of reading at least a portion of unconfirmed messages / notifications.An electronic device (101) according to one embodiment of the present disclosure may perform a function or operation of displaying an additional user interface that allows selection of read processing by chat room, conversation partner, or message, as at least a part of the functions or operations performed according to a “read processing” task.

[0132] An electronic device (101) according to one embodiment of the present disclosure may obtain a user input for selecting a task (e.g., "catch up a message"). The electronic device (101) according to one embodiment of the present disclosure may perform the task (e.g., catch up a message) based on the obtained user input. The electronic device (101) according to one embodiment of the present disclosure may provide a result of performing the task (e.g., a summary message (1230) that summarizes a conversation starting from messages that the user has not checked). In FIG. 12 , when a large number of notifications are received on the electronic device (101), the electronic device (101) may identify a user's preference (e.g., history) of checking notifications at once rather than frequently, as the user's activity history. This is illustrated in FIG. 12 by the phrases "a state in which unread messages are received" and "messages received in multiple chat rooms." According to one embodiment of the present disclosure, an electronic device (101) can, as part of performing a message catch-up task, enter a messaging application via a notification and provide a summary of the content of a new message. This function or operation is illustrated in FIG. 12 by the expressions "entering a specific conversation window of a messenger from a notification" and "providing a summary of a new message."

[0133] FIG. 13 is an exemplary drawing for explaining, from a user interface perspective, a function or operation of an intelligent assistant continuously used during execution of a specified application by an electronic device according to one embodiment of the present disclosure.

[0134] Referring to FIG. 13, an electronic device (101) according to an embodiment of the present disclosure can execute a file converted from a voice recording file into text through a note application. In this state, the electronic device (101) according to an embodiment of the present disclosure can obtain a first user input (710). The electronic device (101) according to an embodiment of the present disclosure can provide at least one recommended task (720). The electronic device (101) according to an embodiment of the present disclosure can obtain an input for selecting one task. The electronic device (101) according to an embodiment of the present disclosure can perform the task in the background and then provide a conversion result of the voice content converted into text for each speaker so that it is suitable for a meeting minutes format. The electronic device (101) according to an embodiment of the present disclosure can re-obtain the first user input (710) from the converted meeting minutes screen. The electronic device (101) according to an embodiment of the present disclosure can provide at least one recommended task (720). An electronic device (101) according to one embodiment of the present disclosure may obtain a user input for selecting a task (e.g., "ending a meeting"). The electronic device (101) according to one embodiment of the present disclosure may, in the background, check a meeting schedule and attendee information related to the meeting. The electronic device (101) according to one embodiment of the present disclosure may, after converting the content into minutes to the attendees via email, provide an execution completion message. In FIG. 13, the electronic device (101) may identify, as a user's activity history, receiving a meeting-related email, performing voice recording and converting the recording file into text during the meeting, and, as user's preference information, identifying, as an activity history, creating minutes after the meeting ends and delivering them to the attendees.As data stored in the electronic device (101), information related to the meeting (e.g., purpose of the meeting, location of the meeting, time of the meeting) may be stored through a mail application, and a recording file containing the meeting content and data converted into text from the recording file may be stored through a voice recording application. According to an embodiment of the present disclosure, when a meeting minutes writing recommendation task is selected, the electronic device (101) may execute a note application, copy the converted text into the note application, and convert it into a meeting note format. According to an embodiment of the present disclosure, when a meeting wrap-up recommendation task is selected, the electronic device (101) may store the meeting schedule and meeting recording time in the electronic device (101) through a calendar application, and, based on the information stored through a mail application, may further store additional information (e.g., attendees, meeting title) in a text file converted into a meeting note format to generate meeting minutes. In addition, as part of an operation for the meeting wrap-up recommendation task, the electronic device (101) according to an embodiment of the present disclosure may send the generated meeting minutes to the meeting attendees through an email.

[0135] Referring to FIG. 13, an electronic device (101) according to an embodiment of the present disclosure may provide, as recommended tasks (1320), tasks of “listening to recording,” “ending a meeting,” and / or “sharing meeting minutes.” “Listening to recording” according to an embodiment of the present disclosure may be a function or operation of outputting a recording file in which the meeting content is recorded. “Sharing meeting minutes” according to an embodiment of the present disclosure may be a function or operation of sending meeting minutes to meeting participants via email. “Ending a meeting” according to an embodiment of the present disclosure may include a function or operation of generating meeting minutes by storing a meeting schedule and a meeting recording time in the electronic device (101) through a calendar application, and further storing additional information (e.g., attendees, meeting title) in a text file converted into a meeting note format based on the stored information through a mail application.

[0136] In the present disclosure, the intelligent assistant that generates recommended tasks and action sequences for each recommended task may utilize a machine learning model or an artificial intelligence neural network. The machine learning model of the present disclosure may be a model trained to detect tasks likely to be executed later based on context. For example, the machine learning model may be trained or learned through sufficient training data on contexts such as activity history, device status, and the current user interface to associate specific contexts with specific tasks and action sequences. The machine learning model of the present disclosure may include various transformer models. The operation of the machine learning model of the present disclosure may include a learning and inference process that finds patterns in data, stores them as models that are generalized rules, and inputs new data into the trained model to obtain results.

[0137] In a mobile device (e.g., the electronic device (101) of FIG. 1), user operations (e.g., functions repeatedly performed in a consistent order) can be monitored to train a machine learning model. The mobile device (e.g., the App / Framework module, the Data platform module) can provide the machine learning model with information about which functions the user used before and after executing a communication application (e.g., email, SMS (short message service), SNS (social network service), phone call, conference), the location of the mobile device at the time, the time zone, the network used, which functions were automatically performed, which settings were manually added, and which topics the user exchanged with a specific party over a certain period of time. The current user interface and previous activity history can serve as inputs for learning, and subsequent user operations can serve as target outputs. The machine learning model can learn the user's usage patterns. This learning can correspond to initial learning or retraining. The learning process of the machine learning model can include forward propagation and backward propagation. For learning, algorithms such as regression, decision trees, neural networks, and k-nearest neighbors can be used. Different machine learning models can be used depending on the target task or input context.

[0138] For convenience of explanation, the following describes the LLM used in the present disclosure, but it is self-evident that the artificial intelligence neural network of the present disclosure may include not only a language model, but also various foundation models such as a code model and an image model, as well as other artificial intelligence neural network models.

[0139] The LLM mentioned in this disclosure may refer to an artificial neural network-based language model that has learned a large amount of text data through pre-training. The LLM may contain a relatively larger number of parameters (e.g., over 10 billion) than existing general language models. The LLM may utilize a transformer artificial neural network structure based on the attention mechanism.

[0140] The attention mechanism is a technique that helps artificial intelligence models focus (attention) on important parts of input data. The attention mechanism can be used to predict output data by predicting the degree to which a portion of time-series input data (e.g., voice or video input data, or input data from a layer of a neural network) contributes to the intermediate or final output of the neural network. Recurrent neural networks (RNNs), which sequentially process each element of a sequence, exhibit poor prediction performance when there is information dependency between long time-series distances. However, the attention mechanism can consider information dependency between long time-series distances by controlling the degree of weight concentration (attention) within the overall (or partial) context of the input data.

[0141] A Transformer can be structured as an encoder-decoder. The encoder processes input data and outputs compressed information (e.g., a contextual representation), while the decoder processes the compressed information and outputs token-based data. Each encoder and decoder can include an independent attention network, or a cross-attention network connecting the encoder and decoder.

[0142] For example, LLM training may involve pre-training and / or fine-tuning. Pre-training involves training the LLM to acquire general linguistic knowledge using large amounts of text data. For example, this may involve self-supervised learning, where the LLM predicts the next word based on the previous word sequence in the text string. Fine-tuning involves training the LLM to be suitable for a specific domain (e.g., chatbot, translation, summarization, Q&A) or task. The LLM may undergo additional supervised learning (or adaptive learning) based on the pre-trained model using a dataset tailored to the domain's purpose. The LLM can perform tasks with text inputs containing natural language, called prompts.

[0143] For example, fine-tuning can be omitted when learning LLMs. The user can control the prompts provided to the LLM to enhance its performance on the desired task. Similar to in-context learning or zero-shot / few-shot learning, the prompts can provide additional task examples and / or guidance for performing the task. Publicly available LLMs include BERT (Bidirectional Encoder Representations from Transformer) and GPT (generative pre-trained transformer).

[0144] A machine learning model (340) (e.g., LLM) according to one embodiment of the present disclosure may additionally receive image (including video) information in addition to text. According to one embodiment of the present disclosure, the image information may be converted into text through a separate pre-conversion (e.g., image recognition, scene recognition), and the machine learning model (340) may include this in a prompt to generate a response. As another example, the electronic device (101) may convert an input image into image embeddings aligned with text through a video encoder, and the machine learning model (340) may generate a response using a separately trained model (e.g., a large multimodal model) using the text embeddings corresponding to the input text.

[0145] The term "LLM" can refer to the language neural network model itself, but can also refer to the model of an LLM-based application (e.g., chatbot, translation, summarization, text classification, sentence generation). For example, an LLM-based chatbot like ChatGPT or an LLM-based translator could also be referred to as "LLM."

[0146] "LLM" may also include an inference engine utilizing the LLM neural network model. For example, "inputting an input prompt to the LLM" may mean "inputting the input prompt to an inference engine based on the LLM." For example, "the output of the LLM for the input prompt" may mean the output information of the last neural network layer of the LLM (or output information modified through further processing) obtained when the input prompt is input to the LLM-based inference engine.

[0147] An electronic device (101) according to one embodiment of the present disclosure includes a display, at least one hardware processor, and a memory, wherein the memory may be configured to store instructions that, when executed by the at least one hardware processor, cause the electronic device to: execute an intelligent assistance based on a first user input for executing the intelligent assistance; provide, through the display, at least one recommended task determined based on context information related to the electronic device in response to the execution of the intelligent assistance; obtain, through the display, a second user input for selecting a recommended task from among the at least one recommended task; execute, through the intelligent application, a task corresponding to the recommended task in the background based on the acquisition of the second user input; and provide, through the display, a guidance message indicating that the task has been completed based on completion of the task executed in the background.

[0148] An electronic device (101) according to one embodiment of the present disclosure includes at least one hardware processor and a memory, and the memory may be configured to store instructions that, when executed by the at least one hardware processor, cause the electronic device to present a first user interface corresponding to a first application, detect an input specified through the first user interface, and obtain at least one recommended task through an intelligent assistant based on the specified input, wherein the intelligent assistant is configured to identify the at least one recommended task based on context information including the first user interface, and includes a machine learning model trained to generate a series of operations for executing each of the at least one recommended task, provide a list of tasks including the at least one recommended task on the first user interface, and perform at least some of the series of operations in the background in response to selection of the at least one recommended task.

[0149] Electronic devices according to various embodiments disclosed in the present disclosure may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of the present disclosure are not limited to the aforementioned devices.

[0150] The various embodiments of the present disclosure and the terminology used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In the present disclosure, each of the phrases "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among the phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0151] The term "module" used in various embodiments of the present disclosure may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0152] Various embodiments of the present disclosure may be implemented as software (e.g., a program (2540)) including one or more instructions stored in a storage medium (e.g., an internal memory (2536) or an external memory (2538)) readable by a machine (e.g., an electronic device (2501)). For example, a processor of the machine (e.g., the electronic device (2501)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0153] According to one embodiment, the method according to various embodiments disclosed in the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0154] According to one embodiment of the present disclosure, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separately arranged in other components. According to one embodiment of the present disclosure, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment of the present disclosure, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In electronic devices, display; At least one hardware processor, and A memory comprising: a computer-readable medium configured to cause the electronic device to: Executing the intelligent assistance based on a first user input for executing the intelligent assistance, Through the display, in response to the execution of the intelligent assistance, providing at least one recommended task determined based on contextual information related to the electronic device, Through the above display, a second user input is obtained for selecting one of the at least one recommended task, Based on the acquisition of the second user input, a task corresponding to the recommended task is executed in the background, and An electronic device characterized in that it is set to store instructions for providing a guidance message indicating that the task has been completed through the display based on the completion of the task executed in the background.

2. In paragraph 1, An electronic device, characterized in that the context information includes at least one of information about at least one application running in the foreground, text and / or images provided through the display, or activity history information of the user.

3. In paragraph 1 or 2, An electronic device, characterized in that the instructions, when executed by the at least one hardware processor, further include instructions that cause the electronic device to generate a first prompt for providing the at least one recommended task using the context information, and input the generated first prompt into a machine learning model.

4. In any one of paragraphs 1 to 3, An electronic device, characterized in that the first prompt includes at least one prompt element among a first prompt element including a first command that causes the machine learning model to suggest the at least one recommended task using the context information, a second prompt element including the activity history information, a third prompt element including context information other than the activity history information, or a fourth prompt element including a description of output data.

5. In any one of paragraphs 1 to 4, An electronic device characterized in that the instructions, when executed by the at least one hardware processor, further include instructions that cause the electronic device to generate a second prompt for performing a task corresponding to the recommended task and input the generated second prompt into the machine learning model.

6. In any one of paragraphs 1 to 5, An electronic device characterized in that the second prompt includes at least one prompt element among a fifth prompt element including a second command to perform the task, a sixth prompt element including information on whether a user's confirmation is required to complete the task, a seventh prompt element including the context information, or an eighth prompt element including a sequence of functions of at least one application to perform the task.

7. In electronic devices, At least one hardware processor, and A memory comprising: a computer-readable medium configured to cause the electronic device to: Present a first user interface corresponding to the first application, Detecting the input specified through the first user interface, Based on the specified input, at least one recommended task is obtained, and the obtaining of the at least one recommended task is configured to identify the at least one recommended task based on context information including the first user interface, and a machine learning model is trained to generate a series of operations for executing each of the at least one recommended task. Providing a list of tasks including at least one recommended task through the first user interface, and An electronic device characterized in that it is configured to store instructions for performing at least some of the series of operations in the background in response to selection of at least one of the recommended tasks.

8. In paragraph 7, The above instructions, when executed by the at least one hardware processor, cause the electronic device to: An instruction for switching from the first user interface to a second user interface corresponding to the second application, and An electronic device, characterized in that it further includes instructions for providing a notification indicating an execution status of at least one recommended task on the second user interface.

9. In paragraph 7 or 8, The machine learning model includes a large language model (LLM) running on the electronic device, An electronic device characterized in that the instructions, when executed by the at least one hardware processor, further include instructions that cause the electronic device to provide a prompt to the giant language model, the prompt including the context information and information about applications that can be provided by the electronic device, such that the giant language model can identify the at least one recommended task.

10. In any one of paragraphs 7 to 9, An electronic device characterized in that the instructions, when executed by the at least one hardware processor, further include instructions that cause the electronic device to, when the at least one recommended task is selected from the list of tasks, provide an additional prompt to the giant language model, the additional prompt including the at least one selected recommended task and a user preference setting, such that the giant language model can generate the series of actions.

11. In any one of paragraphs 7 to 10, An electronic device, characterized in that the context information includes first content provided through the first user interface.

12. In any one of paragraphs 7 to 11, An electronic device, characterized in that the context information includes second content provided through a third user interface corresponding to a third application before the first user interface is provided.

13. In any one of paragraphs 7 to 12, The third application includes a communication application for communicating with a counterpart, An electronic device, characterized in that the above series of operations includes an operation of generating a message through the intelligent assistant and transmitting the generated message to the other party through the third application.

14. In any one of paragraphs 7 to 13, An electronic device, characterized in that the at least one recommended task comprises a plurality of tasks including a first task and a second task, the first task corresponding to a first type requiring user approval in connection with the performance of at least one operation among the series of operations, and the second task corresponding to a second type not requiring user approval.

15. In any one of paragraphs 7 to 14, An electronic device, characterized in that at least some of the above series of operations are performed through another electronic device operatively connected to the electronic device.

Images