Method for managing task execution based on generative artificial intelligence and electronic device therefor

Abstract

Disclosed is an electronic device comprising a memory, and at least one processor. The electronic device may: generate a plurality of sub-prompts from an input for requesting a task; sequentially input the plurality of sub-prompts to one or more artificial intelligence models in order to perform the task; store first result data associated with at least one first sub-prompt among the plurality of sub-prompts when interruption of the task is detected while performing the task, wherein the first result data is generated by processing the at least one first sub-prompt by means of the one or more artificial intelligence models; and after the interruption of the task is detected, perform, on the basis of a resume request for the task, the task by using the first result data and the remaining sub-prompts excluding the at least one first sub-prompt among the plurality of sub-prompts.

Description

Generative artificial intelligence-based task execution management method and electronic device therefor

[0001] Embodiments disclosed in this document relate to a generative artificial intelligence-based task execution management method and an electronic device therefor.

[0002] Generative AI (Generative AI) is an artificial intelligence (AI) system that can generate text, images, and / or other media in response to prompts. Generative AI may include AI models trained by learning the patterns and structures of input training data. Generative AI can use the trained AI models to create new data with similar characteristics to the trained training data. Services that utilize various generative AI models (large language model (LLM), large vision model (LVM), large multi-modal model (LMM)), such as ChatGPT™, Midjourney™, and Copilot, can generate content data such as text, images, and / or video in response to prompts. To receive and process requests from many users, services that utilize generative AI models host the generative AI models on cloud servers or data centers, utilize the server's computing resources to train the AI ​​models on large-scale training data, and execute the generative AI models on the servers. Hosting can refer to deploying and operating a website, application, and / or other service on a server, making it accessible over a network. In services where a server hosting a generative AI model receives requests from a user device, processes them through the model, generates a response, and sends it to the device, the AI ​​model is not directly hosted on the device itself. Most of the computing required to execute the AI ​​model is handled by the server.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art in connection with the present disclosure.

[0004] An electronic device according to an embodiment disclosed in the present document may include a display, a memory, and at least one processor communicatively connected to the display and the memory. The memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to generate a plurality of sub-prompts from an input requesting a task. The memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to sequentially input the plurality of sub-prompts into one or more artificial intelligence models for performing the task. The memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to store first result data associated with at least one first sub-prompt among the plurality of sub-prompts when an interruption of the task is detected during the performance of the task. The first result data may include data generated by processing the at least one first sub-prompt by the one or more artificial intelligence models. The memory may store instructions that cause the electronic device, when individually or in combination executed by the at least one processor, to perform the task using the first result data and the remaining sub-prompts, excluding the at least one first sub-prompt, among the plurality of sub-prompts, based on a resume request for the task after an interruption of the task is detected.

[0005] A method for generating result data for a task request input of an electronic device according to an embodiment disclosed in the present document may include: generating a plurality of sub-prompts from an input requesting a task; generating first result data by processing at least one first sub-prompt among the plurality of sub-prompts by the one or more artificial intelligence models; sequentially inputting the plurality of sub-prompts into one or more artificial intelligence models to perform the task; generating first result data by processing at least one first sub-prompt among the plurality of sub-prompts by the one or more artificial intelligence models; storing first result data associated with at least one first sub-prompt among the plurality of sub-prompts when an interruption of the task is detected during the performance of the task; and performing the task, after the interruption of the task is detected, using the first result data and remaining sub-prompts among the plurality of sub-prompts excluding the at least one first sub-prompt based on a request to resume the task.

[0006] A computer-readable storage medium according to one embodiment disclosed in the present document can store instructions that, when executed by a processor of an electronic device, cause the electronic device to perform a method for generating result data for the task request input.

[0007] Figure 1 illustrates a task execution environment of an electronic device according to one embodiment.

[0008] Figure 2 illustrates modules of an electronic device according to one embodiment.

[0009] Figure 3 illustrates the structure of AI state data according to one embodiment.

[0010] Figure 4 illustrates the structure of prompt data according to one embodiment.

[0011] Figure 5 is a flowchart of a method for generating a sub-prompt according to one embodiment.

[0012] Figure 6 is a flowchart of a method for performing a task using sub-prompts according to one embodiment.

[0013] Figure 7 is a flowchart of a method for performing a task when a task interruption is detected according to one embodiment.

[0014] Figure 8 is a flowchart of a method for performing a task using additional prompts according to one embodiment.

[0015] Figure 9 is a flowchart of a task performing method according to one embodiment.

[0016] Figure 10 is a flowchart of a task execution method by sub-prompt type according to one embodiment.

[0017] Figure 11 is a flowchart of a method for performing a task using additional prompts according to one embodiment.

[0018] FIG. 12 is a flowchart of a method for displaying result data after task interruption detection according to one embodiment.

[0019] Figure 13 is a flowchart of a method for performing tasks in multiple applications according to one embodiment.

[0020] Figure 14a illustrates a process of performing an image generation task according to one embodiment.

[0021] Figure 14b illustrates a process of performing an image generation task according to one embodiment.

[0022] FIG. 15 illustrates various external devices capable of communicating with an electronic device according to one embodiment.

[0023] FIG. 16 is a flowchart of a method for performing a task in multiple electronic devices according to one embodiment.

[0024] FIG. 17a illustrates a UI for performing a task, stopping a task, and resuming a task according to one embodiment.

[0025] Figure 17b illustrates a user interface (UI) for progress by sub-prompt according to one embodiment.

[0026] FIG. 18 is a block diagram of an exemplary electronic device capable of performing the operations described in this document.

[0027] In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0028] Hereinafter, various embodiments of the present invention will be described with reference to the attached drawings. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that the present invention encompasses various modifications, equivalents, and / or alternatives of the embodiments.

[0029] FIG. 1 illustrates a configuration of an electronic device and an operation in which the electronic device responds to a task request according to one embodiment.

[0030] Referring to FIG. 1, according to one embodiment, an electronic device (10) may include a memory (110), a generative artificial intelligence (AI) model (115), a processor (120), a display (130), a communication circuit (140), and / or an interface (150). For example, the electronic device (10) may include a configuration similar to the electronic device (1800) described below with reference to FIG. 18. The electronic device (10) may be referred to as any user device. For example, the electronic device (10) may include a bar-type mobile device, a foldable electronic device (e.g., an electronic device having a foldable display), a rollable electronic device (e.g., an electronic device having a rollable display), a wearable electronic device (e.g., a smart watch and / or a smart ring), an AI speaker, and / or an electronic device configured to provide XR (extended reality) (e.g., an electronic device configured to provide VST (video see-through) content). For example, the memory (110) may correspond to the memory (1820) of FIG. 18. The processor (120) may correspond to the processor (1810) of FIG. 18. The display (130) may correspond to the display (1840) of FIG. 18. The communication circuit (140) may correspond to the communication circuit (1860) of FIG. 18. The configuration of the electronic device (10) illustrated in FIG. 1 is exemplary, and the configuration of the electronic device (10) is not limited thereto. For example, the electronic device (10) may further include a configuration not illustrated in FIG. 1 (e.g., at least one of the configurations of the electronic device (1800) of FIG. 18). For example, the electronic device (10) may not include at least one of the configurations illustrated in FIG. 1 (e.g., the generative AI model (115) and / or the communication circuit (140)).

[0031] The memory (110) can store instructions. When executed by the processor (120), the instructions can cause the electronic device (10) to perform various operations. In various embodiments of the present disclosure, the operation of the electronic device (10) can be referred to as an operation performed by the processor (120) by executing instructions stored in the memory (110). The memory (110) can be configured with hardware similar to the memory (1820) described below with reference to FIG. 18.

[0032] The memory (110) may store a generative AI model (115). For example, the generative AI model (115) may include a large language model (LLM), a large multi-modal model (LMM), and / or a large vision model (LVM). In the present disclosure, the term generative AI model (115) may be referred to as any model trained based on training data, which is configured to generate output based on input. The generative AI model (115) may include, for example, an input layer, at least one hidden layer, and an output layer. Each of the at least one hidden layer may include a plurality of nodes trained to have weights based on the training layer. Although FIG. 1 illustrates the memory (110) as storing one generative AI model (115), embodiments of the present disclosure are not limited thereto. For example, the memory (110) may store a plurality of generative AI models. The plurality of generative AI models may include AI models of different types (e.g., LLM, LVM, and / or LMM). Even for generative AI models of the same type, generative AI models trained based on different data sets may be referred to as different AI models. In FIG. 1, the generative AI model (115) is illustrated as being stored in the memory (110), but embodiments of the present disclosure are not limited thereto. For example, the generative AI model (115) may be stored in an external cloud server and / or data center. When the generative AI model is stored in an external server, the electronic device (10) may communicate with the external server via the communication circuit (140) to request a task from the generative AI model stored in the external server or receive result data generated through the generative AI model.

[0033] For example, LLM can refer to a language model based on an artificial neural network that has learned a large amount of text data through pre-training. An LLM can contain significantly more parameters (e.g., over 100 billion parameters) than conventional language models. In one example, an LLM may include a transformer artificial neural network structure based on an attention mechanism.

[0034] Attention mechanisms are techniques that help AI models focus on important parts of input data. Attention mechanisms can predict the extent to which a portion of time-series input data (e.g., input data such as voice or video, and / or input data from certain layers of a neural network) contributes to the intermediate or final output of a neural network, and can be used to predict output data.

[0035] For example, a transformer may be configured with an encoder-decoder structure. The encoder may process input data and output compressed information (e.g., a contextual representation), and the decoder may process the compressed information and output output data in token units. Each encoder and decoder may include an independent attention network, or a cross-attention network connecting the encoder and decoder.

[0036] For example, the training of an LLM may include pre-training and / or fine-tuning. Pre-training may refer to the process of teaching the LLM general linguistic knowledge using a large amount of text data. For example, pre-training may involve self-supervised learning, which predicts the next word in a text string using previous word sequences. Fine-tuning may refer to the process of training the LLM to be suitable for a specific domain (e.g., chatbot, translation, summarization, Q&A) or task. For example, the LLM may be further supervised (or adaptively trained) using a data set tailored to the domain's purpose based on a pre-trained model. The LLM may perform a task based on a prompt (e.g., text input containing natural language). In one example, fine-tuning may be omitted from the LLM training.

[0037] The term "LLM" can refer to the model itself, as described above, but can also refer to models for LLM-based applications (e.g., chatbots, translation, summarization, text classification, sentence generation). For example, LLM-based chatbots like ChatGPT™ or LLM-based translators can also be referred to as "LLM."

[0038] For example, LVM can be a model based on a transformer architecture (e.g., ViT (Vision Transformer)) that has learned from a large amount of image data through pre-training. For example, LVM can be an application of LLM, a language model used in natural language processing, to vision tasks.

[0039] For example, in LVM, the attention mechanism can be used to extract and combine important features from high-resolution images.

[0040] For example, ViT may include an encoder structure and a multi-layer perceptron (MLP) head configuration. In ViT, the encoder may have a structure similar to the layers of a general transformer. Unlike a general transformer that processes text tokens, the encoder structure of ViT may include a structure for processing image patches. The MLP head may be composed of an input layer, a hidden layer, and / or an output layer. The MLP head may transform data (e.g., a vector) processed by the encoder into an output of a desired form.

[0041] For example, training an LVM may involve pre-training and / or fine-tuning. Pre-training may be referred to as the process of teaching the LVM general visual knowledge using a large amount of image data. Fine-tuning may be referred to as the process of training the LVM to be suitable for a specific domain (e.g., object recognition, image classification, image generation) or task. In one example, fine-tuning may be omitted from LVM training.

[0042] The term "LVM" can refer to the model itself of the aforementioned structure, but can also refer to a model for an LVM-based application (e.g., object recognition, image classification, image generation). For example, LVM can refer to an object recognition system and / or image generator based on ViT, CLIP, DINO, and / or LandingLens.

[0043] For example, an LMM may refer to a multimodal model trained on a large amount of text, image, and / or audio data through pre-training. An LMM may include an encoder for each modality and / or modules for interactions between modalities.

[0044] The term "LMM" can refer to the multimodal neural network model itself, but can also refer to the model for an LMM-based application (e.g., chatbot, translation, image description, text and image classification). For example, an LMM-based chatbot or image description application capable of simultaneously understanding and generating text, images, and / or audio could also be referred to as an "LMM."

[0045] The processor (120) may be communicatively, electrically, operatively, or functionally connected to the memory (110), the display (130), the communication circuitry (140), and / or the interface (150). In various embodiments of the present disclosure, when one component is “operatively” connected to another component, it may mean that the component is connected so as to be able to operate the other component. For example, the component may operate the other component by transmitting a control signal to the other component, either directly or via another component. In various embodiments of the present disclosure, when one component is “functionally” connected to another component, it may mean that the component is connected so as to be able to execute a function of the other component. For example, the component may execute a function of the other component by transmitting a control signal to the other component, either directly or via another component.

[0046] The processor (120) may include at least one processor. For example, the processor (120) may include an application processor (AP), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a tensor processing unit (TPU), and / or a communication processor (CP). The processor (120) may include one chip or a chipset. In the present disclosure, the processor (120) may be referred to as a hardware component having an architecture by at least one processing circuit. For example, the processor (120) may be mounted on a substrate (e.g., a printed circuit board) located within the electronic device (10) and may communicate with other components of the electronic device (10) through at least one conductive path formed on the substrate.

[0047] The display (130) may be configured to display at least one image (e.g., text, an image, and / or a video). In one example, the display (130) may include multiple displays. For example, the display (130) may include a left-eye display and a right-eye display. The display (130) may include a front display and a rear display. The display (130) may include at least one of a see-through display, a flexible display, a rollable display, a foldable display, and / or a rigid display.

[0048] The communication circuit (140) may include at least one circuit (e.g., a modem, a radio frequency integrated circuit, and / or a radio frequency processing circuit) configured to support communication between the electronic device (10) and another electronic device (e.g., an external device (20)). The communication circuit (140) may support wired communication and / or wireless communication. The communication circuit (140) may support short-range wireless communication and / or long-range wireless communication.

[0049] The interface (150) may, in one example, include at least one hardware component for receiving user input. For example, the interface (150) may include a touch-sensitive circuit, at least one microphone, at least one camera, and / or any physical interface (e.g., a knob, a jog & shuttle, and / or a button). The interface (150) may, in one example, include at least one hardware component for outputting a response of the electronic device (10). For example, the interface (150) may include at least one display, at least one speaker, a printer, and / or an optical device. For example, the optical device may include an optical device (e.g., a projector) for projecting an image in an augmented reality (AR) / virtual reality (VR) environment.

[0050] According to one embodiment, the electronic device (10) can receive a user input (160) and generate a response (170) to the input. The electronic device (10) can obtain the input (160) through the interface (150). For example, the electronic device (10) can obtain a keyboard input through a keyboard. For example, the electronic device (10) can obtain a voice input through at least one microphone. For example, the electronic device (10) can obtain a touch input through a touch detection circuit. For example, the electronic device (10) can obtain an input (e.g., a voice input, a keyboard input, and / or a touch input) from an external device (e.g., a Bluetooth earphone and / or a Bluetooth keyboard) communicatively connected through the communication circuit (140).

[0051] For example, the input (160) may include an input requesting the performance of a task. The electronic device (10) may generate at least one prompt for the performance of the task from the input (160). For example, the prompt may include a prompt to be input into the generative AI model (115) for the performance of the task. For example, the response (170) may include a result generated according to the task request of the input (160). For example, the response (170) may include a notification for the interruption when the performance of the generation task is interrupted. In one example, the input (160) may be configured in a mixed form of at least one or more of text data, image data, audio data, video data, graph data, and / or table data.

[0052] The electronic device (10) may generate a response (170) to an input (160) requesting a task on its own, but may also generate a response (170) through collaboration with an external device (20). For example, the external device (20) may receive a request from the electronic device (10), generate all or part of a response (170) to the input (160), and transmit the generated response to the electronic device (10). The external device (20) may be, for example, a smart home appliance, a VST device, a smart watch, a smart phone, a smart ring, and / or a cloud server.

[0053] Figure 2 illustrates software modules of an electronic device according to one embodiment.

[0054] Referring to FIGS. 1 and 2, according to one embodiment, at least a portion of the software structure (200) may be a software module implemented by executing instructions stored in the memory (110) by the processor (120). The software modules within the software structure (200) may be referred to as, for example, applications, programs, and / or threads. Some of the modules illustrated in FIG. 2 may be implemented by dedicated hardware. For example, the network module (250) may be implemented by hardware configured to control the communication circuit (140).

[0055] The configuration of the software structure (200) illustrated in FIG. 2 is an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the configurations of the software structure (200) illustrated in FIG. 2 may be implemented in an external device (20) communicatively connected to the electronic device (10). By exchanging data with the external device (20) using the communication circuit (140), the electronic device (10) and the external device (20) may implement the software structure (200).

[0056] For example, the software structure (200) may include an AI manager (210), a prompt manager (220), a data management module (230), a task request input module (240), a network module (250), and / or a hardware malfunction detection module (260).

[0057] In one embodiment, the AI ​​manager (210) may control the operation of a hardware / software anomaly detection module (211), a generation completion prediction module (213), an AI controller (215), and / or a generative AI model. In one example, the AI ​​manager (210) may include a hardware / software anomaly detection module (211), a generation completion prediction module (213), an AI controller (215), and / or a generative AI model (217).

[0058] The hardware / software anomaly detection module (211) can detect task interruptions based on hardware and software elements. For example, hardware elements may include the device's battery level (e.g., State of Charge), whether the network connection is poor, and / or available computing resources. For example, software elements may include information that can determine whether a process is operating normally (e.g., HEALTH CHECK information and / or PING (packet internet groper) information), the application's response time, whether a thread is in an executable state, and / or the time taken for disk I / O (Input / Output). In one example, the hardware / software anomaly detection module (211) can periodically detect task interruptions. In one example, the hardware / software anomaly detection module (211) can detect task interruptions in response to a specified interrupt. In one example, the hardware / software anomaly detection module (211) can detect a task interruption when a task interruption request is made. The hardware / software anomaly detection module (211) can monitor a task interruption based on the task request.

[0059] For example, the electronic device (10) can detect a task interruption when it is predicted that the battery level (e.g., State of Charge) of the electronic device (10) will be depleted before the task is completed. For example, the electronic device (10) can detect a task interruption when it cannot receive or transmit data required for the task to be performed due to a poor network connection. For example, the electronic device (10) can detect a task interruption when the physical memory (e.g., RAM (random access memory)) of the electronic device (10) for performing the task is insufficient due to the execution of other software (e.g., virtual machine) that requires a lot of memory. For example, the electronic device (10) can detect a task interruption when the storage capacity of the data storage space (e.g., DISK and / or SD (secure digital) card) for performing the task is insufficient. For example, the electronic device (10) can detect a task interruption when the computational power of the processor (e.g., CPU, GPU, NPU, TPU) for performing a task is insufficient due to the execution of other software (e.g., application) that requires a large amount of computation. For example, the electronic device (10) can detect a task interruption when a software process does not respond consecutively more than a specified number of times while performing a task. For example, the electronic device (10) can detect a task interruption when there is no response from an application for a period longer than a specified time. For example, the electronic device (10) can detect a task interruption when a deadlock state occurs in which multiple threads occupy resources that each other need and enter an indefinite waiting state. For example, the electronic device (10) can detect a task interruption when a disk input / output operation required for performing a task takes longer than a specified time.

[0060] Hardware elements can be referenced from hardware status information (320) when referring to FIG. 3. Software elements can be referenced from software status information (310) when referring to FIG. 3.

[0061] The generation completion prediction module (213) can predict the time required to generate result data based on the amount of computation. For example, the generation completion prediction module (213) can predict the overall task execution completion time and the generation completion time of each sub-prompt-specific result data. “Sub-prompt-specific result data” may refer to result data generated when a sub-prompt is input into a generative AI model. The predicted overall task execution completion time and the predicted sub-prompt-specific result data generation completion time may be used to calculate a progress rate and improve the performance of the generative AI model. In one example, the sub-prompt-specific progress rate may be a value obtained by dividing the elapsed time since the generation task for the sub-prompt began by the generation completion prediction module (213) by the generation completion time of each sub-prompt-specific result data. The calculated sub-prompt-specific progress rate may be stored in the prompt data (235) of FIG. 4, which will be described later. In one example, the overall task progress rate may be a value obtained by dividing the elapsed time since the task execution began by the overall task execution completion time predicted by the generation completion prediction module (213). The calculated overall task progress can be stored in the prompt data (235) of FIG. 4, which will be described later. In one example, the performance of the generative AI model can be improved using reinforcement learning. For example, if the actual overall task completion time is shorter than the predicted overall task completion time, a bonus point can be given to improve the overall task performance. For example, if the actual overall task completion time is longer than the predicted overall task completion time, a penalty can be given to improve the overall task performance. For example, if the actual generation time of the result data of a specific sub-prompt is shorter than the predicted generation completion time of each result data of each sub-prompt, training can be performed by giving a bonus point for the corresponding sub-prompt.For example, if the actual generation time of the result data of a specific sub-prompt is longer than the generation completion time of each predicted result data for each sub-prompt, training can be performed by imposing a penalty on the sub-prompt.

[0062] The AI ​​controller (215) can control the generative AI model (217). For example, when a sub-prompt is input to the generative AI model (217), the AI ​​controller (215) can control the generative AI model (217) to generate content corresponding to the input sub-prompt. If a task interruption is detected, the AI ​​controller (215) can control the generative AI model (217) to stop performing the task. If a task interruption is detected, the AI ​​controller (215) can notify the data management module (230) of the task interruption so that the data management module (230) can generate or update AI status data (231).

[0063] A generative AI model (217) may be referred to as an artificial intelligence neural network that generates new types of data (e.g., data that includes information not included in the input information) from input information. The generative AI model (217) may include a model that generates images and / or a model that generates text data. The model that generates images may include, for example, a generative adversarial network (GAN), a variational auto encoder (VAE), a stable diffusion, and / or a diffusion-based generative model that uses a VAE and a transformer architecture. The model that generates text data may include a model trained to output statistically most appropriate output values ​​based on input values. For example, the model that generates text data may include a generative pre-trained transformer (GPT)-based generative model. The generative AI model (217) may include a large language model (LLM), a large multi-modal model (LMM), and / or a large vision model (LVM). Part or all of the generative AI model (217) may be implemented on an external cloud server and / or data center. When referring to FIG. 1, the generative AI model (217) may also be referred to as the generative AI model (115) of FIG. 1.

[0064] The prompt manager (220) can control modules related to sub-prompts. In one example, the prompt manager (220) can include a task request input analysis module (221), a sub-prompt generation module (223), and / or a sub-prompt interpreter (225).

[0065] The task request input analysis module (221) can analyze whether the task request input (160) can be regenerated into multiple sub-prompts. In one example, the task request input analysis module (221) can identify keywords and intents from the task request input (160). For example, keywords may be contexts associated with the input (160). The contexts associated with the input (160) may include time, place, and / or object associated with the input (160). For example, in the input (160) “Please draw a realistic picture of various flowers and squirrels in a forest at sunset,” the keywords may include “sunset,” “in the forest,” “flowers,” and / or “squirrel.” The intent may be a result that the user wants to obtain from the input. For example, in the input (160) “Please draw a realistic picture of various flowers and squirrels in a forest at sunset,” the intent may be “Please draw a realistic picture.”

[0066] The sub-prompt generation module (223) can generate a plurality of sub-prompts from the task request input (160). For example, the sub-prompt generation module (223) can generate a plurality of sub-prompts by merging each of the plurality of keywords identified by the task request input analysis module (221) with an intent. For example, the sub-prompt generation module (223) can generate a plurality of sub-prompts from the input (160) such as “Please draw a realistic picture of sunlight filtering into the jungle during the transition from day to night,” “Please draw a realistic picture that vividly depicts colorful flowers,” and / or “Please draw a realistic picture that depicts an extremely realistic animal with detailed animal fur and soft details by adding a squirrel.”

[0067] In one example, the sub-prompt generation module (223) may generate a plurality of sub-prompts that must be processed sequentially. The plurality of sub-prompts that must be processed sequentially may be referred to as step-by-step sub-prompts. The step-by-step sub-prompts may be used to perform tasks through sequential processing, as described below.

[0068] In one example, the sub-prompt generation module (223) can generate multiple sub-prompts that can be processed independently, regardless of the processing order. The multiple sub-prompts that can be processed independently, regardless of the processing order, can be referred to as element-specific sub-prompts. The element-specific sub-prompts can be used to perform tasks through parallel processing, as described below.

[0069] For example, if the task is to generate an image, the sub-prompt generation module (223) can generate sub-prompts corresponding to a plurality of elements included in the image. For example, the plurality of elements may be objects and / or backgrounds recognizable from the image. For example, if the task is to generate code (e.g., programming code), the sub-prompt generation module (223) can generate sub-prompts corresponding to each function having a single function. For example, the code may include at least one function having a single function. In one example, the plurality of sub-prompts for generating the code may be generated as step-by-step sub-prompts to share parameters between each function.

[0070] The sub-prompt interpreter (225) can preprocess the sub-prompt and input it into the generative AI model. For example, the sub-prompt interpreter (225) can preprocess the sub-prompt into a form that the generative AI model (217) can understand. The preprocessing may include, for example, tokenization and / or normalization. Tokenization may include splitting the input into individual words and / or sub-word chunks. Normalization may include converting the input into a standard format by converting the case and removing unnecessary spaces.

[0071] The data management module (230) may generate, store, and / or update AI status data (231) and / or prompt data (235). The AI ​​status data (231) may include information evaluating the overall status of the electronic device (10) and the AI ​​system and / or data for detecting task interruption, as described below. The prompt data (235) may include information associated with sub-prompts, as described below, such as progress, a list of sub-prompts (430), and / or a list of result data per sub-prompt (440).

[0072] The task request input module (240) can obtain and process an input (160) requesting the performance of a task from a user. In one example, the task request input module (240) can obtain an input (160) from a user through an interface (150). For example, the task request input module (240) can obtain at least one of text (e.g., natural language), an image, audio, and / or a video as the input (160). The task request input module (240) can transfer the obtained task request input to the data management module (230) so that the data management module (230) can store the task request input (160) in task request input information (420).

[0073] The network module (250) may support communication with a network or other electronic devices (e.g., an external device (20)). In one example, the network module (250) may support wired communication and / or wireless communication via the communication circuit (140). The network module (250) may support short-range wireless communication and / or long-range wireless communication via the communication circuit (140). In one example, the network module (250) may be implemented by dedicated hardware. For example, the network module (250) may be implemented by hardware configured to control the communication circuit (140).

[0074] The device status monitoring module (260) can monitor the device resources of the electronic device (10). In one example, the device status monitoring module (260) can monitor the remaining battery capacity (e.g., State of Charge) and / or memory availability of the electronic device (10). The data management module (230) can update or store the observed values ​​obtained through monitoring in AI status data (231).

[0075] Figure 3 illustrates the structure of AI state data according to one embodiment.

[0076] Referring to FIGS. 1 to 3 , according to one embodiment, AI status data (231) may include information evaluating the overall status of the electronic device (10) and the AI ​​system and / or data for detecting task interruption. For example, AI status data (231) may include software status information (310), hardware status information (320), pending sub-prompt information (330), and / or temporary result data information (340).

[0077] Software status information (310) may be information for detecting software abnormalities. Software status information (310) may be referenced by the hardware / software abnormality detection module (211) to detect task interruptions. For example, software status information (310) may include information that can confirm whether a process is operating normally (e.g., HEALTH CHECK information and / or PING (packet internet groper) information), the response time of an application, whether a thread is in an executable state, and / or the time taken for disk I / O. For example, software status information (310) may include information that can confirm the status of an AI solution for performing a generative task. An AI solution for performing a generative task may be referenced as a set of software that includes software that processes AI tasks required for a generative task and / or software that creates final result data for a requested generative task using AI task results processed by the above software. The AI ​​tasks required for the generative task may include, for example, data analysis, pattern recognition, predictive modeling, natural language processing, and / or image segmentation. For example, information that can check the status of the AI ​​solution for performing the generative task may include information that can check whether the software included in the AI ​​solution is operating normally (e.g., HEALTH CHECK information and / or PING (packet internet groper) information), information that can check whether the software included in the AI ​​solution is stopped (e.g., software error log and / or system log), and / or information that can check whether the software included in the AI ​​solution is in a waiting state for a response (e.g., Pingdom that periodically measures the response time).

[0078] In one example, the software status information (310) may include direct information indicating a software abnormality of the electronic device (10). For example, the software status information (310) may include information that can identify sub-prompts that successfully generated result data when the result data generation was completed without a task interruption using sub-prompts. For example, the software status information (310) may include information that can identify sub-prompts that failed to generate result data when the result data generation was not completed using sub-prompts due to a task interruption. The hardware status information (320) may be information for detecting a hardware abnormality. The hardware status information (320) may be referenced by the hardware / software abnormality detection module (211) and used to detect a task interruption. For example, the hardware status information may include information about a battery status (321), a resource status (323), and / or a network status (325).

[0079] For example, the battery status (321) may include information about the remaining battery power (e.g., State of Charge) and / or battery capacity. For example, the resource status (323) may include information about available physical memory (e.g., RAM) and / or processor (e.g., CPU, GPU, NPU, and / or TPU) usage of running processes. For example, the network status (325) may include the strength of a signal in a wireless network (e.g., received signal strength indicator (RSSI)) and / or network bandwidth.

[0080] Pending sub-prompt information (330) may be input into the generative AI model (217) and may include information about sub-prompts for which result generation is in progress. The term “pending sub-prompt” may be referred to as a term referring to sub-prompts for which result generation is in progress and which have been input into the generative AI model. The pending sub-prompt information (330) may include, for example, the content of the pending sub-prompt and / or the progress of the pending sub-prompt.

[0081] The temporary result data information (340) may include the result data generated by the pending sub-prompt until the time when the task interruption is detected. The term “temporary result data” may be referred to as the result data generated by the pending sub-prompt until the time when the task interruption is detected. Referring to FIGS. 8 and 11 , the temporary result data may be referenced when generating additional prompts in operations 835 and 1135. In one example, the electronic device (10) may store the temporary result data in the temporary result data information (340) when the task interruption is detected. In one example, the electronic device (10) may periodically store the temporary result data in the temporary result data information (340). In one example, the electronic device (10) may store the temporary result data in the temporary result data information (340) whenever a designated trigger is triggered.

[0082] Figure 4 illustrates the structure of prompt data according to one embodiment.

[0083] Referring to FIGS. 1, 2, 3, and 4, according to one embodiment, prompt data (235) may include information associated with a sub-prompt. For example, prompt data (235) may include sub-prompt type information (410), task request input information (420), a sub-prompt list (430), and / or a sub-prompt-specific result data list (440).

[0084] FIG. 4 illustrates information included in prompt data (235) in one embodiment, but embodiments of the present disclosure are not limited thereto. For example, the sub-prompt list (430) of the prompt data (235) may further include additional information related to each of the sub-prompts (e.g., progress for each sub-prompt). For example, the task request input information (420) of the prompt data (235) may further include additional information related to the entire task request (e.g., overall task progress).

[0085] The sub-prompt type information (410) may include at least one of information regarding whether the sub-prompt is a sub-prompt that needs to be processed in order and / or information regarding whether the sub-prompt is a sub-prompt that can be processed separately by element. The sub-prompt type information (410) may include information regarding whether the generated sub-prompts correspond to step-by-step sub-prompts or element-by-element sub-prompts.

[0086] The task request input information (420) may include information about the input (160) acquired by the electronic device (10). The task request input analysis module (221) and the sub-prompt generation module (223) may refer to the task request input information (420) to generate sub-prompts. For example, in the input (160) “Please draw a realistic picture of various flowers and squirrels in a forest at sunset,” the task request input information (420) may be “Please draw a realistic picture of various flowers and squirrels in a forest at sunset.”

[0087] The sub-prompt list (430) may include a list of sub-prompts generated based on the task request input information (420). The list of generated sub-prompts may be stored in the form of a list, an array, a queue, a stack, a deque, and / or other structures. The configuration of the sub-prompt list (430) illustrated in FIG. 4 is an example, and the configuration of the sub-prompt list (430) is not limited thereto. The sub-prompt list (430) may include more or fewer sub-prompts than those illustrated in FIG. 4.

[0088] The first sub-prompt (431) may include a plurality of sub-prompts that were input to the generative AI model (217) before the time at which the task interruption was detected and for which the result generation was completed. The “first sub-prompt” may be referred to as a term referring to a plurality of sub-prompts that were input to the generative AI model (217) before the time at which the task interruption was detected and for which the result generation was completed. Sub-prompt-a (433) and / or sub-prompt-b (435) may correspond to examples of the sub-prompts included in the first sub-prompt (341).

[0089] The second sub-prompt (437) may include sub-prompts whose progress is greater than or equal to a specified value when a task interruption is detected among the remaining sub-prompts excluding the first sub-prompt (431). The term “second sub-prompt” may be referred to as sub-prompts whose progress is greater than or equal to a specified value when a task interruption is detected among the remaining sub-prompts excluding the first sub-prompt (431). The second sub-prompt may include sub-prompts whose progress is greater than or equal to a specified value when a task interruption is detected among the pending sub-prompts. Referring to FIG. 11, whether or not to generate an additional prompt may be determined depending on whether the second sub-prompt (437) exists. The result data generated by the second sub-prompt (437) up to the point where an interruption is detected may be stored in the temporary result data information (340). The result data generated by the second sub-prompt (437) up to the point where an interruption is detected may be used to generate an additional prompt.

[0090] Among the pending sub-prompts, pending sub-prompts whose progress is less than a specified value as of the time when task interruption is detected may not be included in the second sub-prompt (437). Among the pending sub-prompts, pending sub-prompts whose progress is greater than a specified value as of the time when task interruption is detected may be included in the second sub-prompt (437). The progress of the second sub-prompt (437) may increase as the task is performed. The second sub-prompts (437) whose progress is 100% and task performance is completed may be excluded from the second sub-prompt. The second sub-prompts (437) whose progress is 100% and task performance is completed may be included in the first sub-prompt.

[0091] The configuration of the sub-prompt list (430) illustrated in FIG. 4 is an example, and the configuration of the sub-prompt list (430) is not limited thereto. The sub-prompt list (430) may include more or fewer sub-prompts than those illustrated in FIG. 4. For example, the sub-prompt list (430) may include sub-prompts whose result generation has been completed (e.g., the first sub-prompt (431)), sub-prompts whose progress rate among pending sub-prompts is greater than a specified value (e.g., the second sub-prompt (437)), sub-prompts whose progress rate among pending sub-prompts is less than a specified value (e.g., the remaining pending sub-prompts except the second sub-prompt), and sub-prompts whose result generation has not been completed, which are not pending after being input to the AI ​​model, and whose progress rate has a value that is 0 or within a specified range (e.g., an error range) from 0.

[0092] The result data list (440) for each sub-prompt may include a list listing result data generated by inputting sub-prompts into the generative AI model (217). The list listing the result data may be stored in the form of a list, array, queue, stack, deck, and / or other structure. The configuration of the result data list (440) for each sub-prompt illustrated in FIG. 4 is an example, and the configuration of the result data list (440) for each sub-prompt is not limited thereto. The result data list (440) for each sub-prompt may include more or less result data than that illustrated in FIG. 4.

[0093] The first result data (441) may include one or more result data generated by inputting the first sub-prompt (431) into the generative AI model (217). “First result data” may be referred to as a term meaning one or more result data generated by inputting the first sub-prompt (431) into the generative AI model (217). For example, result data-a (443) may correspond to result data generated by inputting sub-prompt-a (433) into the generative AI model (217). Result data-b (445) may correspond to result data generated by inputting sub-prompt-b (435) into the generative AI model (217).

[0094] Figure 5 is a flowchart of a method for generating a sub-prompt according to one embodiment.

[0095] The operations described below with reference to FIG. 5 may be referred to as operations of the electronic device (10) of FIG. 1. The order of the operations described below with reference to FIG. 5 is merely an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 5, or may be executed substantially simultaneously with other operations of FIG. 5. At least some of the operations described below with reference to FIG. 5 may be omitted.

[0096] Referring to FIGS. 1, 2, and 5, at operation 505, the electronic device (10) may obtain an input (160) requesting a task. For example, the electronic device (10) may obtain an input (160) from a user through an interface (150) and store the obtained input (160) in prompt data (235) through a task request input module (240). The task may include, for example, image generation, video generation, audio generation, and / or text generation.

[0097] In operation 510, the electronic device (10) may analyze the acquired input (160). For example, the electronic device (10) may analyze the acquired input (160) through the task request input analysis module (221). The electronic device (10) may identify keywords and intents from the task request input (160) through the task request input analysis module (221). Keywords may include tokens associated with a situation and / or background regarding the input (160). A token may be composed of at least one word. A standard for tokenization that divides the input into tokens may include, for example, word tokenization, sentence tokenization, syntactic tokenization, n-gram tokenization, syllable tokenization, semantic tokenization, and / or named entity recognition (NER). In one example, the electronic device (10) can tokenize the input (160) into tokens and select important tokens as keywords by assigning weights to the tokens (e.g., term frequency-inverse document frequency (TF-IDF) and / or an attention mechanism). For example, the electronic device (10) can select tokens with an importance level greater than a specified value among a plurality of tokens as keywords.

[0098] For example, in the input (160) “Please draw a realistic picture of various flowers and squirrels in a forest at sunset,” the keywords may include “sunset,” “forest,” “flowers,” and / or “squirrel.” The intent may be the result that the user wants to obtain from the input. For example, in the input (160) “Please draw a realistic picture of various flowers and squirrels in a forest at sunset,” the intent may be “Please draw a realistic picture.”

[0099] In operation 515, the electronic device (10) may determine whether the acquired input (160) is an input that can be divided into multiple sub-prompts. The electronic device (10) may determine whether the acquired input (160) can be divided into multiple sub-prompts by considering keywords and / or intent.

[0100] For example, if multiple keywords cannot be processed independently and interaction between multiple keywords is important for performing a task, the electronic device (10) may determine that the acquired input (160) is not an input that can be divided into multiple sub-prompts. For example, if performing a task by making the acquired input (160) into a single prompt reduces task complexity or shortens task performance time, the electronic device (10) may determine that the acquired input (160) is not an input that can be divided into multiple sub-prompts.

[0101] For example, the electronic device (10) may determine that the acquired input (160) is an input that can be divided into a plurality of sub-prompts if a plurality of keywords can be processed independently and the interaction between the plurality of keywords is not important for performing a task. For example, if dividing the acquired input (160) into a plurality of sub-prompts reduces task complexity or shortens task performance time, the electronic device (10) may determine that the acquired input (160) is an input that can be divided into a plurality of sub-prompts. In one example, the electronic device (10) may determine that the acquired input (160) can be divided into a plurality of sub-prompts if a plurality of keywords exist and the plurality of keywords are associated with a single intent.

[0102] When performing a task with a single prompt without dividing the acquired input into multiple sub-prompts, if an interruption is detected during task performance, the entire task may need to be restarted from the beginning when resuming the task after the interruption. When performing a task by dividing the acquired input into multiple sub-prompts, if an interruption is detected during task performance, as described later in FIG. 7, the entire task may not need to be restarted from the beginning, but the task may be restarted from the sub-prompt that was input to the generative AI model (217) and was pending at the time of the interruption.

[0103] If it is determined that the input can be divided into sub-prompts (e.g., operation 515-YES), the electronic device (10) can perform operation 520. If it is determined that the input cannot be divided into sub-prompts (e.g., operation 515-NO), the electronic device (10) can terminate the performance of generating the sub-prompt without performing any additional operations.

[0104] In operation 520, the electronic device (10) may generate a sub-prompt using the acquired input (160). The electronic device (10) may generate a plurality of sub-prompts from the acquired input (160) through the sub-prompt generation module (223). For example, the electronic device (10) may generate a plurality of sub-prompts by merging each of the plurality of keywords identified in operation 510 with an intent. For example, the electronic device (10) may identify a plurality of keywords such as “in the forest at sunset,” “various flowers,” and / or “squirrel,” and an intent of “realistic drawing,” from the input (160) “Please draw a realistic picture of various flowers and squirrels in a forest at sunset.” The electronic device (10) may combine each of the identified keywords with an intent to generate multiple sub-prompts, such as “Draw a picture of sunlight filtering through a jungle during the transition from day to night,” “Draw vibrant, colorful flowers,” and / or “Add a squirrel to create an extremely realistic animal with detailed animal fur and soft details.” In one example, the electronic device (10) may generate multiple sub-prompts that must be processed sequentially. The multiple sub-prompts that must be processed sequentially may be referred to as step-by-step sub-prompts. The step-by-step sub-prompts may be used to perform a task through sequential processing, as described below. In one example, the electronic device (10) may generate multiple sub-prompts that may be processed independently, regardless of the processing order. The multiple sub-prompts that may be processed independently, regardless of the processing order, may be referred to as element-by-element sub-prompts. The element-by-element sub-prompts may be used to perform a task through parallel processing, as described below. For example, if the task is to generate an image, the electronic device (10) may generate sub-prompts corresponding to a plurality of elements included in the image.For example, the plurality of elements may include objects and / or backgrounds recognizable from the image. For example, if the task is to generate code, the electronic device (10) may generate sub-prompts corresponding to each function having a single function. For example, the code may include at least one function having a single function. In one example, the plurality of sub-prompts for generating the code may be generated as step-by-step sub-prompts to share parameters between each function.

[0105] Figure 6 is a flowchart of a method for performing a task using sub-prompts according to one embodiment.

[0106] The operations described below with reference to FIG. 6 may be referred to as operations of the electronic device (10) of FIG. 1. The order of the operations described below with reference to FIG. 6 is merely an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 6, or may be executed substantially simultaneously with other operations of FIG. 6. At least some of the operations described below with reference to FIG. 6 may be omitted.

[0107] Referring to FIGS. 1, 2, 4, 5, and 6, in operation 605, the electronic device (10) may obtain an input (160) requesting a task. Referring to FIG. 5, in operation 605, the operation of the electronic device (10) obtaining an input (160) requesting a task may be referred to as described above in operation 505.

[0108] In operation 610, the electronic device (10) may analyze the acquired input (160). Referring to FIG. 5, in operation 610, the operation of the electronic device (10) analyzing the input (160) may be referred to as described above in operation 510.

[0109] In operation 615, the electronic device (10) may generate a sub-prompt using the acquired input (160). Referring to FIG. 5, in operation 615, the operation of generating a sub-prompt using the acquired input (160) by the electronic device (10) may be referred to as described above in operation 520.

[0110] In operation 620, the electronic device (10) may store a plurality of sub-prompts in the prompt data (235). For example, the plurality of sub-prompts generated in operation 615 may be stored in the sub-prompt list (430) in the form of a list, an array, a queue, a stack, a deck, and / or other structures. In one example, the electronic device (10) may assign reference numbers to the generated plurality of sub-prompts in the order in which they should be processed, and store them in the sub-prompt list (430) in the order of the reference numbers. For example, the sub-prompts stored in the sub-prompt list (430) in the order of the reference numbers may include step-by-step sub-prompts. In one example, when there is no priority in processing the plurality of sub-prompts, the electronic device (10) may store the generated plurality of sub-prompts in the sub-prompt list (430) regardless of the order. For example, the sub-prompts stored in the sub-prompt list (430) regardless of the order may include element-specific sub-prompts.

[0111] In operation 625, the electronic device (10) may input a sub-prompt to the generative AI model (217). The electronic device (10) may input the sub-prompt to the generative AI model (217) through a sub-prompt interpreter (225). The sub-prompt may be converted into a form understandable by the generative AI model (217) by being preprocessed through the sub-prompt interpreter (225) and input to the generative AI model (217). The sub-prompt may be preprocessed into a text token form and / or an image patch form through the sub-prompt interpreter (225) and input to the generative AI model (217). While referring to FIG. 1, the memory (110) is illustrated as storing one generative AI model (115), embodiments of the present disclosure are not limited thereto. For example, the memory (110) may store a plurality of generative AI models (115). The plurality of generative AI models (115) may include AI models of different types (e.g., LLM, LVM, and / or LMM). Even in the case of generative AI models (115) of the same type, generative AI models (115) trained based on different data sets may be referred to as different AI models (115). All or part of one generative AI model (115) may be implemented in an external device (20). All or part of the plurality of generative AI models (115) may be implemented in an external device (20). The external device (20) may be, for example, a smart home appliance, a VST device, a smart watch, a smart phone, a smart ring, and / or a cloud server.

[0112] In one example, the sub-prompt may be composed of a mixture of at least one of text data, image data, audio data, video data, graph data, and / or table data.

[0113] In one example, the electronic device (10) may sequentially input sub-prompts into a generative AI model by referring to the order of the sub-prompt list (430). The sub-prompts sequentially input into a generative AI model may be step-by-step sub-prompts.

[0114] In one example, the electronic device (10) may input sub-prompts to a plurality of generative AI models in parallel. The sub-prompts input to the plurality of generative AI models in parallel may be element-specific sub-prompts.

[0115] In operation 630, the electronic device (10) may generate result data for the sub-prompt. In one example, the result data may be generated through at least one layer within the generative AI model (217). The result data may be used to perform a task, as described below in FIG. 10.

[0116] In operation 635, the electronic device (10) may update the generated result data in the prompt data (235). For example, the result data for the sub-prompts generated in operation 630 may be stored in the sub-prompt-specific result data list (440) in the form of a list, array, queue, stack, deck, and / or other structure.

[0117] In operation 640, the electronic device (10) may determine whether result data has been generated for all sub-prompts. For example, the electronic device (10) may refer to the sub-prompt-specific result data list (440) and the sub-prompt list (430) to determine whether result data has been generated for all sub-prompts. For example, the electronic device (10) may determine that result data has been generated for all sub-prompts if the number of sub-prompts stored in the sub-prompt list (430) and the number of result data stored in the sub-prompt-specific result data list (440) are the same. For example, the electronic device (10) may determine that result data has not been generated for all sub-prompts if the number of sub-prompts stored in the sub-prompt list (430) is greater than the number of result data stored in the sub-prompt-specific result data list (440).

[0118] When it is determined that result data for all sub-prompts has been generated (e.g., action 640-YES), the electronic device (10) may terminate the task execution operation using the sub-prompts.

[0119] If it is determined that result data for all sub-prompts has not been generated (e.g., operation 640-NO), the electronic device (10) may input subsequent sub-prompts to the generative AI model (217) in operation 645. For example, the electronic device (10) may find subsequent sub-prompts by referencing reference numbers of sub-prompts stored in the sub-prompt list (430). The subsequent sub-prompt may be a prompt for which result data has not yet been generated, and may be a sub-prompt corresponding to a next number of the reference number of a previously processed sub-prompt.

[0120] Figure 7 is a flowchart of a method for performing a task when a task interruption is detected according to one embodiment.

[0121] The operations described below with reference to FIG. 7 may be referred to as operations of the electronic device (10) of FIG. 1. The order of the operations described below with reference to FIG. 7 is merely an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 7, or may be executed substantially simultaneously with other operations of FIG. 7. At least some of the operations described below with reference to FIG. 7 may be omitted.

[0122] Referring to FIGS. 1 to 4, 6 and 7, in operation 705, the electronic device (10) may input a sub-prompt to the generative AI model (217). Referring to FIG. 6, in operation 705, the operation of the electronic device (10) inputting a sub-prompt to the generative AI model (217) may be referred to as described above in operation 625.

[0123] In operation 710, the electronic device (10) may analyze AI status data (231). For example, the electronic device (10) may refer to software status information (310) and / or hardware status information (320) for analysis.

[0124] In operation 715, the electronic device (10) may determine whether a hardware or software abnormality is identified. For example, the electronic device (10) may analyze hardware status information (320) to identify a hardware abnormality. For example, the electronic device (10) may determine that a hardware abnormality is identified if it checks battery status information (321) and predicts that the battery remaining capacity (e.g., State of Charge) of the electronic device (10) will be depleted before the task is completed. For example, the electronic device (10) may analyze software status information (310) to identify a software abnormality. For example, the electronic device (10) may determine that a software abnormality is identified if a software process does not respond consecutively a specified number of times or more while performing a task.

[0125] The electronic device (10) can identify a software abnormality by checking the software status information (310). In one example, the electronic device (10) can determine that a software abnormality has been identified by checking direct information indicating a software abnormality included in the software status information (310). For example, if information that can identify sub-prompts that failed to generate result data is identified among the software status information (310), the electronic device (10) can determine that a software abnormality has been identified. For example, if information that can identify sub-prompts that succeeded in generating result data is not identified despite the passage of a specified amount of time after performing a task, the electronic device (10) can determine that a software abnormality has been identified.

[0126] If it is determined that a hardware or software anomaly has been identified (e.g., operation 715-YES), the electronic device (10) may perform operation 725. If it is determined that a hardware or software anomaly has not been identified (e.g., operation 715-NO), at operation 720, the electronic device (10) may determine whether other task interruptions are detected.

[0127] At operation 720, the electronic device (10) may determine whether an interruption of another task is detected. For example, the electronic device (10) may determine that an interruption of another task is detected if a specified amount of time has elapsed since the start of the execution of the task. For example, the electronic device (10) may determine that an interruption of another task is detected if the user accidentally presses a force-quit input of the electronic device (10) while the task is being performed. If it is determined that an interruption of another task is detected (e.g., operation 720—YES), the electronic device (10) may perform operation 725. If it is determined that an interruption of another task is not detected (e.g., operation 720—NO), at operation 735, the electronic device (10) may generate result data for the sub-prompt.

[0128] In operation 725, the electronic device (10) may generate or update AI status data (231). For example, information regarding hardware or software anomalies identified in operation 715 may be stored or updated in the AI ​​status data (231). When analyzing the AI ​​status data (231) in a subsequent operation, the electronic device (10) may reflect and analyze the information in the newly stored or updated AI status data (231).

[0129] In operation 730, the electronic device (10) may input a pending sub-prompt to the generative AI model (217). In one example, if the electronic device (10) fails to generate result data due to a task interruption, the electronic device may re-input the pending sub-prompt to the generative AI model to complete the generation of result data. In one example, the pending sub-prompt may include sub-prompts that have been input to the generative AI model (217) and whose result generation is in progress. For example, among multiple sub-prompts such as “Draw sunlight filtering into the jungle during the transition from day to night,” “Draw vividly colorful flowers,” and / or “Add a squirrel to create an extremely realistic animal with detailed animal fur and soft details,” if the sub-prompt “Draw vividly colorful flowers” ​​is input to a generative AI model and the task is performed, and the task interruption is detected and the generation is stopped, the pending sub-prompt may be the sub-prompt “Draw vividly colorful flowers.” The pending sub-prompt may be converted into a form understandable by the generative AI model (217) by being preprocessed by the sub-prompt interpreter (225) and inputted into the generative AI model (217). The pending sub-prompt may be preprocessed into a text token form and / or an image patch form by the sub-prompt interpreter (225) and inputted into the generative AI model (217).

[0130] In operation 735, the electronic device (10) may generate result data for the sub-prompt. Referring to FIG. 6, the operation of generating result data for the sub-prompt by the electronic device (10) in operation 735 may be referred to as described above in operation 630.

[0131] In operation 740, the electronic device (10) may determine whether result data has been generated for all sub-prompts. Referring to FIG. 6, the operation of determining whether result data has been generated for all sub-prompts in operation 740 may be referred to as described above in operation 640.

[0132] When it is determined that result data for all sub-prompts has been generated (e.g., action 740-YES), the electronic device (10) may terminate the task execution operation using the sub-prompts.

[0133] If it is determined that result data for all sub-prompts has not been generated (e.g., operation 740-NO), the electronic device (10) may input subsequent sub-prompts to the generative AI model (217) at operation 745. Referring to FIG. 6, the operation of the electronic device (10) inputting subsequent sub-prompts to the generative AI model at operation 745 may be referred to as described above in operation 645.

[0134] Figure 8 is a flowchart of a method for performing a task using additional prompts according to one embodiment.

[0135] The operations described below with reference to FIG. 8 may be referred to as operations of the electronic device (10) of FIG. 1. The order of the operations described below with reference to FIG. 8 is merely an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 8, or may be executed substantially simultaneously with other operations of FIG. 8. At least some of the operations described below with reference to FIG. 8 may be omitted.

[0136] Referring to FIGS. 1 to 4 and FIGS. 6 to 8, in operation 805, the electronic device (10) may input a sub-prompt to the generative AI model. In one example, the sub-prompt may be configured in a mixed form of at least one of text data, image data, audio data, video data, graph data, and table data. Referring to FIG. 6, in operation 805, the operation of the electronic device (10) inputting the sub-prompt to the generative AI model (217) may be referred to as the content described above in operation 625.

[0137] At operation 810, the electronic device (10) may determine whether a task interruption is detected. For example, the electronic device (10) may determine that a task interruption is detected if a hardware or software abnormality is identified, or if other task interruptions are detected. If it is determined that a task interruption is detected (e.g., operation 810-YES), the electronic device (10) may determine at operation 830 whether the progress of the pending sub-prompt is greater than or equal to a specified value. If it is determined that a task interruption is not detected (e.g., operation 810-NO), the electronic device (10) may generate result data for the sub-prompt or additional prompt at operation 815. Referring to FIG. 7, the operation of the electronic device (10) determining whether a task interruption is detected at operation 810 may be referred to as described above in operation 715 and / or operation 720.

[0138] In operation 815, the electronic device (10) may generate result data for a sub-prompt or an additional prompt. The operation of the electronic device (10) generating result data for a sub-prompt or an additional prompt in operation 815 may be referred to as described above in operation 630.

[0139] In operation 820, the electronic device (10) may determine whether result data has been generated for all sub-prompts. Referring to FIG. 6, the operation of determining whether result data has been generated for all sub-prompts in operation 820 may be referred to as described above in operation 640.

[0140] When it is determined that result data for all sub-prompts has been generated (e.g., action 820-YES), the electronic device (10) may terminate the task execution operation using the sub-prompts.

[0141] If it is determined that result data for all sub-prompts has not been generated (e.g., operation 820-NO), the electronic device (10) may input subsequent sub-prompts to the generative AI model (217) at operation 825. Referring to FIG. 6, the operation of the electronic device (10) inputting subsequent sub-prompts to the generative AI model at operation 825 may be referred to as described above in operation 645.

[0142] At operation 830, the electronic device (10) may determine whether the progress of the pending sub-prompt is greater than or equal to a specified value. In one example, the pending sub-prompt may be a sub-prompt that has been input to the generative AI model (217) and whose output is currently being generated. The pending sub-prompt information (330) may include the progress of the pending sub-prompt. If the progress of the pending sub-prompt is determined to be greater than or equal to the specified value (e.g., operation 830—YES), the electronic device (10) may generate an additional prompt at operation 835. If the progress of the pending prompt is determined to be not greater than or equal to the specified value (e.g., operation 830—NO), the electronic device (10) may re-input the pending sub-prompt to the generative model at operation 845.

[0143] In operation 835, the electronic device (10) may generate an additional prompt. The additional prompt may include a prompt generated using pending sub-prompt information (330) whose progress is greater than or equal to a specified value, the progress of the pending sub-prompt, and / or provisional result data information (340). When the electronic device (10) performs a task using the additional prompt, time and computing resources may be saved compared to repeating the task with the pending sub-prompt. In one example, the additional prompt may be composed of a mixed form of at least one of text data, image data, audio data, video data, graph data, and table data. The additional prompt may include a prompt generated by adding a current status and a request to the pending sub-prompt. The current status may include, for example, information obtained by analyzing task request input information (420) with reference to FIG. 4 (e.g., overall task progress). The current state may include, for example, information obtainable from the provisional result data information (340) as shown in FIG. 3 (e.g., result data information generated by the pending sub-prompt up to the point where the interruption was detected). The request may be, for example, a prompt requesting task resumption based on the pending sub-prompt and the current state. For example, the pending sub-prompt may be, “Please draw vividly colorful flowers.” For example, the current state for generating additional prompts may be, “The input image consists of a first image and a third image that are being generated in parallel with a second image that is incomplete due to a failure during the generation process. An interruption occurred during the generation task in the second layer of the second generative AI model that generates the second image. The overall data generation progress is 70%.” For example, the request for generating additional prompts may be, “Please resume image generation based on the requirements and the current state.”If the progress of the pending sub-prompt is greater than a specified value at the time when a task interruption is detected, the electronic device (10) may generate an additional prompt. The additional prompt may include a prompt generated by adding a current status and a request to the pending sub-prompt. For example, the additional prompt may be, “The requirement is to make the picture feel more vivid with colorful flowers. The currently input image consists of a first image and a third image that are being generated in parallel with a second image that is incomplete due to a failure during the generation process. An interruption occurred during the generation task in the second layer of the second generative AI model that generates the second image. The overall data generation progress is 70%. Please resume image generation based on the requirement, the currently input image, and the overall data generation progress.”

[0144] In operation 840, instead of the pending sub-prompt, the electronic device (10) may input an additional prompt into the generative AI model (217). The additional prompt may be converted into a form understandable by the generative AI model by being preprocessed by the sub-prompt interpreter (225) and inputted into the generative AI model. The additional prompt may be preprocessed into a text token form and / or an image patch form by the sub-prompt interpreter (225) and inputted into the generative AI model. While referring to FIG. 1, the memory (110) is illustrated as storing one generative AI model (115), embodiments of the present disclosure are not limited thereto. For example, the memory (110) may store a plurality of generative AI models (115). The plurality of generative AI models (115) may include AI models of different types (e.g., LLM, LVM, and / or LMM). Even in the case of generative AI models (115) of the same type, generative AI models (115) learned based on different data sets may be referred to as different AI models (115). All or part of one generative AI model (115) may be implemented in an external device (20). All or part of multiple generative AI models (115) may be implemented in an external device (20). The external device (20) may be, for example, a smart home appliance, a VST device, a smart watch, a smart phone, a smart ring, and / or a cloud server.

[0145] In one example, the electronic device (10) may, when an additional prompt is input, utilize information stored in the prompt data (235) together with the additional prompt to resume the task. For example, the electronic device (10) may, by referring to the prompt data (235), utilize additional information associated with the sub-prompt (e.g., the progress of the pending sub-prompt) and / or additional information associated with the overall task request (e.g., the progress of the overall task) together with the additional prompt to perform task resumption. For example, the progress of the pending sub-prompt may include the progress of the pending sub-prompt among the progress rates for each sub-prompt calculated by the generation completion prediction module (213) and inputted into the generative AI model (217). For example, the progress of the pending sub-prompt may include information on which layer the sub-prompt inputted into the generative AI model (217) was at when the task was interrupted. For example, the overall task progress rate may be a value obtained by dividing the elapsed time since the task performance was initiated by the overall task performance completion time predicted by the generation completion prediction module (213). For example, the overall task progress rate may be a ratio of the number of sub-prompts (e.g., the first sub-prompt (431)) that have been completely generated among the entire sub-prompts (e.g., the sub-prompts included in the sub-prompt list (430)) to the number of sub-prompts that have not been completely generated. For example, if the task is to generate an image, the overall task progress rate may include information on pixel areas of the entire image data to be generated where data has not yet been generated. For example, if the task is to generate text, the overall task progress rate may include information on chapters of the entire text to be generated that have not been generated by dividing the entire text to be generated into chapters.

[0146] In one example, if the task is to generate an image, the electronic device (10) may additionally use information about pixel areas of the entire image data to be generated where data has not yet been generated, along with an additional prompt, and / or information about up to which layer of the generative AI model (217) the task has been performed, along with information about the pending sub-prompts. For example, if the task is to generate text, the electronic device (10) may additionally use information about chapters of the entire text to be generated that have not been generated, along with an additional prompt, to resume the task.

[0147] In operation 845, the electronic device (10) may input a pending sub-prompt into the generative AI model (217). Referring to FIG. 7, the operation of the electronic device (10) inputting the pending sub-prompt into the generative AI model (217) in operation 845 may be referred to as described above in operation 730.

[0148] Figure 9 is a flowchart of a task performing method according to one embodiment.

[0149] The operations described below with reference to FIG. 9 may be referred to as operations of the electronic device (10) of FIG. 1. The order of the operations described below with reference to FIG. 9 is merely an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 9, or may be executed substantially simultaneously with other operations of FIG. 9. At least some of the operations described below with reference to FIG. 9 may be omitted.

[0150] With reference to FIGS. 1, 2, 4 to 7, and 9, in operation 905, the electronic device (10) may obtain an input (160) requesting a task. With reference to FIG. 5, the operation of the electronic device (10) obtaining an input (160) requesting a task in operation 905 may be referred to as described above in operation 505.

[0151] In operation 910, the electronic device (10) may generate a plurality of sub-prompts. Referring to FIG. 5, the operation of generating a plurality of sub-prompts by the electronic device (10) in operation 910 may be referred to as described above in operation 520.

[0152] In operation 915, the electronic device (10) may sequentially input a plurality of sub-prompts into one or more generative AI models (217). In one example, the electronic device (10) may sequentially input the sub-prompts into one generative AI model by referring to the order of the sub-prompt list (430). The sub-prompts sequentially input into one generative AI model may be step-by-step sub-prompts. Referring to FIG. 6, the operation of the electronic device (10) sequentially inputting a plurality of sub-prompts into one or more generative AI models (217) in operation 915 may be referred to as the content described above in operation 625.

[0153] At operation 920, the electronic device (10) may determine whether a task interruption is detected during the execution of the task. For example, the electronic device (10) may determine that a task interruption is detected when a hardware or software abnormality is identified, or when other task interruptions are detected. If it is determined that a task interruption is detected during the execution of the task (e.g., operation 920-YES), the electronic device (10) may perform operation 925. If it is determined that a task interruption is not detected during the execution of the task (e.g., operation 920-NO), the electronic device (10) may perform operation 940. Referring to FIG. 7, the operation of the electronic device (10) determining whether a task interruption is detected at operation 920 may be referred to as described above in operation 715 and / or operation 720.

[0154] In operation 925, the electronic device (10) may store first result data (441). The first result data (441) may include one or more result data generated by inputting the first sub-prompt (431) into the generative AI model (217).

[0155] At operation 930, the electronic device (10) may request a task resume. For example, task resume may be referred to as an operation of inputting sub-prompts into the generative AI model (217) for the remaining task performance after the task performance was interrupted.

[0156] In operation 935, the electronic device (10) may perform a task with the remaining sub-prompts except for the first sub-prompt. The first sub-prompt (431) may include a plurality of sub-prompts that were input to the generative AI model (217) before the time point at which the task interruption was detected and for which result generation was completed. In one example, the operation of performing the task with the remaining sub-prompts except for the first sub-prompt may be referred to as an operation of generating result data using sub-prompts for which result data has not yet been generated as of the time point at which the task was interrupted.

[0157] In operation 940, the electronic device (10) may perform a task. For example, the task performance may be referred to as an operation of inputting a sub-prompt into a generative AI model (217) to generate result data, and using the result data to generate a final result.

[0158] Figure 10 is a flowchart of a task execution method by sub-prompt type according to one embodiment.

[0159] The operations described below with reference to FIG. 10 may be referred to as the operations of the electronic device (10) of FIG. 1. The order of the operations described below with reference to FIG. 10 is merely an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be performed in a different order from that of FIG. 10, or may be performed substantially simultaneously with other operations of FIG. 10. At least some of the operations described below with reference to FIG. 10 may be omitted.

[0160] With reference to FIGS. 1, 2, 4 to 6, and 10, in operation 1005, the electronic device (10) may obtain an input (160) requesting a task. With reference to FIG. 5, the operation of the electronic device (10) obtaining an input (160) requesting a task in operation 1005 may be referred to as described above in operation 505.

[0161] In operation 1010, the electronic device (10) may generate a plurality of sub-prompts. Referring to FIG. 5, the operation of the electronic device (10) generating a plurality of sub-prompts in operation 1010 may be referred to as described above in operation 520.

[0162] In operation 1015, the electronic device (10) may determine whether a plurality of sub-prompts should be processed in sequence. For example, if the plurality of generated sub-prompts are step-by-step sub-prompts, the electronic device (10) may determine that the plurality of sub-prompts should be processed in sequence. For example, if the plurality of generated sub-prompts are element-specific sub-prompts, the electronic device (10) may determine that the plurality of sub-prompts do not need to be processed in sequence. The electronic device (10) may determine whether the plurality of sub-prompts should be processed in sequence by referring to sub-prompt type information (410). The sub-prompt type information (410) may include information on whether the sub-prompts are sub-prompts that need to be processed in sequence. The sub-prompt type information (410) may include information on whether the generated sub-prompts correspond to step-by-step sub-prompts or element-specific sub-prompts.

[0163] If it is determined that multiple sub-prompts should be processed sequentially (e.g., operation 1015-YES), in operation 1020, the electronic device (10) may input the sub-prompts to the generative AI model (217) in sequence. The operation of inputting the sub-prompts to the generative AI model (217) in sequence may be referred to as an operation of waiting until the generation of result data for the input sub-prompt is completed and inputting the subsequent sub-prompt to the generative AI model (217) after the generation of result data is completed.

[0164] If it is determined that the multiple sub-prompts do not need to be processed sequentially (e.g., operation 1015-NO), then in operation 1035 the electronic device (10) may input the multiple sub-prompts into one or more generative AI models (217).

[0165] In operation 1025, the electronic device (10) may perform a task by processing the result of a previous sub-prompt using a subsequent sub-prompt. The operation of performing a task by processing the result of a previous sub-prompt using a subsequent sub-prompt may be referred to as “task performance through sequential processing.” In operation 1025, step-by-step sub-prompts may be used to perform the task through sequential processing.

[0166] In operation 1030, the electronic device (10) may determine whether the task is completed. Referring to FIG. 6, in operation 1030, the operation of the electronic device (10) determining whether the task is completed may be referred to as described above in operation 640.

[0167] If the task is determined to be completed (e.g., operation 1030-YES), the electronic device (10) may terminate task execution. If the task is determined to be not completed (e.g., operation 1030-NO), in operation 1020, the electronic device (10) may input the next sub-prompt to the generative AI model in sequence.

[0168] In operation 1035, the electronic device (10) may input a plurality of sub-prompts into one or more generative AI models (217). The operation of inputting a plurality of sub-prompts into one or more generative AI models (217) may be referred to as an operation of inputting a plurality of sub-prompts, which may be processed independently regardless of order, into one or more generative AI models simultaneously, thereby performing parallel processing.

[0169] In operation 1040, the electronic device (10) may perform a task by merging multiple result data from multiple sub-prompts. The operation of performing a task by merging multiple result data from multiple sub-prompts may be referred to as “task performance through parallel processing.” In operation 1040, element-specific sub-prompts may be used to perform a task through parallel processing.

[0170] Figure 11 is a flowchart of a method for performing a task using additional prompts according to one embodiment.

[0171] The operations described below with reference to FIG. 11 may be referred to as operations of the electronic device (10) of FIG. 1. The order of the operations described below with reference to FIG. 11 is merely an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 11, or may be executed substantially simultaneously with other operations of FIG. 11. At least some of the operations described below with reference to FIG. 11 may be omitted.

[0172] With reference to FIGS. 1, 2, 3, 5, 7, 8, 10, and 11, in operation 1105, the electronic device (10) may obtain an input (160) requesting a task. With reference to FIG. 5, the operation of the electronic device (10) obtaining an input (160) requesting a task in operation 1105 may be referred to as described above in operation 505.

[0173] In operation 1110, the electronic device (10) may generate a plurality of sub-prompts. Referring to FIG. 5, the operation of generating a plurality of sub-prompts by the electronic device (10) in operation 1110 may be referred to as described above in operation 520.

[0174] In operation 1115, the electronic device (10) may sequentially input a plurality of sub-prompts into one or more generative AI models. Referring to FIG. 10, the operation of sequentially inputting a plurality of sub-prompts into one or more generative AI models may be referred to as the operation of sequentially inputting sub-prompts into a generative AI model (1025) and / or the operation of sequentially inputting a plurality of sub-prompts into one or more generative AI models (1035).

[0175] In operation 1120, the electronic device (10) may determine whether a task interruption is detected during the execution of the task. For example, the electronic device (10) may determine that a task interruption has been detected if a hardware or software abnormality is identified, or if other task interruptions are detected. Referring to FIG. 7, the operation of the electronic device (10) determining whether a task interruption is detected in operation 1120 may be referred to as described above in operation 715 and / or operation 720.

[0176] If it is determined that an interruption of the task is detected during the execution of the task (e.g., operation 1120-YES), then in operation 1125, the electronic device (10) may store temporary result data. If it is determined that an interruption of the task is not detected during the execution of the task (e.g., operation 1120-NO), then in operation 1150, the electronic device (10) may perform the task.

[0177] In operation 1125, the electronic device (10) may store temporary result data in temporary result data information (340). The temporary result data may include result data generated by the pending sub-prompt until the time when the task interruption is detected. If a second sub-prompt (437) exists, the temporary result data generated by the second sub-prompt (437) among the temporary result data may be referred to as “second result data.”

[0178] In operation 1130, the electronic device (10) may determine whether there is a second sub-prompt (437). For example, when a task execution interruption is detected, the electronic device (10) may determine that there is a second sub-prompt (437) if there are sub-prompts with a progress rate greater than or equal to a specified value among the remaining sub-prompts excluding the first sub-prompt (431). For example, when a task execution interruption is detected, the electronic device (10) may determine that there is no second sub-prompt (437) if there are no sub-prompts with a progress rate greater than or equal to a specified value among the remaining sub-prompts excluding the first sub-prompt (431). When it is determined that there is a second sub-prompt (437) when a task execution interruption is detected (e.g., operation 1130-YES), the electronic device (10) may perform operation 1135. If it is determined that there is no second sub-prompt (437) when a task execution interruption is detected (e.g., operation 1130-NO), then in operation 1155, the electronic device (10) may request a task resumption.

[0179] In operation 1135, the electronic device (10) may generate an additional prompt based on the second result data. The term “second result data” may be referred to as a term referring to temporary result data according to the second sub-prompt (437) among the temporary result data. Referring to FIG. 8, the operation of the electronic device (10) generating an additional prompt in operation 1135 may be referred to as the content described above in operation 835.

[0180] In operation 1140, the electronic device (10) may request a task resumption. Referring to FIG. 9, in operation 1140, the operation of the electronic device (10) requesting a task resumption may be referred to as described above in operation 925.

[0181] In operation 1145, the electronic device (10) may perform the task using an additional prompt instead of the second sub-prompt (437). The additional prompt may be a prompt generated using information from the second sub-prompt (437), the progress of the second sub-prompt (437), and / or second result data information. When the electronic device (10) performs the task using the additional prompt, it may save time and computing resources compared to repeating the task using the second sub-prompt (437).

[0182] In operation 1150, the electronic device (10) may perform a task. If a task interruption is not detected, the electronic device (10) may perform and terminate the task without going through the task interruption and / or task resumption process. Referring to FIG. 9, the operation of the electronic device (10) performing the task in operation 1150 may be referred to as described above in operation 935.

[0183] In operation 1155, the electronic device (10) may request task resumption. Referring to FIG. 9, in operation 1155, the operation of the electronic device (10) requesting task resumption may be referred to as described above in operation 925.

[0184] In operation 1160, the electronic device (10) may perform a task with the remaining sub-prompts except for the first sub-prompt (431). In one example, the operation of performing the task with the remaining sub-prompts except for the first sub-prompt (431) may be referred to as an operation of generating result data using sub-prompts for which result data has not yet been generated as of the time point at which the task was interrupted. Referring to FIG. 9, in operation 1160, the operation of the electronic device (10) performing the task with the remaining sub-prompts except for the first sub-prompt (431) may be referred to as the content described above in operation 935.

[0185] FIG. 12 is a flowchart of a method for displaying result data after task interruption detection according to one embodiment.

[0186] The operations described below with reference to FIG. 12 may be referred to as operations of the electronic device (10) of FIG. 1. The order of the operations described below with reference to FIG. 12 is merely an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 12, or may be executed substantially simultaneously with other operations of FIG. 12. At least some of the operations described below with reference to FIG. 12 may be omitted.

[0187] With reference to FIGS. 1, 2, 4, 5, 7, 9, 11, and 12, in operation 1205, the electronic device (10) may obtain an input (160) requesting a task. With reference to FIG. 5, the operation of the electronic device (10) obtaining an input (160) requesting a task in operation 1205 may be referred to as described above in operation 505.

[0188] In operation 1210, the electronic device (10) may generate a plurality of sub-prompts. Referring to FIG. 5, the operation of generating a plurality of sub-prompts by the electronic device (10) in operation 1210 may be referred to as described above in operation 520.

[0189] In operation 1215, the electronic device (10) may sequentially input a plurality of sub-prompts into one or more generative AI models (217). Referring to FIG. 11, the operation of the electronic device (10) sequentially inputting a plurality of sub-prompts into one or more generative AI models (217) in operation 1215 may be referred to as the content described above in operation 1115.

[0190] In operation 1220, the electronic device (10) may determine whether a task interruption is detected during the execution of the task. For example, the electronic device (10) may determine that a task interruption has been detected if a hardware or software abnormality is identified, or if other task interruptions are detected. Referring to FIG. 7, the operation of the electronic device (10) determining whether a task interruption is detected in operation 1220 may be referred to as described above in operation 715 and / or operation 720.

[0191] If it is determined that an interruption of the task is detected during the execution of the task (e.g., operation 1220-YES), the electronic device (10) may perform operation 1225. If it is determined that an interruption of the task is not detected during the execution of the task (e.g., operation 1220-NO), the electronic device (10) may perform the task at operation 1240.

[0192] In operation 1225, the electronic device (10) may request task resumption. Referring to FIG. 9, in operation 1140, the operation of the electronic device (10) requesting task resumption may be referred to as described above in operation 925.

[0193] In operation 1230, the electronic device (10) can perform a task with the remaining sub-prompts except for the first sub-prompt (431). Referring to FIG. 11, in operation 1230, the operation of the electronic device (10) performing a task with the remaining sub-prompts except for the first sub-prompt (431) can be referred to as described above in operation 1160.

[0194] In operation 1235, the electronic device (10) can display the result data generated after the task is resumed on the display (130). The result data generated after the task is resumed may include the remaining result data excluding the first result data (441) among the entire result data. For example, the electronic device (10) can display the remaining result data excluding the first result data on the display (130). By displaying only the result data generated after the task is resumed on the display (130), the electronic device (10) can prevent user inconvenience caused by displaying the result data redundantly on the display (130) and provide an intuitive UI (user interface) that removes the redundant result data from the result data to be displayed.

[0195] In operation 1240, the electronic device (10) may perform a task. If a task interruption is not detected, the electronic device (10) may perform and terminate the task without going through the task interruption and / or task resumption process. Referring to FIG. 9, the operation of the electronic device (10) performing the task in operation 1240 may be referred to as described above in operation 935.

[0196] In operation 1245, the electronic device (10) can display the entire task execution result on the display (130). If the interruption of the task is not detected, there may not be any result data that is displayed repeatedly on the display (130). In this case, even if the electronic device (10) displays the entire task execution result on the display (130), this does not cause any inconvenience to the user, and therefore, the electronic device (10) can display the entire task execution result on the display (130).

[0197] Figure 13 is a flowchart of a method for performing tasks in multiple applications according to one embodiment.

[0198] The operations described below with reference to FIG. 13 may be referred to as operations of the electronic device (10) of FIG. 1. The order of the operations described below with reference to FIG. 13 is merely an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 13, or may be executed substantially simultaneously with other operations of FIG. 13. At least some of the operations described below with reference to FIG. 13 may be omitted.

[0199] Referring to FIG. 13, for example, a first application (1305) may include at least one of a plurality of applications for document work (e.g., PowerPoint, Word, and / or Excel). For example, a second application (1310) may include at least one of a plurality of applications for document work other than the first application (1305).

[0200] Referring to FIG. 13, in operation 1315, the electronic device (10) may obtain a task request input through the first application (1305). For example, the electronic device (10) may obtain an input (160) from a user through the interface (150) and store the obtained input (160) in prompt data (235). The task may be, for example, a request for image generation, video generation, audio generation, and / or text generation.

[0201] In operation 1320, the electronic device (10) may detect a task execution interruption in the first application (1305) and transmit a task execution request to another application. The other application may include, for example, a second application (1310). For example, when an abnormality occurs in the first application (1305) and a task execution interruption is detected, the electronic device (10) may transmit a task execution request to the second application (1310) so that the task input to the first application (1305) can be performed instead by the second application (1310). When transmitting the task execution request, the electronic device (10) may select and transmit a task that can also be performed by the second application (1310) among the task requests input to the first application (1305). For example, if a task to create a table is interrupted in a first application (1305) (e.g., PowerPoint), the task to create a table can also be performed in a second application (1310) (e.g., Word), and therefore, a request to perform the task to create a table can be passed to the second application (1310).

[0202] At operation 1325, the electronic device (10) may request the second application (1310) to perform a task so that the task can be performed.

[0203] In operation 1330, the electronic device (10) may perform a task in a second application (1310) to generate result data and transmit the generated result data to the first application (1305). For example, the generated result data may include at least one of a table, text, image, and / or other possible work results of a document creation application.

[0204] In operation 1335, the first application (1305) can receive the result data transmitted from the second application (1310). For example, the electronic device (10) can include a shared workspace that can share data between the first application (1305) and the second application (1310). For example, the electronic device (10) can include a layer that can exchange data between applications so that the first application (1305) and the second application (1310) can transmit and / or receive data.

[0205] In operation 1340, the electronic device (10) may complete the task execution. For example, the electronic device (10) may complete the task execution using the result data generated by the second application (1310) received by the first application (1305). In one example, the result data received by the first application in operation 1335 may be result data for the entire task request input acquired in operation 1315. In one example, the result data received by the first application in operation 1335 may be result data for a portion of the task request input acquired in operation 1315.

[0206] Figure 14a illustrates a process of performing an image generation task according to one embodiment.

[0207] The image generation task execution process illustrated in Fig. 14a is exemplary, and the process of performing the image generation task through a generative AI model is not limited thereto.

[0208] The process of sequentially performing an image generation task can be illustrated separately in FIGS. 14A and 14B. In FIG. 14A, each of the first generative AI model (1405), the second generative AI model (1410), and / or the third generative AI model (1415) can correspond to each of the first generative AI model (1405), the second generative AI model (1410), and / or the third generative AI model (1415) in FIG. 14B. Each of the second intermediate result data-a (1450), the occurrence of an error (1455), and / or the second intermediate result data-c (1460) in FIG. 14A can correspond to each of the second intermediate result data-a (1450), the occurrence of an error (1455), and / or the second intermediate result data-c (1460) in FIG. 14B.

[0209] Referring to FIGS. 1, 2, 7, and 14A, according to an embodiment, the electronic device (10) may include a plurality of generative AI models (e.g., a first generative AI model (1405), a second generative AI model (1410), and / or a third generative AI model (1415)). While FIG. 1 illustrates that the memory (110) stores one generative AI model (115), embodiments of the present disclosure are not limited thereto. For example, the memory (110) may store a plurality of generative AI models. The plurality of generative AI models (e.g., the first generative AI model (1405), the second generative AI model (1410), and / or the third generative AI model (1415)) may include AI models of different types (e.g., LLM, LVM, and / or LMM). The plurality of generative AI models (e.g., the first generative AI model (1405), the second generative AI model (1410), and / or the third generative AI model (1415)) may include a Diffusion model, a Stable Diffusion model, and / or a GAN model. Even in the case of generative AI models of the same type, generative AI models trained based on different data sets may be referred to as different AI models. For example, the generative AI model (115) may be stored in an external cloud server and / or data center. When the generative AI model is stored in the external server, the electronic device (10) may communicate with the external server via the communication circuit (140) to request a task from the generative AI model stored in the external server or receive result data generated by the generative AI model.

[0210] In one example, a plurality of generative AI models (e.g., a first generative AI model (1405), a second generative AI model (1410), and / or a third generative AI model (1415)) can share information with each other. For example, each of the plurality of generative AI models can share information through a common database, such as AI state data (231) and / or prompt data (235). For example, at least one or more hidden layers of each of the plurality of generative AI models can be structured to be connected to at least one or more hidden layers of other generative AI models for information sharing. Each of the plurality of generative AI models can reference information of other generative AI models through the connected structure.

[0211] In one example, the sub-prompts may include sub-prompt-a (1420), sub-prompt-b (1425), and / or sub-prompt-c (1430). In one example, the sub-prompts (e.g., sub-prompt-a (1420), sub-prompt-b (1425), and / or sub-prompt-c (1430)) may be composed of at least one mixed form of text data, image data, audio data, video data, graph data, and table data.

[0212] Sub-prompt-a (1420) may include sub-prompts input to the first generative AI model to perform tasks through parallel processing. For example, if the input (160) “Please draw a realistic picture of a forest at dusk with various flowers and a squirrel,” generates multiple sub-prompts such as “Please draw a picture of sunlight filtering into the jungle during the transition from day to night,” “Please draw vividly colored flowers,” and “Please add a squirrel to create an extremely realistic animal with detailed animal fur and soft details,” then sub-prompt-a may be “Please draw a picture of sunlight filtering into the jungle during the transition from day to night.”

[0213] Sub-prompt-b (1425) may include sub-prompts input to the second generative AI model to perform tasks through parallel processing. For example, if the input (160) “Please draw a realistic picture of various flowers and squirrels in a forest at dusk” generates multiple sub-prompts such as “Please draw a picture of sunlight filtering into the jungle during the transition from day to night,” “Please draw vividly colorful flowers,” and “Please add a squirrel to create an extremely realistic animal with detailed animal fur and soft details,” then sub-prompt-b may be “Please draw vividly colorful flowers.”

[0214] Sub-prompt-c (1430) may include sub-prompts input to the third generative AI model to perform tasks through parallel processing. For example, if the input (160) “Draw a realistic picture of a forest at dusk with various flowers and a squirrel,” generates multiple sub-prompts such as “Draw a picture of sunlight filtering into the jungle during the transition from day to night,” “Draw vividly colored flowers,” and “Add a squirrel to create an extremely realistic animal with detailed animal fur and soft details,” then sub-prompt-c may be “Add a squirrel to create an extremely realistic animal with detailed animal fur and soft details.”

[0215] The first intermediate result data-a (1435) may include an intermediate production result for the sub-prompt-a (1420). The term “intermediate production result” may be referred to as an incomplete result data generated in the process of generating result data for the sub-prompt. For example, the first intermediate result data-a (1435) may be an intermediate production result for the sub-prompt-a (1420) with a progress rate of 30%. The first intermediate result data-a (1435) may be an intermediate production result having the same progress rate as the first intermediate result data-b (1440) and / or the first intermediate result data-c (1445). The first intermediate result data-a (1435), the first intermediate result data-b (1440), and / or the first intermediate result data-c (1445) having the same progress rate may be referred to as intermediate production results of the same stage.

[0216] The first intermediate result data-b (1440) may include an intermediate production result for the sub-prompt-b (1425). For example, the first intermediate result data-b (1440) may be an intermediate production result for the sub-prompt-b (1425) with a progress rate of 30%. The first intermediate result data-b (1440) may be an intermediate production result with the same progress rate as the first intermediate result data-a (1435) and / or the first intermediate result data-c (1445). The first intermediate result data-a (1435), the first intermediate result data-b (1440), and / or the first intermediate result data-c (1445) with the same progress rate may be referred to as intermediate production results of the same stage.

[0217] The first intermediate result data-c (1445) may include an intermediate production result for the sub-prompt-c (1430). For example, the first intermediate result data-c (1445) may be an intermediate production result for the sub-prompt-c (1430) with a progress rate of 30%. The first intermediate result data-c (1440) may be an intermediate production result with the same progress rate as the first intermediate result data-a (1435) and / or the first intermediate result data-b (1440). The first intermediate result data-a (1435), the first intermediate result data-b (1440), and / or the first intermediate result data-c (1445) with the same progress rate may be referred to as intermediate production results of the same stage.

[0218] The second intermediate result data-a (1450) may include an intermediate production result for the sub-prompt-a (1420). For example, the second intermediate result data-a (1450) may be an intermediate production result for the sub-prompt-a (1420) with a progress rate of 70%. The second intermediate result data-a (1450) may be an intermediate production result with the same progress rate as the second intermediate result data-c (1460). The second intermediate result data-a (1450) and / or the second intermediate result data-c (1460) with the same progress rate may be referenced as intermediate production results of the same stage.

[0219] When referring to FIG. 7, an error occurrence (1455) may correspond to a hardware or software abnormality identified by the electronic device (10) in operation 715, or other task interruption detected by the electronic device (10) in operation 720. If an error occurrence (1455) occurs, the result data generation task for sub-prompt-b in the second generative AI model (1410) may be stopped.

[0220] The second intermediate result data-c (1460) may include an intermediate production result for the sub-prompt-c (1430). For example, the second intermediate result data-c (1460) may be an intermediate production result for the sub-prompt-c (1430) with a progress rate of 70%. The second intermediate result data-c (1460) may be an intermediate production result with the same progress rate as the second intermediate result data-a (1450). The second intermediate result data-a (1450) and / or the second intermediate result data-c (1460) with the same progress rate may be referenced as intermediate production results of the same stage.

[0221] Figure 14b illustrates a process of performing an image generation task according to one embodiment.

[0222] The image generation task execution process illustrated in Fig. 14b is exemplary, and the process of performing an image generation task using a generative AI model is not limited thereto.

[0223] The process of sequentially performing an image generation task can be illustrated separately in FIGS. 14A and 14B. In FIG. 14A, each of the first generative AI model (1405), the second generative AI model (1410), and / or the third generative AI model (1415) can correspond to each of the first generative AI model (1405), the second generative AI model (1410), and / or the third generative AI model (1415) in FIG. 14B. Each of the second intermediate result data-a (1450), the occurrence of an error (1455), and / or the second intermediate result data-c (1460) in FIG. 14A can correspond to each of the second intermediate result data-a (1450), the occurrence of an error (1455), and / or the second intermediate result data-c (1460) in FIG. 14B.

[0224] With reference to FIGS. 1, 2, 7, 8, 14a, and 14b, according to one embodiment, the electronic device (10) may include a plurality of generative AI models (e.g., a first generative AI model (1405), a second generative AI model (1410), and / or a third generative AI model (1415)). The first generative AI model (1405), the second generative AI model (1410), and / or the third generative AI model (1415) illustrated in FIG. 14b may be referred to as described above in FIG. 14a.

[0225] The second intermediate result data-a (1450) may include an intermediate generation result for the sub-prompt-a (1420). The second intermediate result data-a (1450) illustrated in FIG. 14b may be referenced as described above in FIG. 14a.

[0226] An error occurrence (1455) may correspond to a hardware or software abnormality identified by the electronic device (10) in operation 715, or other task interruption detected by the electronic device (10) in operation 720, as shown in FIG. 7. The error occurrence (1455) illustrated in FIG. 14b may be referred to as described above in FIG. 14a.

[0227] The second intermediate result data-c (1460) may include an intermediate generation result for the sub-prompt-c (1430). The second intermediate result data-c (1460) illustrated in FIG. 14b may be referenced as described above in FIG. 14a.

[0228] The additional prompt (1465) may include an additional prompt generated by the additional prompt generation operation (835) as shown in FIG. 8. The additional prompt (1465) may include an additional prompt generated by the additional prompt generation operation (1130) based on the second result data as shown in FIG. 11. The electronic device (10) may use the additional prompt (1465) to resume a task after it has been interrupted. When the electronic device (10) uses the additional prompt (1465), the first intermediate result data-b does not need to be regenerated, thereby saving generation time and resources required for generation. The additional prompt (1465) may include information about a current state as shown in FIG. 8. For example, the current state may include information (e.g., overall task progress) obtained by analyzing task request input information (420) as shown in FIG. 4. The current state may include, for example, information that can be obtained from the temporary result data information (340) (e.g., result data information generated by the pending sub-prompt up to the point where the interruption was detected) when referring to FIG. 3.

[0229] The second intermediate result data-b (1470) may include a result generated using the additional prompt (1465). For example, the second intermediate result data-b (1470) may include an intermediate generated result for the sub-prompt-b (1425) whose progress is 70%. The second intermediate result data-b (1470) may be an intermediate generated result having the same progress as the second intermediate result data-a (1450) and / or the second intermediate result data-c (1460). The second intermediate result data-a (1450), the second intermediate result data-b (1470), and / or the second intermediate result data-c (1460) having the same progress may be referred to as intermediate generated results of the same stage.

[0230] The third intermediate result data-a (1475) may include an intermediate generation result for the sub-prompt-a (1420). The third intermediate result data-a (1475) may be a result of merging intermediate generation result information of other generative AI models by referencing intermediate generation result information of other generative AI models (e.g., the second generative AI model (1410), the third generative AI model (1415)).

[0231] The third intermediate result data-b (1480) may include an intermediate generation result for the sub-prompt-b (1425). The third intermediate result data-b (1480) may be a result of merging intermediate generation result information of other generative AI models by referencing intermediate generation result information of other generative AI models (e.g., the first generative AI model (1405), the third generative AI model (1415)).

[0232] The third intermediate result data-c (1485) may include an intermediate generation result for the sub-prompt-c (1430). The third intermediate result data-c (1485) may be a result of merging intermediate generation result information of other generative AI models by referencing intermediate generation result information of other generative AI models (e.g., the first generative AI model (1405), the second generative AI model (1410)).

[0233] The final result data (1490) may include the result of the entire task being performed. For example, the final result data (1490) may be a result generated by merging each of the result data for each of the sub-prompt-a (1420), the sub-prompt-b (1425), and / or the sub-prompt-c (1430). For example, the final result data (1490) may be a result generated by merging the third intermediate result data-a (1475), the third intermediate result data-b (1480), and / or the third intermediate result data-c (1485). The final result data (1490) may be a result identical to or similar to the result data obtained by inputting the entire task request input (160) acquired by the electronic device (10) into the generative AI model as shown in FIG. 1.

[0234] FIG. 15 illustrates various external devices capable of communicating with an electronic device according to one embodiment.

[0235] The external devices illustrated in FIG. 15 are exemplary, and the various external devices capable of communicating with the electronic device (10) are not limited thereto. For example, the external device may further include external devices not illustrated in FIG. 15 (e.g., VST devices, cloud servers). For example, the external device may not include at least one of the external devices illustrated in FIG. 15 (e.g., smart ring (1520)). The various external devices illustrated in FIG. 15 capable of communicating with the electronic device (10) may be referred to as external devices (20) when referring to FIG. 1.

[0236] Referring to FIGS. 1 to 4 and FIG. 15, according to one embodiment, an electronic device (1505) may be referred to as the electronic device (10) of FIG. 1. Although FIG. 15 illustrates the electronic device (1505) as a smart phone, the electronic device (1505) is not limited to a smart phone. For example, various devices (e.g., a smart ring, a VST device, a smart watch, a smart home appliance) may be the electronic device (1505).

[0237] The first external device (1510) may include a smart home appliance. The second external device (1515) may include a smart watch. The third external device (1520) may include a smart ring. Unlike the example of FIG. 15, the first external device (1510), the second external device (1520), and / or the third external device (1530) may be any device, including, for example, a smart ring, a VST device, a smart watch, a smart phone, or a cloud server.

[0238] FIG. 16 is a flowchart of a method for performing a task in multiple electronic devices according to one embodiment.

[0239] The operations described below with reference to FIG. 16 may be referred to as operations of the electronic device (10) and / or the external device (20) of FIG. 1. The order of the operations described below with reference to FIG. 16 is merely an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 16, or may be executed substantially simultaneously with other operations of FIG. 16. At least some of the operations described below with reference to FIG. 16 may be omitted.

[0240] Referring to FIGS. 1 and 16, the electronic device (1605) may be referred to as the electronic device (10). Referring to FIGS. 1 and 16, the external device (1610) may be referred to as the external device (20).

[0241] Referring to FIGS. 15 and 16, the electronic device (1605) may be referred to as electronic device (1505). Referring to FIGS. 15 and 16, the external device (1610) may be referred to as one of the first external device (1510), the second external device (1515), and / or the third external device (1520).

[0242] Referring to FIGS. 1, 2, and 16, at operation 1615, the electronic device (1605) may obtain a task request input. For example, the electronic device (10) may obtain an input (160) from a user through the interface (150) and store the obtained input (160) in prompt data (235). The task may be, for example, a request to generate an image, generate a video, or generate text.

[0243] In operation 1620, the electronic device (1605) may detect a task interruption and transmit a task execution request to the external device (1610). For example, the electronic device (1605) may determine that a task interruption has been detected if a hardware or software abnormality is identified during the execution of the task, or if any other task interruption is detected. If an interruption is detected, the electronic device (1605) may interrupt the execution of the task and transmit the task execution request to the external device (1610) along with necessary information so that the task can be resumed on the external device (1610). Referring to FIG. 1, the operation of the electronic device (1605) transmitting the task execution request to the external device (1610) may be performed by the communication circuit (140).

[0244] In operation 1625, the external device (1610) may receive a task performance request along with necessary information from the electronic device (1605). The external device (1610) may request task performance by inputting a prompt to the generative AI model (217) using the received information.

[0245] In operation 1630, the external device (1610) may complete the task execution to generate result data and transmit the generated result data to the electronic device (1605). In one example, the external device (1610) may generate the result data using a generative AI model (115) stored in the memory of the external device (1610). In one example, the external device (1610) may generate the result data using a generative AI model (115) stored in an external cloud server and / or data center. The generated result data may be transmitted to the electronic device (1605) via wired or wireless communication.

[0246] In operation 1635, the electronic device (1605) may receive result data. Referring to FIG. 1, for example, the result data may be received via the communication circuit (140).

[0247] In operation 1640, the electronic device (1605) may complete the task performance. For example, the electronic device (1605) may complete the task performance using the received result data. In one example, the result data received by the electronic device (1605) in operation 1635 may be result data for the entire task request input acquired by the electronic device (1605) in operation 1615. In one example, the result data received by the electronic device (1605) in operation 1635 may be result data for a portion of the task request input acquired by the electronic device (1605) in operation 1615.

[0248] FIG. 17a illustrates a UI for performing a task, stopping a task, and resuming a task according to one embodiment.

[0249] Referring to FIGS. 1, 7, and 17A, according to one embodiment, the electronic device (10) may generate and display a task execution progress UI (1705). For example, the task execution progress UI may include a portion for displaying an estimated time for completion of the entire task execution and a portion for displaying an elapsed time since the task execution. Although the task execution progress UI (1705) is illustrated in FIG. 17A as having a configuration for displaying time, the task execution progress UI (1705) is not limited to the configuration illustrated in FIG. 17A. For example, the task execution progress UI (1705) may be displayed as a percentage (%) rather than time. For example, the task execution progress UI (1705) may be configured in a bar shape, a circle shape, and / or other image shape that may show the progress of task execution.

[0250] Referring to FIG. 17A, according to one embodiment, the electronic device (10) may generate and display a task execution stop notification (1710). Referring to FIG. 7, the electronic device (10) may display the task execution stop notification (1710) when a hardware or software abnormality is identified, or when other task interruptions are detected. The task execution stop notification (1710) may be displayed from the time the electronic device (10) detects a task interruption until the task execution is resumed by a task resume request of the electronic device (10).

[0251] Referring to FIG. 17a, according to one embodiment, when result data generation is completed, the electronic device (10) may generate and display a result data generation completion notification (1715) and display result data (1720).

[0252] In one example, the electronic device (10) may display a result data generation completion notification (1715). For example, the result data generation completion notification (1715) may include a notification notifying that result data generation has been completed in response to a task resumption request after a task interruption. For example, the result data generation completion notification (1715) may be a notification notifying that result data generation has been completed in response to an initial task execution request without the task being interrupted. Although the result data generation completion notification (1715) is illustrated as a notification composed of text in FIG. 17A, the result data generation completion notification (1715) is not limited to the configuration illustrated in FIG. 17A. For example, the result data generation completion notification (1715) may be composed of a shape or symbol in addition to text, or may be composed in a form in which at least one or more of text, a shape, or a symbol is combined.

[0253] The result data (1720) may include the final form of the result that the user intended to obtain through the input. In one example, the result data (1720) may be displayed after the result data generation completion notification (1715) is displayed, or may be displayed simultaneously with the result data generation completion notification (1715). For example, the result data (1720) may be result data generated in response to a task resumption request after a task has been interrupted. For example, the result data (1720) may be result data generated in response to an initial task execution request without the task being interrupted. In one example, the result data (1720) may be a result in the form of text, images, videos, and / or audio.

[0254] Figure 17b illustrates a user interface (UI) for progress by sub-prompt according to one embodiment.

[0255] Referring to FIGS. 1, 2, and 17B, according to one embodiment, the electronic device (10) may generate and display a sub-prompt-specific progress UI (1725). In one example, the sub-prompt-specific progress UI (1725) may be displayed together with the task execution progress UI (1705). In one example, the sub-prompt-specific progress UI (1725) may be displayed separately from the task execution progress UI (1705). Referring to FIG. 2, the sub-prompt-specific progress may be calculated through the generation completion prediction module (213). For example, the sub-prompt-specific progress may be a value obtained by dividing the elapsed time after starting the generation task for the sub-prompt by the generation completion time of each sub-prompt-specific result data predicted by the generation completion prediction module (213). In one example, the sub-prompt-specific progress UI (1725) may be displayed as a percentage (%) and / or a remaining time.

[0256] Referring to FIG. 17b, the sub-prompt-specific progress UI (1725) is illustrated as a configuration in which multiple shapes are colored in different colors, but the configuration of the sub-prompt-specific progress UI is not limited thereto. For example, the sub-prompt-specific progress UI (1725) may be configured in the form of a bar, a circle, and / or other image shapes that can show the progress of each sub-prompt. For example, the sub-prompt-specific progress UI (1725) may be configured in a form that combines text, shapes, symbols, static images, and / or dynamic images. For example, the sub-prompt-specific progress UI (1725) may be configured in a form that combines multiple contents that can show the progress of each sub-prompt.

[0257] FIG. 18 is a block diagram of an exemplary electronic device (1800) capable of performing the operations described in this document.

[0258] Referring to FIG. 18, the electronic device (1800) may be one of various forms of electronic devices, such as a notebook (1890), smartphones (1891) having various form factors (e.g., a bar-type smartphone (1891-1), a foldable-type smartphone (1891-2), or a sliderable (or rollable) type smartphone (1891-3)), a tablet (1892), a cellular phone (not shown), and other similar computing devices (not shown). The components, their relationships, and their functions illustrated in FIG. 18 are exemplary only and do not limit the implementations described or claimed in this document. The electronic device (1800) may be referred to as a mobile device, a user device, a multi-function device, a portable device, or a server.

[0259] The electronic device (1800) may include components including at least one processor (1810) (hereinafter referred to as processor (1810)), at least one memory (1820) (hereinafter referred to as memory (1820)), at least one display (1840) (hereinafter referred to as display (1840)), at least one image sensor (1850) (hereinafter referred to as image sensor (1850)), at least one communication circuit (1860) (hereinafter referred to as communication circuit (1860)), and / or at least one sensor (1870) (hereinafter referred to as sensor (1870)). The above components are merely exemplary. For example, the electronic device (1800) may include other components (e.g., power management integrated circuitry (PMIC), audio processing circuitry, an antenna, a rechargeable battery, or an input / output interface). For example, some components may be omitted from the electronic device (1800). For example, several components can be combined into one component.

[0260] The processor (1810) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing. The processor (1810) may include at least one electrical circuit and may individually or collectively perform distributed processing of instructions (or programs, data, etc.) stored in the memory (1820). The processor (1810) may include a processor assembly including one or more processing circuits. The processor (1810) may include any processing circuit operative to control the performance and operations of one or more components of the electronic device (1800) (e.g., the memory (1820), the display (1840), the image sensor (1850), the communication circuit (1860), and / or the sensor (1870)). For example, the processor (1810) (e.g., an application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). For example, the processor (1810) may be implemented as multiple cores (or at least one core circuit), multiple chips, or multiple chipsets. For example, the processor (1810) may include one or more processing circuits. For example, the processor (1810) may include one or more processing circuits configured to individually and / or collectively perform various functions of the present disclosure. As a non-limiting example, at least a portion of the processor (1810) may be included in a first chip of the electronic device (1800), and at least another portion of the processor (1810) may be included in a second chip of the electronic device (1800) that is different from the first chip of the electronic device (1800).

[0261] For example, the processor (1810) may include a central processing unit (CPU) (1811), a graphics processing unit (GPU) (1812), a neural processing unit (NPU) (1813), an image signal processor (ISP) (1814), a display controller (1815), a memory controller (1816), a storage controller (1817), a communication processor (CP) (1818), and / or a sensor interface (1819). These components of the processor (1810) are merely exemplary. For example, the processor (1810) may further include other components. For example, some components of the processor (1810) may be omitted from the processor (1810). For example, some components of the processor (1810) may be included as separate components of the electronic device (1800) outside the processor (1810). For example, some components of the processor (1810) (e.g., memory controller (1816)) may be included within other components (e.g., at least a portion of memory (1820), an interface (e.g., available for connection to at least one component of the electronic device (100)), a display (1840) and / or an image sensor (1850)).

[0262] The processor (1810) may cause other components of the electronic device (1800) to perform various operations by executing instructions stored in the memory (1820). The CPU (1811) (or central processing circuit) may be configured to control components of the processor (1810) based on the execution of instructions stored in the memory (1820) (e.g., volatile memory (1821) and / or non-volatile memory (1822)). The GPU (1812) (or graphics processing circuit) may be configured to perform parallel operations (e.g., rendering). The NPU (1813) (or neural processing circuit, or artificial intelligence (AI) chip) may be configured to perform operations for an artificial intelligence model (e.g., convolution computation). The ISP (1814) (or image signal processing circuit) may be configured to process a raw image acquired through the image sensor (1850) into a format suitable for a component within the electronic device (1800) or a component of the processor (1810). The display controller (1815) (or display control circuit, or display processing unit (DPU)) may be configured to process an image acquired from the CPU (1811), the GPU (1812), the ISP (1814), or the memory (1820) (e.g., the volatile memory (1821)) into a format suitable for the display (1840). The memory controller (1816) (or memory control circuit) may be configured to control reading data from the volatile memory (1821) and writing data to the volatile memory (1821). The storage controller (1817) (or storage control circuit) may be configured to control reading data from and writing data to the nonvolatile memory (1822).The CP (1818) (communication processing circuit) may be configured to process data obtained from a component of the processor (1810) into a format suitable for transmission to another electronic device via the communication circuit (1860), or to process data obtained from another electronic device via the communication circuit (1860) into a format suitable for processing by the component of the processor (1810). For example, the communication circuit (1860) may include one or more communication circuits. The sensor interface (1819) (or sensing data processing circuit, sensor hub) may be configured to process data on the state of the electronic device (1800) and / or the state of the surroundings of the electronic device (1800), obtained via the sensor (1870), into a format suitable for the component of the processor (1810).

[0263] The memory (1820) may include one or more storage media (or one or more storage devices). For example, the memory (1820) may include a memory assembly including one or more storage media. For example, the one or more storage media may include permanent memory (e.g., non-volatile memory (1822)) such as a hard drive, flash memory, read-only memory (ROM), semi-permanent memory (e.g., volatile memory (1821)) such as random access memory (RAM), any other suitable type of storage (or storage assembly), or any combination thereof. The memory (1820) may include cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (1800). As a non-limiting example, the cache memory may be included within the processor (1810). The memory (1820) may be fixedly embedded within the electronic device (1800) or incorporated into one or more suitable types of components (e.g., a subscriber identity module (SIM) card and / or a secure digital (SD) card) that may be repeatedly inserted into and removed from the electronic device (1800).

[0264] For example, the memory (1820) may store one or more software applications, such as an operating system (or system) software application, a firmware software application, a driver software application, a plug-in (e.g., add-in, add-on, and / or applet) software application, and / or any other suitable software applications. For example, the one or more software applications may include instructions executable by the processor (1810). For example, the memory (1820) may store instructions callable by an application programming interface (API). For example, the memory (1820) may store instructions within a library.

Claims

In electronic devices, memory; and comprising at least one processor communicatively connected to said memory, The memory, when executed individually or collectively by the at least one processor, causes the electronic device to: Generate multiple sub-prompts from input requesting a task, To perform the above task, the plurality of sub-prompts are input to one or more artificial intelligence models, Generating first result data by processing at least one first sub-prompt among the plurality of sub-prompts by the one or more artificial intelligence models, If an interruption of the task is detected during the execution of the task, store the first result data associated with at least one first sub-prompt among the plurality of sub-prompts, An electronic device storing instructions for performing the task using the first result data and the remaining sub-prompts excluding the at least one first sub-prompt among the plurality of sub-prompts, based on a resume request for the task after the interruption of the task is detected. In the first paragraph, The above instructions, when individually or in combination executed by the at least one processor, cause the electronic device to: Identify multiple keywords and intents from the above input, An electronic device that generates the plurality of sub-prompts by merging each of the plurality of keywords with the intent. In the second paragraph, The above instructions, when individually or in combination executed by the at least one processor, cause the electronic device to: The task is performed by generating multiple result data using each of the multiple sub-prompts and merging the multiple result data, or An electronic device that performs the task by processing a previous result of a previous sub-prompt among the plurality of sub-prompts using a following sub-prompt. In the first paragraph, The above instructions, when individually or in combination executed by the at least one processor, cause the electronic device to: Generate additional prompts based on second result data associated with a second sub-prompt, including sub-prompts whose progress is greater than or equal to a specified value when the interruption is detected, among the remaining sub-prompts excluding the first sub-prompt, An electronic device that performs the task using the additional prompt instead of the second sub-prompt based on a request to resume the task. In the first paragraph, Including more displays, When the above instructions are individually or in combination executed by the at least one processor, the electronic device: An electronic device that displays the remaining result data excluding the first result data on the display based on a request for resumption of the above task. In the first paragraph, When the above instructions are individually or in combination executed by the at least one processor, the electronic device: If at least one function abnormality of the hardware or software is identified, or An electronic device that detects interruption of the task in at least one case where a specified amount of time has elapsed since the start time of the execution of the task. In the first paragraph, The above instructions, when individually or in combination executed by the at least one processor, cause the electronic device to: Based on the computational load, identify the processing prediction time for each of the plurality of sub-prompts, Based on the completion of the above task, the processing time of each of the plurality of sub-prompts is identified, An electronic device that performs reinforcement learning on one or more artificial intelligence models based on a comparison of the above processing prediction time and the above processing time. In the first paragraph, An electronic device, wherein each of the plurality of sub-prompts corresponds to each of the plurality of elements included in the image, when the above task is to generate an image. In the first paragraph, An electronic device, wherein each of the plurality of sub-prompts corresponds to a function having one function, when the above task generates code. In the first paragraph, Including more displays, The above instructions, when individually or in combination executed by the at least one processor, cause the electronic device to: When performing the above task, display a UI showing the progress of the above task on the display, If the above task is interrupted, a notification that the task has been stopped is displayed on the display, An electronic device that, when the task is completed based on a request for resumption of the task, displays on the display at least one of a notification indicating that the result data generation is completed or the result data. A method for generating result data for a task request input of an electronic device, An action that generates multiple sub-prompts from input requesting a task; An operation of inputting the plurality of sub-prompts into one or more artificial intelligence models to perform the above task; An operation of generating first result data by processing at least one first sub-prompt among the plurality of sub-prompts by the one or more artificial intelligence models; An operation of storing the first result data associated with at least one first sub-prompt among the plurality of sub-prompts when an interruption of the task is detected during the execution of the task; and A method comprising, after the interruption of the task is detected, performing the task based on a request for resuming the task using the first result data and the remaining sub-prompts excluding the at least one first sub-prompt among the plurality of sub-prompts. In Article 11, The action of generating the above multiple sub-prompts is: An operation of identifying multiple keywords and intents from the above input; and A method comprising an action of generating the plurality of sub-prompts by merging each of the plurality of keywords with the intent. In paragraph 12, The action to perform the above task is: An operation of performing the task by generating a plurality of result data using each of the plurality of sub-prompts and merging the plurality of result data; or A method comprising at least one operation of performing the task by processing a previous result of a previous sub-prompt among the plurality of sub-prompts using a subsequent sub-prompt. In Article 11, The action of performing the above task using the remaining sub-prompts is as follows: An operation of generating an additional prompt based on second result data associated with a second sub-prompt, which includes sub-prompts whose progress is greater than or equal to a specified value when the interruption is detected among the remaining sub-prompts excluding the first sub-prompt; and A method comprising performing a task using the additional prompt instead of the second sub-prompt, based on a request to resume the task. In Article 11, A method further comprising an action of displaying the remaining result data excluding the first result data on a display based on a request for resumption of the above task.

Images