Electronic device and method by which electronic device generates data by using external ai model and on-device ai model

Abstract

An electronic device according to an embodiment comprises a communication circuit, a display, at least one processor, and a memory storing instructions, wherein the instructions may be configured to, when executed by the at least one processor, cause the electronic device to: receive a user input related to a first object, for modifying at least a portion of the first object; generate first information for modifying at least a portion of the first object by means of an external AI model, the external AI model being trained to generate at least one image by using an input including text, speech, or an image input; transmit the first information to the external AI model through the communication circuit; identify second information stored in the memory, on the basis of the user input and at least a portion of the first information; receive first data from the external AI model through the communication circuit; generate second data by modifying at least a portion of the first object on the basis of the second information and the first data, by using an on-device AI model, the on-device AI model being trained to generate at least one image by using an input including text, speech, or an image input; and provide a modified first object corresponding to the second data through the display. Other embodiments may be included.

Description

A method for generating data using external AI models and on-device AI models in electronic devices and electronic devices.

[0001] The present disclosure relates to an electronic device and a method for generating data using an external AI model and an on-device AI model in the electronic device.

[0002] On-device AI models can perform tasks such as summarizing, producing, and / or inferring photos, calls, and / or notes within electronic devices such as smartphones and / or watches, and this can reduce user fatigue and expand the range of utilization of electronic devices by supporting the operation of applications that operate offline with their own AI models without connecting to a server.

[0003] A user of an electronic device can receive assistance in summarizing and / or organizing content, etc., for offline application operations through an on-device AI model, and the on-device AI model can be configured in the form of a voice assistant, a conversational AI model, and / or a generative AI. The electronic device can perform operations according to a user request within a specific application through the on-device AI model, and can distribute and utilize hardware resources as effectively as possible by performing multi-modal operations without causing excessive load on the system and not being limited to a single type of data.

[0004] By generating and providing user-requested objects based on data received from external AI models and data generated by the on-device AI model, external AI models that had previously been excluded due to hardware performance limitations can now utilize some of their functions. Furthermore, the on-device AI model can conduct user propensity and characteristic analysis with greater precision and accuracy. Furthermore, users of the electronic device can maintain the reassurance that the on-device AI model is only handling their information, and by increasing their understanding of themselves, they can broaden the possibilities for AI utilization.

[0005] According to an embodiment, an electronic device may include a communication circuit, a display, at least one processor, and a memory storing instructions. According to an embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to receive a user input related to a first object for modifying at least a portion of the first object. According to an embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to generate first information for modifying at least a portion of the first object by an external AI model, and the external AI model may be trained to generate at least one image using an input including text, voice, or image input. According to an embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to transmit the first information to the external AI model through the communication circuit. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to verify second information stored in the memory based on the user input and at least a portion of the first information. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to receive first data from the external AI model through the communication circuit. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to generate second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, and the on-device AI model may be trained to generate at least one image using an input including a text, voice, or image input.The instructions according to one embodiment, when executed by the at least one processor, may be configured to cause the electronic device to provide a modified first object corresponding to the second data through the display.

[0006] According to an embodiment, an electronic device may include a communication circuit, a display, at least one processor, and a memory storing instructions. According to an embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to receive a user input related to a first object for modifying at least a portion of the first object. According to an embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to generate first information for modifying at least a portion of the first object. According to an embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to transmit the first information to an external device via the communication circuit. According to an embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to verify second information stored in the memory based on the user input and at least a portion of the first information. The instructions according to one embodiment may be configured to cause the electronic device to receive first data from the external device through the communication circuit when executed by the at least one processor. The instructions according to one embodiment may be configured to cause the electronic device to generate second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, and the on-device AI model may be trained to generate at least one image using an input including text, voice, or image input.The instructions according to one embodiment, when executed by the at least one processor, may be configured to cause the electronic device to provide a modified first object corresponding to the second data through the display.

[0007] In one embodiment, a method for generating data using an external AI model and an on-device AI model in an electronic device may include receiving a user input related to a first object for modifying at least a portion of the first object. In one embodiment, the method may include generating first information for modifying at least a portion of the first object by the external AI model, and training the external AI model to generate at least one image using an input including text, voice, or image input. In one embodiment, the method may include transmitting the first information to the external AI model via a communication circuit of the electronic device. In one embodiment, the method may include verifying second information stored in the memory based on the user input and at least a portion of the first information. In one embodiment, the method may include receiving first data from the external AI model via the communication circuit. The method according to one embodiment may include an operation of generating second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, and the on-device AI model is trained to generate at least one image using an input including text, voice, or image input. The method according to one embodiment may include an operation of providing a modified first object corresponding to the second data through a display of the electronic device.

[0008] In one embodiment, a method for generating data using an external AI model and an on-device AI model in an electronic device may include receiving a user input related to a first object for modifying at least a portion of the first object. In one embodiment, the method may include generating first information for modifying at least a portion of the first object. In one embodiment, the method may include transmitting the first information to an external device via a communication circuit of the electronic device. In one embodiment, the method may include identifying second information stored in the memory based on the user input and at least a portion of the first information. In one embodiment, the method may include receiving first data from the external device via the communication circuit. In one embodiment, the method may include generating second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, and the on-device AI model may include training to generate at least one image using an input including text, voice, or image input. The method according to one embodiment may include providing a modified first object corresponding to the second data through a display of the electronic device.

[0009] In one embodiment, a non-volatile storage medium storing commands is provided, wherein the commands, when executed by an electronic device, are configured to cause the electronic device to perform at least one operation, wherein the at least one operation may include receiving a user input related to a first object for modifying at least a portion of the first object. In one embodiment, the at least one operation may include generating first information for modifying at least a portion of the first object by an external AI model, wherein the external AI model is trained to generate at least one image using an input including text, voice, or image input. In one embodiment, the at least one operation may include transmitting the first information to the external AI model via a communication circuit of the electronic device. In one embodiment, the at least one operation may include confirming second information stored in the memory based on the user input and at least a portion of the first information. In one embodiment, the at least one operation may include receiving first data from the external AI model via the communication circuit. In one embodiment, the at least one operation may include generating second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, and the on-device AI model may be trained to generate at least one image using an input including text, voice, or image input. In one embodiment, the at least one operation may include providing a modified first object corresponding to the second data through a display of the electronic device.

[0010] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0011] FIG. 2A is a block diagram of an electronic device according to one embodiment.

[0012] FIG. 2b is a diagram for explaining the operation between an on-device AI model and an external AI model according to one embodiment.

[0013] FIGS. 3a, 3b, 3c, 3d, 3e, and 3f are drawings for explaining an operation of generating data using an external AI model and an on-device AI model in an electronic device according to one embodiment.

[0014] FIGS. 4a, 4b, 4c, 4d, and 4e are drawings for explaining an operation of generating data using an external AI model and an on-device AI model in an electronic device according to one embodiment.

[0015] FIG. 5 is a diagram illustrating an operation for determining whether to use an external AI model in an electronic device according to one embodiment.

[0016] FIGS. 6A and 6B are drawings for explaining an operation for visually distinguishing the operation of an on-device AI model and an external AI model in an electronic device according to one embodiment.

[0017] FIGS. 7A and 7B are drawings for explaining an operation for visually distinguishing the operation of an on-device AI model and an external AI model in an electronic device according to one embodiment.

[0018] FIG. 8 is a diagram for explaining an operation for visually distinguishing the operation of an on-device AI model and an external AI model in an electronic device according to one embodiment.

[0019] FIG. 9 is a diagram illustrating an operation for visually distinguishing the operation of an on-device AI model and an external AI model in an electronic device according to one embodiment.

[0020] FIG. 10 is a diagram for explaining an operation of generating data using an external AI model and an on-device AI model according to one embodiment.

[0021] FIGS. 11A and 11B are diagrams for explaining an operation of generating data using an external AI model and an on-device AI model according to one embodiment.

[0022] FIGS. 12a, 12b, 12c, 12d, and 12e are drawings for explaining operations for visually distinguishing operations of an on-device AI model and an external AI model in an electronic device according to one embodiment.

[0023] FIG. 13 is a flowchart illustrating an operation of generating data using an external AI model and an on-device AI model in an electronic device according to one embodiment.

[0024] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to an embodiment. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). According to an embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0025] The processor (120) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (120) may store a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0026] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0027] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0028] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0029] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0030] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0031] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0032] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0033] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0034] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0035] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0036] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0037] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0038] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0039] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0040] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0041] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0042] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the selected at least one antenna. According to some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0043] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0044] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0045] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0046] FIG. 2a is a block diagram of an electronic device according to an embodiment, and FIG. 2b is a diagram for explaining the operation between an on-device AI model and an external AI model according to an embodiment.

[0047] Referring to the above FIGS. 2A and 2B, the electronic device (201) may include a processor (220), a memory (230), a display (260), and data acquisition devices (250).

[0048] According to one embodiment, the processor (220) may perform overall control operations of the electronic device (201). The processor (220) according to one embodiment may execute software (e.g., the program (140) of FIG. 1) to control at least one other component (e.g., a hardware or software component) of the electronic device (201) connected to the processor (220), and may perform data processing or calculations based on instructions. Instructions according to one embodiment may include instructions configured in a machine language that can be processed by the electronic device (201) or the processor (220). For example, instructions may include instructions corresponding to operation instructions used in a program.

[0049] According to one embodiment, the processor (220) may determine whether to use an external AI (artificial intelligence) model when receiving a user request for processing a first object.

[0050] According to one embodiment, the processor (220) can receive a user's request through a microphone (253) of a data acquisition device (250).

[0051] According to one embodiment, the processor (220) can receive a user's request based on text entered through an input window displayed in a certain area of ​​the display (260).

[0052] A first object according to one embodiment represents a target to be processed according to a user's request (input), and may include various contents such as images, videos, messages, voices, and / or files.

[0053] According to one embodiment, the first object may be selected by the user or by an on-device artificial intelligence (AI) model (231) that analyzes the user's request.

[0054] A user request (input) according to one embodiment may include an action for processing a first object.

[0055] For example, when the processor (220) receives a user's request (input) such as "Add a friend O0 next to me who is wearing trendy clothes these days" while an image including the user is selected, the processor (220) can use the on-device AI model (231) to identify the image as a first object and confirm that "Add a friend O0 next to me who is wearing trendy clothes these days" is the user's request.

[0056] For example, when the processor (220) receives a user request such as “Write a travel report with the photos taken when I went to the Louvre Museum,” it can use the on-device AI model (231) to identify the photos taken when I went to the Louvre Museum stored in the memory (230) as the first object and confirm the user request of “Write a travel report with the photos taken when I went to the Louvre Museum.”

[0057] According to one embodiment, the processor (220), upon receiving a user request, may determine whether to use an external AI (artificial intelligence) model (291) using an on-device AI model (231). According to one embodiment, the external AI model (291) may be trained (learned) to generate at least one image using input including text, voice, or image input.

[0058] According to one embodiment, the processor (220), upon receiving a user request, may determine whether to use at least one of the on-device AI model (231) or the external AI (artificial intelligence) model (291) based on a specified profile or policy, or based on the user's settings or environmental information within the electronic device.

[0059] An on-device AI model (231) according to one embodiment can be trained (learned) to generate at least one image using input including text, voice, or image input.

[0060] According to one embodiment, the on-device AI model (231) can analyze a user's request and distinguish between actions that the on-device AI model (231) can process and actions that the external AI model (291) can process.

[0061] According to one embodiment, the on-device AI model (231) can analyze a user's request and identify actions related to the user's information as actions that the on-device AI model (231) can process, and distinguish actions that are not related to the user's information as actions that can be processed using an external AI model (291).

[0062] According to one embodiment, the on-device AI model (231) analyzes the user's request and, if it is confirmed that the input value (input information) for processing the first object based on the user's request is lower than the reference input value (reference information), it can decide to use the external AI model (291).

[0063] In one embodiment, the on-device AI model (231) may determine that the input value (input information) for processing the first object is below the reference input value (reference information) when the user's feedback on the data generated by processing the object based on the user's request is below the reference satisfaction level (e.g., dissatisfaction).

[0064] According to one embodiment, when the on-device AI model (231) determines that a keyword (e.g., “popular these days”) that cannot be processed is included in the user’s request, the input value (input information) for processing the first object can be confirmed to be less than or equal to a reference input value (reference information).

[0065] According to one embodiment, the on-device AI model (231) can determine that the input value (input information) for processing the first object is less than or equal to the reference input value (reference information) when the number of objects (e.g., images) required to generate data corresponding to the result is less than or equal to a reference number based on the user's tendencies or feedback from previous users.

[0066] According to one embodiment, when the on-device AI model (231) decides to use an external AI model (291), it displays a message asking the user whether to use the external AI model, and when it receives a response corresponding to acceptance from the user, it can decide to use the external AI.

[0067] For example, when the on-device AI model (231) receives a user request such as, "Add a friend O0 next to me who is wearing trendy clothes these days," the on-device AI model (231) may analyze the user request and determine that "friend 00" is the user's information and that the on-device AI model (231) can process the action, or if the input value for analyzing "trendy clothes these days" is lower than a standard input value, the on-device AI model (231) may determine that "trendy clothes these days" is an action that can be processed using an external AI model (291) and decide to use the external AI model (291). For example, when the on-device AI model (231) determines that there is not enough information to process the first object based on the user's request, the on-device AI model (231) may determine that the input value is lower than a standard input value. According to one embodiment, if the on-device AI model (231) determines that a keyword that cannot be processed (e.g., "currently popular") is included in the user's request, the on-device AI model (231) may confirm that an input value (input information) for processing the first object is less than or equal to a reference input value (reference information). If the on-device AI model (231) determines that a keyword that requires processing by an external AI model (291) is included in the user's request (e.g., "currently popular clothing"), the on-device AI model (231) may provide a UI for selecting an external AI model on the display (260) based on information about the type of at least one external AI model that can process the keyword (e.g., external AI models for various product brands, external AI models provided for each product style, and / or external AI models based on user preferences, etc.). According to one embodiment, the on-device AI model (231) may, upon confirmation of selection of a first external AI model among at least one external AI model, display information processed by the first external AI model using graphic elements (e.g., icons and / or logos) representing the first external AI model.

[0068] According to one embodiment, the processor (220) may, when determining to use an external AI model (291) using an on-device AI model (231), generate first information for processing the external AI model (291) based on a request from a first object and a user, and transmit the first information to the external AI model (291).

[0069] According to one embodiment, the on-device AI model (231) may, after generating first information, ask the user whether the first information includes the user's information, and if the user confirms that the first information does not include the user's information, transmit the first information to the external AI model (291).

[0070] According to one embodiment, the on-device AI model (231) can check whether the first object contains user information, and if the first object contains user information, can create a second object by editing (e.g., deleting or modifying) the first object so that the external AI model (291) cannot recognize the user information.

[0071] For example, if the first object is an image containing a user, a second object can be created with the area of ​​the image displaying the user deleted or blurred.

[0072] According to one embodiment, the on-device AI model (231) can generate a prompt (command) to have some of the user's requests processed by the external AI model (291).

[0073] According to one embodiment, the on-device AI model (231) can generate a prompt (command) by considering the usage history of the previous external AI model (291) or the user's feedback.

[0074] According to one embodiment, the on-device AI model (231) can generate a prompt (command) by referencing additional information (data) unrelated to the user's information to convey more specific information to an external AI model (291).

[0075] An on-device AI model (231) according to one embodiment can generate a prompt (command) that does not include user information.

[0076] For example, in response to a user request such as “Add a friend O0 next to me who is wearing trendy clothes these days,” a prompt (command) of “trendy clothes these days” can be generated for an action that can be processed by an external AI model (291), or a prompt (command) of “trendy fashion for slim women these days” can be generated by considering the user’s feedback, etc.

[0077] According to one embodiment, an on-device AI model (231) can generate first information including a prompt and a second object that does not include user information, and transmit the first information to an external AI model (291) through a communication circuit (257).

[0078] According to one embodiment, when the on-device AI model (231) confirms that the first object does not contain user information, it can generate first information including the first object and a prompt and transmit it to the external AI model (291) through the communication circuit (257).

[0079] According to one embodiment, the on-device AI model (231) can determine whether to use the external AI model (291) by using the following Table 1, which is created based on the usage history of the previous external AI model (291) or user feedback.

[0080] Whether the first object's personal information is included? Use of external AI model is required. Processing XX On-device AI model XO External AI model OX On-device AI model OO On-device AI model → External AI model

[0081] According to an embodiment, the on-device AI model (231) can generate and output second data by processing the first object based on a user's request using the on-device AI model (231), as shown in , when the first object does not contain user information and an external AI model (291) is not used. According to an embodiment, the on-device AI model (231) can generate and output second data by processing the first object based on a user's request using the external AI model (291), as shown in , when the first object does not contain user information and an external AI model (291) is used. According to an embodiment, the on-device AI model (231) can generate and output second data by processing the first object based on a user's request using the on-device AI model (231), as shown in , when the first object contains user information and an external AI model (291) is not used.

[0082] According to an embodiment, the on-device AI model (231), as shown in , when the first object includes the user's information and an external AI model (291) is used, the first object including the user's information is edited (e.g., deleted or modified) to transmit the second object generated by editing (e.g., deleting or modifying) the first object and the first information including the prompt to the external AI model (291), and when the first data generated based on the first information is received from the external AI model (291), the on-device AI model (231) can check the second information generated based on at least a portion of the user's request (user's input) and the first information or stored in the memory (230), and can generate and output second data by processing the first object based on the user's request using the second information and the first data received from the external AI model (291).

[0083] According to one embodiment, when transmitting first information including a second object edited (preprocessed) so that user information is not recognized in a first object to an external AI model (291), the on-device AI model (231) may determine not to use the external AI model (291) when a difference between the second data based on the first data received from the external AI model (291) and the second data based on the first data received from the external AI model (291) when transmitting the first information including the first object including user information to the external AI model (291) is greater than a standard.

[0084] According to one embodiment, the on-device AI model (231) may decide not to use the external AI model (291) if there is no network connection.

[0085] According to one embodiment, a processor (220) may process at least some of the operations processed by the on-device AI model by a processor (220) other than the on-device AI model.

[0086] According to one embodiment, the processor (220) may process all or at least some of the operations processed by the on-device AI model, excluding the operations of generating content or data. According to one embodiment, when the processor (220) confirms the decision to use the external AI model (291), the processor (220) may visually distinguish and display information related to the operation of the on-device AI model (231) and information related to the operation of the external AI model (291).

[0087] According to one embodiment, when the processor (220) confirms a decision to use an external AI model (291), the processor (220) divides the display area of ​​the display (260) into a plurality of display areas, displays information related to the operation of the external AI model (291) in a first display area among the plurality of display areas, and displays information related to the operation of the on-device AI model (231) in a second display area among the plurality of display areas, thereby informing the user that the external AI model (291) is currently operating.

[0088] According to one embodiment, when the processor (220) determines a decision to use an external AI model (291) when the electronic device is a bar-type electronic device, the processor (220) may divide the display (260) of the electronic device into two display areas, display information related to the operation of the external AI model (291) in the first display area of ​​the two display areas, and display information related to the operation of the on-device AI model (231) in the second display area of ​​the two display areas.

[0089] According to one embodiment, when the processor (220) determines a decision to use an external AI model (291) in a folded state of the electronic device when the electronic device is a foldable type electronic device, the processor (220) may divide the display (260) of the electronic device into two display areas, display information related to the operation of the external AI model (291) in the first display area of ​​the two display areas, and display information related to the operation of the on-device AI model (231) in the second display area of ​​the two display areas. According to one embodiment, when the processor (220) confirms the decision to use the external AI model (291) in the unfolded state of the electronic device, the processor (220) divides the display area of ​​the display (260) into two or more display areas, displays information related to the operation of the external AI model (291) in a first display area among the plurality of display areas, displays information related to the operation of the on-device AI model (231) in a second display area among the plurality of display areas, and displays various information, for example, a first object including user information, a second object from which user information transmitted to the external AI model (291) is deleted, temporary data corresponding to an intermediate result generated using data partially received from the external AI model (291) prior to displaying the second data, in the remaining display areas.

[0090] According to one embodiment, when the processor (220) determines a decision to use an external AI model (291) in a first state in which the first display area of ​​the flexible display (rollable display and / or foldable display) (260) of the electronic device is exposed and the second display area is housed in a second housing configured to be movable with respect to the first housing when the electronic device is a flexible (rollable and / or foldable) type electronic device, the processor (220) may divide the display (260) of the electronic device into two display areas, display information related to the operation of the external AI model (291) in the first display area of ​​the two display areas, and display information related to the operation of the on-device AI model (231) in the second display area of ​​the two display areas. According to one embodiment, when the processor (220) confirms the decision to use the external AI model (291) in a second state in which the first display area of ​​the flexible display (rollable display and / or foldable display) (260) of the electronic device is exposed and the second display area of ​​the flexible display (260) of the electronic device is exposed to the outside of the second housing, the processor (220) divides the display area of ​​the display (260) into two or more display areas, displays information related to the operation of the external AI model (291) in the first display area among the plurality of display areas, displays information related to the operation of the on-device AI model (231) in the second display area among the plurality of display areas, and displays various information, for example, a first object including user information, a second object from which user information transmitted to the external AI model (291) is deleted, temporary data corresponding to an intermediate result generated using data partially received from the external AI model (291) prior to displaying the second data, on the remaining display areas among the plurality of display areas.According to one embodiment, when the electronic device determines to use the external AI model (291) in the first state, the processor (220) changes to the second state automatically or based on a user's selection, and in the second state, divides the display area of ​​the display (260) of the electronic device into two or more display areas, displays information related to the operation of the external AI model (291) in the first display area among the plurality of display areas, displays information related to the operation of the on-device AI model (231) in the second display area among the plurality of display areas, and displays various information, for example, a first object including user information, a second object from which user information transmitted to the external AI model (291) is deleted, temporary data corresponding to an intermediate result generated using data partially received from the external AI model (291) prior to displaying the second data, etc., in the remaining display areas.

[0091] According to an embodiment, when the electronic device is a VST (video see-through) type electronic device, since the size of the display of the VST type electronic device can be diversified, the processor (220) can divide the display (260) of the electronic device into a plurality of display areas with the user's consent and display the operation of the external AI model (291), the operation of the on-device AI model (231), and / or various contents in each of the plurality of display areas upon confirming the decision to use the external AI model (291).

[0092] According to one embodiment, the processor (220) may confirm the decision to use an external AI model (291) and display an indicator or text indicating that the external AI model (291) is operating in a certain area of ​​the display (e.g., the top of the display, a navigation bar, and / or a tooltip, etc.).

[0093] According to one embodiment, the processor (220) may, when determining the use of an external AI model (291) using an on-device AI model (231), generate (verify) second information according to some of the user's requests based on information stored in the memory (230).

[0094] According to one embodiment, the on-device AI model (231) can generate (verify) second information based on information stored in the memory (230) to perform an action related to the user's information among the user's requests.

[0095] For example, in response to a user request such as "Add a friend O0 who is wearing trendy clothes next to me", the facial image of "friend 000" can be generated (obtained) as secondary information from among the images stored in the memory.

[0096] According to one embodiment, the on-device AI model (231) may generate a specific question or additional question to process the first object based on the user's request while waiting to receive the first data processed based on the first information from the external AI model (291), and may provide the specific question or additional question to the user through a message or voice, thereby providing that the on-device AI model and the external AI model operate in parallel.

[0097] According to one embodiment, while waiting to receive first data processed based on first information from an external AI model (291), the on-device AI model (231) may preferentially provide data corresponding to a result, which may have a somewhat low accuracy, of processing the first object based on a user's request.

[0098] In one embodiment, when the on-device AI model (231) receives a portion of the first data processed based on the first information from the external AI model (291), the on-device AI model (231) may preferentially provide the data generated using the portion of the first data received and the second information generated (confirmed) by the on-device AI model (231).

[0099] According to one embodiment, while waiting to receive first data processed based on first information from an external AI model (291), the on-device AI model (231) may execute an action of explaining the reason for selecting a specific external AI model (291) among a plurality of external AI models through a message or voice.

[0100] According to one embodiment, the on-device AI model (231) may, while waiting to receive first data processed based on first information from an external AI model (291), generate first information containing additional information based on a user's request and transmit the first information to the external AI model (291).

[0101] According to one embodiment, while waiting to receive first data processed based on first information from an external AI model (291), the on-device AI model (231) can generate first information including information corresponding to the user's request based on a user's request and transmit the first information to another external AI model.

[0102] According to one embodiment, when a processor (220) receives first data from an external AI model (291), it can generate and provide second data that processes a first object based on a user's request using second information generated (confirmed) by an on-device AI model (231) and the first data.

[0103] An on-device AI model (231) according to one embodiment can select at least some data based on user information from first data received from an external AI model (291), and learn at least some of the selected data and second information to generate second data.

[0104] According to one embodiment, the on-device AI model (231) can generate second data by performing personalized and regenerative tasks such as reproducing, managing, and summarizing first data received from an external AI model (291) based on user information.

[0105] In one embodiment, the on-device AI model (231) and the external AI model (291) operate individually, but the on-device AI model (231) selects at least some data based on the user's information from the first data generated by the external AI model (291), and the second data can be generated by including the second information generated (confirmed) by the on-device AI model (231) with at least some of the selected data.

[0106] An external AI model (291) according to one embodiment can generate first data through a process of summarizing, analyzing, and processing data acquired based on first information, and transmit the first data to an electronic device (201) based on a user's selection or response or based on specified conditions (e.g., automatic transmission).

[0107] An external AI model (291) according to one embodiment may include at least one of an AI model included in an external server or an on-device AI model of an external electronic device that is connected to the electronic device for communication.

[0108] According to one embodiment, the on-device AI model (231) can, while displaying first data received from an external AI model (291), check at least some data selected by a user among the first data, and learn at least some of the checked data and second information to generate second data.

[0109] According to one embodiment, the on-device AI model (231) may, while displaying first data received from an external AI model (291), confirm selection of at least some data among the first data based on a user's gesture (e.g., slide, drag, short touch, long touch, multi-touch, double touch, etc.), and may generate second data by including the selected at least some data in second information generated (confirmed) by the on-device AI model.

[0110] According to one embodiment, the on-device AI model (231) receives first data from an external AI model (291), and if there is no additional request from the user for the external AI model (291) after a certain period of time has elapsed and it determines that selection of at least some of the necessary data from the first data has been completed, the on-device AI model (231) may terminate the communication connection with the external AI model (291) and learn at least some of the data and second information to generate second data.

[0111] An on-device AI model (231) according to one embodiment may use first data received from an external AI model (291) as learning or input data, and generate second data that is regenerated or summarized based on the learning or input data and second information generated (confirmed) by the on-device AI model.

[0112] According to one embodiment, an external AI model (231) provides simple processing with low security intensity, such as data collection, storage, transmission, summary, organization, analysis, and regeneration, and an on-device AI model (231) can distribute roles to perform tasks such as processing data corresponding to user information.

[0113] According to one embodiment, the on-device AI model (231) may select at least some data based on the user's status (e.g., the country, place, season, or time where the user is currently located) among the first data received from the external AI model (291), and learn the selected at least some data and second information generated (confirmed) by the on-device AI model (291) to generate second data.

[0114] An on-device AI model (231) according to an embodiment can select at least some necessary data from the first data by analyzing the first data received from an external AI model (291) and connecting it with the user's information. An on-device AI model (231) according to an embodiment can generate second data by combining the user's information (e.g., schedule, daily routine, hobbies, tastes, etc.) with at least some data selected from the first data, and can increase the understanding of the user's characteristics by relearning the second data. An on-device AI model (231) according to an embodiment can select only some candidate data from among the candidate data according to the user's status and use them to generate the second data when confirming a plurality of candidate data based on the analysis of the first data received from the external AI model (291).

[0115] For example, when the on-device AI model (231) transmits first information requesting “generation of stylish clothes” to the external A model (291), and receives first data suggesting various clothes that are in fashion from the external AI model (291), the on-device AI model (231) can select clothes with a design preferred by the user based on images that include recent users in the first data, or generate second data by referring only to the patterns of the clothes suggested by the external AI model (291). The on-device AI model (231) can segment the user’s request on its own (e.g., clothes = pattern + type (skirt, pants, etc.)) and segment the first data received from the external AI model (291) accordingly to accept only some of the data. The on-device AI model (231) can generate second data by referring to some of the accepted data and the user’s information. The on-device AI model (291) can generate second data based on the metadata of the first data received from the external AI model (291) for the pattern of the clothing in response to the user's request for "creating stylish clothing" and the selection of the type of clothing based on the user's condition. For example, the on-device AI model (291) can select a type of clothing, such as a short-sleeved T-shirt or loose-fitting pants, if the user is traveling to a hot country, or can select a pattern of clothing with different preferences depending on the user's current location or condition.

[0116] An on-device AI model (231) according to one embodiment can immediately reflect user feedback while processing a user request and change its operation for processing the user request accordingly.

[0117] According to one embodiment, if the on-device AI model (231) fails to receive first data from the external AI model (291) after a certain period of time has elapsed, the on-device AI model may terminate the use of the external AI model or change the type of the external AI model based on the user's selection.

[0118] According to one embodiment, the processor (220) may, by using the on-device AI model (231), select at least some data required from first data received from an external AI model (291) or confirm at least some data selected by the user from the first data, terminate communication with an external server including the external AI model (291), and generate and provide the second data to the user.

[0119] According to one embodiment, the processor (220) may, when selecting at least some data required from first data received from an external AI model (291) using an on-device AI model (231) or confirming at least some data selected by a user from the first data, end an operation of dividing the display (260) into a plurality of display areas to visually indicate termination of communication with an external server including the external AI model (291) or end display of an indicator indicating that the external AI model (291) is operating.

[0120] The memory (230) according to one embodiment may be implemented substantially identically or similarly to the memory (130) of FIG. 1.

[0121] According to one embodiment, an on-device AI model (231) may be stored in the memory (230).

[0122] An on-device AI model (231) according to one embodiment is an AI installed in an electronic device (201) and can support functions such as translation, creation, summary, and organization to ensure user convenience without a network by assisting with work.

[0123] An on-device AI model (231) according to one embodiment can be used to personalize, manage, and regenerate first data received from an external AI model (291).

[0124] The on-device AI model (231) according to one embodiment can specify user information by linking various pieces of information, and thus can actively utilize the hidden layer only for the analysis of user information, such as combining or dividing various meta information and combining it with other information during the inference process. For example, the information on 'watching the OOO musical' and the information on 'small theater seat H-10' in the user's schedule are difficult to individually identify as user information, but can be identified as user information when the two pieces of data are combined. Therefore, the on-device AI model (231) classifies the information in the preloaded app into audio, visual, and text, and then analyzes whether user information can be inferred through the DL (dende layer), and does not accept the input data as is, but changes it into metadata or tokens of the same type, and then analyzes in advance all situations in which user information can be inferred. The on-device AI model (231) can change the weight of the user's information analysis model because information that matches even a part of the meta data after segmenting the user's request target based on the analyzed information has a higher possibility of including the user's information. According to an embodiment, the on-device AI model (231) or processor (230) can search and acquire the latest external data for processing the user's request by using a search augmentation generation (RAG) technology that references a reliable knowledge base of the external device and / or a crawling technology that can search for external data, if it determines that a keyword (e.g., "popular these days") is included in the user's request content, which is determined to increase the request processing accuracy when using data provided by an external device to process some of the user's requests.

[0125] For example, the on-device AI model (231) or processor (230) can use an information retriever, which is one of the components of RAG, to search for information related to a user request from an external large database or document set, and can use an on-device AI model (Generator) (231) to generate data corresponding to the user request based on the searched information.

[0126] For example, the on-device AI model (231) or processor (230) may search for information related to a user request from an external server using crawling, and generate data corresponding to the user request based on the searched information using the on-device AI model (231).

[0127] For example, the on-device AI model (231) or processor (230) may obtain the latest external data related to a user request keyword (e.g., “popular these days”) through a search, and then transmit information related to the external data as an input value to the on-device AI model (231). In order to more clearly perform the user request, the on-device AI model (231) or processor (230) may analyze the obtained external data (e.g., images of popular fashion these days), extract keywords that accurately represent the external data (e.g., main keywords, detailed descriptions, and / or explanatory text related to popular fashion these days), and include the extracted keywords in a prompt for processing the user request, and transmit the extracted keywords to the on-device AI model (231) for processing the user request.

[0128] In one embodiment, the processor (230) may determine whether to use data generated by an external AI model or data obtained through external data retrieval after determining whether external data is required to process a portion of a user's request.

[0129] According to one embodiment, if the processor (320) determines that personal information is included in the first object selected to process a portion of the user's request, the processor (320) may determine not to transmit the first object to an external AI model, but to search and acquire external data using RAG or crawling, etc., and to transmit the acquired external data as an input value to the on-device AI model (231) to process the user request.

[0130] According to one embodiment, the memory (230) may store a plurality of applications that can be connected to a plurality of external AI models.

[0131] A display (260) according to one embodiment may be implemented substantially identically or similarly to the display (160) of FIG. 1.

[0132] According to one embodiment, when the use of an external AI model (291) is determined, a display (260) may display information related to the operation of the external AI model (291) and information related to the operation of the on-device AI model (231) in a plurality of display areas, respectively, to visually distinguish between information related to the operation of the external AI model and information related to the operation of the on-device AI model.

[0133] A display (260) according to one embodiment may include a flexible display, a foldable display, or / and a rollable display in which the size of the visually exposed area on the front of the electronic device can be changed.

[0134] A data acquisition device (250) according to one embodiment may include a camera (251) (e.g., a camera module (180) of FIG. 1), a microphone (253) (e.g., an input module (150) of FIG. 1), a sensor (255) (e.g., a sensor (176) of FIG. 1), and a communication circuit (257) (e.g., a communication module (190) of FIG. 1). In addition to the camera (251), the microphone (253), and the communication circuit (257), the data acquisition devices (250) may further include various types of devices (e.g., various sensors, a touch screen) for acquiring various types of contents described below.

[0135] Data acquisition devices (250) according to one embodiment can acquire various types of data (or content) to be processed based on an AI model. The various types of data may include media data such as images, videos, and audio data, electronic documents, and, without limitation, may further include types of electronic data (e.g., software, sensor values) that can be electronically analyzed by an AI model.

[0136] According to one embodiment, a camera (251) (e.g., at least one front camera (251) and at least one rear camera (251)) can capture still images (or images) and videos. According to one embodiment, at least one camera (251) may include one or more lenses, image sensors, image signal processors, or flashes. In one embodiment, the electronic device (201) may include cameras (251) each having different properties or functions (or purposes). For example, at least one camera (251) may include cameras (251) having different angles of view. The angles of view may include, for example, a super wide angle of 114° to 94°, a wide angle, a normal lens of 84° to 63°, a telephoto of 28° to 8°, and a super telephoto of 6° to 3°. For example, at least one camera (251) may include at least one front camera (251) positioned at the front as described above to capture images and / or videos, and at least one rear camera (251) positioned at the rear to capture images and / or videos.

[0137] According to one embodiment, the microphone (253) can receive sound from outside the electronic device (201). For example, the processor (220) can drive the microphone (253) to receive sound generated from outside through the microphone (253). The sound generated from outside can include voices (or utterances) of speakers (e.g., a user and / or other speakers (or other people)), living noises, and ambient (or background) noises. In one embodiment, the microphone (253) can include a plurality of microphones (253). The processor (220) can form a beamforming that receives sound generated from a specified direction from the electronic device (201) from the sound received using the plurality of microphones (253). Based on the received sound, the sound from the specified direction obtained can be defined as a sub sound. Each of the plurality of microphones (253) is placed in the electronic device (201) so as to be spaced apart by a predetermined distance, and the sound received through each microphone (253) is signal-processed by the time or phase associated with the spaced distance and the direction from which the sound is to be acquired, so that a sub sound can be acquired. Since beamforming technology is a well-known technology, a detailed description thereof will be omitted.

[0138] A sensor (255) according to one embodiment may include various sensors such as an acceleration sensor, a gyroscope sensor, a geomagnetic sensor, and a temperature sensor.

[0139] A communication circuit (257) according to one embodiment can form a communication connection with an external electronic device (e.g., another electronic device or a server) through various types of communication methods, and transmit and / or receive data. As described above, the communication method may include a communication method that establishes a direct communication connection such as Bluetooth and Wi-Fi direct, a communication method that uses an access point (AP) (e.g., Wi-Fi communication), or a communication method that uses cellular communication using a base station (e.g., 3G, 4G / LTE, 5G). Since the communication circuit (227) can be implemented like the communication module (190) described above in FIG. 1, a redundant description will be omitted.

[0140] FIGS. 3a, 3b, 3c, 3d, 3e, and 3f are drawings for explaining an operation of generating data using an external AI model and an on-device AI model in an electronic device according to one embodiment.

[0141] According to an embodiment, referring to FIG. 3A, according to an embodiment, when an electronic device (201) (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) receives a user request, “Add a friend 00 who is wearing trendy clothes next to me” (331), through a microphone (e.g., 253 of FIG. 2A) of the electronic device while displaying a first image (311) representing a first object selected by a user on a display (260) of the electronic device (e.g., the display (260) of FIG. 2A), the electronic device may determine whether to use an external AI model based on the user’s request by using an on-device AI model (e.g., the on-device AI model (231) of FIG. 2B).

[0142] According to an embodiment, referring to FIG. 3b, according to an embodiment, the electronic device (201) uses an on-device AI model to satisfy a user request for "currently trendy clothing" (331a) among the user requests such as "Add a friend 00 next to me who is wearing trendy clothing these days" (331), and confirms that "currently trendy clothing" (331a) is an action that can be processed using an external AI model (e.g., the external AI model (291) of FIG. 2b), and confirms that "friend 00" (331b) is an action that can be processed by the on-device AI model as user information.

[0143] According to one embodiment, the electronic device (201) may use the on-device AI model to decide to use an external AI model based on a history of feedback that the 'clothes' generated by the on-device AI model in a previous similar user request were unsatisfactory, or based on the results of the user's request based on the keyword 'popular these days'.

[0144] In one embodiment, referring to FIG. 3C, according to one embodiment, when the electronic device (201) determines to use an external AI model using the on-device AI model, the electronic device (201) may generate a second image (313) in which the user's display area (311a) corresponding to the information in the first image (311) is deleted. In one embodiment, the electronic device (201) may generate a prompt such as "generate trendy fashion for slim women these days" using the on-device AI model, generate first information including the second image (313) and the prompt, and transmit the first information to the external AI model.

[0145] According to one embodiment, referring to FIG. 3D, when the electronic device (201) determines a decision to use an external AI model for processing a first image (311) based on a user's request, the electronic device (201) divides a display area of ​​the display (260) into a plurality of display areas (261, 263), displays information related to the operation of the external AI model in a first display area (261) among the plurality of display areas, and displays information related to the operation of the on-device AI model in a second display area (263) among the plurality of display areas. According to one embodiment, the electronic device (201) uses an on-device AI model to process a user request, “Add a friend 00 who is wearing trendy clothes next to me” (331), to search for a face image corresponding to “friend 00” among images stored in the memory of the electronic device (e.g., memory (230) of FIG. 2A), and to have the user confirm whether the searched face image is the face image of “friend 00” (333).

[0146] According to one embodiment, referring to FIG. 3e, according to one embodiment, the electronic device (201) may receive as first data a third image (351) including a person (351a) corresponding to “currently popular slim women’s fashion generation” processed based on first information from an external AI model, and may obtain a face image (371) corresponding to “friend 00” obtained from a memory of the electronic device (e.g., memory (230) of FIG. 2a) using an on-device AI model.

[0147] According to one embodiment, as shown in FIG. 3F, according to one embodiment, the electronic device (201) may use an on-device AI model to combine a person (351a) included in a third image (351) with a face image (371) corresponding to “friend 00”, and generate an image (315) of the combined person (391) positioned next to the user (311a) as third data and display the image on the display (260). According to one embodiment, when the electronic device (201) receives a third image (351) including a person (351a) corresponding to “currently popular slim women’s fashion” processed based on first information from an external AI model, the electronic device (201) may terminate the communication connection with the external server including the external AI model and terminate the operation of dividing the display area of ​​the display (260) into a plurality of display areas.

[0148] FIGS. 4a, 4b, 4c, 4d, and 4e are drawings for explaining an operation of generating data using an external AI model and an on-device AI model in an electronic device according to one embodiment.

[0149] According to one embodiment, referring to FIG. 4A, according to one embodiment, an electronic device (201) (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) may check a plurality of images (411, 412, 413, 414, 415, 416, 417, and 418) representing a first object selected by a user among images displayed on a display (260) of the electronic device (e.g., the display (260) of FIG. 2A), and when a user's request, "Write a travel report with the pictures taken when I went to the Louvre Museum" (441), is received through a microphone of the electronic device (e.g., the microphone (253) of FIG. 2A), the electronic device may determine whether to use an external AI model based on the user's request by using an on-device AI model (e.g., the on-device AI model (231) of FIG. 2B).

[0150] According to one embodiment, when the electronic device (201) receives a user request, “Write a travel report with the pictures taken when I went to the Louvre Museum” (441), using an on-device AI model, without the user selecting multiple images, the electronic device (201) can search for multiple images in which location information corresponding to the Louvre Museum is stored among the images stored in the memory of the electronic device (201) (e.g., the memory (230) of FIG. 2).

[0151] According to one embodiment, the electronic device (201) can classify a plurality of images (411, 412, 413, 414, 415, 416, 417, and 418) based on location information and / or time information and analyze each of the plurality of images using an on-device AI model.

[0152] According to one embodiment, the electronic device (201) may determine to use an external AI model (e.g., the external AI model (291) of FIG. 2B) when an input value for outputting data about works included in each of a plurality of images using an on-device AI model is determined to be less than or equal to a reference input value. According to one embodiment, the on-device AI model (231) may determine that the input value is less than or equal to a reference input value when the number of objects (e.g., images) required to generate data (e.g., reports) corresponding to a result based on a user's tendencies or feedback from previous users is less than or equal to a reference number.

[0153] According to one embodiment, as shown in FIG. 4B, according to one embodiment, the electronic device (201) may generate a second image (437) by deleting a user display area (417a) corresponding to the user information from a first image (417) including the user information among a plurality of images (411, 412, 413, 414, 415, 416, 417, and 418) using an on-device AI model. According to one embodiment, the electronic device (201) may generate a prompt such as “Briefly summarize information about the Louvre Museum” or “Briefly summarize information about works exhibited at a specific time in the Louvre Museum” using the on-device AI model, generate first information including the second image (437) and the prompt, and transmit the first information to an external AI model.

[0154] According to one embodiment, the electronic device (201) may use an on-device AI model to transmit the remaining images (411, 412, 413, 414, 415, 416, 417, and 418) excluding the first image (417) and the first information including the prompt to an external AI model, and then generate a second image (437) by deleting the user's display area (417a) corresponding to the user's information from the first image (417), and transmit the generated second image (437) to the external AI model.

[0155] According to one embodiment, the electronic device (201) may use an on-device AI model to generate a second image (437) by deleting a user display area (417a) corresponding to the user information from the first image (417), and then transmit the first information including a plurality of images (411, 412, 413, 414, 415, 416, and 418), the second image (437), and a prompt to the external AI model.

[0156] According to one embodiment, the electronic device (201) may use an on-device AI model to generate a second image (437) by deleting a user display area (417a) corresponding to the user information from the first image (417), and transmit the second image (437) and the first information including the prompt to the external AI model, and then transmit a plurality of images (411, 412, 413, 414, 415, 416, and 418) to the external AI model.

[0157] According to one embodiment, referring to FIG. 4C, according to one embodiment, when the electronic device (201) determines to use an external AI model for processing a plurality of images (411, 412, 413, 414, 415, 416, 417, and 418) based on a user's request, the electronic device (201) divides the display area of ​​the display (260) into a plurality of display areas (261, 263), displays information related to the operation of the external AI model in a first display area (261) of the plurality of display areas, and displays information related to the operation of the on-device AI model in a second display area (263) of the plurality of display areas. The electronic device (201) may perform an operation of confirming the user's answer by asking a specific question for report writing while waiting for first data from the external AI model using the on-device AI model (444).

[0158] According to one embodiment, as shown in FIG. 4d, when the electronic device (201) receives summary information about the works of art of the Louvre Museum as first data processed based on first information from an external AI model, the electronic device (201) may display the summary information about the works of art of the Louvre Museum on the first display area (261) of the display (260).

[0159] According to one embodiment, the electronic device (201) may select a summary description of the works included in each of a plurality of images from among the summary information about the works of art at the Louvre Museum received from an external AI model, or may use all of the summary information about the works of art at the Louvre Museum received from the external AI model.

[0160] According to one embodiment, when summary information about a specific work of art among the summary information about the Louvre Museum works displayed in the first display area (261) is selected by a user's input (451) (e.g., a gesture corresponding to a drag motion from the top to the bottom), the electronic device (201) may use an on-device AI model to add the summary information of an image corresponding to the specific work of art among a plurality of images.

[0161] According to one embodiment, as shown in FIG. 4e, according to one embodiment, the electronic device (201) may generate a report of the Louvre Museum as second data using a plurality of images (411, 412, 413, 414, 415, 416, 417 and 418) and summary information about the Louvre Museum's works received from an external AI model, and display the report on the display (260). When the electronic device (201) receives the summary information about the Louvre Museum's works as first data processed based on the first information from the external AI model, the electronic device (201) may terminate the communication connection with the external server including the external AI model and terminate the operation of dividing the display area of ​​the display (260) into a plurality of display areas.

[0162] According to one embodiment, when the electronic device (201) transmits first information including a second image (437) with user information deleted and a plurality of images (411, 412, 413, 414, 415, 416, and 418) to an external AI model using an on-device AI model, the external AI model may generate related information (e.g., text information such as a summary or description, additional image information, etc.) based on at least one image among the received second image (437) and the plurality of images, and then generate a report including at least one image and the related information and transmit the report to the electronic device (201).

[0163] According to one embodiment, the electronic device (201) uses an on-device AI model to check whether the received report includes a second image (437) with the user's information deleted, and if it is confirmed that the report includes the second image (437), the electronic device (201) can generate a report by replacing the second image (437) with the first image (417) containing the user's information.

[0164] FIG. 5 is a diagram illustrating an operation for determining whether to use an external AI model in an electronic device according to one embodiment.

[0165] According to one embodiment, FIG. 5 <501> As shown in the screen, according to one embodiment, an electronic device (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) transmits a first image (511) including a user (511a) wearing a Santa costume corresponding to the user's information and a first information including a prompt such as "Add appropriate weather information" to an external AI model (e.g., the external AI model (291) of FIG. 2B) using an on-device AI model (e.g., the on-device AI model (231) of FIG. 2B), and then receives the first information from the external AI model. <503> As shown in the screen, a second image (513) with a snowy image added to the first image (511) can be received.

[0166] According to one embodiment, FIG. 5 <505> As shown in the screen, according to one embodiment, the electronic device, using the on-device AI model, transmits the first information including the third image (515) in which the user (511a) wearing a Santa costume corresponding to the user's information is deleted from the first image (511) and the prompt such as "Add appropriate weather information" to an external AI model (e.g., the external AI model (291) of FIG. 2b). <5073> As shown in the screen, a fourth image (517) can be received with an image of rain added to the first image (511).

[0167] According to one embodiment, the electronic device may determine not to use the external AI model if it determines that there is a significant difference between a first image (511) including the user's information and transmitted to an external AI model for the same user's request, and a second image (513) received from the external AI model, and a third image (515) with the user's information deleted and transmitted to the external AI model, and a fourth image (517) received from the external AI model, for the same user's request.

[0168] FIGS. 6A and 6B are drawings for explaining an operation for visually distinguishing the operation of an on-device AI model and an external AI model in an electronic device according to one embodiment.

[0169] According to one embodiment, referring to FIG. 6a, <611> According to one embodiment, as shown in the screen, when the electronic device (601) (e.g., the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIG. 2) is a foldable type electronic device (601), when the decision to use an external AI model is confirmed in the folding state of the electronic device (601), the electronic device (601) divides the first display area (261) of the display of the electronic device (601) into two display areas (261a, 261b), and displays information related to the operation of the external AI model (e.g., the external AI model (291) of FIG. 2b) in the first display area (261a) of the two display areas (261a, 261b), and displays information related to the operation of the on-device AI model (e.g., the on-device AI model (291) of FIG. 2b) in the second display area (261b) of the two display areas (261a, 261b). Information related to the operation of the AI ​​model (231) can be displayed.

[0170] <613> According to one embodiment, as shown in the screen, when the electronic device (601) confirms the decision to use an external AI model in an unfolded state of the electronic device, the electronic device may divide the first display area (261) into two display areas while displaying the first display area (261) and the second display area (263) among the display areas of the electronic device. According to one embodiment, the electronic device (601) may display the operation of an external AI model (e.g., the external AI model (291) of FIG. 2B) in a first display area (261a) among two display areas (261a, 261b), display the operation of an on-device AI model (e.g., the on-device AI model (231) of FIG. 2B) in a second display area (261b) among two display areas (261a, 261b), and display at least one of various pieces of information, for example, a first object including user information, a second object from which user information transmitted to the external AI model is deleted, and temporary data corresponding to an intermediate result generated using data partially received from the external AI model prior to displaying the second data, in a second display area (263).

[0171] <615> According to one embodiment, as shown in the screen, when the electronic device (601) confirms the decision to use an external AI model in an unfolded state of the electronic device, the electronic device (601) may divide the second display area (263) into two display areas (263a, 263b) while displaying the first display area (261) and the second display area (263) among the display areas of the electronic device. According to one embodiment, the electronic device (601) may display at least one of various pieces of information, for example, a first object including user information, a second object from which user information transmitted to the external AI model is deleted, and temporary data corresponding to an intermediate result generated using data partially received from the external AI model prior to displaying the second data, on the first display area (261a). According to one embodiment, the electronic device (601) may display information related to the operation of an external AI model (e.g., the external AI model (291) of FIG. 2b) in a first display area (263a) among two display areas (263a, 263b), and may display information related to the operation of an on-device AI model (e.g., the on-device AI model (231) of FIG. 2b) in a second display area (263b) among two display areas (263a, 263b).

[0172] <617> According to one embodiment, as shown in the screen, when the electronic device (601) confirms the decision to use an external AI model in an unfolded state of the electronic device, the electronic device (601) may display information related to the operation of the external AI model (e.g., the external AI model (291) of FIG. 2B) on the first display area (261) while displaying the first display area (261) and the second display area (263) among the displays of the electronic device (601), and may display information related to the operation of the on-device AI model (e.g., the on-device AI model (231) of FIG. 2B) on the second display area (263).

[0173] According to one embodiment, referring to FIG. 6b, <631> As shown in the screen, according to one embodiment, when the electronic device (601) (e.g., the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIG. 2) is a foldable type electronic device (601), when the decision to use an external AI model is confirmed in the unfolded state of the electronic device, the electronic device (601) may divide the first display area (261) into two display areas while displaying the first display area (261), the second display area (263), and the third display area (265) among the displays of the electronic device. The electronic device (601) may display information related to the operation of an external AI model (e.g., the external AI model (291) of FIG. 2B) in a first display area (261a) among two display areas (261a, 261b), and may display information related to the operation of an on-device AI model (e.g., the on-device AI model (231) of FIG. 2B) in a second display area (261b) among two display areas (261a, 261b). According to one embodiment, the electronic device (601) may display at least one of various pieces of information, for example, a first object including user information, a second object from which user information transmitted to the external AI model is deleted, and temporary data corresponding to an intermediate result generated using data partially received from the external AI model prior to displaying the second data, in a second display area (263), and may display the execution of an application related to processing a user request in a third display area (265).

[0174] <633> As shown in the screen, according to one embodiment, when the electronic device (601) is a foldable type electronic device (601), when the decision to use an external AI model is confirmed in the unfolded state of the electronic device, the electronic device (601) may divide the second display area (263) into two display areas (263a, 263b) while displaying the first display area (261), the second display area (263), and the third display area (265) among the displays of the electronic device. According to one embodiment, the electronic device (601) may display information related to the operation of an external AI model (e.g., the external AI model (291) of FIG. 2b) on a first display area (261), display information related to the operation of an on-device AI model (e.g., the on-device AI model (231) of FIG. 2b) on a first display area (263a) of two display areas (263a, 263b), and display various information, for example, at least one of a first object including user information, a second object from which user information transmitted to the external AI model is deleted, and temporary data corresponding to an intermediate result generated using data partially received from the external AI model prior to displaying the second data, on a second display area (263b) of the two display areas (263a, 263b). According to one embodiment, the electronic device (601) may display the execution of an application related to processing a user request on a third display area (265).

[0175] According to one embodiment, <635> As shown in the screen, according to one embodiment, when the electronic device (601) is a foldable type electronic device (601), when the decision to use an external AI model is confirmed in the unfolded state of the electronic device, the electronic device (601) may divide the third display area (265) into two display areas (265a, 265b) while displaying the first display area (261), the second display area (263), and the third display area (265) among the displays of the electronic device. According to one embodiment, the electronic device (601) may display information related to the operation of an external AI model (e.g., the external AI model (291) of FIG. 2b) on the first display area (261), and display information related to the operation of an on-device AI model (e.g., the on-device AI model (231) of FIG. 2b) on the second display area (263). According to one embodiment, the electronic device (601) may display at least one of various information, for example, a first object including user information, a second object in which user information transmitted to an external AI model is deleted, and temporary data corresponding to an intermediate result generated using data partially received from an external AI model prior to displaying the second data, in a first display area (265a) among two display areas (256a, 265b), and may display the execution of an application related to processing a user request in a second display area (265b) among two display areas (256a, 265b).

[0176] FIGS. 7A and 7B are drawings for explaining an operation for visually distinguishing the operation of an on-device AI model and an external AI model in an electronic device according to one embodiment.

[0177] According to one embodiment, as shown in FIG. 7a, when an electronic device (601) (e.g., the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIG. 2) is a foldable type electronic device (601), when a decision to use an external AI model is confirmed in an unfolded state of the electronic device, the electronic device (601) may divide the first display area (261) into two display areas (261a, 261b) and divide the second display area (263) into two display areas (263a, 263b) while displaying the first display area (261) and the second display area (263) among the displays of the electronic device. According to one embodiment, the electronic device (601) may display a first image (711) (first object) including user information (711a) in a first display area (261a) among two display areas (261a, 261b) separated in a first display area (261), and may generate and display a second image (713) (second object) in which user information (711a) is deleted from the first image for transmission to an external AI model in a second display area (261b) among two display areas (261a, 261b) separated in the first display area (261). According to one embodiment, the electronic device (601) may display information related to the operation of an external AI model (e.g., the external AI model (291) of FIG. 2b) in a first display area (263a) among two display areas (263a, 263b) separated in a second display area (263), and may display information related to the operation of an on-device AI model (e.g., the on-device AI model (231) of FIG. 2b) in a second display area (263b) among two display areas (263a, 263b) separated in a second display area (263).

[0178] According to one embodiment, as shown in FIG. 7b, when the electronic device (601) (e.g., the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIG. 2) is a foldable type electronic device (601), when the decision to use an external AI model is confirmed in the unfolded state of the electronic device, the electronic device (601) may divide the second display area (263) into two display areas (263a, 263b) while displaying the first display area (261) and the second display area (263) among the displays of the electronic device.

[0179] According to one embodiment, the electronic device (601) may display reports on some of the works that were created first in the first display area (261) while generating reports on the works in chronological order. According to one embodiment, the electronic device (601) may display information related to the operation of an external AI model (e.g., the external AI model (291) of FIG. 2B) in the first display area (263a) among the two display areas (263a, 263b) separated in the second display area (263), and display information related to the operation of an on-device AI model (e.g., the on-device AI model (231) of FIG. 2B) in the second display area (263b) among the two display areas (263a, 263b) separated in the second display area (263).

[0180] FIG. 8 is a diagram for explaining an operation for visually distinguishing the operation of an on-device AI model and an external AI model in an electronic device according to one embodiment.

[0181] According to one embodiment, referring to FIG. 8, <811> As shown in the screen, according to one embodiment, when the electronic device (801) (e.g., the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) is a flexible (rollable) type electronic device, in a first state where the first display area (861) of the flexible display (rollable display) (260) of the electronic device (601) is exposed and the second display area (863) is housed within a second housing configured to be movable with respect to the first housing, a decision on the use of an external AI model can be confirmed after a plurality of images (831) for report writing are selected.

[0182] <813> As shown in the screen, according to one embodiment, the electronic device (801) can be switched from a first state to a second state in which a first display area (861) of a flexible display (rollable display) (260) of the electronic device (801) is exposed and a second display area (863) of the flexible display of the electronic device is exposed to the outside of the second housing by sliding movement (851) of the second housing in the electronic device (801) automatically or based on a user's selection. According to one embodiment, when the electronic device (801) confirms the expansion of the display area of ​​the display (260) as it changes from the first state to the second state, the electronic device (801) divides the display area of ​​the display (260) of the electronic device into two display areas (261, 263), displays information related to the operation of an external AI model (e.g., the external AI model (291) of FIG. 2b) in the first display area (261) of the two display areas, and displays information related to the operation of an on-device AI model (e.g., the on-device AI model (231) of FIG. 2b) in the second display area (263) of the two display areas.

[0183] According to one embodiment, when the electronic device (801) determines the decision to use an external AI model in the second state, the electronic device may divide the display (260) of the electronic device into two display areas (261, 263), display information related to the operation of the external AI model (e.g., the external AI model (291) of FIG. 2b) in the first display area (261) of the two display areas, and display information related to the operation of the on-device AI model (e.g., the on-device AI model (231) of FIG. 2b) in the second display area (263) of the two display areas.

[0184] FIG. 9 is a diagram illustrating an operation for visually distinguishing the operation of an on-device AI model and an external AI model in an electronic device according to one embodiment.

[0185] According to one embodiment, as shown in FIG. 9, when the electronic device (901) (e.g., the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) is a VST (video see-through) type electronic device, since the size of the display of the VST type electronic device can be diversified, when the decision to use an external AI model is confirmed, according to one embodiment, the electronic device (901) may divide the display area of ​​the display of the electronic device (901) into a plurality of display areas with the consent of the user, and display information related to the operation of the external AI model (e.g., the external AI model (291) of FIG. 2B), information related to the operation of the on-device AI model (e.g., the on-device AI model (231) of FIG. 2B), and / or various contents in each of the plurality of display areas.

[0186] According to one embodiment, a VST type electronic device (901) capable of executing an AR (Augmented Reality) function, an XR (eXtended Reality) function, and / or a VR (Virtual Reality) function provides a user with an image that is a composite of a real image input through a camera of the electronic device and an artificially generated image, in which a screen artificially processed by the electronic device is added to a real image, and the size of the display displayed to the user can be varied.

[0187] According to one embodiment, when a VST type electronic device (901) decides to use an external AI model, it may ask the user to divide the display area of ​​the display of the electronic device into a plurality of display areas and display the display area, and may recognize the user's response as positive or negative according to various trigger points of the electronic device (901) to determine the subsequent division of the display.

[0188] According to one embodiment, the VST type electronic device (901) can recognize all changes, such as a user wearing the electronic device (901) on the head moving his / her head at a certain speed or faster, performing a specific movement with his / her hand, or requesting a distinction of the display by voice, as a trigger point. According to one embodiment, if the VST type electronic device (901) determines that the user's response is positive based on a signal acquired through a sensor of the electronic device (901), the electronic device can divide the display area of ​​the electronic device into a plurality of display areas and display them, and display information related to the operation of an external AI model (e.g., the external AI model (291) of FIG. 2b), information related to the operation of an on-device AI model (e.g., the on-device AI model (231) of FIG. 2b), and / or various contents in each of the plurality of display areas.

[0189] FIG. 10 is a diagram for explaining an operation of generating data using an external AI model and an on-device AI model according to one embodiment.

[0190] According to one embodiment, with reference to FIG. 10, according to one embodiment, an electronic device (201) (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) displays an image (1011) representing a first object selected by a user on a display (260) of the electronic device (e.g., the display (260) of FIG. 2A) and, when a user's request, "Correct the reflected portion in the image" (1031), is received through a microphone of the electronic device, the electronic device may determine whether to use an external AI model based on the user's request by using an on-device AI model (e.g., the on-device AI model (231) of FIG. 2B).

[0191] According to one embodiment, when the electronic device (201) performs correction on the reflective portion (a1, a2) of the image (1011) using the on-device AI model, a new image that is different from the original image, such as by simply filling in the blank space, can be generated as second data.

[0192] According to one embodiment, when the electronic device (201) performs regeneration to generate the reflective portions (a1, a2) of the image (1011) without learning about the original image when the original image exists, and the user has low trust in the on-device AI model or may accept the result as an illusion (Hallucination), the electronic device (201) may transmit first information including a prompt requesting correction of the image (1011) and the reflective portions (a1, a2) of the image (1011) to an external AI model. According to one embodiment, the electronic device (201) may receive second data obtained by performing correction on the reflective portions (a1, a2) of the image (1011) based on the first information from the external AI model, and display the second data on the display (260). According to one embodiment, the electronic device (201) may use an on-device AI model to request an original image (e.g., an image of the Mona Lisa painting) corresponding to the image (1011) from an external AI model (e.g., an external AI model (291) of FIG. 2B), and may perform correction on the reflected portions (a1, a2) of the image (1011) based on the original image received from the external AI model, thereby generating second data that restores the image (1011) to correspond to the original image, and display the second data on the display (260).

[0193] According to one embodiment, the electronic device (201) may identify a key object included in an image (1011) (e.g., an image of the Mona Lisa painting) using an on-device AI model, and when a restoration request for the identified key object is received from a user, the electronic device (201) may determine whether the identified key object is already a known object. According to one embodiment, if the electronic device (201) determines, using the on-device AI model, that the identified key object is already a known object, the electronic device (201) may determine to use an external AI model to process the user request. According to one embodiment, the electronic device (201) may transmit the image (1011) to the external AI model using the on-device AI model, and receive an image processed based on the user's request from the external AI model and display the processed image on the display (260).

[0194] FIGS. 11A and 11B are diagrams for explaining an operation of generating data using an external AI model and an on-device AI model according to one embodiment.

[0195] According to one embodiment, referring to FIG. 11A, according to one embodiment, when an electronic device (601) (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) performs a call with an external electronic device in an unfolded state of the electronic device (601), the electronic device (601) may display a screen indicating that a call is being performed on a first display area (261) of a display of the electronic device (601), and may display an interactive message summarizing the conversation with the other user using an on-device AI model of the electronic device (e.g., the on-device AI model (231) of FIG. 2B) on a second display area (263) of the display of the electronic device (601).

[0196] According to one embodiment, the electronic device (601) may request the schedule of the other user from the on-device AI model of the external electronic device with which the call is being made using the on-device AI model of the electronic device (e.g., the on-device AI model (231) of FIG. 2b), and based on the schedule of the other user (1111) received from the on-device AI model of the external electronic device and the schedule of the user of the electronic device (1113) generated by the on-device AI model of the electronic device, generate information on dates excluding dates on which the schedules of the two users overlap as second data (1115) and output the information to the second display area (263) of the display of the electronic device (601).

[0197] According to one embodiment, when a request for schedule information of a counterpart user is required from an on-device AI model of an external electronic device performing a call, the electronic device (601) may change the request to request possible schedule information of the counterpart user that does not include personal information among the detailed schedule information of the counterpart user by using the on-device AI model of the electronic device.

[0198] According to one embodiment, when the electronic device (601) requests the schedule information of the other user to the on-device AI model of the external electronic device performing the call, the on-device AI model of the external electronic device transmits the other user's possible schedule information (e.g., information on empty dates such as July 3 to 5) instead of the other user's detailed schedule information (e.g., appointment / schedule information, etc.), and the on-device AI model of the electronic device (601) can receive the other user's possible schedule information that does not include the other user's personal information.

[0199] According to one embodiment, referring to FIG. 11B, according to one embodiment, an electronic device (601) (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) divides a first display area (261) of a display of the electronic device (601) into two display areas (261a, 261b) in an unfolded state of the electronic device (601), displays a message selected by a user in the first display area (261a) of the two display areas (261a, 261b), displays information related to the operation of an external AI model (e.g., the external AI model (291) of FIG. 2B) in the second display area (261b) of the two display areas (261a, 261b), and displays an on-device AI model (e.g., the external AI model (291) of FIG. 2B) in the second display area (263) of the display of the electronic device (601). Information related to the operation of the on-device AI model (231) can be displayed.

[0200] According to one embodiment, in order to process a user's request such as "summarize the message" (1131) using an on-device AI model, the electronic device (601) may generate (verify) second information (1133, 1135) based on the user's information in the on-device AI model based on the content of the message "1 year old credit card" (1151a) and first data (1137) received from an external AI model based on the content of the message "domestic card" (1151b) and display the second data in the second display area (263).

[0201] FIGS. 12a, 12b, 12c, 12d, and 12e are drawings for explaining operations for visually distinguishing operations of an on-device AI model and an external AI model in an electronic device according to one embodiment.

[0202] According to one embodiment, referring to FIG. 12A, according to one embodiment, an electronic device (601) (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2A-2B) may receive a user request of “Change clothes into hanbok” (1211) in a folded state of the electronic device (601), and, if the electronic device (601) confirms the decision to use an external AI model using an on-device AI model (e.g., the on-device AI model (261) of FIG. 2B), may inquire the user about whether to use the external AI model (1213). The electronic device (601) may use the on-device AI model to confirm that the user’s answer is a positive answer (1215), transmit first information to the external AI model, and display text (e.g., external AI operation) (1217) indicating that the external AI model is operating on a certain area (e.g., the top of the display) of the display (260) of the electronic device (601).

[0203] According to one embodiment, referring to FIG. 12B, according to one embodiment, an electronic device (601) (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) may, in an unfolded state of the electronic device (601), display an image (1231) selected by a user on a first display area (261) of a display of the electronic device, and may display second information generated (confirmed) by an on-device AI model and first data generated by an external AI model in a distinct manner on a second display area (263) of the display of the electronic device. According to one embodiment, when the electronic device (601) determines to use an external AI model through an on-device AI model (e.g., the on-device AI model (231) of FIG. 2b) based on a user's request such as "Correct the reflected part in the image" (1233), the electronic device (601) may display a message (1235) notifying the use of an external AI model for the title of the work included in the image and the completeness of the correction, display first data (1237) received from the external AI model, and display a message (1239) notifying that the image (1231) is being corrected in the on-device AI model based on the first data received from the external AI model. As shown in FIG. 12b, according to one embodiment, the electronic device (601) may display the on-device AI and the external AI model by distinguishing them with text (b1, a1) on the second display area (263) of the electronic device (601).

[0204] According to one embodiment, with reference to FIG. 12C, according to one embodiment, when an electronic device (601) (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) performs a call with an external electronic device in an unfolded state of the electronic device (601), the electronic device (601) may display a screen indicating that a call is being performed on a first display area (261) of the display of the electronic device (601), and may display second information generated (confirmed) by an on-device AI model and first data generated by an external AI model in a distinct manner on a second display area (263) of the display of the electronic device (601). According to one embodiment, the electronic device (601) may display an interactive message summarizing the conversation with the other user on the second display (263) using an on-device AI model (e.g., the on-device AI model (231) of FIG. 2B), while displaying first data (e.g., the other user's schedule) (1251) received from an on-device AI model (e.g., an external AI model) of an external electronic device on the second display area (263) of the display of the electronic device (601), display second information (e.g., the schedule of the user of the electronic device) (1253) generated (confirmed) by the on-device AI model of the electronic device, and display information on dates excluding dates on which the schedules of the two users overlap as second data (1255). As shown in FIG. 12c, according to one embodiment, the electronic device (601) can display the on-device AI and the on-device AI model of the external electronic device (e.g., the external AI model) by distinguishing them as text (b1, a1) in the second display area of ​​the electronic device (601).

[0205] According to one embodiment, referring to FIG. 12d, according to one embodiment, the electronic device (601) (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2a to 2b) divides the first display area (261) of the display of the electronic device (601) into two display areas (261a, 261b) in an unfolded state of the electronic device (601), displays a message selected by the user in the first display area (261a) of the two display areas (261a, 261b), displays information related to the operation of an external AI model (e.g., the external AI model (291) of FIG. 2b) in the second display area (261b) of the two display areas (261a, 261b), and displays second information external to the on-device AI model generated (confirmed) in the second display area (263) of the display of the electronic device (601). The first data generated by the AI ​​model can be displayed distinctly. According to one embodiment, when the electronic device (601) determines to use an external AI model to process a user request, such as “summarize the message” (1251), using an on-device AI model (e.g., the on-device AI model (231) of FIG. 2b), on the second display (263), the electronic device (601) can display second information (1253, 1255) generated (confirmed) using the user information in the on-device AI model of the electronic device, and can display first data (1257) generated by the external AI model, on the second display area (263) of the display of the electronic device (601). As shown in FIG. 12d, according to one embodiment, the electronic device (601) can display the on-device AI and the external AI model distinctly as text (b1, a1) on the second display area of ​​the electronic device (601).

[0206] According to one embodiment, referring to FIG. 12E, according to one embodiment, the electronic device (601) (e.g., the electronic device of FIG. 1 and / or the electronic device (201) of FIGS. 2A to 2B) may display a message selected by a user on a first display area (261) of a display of the electronic device (601) in an unfolded state of the electronic device (601), and may display second information generated (confirmed) by an on-device AI model and first data generated by an external AI model in a distinct manner on a second display area (263) of the display of the electronic device (601). According to one embodiment, when the electronic device (601) determines to use an external AI model to process a user request, such as “summarize the message” (1251), on the second display area (263) of the display of the electronic device (601), using the on-device AI model (e.g., the on-device AI model (231) of FIG. 2B), the electronic device (601) may display a message on the second display area (263) of the display of the electronic device (601). In the area (263), the second information (1253, 1255) generated (confirmed) using the user's information in the on-device AI model of the electronic device can be displayed, and the first data (1257) generated in the external AI model can be displayed. As in Fig. 12d, according to one embodiment, the electronic device (601) can display the on-device AI and the external AI model by distinguishing them with text (b1, a1) in the second display area of ​​the electronic device (601).

[0207] An electronic device according to an embodiment (e.g., an electronic device (101) of FIG. 1, an electronic device (201) of FIGS. 2A to 2B, an electronic device (601) of FIGS. 6A to 6B, and / or an electronic device (801) of FIG. 8) may include a communication circuit (e.g., a communication module (190) of FIG. 1, and / or a communication circuit (257) of FIG. 2A), a display (e.g., a display module (160) of FIG. 1, and / or a display (260) of FIG. 2A), at least one processor (e.g., a processor (120) of FIG. 1, and / or a processor (220) of FIGS. 2A to 2B), and a memory (e.g., a memory (130) of FIG. 1, and / or a memory (230) of FIG. 2A) for storing instructions. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to receive a user input related to a first object for modifying at least a portion of the first object. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to generate first information for modifying at least a portion of the first object by an external AI model, and to train the external AI model to generate at least one image using an input including text, voice, or image input. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to transmit the first information to the external AI model via the communication circuit. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to verify second information stored in the memory based on the user input and at least a portion of the first information.The instructions according to one embodiment may be configured to cause the electronic device to receive first data from the external AI model through the communication circuit when executed by the at least one processor. The instructions according to one embodiment may be configured to cause the electronic device to generate second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, and the on-device AI model may be trained to generate at least one image using an input including text, voice, or image input. The instructions according to one embodiment may be configured to cause the electronic device to provide a modified first object corresponding to the second data through the display when executed by the at least one processor.

[0208] The instructions according to one embodiment may be configured to cause the electronic device, when executed by the at least one processor, to perform generation of the first information such that the first information does not include the user's information, at least in part based on a determination that the first object includes the user's information.

[0209] The instructions according to one embodiment may be configured, when executed by the at least one processor, to cause the electronic device to generate the first information based at least in part on the user's input and the first object, at least in part based on a determination that the first object does not contain the user's information. The instructions according to one embodiment may be configured, when executed by the at least one processor, to cause the electronic device to select at least a portion of the first data received from the external AI model based on the user's information. The instructions according to one embodiment may be configured, when executed by the at least one processor, to cause the electronic device to generate the second data based on at least a portion of the selected first data and the second information using the on-device AI model.

[0210] In one embodiment, the commands, when executed by the at least one processor, may be configured to cause the electronic device to select at least a portion of the first data using the on-device AI model based on an input from another user while displaying the first data. In one embodiment, the commands, when executed by the at least one processor, may be configured to cause the electronic device to generate the second data based on at least a portion of the selected first data and the second information.

[0211] The instructions according to one embodiment may be configured to cause the electronic device, when executed by the at least one processor, to provide a first area corresponding to the first information related to the operation of the external AI model and a second area corresponding to the second information related to the operation of the on-device AI model.

[0212] According to one embodiment, the first area may be set differently from the second area. According to one embodiment, the external AI model may include at least one of an AI model included in an external server or another on-device AI model of an external electronic device that is connected to the electronic device for communication.

[0213] The self-device according to the previous embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIGS. 2A to 2B, the electronic device (601) of FIGS. 6A to 6B, and / or the electronic device (801) of FIG. 8) may include a communication circuit (e.g., the communication module (190) of FIG. 1, and / or the communication circuit (257) of FIG. 2A), a display (e.g., the display module (160) of FIG. 1, and / or the display (260) of FIG. 2A), at least one processor (e.g., the processor (120) of FIG. 1, and / or the processor (220) of FIGS. 2A to 2B), and a memory (e.g., the memory (130) of FIG. 1, and / or the memory (230) of FIG. 2A) for storing instructions. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to receive a user input related to a first object for modifying at least a portion of the first object. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to generate first information for modifying at least a portion of the first object. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to transmit the first information to an external device via the communication circuit. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to verify second information stored in the memory based on the user input and at least a portion of the first information. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to receive first data from the external device via the communication circuit.In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to generate second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, and the on-device AI model may be trained to generate at least one image using an input including text, voice, or image input. In one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to provide a modified first object corresponding to the second data through the display.

[0214] FIG. 13 is a flowchart illustrating operations for generating data using an external AI model and an on-device AI model in an electronic device according to an embodiment. The operations for generating data using an external AI model and an on-device AI model may include operations 1301 to 1317. In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, at least two operations may be performed in parallel, or other operations may be added.

[0215] In operation 1301, an electronic device (e.g., electronic device (101) of FIG. 1 and / or electronic device (201) of FIGS. 2A to 2B) may receive a user request for processing a first object.

[0216] An electronic device according to one embodiment may receive a user's request through a microphone of the electronic device (e.g., microphone (253) of FIG. 2A).

[0217] An electronic device according to one embodiment may receive a user's request based on text entered through an input window displayed in a certain area of ​​a display of the electronic device (e.g., display (260) of FIG. 2A).

[0218] A first object according to one embodiment represents a target to be processed according to a user's request, and may include various contents such as images, videos, messages, voices, and / or files.

[0219] In operation 1303, an electronic device (e.g., electronic device (101) of FIG. 1 and / or electronic device (201) of FIGS. 2A to 2B) may determine the use of an external AI model.

[0220] An electronic device according to one embodiment may determine the use of an external AI model (e.g., an external AI model (291) of FIG. 2b) by using an on-device AI model (e.g., an on-device AI model (231) of FIG. 2b) stored in a memory of the electronic device (e.g., a memory (230) of FIG. 2a).

[0221] According to one embodiment, the on-device AI model can analyze a user's request and identify actions related to the user's information as actions that the on-device AI model can process, and distinguish actions that are not related to the user's information as actions that can be processed using an external AI model.

[0222] According to one embodiment, the on-device AI model may analyze a user's request and, if it is determined that an input value (input information) for processing a first object based on the user's request is less than or equal to a reference input value (reference information), it may determine the use of an external AI model.

[0223] According to one embodiment, the on-device AI model may analyze a user's request and, if there is no input value (input information) for processing the first object based on the user's request, determine to use an external AI model.

[0224] In one embodiment, the on-device AI model may, when deciding to use an external AI model, display a message asking the user whether to use the external AI model, and, when receiving a response corresponding to acceptance from the user, decide to use the external AI.

[0225] In operation 1305, an electronic device (e.g., electronic device (101) of FIG. 1 and / or electronic device (201) of FIGS. 2A to 2B) may generate first information to be transmitted to an external AI model.

[0226] An electronic device according to one embodiment may generate first information to be transmitted to an external AI model (e.g., an external AI model (291) of FIG. 2b) using an on-device AI model (e.g., an on-device AI model (231) of FIG. 2b) stored in a memory of the electronic device (e.g., a memory (230) of FIG. 2a).

[0227] According to one embodiment, the on-device AI model can check whether the first object contains user information, and if the first object contains user information, can create a second object by editing (e.g., deleting or modifying) the first object so that the external AI model cannot recognize the user information.

[0228] In one embodiment, the on-device AI model can generate prompts (commands) to cause at least some of the user's requests to be processed by an external AI model.

[0229] According to one embodiment, the on-device AI model (231) can generate a prompt (command) by considering the usage history of a previous external AI model or user feedback.

[0230] In one embodiment, the on-device AI model may generate prompts (commands) by referencing additional information (data) unrelated to the user's information to convey more specific information to an external AI model.

[0231] An on-device AI model (231) according to one embodiment can generate a prompt (command) that does not include user information.

[0232] In one embodiment, an on-device AI model can generate first information including a prompt and a second object that does not include user information.

[0233] In one embodiment, the on-device AI model can generate first information including the first object and a prompt when it is determined that the first object does not contain user information.

[0234] In operation 1307, an electronic device (e.g., electronic device (101) of FIG. 1 and / or electronic device (201) of FIGS. 2A to 2B) may transmit first information to an external AI model.

[0235] An electronic device according to one embodiment may transmit first information to an external AI model (e.g., an external AI model (291) of FIG. 2b) via a communication circuit of the electronic device (e.g., a communication circuit (257)).

[0236] In operation 1309, an electronic device (e.g., electronic device (101) of FIG. 1 and / or electronic device (201) of FIGS. 2A to 2B) may visually distinguish and display information related to the operation of an external AI model and information related to an on-device AI model.

[0237] According to one embodiment, when an electronic device confirms a decision to use an external AI model, the electronic device divides a display area of ​​a display of the electronic device (e.g., display (260) of FIG. 2a) into a plurality of display areas, displays information related to the operation of the external AI model (e.g., external AI model (291) of FIG. 2b) in a first display area among the plurality of display areas, and displays information related to the operation of the on-device AI model in a second display area among the plurality of display areas, thereby informing a user that the external AI model is currently operating.

[0238] According to one embodiment, the processor (220) may confirm the decision to use an external AI model and display an indicator or text indicating that the external AI model (291) is operating in a certain area of ​​the display (e.g., the top of the display, a navigation bar, a tooltip, etc.).

[0239] In operation 1311, an electronic device (e.g., an electronic device (101) of FIG. 1 and / or an electronic device (201) of FIGS. 2A to 2B) can confirm (generate) second information based on information stored in a memory of the electronic device.

[0240] An electronic device according to one embodiment may generate second information based on information stored in a memory of the electronic device (e.g., memory (230) of FIG. 2a) by using an on-device AI model (e.g., on-device AI model (231) of FIG. 2b) stored in a memory of the electronic device (e.g., memory (230) of FIG. 2a).

[0241] According to one embodiment, an electronic device may use an on-device AI model (e.g., an on-device AI model (231) of FIG. 2b) stored in a memory (e.g., a memory (230) of FIG. 2a) of the electronic device to identify second information stored in the memory (230) based at least in part on a user request (user input) and the first information. According to one embodiment, an on-device AI model may generate (identify) second information based on information stored in the memory to perform an action related to the user information among the user requests.

[0242] According to one embodiment, the on-device AI model may generate a specific question or additional question to process the first object based on a user's request while waiting to receive first data processed based on first information from an external AI model (e.g., the external AI model (291) of FIG. 2b), and provide the specific question or additional question to the user via a message or voice, thereby providing that the on-device AI model and the external AI model operate in parallel.

[0243] In one embodiment, the on-device AI model may, while waiting to receive first data processed based on first information from an external AI model, preferentially provide data corresponding to a somewhat accurate result of processing the first object based on a user's request in the on-device AI model.

[0244] In one embodiment, when an on-device AI model receives a portion of first data processed based on first information from an external AI model, the on-device AI model may preferentially provide a portion of the first data received and data generated using second information generated by the on-device AI model.

[0245] In one embodiment, the on-device AI model may, while waiting to receive first data processed based on first information from an external AI model, perform an action of explaining the reason for selecting a specific external AI model among a plurality of external AI models through a message or voice.

[0246] In one embodiment, the on-device AI model may, while waiting to receive first data processed based on first information from an external AI model, generate first information containing additional information based on a user's request and transmit the first information to the external AI model.

[0247] According to one embodiment, while waiting to receive first data processed based on first information from an external AI model, the on-device AI model can generate first information including information corresponding to the user's request and transmit the first information to another external AI model based on the user's request.

[0248] In operation 1313, an electronic device (e.g., electronic device (101) of FIG. 1 and / or electronic device (201) of FIGS. 2A to 2B) can confirm reception of first data from an external AI model.

[0249] An electronic device according to one embodiment may receive first data generated based on first information from an external AI model (e.g., the external AI model (291) of FIG. 2b) through a communication circuit of the electronic device (e.g., the communication circuit (257)).

[0250] In operation 1315, an electronic device (e.g., electronic device (101) of FIG. 1 and / or electronic device (201) of FIGS. 2A to 2B) may generate second data by processing a first object based on a user's request using second information and first data.

[0251] According to one embodiment, when an electronic device receives first data from an external AI model (e.g., an external AI model (291) of FIG. 2b), the electronic device may use an on-device AI model (e.g., an on-device AI model (231) of FIG. 2b) stored in a memory (e.g., a memory (230) of FIG. 2a) of the electronic device to generate and provide second data that processes a first object based on a user's request using second information generated by the on-device AI model and first data received from the external AI model.

[0252] According to one embodiment, an on-device AI model may select at least some data based on user information from first data received from an external AI model, and learn at least some of the selected data and second information or a first object to generate second data.

[0253] According to one embodiment, the on-device AI model can generate second data by performing personalized and regenerative tasks such as reproducing, managing, and summarizing first data received from an external AI model (291) based on user information.

[0254] In one embodiment, an on-device AI model may select at least some data based on user information from first data generated by an external AI model, and generate second data by including at least some of the selected data in second information generated by the on-device AI model.

[0255] In one embodiment, the on-device AI model can display first data received from an external AI model, while verifying at least some data selected by a user from among the first data, and learn the verified at least some data and second information to generate second data.

[0256] In one embodiment, the on-device AI model, while displaying first data received from an external AI model, may generate second data by including the selected at least some data in second information generated by the on-device AI model, if the on-device AI model confirms selection of at least some data among the first data based on a user's gesture (e.g., slide, drag, short touch, long touch, multi-touch, double touch, etc.).

[0257] In one embodiment, the on-device AI model receives first data from an external AI model (291), and if there is no additional request from the user for the external AI model after a certain period of time has elapsed and it determines that selection of at least some of the necessary data from the first data has been completed, the on-device AI model may terminate the communication connection with the external AI model (291) and learn at least some of the data and second information to generate second data.

[0258] According to one embodiment, an on-device AI model may use first data received from an external AI model as learning data, and generate second data that is regenerated or summarized based on the learned data and second information generated by the on-device AI model.

[0259] According to one embodiment, an on-device AI model may select at least some data based on the user's status (e.g., the country, place, season, or time where the user is currently located) from among first data received from an external AI model, and generate second data by learning at least some of the selected data and second information generated by the on-device AI model (231).

[0260] According to one embodiment, the on-device AI model can analyze first data received from an external AI model and select at least some necessary data from the first data by linking it with user information.

[0261] In one embodiment, the on-device AI model can generate second data by combining information about the user (e.g., schedule, daily routine, hobbies, tastes, etc.) with at least some data selected from the first data, and can increase the understanding of the user's characteristics by relearning the second data.

[0262] According to one embodiment, an on-device AI model may, based on analysis of first data received from an external AI model, identify a plurality of candidate data, and then select only some of the candidate data based on the user's status and use them to generate second data.

[0263] In operation 1317, an electronic device (e.g., electronic device (101) of FIG. 1 and / or electronic device (201) of FIGS. 2A to 2B) may display second data.

[0264] An electronic device according to one embodiment may display second data obtained by processing a first object based on a user's request on a display of the electronic device (e.g., display (260) of FIG. 2A).

[0265] In one embodiment, a method for generating data using an external AI model and an on-device AI model in an electronic device (e.g., the electronic device 101 of FIG. 1, the electronic device 201 of FIGS. 2A to 2B, the electronic device 601 of FIGS. 6A to 6B, and / or the electronic device 801 of FIG. 8) may include receiving a user input related to a first object for modifying at least a portion of the first object. In one embodiment, the method may include generating first information for modifying at least a portion of the first object by the external AI model, and training the external AI model to generate at least one image using an input including text, voice, or image input. In one embodiment, the method may include transmitting the first information to the external AI model via a communication circuit of the electronic device. In one embodiment, the method may include verifying second information stored in the memory based on the user input and at least a portion of the first information. The method according to one embodiment may include an operation of receiving first data from the external AI model through the communication circuit. The method according to one embodiment may include an operation of generating second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, and the on-device AI model may include an operation of being trained to generate at least one image using an input including text, voice, or image input. The method according to one embodiment may include an operation of providing a modified first object corresponding to the second data through a display of the electronic device.

[0266] The method according to one embodiment may further include generating the first information such that the first information does not include the user's information, based at least in part on a determination that the first object includes the user's information.

[0267] The method according to one embodiment may further include generating the first information based at least in part on an input of the user and the first object, based at least in part on a determination that the first object does not contain information of the user.

[0268] The method according to one embodiment may include an operation of selecting at least a portion of the first data received from the external AI model based on information about the user. The method according to one embodiment may further include an operation of generating the second data based on at least a portion of the selected first data and the second information using the on-device AI model.

[0269] According to one embodiment, the method may include an operation of selecting at least a portion of the first data using the on-device AI model based on input from another user while displaying the first data. According to one embodiment, the method may further include an operation of generating the second data based on at least a portion of the selected first data and the second information.

[0270] The method according to one embodiment may further include providing a first area corresponding to the first information related to the operation of the external AI model and a second area corresponding to the second information related to the operation of the on-device AI model.

[0271] In one embodiment, the method comprises: the first region is different from the second region.

[0272] According to one embodiment, the method may include an operation of displaying a message indicating whether the external AI model is being used via the display. According to one embodiment, the method may further include an operation of generating the first information based at least in part on another user's input related to the message.

[0273] In one embodiment, a method for generating data using an external AI model and an on-device AI model in an electronic device (e.g., the electronic device 101 of FIG. 1, the electronic device 201 of FIGS. 2A to 2B, the electronic device 601 of FIGS. 6A to 6B, and / or the electronic device 801 of FIG. 8) may include receiving a user input related to a first object for modifying at least a portion of the first object. In one embodiment, the method may include generating first information for modifying at least a portion of the first object. In one embodiment, the method may include transmitting the first information to an external device via a communication circuit of the electronic device. In one embodiment, the method may include confirming second information stored in the memory based on the user input and at least a portion of the first information. In one embodiment, the method may include receiving first data from the external device via the communication circuit. The method according to one embodiment may include an operation of generating second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, and the on-device AI model is trained to generate at least one image using an input including text, voice, or image input. The method according to one embodiment may include an operation of providing a modified first object corresponding to the second data through a display of the electronic device.

[0274] Electronic devices according to embodiments disclosed herein may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments disclosed herein are not limited to the aforementioned devices.

[0275] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0276] The term "module" used in one embodiment of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0277] An embodiment of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101) or an electronic device (301)). For example, a processor (e.g., a processor (520)) of the machine (e.g., an electronic device (301)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0278] According to one embodiment, the method according to one embodiment disclosed in the present document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0279] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101 of FIG. 1; 201 of FIGS. 2a to 2b; 601 of FIGS. 6a to 6b; 801 of FIG. 8a), Communication circuit (190 in Fig. 1; 257 in Fig. 2a) Display (160 in Fig. 1; 260 in Fig. 2a); At least one processor (120 in FIG. 1; 220 in FIGS. 2A to 2B); and It includes a memory (130 in Fig. 1; 230 in Fig. 2a) that stores commands, The above instructions, when executed by the at least one processor, cause the electronic device to: Receiving a user input related to a first object for modifying at least a portion of the first object, generating first information for modifying at least a portion of the first object by an external AI model, wherein the external AI model is trained to generate at least one image using an input including text, voice, or image input, Transmitting the first information to the external AI model through the communication circuit; Verifying second information stored in the memory based on the user's input and at least a part of the first information, Receive first data from the external AI model through the communication circuit; Using an on-device AI model, second data is generated by modifying at least a portion of the first object based on the second information and the first data, and the on-device AI model is trained to generate at least one image using an input including text, voice, or image input. An electronic device configured to provide a modified first object corresponding to the second data through the display.

2. In paragraph 1, The above instructions, when executed by the at least one processor, cause the electronic device to: An electronic device configured to generate the first information such that the first information does not include the user's information, based at least in part on a determination that the first object contains the user's information.

3. In paragraph 1 or 2, The above instructions, when executed by the at least one processor, cause the electronic device to: An electronic device configured to generate the first information based at least in part on an input from the user and the first object, based at least in part on a determination that the first object does not contain information about the user.

4. In any one of paragraphs 1 to 3, The above instructions, when executed by the at least one processor, cause the electronic device to: Selecting at least a portion of the first data received from the external AI model based on the user's information; An electronic device configured to generate the second data based on at least a portion of the selected first data and the second information using the on-device AI model.

5. In any one of paragraphs 1 to 4, The above instructions, when executed by the at least one processor, cause the electronic device to: While displaying the first data, at least a portion of the first data is selected using the on-device AI model based on input from another user, An electronic device configured to generate the second data based on at least a portion of the selected first data and the second information.

6. In any one of paragraphs 1 to 5, The above instructions, when executed by the at least one processor, cause the electronic device to: An electronic device configured to provide a first area corresponding to the first information related to the operation of the external AI model and a second area corresponding to the second information related to the operation of the on-device AI model.

7. In any one of paragraphs 1 to 6, An electronic device wherein the first region is set differently from the second region.

8. In any one of paragraphs 1 to 7, The above instructions, when executed by the at least one processor, cause the electronic device to: Display a message indicating whether the external AI model is used through the above display, An electronic device configured to generate said first information based at least in part on input from another user related to said message.

9. In any one of paragraphs 1 to 8, The above external AI model is an electronic device including at least one of an AI model included in an external server or another on-device AI model of an external electronic device that is connected to the electronic device for communication.

10. In an electronic device (101 of FIG. 1; 201 of FIGS. 2a to 2b; 601 of FIGS. 6a to 6b; 801 of FIG. 8a), Communication circuit (190 in Fig. 1; 257 in Fig. 2a) Display (160 in Fig. 1; 260 in Fig. 2a); At least one processor (120 in FIG. 1; 220 in FIGS. 2A to 2B); and It includes a memory (130 in Fig. 1; 230 in Fig. 2a) that stores commands, The above instructions, when executed by the at least one processor, cause the electronic device to: Receiving user input related to a first object for modifying at least a portion of the first object; Generate first information for modifying at least a part of the first object, Transmitting the first information to an external device through the communication circuit, Verifying second information stored in the memory based on the user's input and at least a part of the first information, Receive first data from the external device through the communication circuit, Using an on-device AI model, second data is generated by modifying at least a portion of the first object based on the second information and the first data, and the on-device AI model is trained to generate at least one image using an input including text, voice, or image input. An electronic device configured to provide a modified first object corresponding to the second data through the display.

11. A method for generating data using an external AI model and an on-device AI model in an electronic device, An action of receiving a user input related to a first object for modifying at least a portion of the first object; An action of generating first information for modifying at least a portion of the first object by an external AI model, wherein the external AI model is trained to generate at least one image using an input including text, voice, or image input; An operation of transmitting the first information to the external AI model through a communication circuit of the electronic device; An operation of verifying second information stored in the memory based on the user's input and at least a portion of the first information; An operation of receiving first data from the external AI model through the communication circuit; An operation of generating second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, wherein the on-device AI model is trained to generate at least one image using an input including text, voice, or image input; and A method comprising providing a modified first object corresponding to the second data through a display of the electronic device.

12. In paragraph 11, A method further comprising: generating the first information so that the first information does not include the user's information, based at least in part on a determination that the first object contains the user's information.

13. In paragraph 11 or 12, A method further comprising: generating the first information based at least in part on an input of the user and the first object, based at least in part on a determination that the first object does not contain information of the user.

14. A method for generating data using an external AI model and an on-device AI model in an electronic device, An action of receiving user input related to a first object to modify at least a portion of the first object; Generate first information for modifying at least a part of the first object, An operation of transmitting the first information to an external device via a communication circuit of the electronic device; An operation of verifying second information stored in the memory based on the user's input and at least a portion of the first information; An operation of receiving first data from the external device through the communication circuit; An operation of generating second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, wherein the on-device AI model is trained to generate at least one image using an input including text, voice, or image input; and A method comprising providing a modified first object corresponding to the second data through a display of the electronic device.

15. In a non-volatile storage medium storing commands, the commands are configured to cause the electronic device to perform at least one operation when executed by the electronic device, wherein the at least one operation is: An action of receiving a user input related to a first object for modifying at least a portion of the first object; An action of generating first information for modifying at least a portion of the first object by an external AI model, wherein the external AI model is trained to generate at least one image using an input including text, voice, or image input; An operation of transmitting the first information to the external AI model through a communication circuit of the electronic device; An operation of verifying second information stored in the memory based on the user's input and at least a portion of the first information; An operation of receiving first data from the external AI model through the communication circuit; An operation of generating second data by modifying at least a portion of the first object based on the second information and the first data using an on-device AI model, wherein the on-device AI model is trained to generate at least one image using an input including text, voice, or image input; and A storage medium comprising an action of providing a modified first object corresponding to the second data through a display of the electronic device.

Images