Electronic device and method for processing artificial intelligence task by using artificial intelligence model, and non-transitory storage medium

Abstract

The present document relates to an electronic device and a method for processing an artificial intelligence task by using an artificial intelligence model, and a non-transitory storage medium. According to one embodiment, the electronic device may comprise communication circuitry, at least one processor, and a memory for storing instructions. When executed individually or collectively by the at least one processor, the instructions can instruct the electronic device to: identify an artificial intelligence task for user input information on the basis of receiving the user input information; establish, through the communication circuitry, a communication connection between the electronic device and at least one external electronic device (403) for distributed processing of the artificial intelligence task by using first short-range wireless communication (603) for security; acquire, from the at least one external electronic device, information related to an available resource and artificial intelligence learning data for distributed processing of the artificial intelligence task; identify, on the basis of the information related to the available resource and the artificial intelligence learning data acquired from the at least one external electronic device, a first task, which is a part of the artificial intelligence task to be distributively processed using a first artificial intelligence model of the electronic device, and a second task, which is the remaining part of the artificial intelligence task to be distributively processed using a second artificial intelligence model of the at least one external electronic device; acquire, on the basis of a first artificial intelligence command for the first task transmitted to the first artificial intelligence model, first result information generated by the first artificial intelligence model; on the basis of a second artificial intelligence command for the second task transmitted to the at least one external electronic device, receive, through the communication circuitry, second result information generated by the second artificial intelligence model, by using the first short-range wireless communication for security; and acquire output information including the first result information and the second result information. Other embodiments are also possible.

Description

Electronic device, method, and non-transient storage medium for processing artificial intelligence tasks using an artificial intelligence model

[0001] The present disclosure relates to an electronic device, a method, and a non-transient storage medium for processing artificial intelligence tasks using an artificial intelligence model.

[0002] With the advancement of digital technology, electronic devices are being provided in various forms such as smartphones, tablet PCs, PDAs, XR (extended reality), VR (virtual reality), AR (augmented reality), or HMDs (head-mounted displays). Electronic devices are also being developed in wearable forms to enhance portability and user accessibility.

[0003] An electronic device may utilize artificial intelligence (AI) models to provide various services. At least some of the various AI models for various services may be implemented as generative AI models. Depending on the implementation, the AI ​​models may operate in a form where multiple AI models are connected.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0005] Electronic devices can be implemented in a form that includes on-device artificial intelligence (AI) models. While applying on-device AI technology to electronic devices offers advantages such as privacy protection and fast response speeds, operating within limited resources like memory and CPU has made it difficult to perform model updates or complex tasks. When electronic devices are implemented in various forms such as XR, VR, AR, or HMDs, difficulties may arise in applying AI technology due to device characteristics, such as limited thermal design, mounting space, battery, or the high data throughput of the chipset. Furthermore, if electronic devices process AI tasks using AI models contained in a server, vulnerabilities regarding security, which is considered critical for electronic devices, may occur.

[0006] The electronic device of the present disclosure aims to provide an electronic device, a method, and a non-transient storage medium for processing artificial intelligence tasks using an artificial intelligence model to distribute and process AI tasks through collaboration with external electronic devices located in the surrounding environment, in order to smoothly provide AI services by resolving usage limitations caused by network latency and security issues when using AI services of a server, and usage limitations caused by insufficient processors and limited capabilities of AI services using an on-device artificial intelligence model within the electronic device.

[0007] According to one embodiment of the present disclosure, an electronic device comprises at least one processor including a communication circuit and a processing circuit, and a memory for storing instructions.

[0008] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device identifies an artificial intelligence task for the user input information based on receiving the user input information.

[0009] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device establishes a communication connection between the electronic device and at least one external electronic device for distributed processing of the artificial intelligence task through the communication circuit using a first short-range wireless communication for security.

[0010] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device obtains information related to resources available for distributed processing of the artificial intelligence task and artificial intelligence learning data from the at least one external electronic device.

[0011] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device identifies a first task which is part of the artificial intelligence task to be distributedly processed using the first artificial intelligence model of the electronic device and a second task which is the remainder of the artificial intelligence task to be distributedly processed using the second artificial intelligence model of the at least one external electronic device, based on information related to the available resources and artificial intelligence learning data obtained from the at least one external electronic device.

[0012] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device obtains first result information generated by the first artificial intelligence model based on a first artificial intelligence command for the first task transmitted to the first artificial intelligence model.

[0013] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device is configured to receive, through the communication circuit, second result information generated by the second artificial intelligence model based on the second artificial intelligence command for the second task transmitted to the at least one external electronic device, using the first short-range wireless communication for security.

[0014] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device obtains output information including the first result information and the second result information.

[0015] According to one embodiment, a method of operation in an electronic device includes an operation of identifying an artificial intelligence operation for user input information based on receiving user input information.

[0016] According to one embodiment, the method includes the operation of establishing a communication connection between the electronic device and at least one external electronic device for distributed processing of the artificial intelligence task through the communication circuit using a first short-range wireless communication for security.

[0017] According to one embodiment, the method includes the operation of obtaining information related to resources available for distributed processing of the artificial intelligence task and artificial intelligence learning data from the at least one external electronic device.

[0018] According to one embodiment, the method includes, based on information related to available resources and artificial intelligence learning data obtained from at least one external electronic device, identifying a first task which is part of the artificial intelligence task to be distributed using a first artificial intelligence model of the electronic device and a second task which is part of the artificial intelligence task to be distributed using a second artificial intelligence model of the at least one external electronic device.

[0019] According to one embodiment, the method includes the operation of obtaining first result information generated by the first artificial intelligence model based on a first artificial intelligence command for the first task transmitted to the first artificial intelligence model.

[0020] According to one embodiment, the method includes the operation of receiving second result information generated by the second artificial intelligence model based on a second artificial intelligence command for the second task transmitted to the at least one external electronic device through the communication circuit using the first short-range wireless communication for security.

[0021] According to one embodiment, the method includes an operation of obtaining output information including the first result information and the second result information.

[0022] According to one embodiment, in a non-transient storage medium storing one or more programs, the one or more programs include a command that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of identifying an artificial intelligence operation for said user input information based on receiving said user input information.

[0023] According to one embodiment, the one or more programs include a command that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of establishing a communication connection between the electronic device and at least one external electronic device for distributed processing of the artificial intelligence task through the communication circuit using a first short-range wireless communication for security.

[0024] According to one embodiment, the one or more programs include a command that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of obtaining information related to resources available for distributed processing of the artificial intelligence task and artificial intelligence learning data from at least one external electronic device.

[0025] According to one embodiment, the one or more programs include an instruction to cause the electronic device to execute, upon execution by at least one processor of the electronic device, an operation to identify a first task which is part of the artificial intelligence task to be distributed to be processed using a first artificial intelligence model of the electronic device and a second task which is part of the artificial intelligence task to be distributed to be processed using a second artificial intelligence model of the at least one external electronic device, based on information related to the available resources and artificial intelligence learning data obtained from the at least one external electronic device.

[0026] According to one embodiment, the one or more programs include a command that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of obtaining first result information generated by the first artificial intelligence model based on a first artificial intelligence command for the first task transmitted to the first artificial intelligence model.

[0027] According to one embodiment, the one or more programs include a command that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of receiving second result information generated by the second artificial intelligence model through the communication circuit using the first short-range wireless communication for security, based on a second artificial intelligence command for the second task transmitted to at least one external electronic device.

[0028] According to one embodiment, the one or more programs include a command that causes the electronic device to execute an operation of obtaining output information including the first result information and the second result information when executed by at least one processor of the electronic device.

[0029] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0030] FIG. 2 is a perspective view showing the structure of an electronic device according to one embodiment.

[0031] FIG. 3a is a perspective view showing the structure of an electronic device according to one embodiment.

[0032] FIGS. 3B and FIGS. 3C are perspective views showing the structure of an electronic device according to one embodiment.

[0033] FIG. 4 is a block diagram showing an example of the configuration of an electronic device according to one embodiment.

[0034] FIG. 5 is a block diagram showing an example of the configuration of an electronic device according to one embodiment.

[0035] FIGS. 6a and 6b are drawings illustrating examples of short-range wireless communication between an electronic device and external electronic devices according to one embodiment.

[0036] FIGS. 7a and 7b are drawings illustrating an example of distributing artificial intelligence tasks with an external electronic device in an electronic device according to one embodiment.

[0037] FIGS. 8A, FIGS. 8B, and FIGS. 8C are drawings illustrating an example of distributing artificial intelligence tasks with an external electronic device in an electronic device according to one embodiment.

[0038] FIG. 9 is a drawing showing an example of an operation method in an electronic device according to one embodiment.

[0039] FIG. 10 is a diagram illustrating an example of secure communication between an electronic device and external electronic devices according to one embodiment.

[0040] FIG. 11 is a drawing showing an example of a method of operation of an electronic device and external electronic devices according to one embodiment.

[0041] FIG. 12 is a diagram illustrating a generative artificial intelligence system according to one embodiment.

[0042] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0043] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity. The term "user" as used in the embodiments of the present disclosure may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0044] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments.

[0045] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0046] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0047] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0048] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0049] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0050] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0051] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0052] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0053] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0054] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0055] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0056] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0057] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0058] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0059] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0060] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0061] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0062] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0063] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0064] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0065] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0066] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0067] FIG. 2 is a perspective view showing the structure of an electronic device according to one embodiment.

[0068] Referring to FIGS. 1 and FIGS. 2, an electronic device (200) according to one embodiment may be an electronic device (101) of FIG. 1, an electronic device (102 or 104) communicating with the electronic device (101) of FIG. 1, or a device capable of providing services related to virtual reality technology that provides a virtual environment similar to the electronic device (101) of FIG. 1. Virtual reality (VR) technology, which is a technology that provides a virtual environment, may be developed into augmented reality (AR), mixed reality (MR), and / or extended reality (XR) that encompasses these. According to one embodiment, virtual reality (VR) technology may be described in a sense that includes augmented reality (AR), mixed reality (MR), and / or extended reality (XR).

[0069] The electronic device (200) may be a device configured to be wearable on a user's body, as illustrated in FIG. 2 (e.g., a head-mounted display (HMD) or a glasses-type AR glasses device). For example, the electronic device (200) may be configured to combine with an external electronic device, such as a mobile device, and may utilize components of the external electronic device (e.g., a display module, a camera module, an audio output module, or other components). Not limited thereto, the electronic device (200) may be implemented in various forms that can be worn on a user's body.

[0070] According to one embodiment, the electronic device (200) may configure a virtual reality space (e.g., an augmented reality space) that displays an augmented reality image corresponding to a real environment captured in the surrounding environment where the user is located, or a virtual image provided (e.g., a 2D or 3D image), and may control a display module (160) to display at least one virtual object corresponding to the user and / or at least one virtual object corresponding to an object for user interaction in the virtual reality space.

[0071] According to one embodiment, the electronic device (200) may include a processor (120), memory (130), display module (160), sensor module (176), camera module (180), charging module (e.g., battery (189) of FIG. 1) and communication module (190) as shown in FIG. 1. The electronic device (200) may further include an acoustic output device (155), an input module (150) as shown in FIG. 1, or other components as shown in FIG. 1. In addition, the electronic device (200) may be configured to include other components necessary to provide augmented reality functions (e.g., services or methods).

[0072] According to one embodiment, the processor (120) is electrically connected to other components and can control other components. The processor (120) can perform various data processing or operations in accordance with the execution of various functions (e.g., operations, services, or programs) provided by the electronic device (200). The processor (120) can perform various data processing or operations to display at least one virtual object related to real objects included in an image captured in real space and / or a virtual object corresponding to a user (e.g., an avatar) in a virtual reality space. The processor (120) can perform various data processing or operations to express user interaction or movement of the virtual object displayed in the virtual reality space.

[0073] Again, with reference to FIG. 2, an electronic device (200) according to one embodiment will be described. As described above, the electronic device (200) is not limited to a glasses-type (e.g., AR glasses) augmented reality device, and can be implemented as various devices capable of providing immersive content (e.g., content based on XR technology) to the user's eyes (e.g., AR head-mounted display type, 2D / 3D head-mounted display device or VR head-mounted display device).

[0074] According to one embodiment, a camera module of an electronic device (200) (e.g., camera module (180) of FIG. 1) may capture still images and / or videos. According to one embodiment, the camera module may be placed within a lens frame and around a first display (251) and a second display (252). According to one embodiment, the camera module may include one or more first cameras (211-1, 211-2), one or more second cameras (212-1, 212-2), and one or more third cameras (213). According to one embodiment, images acquired through one or more first cameras (211-1, 211-2) may be used for detecting hand gestures by a user, tracking the user's head, and / or spatial recognition. One or more first cameras (211-1, 211-2) may be a GS (global shutter) camera or an RS (rolling shutter) camera. One or more first cameras (211-1, 211-2) can perform simultaneous localization and mapping (SLAM) operations through depth imaging. One or more first cameras (211-1, 211-2) can perform spatial recognition and / or motion recognition for 3DoF (depth of field) and / or 6DoF. According to one embodiment, the first cameras (211-1, 211-2) can periodically or non-periodically transmit information related to the trajectory of a user's eye or gaze (e.g., trajectory information) to a processor (e.g., processor (120) of FIG. 1).

[0075] According to one embodiment, the electronic device (200) may use another camera (e.g., a third camera (213)) for hand detection and tracking and user gesture recognition. According to one embodiment, at least one of the first camera (211-1, 211-2) to the third camera module (213) may be replaced with a sensor module (e.g., a LiDAR sensor). For example, the sensor module may include at least one of a vertical cavity surface emitting laser (VCSEL), an infrared sensor, and / or a photodiode.

[0076] According to one embodiment, images acquired through one or more second cameras (212-1, 212-2) may be used to detect and track the user's pupils. One or more second cameras (212-1, 212-2) may be GS cameras. One or more second cameras (212-1, 212-2) may correspond to the left eye and the right eye, respectively, and the performance of one or more second cameras (212-1, 212-2) may be substantially identical. One or more third cameras (213) may be relatively high-resolution cameras. One or more third cameras (213) may perform auto-focusing (AF) and optical image stabilization (OIS) functions. One or more third cameras (213) may be GS (global shutter) cameras or RS (rolling shutter) cameras. One or more third cameras (213) may be color cameras.

[0077] According to one embodiment, the electronic device (200) may include one or more light-emitting elements (214-1, 214-2). The light-emitting elements (214-1, 214-2) are different from the light source described below, which irradiates light onto a screen output area of ​​a display. According to one embodiment, the light-emitting elements (214-1, 214-2) may irradiate light to facilitate pupil detection in detecting and tracking a user's pupil through one or more second cameras (212-1, 212-2). According to one embodiment, the light-emitting elements (214-1, 214-2) may each include an LED. According to one embodiment, the light-emitting elements (214-1, 214-2) may irradiate light in the infrared region. According to various embodiments, the light-emitting elements (214-1, 214-2) may be attached around the frame of the augmented reality device (200). According to one embodiment, a light-emitting element (214-1, 214-2) is positioned around one or more first cameras (211-1, 211-2) and can assist gesture detection, head tracking, and spatial recognition by one or more first cameras (211-1, 211-2) when the augmented reality device (200) is used in a dark environment. According to one embodiment, a light-emitting element (214-1, 214-2) is positioned around one or more third cameras (213) and can assist image acquisition by one or more third cameras (213) when the augmented reality device (200) is used in a dark environment.

[0078] According to one embodiment, the electronic device (200) may include a battery (235-1, 235-2) (e.g., battery (189) of FIG. 1). The battery (235-1, 235-2) may store power to operate the remaining components of the augmented reality device (200).

[0079] According to one embodiment, a display module of an electronic device (200) (e.g., a display module (160) of FIG. 1) may include a first display (251), a second display (252), one or more input optical members (253-1, 253-2), one or more transparent members (290-1, 290-2), and one or more screen display portions (254-1, 254-2). According to one embodiment, the first display (251) and the second display (252) may be light output modules and may include, for example, a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED). According to one embodiment, if the first display (251) and the second display (252) are composed of a liquid crystal display device, a digital mirror display device, or a silicon liquid crystal display device, the augmented reality device (200) may include a light source that irradiates light onto a screen output area of ​​the display. According to one embodiment, if the first display (251) and the second display (252) can generate light themselves, for example, if they are composed of an organic light-emitting diode or a micro LED, the augmented reality device (200) may provide a user with a good quality virtual image (e.g., an image of a virtual reality space) without including a separate light source.

[0080] According to one embodiment, one or more transparent members (290-1, 290-2) included in the electronic device (200) may be positioned to face the user's eyes when the user wears the augmented reality device (200). The one or more transparent members (290-1, 290-2) may include at least one of a glass plate, a plastic plate, or a polymer. When the user wears the augmented reality device (200), the user can see the outside world through the one or more transparent members (290-1, 290-2).

[0081] According to one embodiment, one or more input optical members (253-1, 253-2) included in the electronic device (200) can guide light generated from a first display (251) and a second display (252) to the user's eye. An image based on the light generated from the first display (251) and the second display (252) is formed on one or more screen display portions (254-1, 254-2) on one or more transparent members (290-1, 290-2), and the user can see the image formed on the one or more screen display portions (254-1, 254-2).

[0082] According to one embodiment, the electronic device (200) may include one or more optical waveguides (not shown). The optical waveguides may transmit light generated from the first display (251) and the second display (252) to the user's eye. The augmented reality device (200) may include one optical waveguide each corresponding to the left eye and the right eye. According to one embodiment, the optical waveguides may include at least one of glass, plastic, or polymer. The optical waveguides may include a nano-pattern formed on an inner or outer surface, for example, a polygonal or curved grating structure. The optical waveguides may include a free-form prism, in which case the optical waveguides may provide incident light to the user through a reflective mirror. According to one embodiment, the optical waveguide includes at least one of a diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) or a reflective element (e.g., a reflective mirror), and can guide display light emitted from a light source to the user's eye using at least one diffractive element or reflective element included in the optical waveguide. According to one embodiment, the diffractive element may include an input / output optical member. According to one embodiment, the reflective element may include a member that causes total internal reflection.

[0083] According to one embodiment, the electronic device (200) may include one or more voice input devices (262-1, 262-2, 262-3) and one or more voice output devices (263-1, 263-2).

[0084] According to one embodiment, the electronic device (200) may include a first PCB (270-1) and a second PCB (270-2). The first PCB (270-1) and the second PCB (270-2) may transmit electrical signals to components included in the electronic device (200), such as a first camera (211-1, 211-2), a second camera (212-1, 212-2), a third camera (213), a display (251, 252), an audio module (e.g., the audio module (170) of FIG. 1), and a sensor module (e.g., the sensor module (176) of FIG. 1). According to one embodiment, the first PCB (270-1) and the second PCB (270-2) may be flexible printed circuit boards (FPCB). According to one embodiment, the first PCB (270-1) and the second PCB (270-2) may each include a first substrate, a second substrate, and an interposer disposed between the first substrate and the second substrate.

[0085] FIG. 3a is a perspective view showing the structure of an electronic device according to one embodiment.

[0086] Referring to FIG. 3a, an electronic device (300) according to one embodiment (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2) may be a wearable device such as a head-mounted device (HMD) that can be worn on a user's head to provide an image (e.g., a virtual reality space image) in front of the eyes. The configuration of the electronic device (300) of FIG. 3 may be all or partly the same as the configuration of the electronic device (200) of FIG. 2.

[0087] According to one embodiment, the electronic device (300) may include a housing (310, 320, 330) that can form an exterior and provide a space in which components of the electronic device (300) can be placed.

[0088] According to one embodiment, the electronic device (300) may include a first housing (310) that can surround at least a portion of the user's head. According to one embodiment, the first housing (310) may include a first surface (300a) facing the outside of the electronic device (300) (e.g., in the +X direction).

[0089] According to one embodiment, the first housing (310) may surround at least a portion of the internal space (I). For example, the first housing (310) may include a second surface (300b) facing the internal space (I) of the electronic device (300) and a third surface (300c) opposite to the second surface (300b). According to one embodiment, the first housing (310) may be combined with a third housing (330) to form a closed curve shape surrounding the internal space (I).

[0090] According to one embodiment, the first housing (310) may accommodate at least some of the components of the electronic device (300). For example, a light output module, a circuit board, and a speaker module may be placed within the first housing (310).

[0091] According to one embodiment, a display member (340) corresponding to the left and right eyes of the electronic device (300) may be included. The display member (340) may be placed in a first housing (310). The configuration of the display member (340) of FIG. 3 may be all or partly the same as the configuration of the screen display portion (254-1, 254-2) of FIG. 2.

[0092] According to one embodiment, the electronic device (300) may include a second housing (320) that can be placed on the user's face. According to one embodiment, the second housing (320) may include a fourth surface (300d) that can face at least partially the user's face. According to one embodiment, the fourth surface (300d) may be a surface facing the internal space (I) of the electronic device (300) (e.g., -X direction). According to one embodiment, the second housing (320) may be combined with the first housing (310).

[0093] According to one embodiment, the electronic device (300) may include a third housing (330) that can be seated on the back of the user's head. According to one embodiment, the third housing (330) may be combined with the first housing (310). According to one embodiment, the third housing (330) may accommodate at least some of the components of the electronic device (300). For example, a battery (e.g., the battery (235-1, 235-2) of FIG. 2) may be placed within the third housing (330).

[0094] In order to enhance the user's overall user experience, usage environment, and usability of the head-mounted wearable electronic device (300), it may be necessary for the sensations felt and experienced by the user in VR (virtual reality), AR (augmented reality), and MR (mixed reality) spaces to be as similar as possible to the sensations of the real world.

[0095] FIGS. 3B and FIGS. 3C are perspective views showing the structure of an electronic device according to one embodiment.

[0096] Referring to FIG. 3b and FIG. 3c, in one embodiment, camera modules (311, 312, 313, 314, 315, 316) and / or a depth sensor (317) for acquiring information related to the surrounding environment of an electronic device (300) (e.g., a wearable device) may be disposed on a first surface (310) of the housing.

[0097] In one embodiment, camera modules (311, 312) can acquire images related to the surrounding environment of a wearable electronic device.

[0098] In one embodiment, camera modules (313, 314, 315, 316) can acquire images while the electronic device (300) is worn by a user. Camera modules (313, 314, 315, 316) can be used for hand detection, tracking, and user gesture (e.g., hand movements) recognition. Camera modules (313, 314, 315, 316) can be used for 3DoF, 6DoF head tracking, position (space, environment) recognition, and / or movement recognition. In one embodiment, camera modules (311, 312) may be used for hand detection and tracking and user gestures.

[0099] In one embodiment, the depth sensor (317) may be configured to transmit a signal and receive a signal reflected from the subject, and may be used for determining the distance to the object, such as time of flight (TOF). In place of or additionally to the depth sensor (317), camera modules (313, 314, 315, 316) may determine the distance to the object.

[0100] According to one embodiment, a face recognition camera module (325, 326,) and / or a display (321) (and / or a lens) may be disposed on the second surface (320) of the housing.

[0101] In one embodiment, a face recognition camera module (325, 326) adjacent to the display may be used to recognize the user's face or to recognize and / or track both of the user's eyes.

[0102] In one embodiment, the display (321) (and / or lens) may be disposed on a second surface (320) of the wearable electronic device (300). In one embodiment, the wearable electronic device (300) may not include camera modules (315, 316) among a plurality of camera modules (313, 314, 315, 316). Although not illustrated in FIG. 3b and FIG. 3c, the electronic device (300) may further include at least one of the configurations illustrated in FIG. 2.

[0103] As described above, according to one embodiment, the electronic device (300) may have a form factor for being worn on a user's head. The electronic device (300) may further include a strap and / or a wearing member for being secured on a part of the user's body. The electronic device (300) may provide a user experience based on augmented reality, virtual reality, and / or mixed reality while being worn on the user's head.

[0104] FIG. 4 is a block diagram showing an example of the configuration of an electronic device according to one embodiment.

[0105] Referring to FIGS. 1, FIGS. 2, FIGS. 3a to 3c, and FIGS. 4, an electronic device (401) according to one embodiment may be a device capable of providing services related to virtual reality technology that provides a virtual environment similar to the electronic device (101) of FIG. 1, or the electronic device (200) of FIG. 2 and the electronic device (300) of FIG. 3. Virtual reality (VR) technology, which is a technology that provides a virtual environment, may be developed into augmented reality (AR), mixed reality (MR), and / or extended reality (XR) that encompasses these. According to one embodiment, virtual reality (VR) technology may be described in the sense that it includes augmented reality (AR), mixed reality (MR), and / or extended reality (XR). As shown in FIGS. 2 and FIGS. 3a to 3c, the electronic device (401) may be a device configured to be wearable on a user's body (e.g., a head-mounted display (HMD) or a glasses-type AR glasses device). For example, the electronic device (401) may be implemented in a form that combines an external electronic device, such as a mobile device, and may utilize components of the external electronic device (e.g., the electronic device (102 or 104) of FIG. 1) (e.g., a display module, a camera module, an audio output module, or other components). Not limited thereto, the electronic device (401) may be implemented as a user device, such as a smartphone, a tablet PC, or a PDA, or as various forms of wearable devices that can be worn on the user's body.

[0106] According to one embodiment, when the electronic device (401) is implemented as a device configured to be wearable on a user's body (e.g., a head-mounted display (HMD) or a glasses-type AR glasses device) as illustrated in FIGS. 2 and FIGS. 3a to 3c, the electronic device (401) implements a virtual reality space (e.g., an augmented reality space) that displays an augmented reality image corresponding to a real environment (e.g., a real space) captured in the surrounding environment where the user is located, or a virtual image provided (e.g., a two-dimensional or three-dimensional image), and can display at least one virtual object corresponding to the user and / or at least one virtual object corresponding to an object in the virtual reality space. According to one embodiment, when the electronic device (401) makes the real space of the real environment visible to the user through a transparent member, it can display at least one virtual object anchored to a screen corresponding to the real space.

[0107] An electronic device (401) according to one embodiment may include at least one processor (410) including a processing circuit, a memory (420), a display (430), a camera (440), and a communication circuit (450). Without being limited thereto, the electronic device (401) may be implemented identically or similarly to the electronic device (101) of FIG. 1, the electronic device (200) of FIG. 2, or the electronic device (300) of FIG. 3, and may further include other components of the electronic device (101) of FIG. 1, the electronic device (200) of FIG. 2, or the electronic device (300) of FIG. 3. In addition, the electronic device (401) may be formed to include other components necessary to provide augmented reality (AR), mixed reality (MR), and / or extended reality (XR) functions (e.g., services or methods).

[0108] According to one embodiment, the electronic device (401) may be an on-device device that may include an artificial intelligence (AI) model (e.g., a generative artificial intelligence model). According to one embodiment, the electronic device (401) may communicate with an external electronic device (e.g., the electronic device (102 or 104) of FIG. 1) or a server (e.g., the server (108) of FIG. 1) that includes the artificial intelligence model through a communication circuit (450).

[0109] According to one embodiment, the processor (410) of the electronic device (401) is electrically connected to other components and can control other components. The processor (410) can perform various data processing or operations in accordance with the execution of various functions (e.g., operations, services, or programs) provided by the electronic device (401). The processor (410) can perform various data processing or operations to display at least one virtual object related to an object or use identified in the real space or included in an image captured in the real space (e.g., a real environment) on the virtual reality space. The processor (410) can perform various data processing or operations to express user interaction or movement of the virtual object displayed in the virtual reality space.

[0110] According to one embodiment, the processor (410) may communicate with an external electronic device (403) (e.g., the electronic device (102 or 104) of FIG. 1) or a server (e.g., the server (108) of FIG. 1) through a communication circuit (450) using a wireless or wired communication method. According to one embodiment, the processor (410) may acquire an image (e.g., a 2D image or a 3D image) captured through a camera (e.g., the camera module (180) of FIG. 1) as user input information. According to one embodiment, the processor (410) may acquire voice information as user input information through a microphone (e.g., the input module (150) of FIG. 1). According to one embodiment, the processor (410) may acquire text information as user input information through a keyboard, mouse, or digital pen (e.g., a stylus pen) (e.g., the input module (150) of FIG. 1). According to one embodiment, the processor (410) can display the acquired image or text information in a virtual environment (e.g., virtual reality space) corresponding to the real space through a display (430).

[0111] FIG. 5 is a diagram illustrating an example of the configuration of an electronic device according to one embodiment, and FIG. 6a and FIG. 6b are diagrams illustrating examples of short-range wireless communication between an electronic device and external electronic devices according to one embodiment.

[0112] Referring to FIGS. 4, FIGS. 5, FIGS. 6a and FIGS. 6b, according to one embodiment, a processor (410) of an electronic device (401) can process operations for distributed processing of AI tasks through collaboration with external electronic devices (403, 405) located nearby in order to smoothly provide artificial intelligence (hereinafter referred to as AI) services.

[0113] According to one embodiment, a processor (410) receives user input information (501) for an AI task and can identify an AI task for the user input information. The processor (410) can analyze the user input information to obtain at least one keyword for the AI ​​task (e.g., confirm or identify). The user input information may include at least one of text, voice, or image.

[0114] According to one embodiment, the processor (410) may search for external electronic devices located in the surrounding environment to process operations for distributed processing of AI tasks. The processor (410) may connect with one or more external electronic devices (403 (403a, 403b, 403c) and 405) using a short-range wireless communication method (e.g., BLE, UWB, or Wi-Fi communication method) (hereinafter described as a second short-range wireless communication (601)). In one embodiment, when the processor (410) establishes a secure connection with external electronic devices, it may use location positioning technology to establish a secure connection only with nearby user devices (e.g., within a specified radius) as external electronic devices. Here, the external electronic devices may be devices capable of performing secure communication via short-range wireless communication. The processor (410) can identify (e.g., identify, determine, or select) at least one external electronic device (at least one of 403a, 403b, or 403c) among one or more connected external electronic devices (403, 405) that is capable of distributed processing of AI tasks while maintaining the security of on-device AI tasks. The processor (410) can identify an on-device device that includes an AI model (510) (e.g., a generative AI model) (e.g., capable of AI services) as an external electronic device capable of distributed processing of AI tasks.

[0115] According to one embodiment, the processor (410) can establish a connection for secure communication between the electronic device (401) and at least one external electronic device (at least one of 403a, 403b, or 403c) that is confirmed to be capable of distributed processing of AI tasks, through a communication circuit (450), using a short-range wireless communication for security (hereinafter described as the first short-range wireless communication (603)) within a specified range (610). The first short-range wireless communication (603) may be, for example, an ultra-wideband (UWB) communication method that performs secure communication using a specified security key. For example, the first short-range wireless communication (603) may be a communication method that can perform secure communication with the electronic device (401) and external electronic devices (403, 405) using positioning technology. The ultra-wideband (UWB) communication method utilizes a UWB security protocol (e.g., STS security protocol) to verify whether external electronic devices (403, 405) detected in the vicinity of the electronic device (401) are actually owned by the user. Since the electronic device (401) can maintain security when performing distributed processing of AI tasks by utilizing UWB and eSE rather than cloud-based, it can safely exchange information using UWB packets even when transmitting and receiving data related to AI services. According to one embodiment, the processor (410) can communicate with an authentication server (530) and a communication circuit (450) to perform authentication for secure communication.

[0116] According to one embodiment, the processor (410) may receive information (621, 622, 623) regarding resources related to the hardware capability (HW capability) (e.g., capability of an NPU or CPU and memory) of at least one external electronic device (at least one of 403a, 403b, or 403c) to determine the resources required (e.g., available) for distributed processing of AI tasks. According to one embodiment, the processor (410) may receive (e.g., obtain) AI training data (e.g., second AI training data, 'A' trained data, 'B' trained data, 'C' trained data) trained in at least one external electronic device (at least one of 403a, 403b, or 403c), from at least one external electronic device (at least one of 403a, 403b, or 403c) or from an AI metadatabase (520). The processor (410) may acquire AI training data learned from at least one external electronic device (at least one of 403a, 403b, or 403c) along with information related to resources (621, 622, 623). Here, the information related to available resources (621, 622, 623) of at least one external electronic device (at least one of 403a, 403b, or 403c) is information about various indicators that can represent the hardware (HW) and status of the device required to allocate AI tasks for distributed processing, and may include at least one of the capability of a process (e.g., CPU or NPU) and memory (e.g., hardware capability (HW capability)) or a heat level.

[0117] According to one embodiment, the processor (410) may obtain AI metadata required for an AI task as AI learning data (521) from an AI meta database (520) comprising AI learning data (e.g., first AI learning data) (521) and user model data (522) learned from an electronic device (401). The AI ​​meta database (520) may be contained in memory (420) or in an information providing server (not shown).

[0118] According to one embodiment, the processor (410) can identify a first task which is part of an AI task to be distributed and processed using an AI model (510) (e.g., a first AI model) of an electronic device (401) and a second task which is part of an AI task to be distributed and processed using an AI model (e.g., a second AI model) of at least one external electronic device (at least one of 403a, 403b, or 403c), based on information (621, 622, 623) related to available resources obtained from at least one external electronic device (403a, 403b, or 403c) and / or AI training data (e.g., a second AI training data).

[0119] According to one embodiment, the processor (410) can identify first resource information of an electronic device (401) available to process an AI task, acquire first AI training data (521) learned from the electronic device (401), and acquire second AI training data learned by at least one external electronic device (at least one of 403a, 403b, or 403c). The processor (410) can identify second resource information of at least one external electronic device (at least one of 403a, 403b, or 403c) based on information (621, 622, 623) related to resources available from at least one external electronic device (at least one of 403a, 403b, or 403c). For example, the processor (410) can identify a first task and a second task based on the first resource information, the second resource information, the first training data, and the second training data.

[0120] According to one embodiment, the processor (410) may assign an identified first task to an AI model (510) (e.g., a first AI model) and transmit a first AI command for the first task to the AI ​​model (510). According to one embodiment, the processor (410) may assign an identified second task to an AI model (e.g., a second AI model) of at least one external electronic device (at least one of 403a, 403b, or 403c) and transmit a second AI command for the second task to the AI ​​model (e.g., a second AI model).

[0121] According to one embodiment, the processor (410) may obtain first result information generated by the AI ​​model (510) based on a first AI command for a first task. According to one embodiment, the processor (410) may obtain second result information generated by the AI ​​model based on a second AI command for a second task. According to one embodiment, the processor (410) may obtain output information including the first result information and the second result information, and output the obtained output information. The output information may include at least one of text, voice, or image. For example, if the output information includes text and / or an image, the processor (410) may control the display (430) to display output information including text and / or image information. For example, if the output information of the processor (410) includes voice information, the processor (410) may control the speaker (440) to output output information including voice information.

[0122] According to one embodiment, the processor (410) can extract (e.g., acquire) at least one keyword of an AI task based on user input information (e.g., user command) and use the extracted keyword to compare the correlation (e.g., correlation value) for each hardware capability and / or AI metadata to determine which of at least one external electronic device (at least one of 403a, 403b, or 403c) will perform the AI ​​task. The processor (410) can determine a second task to be distributed to the determined external electronic device, generate a second AI command for the second task based on at least one keyword, and transmit the second AI command to at least one external electronic device (at least one of 403a, 403b, or 403c) via a communication circuit (450) using a first short-range wireless communication for security.

[0123] According to one embodiment, the processor (410) can search for one or more external electronic devices (403, 405) in the surrounding environment of the electronic device using a second short-range wireless communication (601). The processor (410) can identify (e.g., select, determine, or identify) at least one external electronic device (at least one of 403a, 403b, or 403c) among the one or more external electronic devices (403, 405) that is located within a designated security range and capable of first short-range wireless communication (603) for security. The processor (410) selects a first external electronic device (e.g., external electronic device (403a)) among at least one external electronic device whose correlation value is greater than a reference value and whose correlation value is the highest (e.g., highest priority), and based on information regarding the resources of the first external electronic device, checks whether there are sufficient available resources for the second task in the first external electronic device, and based on identifying that there are sufficient available resources for the second task in the first external electronic device, identifies the first external electronic device as a device to process the second task and assigns the second task to the first external electronic device.

[0124] According to one embodiment, the processor (410) may, based on identifying that there are insufficient available resources for the second task in the first external electronic device, re-select a second external electronic device in which the correlation is greater than a reference value and the correlation value is next higher (e.g., next higher priority), identify the second external electronic device as the device to handle the second task, and assign the second task to the second external electronic device.

[0125] According to one embodiment, the processor (410) checks whether there are sufficient available resources of the electronic device (401) to process the AI ​​task, and based on identifying that there are sufficient available resources of the electronic device (401), transmits a third AI command to an AI model (510) (e.g., a first AI model) to process all AI tasks (e.g., a first task and a second task) in the electronic device (401) without distributed processing by at least one external electronic device (at least one of 403a, 403b, or 403c), obtains third result information generated by processing all AI tasks using the AI ​​model (510), and can display the third result information through a display (430) or output it through a speaker (440). Here, the third result information may be at least one of text, voice, or image.

[0126] According to one embodiment, at least one processor (410) may include a central processing unit (CPU) or an application processor (AP), and may include a hardware structure (e.g., an AI chip) specialized for processing an artificial intelligence (AI) model (510) (e.g., a generative AI model). According to one embodiment, at least one processor (410) may perform overall control operations of an electronic device (401) and may perform operations to generate text for message composition using an AI model.

[0127] According to one embodiment, at least one processor (410) can execute instructions stored in memory (420) (e.g., memory (130) of FIG. 1) to implement a software module and can control hardware associated with the function of the software module.

[0128] A memory (420) according to one embodiment may store an AI model (510) (e.g., a generative AI model) and may store various data generated during program execution, including a program (e.g., software or the program (140) of FIG. 1) for distributing AI work on user input information using the AI ​​model (510). A memory (420) according to one embodiment may store instructions that cause at least one processor (410) to perform an operation (or method) for distributing AI work on user input information according to the present disclosure. According to one embodiment, a memory (420) of an electronic device (e.g., a memory (130) of FIG. 1) may store instructions (e.g., instructions) to implement a software module. According to one embodiment, a memory (420) of an electronic device (401) may store result information, output information, and various information related to AI work generated by the AI ​​model (510), and may store information related to the resources of the electronic device (401). The memory (420) may include an AI metadata database (520) containing AI training data learned using an AI model (510) (e.g., a generative AI model, an LLM model (large language model) or a multimodal model).

[0129] A software module of an electronic device according to one embodiment may be configured to include a kernel (or HAL), a framework (e.g., middleware (144) of FIG. 1), and an application (e.g., application (146) of FIG. 1). At least a portion of the software module may be preloaded onto the electronic device or downloadable from a server (e.g., server (108)). The application may include an application received from an external electronic device (e.g., server (108) or electronic device (102, 104)). According to one embodiment, the application may include a preloaded application or a third-party application downloadable from a server. The components of the software module and the names of the components according to the illustrated embodiment may vary depending on the type of operating system. According to one embodiment, at least a portion of the software module may be implemented as software, firmware, hardware, or a combination of at least two of these. At least a portion of the software module may be implemented (e.g., executed) by a processor (e.g., AP). At least a portion of the software module may include at least one of, for example, a module, a program, a routine, a set of instructions, or a process for performing at least one function. A software module of an electronic device according to one embodiment may include a prompt generating module that generates an input prompt for an AI command to be passed as input to an AI model. At least one processor (410) according to one embodiment may control a generative artificial intelligence model (510) that generates result information based on an AI command for distributed processing of an AI task, using the input prompt generated by the prompt generating module as input.

[0130] A display (430) according to one embodiment (e.g., the display (160) of FIG. 1) can display various information based on the control of at least one processor (410). A display (430) according to one embodiment can display a screen (e.g., an execution screen) associated with performing an operation (or method) for text generation using an AI model (510). According to one embodiment, the display (430) can be implemented in the form of a touch screen. When the display (430) is implemented in the form of a touch screen together with an input module, it can display various information generated according to the user's touch operation.

[0131] A communication circuit (450) according to one embodiment (e.g., the communication module (190) of FIG. 1) may include a wireless communication module (e.g., a cellular module, a Wi-Fi (wireless-fidelity) module, a Bluetooth module, a UWB module, or a NFC (near field communication) module). A communication circuit (450) according to one embodiment may communicate with an external electronic device including an on-device AI model (e.g., the external electronic device (403: 403a, 403b, 403c) of FIG. 5, FIG. 6a, FIG. 6b, FIG. 7a, FIG. 7b, FIG. 1, FIG. 8a, FIG. 1, FIG. 8b, FIG. 2, FIG. 8b, FIG. 3, FIG. 8c, FIG. 8c, FIG. 8c). A communication circuit (450) according to one embodiment may transmit an AI command (input prompt) to an external electronic device based on the control of a processor (410). A communication circuit (450) according to one embodiment can receive result information generated based on an AI command (input prompt) using an AI model from an external electronic device.

[0132] FIGS. 7a and 7b are drawings illustrating an example of distributed processing of artificial intelligence tasks with an external electronic device in an electronic device according to one embodiment. FIGS. 8a, 8b, and 8c are drawings illustrating an example of distributed processing of artificial intelligence tasks with an external electronic device in an electronic device according to one embodiment.

[0133] Referring to FIG. 7a, an electronic device (401) according to one embodiment can search for a plurality of external electronic devices (403a, 403b, 403c, and 405) via a second short-range wireless communication within a designated range (610). For example, the operation of searching for external electronic devices may be performed when a user uses (e.g., wears) the electronic device (401) or when user input for an AI task is received from the user. Not limited thereto, the operation of searching for external electronic devices may be performed when an event requiring communication with an external electronic device occurs or by a user request.

[0134] According to one embodiment, an electronic device (401) may receive user input information for an AI task from a user. The electronic device (401) may analyze the user input information to obtain keywords necessary for the AI ​​task. According to one embodiment, as illustrated in FIG. 7a, for example, if the user input information is an image (710) of a kitchen where a mother is standing, the electronic device (4011) may analyze the objects included in the image (710) (e.g., a first object (711), a second object (712), a third object (713)) to obtain key keywords such as “mother” corresponding to the first object (711), “kitchen” corresponding to the second object (712), and “design” corresponding to the third object (713). The user input information may also be received as voice or text information, such as “design the kitchen to suit Mom’s taste,” in addition to the image. According to one embodiment, the electronic device (401) can obtain a first keyword (e.g., “novel” or “photo”) (721), a second keyword (e.g., “daughter”) (722), or a third keyword (e.g., “last trip”) (723), for example, when user input information (720) is voice or text information such as “Can you make a novel or photo of the last trip for my daughter?” as illustrated in FIG. 7b.

[0135] According to one embodiment, the electronic device (401) can determine whether AI work (e.g., AI service) is possible among a plurality of discovered external electronic devices (403a, 403b, 403c and 405), and can select (e.g., confirm) the external electronic devices (403a, 403b, 403c) capable of AI work. The electronic device (401) can establish a secure communication connection with the external electronic devices (403a, 403b, 403c) through a first short-range wireless communication. As illustrated in FIGS. 8A, 8B, and 8C, the electronic device (401) may receive information related to resources (e.g., a hardware capability table) (810a, 810b, 810c) and / or AI training data (e.g., a pre-trained AI data set) (820a, 810b, 810c) from external electronic devices (403a, 403b, 403c) connected via secure communication. For example, the information related to resources (e.g., a hardware capability table) (810) may include at least one of the following data: processor processing level, processor capability, processor occupancy, memory occupancy, memory allocation, or heat generation level (e.g., current heat dissipation budget) of each external electronic device (403a, 403b, 403c). For example, AI training data (e.g., pre-trained AI data set) (820) may include AI data that has been pre-trained for each external electronic device (403a, 403b, 403c). The first external electronic device (403a) is trained with content called 'A', where 'A' may be a specific keyword or various information such as a specific place or environment. The second external electronic device (403b) is trained with content called 'B', where 'B' may be a specific keyword or various information such as a specific place or environment.The third external electronic device (403c) is trained with the content of 'C', which may be a specific keyword or various information such as a specific place or environment.

[0136] According to one embodiment, as illustrated in FIG. 7a, an electronic device (401) can determine a correlation value (e.g., correlation or priority) by comparing acquired keywords (e.g., “mom,” “kitchen,” and “design”) with AI training data received from each external electronic device (403a, 403b, and 403c). The electronic device (401) can determine that the correlation value of each of the external electronic devices (403a, 403b, and 403c) is greater than a specified reference value. For example, the electronic device (401) can identify the first external electronic device (403a) among the external electronic devices (403a, 403b, and 403c) as a device (e.g., mom's mobile phone) that has training data with a high correlation to the keyword “mom.” For example, the electronic device (401) can be identified as a device (e.g., an AI appliance such as a refrigerator or vacuum cleaner) in which the second external electronic device (403b) among the external electronic devices (403a, 403b, and 403c) has training data that is highly correlated with the keyword “kitchen”. For example, the electronic device (401) can be identified as a device (e.g., a laptop) in which the third external electronic device (403c) among the external electronic devices (403a, 403b, and 403c) has training data that is highly correlated with the keyword “design”. Here, an external electronic device in which the correlation value is greater than a specified reference value may be in a state where the resources (e.g., NPU and memory) of the external electronic device are abundant and heat generation is low.

[0137] According to one embodiment, as illustrated in FIG. 7b, an electronic device (401) can determine a correlation value (e.g., correlation or priority) by comparing acquired keywords (721, 722, 723) (e.g., “novel,” “photo,” “daughter,” or “last trip”) with AI training data (731, 732, 733) received from each external electronic device (403a, 403b, and 403c). The electronic device (4011) can determine that the correlation value of each of the external electronic devices (403a, 403b, and 403c) is greater than a specified reference value. For example, the electronic device (401) can be identified as a device (e.g., a tablet) in which the first external electronic device (403a) among the external electronic devices (403a, 403b, and 403c) has learning data that is highly correlated with the first keyword (721) for “novel” or “photo”. For example, the electronic device (401) can be identified as a device (e.g., a daughter’s mobile phone or a family’s mobile phone) in which the second external electronic device (403b) among the external electronic devices (403a, 403b, and 403c) has learning data that is highly correlated with the second keyword (722) for “daughter”. For example, the electronic device (401) can be identified as a device (e.g., a laptop) in which the third external electronic device (403c) among the external electronic devices (403a, 403b and 403c) has learning data that is highly correlated with the third keyword (723) for “past trip”.

[0138] According to one embodiment, the electronic device (401) can determine whether each external electronic device (403a, 403b, 403c) currently has sufficient available resources (e.g., whether the correlation is higher than a specified value) based on information (621, 622, 623) related to resources received from each external electronic device (403a, 403b, 403c) and / or AI learning data (731, 732, 733). When an electronic device (401) determines that at least one of the external electronic devices (403a, 403b, 403c) has sufficient available resources (e.g., has a correlation higher than a specified value), it can determine the tasks to be distributed for AI tasks (e.g., determine the level of computation, the field of computation, or the amount of computation) based on information (621, 622, or 623) related to the resources of at least one external electronic device (403a, 403b, or 403c) that has sufficient available resources and / or AI training data (731, 732, or 733), and assign (e.g., distribute, or share) the determined tasks to at least one external electronic device (403a, 403b, or 403c) that has sufficient available resources. According to one embodiment, the electronic device (401) may, for example, assign a task related to a keyword (e.g., “Mom”) for a first object (711) as part of an AI task to a first external electronic device (403a) as part of an AI task, assign a task related to a keyword (e.g., “Kitchen”) for a second object (712) as part of an AI task to a second external electronic device (403b) as part of an AI task, and assign a task related to a keyword (e.g., “Design”) for a third object (713) as part of an AI task to a third external electronic device (403c) as part of an AI task.According to one embodiment, the electronic device (401) may, for example, assign a task related to a first keyword (721) for “novel” and / or “photo” as part of an AI task to a first external electronic device (403a) as part of an AI task, assign a task related to a second keyword (722) for “daughter” as part of an AI task to a second external electronic device (403b), and assign a task related to a third keyword (723) for “past trip” as part of an AI task to a third external electronic device (403c), as shown in FIG. 7b. In this way, the electronic device (401) can perform higher accuracy and faster processing by utilizing the processors and memories of each external electronic device (403a, 403b and 403c) that have a high correlation, and allocating a portion of the AI ​​processing to distributed processing of the better results, learned AI data, models, and external electronic devices (403a, 403b or 403c), thereby reducing the load and improving performance.

[0139] According to one embodiment, the electronic device (401) can transmit AI commands for tasks (e.g., part of an AI task) assigned to each external electronic device (403a, 403b, and 403c) to each external electronic device (403a, 403b, and 403c) via a first short-range wireless communication for security. According to one embodiment, the electronic device (401) can receive result information generated from each external electronic device (403a, 403b, and 403c) via the first short-range wireless communication for security, and combine the result information of each external electronic device (403a, 403b, and 403c) (e.g., second result information) and the result information generated by the AI ​​model (510) of the electronic device (401) (e.g., first result information) to generate final output information (503) (e.g., text information, voice information, 2D image, or 3D image).

[0140] According to one embodiment, the electronic device (401) may obtain result information (e.g., text information, voice information, 2D image or 3D image) generated by performing a task related to a keyword for “mom” by an AI model of the first external electronic device (403a) as shown in FIG. 7a, obtain result information (e.g., text information, voice information, 2D image or 3D image) generated by performing a task related to a keyword for “kitchen” by an AI model of the second external electronic device (403b), and obtain result information (e.g., text information, voice information, 2D image or 3D image) generated by performing a task related to a keyword for “design” by an AI model of the third external electronic device (403c). According to one embodiment, the electronic device (401) may combine result information (e.g., first result information) generated by the AI ​​model (510) of the electronic device (401) and result information (e.g., second result information) of each external electronic device (403a, 403b, and 403c) to generate output information including, for example, an image of a kitchen design that suits Mom's taste or information about a description of the design. The result information (e.g., first result information) generated by the AI ​​model (510) of the electronic device (401) may be information generated by processing, for example, a task on main keywords (e.g., “Mom,” “Kitchen,” “Design”) and another keyword (e.g., “Furniture”) or a task related to at least some of the main keywords (e.g., a task different from the task assigned to each external electronic device (403a, 403b, and 403c)).

[0141] According to one embodiment, the electronic device (401) may obtain result information (741) (e.g., text information, voice information, 2D image or 3D image) generated by performing an operation related to a first keyword (721) for “photo” and / or “novel” by an AI model of a first external electronic device (403a), as shown in FIG. 7b, obtain result information (742) (e.g., text information, voice information, 2D image or 3D image) generated by performing an operation related to a second keyword (722) for “daughter” by an AI model of a second external electronic device (403b), and obtain result information (743) (e.g., text information, voice information, 2D image or 3D image) generated by performing an operation related to a third keyword (723) for “past trip” by an AI model of a third external electronic device (403c). According to one embodiment, the electronic device (401) may combine result information (e.g., first result information) generated by the AI ​​model (510) of the electronic device (401) and / or result information (e.g., second result information) (741, 742, 743) of each external electronic device (403a, 403b and 403c) to generate output information (740) including, for example, a novel or photos of a past trip for a daughter. The result information (e.g., first result information) generated by the AI ​​model (510) of the electronic device (401) may be information generated by processing, for example, a task on major keywords (721, 722, 723) (e.g., “photo”, “novel”, “daughter”, “past trip”) and another keyword (e.g., “sea”) or a task related to at least some of the major keywords (e.g., a task different from the task assigned to each external electronic device (403a, 403b and 403c)).

[0142] In this way, the electronic device (401) can perform higher accuracy and faster processing by utilizing the processors and memories of each external electronic device (403a, 403b, and 403c) that have a high correlation, and allocating a portion of the AI ​​processing to be distributed among the external electronic devices (403a, 403b, and 403c), thereby reducing the load and improving performance. The electronic device can perform the final AI service while compensating for insufficient HW performance and maintaining sensitive AI information through secure communication.

[0143] Operations for distributed processing of AI tasks in at least one external electronic device discovered in the electronic device (401) of the present disclosure and the surrounding environment can be utilized in various services such as inter-vehicle services, home IoT services, or services provided in an office environment.

[0144] In this way, the main components of an electronic device have been described through the electronic device described in the above-described embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (200) of FIG. 2, the electronic device (300) of FIG. 3a and FIG. 3c, and the electronic device (401) of FIG. 4). However, in various embodiments, not all components illustrated in FIG. 1 to 4 are essential components, and the electronic device may be implemented with more components than those illustrated, or with fewer components. Additionally, the location of the main components of the electronic device described in FIG. 1 to 4 may be changed according to various embodiments.

[0145] FIG. 9 is a diagram illustrating an example of an operation method in an electronic device according to one embodiment, and FIG. 10 is a diagram illustrating an example of secure communication between an electronic device and external electronic devices according to one embodiment. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0146] Referring to FIGS. 9 and 10, in operation 901, an electronic device (401) according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3a to 3c and electronic device (401) of FIG. 4) receives user input information (e.g., user input information (501) of FIG. 5) for an artificial intelligence (hereinafter referred to as AI) task and can identify an AI task for the user input information. The user input information may include at least one of text, voice, or image.

[0147] In operation 903, an electronic device (401) according to one embodiment searches for external electronic devices located in a surrounding environment and can identify (e.g., select or determine) at least one external electronic device (e.g., at least one of the first external electronic device (403a), second external electronic device (403b), ..., Nth external electronic device (403n) of FIG. 10) that is located within a specified range and capable of performing AI tasks (e.g., AI services) while maintaining security. The electronic device (401) can establish a connection for secure communication with the identified at least one external electronic device through short-range wireless communication for security (hereinafter described as the first short-range wireless communication). For example, when the electronic device (401) establishes a secure connection with external electronic devices, it may use location positioning technology to establish a secure connection only with nearby user devices (e.g., within a specified radius) as external electronic devices. Here, the external electronic devices may be devices capable of performing secure communication through short-range wireless communication. The electronic device (401) can discover external electronic devices using a short-range wireless communication method (e.g., BLE, UWB, or Wi-Fi communication method) (hereinafter described as the second short-range wireless communication). The first short-range wireless communication may be, for example, an ultra-wideband (UWB) communication method that performs secure communication using a designated security key. The ultra-wideband (UWB) communication method can verify whether external electronic devices discovered in the vicinity of the electronic device (401) are actually owned by the user by utilizing a UWB security protocol (e.g., STS (scrambled time stamp) security protocol). Since the electronic device (401) can maintain security when performing distributed processing of AI tasks by utilizing UWB and eSE rather than cloud-based, it can safely exchange information using UWB packets even when transmitting and receiving data related to AI services.As illustrated in FIG. 10, according to one embodiment, an electronic device (401) can transmit an information request message (e.g., request AI thread) (1001) via secure communication to at least one external electronic device (e.g., at least one of the first external electronic device (403a), second external electronic device (403b), …, and Nth external electronic device (403n) of FIG. 10).

[0148] In operation 905, an electronic device (401) according to one embodiment may receive information related to resources regarding the capabilities of the processes (e.g., CPU or NPU) and memory (e.g., hardware capability) of at least one external electronic device from at least one external electronic device to identify resources required (e.g., available) for distributed processing of AI tasks. According to one embodiment, the electronic device (401) may obtain AI training data (e.g., second AI training data, metadata) learned from at least one external electronic device from an information providing server including at least one external electronic device or an AI metadata database. For example, the electronic device (401) may obtain AI training data learned from at least one external electronic device together with the information related to resources when obtaining information related to resources. As illustrated in FIG. 10, according to one embodiment, an electronic device (401) may receive, via secure communication, an information providing message (e.g., read for AI thread) (1003) containing information related to resources and / or AI learning data from at least one external electronic device (e.g., at least one of the first external electronic device (403a), the second external electronic device (403b), ..., the Nth external electronic device (403n) of FIG. 10). Here, the information related to resources of at least one external electronic device is information necessary to assign AI tasks to be distributed, and may include at least one of a process (e.g., CPU or NPU) and memory capability (e.g., hardware capability (HW capability)) or heat level. According to one embodiment, the electronic device (401) may obtain AI learning data learned by the electronic device (401) (e.g., first learning data) and information related to resources of the electronic device (401) (e.g., first resource information) from a memory (420) or an information providing server (not shown).

[0149] In operation 907, an electronic device (401) according to one embodiment can identify a first task which is part of an AI task to be distributed and / or a second task which is part of an AI task to be distributed and / or an AI model of at least one external electronic device (hereinafter described as the second AI model), based on information related to available resources obtained from at least one external electronic device (e.g., second resource information) and / or AI training data (e.g., second AI training data). According to one embodiment, the electronic device (401) can identify the first task and / or the second task based on the first resource information, the second resource information, the first training data, and the second training data. According to one embodiment, the electronic device (401) may assign the identified first task to a first AI model and assign the identified second task to an AI model (e.g., a second AI model) of at least one external electronic device (e.g., a first external electronic device (403a) and a Nth electronic device (403n)). As illustrated in FIG. 10, according to one embodiment, the electronic device (401) may assign the identified second task to an AI model (e.g., a second AI model) of the first external electronic device (403a) and the Nth electronic device (403n) and transmit an AI service request message (e.g., a multicast request AI service (1005)) containing a second AI command for the second task to the first external electronic device (403a) and the Nth electronic device (403n).

[0150] In operation 909, an electronic device (401) according to one embodiment can obtain first result information generated by the first AI model based on a first AI command for a first task transmitted to the first AI model.

[0151] In operation 911, an electronic device (401) according to one embodiment may obtain second result information generated by a second AI model from at least one external electronic device based on a second AI command for a second task transmitted to the second AI model. As illustrated in FIG. 10, according to one embodiment, the electronic device (401) may transmit an AI service request message (e.g., multicast request AI service) (1005) containing a second AI command to at least one external electronic device (e.g., at least one of the first external electronic device (403a), the second external electronic device (403b), ..., the Nth external electronic device (403n) of FIG. 10) via secure communication. Subsequently, the electronic device (401) can receive a response message (e.g., response) (1007) containing second result information from at least one external electronic device (e.g., at least one of the first external electronic device (403a), second external electronic device (403b), …, and Nth external electronic device (403n) of FIG. 10) via secure communication.

[0152] 913 In operation, an electronic device according to one embodiment may acquire (e.g., generate) output information including acquired first result information and second result information.

[0153] In operation 915, electronic device output information according to one embodiment may be output (e.g., provided). The output information may include at least one of text, voice, or image (e.g., 2D image or 3D image). For example, if the output information includes text and / or image, the electronic device may control a display (e.g., display module (160) of FIG. 1, screen display portion (254-1, 254-2) of FIG. 2, display member (340) of FIG. 3a, display (321) of FIG. 3b, or display (430) of FIG. 4) to display output information including text and / or image information. For example, if the output information of the processor (410) includes voice information, the electronic device may control a speaker (e.g., sound output module (155) of FIG. 1, or speaker (440) of FIG. 4) to output output information including voice information.

[0154] In the operation 907 of FIG. 9 described above, an electronic device according to one embodiment can obtain at least one keyword of an AI task based on user input information. The electronic device can check the correlation of the electronic device (e.g., a first correlation value) by comparing the first AI training data with at least one keyword. The electronic device can check the correlation of at least one external electronic device (e.g., a second correlation value) by comparing the second AI training data with at least one keyword. According to one embodiment, based on the correlation of the electronic device (e.g., a first correlation value), the electronic device can identify and assign a first task to be distributed and processed by the electronic device, and transmit a first AI command (e.g., a first input prompt) for the first task generated based on at least one keyword to the first AI model. According to one embodiment, the electronic device may identify and assign a second task to be distributed and processed by at least one external electronic device based on the correlation (e.g., a second correlation value) of at least one external electronic device, generate a second AI command (e.g., a second input prompt) for the second task based on at least one keyword, and transmit the second AI command to at least one external electronic device using a first short-range wireless communication for security. The electronic device may set priorities for AI tasks based on a first correlation value and at least one second correlation value for at least one keyword. For example, if the first correlation value of a specific keyword is greater than a reference value and greater than at least one second correlation value, the electronic device may identify a task related to the specific keyword as a first task that can be processed by the first AI model of the electronic device and assign the identified first task to the first AI model of the electronic device.For example, if the first correlation value of a specific keyword is greater than a reference value and is less than or equal to at least one second correlation value, the electronic device may identify a task related to the specific keyword as a second task that can be processed by a second AI model of at least one external electronic device (e.g., the electronic device having the highest correlation value among multiple external electronic devices) and assign the identified second task to the second AI model of at least one electronic device. For example, if the first correlation value of a specific keyword is smaller than the reference value, the electronic device may determine that there is insufficient AI training data for the specific keyword and, instead of processing the AI ​​task related to the specific keyword in the first AI model of the electronic device, transmit an AI command for the specific keyword to an external electronic device with a high correlation value for the specific keyword. For example, if the first correlation value for all identified keywords is smaller than the reference value or if the electronic device lacks resources, the electronic device may not process the AI ​​task in the first AI model of the electronic device and may process the AI ​​task only in at least one external electronic device connected via secure communication within a specified range, or distribute the processing of the AI ​​task among multiple external electronic devices connected via secure communication within a specified range.

[0155] According to one embodiment, if the resources available to the electronic device are sufficient to process the AI ​​task and the AI ​​task is a task that can be processed quickly without delay (within a specified time), the electronic device may allocate the entire AI task to a first AI model of the electronic device without distributed processing by at least one external electronic device, and obtain result information (e.g., third result information) generated based on an AI command (e.g., third AI command) from the first AI model. The electronic device may display the third result information through the display of the electronic device or output it through the speaker of the electronic device. Here, the third result information may be at least one of text, voice, or image.

[0156] In the operation of FIG. 907 described above, when a plurality of external electronic devices located within a designated security range are detected, the electronic device according to one embodiment selects a first external electronic device among the plurality of external electronic devices that has a correlation value greater than a reference value and has the highest correlation value (e.g., highest priority), and can determine whether there are sufficient available resources for a second task in the first external electronic device based on information related to the resources of the first external electronic device. Based on identifying that there are sufficient available resources for the second task in the first external electronic device, the electronic device can identify the first external electronic device as a device to process the second task and assign the second task to the first external electronic device. According to one embodiment, based on identifying that there are insufficient available resources for the second task in the first external electronic device, the electronic device can again select a second external electronic device among the plurality of external electronic devices that has a correlation value greater than a reference value and has the next highest priority correlation value, identify the again selected second external electronic device as a device to process the second task, and assign the second task to the second external electronic device.

[0157] FIG. 11 is a diagram illustrating an example of the operation method of an electronic device and an external electronic device according to one embodiment. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0158] Referring to FIG. 11, in operation 1101, an electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3a to 3c and electronic device (401) of FIG. 4) receives user input information (e.g., user input information (501) of FIG. 5) for an artificial intelligence (hereinafter referred to as AI) task, and analyzes the user input information to obtain at least one keyword for the AI ​​task (e.g., confirm, identify, or determine). The user input information may include at least one of text, voice, or image.

[0159] In operation 1103, an electronic device according to one embodiment may search for external electronic devices located in a surrounding environment and identify (e.g., select or determine) at least one external electronic device among the external electronic devices that is located within a specified range and capable of performing AI tasks (e.g., AI services) while maintaining security (e.g., at least one external electronic device of FIG. 4 (e.g., at least one external electronic device of FIG. 6a and FIG. 6b (at least one of 403a, 403b, or 403c)). The electronic device may establish a connection for secure communication with the identified at least one external electronic device through short-range wireless communication for security (hereinafter described as the first short-range wireless communication). The electronic device may search for external electronic devices using a short-range wireless communication method (e.g., BLE, UWB, or Wi-Fi communication method) (hereinafter described as the second short-range wireless communication). The first short-range wireless communication may be, for example, an ultra-wideband (UWB) communication method that performs secure communication using a specified security key. Ultra-Wideband (UWB) communication methods utilize UWB security protocols (e.g., STS security protocol) to verify whether external electronic devices detected in the vicinity of an electronic device actually belong to the user. By utilizing UWB and eSE rather than a cloud-based approach, electronic devices can maintain security when performing distributed processing of AI tasks; consequently, information can be securely exchanged using UWB packets even during the transmission and reception of data related to AI services.

[0160] In operation 1105, an electronic device according to one embodiment may receive from at least one external electronic device information related to resources regarding the capabilities of the process (e.g., CPU or NPU) and memory (e.g., hardware capability) of at least one external electronic device to determine the resources required (e.g., available) for distributed processing of AI tasks, and AI training data (e.g., second AI training data, or metadata) learned from at least one external electronic device. According to one embodiment, the electronic device may obtain AI training data (e.g., second AI training data, or metadata) learned from at least one external electronic device from an information providing server including an AI meta database (e.g., AI meta database (520) of FIG. 5). An electronic device according to one embodiment may obtain AI training data learned from the electronic device (e.g., first training data) and information related to the resources of the electronic device (e.g., first resource information) contained in memory (e.g., memory (420) of FIG. 4) or from an information providing server (not shown). Here, information related to the resources of at least one external electronic device is information necessary to allocate AI tasks for distributed processing, and may include at least one of the capabilities of a process (e.g., CPU or NPU) and memory (e.g., hardware capability (HW capability)) or a heat level.

[0161] In operation 1107, an electronic device according to one embodiment can determine (e.g., obtain, identify, or determine) the correlation value of the electronic device and the correlation value of at least one external electronic device by comparing information related to the resources of the electronic device (e.g., first resource information) and information related to the resources of at least one external electronic device (e.g., second resource information) with at least one keyword. Based on the determined correlation values, the electronic device can set the priority of each device (electronic device and at least one external electronic device).

[0162] 1109 In operation, an electronic device according to one embodiment may select the device with the highest priority (e.g., an electronic device or an external electronic device) to check available resources in the order of set priorities.

[0163] In operation 1111, an electronic device according to one embodiment can check whether there are sufficient resources available in the selected device. If, as a result of the check, there are sufficient resources available in the selected device, the electronic device can perform operation 1115. On the other hand, if there are insufficient resources available in the checked device, in operation 1113, another device with a higher priority in the next order can be selected again, and then operation 1111 can be performed again.

[0164] In operation 1115, an electronic device according to one embodiment may assign part or all of the AI ​​task to a selected device and transmit an AI command to the AI ​​model (e.g., a second AI model) of the selected device (e.g., the first external electronic device (403a) and the Nth external electronic device (403n) of FIG. 10) to process part or all of the assigned AI task. According to one embodiment, if there are multiple identified keywords, the electronic device may check the correlation value for each keyword and set priorities for each keyword to enable fast AI task processing for each keyword. The electronic device may repeat operations 1109 and 1111 for each keyword, and when the verification of all keywords is complete, perform operation 1115 and assign tasks for each keyword to the selected devices respectively so as to distribute the AI ​​task processing for each keyword.

[0165] In operation 1117, an electronic device according to one embodiment may acquire result information generated by an AI model of a selected device and output output information including the acquired result information. Here, the result information may be information corresponding to a task assigned in relation to a specific keyword (e.g., part or all of an AI task). According to one embodiment, when the electronic device receives result information distributedly processed by a plurality of devices, it may combine the received result information to generate output information and output the generated output information. Here, the output information may include at least one of text, voice, or image (e.g., 2D image or 3D image). For example, if the output information includes text and / or image, the electronic device may control a display (e.g., display module (160) of FIG. 1, screen display part (254-1, 254-2) of FIG. 2, display member (340) of FIG. 3a, display (321) of FIG. 3b, or display (430) of FIG. 4) to display output information including text and / or image information. For example, if the output information of the processor (410) includes voice information, the speaker (e.g., the acoustic output module (155) of FIG. 1, or the speaker (440) of FIG. 4) can be controlled to output the output information including voice information.

[0166] FIG. 12 is a diagram illustrating a generative artificial intelligence system according to one embodiment.

[0167] In a generative artificial intelligence system (1200) according to one embodiment, a user query / response interface (1210) (e.g., the input module of FIG. 1 or the display module (160) of FIG. 1, the display (430) of FIG. 4) can receive user input. The user input may be in the form of natural language, images and / or videos, but is not limited to any other form. Additionally, context information may be transmitted along with the user input. The context information may include various additional information at the time of user input. For example, the additional information may include information about the application currently being used by the user or the user's location information. Furthermore, the user input may be in a mixed form of the aforementioned natural language, images, sounds, or context information. Additionally, the user input may be in a non-natural language form, such as selecting a menu. The user query / response interface (1210) can output the results of the generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of an action requested by the user. The user query / response interface (1210) can output the results of the generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of an action requested by the user.

[0168] An AI framework (1240) (e.g., the processor (120) of FIG. 1 or the processor (410) of FIG. 4) can receive input from a user and coordinate and control each component necessary to perform the user's intent based on the user's query.

[0169] User input received from the user query / response interface (1210) can be transmitted to a prompt design component (1241) (e.g., the processor (120) of FIG. 1, or the processor (410) of FIG. 4). The prompt design component (1241) can be used to generate prompts suitable for inputting user input into a large language model (LLM) or a large multimodal model (LMM). The prompt design component (1241) may be an AI component that uses machine learning algorithms or neural networks to develop better prompts over time. The prompt design component (1241) can generate prompts by accessing a knowledge component containing user preference data, a prompt library, and prompt examples based on user input, and can transmit the generated prompts to the LLM or LMM.

[0170] An API / Plug-in management component (1242) (e.g., the processor (120) of FIG. 1, or the processor (410) of FIG. 4) can perform the role of communicating with external information when there is a request for additional information when user input is passed as input to a generative model (e.g., the AI ​​model (510) of FIG. 5 or a cloud AI model). The API / Plug-in management component (942) establishes a channel to communicate with the outside of the AI ​​interface through the API, and through the established channel, it can enable access to various data sources (e.g., a knowledge repository (1220)) (e.g., the memory (130) of FIG. 1, the memory (420) of FIG. 4). Additionally, the API / plugin management component (1242) may request the application / service component (1230) (e.g., the processor (120) of FIG. 1, the processor (410) of FIG. 4) via the API when the application or service needs to perform an action that ultimately performs user input rather than an intermediate result. Information obtained from the outside may be used to generate a prompt in the prompt design component (1241) along with user input, or it may be passed as input to a generative model (e.g., the AI ​​model (510) of FIG. 5 or a cloud AI model).

[0171] An output modification component (or refiner component) (1243) (e.g., processor (120) of FIG. 1) can finely tune the output of a generative model (e.g., AI model (510) of FIG. 5 or cloud AI model). For example, the output modification component (1243) can verify whether the content generated through LLM and / or LMM is irrelevant, contains biased content, or contains harmful content. Additionally, the output modification component (1243) can determine the extent to which the output matches the desired result and, if additional processing is required, proceed with that process. Furthermore, the output modification component (1243) can configure and provide hints to the user to avoid unwanted output.

[0172] A generative AI model (1260) (e.g., the AI ​​model (510) of FIG. 5 or a cloud AI model) can generally refer to an artificial intelligence neural network that generates new forms of data based on user input information. A generative AI model (1260) may include a model that generates images and / or a model that generates language. Models that generate images include, but are not limited to, a generative adversarial network (GAN) or a variational autoencoder (VAE), and examples include a Diffusion-based generative model using a VAE and a Transformer structure. Models that generate language are models trained to output the most statistically appropriate output value based on input values, and examples include models such as CHAT-GPT 3 or CHAT-GPT 4. Additionally, there are large multimodal models (LMMs) that can recognize various forms of data input, such as text, images, or voice, and generate new data corresponding to them.

[0173] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3a and FIG. 3c and electronic device (401) of FIG. 4) may include a communication circuit (e.g., communication module (190) of FIG. 1, communication circuit (450) of FIG. 4), at least one processor including a processing circuit (e.g., processor (120) of FIG. 1, processor (410) of FIG. 4), and a memory for storing instructions (e.g., memory (130) of FIG. 1, memory (420) of FIG. 4).

[0174] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may identify an artificial intelligence task for the user input information based on receiving the user input information (e.g., user input information (501) of FIG. 5).

[0175] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be able to establish a communication connection between the electronic device and at least one external electronic device for distributed processing of the artificial intelligence task (e.g., external electronic device (403: 403a, 403b, 403c) of FIG. 5, FIG. 6a, FIG. 6b, FIG. 7a, FIG. 7b, first external electronic device (403a) of FIG. 8a, second external electronic device (403b) of FIG. 8b, and third external electronic device (403c) of FIG. 8c) through the communication circuit using a first short-range wireless communication for security.

[0176] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may obtain information related to resources available for distributed processing of the artificial intelligence task and artificial intelligence learning data from the at least one external electronic device.

[0177] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may identify a first task which is part of the artificial intelligence task to be distributed and processed using the first artificial intelligence model of the electronic device (e.g., the artificial intelligence model (510) of FIG. 5) and a second task which is part of the artificial intelligence task to be distributed and processed using the second artificial intelligence model of the at least one external electronic device, based on information related to the available resources and artificial intelligence learning data obtained from the at least one external electronic device.

[0178] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may obtain first result information generated by the first artificial intelligence model based on a first artificial intelligence command for the first task transmitted to the first artificial intelligence model.

[0179] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be able to receive second result information generated by the second artificial intelligence model based on the second artificial intelligence command for the second task transmitted to the at least one external electronic device through the communication circuit using the first short-range wireless communication for security.

[0180] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device can obtain output information (e.g., output information (503) of FIG. 5) including the first result information and the second result information.

[0181] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be configured to obtain at least one keyword for the artificial intelligence task based on the user input information, check the correlation value between the artificial intelligence learning data and the at least one keyword, assign the second task for the at least one keyword to the at least one external electronic device based on the correlation value being greater than a reference value, and transmit the second artificial intelligence command to the at least one external electronic device through the communication circuit using the first short-range wireless communication for security, which requests the processing of the second task for the at least one keyword by the second artificial intelligence model of the at least one external electronic device.

[0182] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be configured to determine whether there is sufficient available resource in the at least one external electronic device based on information related to the available resource obtained from the at least one external electronic device, and to transmit the AI ​​command requesting the processing of the second task to the at least one external electronic device based on the determination that there is sufficient available resource in the at least one external electronic device.

[0183] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device transmits an AI command requesting the first artificial intelligence model of the electronic device to process both the first task and the second task based on the finding that the available resources in the at least one external electronic device are insufficient.

[0184] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device selects the first external electronic device with the largest correlation value among the plurality of external electronic devices based on the connection of a plurality of external electronic devices using the first short-range wireless communication to distribute the artificial intelligence task, checks whether there are sufficient available resources for the second task in the first external electronic device based on information related to the resources of the first external electronic device, assigns the second task to the first external electronic device based on identifying that there are sufficient available resources for the second task in the first external electronic device, selects the second external electronic device with the next largest correlation value among the plurality of external electronic devices based on identifying that there are insufficient available resources for the second task in the first external electronic device, and assigns the second task to the second external electronic device.

[0185] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device obtains first resource information of the electronic device available to process the artificial intelligence task, checks whether there is sufficient available resource in the electronic device based on the first resource information, and, based on identifying that there is sufficient available resource in the electronic device, transmits a third artificial intelligence command to the first artificial intelligence model to process both the first task and the second task by the first artificial intelligence model of the electronic device without distributed processing by the at least one external electronic device, obtains third result information generated by the first artificial intelligence model, and, based on identifying that there is insufficient available resource in the electronic device, assigns the first task to the electronic device based on the first resource information of the electronic device, and assigns the second task to the at least one external electronic device based on second resource information corresponding to information related to the available resource of the at least one external electronic device.

[0186] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may search for one or more external electronic devices in the surrounding environment of the electronic device using a second short-range wireless communication, and select the at least one external electronic device among the one or more external electronic devices that is located within a designated security range and capable of a secure connection using the first short-range wireless communication.

[0187] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be configured to display the output information through the display of the electronic device (e.g., the display module (160) of FIG. 1, the screen display portion (254-1, 254-2) of FIG. 2, the display member (340) of FIG. 3a, the display (321) of FIG. 3b, the display (430) of FIG. 4) or output it through the speaker of the electronic device (e.g., the sound output module (155) of FIG. 1, the speaker (440) of FIG. 4). According to one embodiment, the first short-range wireless communication is an ultra-wideband (UWB) communication method that performs secure communication using a designated security key, and the output information may be at least one of text, voice, or image.

[0188] According to one embodiment, the user input information includes at least one of text, voice, or image, and the information related to the available resources obtained from the at least one external electronic device is information related to hardware capability and may include at least one of the process occupancy level, processing capability, memory allocation, or heat level.

[0189] According to one embodiment, a method of operation in an electronic device may include an operation of identifying an artificial intelligence operation for user input information based on receiving user input information (e.g., user input information (501) of FIG. 5).

[0190] According to one embodiment, the method may include the operation of establishing a communication connection between the electronic device and at least one external electronic device for distributed processing of the artificial intelligence task (e.g., external electronic devices of FIG. 5, FIG. 6a, FIG. 6b, FIG. 7a, FIG. 7b (403: 403a, 403b, 403c), first external electronic device of FIG. 8a (403a), second external electronic device of FIG. 8b (403b) and third external electronic device of FIG. 8c (403c)) through a communication circuit of the electronic device (e.g., communication module (190) of FIG. 1, communication circuit (450) of FIG. 4) using a first short-range wireless communication for security.

[0191] According to one embodiment, the method may include the operation of obtaining information related to resources available for distributed processing of the artificial intelligence task and artificial intelligence learning data from the at least one external electronic device.

[0192] According to one embodiment, the method may include an operation of identifying a first task, which is part of the artificial intelligence task to be distributed and processed using a first artificial intelligence model of the electronic device (e.g., the artificial intelligence model (510) of FIG. 5), and a second task, which is part of the artificial intelligence task to be distributed and processed using a second artificial intelligence model of the at least one external electronic device, based on information related to the available resources and artificial intelligence learning data obtained from the at least one external electronic device.

[0193] According to one embodiment, the method may include an operation of obtaining first result information generated by the first artificial intelligence model based on a first artificial intelligence command for the first task transmitted to the first artificial intelligence model.

[0194] According to one embodiment, the method may include the operation of receiving second result information generated by the second artificial intelligence model through the communication circuit using the first short-range wireless communication for security, based on a second artificial intelligence command for the second task transmitted to the at least one external electronic device.

[0195] According to one embodiment, the method may include an operation of obtaining output information (e.g., output information (503) of FIG. 5) including the first result information and the second result information.

[0196] According to one embodiment, the method may include an operation of identifying an artificial intelligence task for the user input information, which comprises an operation of obtaining at least one keyword for the artificial intelligence task based on the user input information and an operation of checking the correlation value between the artificial intelligence learning data and the at least one keyword. According to one embodiment, the method may further include an operation of assigning the second task for the at least one keyword to the at least one external electronic device based on the correlation value being greater than a reference value, and an operation of transmitting the second artificial intelligence command requesting the second task for the at least one keyword to be processed by the second artificial intelligence model of the at least one external electronic device through the communication circuit using the first short-range wireless communication for security.

[0197] According to one embodiment, the method may further include an operation of determining whether there is sufficient available resource in the at least one external electronic device based on information related to the available resource obtained from the at least one external electronic device, and an operation of transmitting the AI ​​command requesting the processing of the second task to the at least one external electronic device based on determining that there is sufficient available resource in the at least one external electronic device.

[0198] According to one embodiment, the method may further include the operation of transmitting an AI command requesting the first artificial intelligence model of the electronic device to process both the first task and the second task based on determining that the available resources in the at least one external electronic device are insufficient.

[0199] According to one embodiment, the method may further include: selecting a first external electronic device with the largest correlation value among the plurality of external electronic devices based on the connection of a plurality of external electronic devices using the first short-range wireless communication to distribute the artificial intelligence task; checking whether there are sufficient available resources for the second task in the first external electronic device based on information related to the resources of the first external electronic device; assigning the second task to the first external electronic device based on identifying that there are sufficient available resources for the second task in the first external electronic device; and selecting a second external electronic device with the next largest correlation value among the plurality of external electronic devices to assign the second task to the second external electronic device based on identifying that there are insufficient available resources for the second task in the first external electronic device.

[0200] According to one embodiment, the method may further include: acquiring first resource information of the electronic device available to process the artificial intelligence task; determining whether there is sufficient available resource in the electronic device based on the first resource information; transmitting a third artificial intelligence command to the first artificial intelligence model to process both the first task and the second task by the first artificial intelligence model of the electronic device without distributed processing by the at least one external electronic device based on identifying that there is sufficient available resource in the electronic device, and acquiring third result information generated by the first artificial intelligence model; and assigning the first task to the electronic device based on the first resource information of the electronic device based on identifying that there is insufficient available resource in the electronic device, and assigning the second task to the at least one external electronic device based on second resource information corresponding to information related to the available resource of the at least one external electronic device.

[0201] According to one embodiment, the method may further include the operation of searching for one or more external electronic devices in the surrounding environment of the electronic device using a second short-range wireless communication, and the operation of selecting at least one external electronic device among the one or more external electronic devices that is located within a designated security range and capable of a secure connection using the first short-range wireless communication.

[0202] According to one embodiment, the method may further include the operation of displaying the output information through a display of the electronic device (e.g., display module (160) of FIG. 1, screen display portion (254-1, 254-2) of FIG. 2, display member (340) of FIG. 3a, display (321) of FIG. 3b, display (430) of FIG. 4) or outputting it through a speaker of the electronic device (e.g., sound output module (155) of FIG. 1, speaker (440) of FIG. 4).

[0203] According to one embodiment, the first short-range wireless communication is an ultra-wideband (UWB) communication method that performs secure communication using a designated security key, and the output information may be at least one of text, voice, or image.

[0204] According to one embodiment, the user input information includes at least one of text, voice, or image, and the information related to the available resources obtained from the at least one external electronic device is information related to hardware capability and may include at least one of the process occupancy level, processing capability, memory allocation, or heat level.

[0205] According to one embodiment, in a non-transient storage medium storing one or more programs, the one or more programs may include a command to cause the electronic device to execute an operation to identify an artificial intelligence operation on user input information based on receiving user input information (e.g., user input information (501) of FIG. 5) when executed by at least one processor (e.g., processor (120) of FIG. 1, processor (410) of FIG. 4) of an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3a and FIG. 3c and electronic device (401) of FIG. 4).

[0206] According to one embodiment, the one or more programs may include a command to cause the electronic device, when executed by at least one processor of the electronic device, to execute an operation to establish a communication connection between the electronic device and at least one external electronic device (e.g., external electronic device of FIG. 5, FIG. 6a, FIG. 6b, FIG. 7a, FIG. 7b (403: 403a, 403b, 403c)), the first external electronic device (403a) of FIG. 8a, the second external electronic device (403b) of FIG. 8b, and the third external electronic device (403c) of FIG. 8c) for distributed processing of the artificial intelligence task through the communication circuit using a first short-range wireless communication for security.

[0207] According to one embodiment, the one or more programs may include a command that causes the electronic device to execute an operation of obtaining information related to resources available for distributed processing of the artificial intelligence task and artificial intelligence learning data from the at least one external electronic device when executed by at least one processor of the electronic device.

[0208] According to one embodiment, the one or more programs may include a command to cause the electronic device to execute, when executed by at least one processor of the electronic device, an operation to identify a first task which is part of the artificial intelligence task to be distributed and processed using a first artificial intelligence model of the electronic device (e.g., the artificial intelligence model (510) of FIG. 5) and a second task which is part of the artificial intelligence task to be distributed and processed using a second artificial intelligence model of the at least one external electronic device, based on information related to the available resources and artificial intelligence learning data obtained from the at least one external electronic device.

[0209] According to one embodiment, the one or more programs may include a command that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of obtaining first result information generated by the first artificial intelligence model based on a first artificial intelligence command for the first task transmitted to the first artificial intelligence model.

[0210] According to one embodiment, the one or more programs may include a command to cause the electronic device to execute, when executed by at least one processor of the electronic device, to receive second result information generated by the second artificial intelligence model based on the second artificial intelligence command for the second task transmitted to the at least one external electronic device through the communication circuit of the electronic device (e.g., communication module (190) of FIG. 1, communication circuit (450) of FIG. 4) using the first short-range wireless communication for security.

[0211] According to one embodiment, the one or more programs may include a command that causes the electronic device to execute an operation to obtain output information (e.g., output information (503) of FIG. 5) including the first result information and the second result information when executed by at least one processor of the electronic device.

[0212] Various effects that can be identified directly or indirectly through the present disclosure may be provided. The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0213] Furthermore, the embodiments disclosed in this document are presented for the purpose of explaining and understanding the disclosed technical content and are not intended to limit the scope of the technology described in this document. Accordingly, the scope of this document should be interpreted to include all modifications or various other embodiments based on the technical concept of this document.

[0214] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0215] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0216] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0217] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0218] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0219] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101, 200, 300, 401), Communication circuit (190, 450); At least one processor (120, 410) including a processing circuit; and It includes memory (130, 420) for storing instructions, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Based on receiving user input information, identify artificial intelligence operations for said user input information, and A communication connection is established between the electronic device and at least one external electronic device (403) for distributed processing of the artificial intelligence task through the communication circuit using a first short-range wireless communication (603) for security, and Information related to resources available for distributed processing of the above artificial intelligence task and artificial intelligence training data are obtained from the at least one external electronic device, and Based on information related to the available resources obtained from at least one external electronic device and the artificial intelligence learning data, a first task which is part of the artificial intelligence task to be distributed and processed using the first artificial intelligence model of the electronic device and a second task which is the remaining part of the artificial intelligence task to be distributed and processed using the second artificial intelligence model of the at least one external electronic device are identified, Based on the first artificial intelligence command for the first task transmitted to the first artificial intelligence model, first result information generated by the first artificial intelligence model is obtained, and Based on a second artificial intelligence command for the second task transmitted to at least one external electronic device, second result information generated by the second artificial intelligence model is received through the communication circuit using the first short-range wireless communication for security, and An electronic device that causes to obtain output information including the first result information and the second result information.

2. In paragraph 1, when the instructions are executed individually or collectively by the at least one processor, the electronic device: Based on the above user input information, at least one keyword for the above artificial intelligence task is obtained, and Check the correlation value between the artificial intelligence learning data and at least one keyword, and Based on the correlation value being greater than the reference value, the second task for the at least one keyword is assigned to the at least one external electronic device, and An electronic device that causes the second artificial intelligence command requesting the second artificial intelligence model of the at least one external electronic device to process the second task for the at least one keyword to be transmitted to the at least one external electronic device through the communication circuit using the first short-range wireless communication for security.

3. In claim 1 or 2, when the instructions are executed individually or collectively by the at least one processor, the electronic device: Based on information related to the available resources obtained from the at least one external electronic device, determining whether the available resources in the at least one external electronic device are sufficient, An electronic device that causes the second artificial intelligence command requesting the processing of the second task to be transmitted to the at least one external electronic device based on the determination that the available resources are sufficient in the at least one external electronic device.

4. In any one of claims 1 to 3, when the instructions are executed individually or collectively by the at least one processor, the electronic device: An electronic device that causes an artificial intelligence command to be transmitted to the first artificial intelligence model of the electronic device to process both the first task and the second task, based on the finding that the available resources are insufficient in at least one external electronic device.

5. In any one of claims 1 to 4, when the instructions are executed individually or collectively by the at least one processor, the electronic device: Based on the connection of multiple external electronic devices using the first short-range wireless communication to distribute the processing of the above artificial intelligence task, the first external electronic device with the largest correlation value among the multiple external electronic devices is selected, Based on information regarding the resources of the first external electronic device, determine whether there are sufficient available resources for the second task in the first external electronic device, and Based on identifying that there are sufficient available resources for the second task in the first external electronic device, the second task is assigned to the first external electronic device, and Based on identifying that the available resources for the second task are insufficient in the first external electronic device, a second external electronic device having a correlation value greater than the next largest among the plurality of external electronic devices is selected, and An electronic device that causes the second task to be assigned to the second external electronic device.

6. In any one of claims 1 to 5, when the instructions are executed individually or collectively by the at least one processor, the electronic device: Acquire first resource information of the electronic device available to process the above artificial intelligence task, and Based on the above first resource information, check whether there are sufficient resources available in the electronic device, and Based on identifying that the available resources of the electronic device are sufficient, a third artificial intelligence command is transmitted to the first artificial intelligence model to process both the first task and the second task by the first artificial intelligence model of the electronic device without distributed processing by the at least one external electronic device, and third result information generated by the first artificial intelligence model is obtained. An electronic device that, based on identifying that the available resources of the electronic device are insufficient, causes the first task to be assigned to the electronic device based on the first resource information of the electronic device, and causes the second task to be assigned to the at least one external electronic device based on second resource information corresponding to information regarding the available resources of the at least one external electronic device.

7. In any one of claims 1 through 6, when the instructions are executed individually or collectively by the at least one processor, the electronic device: One or more external electronic devices are searched for in the surrounding environment of the electronic device using a second short-range wireless communication, and Among the above one or more external electronic devices, at least one external electronic device is selected that is located within a designated security range and is capable of a secure connection using the first short-range wireless communication, and Causes the above output information to be displayed through the display (160, 430) of the electronic device or output through the speaker (155, 440) of the electronic device, The above-mentioned first short-range wireless communication is an ultra-wideband (UWB) communication method that performs secure communication using a designated security key, and The above output information is at least one of text, voice, or image, and the above user input information includes at least one of text, voice, or image, and The information regarding the available resources obtained from at least one external electronic device is information regarding hardware capabilities, and includes at least one of a process occupancy level, processing capability, memory allocation, or heat generation level, an electronic device.

8. A method of operation in an electronic device (101, 200, 300, 401), An operation to identify an artificial intelligence operation for said user input information based on receiving said user input information; An operation of establishing a communication connection (403) between the electronic device and at least one external electronic device for distributed processing of the artificial intelligence task through the communication circuit (190, 450) of the electronic device using a first short-range wireless communication (603) for security; The operation of obtaining information related to available resources and artificial intelligence training data from at least one external electronic device to distribute and process the above artificial intelligence task; An operation to identify, based on information related to the available resources and artificial intelligence learning data obtained from at least one external electronic device, a first task which is part of the artificial intelligence task to be distributed and processed using a first artificial intelligence model of the electronic device, and a second task which is the remaining part of the artificial intelligence task to be distributed and processed using a second artificial intelligence model of the at least one external electronic device; An operation to obtain first result information generated by the first artificial intelligence model based on the first artificial intelligence command for the first task transmitted to the first artificial intelligence model; The operation of receiving second result information generated by the second artificial intelligence model based on a second artificial intelligence command for the second task transmitted to at least one external electronic device through the communication circuit using the first short-range wireless communication for security; and A method comprising the operation of obtaining output information including the first result information and the second result information.

9. In paragraph 8, the operation of identifying an artificial intelligence operation on the user input information is, An operation to obtain at least one keyword for the artificial intelligence task based on the above user input information; and It includes an operation to check the correlation value between the artificial intelligence learning data and at least one keyword, and The above method is, An action of assigning the second task for the at least one keyword to the at least one external electronic device based on the correlation value being greater than the reference value; and A method further comprising the operation of transmitting the second artificial intelligence command, which requests the second artificial intelligence model of the at least one external electronic device to process the second task for the at least one keyword, to the at least one external electronic device through the communication circuit using the first short-range wireless communication for security.

10. In either paragraph 8 or 9, the above method is, An operation to determine whether there is sufficient available resource in the at least one external electronic device based on information related to the available resource obtained from the at least one external electronic device; The operation of transmitting the AI ​​command requesting the processing of the second task to the at least one external electronic device based on confirming that the available resources are sufficient in the at least one external electronic device; and A method further comprising the operation of transmitting an AI command requesting the first artificial intelligence model of the electronic device to process both the first task and the second task based on determining that the available resources are insufficient in the at least one external electronic device.

11. In any one of paragraphs 8 to 10, the above method is, Based on the connection of multiple external electronic devices using the first short-range wireless communication to distribute the processing of the above artificial intelligence task, the operation of selecting the first external electronic device with the largest correlation value among the multiple external electronic devices; An operation to determine whether there are sufficient available resources for the second operation in the first external electronic device based on information related to the resources of the first external electronic device; An action of assigning the second task to the first external electronic device based on identifying that there are sufficient available resources for the second task in the first external electronic device; and A method further comprising the operation of selecting a second external electronic device having a correlation value in the following order to assign the second task to the second external electronic device, based on identifying that there are insufficient available resources for the second task in the first external electronic device.

12. In any one of paragraphs 8 through 11, An operation to acquire first resource information of the electronic device available to process the above artificial intelligence task; An operation to check whether there are sufficient resources available in the electronic device based on the first resource information above; Based on identifying that the available resources of the electronic device are sufficient, the operation of transmitting a third artificial intelligence command to the first artificial intelligence model so that both the first task and the second task are processed by the first artificial intelligence model of the electronic device without distributed processing by the at least one external electronic device, and obtaining third result information generated by the first artificial intelligence model; and A method further comprising the operation of, based on identifying that the available resources of the electronic device are insufficient, assigning the first task to the electronic device based on the first resource information of the electronic device, and assigning the second task to the at least one external electronic device based on second resource information corresponding to information related to the available resources of the at least one external electronic device.

13. In any one of paragraphs 8 to 12, the method is, An operation of searching for one or more external electronic devices in the surrounding environment of the electronic device using a second short-range wireless communication; A method further comprising: an operation of selecting at least one external electronic device among the above one or more external electronic devices that is located within a designated security range and capable of a secure connection using the first short-range wireless communication; and an operation of displaying the output information through the display (160, 430) of the electronic device or outputting it through the speaker (155, 440) of the electronic device.

14. In any one of paragraphs 8 through 13, The above-mentioned first short-range wireless communication is an ultra-wideband (UWB) communication method that performs secure communication using a designated security key, and The above output information is at least one of text, voice, or image, and The above user input information includes at least one of text, voice, or image, and A method wherein information related to the available resources obtained from at least one external electronic device is information related to hardware capability, comprising at least one of the process occupancy level, processing capability, memory allocation, or heat level.

15. In a non-transient storage medium storing one or more programs, the one or more programs, when executed by at least one processor (120, 410) of an electronic device (101, 200, 300, 401), cause the electronic device: An operation to identify an artificial intelligence operation for said user input information based on receiving said user input information; An operation of establishing a communication connection between the electronic device and at least one external electronic device (403) for distributed processing of the artificial intelligence task through the communication circuit (190, 450) of the electronic device using a first short-range wireless communication (603) for security; The operation of obtaining information related to available resources and artificial intelligence training data from at least one external electronic device to distribute and process the above artificial intelligence task; An operation to identify, based on information related to the available resources and artificial intelligence learning data obtained from at least one external electronic device, a first task which is part of the artificial intelligence task to be distributed and processed using a first artificial intelligence model of the electronic device, and a second task which is the remaining part of the artificial intelligence task to be distributed and processed using a second artificial intelligence model of the at least one external electronic device; An operation to obtain first result information generated by the first artificial intelligence model based on the first artificial intelligence command for the first task transmitted to the first artificial intelligence model; The operation of receiving second result information generated by the second artificial intelligence model based on a second artificial intelligence command for the second task transmitted to at least one external electronic device through the communication circuit using the first short-range wireless communication for security; and A non-transient storage medium comprising an executable command to execute an operation to obtain output information including the first result information and the second result information.

Images