Electronic device, method, and storage medium for switching artificial intelligence model

Abstract

In the present disclosure, an electronic device may comprise: a communication circuit; a memory including one or more storage media for storing instructions; and at least one processor including a processing circuit. When the instructions are individually or collectively executed by the at least one processor, the instructions may cause the electronic device to perform at least one operation. The at least one operation may include an operation of acquiring a second AI model trained and released by a first AI model of a cloud server via the communication circuit. The at least one operation may include an operation in which an inference process is performed by the first AI model, and the inference process being performed in the first AI model is inherited by the second AI model and continuously processed on the basis of the quality of communication with the cloud server.

Description

Electronic device, method, and storage medium for converting artificial intelligence models

[0001] The present disclosure relates to a device, method, and storage medium for switching an artificial intelligence (AI) model by considering a communication situation in an electronic device.

[0002] An artificial neural network (ANN) refers to a computational architecture that models the biological brain. Technologies such as deep learning and machine learning can be implemented based on ANNs. As an example of an ANN, a deep neural network may have a multilayer structure containing multiple layers.

[0003] AI models are being used in a variety of ways to analyze natural language, such as images or speech. For example, AI models can provide generative AI services such as interpretation, translation, summarization, image generation, or augmented reality (AR) experiences.

[0004] AI models can be categorized as lightweight models (e.g., small language models (SLMs)) that operate on-device or large models (e.g., large language models (LLMs)) that operate in the cloud. In this case, AI services requiring relatively low computational throughput are preferably processed by lightweight models capable of fast processing, while AI services requiring high computational throughput are preferably processed by large models that offer slower processing speeds but high quality. However, cloud-based AI services can be affected by changes in network conditions.

[0005] The above information may be provided as background information to aid in understanding this document. None of the above is claimed to be prior art related to this document or can be used to determine prior art.

[0006] According to one embodiment, an electronic device may include a communication circuit. The electronic device may include a memory including one or more storage media for storing instructions. The electronic device may include at least one processor including a processing circuit. When the instructions are individually or collectively executed by the at least one processor, the instructions may cause the electronic device to perform at least one operation. The at least one operation may include an operation of obtaining a second AI model that has been pre-trained and distributed by a first AI model of a cloud server through the communication circuit. The at least one operation may include an operation of performing an inference process by the first AI model and inheriting and continuously processing the inference process being performed by the first AI model based on a communication quality with the cloud server.

[0007] According to one embodiment, a storage medium storing computer-readable instructions may cause the instructions, when executed by at least a part of at least one processor of an electronic device, to cause the electronic device to perform at least one operation. The at least one operation may include an operation of obtaining a second AI model that has been pre-trained and distributed by a first AI model of a cloud server. The at least one operation may include an operation of performing an inference process by the first AI model and, based on a communication quality with the cloud server, inheriting and continuously processing the inference process being performed by the first AI model in the second AI model.

[0008] According to one embodiment, a method of operating an electronic device may include acquiring a second AI model that has been pre-trained and distributed by a first AI model of a cloud server. The method may include performing an inference process by the first AI model, and then, based on the quality of communication with the cloud server, taking over the inference process being performed by the first AI model and continuously processing it by the second AI model.

[0009] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0010] FIG. 1 is a block diagram of an exemplary electronic device capable of performing the operations described herein.

[0011] FIG. 2 is an exemplary block diagram for providing generative AI functionality in an electronic device according to one embodiment.

[0012] FIG. 3 is a block diagram of an exemplary AI system capable of performing the operations described in this document.

[0013] FIG. 4 is a diagram illustrating the configuration of a generative AI system that provides generative AI services based on a network environment according to one embodiment.

[0014] FIG. 5 is a state transition diagram for providing a linked inference service in a generative AI system according to one embodiment.

[0015] FIG. 6 is a flowchart illustrating a method for switching an AI model to perform an inference service based on communication quality in a cloud server according to one embodiment.

[0016] FIG. 7a or FIG. 7b is a flowchart illustrating a method for switching an AI model to perform an inference service based on communication quality in an electronic device according to one embodiment.

[0017] FIG. 8 is a diagram for explaining a knowledge distillation technique in a generative AI system according to one embodiment.

[0018] FIG. 9 is a diagram for explaining model distribution in a generative AI system according to one embodiment.

[0019] FIG. 10 is a diagram for explaining communication quality to be considered when switching models in a generative AI system according to one embodiment.

[0020] FIG. 11a is a diagram for explaining preparation for conversion of an AI model in a generative AI system according to one embodiment.

[0021] FIG. 11b is a diagram illustrating preparations for completing inference without switching an AI model in a generative AI system according to one embodiment.

[0022] FIG. 12a is a diagram illustrating an exemplary scenario for providing an inference service by conversion of an AI model in a generative AI system according to one embodiment.

[0023] FIG. 12b is a diagram illustrating an exemplary scenario for providing an inference service by conversion of an AI model in a generative AI system according to one embodiment.

[0024] FIG. 12c is a diagram illustrating an exemplary scenario for providing an inference service by conversion of an AI model in a generative AI system according to one embodiment.

[0025] FIG. 12d is a diagram illustrating an exemplary scenario for providing an inference service by conversion of an AI model in a generative AI system according to one embodiment.

[0026] FIG. 12e is a diagram illustrating an exemplary scenario for providing an inference service by conversion of an AI model in a generative AI system according to one embodiment.

[0027] FIG. 13 is a block diagram of an electronic device within a network environment according to various embodiments.

[0028] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0029] FIG. 1 is a block diagram of an exemplary electronic device (100) capable of performing the operations described in this document.

[0030] Referring to FIG. 1, the electronic device (100) may be one of various forms of electronic devices, such as a notebook (190), smartphones (191) having various form factors (e.g., a bar-type smartphone (191-1), a foldable-type smartphone (191-2), or a sliderable (or rollable) type smartphone (191-3)), a tablet (192), a cellular phone (not shown), and other similar computing devices (not shown). The components, their relationships, and their functions illustrated in FIG. 1 are exemplary only and do not limit the implementations described or claimed in this document. The electronic device (100) may be referred to as a mobile device, a user device, a multi-function device, a portable device, or a server.

[0031] The electronic device (100) may include components including at least one processor (110) (hereinafter, referred to as 'processor (110)'), at least one memory (120) (hereinafter, referred to as 'memory (120)'), at least one display (140) (hereinafter, referred to as 'display (140)'), at least one image sensor (150) (hereinafter, referred to as 'image sensor (150)'), at least one communication circuit (160) (hereinafter, referred to as 'communication circuit (160)'), and / or at least one sensor (170) (hereinafter, referred to as 'sensor (170)'). The components are merely exemplary. For example, the electronic device (100) may include other components (e.g., power management integrated circuitry (PMIC), audio processing circuitry, an antenna, a rechargeable battery, or an input / output interface). For example, some components may be omitted from the electronic device (100). For example, several components may be integrated into a single component. For example, the electronic device (100) may further include at least some of the configurations and / or functions not shown. At least some of the respective components of the electronic device shown (or not shown) may be operatively, functionally, and / or electrically connected to each other.

[0032] The processor (110) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing. The processor (110) may include at least one electrical circuit and may individually or collectively perform distributed processing of instructions (or programs, data) stored in the memory (120). The processor (110) may include a processor assembly including one or more processing circuits. The processor (110) may include any processing circuit operative to control the performance and operations of one or more components (e.g., the memory (120), the display (140), the image sensor (150), the communication circuit (160), and / or the sensor (170)) of the electronic device (100). For example, the processor (110) (e.g., the application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). For example, the processor (110) may be implemented with multiple cores (or at least one core circuit), multiple chips, or multiple chipsets. For example, the processor (110) may include one or more processing circuits. For example, the processor (110) may include one or more processing circuits configured to individually and / or collectively perform various functions of the present disclosure. As a non-limiting example, at least a portion of the processor (110) may be included in a first chip of the electronic device (100), and at least another portion of the processor (110) may be included in a second chip of the electronic device (100) that is different from the first chip of the electronic device (100).

[0033] For example, the processor (110) may include a central processing unit (CPU) (311), a graphics processing unit (GPU) (112), a neural processing unit (NPU) (113), an image signal processor (ISP) (114), a display controller (115), a memory controller (116), a storage controller (117), a communication processor (CP) (118), and / or a sensor interface (119). These components of the processor (110) are merely exemplary. For example, the processor (110) may further include other components. For example, some components of the processor (110) may be omitted from the processor (110). For example, some components of the processor (110) may be included as separate components of the electronic device (100) outside the processor (110). For example, some components of the processor (110) (e.g., memory controller (116)) may be included within other components (e.g., at least a portion of memory (120), an interface (e.g., available for connection to at least one component of the electronic device (100)), a display (140) and / or an image sensor (150)).

[0034] The processor (110) may cause other components of the electronic device (100) to perform various operations by executing instructions stored in the memory (120). The CPU (111) (or central processing circuit) may be configured to control components of the processor (110) based on the execution of instructions stored in the memory (120) (e.g., volatile memory (121) and / or non-volatile memory (122)). The GPU (112) (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (113) (or neural processing circuit, or artificial intelligence (AI) chip) may be configured to execute operations for an AI model (e.g., convolution computation). The ISP (114) (or image signal processing circuit) may be configured to process a raw image acquired through the image sensor (150) into a format suitable for a component within the electronic device (100) or a component of the processor (110). The display controller (115) (or display control circuit, or DPU (display processing unit)) may be configured to process an image acquired from the CPU (111), GPU (112), ISP (114), or memory (120) (e.g., volatile memory (121)) into a format suitable for the display (140). The memory controller (116) (or memory control circuit) may be configured to control reading data from the volatile memory (121) and writing data to the volatile memory (121). The above storage controller (117) (or storage control circuit) may be configured to control reading data from the non-volatile memory (122) and writing data to the non-volatile memory (122).The CP (118) (communication processing circuit) may be configured to process data obtained from a component of the processor (110) into a format suitable for transmitting to another electronic device via the communication circuit (160), or to process data obtained from another electronic device via the communication circuit (160) into a format suitable for processing by a component of the processor (110). For example, the communication circuit (160) may include one or more communication circuits. The sensor interface (119) (or sensing data processing circuit, sensor hub) may be configured to process data on the state of the electronic device (100) and / or the state of the surroundings of the electronic device (100), obtained via the sensor (170), into a format suitable for a component of the processor (110).

[0035] The memory (120) may include one or more storage media (or one or more storage devices). For example, the memory (120) may include a memory assembly including one or more storage media. For example, the one or more storage media may include permanent memory (e.g., non-volatile memory (122)) such as a hard drive, flash memory, read-only memory (ROM), semi-permanent memory (e.g., volatile memory (121)) such as random access memory (RAM), any other suitable type of storage (or storage assembly), or any combination thereof. The memory (120) may include a cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (100). As a non-limiting example, the cache memory may be included within the processor (110). The memory (120) may be fixedly embedded within the electronic device (100) or incorporated into one or more suitable types of components (e.g., a subscriber identity module (SIM) card and / or a secure digital (SD) card) that may be repeatedly inserted into and removed from the electronic device (100).

[0036] For example, the memory (120) may store one or more software applications, such as an operating system (or system) software application, a firmware software application, a driver software application, a plug-in (e.g., add-in, add-on, and / or applet) software application, and / or any other suitable software applications. For example, the one or more software applications may include instructions executable by the processor (110). For example, the memory (120) may store instructions callable by an application programming interface (API). For example, the memory (120) may store instructions within a library.

[0037] According to one example, the electronic device (100) can execute at least one instance of an AI model. The instance may be an object corresponding to a program (or application), such as an AI model, for example. The instance may be named a replica, a pod, a container, or a virtual machine, and there is no limitation on the name thereof. The number of instances may correspond to the size of a resource (e.g., a GPU (112) or an NPU (113)), and accordingly, the number of instances may be used interchangeably with the size of the resource, or the instances may be used interchangeably with the resource.

[0038] As an example, a plurality of user requests may be input to the electronic device (100). The user requests may be associated with a service. The user request may be processed by a first instance of a first AI model, and a first processing result may be provided from the first instance of the first AI model. The first processing result may be processed by a first instance of a second AI model, and accordingly, a second processing result may be provided by the first instance of the second AI model. By serial processing of the processing results, the first instance of the M-th AI model may receive and process the N-1-th processing result. The first instance of the M-th AI model may provide the N-th processing result as a response. Accordingly, a response corresponding to the user request may be provided.

[0039] Based on the above-described process, responses corresponding to each of a plurality of user requests may be provided. Meanwhile, since processing must be performed by an instance, the time for providing responses corresponding to each of a plurality of user requests (hereinafter referred to as “response time”) may take a relatively long time. The response time may affect the latency of the instance. In order to reduce the response time, the electronic device (100) may increase the number of instances of at least one AI model, which may be referred to as scaling out. However, there may be a limit to increasing the number of instances due to hardware and / or software constraints of the electronic device (100) and / or parameter restrictions of the AI ​​model (e.g., large language model (LLM)).

[0040] FIG. 2 is an exemplary block diagram for providing a generative artificial intelligence (AI) function in an electronic device (e.g., electronic device (100) of FIG. 1) (hereinafter referred to as 'electronic device (100)') according to one embodiment.

[0041] Referring to FIG. 2, the electronic device (100) may include a processor (110) (e.g., the processor (110) of FIG. 1), a memory (120) (e.g., the memory (120) of FIG. 1), and / or an interface (I / F) (220). The electronic device (100) may be a device for providing a service linked to at least one AI system (210) (hereinafter, referred to as 'AI system (210)').

[0042] The AI ​​system (210) may include at least one AI model (hereinafter referred to as "AI model"). For example, the AI ​​system (210) may analyze received messages to generate a summary message. For example, the summary message may include one or more pieces of content (hereinafter referred to as "content") that have been reprocessed from received messages so that a user can easily recognize the content of the received messages. The content may be, for example, at least one of text, images, audio, or video.

[0043] The AI ​​system (210) may be based on natural language processing (NLP). NLP is a technology that allows, for example, an electronic device (200) to understand or process natural language input (hereinafter referred to as a "prompt") that can be expressed in voice and / or text. The electronic device (200) can understand natural language through NLP and, based on this, determine human intention or convey information in a language that humans can understand. To understand human language, NLP can learn the order of words or tokens and predict the probability of the next word or token in a given text. A token is a basic unit for processing or understanding a prompt in an AI model. Major NLP technologies include tokenization, part-of-speech tagging, syntax analysis, named entity recognition, or sentiment analysis for prompts corresponding to user input.

[0044] The I / F (220) can receive a query (230) and transmit the received query (230) to the processor (110). The query (230) can be a medium that guides the AI ​​system (210) to perform a task or generate a result in a desired direction. The query (230) can be the only window through which the user can communicate with the AI ​​system (210). The query (230) needs to be clear and specific in order to obtain an answer close to the desired result from the AI ​​system (210). According to an example, the I / F (220) can receive a response (240) (e.g., a summary message and / or a response message) processed by the AI ​​system (210) based on the query (230) (e.g., received messages), and output a response (240) converted into a natural language form (e.g., text, image, audio, or video) that can be recognized by humans. I / F (220) can input or output natural language in the form of voice and / or text, for example, with at least one component such as a keyboard, a touch panel, a display, and / or a speaker.

[0045] The processor (110) may execute software (e.g., a program) to control at least one other component (e.g., a hardware or software component) of the electronic device (100) to which it is electrically connected. The processor (110) may perform various data processing or operations. As at least a part of the data processing or operations, the processor (110) may store instructions or data received from another component (e.g., an I / F (220)) in the memory (120) (e.g., a volatile memory, but without limitation). As at least a part of the data processing or operations, the processor (110) may process instructions or data stored in the memory (120) (e.g., a volatile memory, but without limitation). As at least a part of the data processing or operations, the processor (110) may store data resulting from processing instructions or data in the memory (120) (e.g., a non-volatile memory, but without limitation).

[0046] The memory (120) may store various data used by at least one component (e.g., processor (110) and / or I / F (220)) of the electronic device (100). The data may include, for example, software (e.g., program) and input data or output data for commands related thereto. The memory (120) may also store at least one AI model (e.g., LLM, LVM (large vision models), LMM (large multimodal models)) for instance execution.

[0047] The memory (120) can store at least one instruction. The processor (110) can execute at least one instruction stored in the memory (120). When the at least one instruction is executed by the processor (110), the at least one instruction can cause the electronic device (100) to perform at least one operation. For example, as the at least one instruction is executed by the processor (110), at least one other component can be controlled, and / or various data processing or calculations can be performed. The performance of one operation by the processor (110) can mean, for example, that the operation is performed by (or under the control of) one entity included in the processor (110) (for example, the main processor, but without limitation). The performance of one operation can mean, for example, that a specific operation is performed by (or under the control of) multiple entities (for example, multiple processors). The fact that multiple operations are performed may mean, for example, that all of the multiple operations are performed by (or under the control of) one entity (e.g., but not limited to, a main processor (e.g., CPU (111) of FIG. 1).). The fact that multiple operations are performed may mean, for example, that some of the multiple operations are performed by at least one entity, and some of the remaining operations are performed by at least one other entity. At least one instruction causing the performance of one or more operations may be stored, for example, in one memory, or may be stored distributedly in each of a plurality of memories.

[0048] In the electronic device (100), the AI ​​system (210) may share resources (e.g., data processing or computational power) corresponding to part or all of at least one processor included in the processor (110) and / or resources (e.g., data recording area) corresponding to part or all of the memory (120). For example, the AI ​​system (210) may be operated by at least one of the CPU (111), the GPU (112), or the NPU (113). The AI ​​system (210) may be performed solely by the CPU (111), for example, by being allocated at least a portion of the memory (120). The AI ​​system (210) may be performed solely by the GPU (112), for example, by being allocated at least a portion of the memory (120). The AI ​​system (210) may be performed solely by the NPU (113), for example, by being allocated at least a portion of the memory (120). The AI ​​system (210) can be performed by the CPU (111) and the GPU (112) in cooperation, for example, by being assigned at least a portion of the memory (120). The AI ​​system (210) can be performed by the CPU (111) and the NPU (113) in cooperation, for example, by being assigned at least a portion of the memory (120). The AI ​​system (210) can be performed by the GPU (112) and the NPU (113) in cooperation, for example, by being assigned at least a portion of the memory (120). The AI ​​system (210) can be performed by the CPU (111), the GPU (112), and the NPU (113) in cooperation, for example, by being assigned at least a portion of the memory (120). The various embodiments to be described later in the present disclosure are not limited to the combination of components for performing the AI ​​system (210), and can be implemented and / or applied based on any combination.

[0049] FIG. 3 is a block diagram of an exemplary AI system (300) capable of performing the operations described in this document. The AI ​​system (300) may be a generative AI system, but will be referred to as the "AI system (300)" hereinafter.

[0050] Referring to FIG. 3, the AI ​​system (300) may include a User Query / Response Interface (310) (e.g., I / F (220) of FIG. 2) (hereinafter, referred to as 'I / F (310)'), an AI framework (320), a generative AI model (330), a database (340), or an Application / Service Component (350).

[0051] The I / F (310) can receive data or user input acquired or generated by an electronic device (e.g., the electronic device (100) of FIG. 1). The data acquired or generated by the electronic device (100) may include image or video data generated using a processor (e.g., the processor (110) of FIG. 1), values ​​transmitted through a sensor or sensor hub (e.g., external illuminance, an angle of the terminal, a display (e.g., the display (140) of FIG. 1) or the temperature of the electronic device (100), display (140) size or expansion / reduction information, an image captured by an image sensor (e.g., the image sensor (150) of FIG. 1)). The user input may be in the form of natural language, touch coordinates or stylus coordinates acquired through a touch panel or digitizer included in the display (140), images, and / or videos, but is not limited thereto. In addition, context information may also be transmitted when transmitting the user input. Contextual information can include various additional information at the time of user input. For example, this additional information may include information about the application the user is currently using or the user's location. Furthermore, user input may also be a mixture of natural language, images, sounds, and contextual information described above. Furthermore, user input may also take the form of non-natural language input, such as selecting a menu.

[0052] The I / F (310) can output the results of analyzing the output and / or input of the AI ​​system (300). The output may be in the form of natural language or specific content. The output may also be provided in the form of an action requested by the user. The output may also be provided in the form of a specific value specified by the user. The I / F (310) can output the results of the generative AI system (300) to the user. The output may be in the form of natural language or specific content. The output may also be provided in the form of an action requested by the user.

[0053] The AI ​​framework (320) can receive user input and coordinate and control each component necessary to carry out the user's intention based on the user's query. For example, the AI ​​framework (320) may include a prompt design component (321), an API / plug-in management component (323), or an output modification component (or refiner component) (325).

[0054] User input received from I / F (310) can be transmitted to a prompt design component (321). The prompt design component (321) can be used to generate a prompt suitable for inputting the user input into a generative AI model (330) (e.g., LLM, LVM, or LMM). The prompt design component (321) can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The prompt design component (321) can access user preference data (343), a prompt library (341), and a knowledge component including prompt examples based on the user input to generate a prompt, and can transmit the generated prompt to the generative AI model (330), which is an LLM or LMM.

[0055] The API / plugin management component (323) may communicate with external information when a request for additional information is made when passing user input as input to the generative AI model (330). The API / plugin management component (323) establishes a channel for communicating with the outside of the AI ​​interface through the API, and enables access to various data sources (e.g., knowledge repositories (345)) through the established channel.

[0056] The API / plugin management component (323) may request the application / service component (323) to perform an action that ultimately requires user input, rather than an intermediate result, through an API when the action needs to be performed by the application or service. Information obtained from an external source may be used to generate a prompt in the prompt design component (321) along with user input, or may be passed as input to a generative AI model (330).

[0057] The output tuning component (325) (also referred to as a refiner component) can fine-tune or reprocess the output from the generative AI model (330). The output tuning component (325) can verify, for example, whether the content generated by the generative AI model (330) is irrelevant, biased, or harmful. The output tuning component (325) can determine to what extent the content matches the user's desired result and, if additional processing is required, can proceed with the process. The output tuning component (325) can additionally configure and provide the user with hints to avoid unwanted output.

[0058] A generative AI model (330) may generally refer to an AI neural network that creates new types of data based on user input information. A generative AI model (330) may include a model that generates images and / or a model that generates language. The model that generates images may include, for example, a generative adversarial network (GAN) or a variational autoencoder (VAE). The model that generates images may be, for example, a diffusion-based AI model that uses a VAE and a transformer structure. The model that generates language may be a model trained to output the most statistically appropriate output value based on input values. Representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. In addition, there is also an LMM as an AI model (330) that can recognize various types of data input, such as text, images, voice, and video, and generate new data corresponding thereto.

[0059] FIG. 4 is a diagram illustrating the configuration of a generative AI system (400) that provides generative AI services based on a network environment according to one embodiment.

[0060] The generative AI system (400) of the present disclosure may include a cloud server (410) and / or an electronic device (440) connected via a network (420) to provide an AI service, such as an inference service, based on a network environment. According to one example, the generative AI system (400) may take over and process an inference service processed by a generative AI model (e.g., cloud server LLM) loaded on a cloud server (410) by a generative AI model (e.g., sLM) loaded on an electronic device (440) in consideration of the quality of the network (420) (hereinafter, referred to as “communication quality”). In this case, even if a situation occurs where the communication quality is poor, an uninterrupted user experience can be provided.

[0061] Referring to FIG. 4, a cloud server (410) can provide an inference service. The cloud environment in which the cloud server (410) can provide the inference service may be an IT (information technology) environment that provides virtualized servers accessible through a vast network and programs and databases running on the servers. The cloud service may be a service provided based on cloud computing. The cloud service may include infrastructure services, platform services, or software services. The cloud server (410) may be a virtual server that quickly builds a server environment through a virtual infrastructure without a separate physical server. The cloud server (410) is not limited to a physical location and can be accessed without temporal and / or spatial constraints via a network (420) such as the Internet.

[0062] The cloud server (410) can provide LLM-based services to users when the network (420) condition is good. The cloud server (410) can include a senior model (411) (e.g., cloud server LLM), a knowledge distiller (413), a dynamic model provider (415), or a check point provider (417).

[0063] The senior model (411) may be an LLM foundation model generated through pre-training. The senior model (411) is an AI model that can be adopted to provide AI services when the network environment connecting the electronic device (440) is smooth. The senior model (411) supports the application of knowledge distillation techniques to create a successor model (sLM) to continuously provide AI services even when the network environment is poor.

[0064] The knowledge distiller (413) may perform knowledge distillation to provide a pre-trained LLM foundation model to the electronic device (440). The knowledge distiller (413) may generate a successor model (446) of the electronic device (440) from the senior model (411) (e.g., LLM). The knowledge distiller (411) may periodically and / or aperiodically distribute relevant information to the electronic device (440) to enable inference for the successor model (446) or continuous AI services by the successor model (446) as needed.

[0065] The dynamic model provider (415) is a functional module that allows variable loading of an AI model depending on network conditions and / or system resources. The dynamic model provider (415) can recognize when the network conditions are not good and provide some or all of the weights to the electronic device (440) so that inference for the AI ​​service can be continuously performed in the on-device environment, thereby providing specific layers or tokens to the electronic device (440). For example, the dynamic model provider (415) can provide a partial AI model corresponding to a specific part (e.g., a specific layer or module) of the successor model (446) to the electronic device (440) when necessary. The dynamic model provider (415) can support variable resolution. For example, the dynamic model provider (415) can adjust the resolution of the input or adjust the precision of the parameters according to the cloud environment and / or the on-device environment. For example, an AI model for image classification can improve performance by processing more detailed information when receiving high-resolution images as input. On the other hand, if the network condition is poor or there are memory constraints, the AI ​​model can reduce processing time and save resources by receiving low-resolution images as input. For example, the precision of parameters can be improved by performing high-precision (32-bit, 64-bit) calculations in the cloud, and can be reduced processing time or memory usage by performing low-precision (16-bit, 8-bit) calculations on-device. The dynamic model provider (415) can provide some inferences that should be processed by the senior model (411) depending on specific conditions (network condition, available memory) to be processed by the successor model (446) of the electronic device (440).

[0066] The checkpoint provider (417) can store the overall state of the senior model (411) so that the electronic device (440) can continue to perform tasks when the network (420) is unstable. For example, the checkpoint provider (417) can store information about the parameters (weights and biases) of the model, the inference state, the random state, or the protocol. The parameters of the model can include, for example, the weights and biases of all layers of the senior model (411). The information about the inference state can be, for example, information about all states related to the current inference time point (e.g., the current layer, token). The information about the random state can include a random number used in the process of dropout and sampling for the reproducibility of the inference process by the senior model (411). The protocol can, for example, indicate a communication protocol for transmitting the stored checkpoints to the electronic device (440).

[0067] The network (420) and the base station (430) can form a passage for transmitting signals or data between the cloud server (410) and the electronic device (440).

[0068] The electronic device (440) can selectively apply a cloud-based senior model (411) or an on-device-based successor model (446) based on communication quality to provide AI services to the user. For example, when the network (420) condition is good and the communication quality is good, the electronic device (440) can operate to provide AI services by the senior model (411) of the cloud server (410). For example, when the network (420) condition is bad and the communication quality is poor, the electronic device (440) can operate the successor model (446) distributed from the cloud server (410) to take over and provide AI services provided by the senior model (411). The electronic device (440) may include a successor model downloader (441), a network status checker (442), a model switcher (443), a check point receiver (444), a dynamic model loader (445), or a junior model (446) (e.g., on device sLM).

[0069] The successor model downloader (441) can download and store a successor model (446) that can be periodically and / or aperiodically distributed from a cloud server (410) when the network (420) environment is good.

[0070] The network status checker (442) can continuously check the status of the network (420), such as throughput, latency, or received signal strength indicator (RSSI), to obtain communication quality. The network status checker (442) can, for example, ensure that the AI ​​service provided by the senior model (411) of the cloud server (410) can be maintained when the communication quality is good. The network status checker (442) can generate an event to switch the AI ​​model that provides the AI ​​service from the senior model (411) to the successor model, for example, when the communication quality continues to be below a critical level for a predetermined period of time.

[0071] When a model conversion request is made from a network status checker (442) due to poor communication quality, the model converter (443) may request the dynamic model provider (415) and checkpoint provider (417) provided in the cloud server (410) to provide information related to the progress status of the AI ​​service, the senior model status, or the inference status.

[0072] If the checkpoint receiver (444) determines that the communication quality is unstable during the inference process performed by the senior model (411) and that it will be difficult to maintain the AI ​​service provided based on the cloud environment, the checkpoint receiver (444) may receive inference status information of the senior model (411) from the cloud server (410). The inference status information may include, for example, information about a layer state, a bias state, an input token state, or a random state. The layer state may include, for example, parameters updated during learning in each layer of the neural network. As an example, the layer state may be composed of two elements: weights and activation values. Weights are values ​​that determine how much each neuron influences the input. Weights are updated during the learning process and may play an important role in the performance of the senior model (411). Activation values ​​may be output values ​​generated by neurons in a specific layer according to the input. The activation values ​​may be passed on to the next layer and may contribute to the prediction results of the senior model (411). The bias state is a constant value added to each neuron, allowing the senior model (411) to learn data more flexibly. The bias state adjusts the neuron's activation function and can be learned along with the weights. This allows activation to occur even when the input data is 0, which can help the senior model (411) learn various patterns. The input token state can indicate the state of the data given as input (e.g., sentences, words, image pixels). The input token state can be mainly expressed in two ways. One of them is embedding, which converts the input token into a vector in a high-dimensional space. Each token is represented as a dense vector containing meaning, which can support the senior model (411) to learn effectively.Another type, positional encoding, is a vector containing the positional information of input tokens, allowing the master model (411) to understand the order of the sequence. This can be particularly important in transformer models. Random states can be used to control randomness during model training. Random states can influence initial weight settings, data sampling, or batch generation. A fixed random state can reproduce the same results each time the same data and algorithm are used. Therefore, random states can be an important factor in increasing the reproducibility of experiments.

[0073] The checkpoint receiver (444) receives all information about the current inference status being performed by the senior model (411) from the cloud server (410), and can perform a process of preparing to continue inference in the successor model (446) by additionally receiving knowledge distillation information.

[0074] The dynamic model loader (445) can download specific layers and / or tokens of the senior model (411) required for the work of the inference state currently in progress by the senior model (411) or the successor model (446). For example, the dynamic model loader (445) can perform functions such as model partitioning, partial loading, local caching, or communication status checking. Model partitioning can, for example, divide the model into multiple parts (e.g., layer units). Partial loading can, for example, load only the weights of the parts required during the inference process. Local caching can, for example, cache pre-downloaded weights locally so that they can be reused. Communication status checking can, for example, continuously monitor network conditions and load weights when necessary.

[0075] The successor model (446) can dynamically update its weights, bias, random state, token state, and inferencing layer in advance when the network environment is predicted to deteriorate to the point where AI services in a cloud environment cannot be supported. The successor model (446) can inherit and perform inference in an on-device environment without the assistance of the senior model (411). This enables uninterrupted service to users.

[0076] A cloud server (410) or electronic device (440) including components as described above can perform the following operations. The following operations are not described as being performed in a chronological order, and the order of the operations may be changed as needed.

[0077] The cloud server (410) can perform pre-learning on a successor model (446) using a senior model (411) or a knowledge distiller (413), and distribute the pre-learned successor model (446) to an electronic device (440).

[0078] The successor model downloader (441) of the electronic device (440) can pre-download or update the successor model (446) from the cloud server (410) according to a specific cycle. The pre-downloaded successor model (446) can perform a preparatory operation for the successor model (446) to take over and process the inference for the corresponding AI service in a section where it is determined that the communication environment may become poor in the future (e.g., section B (1020) of FIG. 10). The pre-downloaded successor model (446) can be driven to process the remaining inference for the corresponding AI service in the on-device environment on behalf of the senior model (411) in a section where the communication quality is so poor that it cannot support the AI ​​service in the cloud environment (e.g., section C (1030) of FIG. 10).

[0079] The network status checker (442) of the electronic device (440) can measure the communication quality and perform a model switching preparation operation or a model switching operation by the model switcher (443) based on the measured communication quality. According to one example, the network status checker (442) can measure the communication quality based on at least one of throughput, latency, or signal strength (RSSI). If the communication quality is continuously maintained for a predetermined period of time in a predetermined section (e.g., section B (1020) of FIG. 10), the network checker (442) can determine that a situation may arise in which communication with the cloud server (410) may become difficult in the future and can cause the model switcher (443) to operate.

[0080] The model converter (443) of the electronic device (440) may perform an operation to obtain a checkpoint related to an inference operation in the senior model (411). In one example, the model converter (443) may call a checkpoint receiver (444). The called checkpoint receiver (444) may receive information that may indicate a current inference state of the senior model (411) from the checkpoint provider (417) of the cloud server (410). The inference state information that may indicate the inference state may include information about the layer information, bias state, token information, and random state of the senior model (411) in which inference is in progress. The inference state information may be used to determine whether to continue the inference by the senior model (411) in the next step or to continue the inference by taking over to the successor model (446). If it is determined to continue the inference by the senior model (411), the inference state information may be utilized in the next inference step to be performed by the senior model (411).

[0081] The electronic device (440) may determine how to process the inference to be performed later based on the number of layers that the senior model (411) is inferring for the AI ​​service. For example, the checkpoint receiver (444) may count the number of remaining layers to complete the inference according to the AI ​​service based on the previously acquired checkpoints. The dynamic model loader (445) of the electronic device (440) may determine that the number of remaining layers for future inference is large if the number of remaining layers is greater than a certain threshold level and may prepare to drive the successor model (446). If the number of remaining layers is not greater than a certain threshold level, the dynamic model loader (445) may determine that the number of remaining layers for future inference is small and may prepare to continue the inference by the senior model (411).

[0082] The electronic device (440) can load a dynamic model according to a determined method to continue the remaining inference. For example, if the number of layers remaining until the inference is completed exceeds a certain threshold level, the model converter (433) can prepare to run the successor model (446) so that the successor model (446) can be used. The process of preparing to run the successor model (446) may include a process of receiving information (e.g., synchronization and mapped layer information) necessary to inherit the inference created during the knowledge distillation process from the knowledge distiller (413) of the cloud server (410).

[0083] For example, if the number of layers remaining until the inference is completed is less than a certain threshold level, the electronic device (440) can call the dynamic model loader (445) to download only some of the necessary weight parameter information of the senior model (411) in order to continue the inference being processed by the senior model (411) based on the senior model (411).

[0084] The electronic device (440) can take over and process inference in the on-device environment for dynamic models and checkpoints. In one example, after completing the loading of the dynamic model, the electronic device (440) can take over and continue processing the inference being processed by the senior model (411) with a model completed by loading the dynamic model in a section (e.g., section C (1030) of FIG. 10) where the communication quality determines that AI service support in the cloud environment is no longer possible.

[0085] When the electronic device (440) completes all inference operations for the AI ​​service, it can store the successor model (446) so that it can provide the AI ​​service to the user based on the pre-downloaded successor model (446).

[0086] Although not mentioned in the above-described operations, if a situation is detected where it is difficult to provide AI services in an on-device environment, the inference operation may be continuously performed by switching to a cloud environment. For example, a situation where it is difficult to provide AI services may include a situation where the battery of the electronic device (440) is below a critical level (e.g., 15%). In this case, if it is determined that it is no longer possible to perform inference operations on the electronic device (440), the remaining inference may be processed by an AI model supporting AI services in a cloud environment. For example, if a user receives an AI service request that is difficult to process in an on-device environment, the AI ​​service may be processed by an AI model operating in a cloud environment.

[0087] FIG. 5 is a state transition diagram for providing a linked inference service in a generative AI system (e.g., the generative AI system (400) of FIG. 4) according to one embodiment.

[0088] Referring to FIG. 5, the generative AI system (400) can perform an AI service by a generative AI model (e.g., a senior model (411) of FIG. 4) pre-trained in a cloud server (e.g., a cloud server (410) of FIG. 4) if the communication quality is good enough to provide an AI service based on a cloud environment (operation 510). The communication quality can be measured based on at least one of throughput, latency, or signal strength (RSSI) corresponding to a wireless channel connecting an electronic device (e.g., an electronic device (440) of FIG. 4) to the cloud server (410), for example (operation 511).

[0089] The generative AI system (400) can determine whether to perform the AI ​​service by a senior model (411) in the cloud server (410) or by a successor model (e.g., successor model (446) of FIG. 4) in the electronic device (440) by considering the memory usage of the electronic device (440) and / or the cloud server (410) (operation 510).

[0090] In the generative AI system (400), the cloud server (410) may perform knowledge distillation according to a specific cycle to prepare for a situation where communication quality is poor, and perform a preparatory operation to take over the corresponding AI service in an on-device environment and continue inference (operation 513). The preparatory operation to take over the corresponding AI service and continue inference may, for example, generate a successor model (e.g., the successor model (446) of FIG. 4) in which knowledge has been transferred by the senior model (411) based on knowledge distillation, and distribute the generated model to an electronic device (440). The preparatory operation may be performed by the senior model (411) and the knowledge distiller (413) provided in the cloud server (410). The preparatory operation to take over the corresponding AI service and continue inference may include, for example, an operation of preparing information regarding checkpoints. The operation of preparing checkpoints may be performed by a checkpoint provider (407) provided in the cloud server (410).

[0091] The generative AI system (400) enables the cloud server (410) to periodically or aperiodically transmit inference status information to the electronic device (440) at a specific point in time while the AI ​​service is being performed by the senior model (411) (operation 515). The inference status information may include information about the progress status, model status, or inference status of the AI ​​service being provided by the senior model (411). For example, the cloud server (410) may transmit some or all of the weights or biases to the electronic device (440). For example, the cloud server (410) may provide token information or random state information to the electronic device (440). For example, the cloud server (410) may provide information about specific layers, tokens, or random states to the electronic device (440) so that the inference for the corresponding AI service can be continued in a cloud environment or in an on-device environment. For example, the cloud server (410) may additionally provide knowledge distillation information to an electronic device (440). As this has been specifically described above with reference to FIG. 4, reference may be made to that description.

[0092] When a downward model switching event occurs to switch models from a senior model (411) to a successor model (446), the generative AI system (400) may perform an operation to take over and process the remaining inferences for the corresponding AI service in the successor model (446) (operation 520). For example, a model switching request may occur in a situation where the communication quality deteriorates to the extent that an AI model must be switched (e.g., section C (1030) of FIG. 10). If a model switching is not requested, the cloud server (410) may continue to perform an AI service based on the senior model (411). The generative AI system (400) may perform a model switching operation before the model switching. In the model switching preparation operation, the generative AI system (400) may transmit information about specific layers and / or tokens corresponding to the inference progress status to the electronic device (440) by the cloud server (410) in the model switching preparation operation (operation 515).

[0093] The generative AI system (400) may monitor whether the electronic device (440) can perform an inference operation according to the AI ​​service in a cloud environment due to an improvement in the network status while performing an operation to process the remaining inference for the AI ​​service in the successor model (446) (operation 521). In addition, the memory usage status of the electronic device (440) may also be monitored (operation 521).

[0094] The generative AI system (400) can generate checkpoints based on the inferences being processed by the electronic device (440) while the successor model (446) performs the operation of processing the remaining inferences for the corresponding AI service. The checkpoints generated by the electronic device (440) can be distributed to the cloud server (410) when communication quality deteriorates.

[0095] The generative AI system (400) may generate an upstream model transition event (operation 525) when it determines that a transition from the successor model (446) to the senior model (411) is possible, taking into account communication quality and / or memory status. When an upstream model transition event occurs, the generative AI system (400) may transfer the inference operation being processed by the successor model (446) to the senior model (411) for processing.

[0096] A specific description of the above-described operations 510, 511, 513, 515, 520, 521, 523 or 525 will be specifically described below with reference to FIG. 6, FIG. 7a or FIG. 7b.

[0097] FIG. 6 is a flowchart illustrating a method for switching an AI model to perform an inference service based on communication quality in a cloud server (e.g., cloud server (410) of FIG. 4) according to one embodiment.

[0098] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0099] Referring to FIG. 6, the cloud server (410) can determine, in operation 611, whether an AI service is requested from an electronic device (e.g., the electronic device (440) of FIG. 4). In the following description, AI services and inference services will be used in parallel, but this is not intended to be limiting. The AI ​​service may be, for example, one of the following services: interpretation, translation, summarization, image generation, or AR experience provision.

[0100] The cloud server (410) may, at operation 613, perform an inference service by a senior model (411) for the requested AI service. The inference service is a service in which a machine learning model (e.g., a generative AI model such as LLM) trained through pre-training can derive a logical conclusion for a specific query. For example, a generative AI model pre-trained using sample text can perform an inference service that analyzes new text that has not been previously trained and derives a conclusion. The inference service may be affected by the performance of the generative AI model. For example, the higher the computational power, the faster the generative AI model can provide high-quality results. In the following description, the generative AI model provided in the cloud server (410) will be referred to as a “senior model (e.g., the senior model (411) of FIG. 4)” to indicate that it can preferentially process the requested inference service, but this is merely an exemplary expression.

[0101] The cloud server (410) may, at operation 615, generate a successor model (e.g., the successor model (446) of FIG. 4) or perform pre-training by transferring knowledge acquired through inference of the senior model (411) based on knowledge distillation. In this case, the performance of the senior model (411), which is a large model, may be transferred to the successor model (446), which is a small model, so that the inference performance of the lightweight successor model (446) can be maintained to the maximum extent possible.

[0102] The cloud server (410) may, in operation 617, distribute the successor model (446) generated based on pre-training to the electronic device (440) periodically or in response to the occurrence of a specific event. For example, the cloud server (410) may distribute the successor model (446) to the electronic device (440) whenever necessary in a situation where the communication quality with the electronic device (440) is good (e.g., section A (1010) of FIG. 10).

[0103] The cloud server (410) may determine, at operation 619, whether an event to provide inference status information to the electronic device (440) has occurred. The event to provide inference status information may be triggered by a request for inference status information from the electronic device (440) to prepare for switching the AI ​​model. For example, preparation for switching the AI ​​model may be triggered by a situation (e.g., section B (1020) of FIG. 10) that is expected to deteriorate to the point where the AI ​​model needs to be switched for a certain period of time.

[0104] If an event to provide inference status information to the electronic device (440) does not occur, the cloud server (410) may proceed to operation 623. In response to a request for inference status information, the cloud server (410) may transmit inference status information for preparing AI model conversion to the electronic device (440) in operation 621. The inference status information may include information about the progress status, model status, or inference status of the AI ​​service being provided by the senior model (411). For example, the cloud server (410) may transmit some or all of the weights or biases to the electronic device (440). For example, the cloud server (410) may provide token information or random state information to the electronic device (440). For example, the cloud server (410) may provide information about specific layers, tokens, or random states to the electronic device (440) so that inference for the corresponding AI service can be continued in a cloud environment or inference for the corresponding AI service can be continued in an on-device environment. For example, the cloud server (410) may additionally provide knowledge distillation information to an electronic device (440). As this has been specifically described above with reference to FIG. 4, reference may be made to that description.

[0105] The cloud server (410) may determine, at operation 623, whether a model switch request is made by the electronic device (440). For example, the model switch request may be provided by the electronic device (440) when a situation occurs in which the communication quality deteriorates to the extent that an AI model must be switched (e.g., section C (1030) of FIG. 10).

[0106] If no model conversion is requested, the cloud server (410) can determine whether all inference processing for the corresponding AI service has been completed in operation 625. If all inference processing for the corresponding AI service has been completed, the cloud server (410) can transmit the processing result for the corresponding AI service to the electronic device (440) in operation 627. If all inference processing for the corresponding AI service has not been completed, the cloud server (410) can continue to perform inference processing for the corresponding AI service in operation 613.

[0107] Upon receiving a model conversion request, the cloud server (410) may, at operation 629, convert the model to an AI model operating in the on-device environment to perform the remaining inference for the AI ​​service. For example, the AI ​​model operating in the on-device environment may be a dynamic model or a successor model (446) for the senior model (411).

[0108] FIG. 7A or FIG. 7B is a flowchart illustrating a method for switching an AI model to perform an inference service based on communication quality in an electronic device (e.g., the electronic device (440) of FIG. 4) according to one embodiment.

[0109] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0110] Referring to FIG. 7A or FIG. 7B , the electronic device (440) may determine, in operation 711, whether an AI service is requested by the user. For example, AI services that may be requested by the user may include services such as interpretation, translation, summarization, image generation, or provision of AR experiences.

[0111] The electronic device (440) may, at operation 713, request processing of an AI service based on a senior model (e.g., the senior model (411) of FIG. 4) from a cloud server (e.g., the cloud server (410) of FIG. 4) based on a user's request. The senior model-based inference service may be, for example, an inference service to be provided by the senior model (411) based on a user's request.

[0112] The electronic device (440) may, at operation 715, download and / or update a successor model (e.g., the successor model (446) of FIG. 4) from the cloud server (410). The successor model (446) may be generated by transferring knowledge acquired by the cloud server (410) through inference of the senior model (411) based on knowledge distillation. In this case, the performance of the successor model (446), which is a small model, may be transferred from the senior model (411), which is a large model, so that the inference performance can be maintained to the maximum extent even in the lightweight successor model (446). The successor model (446) may be distributed from the cloud server (410) periodically or in response to the occurrence of a specific event. For example, the electronic device (440) can receive information related to the successor model (446) from the cloud server (410) whenever necessary in a situation where the communication quality with the cloud server (410) is good (e.g., section A (1010) of FIG. 10). The successor model (446) can be prepared for operation when a situation where the communication quality is expected to deteriorate to the extent that the AI ​​model needs to be switched (e.g., section B (1020) of FIG. 10) is maintained for a certain period of time. The successor model (446) can be operated to take over and process the corresponding AI service in a situation where the communication quality deteriorates to the extent that the AI ​​model needs to be switched (e.g., section C (1030) of FIG. 10).

[0113] The electronic device (440) may determine, at operation 717, whether a model switch preparation event has occurred. In one example, the model switch preparation event may occur when it is necessary to prepare for the AI ​​model switch. For example, the AI ​​model switch preparation may be triggered by a situation (e.g., section B (1020) of FIG. 10) that is expected to deteriorate to the point where the AI ​​model needs to be switched, persisting for a certain period of time.

[0114] If a model transition preparation event has not occurred, the electronic device (440) can determine whether all inference processing for the corresponding AI service has been completed in operation 733. If all inference processing for the corresponding AI service has been completed, the electronic device (440) can receive and output the inference processing result for the corresponding AI service from the cloud server (410) in operation 735. If all inference processing for the corresponding AI service has not been completed, the electronic device (440) can continue to perform inference processing for the corresponding AI service by the senior model (411) based on the cloud environment in operation 715.

[0115] When a model transition preparation event occurs, the electronic device (440) may request and obtain inference status information from the cloud server (410) in operation 719. The electronic device (440) may use the inference status information to determine whether to continue inference based on the senior model (411) or the successor model (446) in the next step. When the electronic device (440) determines to continue inference based on the senior model (411) in the next step, the inference status information may be utilized in the next inference step based on the senior model (411). The inference status information may include information regarding the progress status of the AI ​​service provided by the senior model (411), the model status, or the inference status. For example, the electronic device (440) may receive some or all of the weights or biases from the cloud server (410). For example, the electronic device (440) may receive token information or random status information from the cloud server (410). For example, the electronic device (440) may receive information regarding specific layers, tokens, or random states from the cloud server (410) to enable it to continue inference regarding the AI ​​service in a cloud environment, or to continue inference regarding the AI ​​service in an on-device environment. For example, the electronic device (440) may additionally receive knowledge distillation information from the cloud server (410). This has been specifically described above with reference to FIG. 4, and reference may be made to that description.

[0116] The electronic device (440) may determine, in operation 721, whether a model transition preliminary event has occurred. The model transition preliminary event may occur depending on whether the successor model (446) requires preparation for operation. In one example, the electronic device (440) may determine the number of remaining layers to perform inference for the corresponding AI service based on inference status information, and may make a determination based on the determined number of remaining layers. For example, the electronic device (440) may determine that a model transition preliminary event has not occurred if the number of remaining layers is below a threshold level. For example, the electronic device (440) may determine that a model transition preliminary event has occurred if the number of remaining layers is above a threshold level.

[0117] If a model transition preliminary event does not occur, the electronic device (440) may prepare to continue inference based on the senior model (411) at operation 723. For example, the electronic device (440) may download only some dynamic models that can be additionally processed in the on-device environment from the cloud server (410) (see FIG. 11b). This is because, since the number of layers remaining until inference completion is small, the electronic device (440) may download some weight parameter information of the senior model (411) necessary to continue inference based on the senior model (411) from the cloud server (410).

[0118] When a model transition preliminary event occurs, the electronic device (440) may, in operation 725, download knowledge distillation information from the cloud server (410) and perform operation preparation for the successor model (446) so that the previously prepared successor model (446) can take over the inference for the AI ​​service performed by the senior model (411). For example, the operation preparation process of the successor model (446) may include a process of receiving information (e.g., synchronization and mapped layer information) from the cloud server (410) that can continue the inference between the senior model (411) and the successor model (446) generated in the knowledge distillation process performed in the cloud server (410). The electronic device (440) may load the successor model (446) and prepare for use.

[0119] The electronic device (440) may determine, at operation 727, whether a model switching event has occurred. For example, a model switching event may occur due to a situation in which communication quality has deteriorated to the point where an AI model must be switched (e.g., section C (1030) of FIG. 10).

[0120] The electronic device (440) can continuously perform the preparation process for model switching until the corresponding AI service is completed or the model switching event occurs, if a model switching event does not occur.

[0121] When a model switching event occurs, the electronic device (440) can switch to a model based on the on-device environment in operation 729 and continue to perform the inference service inherited from the senior model (411). For example, if the number of remaining layers is small for each threshold level, the electronic device (440) can perform inference corresponding to the remaining layers on a model completed by loading the dynamic model acquired for the senior model (411). For example, if the number of remaining layers is large for each threshold level, the electronic device (440) can perform inference corresponding to the remaining layers on a model completed by loading the dynamic model acquired for the successor model (446).

[0122] The electronic device (440) can determine, in operation 731, whether all inference processing for the corresponding AI service has been completed. If all inference processing for the corresponding AI service has been completed, the electronic device (440) can receive and output the inference processing result for the corresponding AI service from the cloud server (410) in operation 735. If all inference processing for the corresponding AI service has not been completed, the electronic device (440) can continue to perform inference processing for the corresponding AI service in the on-device environment in operation 729.

[0123] The electronic device (440), in operation 737, stores the successor model (446) already downloaded within the on-device so that the successor model (446) can be used for AI services in the future.

[0124] FIG. 8 is a diagram for explaining a knowledge distillation technique in a generative AI system (e.g., the generative AI system (400) of FIG. 4) according to one embodiment.

[0125] Referring to FIG. 8, knowledge distillation in a generative AI system (400) can be said to be a series of processes of extracting knowledge that may be valid in a small model (student model) (820) (e.g., successor model (446)) from knowledge learned by a large model (teacher model) (810) (e.g., senior model (411) of FIG. 4), and transferring (830) the extracted knowledge to the small model (820).

[0126] For example, a large model (810) may perform a distillation operation to extract knowledge from the learned knowledge so that the performance of a small model (820) can be maintained as much as possible. The large model (810) may create a pre-trained small model (820) based on the extracted knowledge. The large model (810) may deploy the created small model (820). The small model (820) may be an AI model that is relatively lightweight compared to the large model (810). The large model (810) may be, for example, a relatively complex AI model compared to the small model (820). The small model (820) may be, for example, a relatively simple AI model compared to the large model (810). Therefore, the prediction accuracy of the large model (810) by the inference service may be relatively higher than that of the small model (820), but the prediction time may be relatively longer than that of the small model (820). The small model (810) can perform a series of processes to obtain results similar to the results (e.g., class-specific probabilities) obtained by the softmax function used in the output layer, which is the last layer in the neural network of the large model (810), for example, based on the knowledge distillation technique. The softmax function is a function that normalizes all input values ​​into values ​​between 0 and 1 as outputs, and has the characteristic that the sum of the output values ​​is always 1.

[0127] FIG. 9 is a diagram for explaining model distribution in a generative AI system (e.g., the generative AI system (400) of FIG. 4) according to one embodiment.

[0128] Referring to FIG. 9, a server (910) of a generative AI system (400) (e.g., a cloud server (410) of FIG. 4) may generate a small model (e.g., a senior model (411) of FIG. 4 or a large model (810) of FIG. 8) (hereinafter, referred to as a “large model (810)”) through training using original data using a large model (e.g., a successor model (446) of FIG. 4 or a small model (820) of FIG. 8) (hereinafter, referred to as a “small model (820)”). The server (910) may distribute (930) the small model (820) generated by the large model (810) to an electronic device (920), such as a smartphone, having a relatively small processing capacity.

[0129] FIG. 10 is a diagram for explaining communication quality to be considered when switching models in a generative AI system (e.g., generative AI system (400) of FIG. 4) according to one embodiment.

[0130] Referring to Figure 10, communication quality (Q) can change frequently in a wireless communication environment. The figure assumes a graph (1000) in which communication quality (Q) decreases over time.

[0131] For example, in the section A (1010) up to the time point t1 when the communication quality (Q) in the graph (1000) reaches the first threshold level (th1), the communication quality (Q) is maintained at a level good enough to enable providing AI services in a cloud environment. Therefore, in the section A (1010), the electronic device (the electronic device (440) of FIG. 4) can receive AI services from a large model (e.g., the senior model (411) of FIG. 4 or the large model (810) of FIG. 8) mounted on a cloud server (e.g., the cloud server (410) of FIG. 4).

[0132] For example, in the graph (1000), the time interval (Δt) between t1 and t2, during which the communication quality (Q) is maintained between the first threshold level (th1) and the second threshold level (th2), section B (1020) may correspond to a situation in which the communication quality (Q) is capable of providing an AI service in a cloud environment but may worsen. Accordingly, in section B (1020), the electronic device (440) may prepare for a transition from a large model (810) mounted on the cloud server (410) to a small model (e.g., the successor model (446) of FIG. 4 or the small model (820) of FIG. 8). The transition to the small model (820) may be performed by, for example, receiving inference status information from the cloud server (410) according to processing of the AI ​​service in the large model (810). The inference state information transmitted from the cloud server (410) may include layer state information, bias state information, input token state information, or random state information regarding the large model (810).

[0133] For example, in section C (1030), which is after time t3 when the communication quality (Q) in the graph (1000) reaches the second threshold level (th2), the communication quality (Q) may be such that it is difficult to provide AI services in a cloud environment. Therefore, in section C (1030), the electronic device (440) may switch the generative AI model that performs inference for the remaining AI services from a large model (810) to a small model (820).

[0134] FIG. 11a is a diagram for explaining preparation for conversion of an AI model in a generative AI system (e.g., the generative AI system (400) of FIG. 4) according to one embodiment.

[0135] Referring to FIG. 11A, an electronic device (1120) (e.g., the electronic device (440) of FIG. 4) may, at operation 1130, request provision of a checkpoint from a server (1110) (e.g., the cloud server (410) of FIG. 4). For example, the electronic device (1120) may request provision of a checkpoint from the server (1110) in response to the occurrence of a model transition preparation event. The checkpoint may be an indicator indicating a status in which inference for an AI service has been performed in a large model (e.g., the senior model (411) of FIG. 4 or the large model (810) of FIG. 8) mounted on the server (1110).

[0136] The server (1110) may, in operation 1140, transmit inference state information to the electronic device (1120) in response to a request from the electronic device (1120) for synchronizing mutual checkpoints. The inference state information may include, for example, layer state information, bias state information, input token state information, or random state information regarding the large model (810). The inference state information may be used to determine whether to continue performing inference by the large model (810) in the next step or to allow the small model (820) to take over the inference. For example, if it is determined to continue performing inference by the large model (810), the inference state information may be utilized in the next inference step to be performed by the large model (810).

[0137] FIG. 11b is a diagram illustrating preparations for completing inference without switching an AI model in a generative AI system (e.g., the generative AI system (400) of FIG. 4) according to one embodiment.

[0138] Referring to FIG. 11B, the electronic device (1120) (e.g., the electronic device (440) of FIG. 4) may, in operation 1150, request provision of a dynamic model from the server (1110) (e.g., the cloud server (410) of FIG. 4). For example, if the number of remaining layers is less than a threshold level until the inference according to the corresponding AI service is completed in the large model (e.g., the senior model (411) of FIG. 4 or the large model (810) of FIG. 8) mounted on the server (1110), the electronic device (1120) may request some weight parameter information necessary to continue the inference by the large model (810) from the server (1110).

[0139] The server (1110) may, in operation 1160, transmit the dynamic model to the electronic device (1120) in response to a request from the electronic device (1120). The electronic device (1120) may, for example, download only some necessary weight parameter information of the large model (810) from the server (1110) so that inference by the large model (810) can continue to be performed.

[0140] FIG. 12a, FIG. 12b, FIG. 12c, FIG. 12d, or FIG. 12e are diagrams for explaining an exemplary scenario of providing an inference service by conversion of an AI model in a generative AI system (e.g., the generative AI system (400) of FIG. 4) according to one embodiment.

[0141] Referring to FIG. 12A, an electronic device (e.g., an electronic device (440) of FIG. 4) may access a server (e.g., a cloud server (410) of FIG. 4) based on a cloud environment to provide AI services such as real-time news summaries, real-time sentence generation, real-time writing assistants, real-time translations, or real-time image generation (1210). While the AI ​​services are provided by a generative AI model (e.g., a senior model (411) of FIG. 4) of the cloud server (410), the electronic device (440) may measure communication quality (e.g., throughput, latency, signal strength (RSSI)) (1211).

[0142] The electronic device (440) may guide the user to switch AI models when communication quality degrades to a level where it is difficult to provide network-based AI services. For example, the electronic device (440) may display a pop-up message on the screen saying, "The Internet environment is poor. We will continue the service without a cloud server!" (1213).

[0143] The electronic device (440) can switch to a generative AI model operating in an on-device environment (e.g., a successor model (446) of FIG. 4) to continue providing a user experience according to the AI ​​service provided in the cloud environment. For example, in a situation where the communication quality is good, the electronic device (440) can download an SLM to be used as the successor model (446) from the cloud server (410) or update the model based on knowledge distillation. When the electronic device (440) completes a task corresponding to the AI ​​service, it can output a message guiding the task status. For example, the electronic device (440) can output a pop-up message on the screen saying, "Task completed! Please note that this service only includes information up to n days ago!" (1215).

[0144] Referring to FIG. 12B, the electronic device (440) can provide an AI service, such as interpretation, to a user in a cloud environment or an on-device environment. For example, the electronic device (440) can display an input window (1221) and an output window (1224) for the interpretation service on a single screen (1220). The user can input a natural language (1222) (e.g., English) to be translated by pressing a microphone identifier (1223) provided in the input window (1221). If the communication quality is good, the electronic device (440) can provide the result (1225) of the interpretation service by a generative AI model (e.g., the lead model (411) of FIG. 4) supported in the cloud environment through the output window (1224). When the communication quality falls to a level where it is difficult to provide a network-based interpretation service, the electronic device (440) can switch to a generative AI model (e.g., the successor model (446) of FIG. 4) operating in an on-device environment to provide a seamless interpretation service to the user.

[0145] Referring to FIG. 12c, an electronic device (440) can provide an augmented reality (AR) service to a user at a travel destination (1230). The electronic device (440) can pre-download a generative AI model (e.g., the successor model (446) of FIG. 4) from a cloud server (410) when good communication quality is provided at the travel destination. If the communication quality degrades to a level where it is difficult to provide network-based AI services, the electronic device (440) can guide the user to switch AI models. For example, the electronic device (440) can display a pop-up message on the screen that says, "The Internet environment is poor. We will continue the service without the cloud server!" (1233).

[0146] The electronic device (440) can provide a user experience according to the AI ​​service by switching to a generative AI model that operates in an on-device environment that has been downloaded in advance. When the electronic device (440) completes a task corresponding to the AI ​​service, it can output a message guiding the task status. For example, the electronic device (440) can output a pop-up message on the screen saying, "This provides information about this monkey! It looks about how old it is~ It's a monkey of xx type..." (1235).

[0147] Referring to FIG. 12d, the electronic device (440) can use a map service to search for a route that a user (1243) will travel in advance, and predict the communication quality of the searched route to identify an area (1245) where AI service in a cloud environment is possible or a shaded area (1241) where AI service in a cloud environment is impossible (1240). The electronic device (440) can download in advance a generative AI model (e.g., a successor model (446) of FIG. 4) for providing an AI service based on the on-device environment in the shaded area (1241). Even if the communication quality suddenly deteriorates while the electronic device (440) is using an AI service in a cloud environment while moving, the electronic device (440) can load the dynamic model to provide a user experience according to the AI ​​service without interruption.

[0148] Referring to FIG. 12E, when climbing a high altitude where communication conditions may be unstable, the generative AI model to provide the AI ​​service may be switched based on the communication quality. For example, if the communication quality is good, a large model based on a cloud environment (e.g., the senior model (411) of FIG. 4 or the large model (810) of FIG. 8) may be selected as the generative AI model to provide the AI ​​service. For example, if the communication quality is poor, a small model based on an on-device environment (e.g., the successor model (446) of FIG. 4 or the small model (820) of FIG. 8) may be selected as the generative AI model to provide the AI ​​service.

[0149] For example, the electronic device (440) can monitor changes in communication quality based on at least one of throughput, latency, or received signal strength (RSSI). If the communication quality is maintained at a certain level (e.g., section B (1020) of FIG. 10) for a certain period of time, the electronic device (440) can download layer state information of an AI model for real-time route guidance from a server (e.g., cloud server (410) of FIG. 4).

[0150] The electronic device (440) can determine the number of remaining layers for performing inference for route guidance based on the downloaded layer status information. For example, if the number of remaining layers is below a critical level, the electronic device (440) can download only some dynamic models that can be additionally processed in the on-device environment from the cloud server (410) (see FIG. 11b). For example, if the number of remaining layers is above a critical level, the electronic device (440) can download knowledge distillation information from the cloud server (410) and prepare a pre-prepared small model (820) to take over the route guidance inference performed by the large model (810). When the communication quality falls to a poor level (e.g., section C (1030) of FIG. 10), the electronic device (440) can receive knowledge distillation information from a pre-prepared dynamic model or cloud server (410) and the prepared small model (820) can continue to process the inference for route guidance by taking over from the large model (810).

[0151] FIG. 13 is a block diagram of an electronic device (1301) (e.g., electronic device (440) of FIG. 4) within a network environment (1300) according to various embodiments.

[0152] Referring to FIG. 13, in a network environment (1300), an electronic device (1301) may communicate with an electronic device (1302) via a first network (1398) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (1304) or a server (1308) via a second network (1399) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (1301) may communicate with the electronic device (1304) via the server (1308). According to one embodiment, the electronic device (1301) may include a processor (1320), a memory (1330), an input module (1350), an audio output module (1355), a display module (1360), an audio module (1370), a sensor module (1376), an interface (1377), a connection terminal (1378), a haptic module (1379), a camera module (1380), a power management module (1388), a battery (1389), a communication module (1390), a subscriber identification module (1396), or an antenna module (1397). In some embodiments, the electronic device (1301) may omit at least one of these components (e.g., the connection terminal (1378)), or may have one or more other components added. In some embodiments, some of these components (e.g., sensor module (1376), camera module (1380), or antenna module (1397)) may be integrated into a single component (e.g., display module (1360)).

[0153] The processor (1320) may, for example, execute software (e.g., a program (1340)) to control at least one other component (e.g., a hardware or software component) of the electronic device (1301) connected to the processor (1320) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (1320) may store commands or data received from other components (e.g., a sensor module (1376) or a communication module (1390)) in a volatile memory (1332), process the commands or data stored in the volatile memory (1332), and store result data in a non-volatile memory (1334). According to one embodiment, the processor (1320) may include a main processor (1321) (e.g., a central processing unit or an application processor) or a secondary processor (1323) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (1321). For example, when the electronic device (1301) includes the main processor (1321) and the secondary processor (1323), the secondary processor (1323) may be configured to use less power than the main processor (1321) or to be specialized for a given function. The secondary processor (1323) may be implemented separately from the main processor (1321) or as a part thereof.

[0154] The auxiliary processor (1323) may control at least a portion of functions or states associated with at least one component (e.g., the display module (1360), the sensor module (1376), or the communication module (1390)) of the electronic device (1301), for example, on behalf of the main processor (1321) while the main processor (1321) is in an inactive (e.g., sleep) state, or together with the main processor (1321) while the main processor (1321) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (1323) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (1380) or a communication module (1390)). In one embodiment, the auxiliary processor (1323) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1301) where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1308)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0155] The memory (1330) can store various data used by at least one component (e.g., the processor (1320) or the sensor module (1376)) of the electronic device (1301). The data can include, for example, software (e.g., the program (1340)) and input data or output data for commands related thereto. The memory (1330) can include volatile memory (1332) or non-volatile memory (1334).

[0156] The program (1340) may be stored as software in memory (1330) and may include, for example, an operating system (1342), middleware (1344), or an application (1346).

[0157] The input module (1350) can receive commands or data to be used in a component of the electronic device (1301) (e.g., a processor (1320)) from an external source (e.g., a user) of the electronic device (1301). The input module (1350) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0158] The audio output module (1355) can output audio signals to the outside of the electronic device (1301). The audio output module (1355) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0159] The display module (1360) can visually provide information to an external device (e.g., a user) of the electronic device (1301). The display module (1360) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (1360) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0160] The audio module (1370) can convert sound into an electrical signal, or vice versa. According to one embodiment, the audio module (1370) can acquire sound through the input module (1350), output sound through the sound output module (1355), or an external electronic device (e.g., electronic device (1302)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (1301).

[0161] The sensor module (1376) can detect the operating status (e.g., power or temperature) of the electronic device (1301) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (1376) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0162] The interface (1377) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (1301) with an external electronic device (e.g., the electronic device (1302)). In one embodiment, the interface (1377) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0163] The connection terminal (1378) may include a connector through which the electronic device (1301) may be physically connected to an external electronic device (e.g., the electronic device (1302)). According to one embodiment, the connection terminal (1378) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0164] The haptic module (1379) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (1379) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0165] The camera module (1380) can capture still images and videos. According to one embodiment, the camera module (1380) may include one or more lenses, image sensors, image signal processors, or flashes.

[0166] The power management module (1388) can manage power supplied to the electronic device (1301). According to one embodiment, the power management module (1388) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0167] A battery (1389) may power at least one component of the electronic device (1301). In one embodiment, the battery (1389) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0168] The communication module (1390) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (1301) and an external electronic device (e.g., electronic device (1302), electronic device (1304), or server (1308)), and the performance of communication through the established communication channel. The communication module (1390) may operate independently from the processor (1320) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1390) may include a wireless communication module (1392) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (1394) (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device (1304) via a first network (1398) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1399) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a local area network or a wide area network)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1392) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (1396) to verify or authenticate the electronic device (1301) within a communication network such as the first network (1398) or the second network (1399).

[0169] The wireless communication module (1392) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimizing terminal power and connecting multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1392) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (1392) can support various technologies for securing performance in high-frequency bands, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1392) can support various requirements specified in the electronic device (1301), an external electronic device (e.g., the electronic device (1304)), or a network system (e.g., the second network (1399)). According to one embodiment, the wireless communication module (1392) may support a peak data rate (e.g., 20 Gbps or more) for eMBB implementation, a loss coverage (e.g., 164 dB or less) for mMTC implementation, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL) each, or 1 ms or less for round trip) for URLLC implementation.

[0170] The antenna module (1397) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (1397) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (1397) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (1398) or the second network (1399), may be selected from the plurality of antennas by, for example, the communication module (1390). A signal or power may be transmitted or received between the communication module (1390) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (1397).

[0171] According to various embodiments, the antenna module (1397) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0172] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0173] According to one embodiment, commands or data may be transmitted or received between the electronic device (1301) and an external electronic device (1304) via a server (1308) connected to a second network (1399). Each of the external electronic devices (1302 or 1304) may be the same or a different type of device as the electronic device (1301). According to one embodiment, all or part of the operations executed in the electronic device (1301) may be executed in one or more of the external electronic devices (1302, 1304, or 1308). For example, when the electronic device (1301) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1301) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1301). The electronic device (1301) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (1301) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (1304) may include an Internet of Things (IoT) device. The server (1308) may be an intelligent server utilizing machine learning and / or a neural network.In one embodiment, an external electronic device (1304) or server (1308) may be included in the second network (1399). The electronic device (1301) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0174] Although not included in the scenario described above, if a user wearing AR glasses receives a real-time surrounding image generation service based on a cloud environment and then enters a location with poor network communication conditions, such as underground, the model transition method proposed in this document may be utilized to provide uninterrupted service.

[0175] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary knowledge in the technical field to which the present disclosure pertains.

[0176] According to an example, the electronic device (440) may include a communication circuit (160). The electronic device (440) may include a memory (120) including one or more storage media for storing instructions. The electronic device (440) may include at least one processor (110) including a processing circuit. When the instructions are individually or collectively executed by the at least one processor (110), the instructions may cause the electronic device (100) to perform at least one operation. The at least one operation may include obtaining a second AI model (446) that is pre-trained and deployed by a first artificial intelligence (AI) model (411) of a cloud server (410) through the communication circuit (160). The above at least one operation may include an operation in which an inferencing process is performed by the first AI model (411), and the inferencing process being performed by the first AI model (411) is inherited and continuously processed by the second AI model (446) based on the communication quality with the cloud server (410).

[0177] In one example, when the instructions are individually or collectively executed by at least one processor (210), they may cause the electronic device (100) to perform an operation of updating the second AI model (446) based on knowledge distillation in the first AI model (411) while the inference process is performed in the first AI model (411).

[0178] For example, when the instructions are individually or collectively executed by at least one processor (210), they may cause the electronic device (100) to perform an operation of receiving, from the cloud server (410), inference status information regarding the inference process being performed in the first AI model (411).

[0179] According to an example, the inference state information may include layer state information, bias state information, input token state information, or random state information regarding the first AI model (411).

[0180] For example, when the instructions are individually or collectively executed by at least one processor (210), the electronic device (100) may be caused to perform an operation of receiving information about layers synchronized and mapped for inference succession from the cloud server (410) and preparing to drive the second AI model (446) if the communication quality is maintained within a first threshold range for a specific period of time and the number of layers remaining until the inference process by the first AI model (411) is completed is greater than a specific number.

[0181] According to an example, when the instructions are individually or collectively executed by at least one processor (210), the electronic device (100) may be caused to perform an operation of downloading information about weight parameters for continuing the inference process by the first AI model (411) from the cloud server (410) if the number of layers remaining until the inference process by the first AI model (411) is completed is equal to or less than a specific number.

[0182] For example, when the instructions are individually or collectively executed by at least one processor (210), they may cause the electronic device (100) to perform an operation of obtaining the communication quality based on at least one of throughput, latency, or received signal strength (RSSI).

[0183] According to one example, a storage medium (120) storing computer-readable instructions may cause the electronic device (440) to perform at least one operation when the instructions are executed by at least a part of at least one processor (110) of the electronic device (440). The at least one operation may include an operation of acquiring a second AI model (446) that has been pre-trained and deployed by a first artificial intelligence (AI) model (411) of a cloud server (410). The at least one operation may include an operation of performing an inferencing process by the first AI model (411) and taking over and continuously processing the inferencing process being performed by the first AI model (411) based on a communication quality with the cloud server (410).

[0184] In one example, the at least one operation may include updating the second AI model (446) based on knowledge distillation in the first AI model (411) while the inference process is performed in the first AI model (411).

[0185] According to an example, the at least one operation may include an operation of receiving inference status information regarding the inference process being performed in the first AI model (411) from the cloud server (410).

[0186] According to an example, the inference state information may include layer state information, bias state information, input token state information, or random state information regarding the first AI model (411).

[0187] According to an example, the at least one operation may include an operation of preparing to drive the second AI model (446) by receiving information about layers synchronized and mapped for inference succession from the cloud server (410) when the communication quality is maintained within a first threshold range for a specific period of time and the number of layers remaining until the inference process by the first AI model (411) is completed is greater than a specific number.

[0188] According to an example, the at least one operation may include an operation of downloading information about weight parameters for continuing the inference process by the first AI model (411) from the cloud server (410) if the number of layers remaining until the inference process by the first AI model (411) is completed is less than or equal to a specific number.

[0189] In one example, the at least one operation may include an operation of obtaining the communication quality based on at least one of throughput, latency, or received signal strength (RSSI).

[0190] According to an example, the operating method of the electronic device (440) may include an operation of acquiring a second AI model (446) that has been pre-trained and deployed by a first artificial intelligence (AI) model (411) of a cloud server (410). The operating method may include an operation of performing an inferencing process by the first AI model (411) and then taking over and continuously processing the inferencing process being performed by the first AI model (411) based on the communication quality with the cloud server (410) by the second AI model (446).

[0191] According to an example, the method of operation may include an operation of updating the second AI model (446) based on knowledge distillation in the first AI model (411) while the inference process is performed in the first AI model (411).

[0192] According to an example, the operating method may include an operation of receiving inference status information regarding the inference process being performed in the first AI model (411) from the cloud server (410).

[0193] According to an example, the inference state information may include layer state information, bias state information, input token state information, or random state information regarding the first AI model (411).

[0194] According to an example, the operating method may include an operation of preparing to drive the second AI model (446) by receiving information about layers synchronized and mapped for inference succession from the cloud server (410) when the communication quality is maintained within a first threshold range for a specific period of time and the number of layers remaining until the inference process by the first AI model (411) is completed is greater than a specific number.

[0195] According to an example, the method of operation may include an operation of obtaining the communication quality based on at least one of throughput, latency, or received signal strength (RSSI).

[0196] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains.

[0197] It should be understood that the embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to a specific embodiment, but rather to encompass various modifications, equivalents, or substitutes of the embodiment. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the item, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0198] The term "module" used in one embodiment of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0199] An embodiment of the present document may be implemented as software including one or more instructions stored in a storage medium (e.g., memory (120)) readable by a machine (e.g., electronic device (440)). For example, a processor (e.g., processor (110)) of the machine (e.g., electronic device (440)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0200] According to one embodiment, the method according to one embodiment disclosed in the present document may be provided as a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0201] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (440), Communication circuit (160); A memory (120) including one or more storage media for storing instructions; and At least one processor (110) including a processing circuit, When the above instructions are executed individually or collectively by at least one processor (110), they cause the electronic device (440) to perform at least one operation, At least one of the above actions, An operation of acquiring a second AI model (446) that has been pre-trained and deployed by a first artificial intelligence (AI) model (411) of a cloud server (410) through the above communication circuit (160); and An operation in which an inferencing process is performed by the first AI model (411), and the inferencing process being performed by the first AI model (411) is inherited and continuously processed by the second AI model (446) based on the communication quality with the cloud server (410). An electronic device (440) comprising:

2. In paragraph 1, When the above instructions are individually or collectively executed by at least one processor (210), the electronic device (440) causes: While the inference process is performed in the first AI model (411), an operation of updating the second AI model (446) based on knowledge distillation in the first AI model (411) An electronic device (440) that causes the device to perform a function.

3. In paragraph 1, When the above instructions are individually or collectively executed by at least one processor (210), the electronic device (440) causes: An operation of receiving inference status information regarding the inference process being performed in the first AI model (411) from the cloud server (410). An electronic device (440) that causes the device to perform a function.

4. In paragraph 3, An electronic device (440) wherein the above-mentioned inference state information includes layer state information, bias state information, input token state information, or random state information regarding the first AI model (411).

5. In paragraph 3 or 4, When the above instructions are individually or collectively executed by at least one processor (210), the electronic device (440) causes: If the above communication quality is maintained for a specific period of time within the first critical range and the number of layers remaining until the inference process by the first AI model (411) is completed is greater than a specific number, an operation of receiving information about layers synchronized and mapped for inference succession from the cloud server (410) and preparing to drive the second AI model (446) An electronic device (440) that causes the device to perform a function.

6. In any one of paragraphs 1 to 5, When the above instructions are individually or collectively executed by at least one processor (210), the electronic device (440) causes: If the number of layers remaining until the inference process by the first AI model (411) is completed is less than or equal to a specific number, an operation of downloading information about weight parameters for continuing the inference process by the first AI model (411) from the cloud server (410). An electronic device (440) that causes the device to perform a function.

7. In any one of paragraphs 1 to 6, When the above instructions are individually or collectively executed by at least one processor (210), the electronic device (440) causes: An operation of obtaining the communication quality based on at least one of throughput, latency, or received signal strength (RSSI). An electronic device (440) that causes the device to perform a function.

8. In the operating method of the electronic device (440), An operation of acquiring a second AI model (446) that has been pre-trained and deployed by a first artificial intelligence (AI) model (411) of a cloud server (410); and An operation in which an inferencing process is performed by the first AI model (411), and the inferencing process being performed by the first AI model (411) is inherited and continuously processed by the second AI model (446) based on the communication quality with the cloud server (410). A method comprising:

9. In paragraph 8, A method comprising an operation of updating the second AI model (446) based on knowledge distillation in the first AI model (411) while the inference process is performed in the first AI model (411).

10. In paragraph 8, A method including an operation of receiving inference status information regarding the inference process being performed in the first AI model (411) from the cloud server (410).

11. In paragraph 10, A method in which the above inference state information includes layer state information, bias state information, input token state information, or random state information regarding the first AI model (411).

12. In paragraph 10 or 11, A method comprising an operation of preparing to drive the second AI model (446) by receiving information about layers synchronized and mapped for inference succession from the cloud server (410) when the communication quality is maintained for a specific period of time within the first critical range and the number of layers remaining until the inference process by the first AI model (411) is completed is greater than a specific number.

13. In any one of paragraphs 8 to 12, A method including an operation of downloading information on weight parameters for continuing the inference process by the first AI model (411) from the cloud server (410) when the number of layers remaining until the inference process by the first AI model (411) is completed is less than or equal to a specific number.

14. In any one of paragraphs 8 to 13, A method comprising an operation of obtaining the communication quality based on at least one of throughput, latency, or received signal strength (RSSI).

15. In a storage medium (120) that stores instructions that can be read by a computer, The above instructions, when executed by at least a portion of at least one processor (110) of the electronic device (440), cause the electronic device (440) to perform at least one operation; At least one of the above actions: An operation of acquiring a second AI model (446) that has been pre-trained and deployed by a first artificial intelligence (AI) model (411) of a cloud server (410); and An operation in which an inferencing process is performed by the first AI model (411), and the inferencing process being performed by the first AI model (411) is inherited and continuously processed by the second AI model (446) based on the communication quality with the cloud server (410). A storage medium (120) including:

Images