Distributed artificial intelligence service splitting and handover decision mechanism combining user equipment capability and network conditions

Abstract

In one aspect of the disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus can be a device. The device evaluates processing requirements of an Artificial Intelligence (AI) service, resources of a local device, resources of a server, and network connectivity resources between the local device and the server. The device splits the AI service into multiple parts based on the evaluation results and a set of decision criteria for distributed AI splitting and switching. The set of decision criteria includes privacy requirements, real-time response requirements, and quality-latency tradeoffs. The device switches respective split parts of the AI service between the local device and the server based on the evaluation results and the set of decision criteria.

Description

[0001] Cross-referencing

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 63 / 709,639, entitled “A Distributed AI Service Splitting Decision-Making Approach Based on User Equipment Capabilities and Network Conditions,” filed October 21, 2024, the entire contents of which are hereby expressly incorporated by reference. Technical Field

[0003] This disclosure generally relates to wireless communications, and more specifically, to techniques for making decisions on the splitting and switching of distributed artificial intelligence (AI) services, taking into account user equipment capabilities and network conditions. Background Technology

[0004] The statements in this section are provided only as background information in relation to this disclosure and may not constitute prior art.

[0005] Wireless communication systems are widely deployed to provide a variety of telecommunications services such as telephone, video, data, messaging, and broadcasting. Typical wireless communication systems employ multiple access technologies that enable communication with multiple users by sharing available system resources. Examples of such multiple access technologies include code division multiple access (CDMA) systems, time division multiple access (TDMA) systems, frequency division multiple access (FDMA) systems, orthogonal frequency division multiple access (OFDMA) systems, single-carrier frequency division multiple access (SC-FDMA) systems, and time division synchronous code division multiple access (TD-SCDMA) systems.

[0006] These multiple access technologies have been adopted in various telecommunications standards to provide a common protocol enabling different wireless devices to communicate at the municipal, national, regional, and even global levels. An example telecommunications standard is 5G New Radio (NR). 5G NR is part of the Continuous Mobile Broadband Evolution program (3GPP) to meet new requirements related to latency, reliability, security, scalability (e.g., the Internet of Things (IoT),) and other requirements. Some aspects of 5G NR can be based on the 4G Long Term Evolution (LTE) standard. There is a need for further improvements in 5G NR technology. These improvements can also be applied to other multiple access technologies and telecommunications standards that adopt them. Summary of the Invention

[0007] The following content presents one or more aspects in a simplified form to provide a basic understanding of these aspects. This summary is not a comprehensive overview of all hypothetical aspects, nor is it intended to identify key or important elements of all aspects, nor is it intended to define the scope of any or all aspects. Its sole purpose is to present certain concepts of one or more aspects in a simplified form as a prelude to a more detailed description thereafter.

[0008] In one aspect of this disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus can be a device. The device assesses the processing requirements of an artificial intelligence (AI) service, the resources of a local device, the resources of a server, and the network connectivity resources between the local device and the server. Based on the assessment results and a set of decision criteria for distributed AI splitting and switching, the device splits the AI ​​service into multiple parts. The set of decision criteria includes privacy requirements, real-time response requirements, and quality-latency tradeoffs. Based on the assessment results and the set of decision criteria, the device switches the appropriate split parts of the AI ​​service between the local device and the server.

[0009] To achieve the foregoing and related objectives, one or more aspects include the features described in detail below and specifically pointed out in the claims. The following description and accompanying drawings illustrate certain exemplary features of one or more aspects. However, these features only illustrate a portion of how the principles of the various aspects can be applied, and this description is intended to cover all such aspects and their equivalents. Attached Figure Description

[0010] Figure 1 The diagram illustrates an example of a wireless communication system and access network.

[0011] Figure 2 This diagram illustrates the communication between a base station and a user equipment (UE) in the access network.

[0012] Figure 3 An example logical architecture for a distributed access network is shown.

[0013] Figure 4 An example physical architecture of a distributed access network is shown.

[0014] Figure 5 The diagram illustrates the deployment architecture of virtual and human sales assistant mixed reality services in the retail industry.

[0015] Figure 6 The illustration shows the design of a mixed reality service featuring virtual and personal sales assistants.

[0016] Figure 7 The diagram illustrates three types of collaborative AI personal networks (PNs) with distributed generative AI.

[0017] Figure 8 The diagram illustrates the collaborative AI Personal Network (PN) between the in-body PN and the family PN.

[0018] Figure 9 The illustration shows a scenario of an "independent PN AI assistant".

[0019] Figure 10 The diagram illustrates a "distributed PN AI assistant" scenario.

[0020] Figure 11 This is an illustration showing an "information sharing" scenario.

[0021] Figure 12 The diagram illustrates a "knowledge synchronization" scenario.

[0022] Figure 13 The illustration shows a "fall detection" scenario.

[0023] Figure 14A -B is a schematic diagram illustrating the distributed AI splitting and switching decision mechanism, as well as network analysis between user devices and the service network.

[0024] Figure 15 The diagram illustrates Retrieval Augmented Generation (RAG).

[0025] Figure 16 The diagram illustrates the processes of active / passive access technology and active / passive network modes.

[0026] Figure 17 The diagram illustrates the parameters and API concepts for user equipment analysis.

[0027] Figure 18The flowchart illustrates the decision-making process for splitting and switching distributed AI services. Detailed Implementation

[0028] The detailed description below, taken in conjunction with the accompanying drawings, is intended to describe various configurations and is not intended to represent the only configuration in which the concepts described herein can be implemented. The detailed description includes specific details to provide a thorough understanding of the various concepts. However, those skilled in the art will understand that these concepts can be implemented without these specific details. In some cases, to avoid obscuring these concepts, known structures and components are shown in block diagram form.

[0029] Several aspects of telecommunications systems will now be introduced in conjunction with various apparatuses and methods. These apparatuses and methods will be described in detail below and illustrated with accompanying drawings as various modules, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements can be implemented by electronic hardware, computer software, or any combination thereof. Whether an element is implemented in hardware or software depends on the specific application and design constraints imposed on the overall system.

[0030] For example, a single element, any portion of an element, or any combination of elements may be implemented as a “processing system” comprising one or more processors. Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, system-on-a-chip (SoCs), baseband processors, field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functions described in this disclosure. One or more processors in the processing system can execute software. Software should be understood broadly as instructions, instruction sets, code, code segments, program code, programs, subroutines, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., regardless of whether it is referred to as software, firmware, middleware, microcode, hardware description languages, or other names.

[0031] Therefore, in one or more example aspects, the functionality can be implemented by hardware, software, or any combination thereof. If implemented by software, these functions can be stored or encoded as one or more instructions or codes stored on a computer-readable medium. Computer-readable media include computer storage media. Storage media can be any existing medium accessible to a computer. For example, but not limited to, such computer-readable media can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), optical disc storage, magnetic disk storage, other magnetic storage devices, combinations of computer-readable media of the above types, or any medium that can be used to store computer-executable code in the form of instructions or data structures and is accessible to a computer.

[0032] Figure 1 The illustration shows an example of a wireless communication system and access network 100. The wireless communication system (also known as a wireless wide area network (WWAN)) includes a base station 102, user equipment (UE) 104, an evolved packet core network (EPC) 160, and another core network 190 (e.g., a 5G core network (5GC)). Base station 102 may include macrocells (high-power cellular base stations) and / or small cells (low-power cellular base stations). Macrocells include base stations. Small cells include home microcells, picocells, and microcells.

[0033] Base station 102 configured as 4G LTE (collectively referred to as Evolved Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (E-UTRAN)) can be connected to EPC 160 via backhaul link 132 (e.g., SI interface). Base station 102 configured as 5G NR (collectively referred to as Next Generation Radio Access Network (NG-RAN)) can be connected to core network 190 via backhaul link 184. Among other functions, base station 102 can perform one or more of the following functions: user data transmission, radio channel encryption and decryption, integrity protection, header compression, mobility control functions (e.g., handover, dual connectivity), inter-cell interference coordination, connection establishment and release, load balancing, non-access stratum (NAS) message distribution, NAS node selection, synchronization, radio access network (RAN) sharing, multimedia broadcast multicast service (MBMS), user and device tracking, RAN information management (RIM), paging, location, and warning message delivery. Base stations 102 can communicate with each other directly or indirectly (e.g., via EPC 160 or core network 190) via backhaul link 134 (e.g., X2 interface). Backhaul link 134 can be wired or wireless.

[0034] Base station 102 can wirelessly communicate with UE 104. Each base station 102 can provide communication coverage for a corresponding geographic coverage area 110. Overlapping geographic coverage areas 110 may exist. For example, small cell 102' may have a coverage area 110' that overlaps with the coverage areas 110 of one or more macro base stations 102. A network containing small cells and macro cells may be referred to as a heterogeneous network. Heterogeneous networks may also include evolved home nodes (eNBs) (HeNBs) that can provide service to a restricted group called a closed subscriber group (CSG). The communication link 120 between base station 102 and UE 104 may include uplink (UL) (also known as reverse link) transmission from UE 104 to base station 102 and / or downlink (DL) (also known as forward link) transmission from base station 102 to UE 104. Communication link 120 may employ multiple-input multiple-output (MIMO) antenna technologies, including spatial multiplexing, beamforming, and / or transmit diversity. The communication link may be implemented via one or more carriers. Base station 102 / UE 104 can use spectrum with a bandwidth of up to 7 MHz per carrier (e.g., 5, 10, 15, 20, 100, 400 MHz, etc.) to allocate a total of Yx MHz (x component carriers) for transmission in carrier aggregation in each direction. Carriers may be adjacent or non-adjacent. Carrier allocation may be asymmetrical in the DL and UL directions (e.g., more or fewer carriers allocated to DL than to UL). Component carriers may include primary component carriers and one or more secondary component carriers. The primary component carrier may be referred to as the primary cell (PCell), and the secondary component carriers may be referred to as secondary cells (SCells).

[0035] Some UEs 104 can communicate with each other via device-to-device (D2D) communication link 158. D2D communication link 158 can use DL / UL WWAN spectrum. D2D communication link 158 can use one or more sidelink channels, such as Physical Sidelink Broadcast Channel (PSBCH), Physical Sidelink Discovery Channel (PSDCH), Physical Sidelink Shared Channel (PSSCH), and Physical Sidelink Control Channel (PSCCH). D2D communication can be implemented through various wireless D2D communication systems, such as FlashLinQ, WiMedia, Bluetooth, ZigBee, Wi-Fi based on the IEEE 802.11 standard, LTE, or NR.

[0036] The wireless communication system may also include a Wi-Fi access point (AP) 150 that communicates with the Wi-Fi station (STA) 152 via a communication link 154 in the 5 GHz unlicensed spectrum. When communicating in the unlicensed spectrum, the STA 152 / AP 150 may perform a clear channel assessment (CCA) before communication to determine whether the channel is available.

[0037] Small cell 102' can operate in licensed and / or unlicensed spectrum. When operating in unlicensed spectrum, small cell 102' can employ a new radio interface (NR) and use the same 5 GHz unlicensed spectrum as Wi-Fi access point 150. Employing NR in unlicensed spectrum can improve the coverage and / or increase the capacity of the access network.

[0038] Base station 102, whether a small cell 102' or a large cell (e.g., a macro base station), may include evolved NodeBs (eNBs), gNodeBs (gNBs), or other types of base stations. Some base stations, such as gNB 180, can communicate with user equipment (UE) 104 in conventional sub-6 GHz spectrum, millimeter wave (mmW) frequencies, and / or near-millimeter wave frequencies. When gNB 180 operates at millimeter wave or near-millimeter wave frequencies, gNB 180 may be referred to as a millimeter wave base station. Extremely high frequency (EHF) belongs to the radio frequency (RF) portion of the electromagnetic spectrum. The EHF range is from 30 GHz to 300 GHz, with wavelengths between 1 mm and 10 mm. Radio waves in this band may be referred to as millimeter waves. Near-millimeter waves extend down to 3 GHz with wavelengths of 100 mm. Ultra-high frequency (SHF) bands are between 3 GHz and 30 GHz, also known as centimeter waves. Communication using millimeter wave / near-millimeter wave RF bands (e.g., 3 GHz - 300 GHz) has extremely high path loss and short range. The millimeter-wave base station 180 can communicate with the UE 104 using beamforming 182 to compensate for extremely high path loss and short distance.

[0039] Base station 180 can transmit beamforming signals to UE 104 in one or more transmit directions 108a. UE 104 can receive beamforming signals from base station 180 in one or more receive directions 108b. UE 104 can also transmit beamforming signals to base station 180 in one or more transmit directions. Base station 180 can receive beamforming signals from UE 104 in one or more receive directions. Base station 180 / UE 104 can perform beam training to determine their respective optimal receive and transmit directions. The transmit and receive directions of base station 180 can be the same or different. The transmit and receive directions of UE 104 can be the same or different.

[0040] The evolved packet core (EPC) 160 may include a mobility management entity (MME) 162, other MMEs 164, a serving gateway 166, a multimedia broadcast multicast service (MBMS) gateway 168, a broadcast multicast service center (BM-SC) 170, and a packet data network (PDN) gateway 172. MME 162 can communicate with the Home Subscriber Server (HSS) 174. MME 162 is the control node that handles signaling between UE 104 and EPC 160. Typically, MME 162 provides bearer and connection management. All user Internet Protocol (IP) packets are transmitted through the serving gateway 166, which is itself connected to the PDN gateway 172. The PDN gateway 172 provides UE IP address allocation and other functions. The PDN gateway 172 and BM-SC 170 are connected to the IP service 176. The IP service 176 may include the Internet, intranets, IP Multimedia Subsystem (IMS), packet-switched (PS) streaming services, and / or other IP services. The BM-SC170 provides configuration and delivery functions for MBMS user services. The BM-SC170 can serve as an entry point for MBMS transmissions by content providers, authorizing and initiating MBMS bearer services within a Public Land Mobile Network (PLMN), and scheduling MBMS transmissions. The MBMS gateway 168 can distribute MBMS traffic to base station 102, which belongs to a Multicast-Broadcast Single Frequency Network (MBSFN) area and broadcasts specific services, and is responsible for session management (start / stop) and collecting billing information related to enhanced MBMS.

[0041] The core network 190 may include Access and Mobility Management Functions (AMF) 192, other AMFs 193, Location Management Functions (LMF) 198, Session Management Functions (SMF) 194, and User Plane Functions (UPF) 195. AMF 192 can communicate with Unified Data Management (UDM) 196. AMF 192 is the control node that handles signaling between UE 104 and the core network 190. Typically, SMF 194 provides QoS flow and session management. All user Internet Protocol (IP) packets are transmitted through UPF 195. UPF 195 provides UE IP address allocation and other functions. UPF 195 connects to IP service 197. IP service 197 may include the Internet, intranet, IP Multimedia Subsystem (IMS), packet-switched (PS) streaming service, and / or other IP services.

[0042] A base station may also be referred to as a gNB, Node B, evolved Node B (eNB), access point, base transceiver, wireless base station, wireless transceiver, transceiver function, basic services set (BSS), extended services set (ESS), transmit / receive point (TRP), or other applicable terms. Base station 102 provides UE 104 with access to EPC 160 or core network 190. Examples of UE 104 include cellular phones, smartphones, Session Initiation Protocol (SIP) phones, laptops, personal digital assistants (PDAs), satellite broadcasting, GPS, multimedia devices, video devices, digital audio players (such as MP3 players), cameras, game consoles, tablets, smart devices, wearable devices, vehicles, electricity meters, gas pumps, large or small kitchen appliances, medical devices, implants, sensors / actuators, displays, or any other similarly functional devices. Some UE 104 devices may be referred to as Internet of Things (IoT) devices (such as parking meters, gas pumps, toasters, vehicles, heart monitors, etc.). UE 104 may also be referred to as site, mobile station, user station, mobile unit, user unit, radio unit, remote unit, mobile device, radio device, wireless communication device, remote device, mobile user station, access terminal, mobile terminal, radio terminal, remote terminal, handheld device, user agent, mobile client, client or other applicable terms.

[0043] Although this disclosure may relate to 5G New Radio (NR), it may also be applied to other similar fields, such as LTE, LTE-Advanced (LTE-A), Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM) or other wireless / radio frequency access technologies.

[0044] Figure 2This is a block diagram illustrating communication between a base station 210 and a user equipment (UE) 250 in an access network. In the downlink, IP packets from the evolved packet core (EPC) 160 can be provided to the controller / processor 275. The controller / processor 275 implements Layer 3 and Layer 2 functions. Layer 3 includes the radio resource control (RRC) layer, and Layer 2 includes the packet data convergence protocol (PDCP) layer, the radio link control (RLC) layer, and the medium access control (MAC) layer. The controller / processor 275 provides RRC layer functions related to system information (e.g., Master Information Block (MIB), System Information Blocks (SIBs)) broadcasting, RRC connection control (e.g., RRC connection paging, RRC connection establishment, RRC connection modification, and RRC connection release), radio access technology (RAT) mobility, and measurement configuration for UE measurement reporting; PDCP layer functions related to header compression / decompression, security (encryption, decryption, integrity protection, integrity verification), and handover support functions; RLC layer functions related to upper layer packet data unit (PDU) transmission, error correction via Automatic Repeat Request (ARQ), RLC Service Data Unit (SDU) splicing, segmentation, and reassembly, RLC data PDU resegmentation, and RLC data PDU reordering; and mapping between logical channels and transport channels, multiplexing of MAC SDUs on transport blocks (TBs), and MAC... SDU performs MAC layer functions related to demultiplexing on TB, scheduling information reporting, error correction via Hybrid Automatic Repeat Request (HARQ), priority processing, and logical channel priority ordering.

[0045] Transmit (TX) processor 216 and receive (RX) processor 270 implement Layer 1 functions related to various signal processing functions. Layer 1 includes the physical (PHY) layer, which may include error detection of the transport channel, forward error correction (FEC) encoding / decoding of the transport channel, interleaving, rate matching, mapping to the physical channel, modulation / demodulation of the physical channel, and multiple-input multiple-output (MIMO) antenna processing. TX processor 216 processes signal constellation mapping according to various modulation schemes (e.g., binary phase shift keying (BPSK), quadrature phase shift keying (QPSK), M-order phase shift keying (M-PSK), M-order quadrature amplitude modulation (M-QAM)). The encoded and modulated symbols can then be split into parallel streams. Each stream can then be mapped to an orthogonal frequency division multiplexing (OFDM) subcarrier, multiplexed with a reference signal (e.g., a pilot) in the time and / or frequency domains, and then combined via inverse fast Fourier transform (IFFT) to generate a physical channel carrying the time-domain OFDM symbol stream. The OFDM stream is spatially precoded to generate multiple spatial streams. Channel estimates from channel estimator 274 are used to determine coding and modulation schemes, as well as spatial processing. The channel estimates can be derived from a reference signal and / or channel state feedback transmitted by UE 250. Each spatial stream can then be provided to a different antenna 220 via a separate transmitter 218TX. Each transmitter 218TX can be transmitted using a radio frequency (RF) carrier modulated with the corresponding spatial stream.

[0046] At UE 250, each receiver 254RX receives signals through its corresponding antenna 252. Each receiver 254RX recovers the information modulated onto the RF carrier and provides the information to the receive (RX) processor 256. The TX processor 268 and RX processor 256 implement Layer 1 functions related to various signal processing functions. The RX processor 256 can perform spatial processing on the information to recover any spatial stream directed towards UE 250. If there are multiple spatial streams directed towards UE 250, they can be merged by the RX processor 256 into a single OFDM symbol stream. The RX processor 256 then uses a Fast Fourier Transform (FFT) to transform the OFDM symbol stream from the time domain to the frequency domain. The frequency domain signal consists of a separate OFDM symbol stream for each OFDM signal subcarrier. The symbols on each subcarrier, along with the reference signal, are recovered and demodulated by determining the most probable signal constellation point transmitted by base station 210. These soft decisions can be based on the channel estimate calculated by channel estimator 258. The soft decision is then decoded and deinterleaved to recover the data and control signals initially transmitted by base station 210 on the physical channel. The data and control signals are then provided to controller / processor 259, which implements Layer 3 and Layer 2 functions.

[0047] Controller / processor 259 may be associated with memory 260, which stores program code and data. Memory 260 may be referred to as computer-readable medium. In the uplink, controller / processor 259 provides demultiplexing, packet reassembly, decryption, header decompression, and control signal processing between transport and logical channels to recover IP packets from EPC 160. Controller / processor 259 is also responsible for error detection using acknowledgment (ACK) and / or denial (NACK) protocols to support HARQ operation.

[0048] Similar to the downlink transmission functions described in base station 210, controller / processor 259 provides RRC layer functions related to system information (e.g., MIB, SIBs) acquisition, RRC connection, and measurement reporting; PDCP layer functions related to header compression / decompression and security (encryption, decryption, integrity protection, integrity verification); RLC layer functions related to upper-layer PDU transmission, error correction via ARQ, RLCSDU splicing, segmentation, and reassembly, RLC data PDU resegmentation, and RLC data PDU reordering; and MAC layer functions related to mapping between logical channels and transport channels, multiplexing MAC SDUs on TB, demultiplexing MAC SDUs from TB, scheduling information reporting, error correction via HARQ, priority processing, and logical channel priority.

[0049] The channel estimate derived by channel estimator 258 based on a reference signal or feedback from base station 210 can be used by TX processor 268 to select appropriate coding and modulation schemes and facilitate spatial processing. The spatial stream generated by TX processor 268 can be provided to different antennas 252 via individual transmitters 254TX. Each transmitter 254TX can modulate an RF carrier with the corresponding spatial stream for transmission. Uplink transmission processing at base station 210 is similar to that described in relation to the UE 250 receiving function. Each receiver 218RX receives signals through its corresponding antenna 220. Each receiver 218RX recovers the information modulated onto the RF carrier and provides the information to RX processor 270.

[0050] Controller / processor 275 may be associated with memory 276, which stores program code and data. Memory 276 may be referred to as computer-readable medium. In the uplink, controller / processor 275 provides demultiplexing, packet reassembly, decryption, header decompression, and control signal processing between the transport channel and the logical channel to recover IP packets from user equipment (UE) 250. IP packets from controller / processor 275 may be provided to evolved packet core (EPC) 160. Controller / processor 275 is also responsible for error detection using acknowledgment (ACK) and / or negative acknowledgment (NACK) protocols to support Hybrid Automatic Repeat Request (HARQ) operation.

[0051] New Radio (NR) can refer to a wireless device configured to operate under a new radio interface (e.g., different from an air interface based on Orthogonal Frequency Division Multiple Access (OFDMA)) or a fixed transport layer (e.g., different from Internet Protocol (IP)). NR can use Orthogonal Frequency Division Multiplexing (OFDM) with a Cyclic Prefix (CP) in both uplink and downlink, and may include support for half-duplex operation using Time Division Duplex (TDD). NR may include enhanced mobile broadband (eMBB) services targeting wide bandwidth (e.g., above 80 MHz), millimeter wave (mmW) services targeting high carrier frequencies (e.g., 60 GHz), massive machine-type communications (mMTC) services targeting backward-incompatible MTC technologies, and / or mission-critical services targeting ultra-reliable low-latency communications (URLLC) services.

[0052] It can support a single component carrier bandwidth of 100 MHz. For example, an NR resource block (RB) can span 12 subcarriers with a subcarrier bandwidth of 60 kHz and a duration of 0.25 ms, or a bandwidth of 30 kHz and a duration of 0.5 ms (similarly, a 15 kHz subcarrier spacing (SCS) has a bandwidth of 50 MHz over a duration of 1 ms). Each radio frame can consist of 10 subframes (10, 20, 40, or 80 NR slots) with a length of 10 ms. Each slot can indicate the link direction used for data transmission (i.e., downlink (DL) or uplink (UL)), and the link direction of each slot can be dynamically switched. Each slot can include DL / UL data and DL / UL control data.

[0053] The NR Radio Access Network (RAN) may include Central Units (CUs) and Distributed Units (DUs). NR Base Stations (BSs) (e.g., gNBs, 5G Node Bs, Node Bs, Transmission Reception Points (TRPs), Access Points (APs)) may correspond to one or more base stations. NR cells may be configured as Access Cells (ACells) or Data-Only Cells (DCells). For example, the RAN (e.g., Central Units or Distributed Units) may configure these cells. DCells may be used for carrier aggregation or dual connectivity and may not be used for initial access, cell selection / reselection, or handover. In some cases, DCells may not transmit synchronization signals (SS), and in others they may transmit SS. NR base stations may transmit downlink signals to User Equipment (UEs) indicating the cell type. Based on the cell type indication, UEs can communicate with NR base stations. For example, UEs can determine the NR base station for cell selection, access, handover, and / or measurement based on the indicated cell type.

[0054] Figure 3 An example logical architecture of a distributed radio access network (RAN) 300 according to various aspects of this disclosure is illustrated. A 5G access node 306 may include an access node controller (ANC) 302. The ANC may be the central unit (CU) of the distributed RAN. The backhaul interface to the next-generation core network (NG-CN) 304 may terminate at the ANC. The backhaul interface to the adjacent next-generation access node (NG-AN) 310 may terminate at the ANC. The ANC may include one or more transceiver points (TRPs) 308 (also referred to as base stations, NR base stations, Node B, 5G NB, AP, or other terms). As mentioned above, TRP can be used interchangeably with "cell".

[0055] TRP 308 can be a Distributed Unit (DU). A TRP can connect to one ANC (ANC 302) or multiple ANCs (not shown). For example, for RAN sharing, Radio-as-a-Service (RaaS), and service-specific ANC deployments, a TRP can connect to multiple ANCs. A TRP can include one or more antenna ports. A TRP can be configured to provide service to user equipment individually (e.g., dynamically selected) or jointly (e.g., jointly transmitted).

[0056] The local architecture of the distributed RAN 300 can be used to illustrate the fronthaul definition. This architecture can be defined as a fronthaul solution supporting different deployment types. For example, the architecture can be based on transport network capabilities (e.g., bandwidth, latency, and / or jitter). The architecture can share features and / or components with LTE. Depending on certain aspects, the Next Generation Access Node (NG-AN) 310 can support dual connectivity with NR. The NG-AN can share a common fronthaul for both LTE and NR.

[0057] This architecture enables collaboration between and within TRPs 308. For example, collaboration can be pre-defined within a TRP and / or implemented across TRPs via ANC 302. Depending on certain aspects, inter-TRP interfaces may not be required or may not exist.

[0058] In some respects, the architecture of the distributed RAN 300 can have a dynamic configuration with logical function splitting. Packet Data Convergence Protocol (PDCP), Radio Link Control (RLC), and Media Access Control (MAC) protocols can be adaptively placed in the ANC or TRP.

[0059] Figure 4An example physical architecture of a distributed RAN 400 according to various aspects of this disclosure is illustrated. A centralized core network unit (C-CU) 402 can host core network functions. The C-CU can be centrally deployed. C-CU functions can be offloaded (e.g., to Advanced Wireless Services (AWS)) to handle peak capacity. A centralized RAN unit (C-RU) 404 can host one or more ANC functions. Optionally, the C-RU can host core network functions locally. The C-RU can be distributed. The C-RU can be located closer to the network edge. A distributed unit (DU) 406 can host one or more TRPs. DUs with radio frequency (RF) capabilities can be located at the network edge.

[0060] The traditional and common role of user devices is to provide connectivity within the network for dedicated servers, such as server-client services, or to facilitate peer-to-peer communication, such as telephone and video calls, through these servers. However, next-generation networks, including mobile networks, are expected to support a variety of user devices, artificial intelligence (AI) applications, and distributed computing and communication sharing.

[0061] The recent rapid proliferation of AI and machine learning (ML) applications, especially generative AI services, has brought new challenges in determining where computational tasks should be performed. User devices, such as smartphones, now possess sufficient computing power to run certain AI / ML models locally, while web servers offer even greater processing power for complex tasks. This raises the question for every user interaction: should AI / ML processing occur on the user's device, on a web server, or split between the two?

[0062] Prior to the widespread adoption of artificial intelligence / machine learning (AI / ML) and generative AI applications, most applications and services were deployed in static configurations. The recent popularity of these applications has raised significant concerns about privacy and service costs, particularly the high costs associated with large language model (LLM) services. These concerns have spurred an increased demand for distributed AI services that can run portions of AI / ML functionality on user devices to address privacy concerns and, where appropriate, enhance the user experience.

[0063] Deciding where to execute AI / ML functions involves several complex factors that must be evaluated in real time. The computing power of user devices varies significantly across different devices; factors such as CPU power, GPU availability, and memory capacity affect what can be processed locally. Battery status is another critical consideration, as local AI / ML processing can consume substantial amounts of power, making remaining battery level and state of charge important decision-making factors. Network conditions add another layer of complexity; link bandwidth, end-to-end latency, and the availability of various wireless access technologies all affect the feasibility and efficiency of offloading computational tasks to a network server.

[0064] Multiple Radio Access Technologies (M-RATs), such as Wi-Fi and cellular technologies (including 4G and 5G), typically coexist and provide complementary wireless services to users. Many user devices, including smartphones, tablets, and laptops, are already capable of connecting to multiple access technologies simultaneously at the wireless layer. Traditionally, the selection of access networks has been policy-driven, such as automatically selecting Wi-Fi when available, or based on user preferences, without considering the specific needs of applications or services. Once an application session is connected via a specific Radio Access Technology (RAT), the connection is typically maintained until a wireless handover within the technology is triggered due to poor wireless coverage, or the user manually switches to a different RAT.

[0065] The distributed AI service platform disclosed in this paper addresses these challenges by providing a mechanism to dynamically determine the optimal distribution of AI / ML processing tasks between user devices and network nodes. This mechanism makes real-time decisions for each user query, considering the current state of the user device, network conditions, and the nature of the computational task. The platform operates within an AI / ML framework based on distributed computing principles, strategically splitting tasks between local devices and network-based computing resources to optimize performance, reduce network traffic, and improve battery efficiency.

[0066] This mechanism specifically targets several key service elements in distributed AI service delivery. AI, acting as an assistant, can provide omnichannel capabilities and address various service challenges. For example, in retail applications, a virtual customer service assistant can accompany a customer throughout their store visit, providing continuous assistance until they leave. Smartphones, as the platform for initial and local AI / ML computation, handle the initial stages of AI / ML tasks and then decide whether to offload additional computational tasks to network devices. When local processing is inefficient or unavailable, edge servers and other network nodes perform significant computations for tasks offloaded from user devices. The system reduces network traffic by preventing unnecessary information from being transmitted from user devices to network devices, particularly by reducing uplink traffic.

[0067] Existing M-RAT interoperability solutions, such as those defined in the Third Generation Partnership Project (3GPP), primarily focus on traffic offloading from cellular to Wi-Fi, or traffic routing, handover, and splitting. Some solutions also involve traffic routing and aggregation in M-RAT environments. However, these approaches prioritize achieving high throughput and low end-to-end network latency without considering the unique needs of distributed applications, user equipment capabilities, or device conditions such as battery status. The mechanism disclosed in this paper differentiates itself from existing approaches by comprehensively considering all these factors when deciding how to distribute AI / ML processing tasks.

[0068] The distributed AI service splitting and switching decision mechanism can be applied to a variety of use cases and scenarios. While this disclosure provides specific examples, including virtual and personal assistant mixed reality services and collaborative AI personal networks with distributed generative AI, its underlying principles can be extended to numerous other application areas. The same concept can be applied to food service, libraries, transportation hubs, and other environments requiring personalized AI assistants. This approach allows for flexible and personalized assistance while retaining sensitive personal information on the user's device and leveraging more powerful public AI / ML models on network servers where appropriate. Therefore, this mechanism establishes a framework for future value creation opportunities for device manufacturers, service providers, and network operators.

[0069] Using a customer entering a shopping mall or retail store as an example application, a virtual and personal assistant can be assigned or activated on the customer's device to provide assistance during the customer's visit. For first-time visitors, a default 3D avatar appears on the user's device display, whether it's smart glasses, a smartphone, or other display-enabled device, and provides an initial greeting. This avatar can be presented as mixed reality, overlaid on the real-world view, or as augmented reality, displayed independently of the real-world view. User interaction with the virtual assistant can be via audio, text, or both, depending on device capabilities and user preferences.

[0070] During the shopping process, if customers have any questions, a virtual store assistant can interact with them and provide comprehensive services. These services include product information such as manufacturer details, expiration date, price, discounts, warranty, ingredients, nutritional information, and calorie count per serving. The assistant can also provide operational information, including assembly instructions, installation procedures, and usage guides. Purchasing services can be completed through the assistant, including order placement and order status tracking. The system provides product availability information, including current in-store stock, the location of nearby stores with the desired product in stock, and out-of-stock details. When authorized by the user, the service can also provide personalized recommendations based on shopping history, declared preferences, and other relevant criteria.

[0071] Figure 5Figure 500 illustrates the deployment architecture of virtual and personal assistant mixed reality services in the retail industry. This architecture is part of a distributed AI service platform that implements a mechanism to dynamically split and switch AI / ML computing tasks between user devices and network nodes based on real-time conditions, including device capabilities, battery status, and network conditions.

[0072] The virtual assistant service network comprises multiple interconnected components that collaborate to provide distributed AI services. User devices, including smartphones, smart glasses, and smartwatches, serve as the initial point of user interaction and can perform local AI / ML calculations where appropriate. Product registration servers maintain the inventory and product information required for retail applications. The network service provides the connectivity infrastructure that enables communication between user devices, service agents, and servers containing service models and service engine functionalities.

[0073] The service engine comprises multiple computing modules strategically distributed across the service network to optimize performance and resource utilization. These modules include Speech-to-Text (STT) for converting speech input to text, Text-to-Speech (TTS) for generating audio responses, a set of large language models (LLMs) for natural language processing and response generation, and sub-modules for generating and rendering 3D virtual assistant avatars. These sub-modules are not confined to a single machine but can be distributed across different network locations based on available computing resources, network latency, and the specific needs of each user interaction. This distributed approach enables the system to allocate computing tasks to the most appropriate locations, whether processing privacy-sensitive or simple queries on the user's device or computationally intensive operations on a web server.

[0074] Server infrastructure combines a service model containing a knowledge base with a service engine that performs computational processing. It can be deployed in various configurations to meet diverse operational needs. Servers can be installed as dedicated units in a Customer Premises Network (CPN), labeled "1," providing low-latency access for users within retail locations. Alternatively, servers can be hosted on shared computing and storage infrastructure managed by a Mobile Network Operator (MNO), labeled "2," allowing centralized management and resource sharing across multiple locations. Fixed network operators, including Digital Subscriber Line (DSL), fiber optic, coaxial cable, and Wi-Fi providers, as well as satellite network operators, labeled "3," offer another deployment option for broad coverage. Cloud network operators, labeled "4," provide scalable infrastructure that can dynamically adjust to different computing needs.

[0075] To enhance reliability and performance, server infrastructure can be deployed across multiple sites or networks simultaneously. This multi-site deployment enables load balancing, evenly distributing computational workloads, providing redundancy to improve reliability through failover capabilities, and improving end-to-end performance by allowing users to connect to the nearest or least congested server. Modular architecture allows service models and service engine components to be deployed in different physical locations, maintaining a single centralized service model containing the knowledge base while running multiple distributed service engine instances for computational processing. This separation enables efficient resource utilization, allowing computationally intensive tasks to be handled by the most powerful available hardware while maintaining the consistency of the knowledge base.

[0076] The service broker acts as the initial service landing point for user equipment to establish its first communication, then directs it to the appropriate server. The broker performs load balancing and routing decisions based on current network conditions, server availability, and the nature of the user's request. As an intermediary, the service broker shields user equipment from the complexities of the distributed server infrastructure while ensuring that each request is handled by the most suitable available resources.

[0077] Figure 6This demonstrates a mixed reality service design that integrates a virtual and personal sales assistant between user devices (such as smartphones) and service servers. The system architecture eliminates intermediate network elements, including Customer Premises Equipment (CPE), Evolved Residential Gateway (eRG), Customer Premises Network (CPN), and Mobile Network Operator (MNO) infrastructure, to focus on core service components and their interactions.

[0078] The virtual assistant service requires comprehensive access to detailed product information, thus necessitating the preparation of all product information and the construction of a corresponding service model. The server architecture implements a dual database system for efficient information retrieval. Figure 6 The top vector database 602 shown represents the product list and information registration database, serving as the primary repository for textual product information. This database stores all product items and their associated metadata in an inventory database structure. The stored data can be stored as plain text for direct retrieval or converted into context vectors for similarity-based searches. The comprehensive information catalog includes a unique product name for clear product identification, a category for organizational hierarchy, detailed product descriptions, general pricing information, applicable discount rates, warranty terms and coverage, expiration dates for perishable goods, best-before dates for best-quality items, currently displayed inventory quantities, and inventory levels.

[0079] Figure 6 The bottom vector database 604 shown represents the product item image registration database, enabling visual product recognition. This database stores multiple photos of each product item taken from different angles, along with corresponding product name embeddings generated by a deep learning model. The system uses a Contrastive Language-Image Pre-Training (CLIP) model to generate these embeddings, creating a unified vector representation to match visual and textual product information. This dual-database architecture provides flexible query paths, allowing products to be referenced by product name or product index as the lookup key. While these databases could be merged into a single unified database for simplified management, separate maintenance allows for optimized query performance through dedicated indexing strategies for text and visual searches.

[0080] The service architecture distributes dedicated modules between user devices and web servers to optimize computing resource utilization. The local model module within the user device serves as a multilingual service component, containing a variety of compact machine learning models carefully selected for efficient operation on resource-constrained devices. These models include smaller variants of the Large Language Model (LLM), visual-language models such as LLaVA (with 7 billion and 13 billion parameters), and other specialized models optimized for mobile deployments. When the service is served locally on the device, this module provides language-specific responses based on user preferences, maintaining responsiveness even without a network connection.

[0081] The AI / ML model module within the service engine on the web server provides complementary multilingual service capabilities, offering a larger and richer selection of models. These server-side models include full-scale large-scale language models with tens of billions of parameters, comprehensive vision-language models, and domain-specific models impractical for deployment on mobile devices. When services are delivered via high-performance server infrastructure, this module provides higher accuracy and capabilities while adapting to user language preferences.

[0082] The system achieves distributed speech processing capabilities through strategically deployed Speech-to-Text (STT) and Text-to-Speech (TTS) modules. The STT module on the user device converts voice-based user prompts and commands into text format for processing. Both the user device and the server are equipped with TTS modules that convert text responses into audio streams; the choice between local or server-side TTS processing depends on the current operating conditions.

[0083] Deciding where to perform TTS processing requires evaluating multiple trade-offs between network latency, data transfer requirements, and computing resources. When TTS runs on the user's device, the system does not need to communicate back and forth with the server, thus reducing network latency and generating immediate responses. This approach only requires sending text data to the device, typically 100 to 1000 bytes per query, while audio files (such as a five-second MP3 clip) can reach 50 to 100 kilobytes, greatly reducing data transfer. The faster response time resulting from local processing creates a smoother, more interactive user experience. However, device-based TTS has limitations, including the computing power limitations of older or low-spec devices, increased battery consumption due to processing load, and increased application size due to embedded TTS libraries.

[0084] Server-side TTS processing delivers consistently high quality and more natural speech by accessing more powerful computing resources and sophisticated speech synthesis models. This approach offloads the computational load from the user device, making it particularly suitable for devices with limited processing power and simplifying system updates, as improvements can be centrally deployed without requiring application updates. Disadvantages include potential response delays due to network latency, especially in areas with poor network connectivity, increased data transfer requirements due to audio file transmission, and reliance on a stable internet connection to ensure functionality.

[0085] To balance these competing needs, the system implements a hybrid approach, delivering a seamless user experience while optimizing device performance and battery life. This balancing strategy selectively offloads TTS processing to a web server for complex or lengthy responses, reducing the smartphone's computational load as needed. The system employs optimized 3D models and efficient rendering techniques to minimize GPU load when avatar rendering is required. For optimal performance, the hybrid approach uses local TTS for rapid processing of short, simple statements, while server-side TTS is used for complex interactions requiring higher quality or involving numerous responses, provided network conditions allow for data transmission.

[0086] The selective use of modules coexisting in both the user device and the service engine can dynamically respond based on operational conditions. When network bandwidth is limited, the system prioritizes using local modules to avoid network latency. When the user device has low battery power or insufficient available memory, the system will offload tasks to the network server to maintain device functionality. This adaptive approach maintains service availability under different conditions and optimizes the user experience based on available resources.

[0087] The second major application scenario explores collaborative AI personal networks (PNs) with distributed generative artificial intelligence (AI), focusing on the dynamic interaction between complex AI models in central nodes (such as customer premises equipment, CPEs) and simpler models in user devices (such as smartphones). This scenario addresses the challenge of continuously delivering context-aware personal assistance services across diverse environments and devices, while maintaining user privacy and optimizing computational resources.

[0088] The collaborative AI personal network architecture comprises three interconnected network types that seamlessly exchange information to provide comprehensive personal assistance services. The In-Body Personal Network is centered around wearable devices and smartphones that continuously collect health data and activity information. This network uses AI models to generate real-time health insights and alerts, while minimizing reliance on cloud connectivity through primarily local processing. The smartphone within this network acts as a local AI / machine learning (ML) computing platform, employing encrypted local storage to preserve personal information and experiences to protect user privacy. In typical use, the smartphone tracks daily activities, conversations, schedules, and health metrics from connected wearable devices, leveraging this accumulated data as a comprehensive source for the AI ​​assistant to understand user context and preferences.

[0089] The in-vehicle personal network operates in autonomous and connected vehicles, integrating AI systems to process sensor data in real time for navigation, safety features, and traffic management. This network leverages edge computing capabilities to enable low-latency decision-making required for vehicle operation. When the driver enters the vehicle, the in-vehicle personal network connects to the in-vehicle network, synchronizing the databases of the two systems. The vehicle's driving assistant, relying on more powerful computing hardware than mobile devices, can learn about the driver's recent experiences, physical condition, and emotional state through synchronized data, thereby providing context-aware assistance services.

[0090] The In-House Personal Network manages smart home automation through AI applications, controlling and coordinating home devices and environmental sensors. Over time, the network learns user preferences, optimizes energy consumption patterns, and processes data locally to maintain both privacy and responsiveness. When a user arrives home, the in-vehicle personal network connects to the home network and transmits relevant journey and activity data to the home network database. Home CPE devices possess powerful processing capabilities and storage capacity, integrating information about the user's daily experiences, current physical condition, and emotional state. Based on this comprehensive understanding, the home assistant automatically adjusts environmental settings, including temperature, lighting intensity and color, and ambient music selection, to match the user's current needs and preferences.

[0091] Collaboration among these personal networks enables complex scenarios through selective information sharing while preserving user privacy. When environments change, the networks perform intelligent knowledge transfer, synchronizing only relevant data while isolating sensitive information within appropriate domains. This distributed approach allows each network to specialize in its domain while collectively contributing to a unified personal assistive experience, enabling users to seamlessly adapt to their daily activities.

[0092] Figure 7This diagram (700) illustrates three collaborative AI personal networks and distributed generative AI, demonstrating how the distributed AI service splitting and switching mechanism operates in different personal network environments. The architecture comprises three interconnected personal networks that work together to provide continuous AI assistance to users as they switch between different environments in their daily activities.

[0093] An in-body personal network forms the basis for personal data collection and processing. This network integrates wearable devices to monitor health metrics, with smartphones serving as the primary AI / ML computing platform. Smartphones continuously collect health data and leverage AI to provide real-time health insights and alerts, while maintaining data privacy through local processing. This approach reduces reliance on continuous cloud connectivity by performing initial AI / ML computations on the device itself. Smartphones track user activity, experiences, and health metrics throughout the day, creating a comprehensive data source for the personal AI assistant. As shown in step 1 of the diagram, the system updates personal experience data during daily activities; the text "I got a promotion today" represents an example of captured personal information.

[0094] The in-vehicle personal network provides AI assistance during traffic operations through autonomous vehicle systems. This network includes an AI system that processes sensor data in real time for navigation, safety features, and traffic management. The vehicle system utilizes edge computing capabilities to achieve low-latency decision-making required for safe vehicle operation. When a user enters the vehicle, as shown in step 2, the phone is placed inside, and the internal personal network establishes a connection with the in-vehicle personal network. This connection enables database synchronization between the two systems. In step 3, the vehicle assistant provides voice interaction, stating, "Congratulations on your promotion. Would you like to go to a party?" This demonstrates that the driving assistant understands the user's recent experiences and emotional state through synchronized data, possessing more powerful computing resources than a smartphone. The user's response in step 4, "No, go home," instructs the vehicle's navigation system.

[0095] The home personal network manages the home environment through a smart home automation system. This network includes AI applications that control home devices, environmental sensors, and energy management systems. The network processes data locally, protecting privacy while ensuring responsiveness, and learns user preferences to optimize energy consumption patterns. Step 5 displays the driving assistant informing the user that communication with the home system is in progress. When the user arrives home, the in-vehicle personal network transmits relevant journey and activity data to the home personal network database. The CPE devices in the home have greater processing and storage capabilities than mobile devices, integrating information about the user's daily experiences, physical condition, and emotional state. Step 6 displays the home assistant's voice interaction stating, "Vehicle detected, you have arrived," confirming a successful network switch. In step 7, the home assistant automatically adjusts environmental settings, including temperature, lighting, music selection, and other home parameters, based on a comprehensive understanding of the user's current state and the day's events.

[0096] The collaborative operation of these three personal networks demonstrates the practical application of a distributed AI service splitting and switching mechanism. Each network makes real-time decisions on whether to process AI / ML tasks locally, offload them to a more powerful device within the same personal network, or to another connected personal network. The in-vehicle PN primarily processes personal and privacy data on the smartphone to maintain privacy. The in-vehicle PN, with its more powerful hardware, can handle computationally intensive tasks offloaded from the smartphone and adds vehicle-specific processing. The home PN, with the strongest computing resources through CPE devices, can handle complex AI / ML tasks for all connected devices while maintaining the home environment.

[0097] Knowledge transfer between these networks is selective to maintain privacy and ensure seamless service continuity. Personal experiences and preferences stored in the in-body PN are shared with the in-vehicle PN when the user enters the vehicle, enabling the driving assistant to provide context-aware assistance. Similarly, data accumulated by the in-body and in-vehicle PNs is synchronized with the home PN upon arrival at home, allowing the home assistant to create an optimal environment based on a complete understanding of the user's day. This distributed approach allows each personal network to focus on its domain-specific functions while collectively contributing to a unified personal assistance experience that adapts as the user moves between different environments.

[0098] The system maintains user privacy through intelligent data isolation and selective sharing. Sensitive personal information remains encrypted on the user's smartphone unless explicitly required by the connected personal network. A distributed AI service splitting and switching mechanism determines in real time which data should be shared and which AI / ML models should be used, based on the current network environment, available computing resources, and the nature of the task being performed. This dynamic decision-making process optimizes performance while conserving battery life on mobile devices and maintaining responsive service across all three personal network environments.

[0099] Figure 8 This is a schematic diagram (800) illustrating a collaborative AI personal network between an in-body personal network and an in-house personal network. The diagram demonstrates two distinct collaboration mechanisms: knowledge transfer between personal networks belonging to the same user, and information sharing between personal networks belonging to different individuals. This architecture represents a practical implementation of a distributed AI service splitting and switching mechanism in a multi-device, multi-network environment.

[0100] The in-body personal network primarily operates on the user's smartphone, which serves as the central hub for personal data collection and processing. Once granted appropriate permissions, the smartphone collects user information from multiple sources, including the device's camera for visual context, a connected smartwatch for health and activity metrics, a microphone for audio input and environmental awareness, conversations and interactions, internet browsing activity, phone usage patterns, calendar entries for scheduling information, a contact list for relationship data, and notes or documents stored on the device. This comprehensive data collection creates a rich personal context for AI assistants, which can then leverage this data to provide personalized services.

[0101] The smartphone architecture within the on-premises personal network comprises several dedicated modules. The multilingual service module includes various artificial intelligence / machine learning (AI / ML) models optimized for mobile deployment, including smaller versions of large language models such as Llama3 with 8 billion parameters, LLaVA visual language models with 7 billion and 13 billion parameters, and OpenChat models. These models were chosen for their efficient operation within the computational constraints of mobile devices. The PN Assistant module acts as a personal network assistant, coordinating interactions between different AI models and managing user queries. The context and network-aware model selection module determines whether queries should be processed locally or offloaded to more powerful computing resources. The device analytics module monitors the smartphone's operational status, and the network analytics module assesses connectivity conditions.

[0102] All personal information collected by smartphones is processed and stored in a local vector database to maintain privacy. This database stores embeddings generated from user's personal data, creating a searchable knowledge base that is always under user control. This local storage approach allows personal AI assistants to function even when the smartphone lacks an internet connection, using locally deployed machine learning models for reasoning and response generation.

[0103] Home Personal Networks (PNs) are centered around Customer Premises Equipment (CPEs) within the user's home environment. CPEs offer significantly more computing resources than mobile devices, including more powerful processors, larger memory capacities, and a more stable power supply. This enhanced capability allows them to host larger and more complex AI / ML models, which are impractical to deploy on battery-powered mobile devices. Home PNs also include a multilingual service module, full-scale models, a PN assistant for home services, and their own network analytics capabilities.

[0104] Knowledge transfer, labeled "Knowledge Transfer (Synchronization)" in the diagram, represents the synchronization process between personal networks belonging to the same user or jointly owned. When a user's smartphone connects to their home personal network, typically after arriving home, the vector database on the smartphone synchronizes with the database on the CPE. This synchronization transfers relevant user data, experience, and context accumulated throughout the day to the home system. The CPE, with its greater processing power and larger machine learning models, can provide enhanced support while the user is at home. This knowledge transfer is bidirectional, allowing the home system to update relevant information to mobile devices when the user leaves the home environment.

[0105] Information sharing, as illustrated between two independent smartphones in the diagram, represents another collaborative mechanism. This process occurs between personal networks belonging to different individuals and operates under strict privacy constraints. When a user explicitly authorizes, their corresponding PN assistant can exchange filtered information needed for collaborative tasks while maintaining the privacy of each user's personal data. For example, when coordinating a meeting, the assistant can share availability information without exposing details of each person's calendar. This selective information sharing allows virtual assistants from different personal networks to interact and supports multi-user scenarios, while minimizing direct interpersonal interaction and protecting individual privacy.

[0106] The diagram also illustrates connections to external network infrastructure via network operators / internet / cloud components, which may include services from service providers such as Google, Meta, or Microsoft. These external connections enable personal networks to access cloud-based services when needed, although the architecture prioritizes local and edge processing to maintain privacy and reduce latency. The distributed AI service splitting and switching mechanism continuously evaluates whether to process queries locally on the smartphone, in a more robust CPE within the home network, or via external cloud services, making decisions based on factors such as query nature, available computing resources, network conditions, and privacy requirements.

[0107] Figure 9This is Schematic 900, illustrating a standalone PN AI assistant scenario, representing the baseline operating mode of a collaborative AI personal network system. In this configuration, the in-body personal network operates independently on a single device (such as smartphone 1), without relying on external network resources or cloud services. The smartphone contains all the components necessary for autonomous operation, including a context and network-aware model selection module, multilingual and visual models for local AI / ML processing, device and network analytics capabilities for monitoring operating conditions, and the PN AI assistant coordinating the interaction of these components. This standalone configuration demonstrates the system's ability to provide AI assistance even when network connectivity is unavailable or privacy requirements necessitate fully local processing. The distributed AI service splitting and switching mechanism operates entirely within the device, with all computational decisions based solely on local resource availability and the complexity of user queries.

[0108] Figure 10 Diagram 1000 illustrates a distributed PN AI assistant scenario, where the in-body personal network dynamically expands its computing power by collaborating with a network server. When smartphone 1 determines that its local model cannot adequately support a user's request, whether due to model limitations, computational constraints, or battery factors, a distributed AI service splitting and switching mechanism is activated to identify and utilize an appropriate model from the network infrastructure. In the example, the supporting model resides on a multi-cluster edge cloud server, which provides a more powerful AI / ML model than the mobile device. The system makes offloading decisions based on a real-time assessment of multiple factors, including query complexity, available network bandwidth, latency to the edge server, device battery status, and the expected improvement in response quality using a more powerful network model. Communication between the smartphone and the edge cloud server preserves the user context while offloading only the computationally intensive parts of the AI / ML processing, thus protecting the privacy of sensitive information remaining on the device.

[0109] Figure 11This diagram (1100) illustrates an information-sharing scenario embodying controlled collaboration between personal networks belonging to different individuals. This scenario addresses the challenge of enabling AI assistants to coordinate user activities while maintaining strict privacy boundaries. When explicitly authorized by the user, virtual assistants on smartphones 1 and 2 can establish filtered information exchange channels. A distributed AI service splitting and switching mechanism within each device independently determines which information can be securely shared based on user-defined privacy policies and the nature of the collaborative task. For example, when coordinating a meeting between two users, the assistant can share available time slots without revealing detailed calendar content, or exchange proximity information without disclosing specific locations. Each PN AI assistant processes shared information using its local model to maintain privacy, transmitting only the processing results, not the original personal data. This selective information-sharing protocol allows virtual assistants to facilitate multi-user interaction, reduce the need for direct interpersonal communication, and maintain personal privacy boundaries through intelligent filtering of exchanged data.

[0110] Figure 12 Figure 1200 illustrates a knowledge synchronization scenario, demonstrating the seamless transfer of accumulated knowledge between personal networks within the same ownership domain. When a user switches from their In-Body Personal Network (PN) to their In-House Personal Network (PN), the system initiates a knowledge transfer process, synchronizing relevant data between the smartphone 1 and the Customer Premises Equipment (CPE). This synchronization goes beyond simple data copying; it involves intelligent selection of information relevant to the home environment context. A distributed artificial intelligence (AI) service splitting and switching mechanism evaluates information from the mobile device, determining which information enhances the home assistant's ability to provide personalized services to the user. The CPE device in the home PN, equipped with a more powerful processor, larger memory capacity, and stable power supply, can utilize larger machine learning (ML) models to process the synchronized data—models impractical to run on battery-powered mobile devices. During the knowledge transfer process, the system maintains bidirectional synchronization, enabling the home network to update relevant information to the mobile device for future use by the user outside the home. A synchronized knowledge base enables the home assistant to understand the user's daily experiences, current state, and emotional state, thereby automatically adjusting environmental settings, such as temperature, lighting, and music, to match the user's current needs.

[0111] Figure 13Figure 1300 illustrates a fall detection scenario, showcasing the system's emergency response capabilities through the "Virtual of Me" concept. In this key use case, when the system detects a user has fallen and lost consciousness or is otherwise unable to move, the Personal Network (PN) AI assistant transforms from a passive assistant into an active representative, capable of communicating on the user's behalf. The "Virtual of Me" functionality enables the AI ​​assistant to synthesize the user's communication style and relevant medical information, interacting with emergency services, first responders, and designated emergency contacts. A distributed AI service splitting and switching mechanism immediately prioritizes this emergency function, allocating all available local resources while establishing connections to network services to enhance communication capabilities. The system can simultaneously contact multiple entities, including other virtual assistants belonging to emergency contacts, human first responders, and emergency services such as 911. The virtual assistant provides critical information, including the user's location, the nature of the emergency detected through sensor data, authorized relevant medical history, and can maintain continuous communication with responders until aid arrives. This autonomous emergency response capability operates even when the user is unable to interact with the device, demonstrating the system's function as a protective digital agent in critical situations.

[0112] These five scenarios collectively demonstrate the diversity and adaptability of the distributed AI service splitting and switching mechanism in collaborative AI personal network systems. Each scenario represents a different operating mode that the system can dynamically select based on current conditions, user needs, and available resources. The system continuously evaluates whether to operate in independent mode to achieve maximum privacy and independence, in distributed mode to enhance capabilities, in information-sharing mode for multi-user collaboration, in knowledge synchronization mode to achieve context transfer between environments, or in emergency mode for critical intervention. This dynamic selection process is transparent to the user, with the distributed AI service splitting and switching mechanism making real-time decisions based on a comprehensive analysis of device capabilities, network conditions, query needs, and contextual information.

[0113] Figure 14A -B represents Figures 1400 and 1450, respectively, illustrating the distributed AI splitting and switching decision-making mechanism and network analysis between user devices and the service network. This mechanism represents the core decision-making system, which determines in real time whether AI / ML processing should be performed on user devices or offloaded to network servers. The example service network consists of user devices (such as...) Figure 14A (as shown in the image) and distributed in, for example, smartphones Figure 14B The network infrastructure shown consists of multiple server clusters.

[0114] The decision-making mechanism makes optimal splitting decisions by collecting comprehensive information about equipment capabilities and network conditions. Figure 14AThe smartphone contains multiple functional modules that work together to analyze user queries and determine the appropriate processing location. Module B1 receives user prompts, which can be provided via text, voice, gestures, or other input methods. Module B2 uses a context-aware multimodal model selector to analyze prompts, identify the nature of the query, and select the appropriate AI model for a specific application or modality (such as chat, visual-language, language, or domain-specific processing). Module B3 analyzes whether the query involves personal or privacy information, a key decision point for privacy protection. Module B4 contains a Retrieval Augmented Generation (RAG) system, which maintains embeddings in a vector database derived from on-device personal and professional information from the local device.

[0115] Module B5 stores local AI / ML models, including smaller versions of language models such as Llama3 with 8 billion parameters, visual-language models such as LLaVA with 7 billion and 13 billion parameter variants, OpenChat with 7 billion parameters, and Llama3-chatQA with 8 billion parameters. These models are specifically chosen for efficient operation under the computational constraints of mobile devices. Module B6 performs suggestive AI analysis, determining whether a query is simple enough for local processing by evaluating factors such as query length, expected response length, and expected response time. Module B7 manages AI agent selection, choosing appropriate agents based on query requirements, such as visual agents including avatars, chat agents, or summary agents.

[0116] Module B8 performs AI analysis of user behavior, comparing expected response times with estimated user tolerance for response latency. This module continuously learns through user interaction, understanding when users might cancel or resubmit queries due to impatience. Module B9 performs device analysis, assessing available computing resources and battery status, including current battery level and whether the device is charging or discharging. Module B10, known as Context & Network Aware GeoModelSwitching, acts as a central decision engine, determining whether to use a local or remote model based on a comprehensive analysis of all available information. GeoModelSwitching refers to the model's geographic location within the distributed infrastructure, rather than the user's physical location.

[0117] Module B11 performs network analysis at Layers 1, 2, and 3 and above, identifying the optimal network interface based on link capacity status or predictions and latency measurements to remote models. This analysis can monitor existing traffic in passive mode or send probe messages to measure network performance in active mode. Module B12 manages communications through various RadioAccess Technologies (RATs), which may include WiFi, 4G, 5G, or other available network connections. Module B13 delivers responses to the user through selected output modalities such as text, audio, visual, or other means.

[0118] The system implements twenty-four different data flow paths, labeled A1 to A24, connecting these functional modules and defining possible processing routes for user queries. Path A1 passes the initial prompt from module B1 to the prompt analysis in module B2. Path A2 branches from module B2, pointing to AI agent selection and personal query analysis respectively. Path A3 routes to module B7 for AI agent selection. Path A4 directs personal queries to the RAG system in module B4. Path A5 provides a direct path for using the original query without RAG enhancement. Path A6 integrates the paths from the RAG system into the local model of module B5. Path A7 passes the response from the local model to module B13. Path A8 connects AI agent selection to the response mechanism.

[0119] Path A9 passes the non-personal query from module B3 to module B6 for complexity analysis. Path A10 routes the query from module B6 to module B8 for user behavior analysis. Path A11 connects module B8 to module B4 when local processing is deemed appropriate. Path A12 provides an alternative path for retrieving information from module B6 to module B4. Path A13 connects the RAG output to the network analysis path. Path A14 represents an alternative flow from module B8 to module B4 when response time analysis indicates that local processing is still feasible. Path A15 extends from module B8 to module B10 for distributed processing evaluation.

[0120] Path A16 transfers device analysis results from module B9 to module B10. Path A17 inputs network analysis information from module B11 to module B10. Path A18 transfers the merged network analysis results to module B10. Path A19 establishes communication from module B10 to module B12. Path A20 returns the response from module B10 to module B7 for display. Path A21 represents bidirectional communication between user equipment and network infrastructure via module B12. Path A22 provides an additional communication path for network connectivity. Path A23 routes complex queries directly from module B6 to module B10 for distributed processing evaluation. Path A24, when the system determines that local processing is optimal despite query complexity, implements local model selection by returning from module B10 to module B5.

[0121] The distributed AI splitting and switching mechanism implements five main operation scenarios, demonstrating different query processing paths in the system. These five scenarios are: (1) using the local model to process personal queries, (2) using the local model to process non-personal, simple queries, (3) using the remote model to process non-personal, simple queries, (4) using the remote model to process non-personal, complex queries, and (5) using the local model to process non-personal, complex queries.

[0122] In the first scenario, where a local model is used to process personal queries, the system processes queries containing privacy or personal information entirely on the device to protect privacy. The process proceeds along path A1 from module B1 to module B2 for initial prompt analysis, then along path A2 to module B3, where the system identifies the query as a personal query. For personal queries, the system follows two possible paths: one is to retrieve relevant personal context via path A4 through the RAG system, then along path A6 to the local model; the other is to directly reach the local model along path A5 if context retrieval is not required. The response is generated by module B5 and passed to module B13 via path A7. Simultaneously, path A3 selects an appropriate AI agent from module B2 to process and display the response in module B7.

[0123] In the second scenario, where a local model is used to handle impersonal, simple queries, the system processes simple queries that do not require advanced computing resources. After analysis via paths A1 and A2, module B3 identifies the query as impersonal and routes it to module B6 via path A9. Simplicity analysis determines that the query can be processed locally, and the process proceeds along path A10 to module B8. When user behavior analysis confirms that the expected response time meets the user's tolerance, the system retrieves the necessary information from module B4 along path A11, then proceeds along path A6 to the local model of module B5, and the response is delivered via path A7.

[0124] In the third scenario, where a remote model is used to process non-personal, simple queries, the system should address situations where local processing response time exceeds the user's tolerance. After initial analysis via paths A1, A2, and A9, when module B8 determines via path A10 that the expected local response time exceeds the user's tolerance, the system evaluates remote processing options via path A15 to module B10. Module B10 analyzes available network servers; if a suitable remote model, such as server S2, is identified, relevant context is retrieved from module B4, and the enhanced query is passed to module B12 via path A19 for communication. The query travels via path A21 to the base station and further to server S2. The response travels along the same path via paths A21 and A19 to module B10, and is displayed by the AI ​​agent selected by module B7 via path A20.

[0125] In the fourth scenario, where a remote model is used to process impersonal, complex queries, the system handles computationally intensive queries that exceed local processing capabilities. After analysis, module B6 identifies the query as complex via path A9 and triggers a dual operation. Path A12 initiates the retrieval of relevant information from module B4, while path A23 simultaneously routes to module B10 for distributed processing evaluation. Module B10 determines the optimal remote server, retrieves relevant information from module B4, and passes the enhanced query to module B12 via path A19. Communication proceeds via path A21 to the base station and selected servers, such as server S2. The response travels along paths A21 and A19 to module B10 and is displayed by the selected AI agent via path A20.

[0126] In the fifth scenario, which uses a local model to handle impersonal, complex queries, the system must process complex queries locally, despite the higher computational demands. This occurs when no better remote model is available, network conditions are poor, or device battery constraints prevent remote processing. After complexity evaluation by module B6, the system evaluates remote options via path A23 to module B10. When module B10 determines that remote processing is infeasible or unprofitable, the query is returned to the local processing module B5 via path A24. The query is processed using the local model, and the response is passed to module B13 via path A7.

[0127] Device capability parameters influencing the splitting decision include geographic location information to determine distance to available servers, battery status including remaining power and charging status, battery charging rate indicating whether power is increasing or decreasing, IP addresses assigned to each Radio Access Technology (RAT) revealing available network connectivity, and user mobility characteristics such as whether they are stationary, walking, or using transportation. The system also monitors historical network throughput performance to understand achievable data transmission rates, monitors packet latency performance to assess network latency, and evaluates the transmission buffer status of network congestion through backlog measurements of the underlying chipset.

[0128] Module B10's Context & Network AwareGeoModelSwitching module makes comprehensive, split-factor decisions by simultaneously evaluating multiple factors. For queries identified as privacy-sensitive in Module B3, this module forces local processing regardless of other factors, protecting user privacy by preventing personal information from leaving the device. For non-privacy queries, this module evaluates query complexity, available device resources including CPU, GPU, and memory availability, battery status and charging status, network conditions for each available RAT, and the relative performance capabilities of local and remote models. When network quality is poor, this module may choose local processing, even using a smaller, potentially less accurate model, to maintain service availability. When the device battery is low and not charging, this module may limit processing to conserve remaining power, unless the user overrides this protection.

[0129] Figure 14B The distributed server architecture shown implements a multi-tiered deployment strategy across four different server types. Servers labeled S1 are installed within the Customer Premises Network (CPN), providing minimal latency access to users in retail or enterprise locations while maintaining complete control over the service infrastructure. Servers labeled S2 are hosted by a Mobile Network Operator (MNO), providing broader geographic coverage via cellular networks and having computing resources managed by professionals. Servers labeled S3 run on infrastructure managed by a Fixed Network Operator, including DSL, fiber, cable, and WiFi providers, as well as satellite network operators, extending service coverage beyond traditional mobile networks. Servers labeled S4 utilize the infrastructure of a Cloud Network Operator, providing virtually unlimited scalability and handling peak loads and compute-intensive tasks through dynamic resource allocation.

[0130] Each server combines service model and service engine functionalities. The service model maintains a comprehensive knowledge base containing domain-specific information, while the service engine performs computations using various artificial intelligence / machine learning (AI / ML) models. The modular architecture allows these components to be deployed together on the same server or distributed across different physical locations. A single centralized service model can support multiple distributed service engine instances, achieving efficient resource utilization, where the knowledge base remains consistent, while computational resources are allocated according to local demand patterns.

[0131] Several real-time factors evaluated by Block B10 determine which server handles a specific AI / ML task. Query characteristics, including complexity and data requirements, influence whether the local customer premises network (CPN) server can handle the task or whether cloud resources are needed. Network conditions between the user device and each potential server determine which option provides the best response time. Privacy requirements may necessitate that certain queries be retained within the CPN server. The current load of each server in the distributed infrastructure helps route requests to servers with available capacity. This mechanism continuously evaluates these factors to make optimal routing decisions, balancing performance, privacy, and resource utilization across the distributed server infrastructure.

[0132] Server deployment follows a layered architecture to accommodate varying latency, privacy, and computing needs. Servers installed as dedicated units within the Customer Premises Network (CPN), labeled "S1," provide minimal-latency access for users within retail or enterprise locations. These local servers can handle privacy-sensitive queries without transferring data outside the premises and provide instant response times for location-specific services. CPN deployments allow enterprises complete control over their service infrastructure while benefiting from distributed AI processing capabilities.

[0133] Servers hosted on infrastructure managed by mobile network operators (MNOs), designated "S2," provide middleware computing resources accessible via cellular networks. These MNO-hosted servers offer wider geographic coverage than CPN servers and can serve multiple retail locations or businesses within the operator's service area. MNO infrastructure provides reliable connectivity and professional computing resource management, making it suitable for enterprises that require robust AI services but are unwilling to invest in their own server infrastructure.

[0134] Fixed network operators and satellite network operators, designated "S3," offer alternative deployment options to extend service coverage beyond traditional mobile networks. Fixed network operators, including digital subscriber line (DSL), fiber, cable, and Wi-Fi providers, offer high-bandwidth connectivity suitable for data-intensive AI operations. Satellite network operators enable service coverage in remote areas inaccessible by terrestrial networks, providing comprehensive geographic coverage for distributed AI service platforms.

[0135] Cloud network operators, designated "S4," provide the highest tier of computing resources with virtually unlimited scalability. These cloud deployments can dynamically allocate processing power based on fluctuating demand, making them ideal for handling peak loads or compute-intensive tasks that exceed the capacity of edge servers. Cloud infrastructure also facilitates the rapid deployment of updated AI models and service enhancements throughout the distributed system.

[0136] Distributed deployments across multiple sites and networks achieve several operational goals beyond basic service delivery. Load balancing distributes computing workloads across available servers, preventing any single node from overloading and maintaining consistent service network performance. Multi-site architecture provides redundancy through automatic failover capabilities; if a server becomes unavailable, user requests can be seamlessly redirected to other servers, ensuring uninterrupted service. Geographic distribution improves end-to-end performance, minimizes network latency, and maximizes throughput by connecting users to the nearest available server.

[0137] Modular architecture allows service models and service engine components to be deployed together on the same server or distributed across different physical locations, depending on operational needs. A single centralized service model containing a comprehensive knowledge base can support multiple distributed service engine instances, each performing computations for users in its corresponding geographic region. This separation enables efficient resource utilization; the knowledge base remains consistent across the platform, while computational resources are allocated according to local demand patterns. The service model remains synchronized across all deployments, ensuring users receive consistent information regardless of which server processes their requests.

[0138] Multiple factors, evaluated in real-time by a distributed AI splitting and switching mechanism, determine which server (S1, S2, S3, or S4) handles a specific AI / ML task. Query characteristics, including complexity and data requirements, influence whether the local CPN server can handle the task or whether more powerful cloud resources are needed. Network conditions between the user device and each potential server determine which option provides the best response time. Privacy requirements may necessitate that certain queries remain within the CPN server to prevent sensitive data from traversing public networks. The current load of each server in the distributed infrastructure helps the system route requests to servers with available capacity. This mechanism continuously evaluates these factors to make optimal routing decisions, balancing performance, privacy, and resource utilization within the distributed server infrastructure.

[0139] Figure 15 This is a diagram (1500) illustrating the Retrieval Augmented Generation (RAG) framework as part of a distributed AI service splitting and switching mechanism. The RAG system provides a method for storing and retrieving personalized or specialized information that must be maintained locally on the user's device to protect privacy, or that contains domain-specific knowledge unavailable to general AI models deployed on web servers.

[0140] The RAG framework operates through a systematic process of information collection, embedding generation, and retrieval. Information flows into the system from various authorized sources, represented by information icons at the top of the graph. These sources include documents, notes, camera input for visual data, smartwatch data for health and activity metrics, and other forms of personal or professional data. Users or system operators must explicitly authorize the collection of this information to protect privacy and comply with data protection requirements.

[0141] The collected information is transformed into numerical representations called embeddings using specially designed machine learning models. These models convert diverse data types—whether text, visual, or sensor data—into fixed-dimensional vectors, capturing the semantic meaning of the original information. Embedding vectors are represented in the graph as numerical arrays, such as [0.5, 0.3, ... 0.1, ...] and [0.2, 0.1, ... 0.9, ...], encoding knowledge in a format that AI models can efficiently process and compare.

[0142] The vector database serves as the central repository for these embeddings, maintaining transformed representations of personal and specialized information. When a user submits a suggestion or query to the system, the RAG framework uses the same encoding process to transform that suggestion into an embedding. The system then performs a retrieval operation by comparing the query embedding with the embeddings stored in the vector database, identifying the most relevant information based on semantic similarity rather than exact keyword matching.

[0143] The retrieved information, presented as a vector [0.2, 0.1, ... 0.9, 0.8, 0.6, ... 0.7], provides context-relevant data that enhances the response generation capabilities of AI / ML models. This retrieval process allows the AI ​​assistant to access specific personal information or expertise when answering user queries, while keeping this sensitive data securely on the local device rather than transferring it to an external server.

[0144] The integration of the RAG framework with a distributed AI service splitting and switching mechanism provides crucial capabilities for privacy-preserving AI services. When the distributed decision-making mechanism identifies a query involving personal or privacy information, it instructs processing to remain on the local device. At this point, the RAG system can access the necessary personal context from the local vector database. This architecture allows the system to provide personalized responses without compromising user privacy, without transmitting sensitive data over the network.

[0145] The service engine component works in conjunction with the RAG framework to provide AI services in a distributed architecture. The service engine exists in two complementary forms: a smaller, optimized version deployed on user devices, and a larger, more powerful version deployed on web servers. This distributed deployment strategy allows the system to balance computational demands, privacy considerations, and response quality based on the specific characteristics of each user's query.

[0146] On user devices, the service engine integrates a curated selection of compact artificial intelligence / machine learning (AI / ML) models specifically chosen for efficient operation under mobile computing constraints. These models include parameter-reduced versions of language models, such as Llama3 with 8 billion parameters; vision-language models like LLaVA, offering 7 billion and 13 billion parameter configurations; and specific models like OpenChat and Llama3-chatQA optimized for conversational interactions. The selection of these models prioritizes efficiency and responsiveness while maintaining sufficient capacity to handle common queries and privacy-sensitive tasks that must remain on the device.

[0147] Web-hosted service engines feature a significantly larger and more diverse collection of AI / ML models. These server-side deployments include full-scale language models with tens of billions of parameters, comprehensive vision-language models with enhanced multimodal understanding capabilities, and domain-specific models that exceed the memory and computing power of mobile devices. The distributed architecture allows access to these powerful models when needed, while retaining the option of local processing where appropriate.

[0148] The service engine can also integrate complementary sub-modules to enhance core AI / ML capabilities. The speech-to-text module converts speech input into text format for language model processing. The text-to-speech module generates natural-sounding audio responses based on text output. Virtual assistant avatar generation creates visual representations for mixed reality applications. 3D rendering capabilities support immersive user interfaces when applications require visual interaction. These sub-modules can be strategically distributed across the network, with deployment decisions based on factors such as available computing resources, network latency, battery status, and the specific requirements of each interaction.

[0149] The coordination between the Retrieval Enhancement Generation (RAG) framework and the distributed service engine components enables the system to intelligently decide where each query should be processed. Personal queries that require access to information in the local RAG vector database are processed on the device to maintain privacy. Complex queries that require a more robust model but do not involve personal information can be offloaded to a web server. Queries that combine personal context with advanced processing needs can be processed in a hybrid manner, where the RAG system retrieves relevant personal context locally and enhances the query sent to the web server in a privacy-preserving manner without exposing raw personal data.

[0150] The inference performance estimate for the local model includes inference time (excluding response propagation latency). ), which is The function, model accuracy ( ), required computing power ( ), battery consumption ( and response data size () Because the local service engine resides on the user device, there is a delay in the transmission of responses from the service engine to the user device. It is considered zero. Based on the number of local model candidates and the capabilities of the user's device, multiple sets of inference performance estimates for multiple models can be prepared (e.g., , ,……as well as ).

[0151]

[0152] Performance estimation for inference using models in a remote network server service engine includes inference time (excluding response delivery latency). ), which is The function, model accuracy ( ), the amount of computing power required ( ), battery consumption ( ), response data size ( ), and the total network latency for transmitting responses via network connection status ( ).parameter Based on the total size of the response data that needs to be transmitted over the network ( Estimate the achievable network connectivity between user equipment and candidate network server service engines. or And the end-to-end round-trip latency between the user equipment and the candidate network server service engine ( The RTT (Round-Trip Time) can affect the actual throughput, depending on the network protocol used to deliver the response data, such as TCP or UDP. This refers to the achievable air interface data rate, which depends on the characteristics of RAT layers 1 and 2. This refers to the data rate at IP and higher layers. exist When unavailable, this becomes especially important. Depending on the number of candidate service engines for the remote web server and the capabilities of the user's device, multiple sets of inference performance estimates for multiple models and multiple service engines may be prepared (e.g., , ,……as well as ).

[0153]

[0154] The following two example pseudocode snippets represent advanced service engine selection logic, assuming only one inference performance estimate each for the local device and web server service engines. However, this pseudocode is scalable to accommodate more than two inference performance estimates. This pseudocode assumes that parameters are considered by the user-configured battery (…). This parameter serves as a strict priority factor for service engine selection. If If the value is 0, then the user device's battery status does not affect the service engine selection decision. If... The value is equal to 1 and the expected remaining device battery is below the user-defined threshold. If the battery is not charged, the service or application may be terminated. However, if the user device is charging, the battery constraint is excluded from the decision-making process. The alpha parameter ( This parameter determines the relative weighted importance of response time latency and model accuracy in the switching score calculation. When alpha approaches 1, the system prioritizes minimizing response time; when alpha approaches 0, the system prioritizes maximizing model accuracy. If this parameter is set too small, the impact of inference response time on the selection of the target service engine will be minimal.

[0155] Pseudocode 1 illustrates the advanced service engine selection logic that incorporates user device battery consumption estimates. This method employs two different battery consumption estimates to make a more accurate decision regarding service engine selection.

[0156] parameter This represents an estimate of the user device's battery consumption when performing inference on the user device, which is a function of the required computing power. calculate:

[0157] function Based on the device's power consumption characteristics during artificial intelligence / machine learning (AI / ML) processing tasks, computing demands are mapped to expected battery consumption.

[0158] parameter This represents an estimate of user device battery consumption when performing inference on a remote network server. In this scenario, user device battery consumption stems from data communication activities while receiving inference results from the network server. This consumption is a function of several network-related factors. calculate:

[0159] function Consider the power consumption during network transmission, which varies with the total size of the response data, network connectivity characteristics including channel rate or data rate, round-trip time, and the transport layer protocol used.

[0160] If the estimated remaining battery capacity of the device after inference is lower than If a candidate service engine is not selected due to battery constraints, the service or application may be terminated to extend battery life. If a user wishes to continue service regardless of battery status when no service engine is available due to battery limitations, they can choose to override the selection. Options for setting up switching scores. If both the local and remote service engines meet battery requirements, a switching score is calculated using a weighted combination of inference latency and model accuracy. This determines the optimal service engine.

[0161] The switching score formula balances two competing objectives: minimizing total response time and maximizing model accuracy. The first term of the formula normalizes the difference in total response time between remote and local processing, with a positive value indicating that local processing is faster. The second term represents the difference in accuracy between local and remote models, with a positive value indicating that the local model provides higher accuracy. The alpha parameter controls the relative importance of these two factors in the final decision.

[0162] Battery considerations = 0 or 1; # User configuration

[0163] if (Battery consideration == 1 AND Battery charging == False) {

[0164] Local battery consumption = g (local computing power)

[0165] Remote battery consumption = h(remote data size, channel rate or data rate, round-trip time, transmission protocol)

[0166] Local battery score = ((remaining battery power - local battery consumption) - battery threshold) / battery threshold

[0167] Remote battery score = ((remaining battery power - remote battery consumption) - battery threshold) / battery threshold

[0168] if (local battery score >= 0 AND remote battery score >= 0) {

[0169] Switching score = α (Remote inference time + Remote latency - Local inference time) /

[0170] max(remote inference time + remote latency, local inference time) +

[0171] (1 - α) (Local accuracy - Remote accuracy)

[0172] if (toggle score > 0) {

[0173] Service Engine Selection = Local Service Engine

[0174] } else {

[0175] Service Engine Selection = Remote Service Engine

[0176] }

[0177] } else if (local battery score >= 0) {

[0178] Service Engine Selection = Local Service Engine

[0179] } else if (remote battery score >= 0) {

[0180] Service Engine Selection = Remote Service Engine

[0181] } else {

[0182] exit

[0183] }

[0184] } else {

[0185] Switching score = α (Remote inference time + Remote latency - Local inference time) /

[0186] max(remote inference time + remote latency, local inference time) +

[0187] (1 - α) (Local accuracy - Remote accuracy)

[0188] if (toggle score > 0) {

[0189] Service Engine Selection = Local Service Engine

[0190] } else {

[0191] Service Engine Selection = Remote Service Engine

[0192] }

[0193] }

[0194] The method in pseudocode 1 provides strict control over the system through battery consumption estimation. However, this method requires additional computational overhead to estimate battery consumption, and the battery consumption of a single inference operation may not be sufficient to justify such detailed calculations. Therefore, the service engine selection logic can be simplified, as shown in pseudocode 2, which only compares the remaining battery power with a battery threshold during calculation, without estimating the consumption of the current inference operation.

[0195] Battery consideration = 0 or 1; # User input

[0196] if (Battery consideration == 1 AND Battery charging == False) {

[0197] Battery score = (Remaining battery capacity - Battery threshold) / Battery threshold

[0198] if (battery score > 0) {

[0199] Switching score = α (Remote inference time + Remote latency - Local inference time) /

[0200] max(remote inference time + remote latency, local inference time) +

[0201] (1 - α) (Local accuracy - Remote accuracy)

[0202] if (toggle score > 0) {

[0203] Service Engine Selection = Local Service Engine

[0204] } else {

[0205] Service Engine Selection = Remote Service Engine

[0206] }

[0207] } else {

[0208] exit

[0209] }

[0210] } else {

[0211] Switching score = α (Remote inference time + Remote latency - Local inference time) /

[0212] max(remote inference time + remote latency, local inference time) +

[0213] (1 - α) (Local accuracy - Remote accuracy)

[0214] if (toggle score > 0) {

[0215] Service Engine Selection = Local Service Engine

[0216] } else {

[0217] Service Engine Selection = Remote Service Engine

[0218] }

[0219] }

[0220] These two approaches represent different trade-offs between computational complexity and decision accuracy. The first approach, with battery consumption estimation, attempts to predict the battery impact of processing each query. When a user submits a query and expects a response, this approach predicts the length of the generated answer and the battery consumption required to generate that answer. If these predictions prove accurate, this approach can make more precise service engine selection decisions. However, prediction accuracy can vary, and the computational overhead of making these predictions may not be reasonable for typical queries.

[0221] The practical consideration behind this simplification approach stems from the observation that most artificial intelligence (AI) inference operations consume relatively little battery power. Unless the generated response is exceptionally long, such as tens or hundreds of pages, the battery consumption for processing a typical half-page or shorter query usually does not exceed 0.5% to 1% of the total battery capacity. Therefore, executing a single query without pre-estimating battery consumption does not lead to battery depletion issues.

[0222] The simplified approach implements a direct binary battery consideration mechanism. Users can configure system preferences, such as completely disabling battery consideration to always use the AI ​​service, or setting a battery threshold percentage to pause the AI ​​service when the battery level falls below that threshold. For example, users can configure the system to disable the AI ​​service when the battery level is below 20%. These represent strict, user-defined operational boundaries, rather than dynamic optimization.

[0223] When a user allows access to AI services under their configured battery settings, the simplified system assesses whether the current battery level exceeds a threshold to process the next prompt. The system then calculates a switching score, taking into account the processing time for answer generation, total network latency, and the difference in inference time between local and remote processing. Based on this comprehensive assessment, the system determines whether local or remote processing is more suitable for the current conditions. By eliminating battery consumption estimation, the simplified approach transforms complex calculations into a more direct decision-making process while still considering the key factors influencing user experience.

[0224] The decision between passive and active modes represents another dimension of the distributed AI service splitting and switching mechanism. Wireless access technologies supporting both Layer 1 and Layer 2 can operate in either passive or active mode when future network standards (such as the anticipated 6G system) provide the necessary interfaces.

[0225] In passive mode, user equipment monitors its network links without actively interacting with base stations or access points. The device observes radio conditions, link quality, and network performance metrics to inform its processing decisions. This method minimizes network overhead because it does not generate additional signaling traffic, but the accuracy of network condition assessment may be limited to observable metrics.

[0226] The proactive mode enables devices to actively request status information from the network infrastructure. Devices can query base stations or access points to obtain detailed status reports containing wireless conditions, available bandwidth, data transmission capabilities, and other network performance metrics. While this method generates additional network signaling traffic, it provides more accurate and timely information for making optimal service engine selection decisions.

[0227] Even when the base station does not support Layer 1 and Layer 2 interfaces, the distributed AI service mechanism can still operate through Layer 3 analysis. Layer 3 passive mode infers network conditions by monitoring network traffic patterns of other applications without generating probe traffic. The system tracks data volume and time patterns to estimate available bandwidth and network latency. Layer 3 active mode sends a controlled amount of probe traffic to more accurately measure network performance, accepting the trade-off of additional network overhead for higher measurement accuracy.

[0228] This architecture adapts to all four operational scenarios: Layer 1 / 2 support with a passive mode for minimal-overhead monitoring, Layer 1 / 2 support with an active mode for accurate network assessment, Layer 3 support with a passive mode for basic network inference, and Layer 3 support with an active mode for accurate IP layer measurements. This comprehensive approach enables the system to adapt to different network infrastructure capabilities while maintaining optimal service engine selection across various deployment environments.

[0229] Figure 16 Figure 1600 illustrates the operational modes of network analysis and access technology monitoring within a distributed artificial intelligence (AI) service splitting and switching mechanism. The figure distinguishes between two monitoring methods: an active / passive network (NW) mode associated with module B11 used for Layer 3 and above analysis, and an active / passive access technology (AT) mode associated with module B12 used for Layer 1 and Layer 2 analysis.

[0230] Network active mode, in Figure 16Marked as "Network Proactive Mode," this mode operates at the application layer and is used to measure end-to-end performance between user equipment and network servers. In this mode, the system sends uplink probe messages and receives downlink probe messages to directly measure throughput and latency characteristics. This proactive probing can evaluate performance using various predetermined message sizes during inference activity, providing accurate real-time measurements. However, this approach generates additional network traffic beyond normal application data. Depending on the probing strategy, the timing of performance measurements may not perfectly coincide with the actual inference time. When probing occurs immediately before inference operations, it can introduce additional latency to the inference process itself.

[0231] Network passive mode, in Figure 16 This method, labeled "Passive Network Mode," monitors existing user traffic performance without generating dedicated probe messages. It measures throughput and latency performance by observing existing message patterns of applications already running on the device. Because it doesn't generate additional probe traffic, this method avoids network overhead and eliminates the need for a dedicated traffic probe module. A limitation of this passive approach is that when network communication is infrequent, it may lack sufficient measurement logs to generate accurate network performance analysis.

[0232] Access technology active mode, in Figure 16 The "AT Active Mode" provides a different monitoring method that operates at the radio access layer. In this mode, the user equipment (UE) actively requests access technology status information from the base station or Wi-Fi access point. The device sends an AT status information request and receives a response containing detailed radio layer metrics. Unlike the network-active mode, which generates user traffic on the data plane, the AT Active Mode operates via control plane signaling. The client obtains the estimated available link capacity of the UE directly from the "Network Analytics (L1+L2)" module deployed in the access point infrastructure (such as Wi-Fi APs or cellular base stations). This method provides accurate radio layer information without consuming user data bandwidth.

[0233] Access technology passive mode, in Figure 16 Marked as "AT Passive Mode," this mode acquires existing local AT status information within the user equipment without communicating with the network infrastructure. The device monitors its own wireless interface status, including signal strength, modulation parameters, and error rate, using information maintained by the device's wireless subsystem. This mode operates entirely within the device, eliminating any network signaling overhead while still providing basic wireless condition awareness.

[0234] The hybrid approach combines elements of both active and passive methods to maximize their respective advantages. The hybrid strategy between active and passive modes can be dynamically adjusted based on the current network communication patterns. When network traffic is high, the system may rely more on passive monitoring to avoid unnecessary overhead. When accurate measurement is critical but natural traffic is sparse, the system may increase active probing to maintain measurement accuracy. While this hybrid approach is more complex to implement than purely active or purely passive methods, it offers flexibility to adapt to different network conditions and application requirements.

[0235] Module B12, designated as the communications module, covers all network connectivity technologies available for wireless and wired transmission on the user equipment. This module primarily handles Layer 1 (Physical Layer) and Layer 2 (Data Link Layer) information, with specific parameters varying depending on the network access technology used. For wireless access technologies such as Wi-Fi, 4G, and 5G, typical parameters include modulation and coding scheme (MCS), signal-to-interference-noise ratio (SINR), and block error rate (BLER). For 5G networks, additional parameters such as Access Traffic Bootstrapping-Switching-Offloading (ATSSS) are defined according to 3GPP specifications.

[0236] Access network status monitoring in Module B12 can employ active, passive, or hybrid methods similar to those used in Module B11 for network analysis, but the operations differ. For the active method, Module B12 deploys a corresponding module called "Network Analysis (L1+L2)" in the service broker and base station infrastructure, such as... Figure 14B As shown. The key difference between Module B12 and Module B11's active mode lies in the way information is obtained. Module B12's active mode does not insert additional user traffic into the air interface. Instead, it obtains the estimated available link capacity from the client through control plane signaling with the "Network Analysis (L1+L2)" module in the access network infrastructure.

[0237] When a "network analytics (L1+L2)" module is not deployed in the access point or base station, meaning that current 5G deployments lack standardized APIs for accessing Layer 1 and Layer 2 information, the system reverts to a passive monitoring approach. This limitation of the current network infrastructure requires reliance on the device's local observation of wireless conditions, rather than authoritative information from the network side.

[0238] In order for the network analysis module B11 to utilize the connection state information collected by the module B12, an application programming interface (API) must exist between the two. This API facilitates cross-protocol layer optimization, enabling higher-level network analysis to benefit from lower-level radio information. This interface is provided by... Figure 17The "control" arrows in the diagram illustrate the parameters and data structures exchanged between the device analysis, network analysis, and communication modules. This cross-layer information sharing enables the distributed AI service splitting and switching mechanism to make informed decisions based on a comprehensive understanding of wireless layer conditions and end-to-end network performance.

[0239] Figure 17 Figure 1700 illustrates the parameters used for user device analysis and the conceptual architecture of the application programming interface (API) for cross-layer information exchange that enables the distributed AI service splitting and switching mechanism. The accuracy of decisions involving distributing AI processing across user devices and network nodes depends on a comprehensive analysis of device status and network connectivity conditions. Figure 17 The specific parameters required for this analysis are shown, and the API framework for obtaining these parameters across different protocol layers and system boundaries is described.

[0240] like Figure 17 The architecture shown is divided into two distinct partitions separated by a Host Interface Abstraction Layer (HIAL). The host partition 1702, located above the HIAL, contains upper-layer software components, including applications, transport protocols, IP networking, and device and network analysis modules. The hardware partition 1704, located below the HIAL, encompasses physical layer components, including multiple RAT protocol stacks supporting various Radio Access Technologies (RATs), such as RAT-1 to RAT-n, each RAT implementing its corresponding Layer 1 physical protocol and Layer 2 data link protocol.

[0241] The Host Interface Abstraction Layer (HIAL) is a software component that abstracts and standardizes the communication interface between the host system and the underlying hardware or external devices. This abstraction layer acts as an intermediary, enabling the host system to communicate with various hardware components and wireless access technologies using unified commands and protocols, independent of specific hardware implementation details. This architectural approach allows the API framework for distributed AI service splitting and switching mechanisms to operate independently of whether the host and hardware components are physically integrated (as in a smartphone) or separated across different devices. The host-side HIAL interface maintains compatibility with various user device types, providing platform independence, portability, flexibility, and scalability for distributed AI service deployment.

[0242] The communication device implementing this architecture comprises two distinct operational planes coexisting within the system. The data plane includes modules responsible for handling the actual flow of data through the device. These modules move data at the HIAL boundary and modify the data flow during runtime operations. The data plane processes the actual AI inference requests and responses flowing between user applications and local or remote AI models. An upper-layer application interface, located above the Media Access Control (MAC) layer, provides a mechanism for packet exchange between upper-layer applications and the underlying communication stack.

[0243] The control plane consists of modules that do not directly modify or process user data streams; instead, they are responsible for system configuration and monitoring. These control plane modules configure different data plane components to meet specific operational needs and provide APIs to exchange device and network status information via HIAL. The control plane API enables the distributed AI service splitting and switching mechanism to collect the parameters needed to make informed decisions to determine where AI workloads should be processed.

[0244] The device analytics module needs to access critical hardware and resource information via the control plane API. These parameters include CPU utilization, indicating the current processing load and available computing power for local AI model execution. Memory availability determines whether there is sufficient RAM for locally loading and running AI models. Remaining battery parameters provide the current battery level as a percentage, while the battery charging rate indicates whether the battery is charging or discharging and at what rate. Charging mode distinguishes between different power sources, such as wall adapter charging versus USB charging, which affects the available power budget. Geolocation information provides the device's current physical location, which can influence the selection of nearby edge servers to reduce latency.

[0245] The network analysis module needs to access Layer 1 and Layer 2 information elements via the HIAL control plane API. The amount of data waiting to be transmitted in the transmit buffer indicates the data backlog, reflecting the current network congestion status. The list of supported RATs identifies the wireless access technologies that the device can utilize, such as Wi-Fi, 4G, 5G, or Bluetooth. Support for multiple RAT technologies, including Access Traffic Steering, Switching, and Splitting (ATSSS) and Multipath TCP (MPTCP), indicates that the device can utilize multiple network connections simultaneously to improve performance. RAT status parameters include Received Signal Strength Indicator (RSSI), Modulation and Coding Scheme (MCS), Signal to Interference and Noise Ratio (SINR), and Block Error Rate (BLER), providing detailed wireless link quality metrics. The idle or active state of each wireless interface affects power consumption and connection establishment latency.

[0246] For Layer 3 and above network analysis, the module monitors throughput performance to understand the actual data transfer rates achievable on the current network connection. Network latency performance measurements capture the end-to-end latency between user devices and potential AI processing servers in the network. These higher-level metrics complement the wireless layer information, providing a complete view of network capabilities for decisions regarding distributed AI service splitting and switching.

[0247] Device hardware and resource information, as well as Layer 1 and Layer 2 wireless parameters, must be exposed to higher layers via the HIAL control plane API. This exposure enables device analytics and network analytics modules located above the host partition HIAL to access real-time hardware and wireless status information. Figure 17 The configuration module shown receives user input to configure operating parameters, such as preventing the AI ​​service from being uninstalled to keep the device running when the battery level is below a minimum remaining battery threshold. The device and network analytics modules also establish direct communication with the transport and IP layers within the host partition to obtain accurate throughput measurements and latency estimates, reflecting real-world application layer performance.

[0248] This comprehensive parameter collection and API framework enables distributed AI service splitting and switching mechanisms to make informed decisions based on current device capabilities, battery constraints, and network conditions. The cross-layer information flow facilitated by the HIAL API allows the system to optimize AI processing distribution in real time, balancing factors such as computing demands, power consumption, network latency, and privacy, while providing the best user experience while protecting device resources.

[0249] Figure 18 This is a flowchart (1800) illustrating a method for making decisions regarding the splitting and switching of distributed AI services. This method can be executed by a device (such as a user device or smartphone). In operation 1802, the device assesses the processing requirements of the AI ​​service, the resources of the local device, the resources of the server, and the network connectivity resources between the local device and the server. In operation 1804, based on the assessment results and a set of decision criteria for distributed AI splitting and switching, the device splits the AI ​​service into multiple parts. This set of decision criteria includes privacy requirements, real-time response requirements, and quality-latency tradeoffs. In operation 1806, based on the assessment results and the set of decision criteria, the device switches the corresponding split portion of the AI ​​service between the local device and the server.

[0250] In some implementations, AI services include virtual and personal assistant mixed reality services or collaborative AI personal networks (PN) services with distributed generative AI. Servers include service clusters deployed across multiple sites or networks for load balancing, reliability, or performance optimization.

[0251] In some implementations, to switch the corresponding split of the AI ​​service, the device can run the AI ​​service independently on the local device's PN, run the AI ​​service on the local device's PN and a server in the edge cloud, run the AI ​​service on the local device's PN and a PN on another user device through information sharing, or run the AI ​​service on the local device's PN and a server in the Customer Premises Equipment (CPE) through knowledge synchronization.

[0252] In some implementations, the server includes one or more service models and one or more service engines. These service models and engines are deployed in different locations.

[0253] In some implementations, both the local device and the server include multiple AI models with different specializations. These multiple AI models with different specializations include language models, vision models, speech recognition models, or anomaly detection models. The number of parameters varies among these multiple AI models in each category.

[0254] In some implementations, the decision criteria further include: for real-time applications requiring real-time response, executing AI services on local devices to reduce latency; for applications prioritizing quality over latency, executing AI services on servers; or for hybrid environments, primarily executing AI services on local devices, but falling back to server execution when local devices cannot support AI services, or when higher quality is required and network conditions permit.

[0255] In some implementations, to switch the corresponding split of the AI ​​service, the device analyzes the user prompt and selects an AI model for a specific application or modality based on the nature of the prompt. In response to the user prompt involving personal or privacy information, the device uses a Retrieval Augmented Generation (RAG) framework to retrieve relevant information from a local vector database and generates a response within the local device.

[0256] In some implementations, the device utilizes a pre-trained initial inference prediction model to predict the inference performance of a specific AI model for a given cue in various computing environments. Metrics for this inference performance include one or any combination of inference latency, throughput, model accuracy, computational power, size of the response data to be transmitted, power consumption, and energy efficiency.

[0257] In some implementations, the device uses user behavior models and user prompts to predict user-specific inference response time requirements. The device compares this inference response time requirement with the predicted inference performance. Based on the comparison result, the device selects either a local service engine on the local device or a server service engine on the server.

[0258] In some implementations, the device applies a battery constraint to exclude service engines from the candidate list if the estimated or calculated remaining battery power is below a user-defined threshold. This battery constraint is ignored if the local device is charging.

[0259] In some implementations, to select the local service engine or the server service engine, the device calculates a handover score based on a weighted combination of inference time and accuracy differences between the local device and the server. Weighting parameters determine the relative importance of response time and accuracy. This handover score incorporates battery consumption estimates. Local battery consumption is estimated as a function of local computing power, while remote battery consumption is estimated as a function of response data size, channel rate or data rate, round-trip time, and transmission protocol.

[0260] In some implementations, the device performs device analysis to determine the resources of the local device. The resources of the local device include one or any combination of CPU utilization, memory availability, remaining battery power, battery charging rate, battery charging mode, and geographic location information.

[0261] In some implementations, the device performs network analysis to determine network connectivity resources between the local device and the server. These network connectivity resources include one or any combination of link capacity status, predicted link capacity status, and network latency. The network analysis includes analyzing throughput performance and network latency performance between the local device running the AI ​​service and the server.

[0262] In some implementations, the throughput and network latency performance are measured using an active method. This active method is performed by estimating the throughput and network latency performance using a dedicated traffic probing module between the local device and the server.

[0263] In some implementations, the throughput and network latency performance are measured using a passive method. This passive method is performed by measuring the throughput and network latency performance without generating additional traffic using existing messaging.

[0264] In some implementations, the throughput and network latency performance are measured using a hybrid approach that combines active and passive methods. This hybrid approach dynamically adjusts the combination strategy between the active and passive methods based on network communication usage.

[0265] It should be understood that the specific order or hierarchy of the modules in the disclosed process / flowchart is merely exemplary. Based on design preferences, it should be understood that the specific order or hierarchy of the modules in the process / flowchart can be rearranged. Furthermore, some modules can be combined or omitted. The accompanying method claims present the elements of the modules in an exemplary order and are not limited to the specific order or hierarchy shown.

[0266] The foregoing description is intended to enable those skilled in the art to implement the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to limit the aspects shown herein, but rather to provide a full scope consistent with the language of the claims, wherein a singular reference to an element does not mean “only one” unless explicitly stated otherwise, but rather “one or more.” The term “exemplary” is used herein to mean “as an example, instance, or illustration.” Any aspect described as “exemplary” is not necessarily to be construed as superior to other aspects. Unless otherwise explicitly stated, the term “some” means one or more. Combinations such as “at least one A, B, or C,” “one or more A, B, or C,” “at least one A, B, and C,” “one or more A, B, and C,” and “A, B, C, or any combination thereof” include any combination of A, B, and / or C, and may include multiple A, multiple B, or multiple C. Specifically, combinations such as "at least one A, B, or C", "one or more A, B, or C", "at least one A, B, and C", "one or more A, B, and C", and "A, B, C, or any combination thereof" can be only A, only B, only C, A and B, A and C, B and C, or A, B, and C, wherein any such combination may include one or more members of A, B, or C. All structural and functional equivalents of the various aspects described in this disclosure, whether known or subsequently known to those skilled in the art, are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, the disclosure herein is not intended for public disclosure, whether or not it is expressly recited in the claims. The terms "module," "mechanism," "element," "device," etc., are not substitutes for "means." Therefore, no claim element should be construed as means plus function unless the element expressly uses the phrase "means for..."

Claims

1. A wireless communication method for a device, comprising: Assess the processing requirements of the artificial intelligence (AI) service, the resources of the local device, the resources of the server, and the network connectivity resources between the local device and the server; Based on the assessment results and a set of decision criteria for the distributed AI splitting and switching, the AI ​​service was split into multiple parts, including privacy requirements, real-time response requirements, and quality-latency trade-offs. as well as Based on the assessment results and the decision-making criteria, the corresponding split of the AI ​​service was switched between the local device and the server.

2. The method of claim 1, wherein the AI ​​service includes a virtual and personal assistant mixed reality service or a collaborative AI personal network (PN) service with distributed generative AI, and wherein the server includes a service cluster deployed across multiple sites or networks for load balancing, reliability, or performance optimization.

3. The method of claim 2, wherein switching the corresponding split portion of the AI ​​service includes: The AI ​​service can be run independently on the local device's PN, on the local device's PN and on the server in the edge cloud, through information sharing on the local device's PN and on another user's PN, or through knowledge synchronization on the local device's PN and on the server in the Customer Premises Equipment (CPE).

4. The method of claim 1, wherein the server includes one or more service models and one or more service engines, and wherein the one or more service models and the one or more service engines are deployed in different locations.

5. The method of claim 1, wherein both the local device and the server include multiple AI models with different specializations.

6. The method of claim 5, wherein the plurality of AI models with different specializations include a language model, a vision model, a speech recognition model, or an anomaly detection model, and wherein the plurality of AI models in each category differ in the number of parameters.

7. The method of claim 1, wherein the set of decision criteria further comprises: For real-time applications that require real-time responses, the AI ​​service is executed on the local device to reduce latency; For applications where high quality is prioritized over latency, execute the AI ​​service on this server; or In a hybrid environment, the AI ​​service is primarily executed on the local device, but when the local device cannot support the AI ​​service, or when higher quality is required and network conditions permit, it is rolled back to the server for execution.

8. The method of claim 1, wherein switching the corresponding split portion of the AI ​​service comprises: Analyze user prompts; as well as Based on the nature of this user prompt, select an AI model for a specific application or modality.

9. The method of claim 8, wherein in response to the user prompt involving personal or privacy information, relevant information is retrieved from a local vector database using a Retrieval Enhanced Generation (RAG) framework, and a response is generated within the local device.

10. The method of claim 1, further comprising: Using a pre-trained initial inference prediction model, predict the inference performance of a specific AI model for a given prompt in various computing environments, wherein the inference performance metrics include one or any combination of inference latency, throughput, model accuracy, computing power, size of the response data to be transmitted, power consumption, and energy efficiency.

11. The method of claim 10, further comprising: Using user behavior models and user cues, predict users’ specific reasoning response time requirements; Compare the inference response time requirement with the inference performance of the prediction; as well as Based on the comparison results, either the local service engine in the local device or the server service engine in the server is selected.

12. The method of claim 11, further comprising: Apply a battery constraint to exclude service engines from the candidates if the estimated or calculated remaining battery power of the device is below a user-defined threshold, where the battery constraint is ignored if the local device is charging.

13. The method of claim 12, wherein selecting the local service engine or the server service engine comprises: A switching score is calculated based on a weighted combination of the differences in inference time and accuracy between the local device and the server, where the weight parameters determine the relative importance of response time and accuracy.

14. The method of claim 13, wherein the handover score incorporates a battery consumption estimate, and wherein local battery consumption is estimated as a function of local computing power, and remote battery consumption is estimated as a function of response data size, channel rate or data rate, round-trip time, and transmission protocol.

15. The method of claim 1, further comprising: Perform device analysis to determine the resources of the local device, wherein the resources of the local device include one or any combination of CPU utilization, memory availability, remaining battery power, battery charging rate, battery charging mode, and geolocation information.

16. The method of claim 1, further comprising: Perform network analysis to determine network connectivity resources between the local device and the server, wherein the network connectivity resources include one or any combination of link capacity status, predicted link capacity status, and network latency.

17. The method of claim 16, wherein the network analysis includes analyzing throughput and network latency performance between the local device and the server running the AI ​​service.

18. The method of claim 17, wherein the throughput performance and the network latency performance are measured using an active method, wherein the active method is performed by estimating the throughput performance and the network latency performance using a dedicated traffic probing module between the local device and the server.

19. The method of claim 17, wherein the throughput performance and the network latency performance are measured using a passive method, wherein the passive method is performed by measuring the throughput performance and the network latency performance without generating additional traffic using existing messaging.

20. The method of claim 17, wherein the throughput performance and the network latency performance are measured using a hybrid method combining an active method and a passive method, wherein the hybrid method dynamically adjusts the hybrid strategy between the active method and the passive method based on network communication usage.

21. An apparatus for wireless communication, comprising: Memory; as well as At least one processor coupled to the memory is configured as follows: Assess the processing requirements of the artificial intelligence (AI) service, the resources of the local device, the resources of the server, and the network connectivity resources between the local device and the server; Based on the assessment results and a set of decision criteria for the distributed AI splitting and switching, the AI ​​service was split into multiple parts, including privacy requirements, real-time response requirements, and quality-latency trade-offs. as well as Based on the assessment results and the decision-making criteria, the corresponding split of the AI ​​service was switched between the local device and the server.

22. A computer-readable medium storing computer-executable code for wireless communication of a device, comprising code for: Assess the processing requirements of the artificial intelligence (AI) service, the resources of the local device, the resources of the server, and the network connectivity resources between the local device and the server; Based on the assessment results and a set of decision criteria for the distributed AI splitting and switching, the AI ​​service was split into multiple parts, including privacy requirements, real-time response requirements, and quality-latency trade-offs. as well as Based on the assessment results and the decision-making criteria, the corresponding split of the AI ​​service was switched between the local device and the server.

Images