Method, architecture, apparatus, and system for distributed artificial intelligence

Abstract

A method, a method, an architecture, an apparatus, a system, a device, and a computer program product for distributed artificial intelligence (AI), in which a first device partitions a unit of media data corresponding to a media content item into a plurality of media segments, determines for each media segment whether inference using a machine learning model, a machine learning unit, and an inference unit of the first device is to be performed externally to the first device or in the first device, transmits content data corresponding to the media segment and information indicating the determined machine learning model for inference and the machine learning unit, and obtains processed data in which the content data is processed using at least the machine learning unit.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of European Application No. 22306669.7, filed November 4, 2022, which is incorporated herein by reference in its entirety. [Background technology]

[0002] The present disclosure relates generally to the fields of communications, software, and coding, including, for example, methods, architectures, apparatus, and systems directed to collaborative artificial intelligence (AI). Summary of the Invention

[0003] In a first aspect, the present principles relate to a method including: a first device partitioning a unit of media data corresponding to a media content item into a plurality of media segments; determining, for each media segment, a machine learning model, a machine learning unit, and whether inference using the machine learning unit is to be performed external to the first device or in an inference unit of the first device; transmitting content data corresponding to the media segment and information indicative of the determined machine learning model for inference and machine learning unit; and obtaining processed data resulting from processing the content data using at least the machine learning unit.

[0004] In a second aspect, the present principles relate to a first device comprising at least one hardware processor configured to partition a unit of media data corresponding to a media content item into a plurality of media segments, determine for each media segment a machine learning model, a machine learning unit, and whether inference using the machine learning unit is to be performed external to the first device or by an inference unit of the first device, transmit content data corresponding to the media segment and information indicative of the determined machine learning model for inference and machine learning unit, and obtain processed data resulting from processing the content data using at least the machine learning unit. [Brief explanation of the drawings]

[0005] A more detailed understanding can be had from the following detailed description, given in conjunction with the drawings that accompany this specification by way of example. As with the detailed description, the figures in such drawings are examples. As such, the figures and detailed description should not be considered limiting, as other equally effective examples are possible and possible. Furthermore, like reference numerals ("ref") within the figures indicate like elements. [Figure 1A] FIG. 1 is a system diagram illustrating an example communication system. [Figure 1B] 1B is a system diagram illustrating an example wireless transmit / receive unit (WTRU) that may be used within the communication system shown in FIG. 1A. [Figure 1C] 1B is a system diagram illustrating an example radio access network (RAN) and an example core network (CN) that may be used within the communication system shown in FIG. 1A. [Figure 1D] FIG. 1B is a system diagram illustrating a further exemplary RAN and a further exemplary CN that may be used within the communication system shown in FIG. 1A. [Figure 2] Here are different examples of splitting AI / ML models: [Figure 3] 1 shows an example of a split topology where the UE captures sensing / media data and acts as an uplink data source. [Figure 4] 1 shows a flowchart of a method for UE content processing in a distributed AI / ML environment. [Figure 5] 1 illustrates an exemplary system according to an embodiment of the present principles. [Figure 6] 1 illustrates an example of inference according to an embodiment of the present principles. [Figure 7A] 7 illustrates one embodiment of the data flow for the example shown in FIG. 6. [Figure 7B] 7 illustrates one embodiment of the data flow for the example shown in FIG. 6. [Figure 7C] 7 illustrates one embodiment of the data flow for the example shown in FIG. 6. DETAILED DESCRIPTION OF THE INVENTION

[0006] In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments and / or examples described herein. It will be understood, however, that such embodiments and examples may be practiced without some or all of the specific details set forth herein. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the following description. Furthermore, embodiments and examples not specifically described herein may be practiced in place of, or in combination with, embodiments and other examples provided explicitly, implicitly, and / or inherently (collectively "provided") herein, as described, disclosed, or otherwise. Although various embodiments are described and / or claimed herein in which apparatuses, systems, devices, etc. and / or any elements thereof perform operations, processes, algorithms, functions, etc. and / or any portions thereof, it should be understood that any embodiment described and / or claimed herein assumes that any apparatus, system, device, etc. and / or any elements thereof are configured to perform any operations, processes, algorithms, functions, etc. and / or any portions thereof.

[0007] Exemplary Communication System The methods, apparatus, and systems provided herein are well suited for communications involving both wired and wireless networks. An overview of various types of wireless devices and infrastructure is provided with respect to Figures 1A-1D, and various elements of the networks may utilize, perform, be arranged according to, and / or be adapted and / or configured for the methods, apparatus, and systems provided herein.

[0008] 1A is a system diagram illustrating an example communication system 100 in which one or more disclosed embodiments can be implemented. The communication system 100 can be a multi-access system that provides content, such as voice, data, video, messaging, broadcasts, etc., to multiple wireless users. The communication system 100 can enable the multiple wireless users to access such content through sharing of system resources, including wireless bandwidth. For example, the communication system 100 can employ one or more channel access schemes, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tailed (ZT) unique word (UW) discrete Fourier transform (DFT) spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multicarrier (FBMC), etc.

[0009] 1A, communications system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, radio access networks (RANs) 104 / 113, core networks (CNs) 106 / 115, public switched telephone networks (PSTNs) 108, the Internet 110, and other networks 112, although it will be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include (or may be) user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a mobile phone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (IoT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, medical equipment and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in the context of an industrial and / or automated processing chain), a consumer electronics device, a device operating in a commercial and / or industrial wireless network, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may be referred to interchangeably as a UE.

[0010] The communications system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communications networks, such as, for example, the CN 106 / 115, the Internet 110, and / or the network 112. By way of example, the base stations 114a, 114b may be any of a base transceiver station (BTS), a Node-B (NB), an eNode-B (eNB), a Home Node-B (HNB), a Home eNode-B (HeNB), a gNode-B (gNB), a NR Node-B (NR NB), a site controller, an access point (AP), a wireless router, etc. Although the base stations 114a, 114b are each shown as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0011] The base station 114a may be part of the RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), a relay node, etc. The base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies may be licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide wireless service coverage for a particular geographic area, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one for each sector of the cell. In one embodiment, the base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each or any sector of the cell. For example, beamforming may be used to transmit and / or receive signals in desired spatial directions.

[0012] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0013] More specifically, as noted above, the communications system 100 may be a multiple-access system, but may use one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base station 114a of the RAN 104 / 113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 116 using Wideband CDMA (WCDMA). WCDMA may include communications protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink Packet Access (HSDPA) and / or High-Speed ​​Uplink Packet Access (HSUPA).

[0014] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).

[0015] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR radio access, which may establish the air interface 116 using New Radio (NR).

[0016] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may jointly implement LTE radio access and NR radio access, e.g., using dual connectivity (DC) principles. Thus, the air interface utilized by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to and from multiple types of base stations (e.g., eNBs and gNBs).

[0017] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a wireless technology such as IEEE 802.11 (i.e., Wireless Fidelity (Wi-Fi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rates for GSM Evolution (EDGE), GSM EDGE (GERAN), or the like.

[0018] 1A may be, for example, a wireless router, a Home Node B, a Home eNode B, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, etc. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish either a small cell, a picocell, or a femtocell. 1A, the base station 114b may be directly connected to the Internet 110. Therefore, the base station 114b may not need to access the Internet 110 via the CN 106 / 115.

[0019] The RAN 104 / 113 may communicate with the CN 106 / 115, which may be any type of network configured to provide voice, data, application, and / or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have various quality of service (QoS) requirements, such as different throughput, latency, error tolerance, reliability, data throughput, mobility, etc. The CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions such as user authentication. Although not shown in FIG. 1A , it will be understood that the RAN 104 / 113 and / or the CN 106 / 115 may communicate directly or indirectly with other RANs that use the same RAT as the RAN 104 / 113 or a different RAT. For example, in addition to being connected to a RAN 104 / 113 that may utilize NR radio technology, the CN 106 / 115 may also communicate with another RAN (not shown) that uses any of GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or Wi-Fi radio technologies.

[0020] The CN 106 / 115 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network providing plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices, which use common communication protocols such as the transmission control protocol (TCP), the user datagram protocol (UDP), and / or the internet protocol (IP) of the TCP / IP Internet protocol suite. The network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the network 112 may include another CN connected to one or more RANs, which may use the same RAT as the RAN 104 / 114 or a different RAT.

[0021] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links.) For example, the WTRU 102c shown in FIG. 1A may be configured to communicate with a base station 114a that may employ cellular-based wireless technology and with a base station 114b that may employ IEEE 802.2 wireless technology.

[0022] 1B is a system diagram illustrating an example WTRU 102. As shown in FIG. 1B, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other elements / peripherals 138. It will be understood that the WTRU 102 may include any subcombination of the foregoing elements while remaining consistent with an embodiment.

[0023] The processor 118 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other function that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. While FIG. 1B depicts the processor 118 and the transceiver 120 as separate components, it will be understood that the processor 118 and the transceiver 120 may be integrated together, for example, in an electronic package or chip.

[0024] The transmit / receive element 122 can be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) over the air interface 116. For example, in one embodiment, the transmit / receive element 122 can be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive element 122 can be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals, for example. In one embodiment, the transmit / receive element 122 can be configured to transmit and / or receive both RF and light signals. It will be understood that the transmit / receive element 122 can be configured to transmit and / or receive any combination of wireless signals.

[0025] 1B as a single element, the WTRU 102 may include any number of transmit / receive elements 122. For example, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0026] The transceiver 120 may be configured to modulate signals transmitted by the transmit / receive element 122 and demodulate signals received by the transmit / receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs, such as, for example, NR and IEEE 802.11.

[0027] The processor 118 of the WTRU 102 may be coupled to and may receive user-entered data from a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) or organic light-emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. Additionally, the processor 118 may access information from and store data in any type of suitable memory, such as non-removable memory 130 and / or removable memory 132. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, etc. In other embodiments, the processor 118 may access information from and store data in memory that is not physically located on the WTRU 102, such as on a server or home computer (not shown).

[0028] The processor 118 may receive power from the power source 134 and may be configured to distribute and / or control the power to other components in the WTRU 102. The power source 134 may be any suitable device for providing power to the WTRU 102. For example, the power source 134 may include one or more dry batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0029] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or instead of, information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) over the air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may obtain location information by way of any suitable location-determination method while remaining consistent with an embodiment.

[0030] The processor 118 may further be coupled to other elements / peripherals 138, which may include one or more software and / or hardware modules / units that provide additional features, functionality, and / or wired or wireless connectivity. For example, the elements / peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (e.g., for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, an internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, etc. The elements / peripherals 138 may include one or more sensors, which may be one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor, a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0031] The WTRU 102 may include a full-duplex radio where transmission and reception of some or all of the signals (e.g., associated with a particular subframe for both the uplink (e.g., for transmission) and downlink (e.g., for reception)) may be parallel and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference either through hardware (e.g., a choke) or signal processing via a processor (e.g., via a separate processor (not shown) or processor 118). In one embodiment, the WTRU 102 may include a half-duplex radio for transmission and reception of some or all of the signals (e.g., associated with a particular subframe for either the uplink (e.g., for transmission) or downlink (e.g., for reception)).

[0032] 1C is a system diagram illustrating the RAN 104 and the CN 106 according to an embodiment. As mentioned above, the RAN 104 may communicate with the WTRUs 102a, 102b, and 102c over the air interface 116 using E-UTRA radio technology. The RAN 104 may also communicate with the CN 106.

[0033] The RAN 104 may include eNode-Bs 160a, 160b, and 160c, although it will be understood that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, and 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, and 102c over the air interface 116. In one embodiment, the eNode-Bs 160a, 160b, and 160c may implement MIMO technology. Thus, the eNode-B 160a may, for example, use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a.

[0034] Each of the eNode-Bs 160a, 160b, and 160c may be associated with a particular cell (not shown) and configured to handle radio resource management decisions, handover decisions, scheduling of users in the uplink (UL) and / or downlink (DL), etc. As shown in FIG. 1C, the eNode-Bs 160a, 160b, 160c may communicate with each other via an X2 interface.

[0035] 1C may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (PGW) 166. Although each of the foregoing elements is shown as part of the CN 106, it will be understood that any one of these elements may be owned and / or operated by an entity other than the CN operator.

[0036] The MME 162 may be connected to each of the eNode-Bs 160a, 160b, and 160c in the RAN 104 via an S1 interface and may function as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, activating / deactivating bearers, selecting a particular serving gateway during initial connection of the WTRUs 102a, 102b, 102c, etc. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA.

[0037] The SGW 164 may be connected to each of the eNode-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to and from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions such as anchoring the user plane during inter-eNode-B handovers, triggering paging when DL data is available to the WTRUs 102a, 102b, 102c, managing and storing context for the WTRUs 102a, 102b, 102c, etc.

[0038] The SGW 164 may be connected to a PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks such as the Internet 110 to facilitate communication between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0039] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional landline communications devices. For example, the CN 106 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0040] Although the WTRU is depicted in FIGS. 1A-1D as a wireless terminal, it is contemplated that in certain representative embodiments such a terminal may use a wired communication interface (e.g., temporarily or permanently) with the communication network.

[0041] In an exemplary embodiment, the other network 112 may be a WLAN.

[0042] A WLAN in infrastructure basic service set (BSS) mode may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access or interface to a distribution system (DS) or another type of wired / wireless network that carries traffic into and / or out of the BSS. Traffic originating from outside the BSS to a STA may arrive through the AP and be delivered to the STA. Traffic originating from a STA to a destination outside the BSS may be sent to the AP and transmitted to the respective destination. Traffic between STAs within a BSS may be transmitted, for example, through the AP, where the source STA may send traffic to the AP, and the AP may deliver the traffic to the destination STA. Traffic between STAs within a BSS may be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic may be transmitted between (e.g., directly between) a source STA and a destination STA using a direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunneled DLS (TDLS). A WLAN using an Independent BSS (IBSS) mode may not have an AP, and STAs within or using the IBSS (e.g., all of the STAs) may communicate directly with each other. The IBSS communication mode is sometimes referred to herein as an "ad hoc" communication mode.

[0043] When using 802.11ac infrastructure mode operation or a similar mode of operation, an AP may transmit beacons on a fixed channel, such as a primary channel. The primary channel may be a fixed width (e.g., a wide 20 MHz bandwidth) or a width dynamically set via signaling. The primary channel may be the operating channel of the BSS and may be used by STAs to establish a connection with the AP. In certain representative embodiments, carrier sense multiple access with collision avoidance (CSMA / CA) may be implemented, for example, in an 802.11 system. With CSMA / CA, STAs (e.g., all STAs), including the AP, may sense the primary channel. If the primary channel is detected / sensed and / or determined to be busy by a particular STA, the particular STA may back off. One STA (e.g., only one station) may transmit on a particular BSS at any time.

[0044] High throughput (HT) STAs may use a 40 MHz wide channel for communication, for example, through a combination of a primary 20 MHz channel with adjacent or non-adjacent 20 MHz channels to form a 40 MHz wide channel.

[0045] A very high throughput (VHT) STA may support channels of 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz width. A 40 MHz and / or 80 MHz channel may be formed by combining contiguous 20 MHz channels. A 160 MHz channel may be formed by combining eight contiguous 20 MHz channels or two non-contiguous 80 MHz channels, which may be referred to as an 80+80 configuration. For the 80+80 configuration, after channel encoding, the data may be passed through a segment parser that may separate the data into two streams. Inverse fast Fourier transform (IFFT) processing and time-domain processing may be performed separately on each stream. The streams may be mapped to two 80 MHz channels, and the data may be transmitted by the transmitting STA. At the receiver of the receiving STA, the above operations for the 80+80 configuration may be reversed, and the combined data may be transmitted to a medium access control (MAC) layer entity.

[0046] Sub-1 GHz operating modes are supported by 802.11af and 802.11ah. The channel operating bandwidths and carriers are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV White Space (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to representative embodiments, 802.11ah can support metered control / machine-based communication (MTC), such as MTC devices within a macro coverage area. MTC devices may have limited functionality, including support (e.g., support only) of specific bandwidths and / or limited bandwidths. MTC devices may include batteries with above-threshold battery life (e.g., to maintain very long battery life).

[0047] WLAN systems that can support multiple channels and channel bandwidths, such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include a channel that can be designated as a primary channel. The primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be configured and / or limited by the STA from among all STAs operating in the BSS that support the smallest bandwidth operating mode. In the 802.11ah example, the primary channel for a STA (such as an MTC-type device) that supports (e.g., only supports) 1 MHz mode may be 1 MHz wide, even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or network allocation vector (NAV) configuration may depend on the status of the primary channel. For example, if the primary channel is busy because a STA (that only supports 1 MHz mode of operation) is transmitting to the AP, the entire available frequency band may be considered busy, even though most of the frequency band may remain idle and available.

[0048] In the United States, the available frequency band that can be used with 802.11ah is 902MHz to 928MHz. In South Korea, the available frequency band is 917.5MHz to 923.5MHz. In Japan, the available frequency band is 916.5MHz to 927.5MHz. The total available bandwidth for 802.11ah is 6MHz to 26MHz depending on the country code.

[0049] 1D is a system diagram illustrating the RAN 113 and the CN 115 according to an embodiment. As described above, the RAN 113 may employ NR radio technology to communicate with the WTRUs 102a, 102b, and 102c over the air interface 116. The RAN 113 may also communicate with the CN 115.

[0050] The RAN 113 may include gNBs 180a, 180b, and 180c, although it will be understood that the RAN 113 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, and 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, and 102c over the air interface 116. In an embodiment, the gNBs 180a, 180b, and 180c may implement MIMO technology. For example, the gNBs 180a and 180b may utilize beamforming to transmit signals to and / or receive signals from the WTRUs 102a, 102b, and 102c. Thus, the gNB 180a may, for example, transmit wireless signals to and / or receive wireless signals from the WTRU 102a using multiple antennas. In one embodiment, the gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit multiple component carriers to the WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum, and the remaining component carriers may be on licensed spectrum. In one embodiment, the gNBs 180a, 180b, and 180c may implement Coordinated Multi-Point (CoMP) technology. For example, the WTRU 102a may receive coordinated transmissions from the gNBs 180a and 180b (and / or 180c).

[0051] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with scalable numerology. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of different or scalable lengths (e.g., different lengths of absolute time including and / or lasting different numbers of OFDM symbols).

[0052] The gNBs 180a, 180b, 180c can be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c without accessing another RAN (e.g., eNode-Bs 160a, 160b, 160c, etc.). In a standalone configuration, the WTRUs 102a, 102b, 102c may utilize one or more of the gNBs 180a, 180b, 180c as mobility anchor points. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in unlicensed bands. In a non-standalone configuration, the WTRUs 102a, 102b, 102c may communicate with and connect to a gNB 180a, 180b, 180c while also communicating with and connecting to another RAN, such as an eNode-B 160a, 160b, 160c. For example, the WTRUs 102a, 102b, 102c may implement DC principles to communicate with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c substantially simultaneously. In a non-standalone configuration, the eNode-Bs 160a, 160b, 160c may act as mobility anchors for the WTRUs 102a, 102b, 102c, and the gNBs 180a, 180b, 180c may provide additional coverage and / or throughput for serving the WTRUs 102a, 102b, 102c.

[0053] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, support for network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data to User Plane Functions (UPFs) 184a, 184b, routing of control plane information to Access and Mobility Management Functions (AMFs) 182a, 182b, etc. As shown in FIG. 1D, the gNBs 180a, 180b, 180c may communicate with each other via an Xn interface.

[0054] 1D may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and at least one Data Network (DN) 185a, 185b. While each of the foregoing elements is shown as part of the CN 115, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0055] The AMF 182a, 182b may be connected to one or more gNBs 180a, 180b, 180c in the RAN 113 via an N2 interface and may function as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, supporting network slicing (e.g., handling different protocol data unit (PDU) sessions with different requirements), selecting a particular SMF 183a, 183b, managing registration areas, terminating NAS signaling, mobility management, etc. Network slicing may be used by the AMF 182a, 182b to customize the CN support of the WTRUs 102a, 102b, 102c, for example, based on the type of service being utilized by the WTRUs 102a, 102b, 102c. Different network slices may be established for different use cases, for example, services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, services with MTC access, etc. The AMF 162 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) that use other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies like Wi-Fi.

[0056] The SMFs 183a and 183b can be connected to the AMFs 182a and 182b in the CN 115 via an N11 interface. The SMFs 183a and 183b can also be connected to the UPFs 184a and 184b in the CN 115 via an N4 interface. The SMFs 183a and 183b can select and control the UPFs 184a and 184b and configure the routing of traffic through the UPFs 184a and 184b. The SMFs 183a and 183b may perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notification. The PDU session type can be IP-based, non-IP-based, Ethernet-based, etc.

[0057] The UPFs 184a, 184b may be connected to one or more gNBs 180a, 180b, 180c in the RAN 113 via an N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks such as the Internet 110, for example to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPFs 184, 184b may perform other functions such as routing and forwarding packets, enforcing user plane policy, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing a mobility anchor, etc.

[0058] The CN 115 may facilitate communication with other networks. For example, the CN 115 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between the CN 115 and the PSTN 108. Additionally, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to local data networks (DNs) 185a, 185b via UPFs 184a, 184b via an N3 interface to the UPFs 184a, 184b and an N6 interface between the UPFs 184a, 184b and the DNs 185a, 185b.

[0059] 1A-1D and the corresponding description thereof, one or more or all of the functions described herein with respect to any of the WTRUs 102a-d, base stations 114a-b, eNode-Bs 160a-c, MME 162, SGW 164, PGW 166, gNBs 180a-c, AMFs 182a-b, UPFs 184a-b, SMFs 183a-b, DNs 185a-b, and / or any other elements / devices described herein may be performed by one or more emulation elements / devices (not shown). The emulation devices may be one or more devices configured to emulate one or more or all of the functions described herein. For example, the emulation devices may be used to test other devices and / or simulate network and / or WTRU functions.

[0060] The emulation device may be designed to implement one or more tests of other devices in a lab environment and / or an operator network environment. For example, one or more emulation devices may perform one or more or all functions while fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices in the communication network. One or more emulation devices may perform one or more or all functions while temporarily implemented / deployed as part of a wired and / or wireless communication network. The emulation device may be directly coupled to another device for testing purposes and / or may use terrestrial wireless communication to perform the tests.

[0061] One or more emulation devices may perform one or more functions, inclusive, while not being implemented / deployed as part of a wired and / or wireless communication network. For example, the emulation devices may be utilized in test scenarios in a test lab and / or in an undeployed (e.g., test) wired and / or wireless communication network to implement testing of one or more components. One or more emulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas) may be used by the emulation devices to transmit and / or receive data.

[0062] Introduction It will be appreciated that artificial intelligence (AI), and particularly machine learning (ML), can be a very powerful tool in various devices, but the resource (e.g., processing) requirements may be too great for devices with relatively limited resources. Such devices, denoted herein as UEs, may be end-user devices, particularly mobile devices such as smartphones and tablets. For this reason, a common solution is to have one or more other devices perform at least some of the computations. Note that "device" (also called "node") in this context may refer to multiple devices acting as one, for example, in the case of server banks and cloud computing.

[0063] Apart from being performed on a single device, AI / ML inference can therefore be separated (i.e., split) across different points (e.g., from a UE to an edge or cloud device) for a particular AI / ML application, which may be based, for example, on a deep neural network (DNN). An AI / ML application may be split, for example, along the interface between two layers in a DNN, or between different parts, where one part (e.g., a part that detects facial features) can provide results as input to a subsequent part (e.g., using the detected facial features for a person's mood). A corresponding AI / ML model, i.e., a computer program (e.g., trained, i.e., with appropriate parameters), can therefore be split into several AI / ML model subsets running on different devices / servers. Each AI / ML model subset is an independent piece of software running on a split node, such as a UE, an edge device, or the like, in the cloud or in the access network.

[0064] FIG. 2 illustrates different examples of AI / ML model partitioning. An exemplary AI / ML model can be partitioned in different ways. For example, partitioned model M can include AI / ML model subsets {M0, M1}. Similarly, partitioned model M' can include {M'0, M'1} and partitioned model M"{M0", M1", M2"}. These three exemplary partitioned models M, M', and M" provide the same service and, as can be seen from the figure, can be partitioned into different numbers of subsets. The boundaries between different subsets are called split points and are indicated by exemplary arrows in one example. Similarly, another model M can include AI / ML model subsets {N0, N1}, and model N' can include {N'0, N'1, N'2}. Models M and N can have the same DNN layer composition with the same split point, but they can differ, for example, in neuron weight values, bias values, or quantization levels of any input values.

[0065] Furthermore, different subsets may run or be intended to run on different devices. Which device a subset runs on may depend on different conditions. Exemplary conditions include device capabilities, device load, and network load. Thus, a particular subset, e.g., M0, may run either on an edge device or on a UE, for example.

[0066] FIG. 3 shows an example of a split topology in which the UE captures sensing data and acts as an uplink data source.

[0067] Assume that an AI / ML model subset is obtained (e.g., downloaded, streamed) from a device such as a cloud or edge server node. Media data is provided to subset M0 by a UE (e.g., an application capturing video). In this example, assume that M0 processes the media data and outputs intermediate data to subset M1. M1 can process (i.e., infer) the AI ​​data to obtain results, or forward different AI data to M2, which processes different intermediate data to obtain results. The results are possibly returned to the UE via the subset numbered immediately below (e.g., M1 can provide results via M0). The results can be provided to an application or used directly by the UE.

[0068] In example (a), both M0 and M1 are present on the UE. The UE may execute subset M0 while downloading other subset M1, and may obtain a first partial result depending on the model architecture.

[0069] In example (b), M0 resides on a first network node (eg, an edge node) and M1 resides on a second network node (eg, a cloud node).

[0070] In example (c), M0 resides on the UE and M1 resides on the first network node.

[0071] In example (d), M0 resides in the UE and M1 resides in a second network node.

[0072] In example (e), M0 resides on the second network node and M1 resides in the UE.

[0073] In example (f), M0 resides on the UE, M1 resides on a first network node, and M2 resides on a second network node. The network (edge / cloud) sends the final result back to the UE upon completion of successive model subsets.

[0074] The results can be rendered in various ways, for example as a video or as a textual piece of information: it can be a textual indication of the recognized object, an output score, a bounding box, or augmented media data.

[0075] The present principles relate to a split topology with UE endpoints as sources of media streaming data, with results delivered on a media sequence basis from network / edge endpoints, providing, for example, enhanced video (e.g., removing artifacts) or video overlay (e.g., pose detection, pedestrian detection). Considerations to take into account include one or more of the following:

[0076] Partition model dependent on UE constraints and requirements The division of the model may depend on one or more of the UE's processing capabilities, current network conditions, and remote network processing capabilities. These capabilities and conditions typically change over time and may affect application-level requirements, for example, to guarantee an expected end-to-end latency from media capture to obtaining the result, or to guarantee the quality of the result.

[0077] Decision model distribution classification A distributed model that performs inference from a single media stream may be suboptimal if certain content can only be processed after previous content has been processed. As a result, the media stream may be paced at a certain frequency. Meanwhile, the AI / ML model can handle a limited number of data packets per unit time. In this context, the AI / ML model may not properly absorb (i.e., process) the media stream data, which may result in slow processing. Such processing issues may increase if the media rate of the input content is greater than the model subset inference media rate of the current content at the UE. For example, video may stall, or more jitter may be observed in the network.

[0078] Content-aware dependency of model processing Media (i.e., content) data capture in the UE feeds models into processing, which may be split between the UE and the network. Input content may have distinct characteristics. For example, a camera may take a long video shot during which nothing happens, or it may take a short, intense video shot with a lot of movement. As another example, the lighting conditions in a video sequence may change from dark to bright, or vice versa. Depending on the characteristics of the content, the requirements for model execution may vary. This may result in different models and / or subsets, e.g., UE-only, or distributed across the UE and the network.

[0079] Multimedia Data Entry A mobile device UE may capture a video stream and an audio stream, where both streams are inputs for one or more model inferences. As an example, a robot or drone UE may capture several media content items at a time. The models and / or subsets, e.g., UE only, network only, or distributed among UE networks, can depend, for example, on the number and type of content.

[0080] Issued sequentially The partitioned models can be managed by model distribution topologies, but these topologies can change dynamically, so that intermediate data packets may not be sent sequentially to the network.

[0081] overview According to the present principles, the UE can provide (input) content segmentation adapted to model inference, content-aware decisions for model execution selection and delivery, and scalable and self-contained segment delivery adapted to content data or intermediate data.

[0082] FIG. 4 shows a flowchart of a method for UE content processing in a distributed AI / ML environment.

[0083] In step S410, the UE segments the input content adapted to model inference into variable media segments containing inference process support information.

[0084] The UE can partition input content into variable segments that are adapted to be fed to an inference engine that runs a model subset on these segments. A pre-processing unit (e.g., a processor) within the UE can analyze the content (audio, video) and partition the content into segments to accommodate expected inference processing stages.

[0085] The UE segmentation unit detects variations that may affect inference. The UE segmentation may provide segmentation information used for inference. The segmentation information may include one or more of light information, content shot information, number of people / objects, video plan composition / changes, segment dependencies, and plan characteristics.

[0086] The light information may include, for example, information about the brightness of the image. The applied model (biases, weights) may differ based on the light characteristics of the segment.

[0087] The content shot information can indicate long shots with little action, or short, intense video shots. The model can easily calculate long duration segments where not much is happening, but it may be more difficult to calculate shorter video segments with a lot of activity.

[0088] The number of people / objects in the plan may indicate the number of people and / or objects in the shot. The number of computing units may vary accordingly.

[0089] Video plan composition / modification information: Model execution can be more efficient and quality can be higher if the entire video plan is processed.

[0090] Segment dependency information. A segment, and therefore processing, may be highly dependent on a previous (e.g., immediately preceding) segment. For example, there may be cases where the segmentation does not correspond to a completely independent frame boundary. In such cases, the current segment may preferably be processed by the same inference engine running the same model subset as the segments on which it depends.

[0091] The plan characteristic information may indicate, for example, orientation, resolution, and frame.

[0092] Based on the information, the content-aware segmentation unit can partition the input content into segments of different duration and / or size.

[0093] In one embodiment, the segmentation unit only considers content-related considerations as described above. The segmentation unit may provide the segments and corresponding segment metadata to a local decision module that is responsible for deciding how to configure a model to process the segment. The segment metadata may include information for model selection and estimated processing resources for processing the segment.

[0094] In one embodiment, an application can configure media segmentation characteristics, ranges, or limits, for example, in terms of minimum, average, and maximum media segment duration, bit rate, or segment size.

[0095] In one embodiment, the segmentation unit may include a selection (see below), for which monitoring information about the current UE or network endpoint and network conditions may also be taken into account for the segmentation itself.

[0096] In one embodiment, the UE can deliver inference processing assistance information to an inference unit located on a network endpoint, which may be the case, for example, when a media segment is processed first on the network side.

[0097] In step S420, the UE selects a model, a model subset, and an inference unit adapted to the input media segment.

[0098] The UE may select a model, a model subset, and an inference unit for each model subset to process the input media segment. For each given input content segment, the UE performs one or more actions.

[0099] The UE may calculate input information, which may include, for example, one or more of segmentation unit information, UE or network capabilities and current network conditions, inference unit list and capabilities, application level requirements, model characteristic information, and user or task specific requirements, which are further described.

[0100] The segmentation unit information has already been explained.

[0101] UE or network capabilities and current network conditions. For example, depending on initial configuration settings, this may include the processing power required to compute certain types of segments depending on their size and complexity. The information may also include an indication of current network congestion information.

[0102] List and Capabilities of Inference Units. The UE and network can provide lists of inference units with different processing capabilities. As an example, an inference unit (UE or network) may be adapted to process a particular subset of a given model, e.g., low processing power, while another unit is further adapted to process other model subsets. As another example, the UE may select a local inference unit for the incoming current segment instead of the network segment when the network is congested.

[0103] Application-level requirements. For example, a user or application can provide information about the processing power allocated to AI / ML processing for all inference units in the UE. The information can change in response to events. For example, the start of another application can result in a reduction in processing power available for AI / ML processes, and the UE can, for example, select a network inference unit instead of a local inference unit to meet the requirements.

[0104] The model characteristic information indicates whether the model requires split inference with a task-specific model executed on the UE or on the network as a first subset or a final subset.

[0105] User or task specific requirements: for example, it may be necessary to perform a processing task on the end device to protect privacy or because the task is latency sensitive.

[0106] The UE may select a model to apply from a list of models, where different models may be generally identical with different internal compositions (e.g., neurons, weights, biases, quantization).

[0107] The UE may select the topology to be used for the segment, which may be UE inference, network inference, or segment inference unit. In the latter case, the decision module selects the UE subset and network subset distribution to apply.

[0108] The UE may encapsulate the information as input to a selected inference unit, which may be one of several UE inference units or one or several network inference units. The data payload may be media data when the entire model is processed in the network, or intermediate data distribution information when a subset is processed by the device.

[0109] The UE may mediate the delivery of segments from the segmentation unit to selected inference units at the UE or in the network.

[0110] In step S430, the UE provides a scalable self-contained content segment package.

[0111] The UE may encapsulate information for any inference unit running in any network endpoint to execute the remaining network model subset of a given content segment. This allows one inference unit to start processing a new segment instead of waiting for completion of a previous segment in another inference unit, including the UE inference unit. The network endpoint receives and calculates information from any UE on how to process incoming inference data or intermediate data. The information may include one or more of the following elements, which will be described further: model identifier, originating content identifier, full model, model split point, inference unit information, content segment number, UE identifier, content segment information, segment data payload type, segment data payload compression profile, compression information, and segmentation information.

[0112] The model identifier indicates the trained model used for a given content segment.

[0113] The outgoing content identifier identifies the incoming outgoing content flow. This may be needed when multiple UE or network endpoints perform model inference subsets from content of the same origin. This may be needed to identify a content flow when a UE captures multiple content flows.

[0114] The full model indicates that the UE or the network processes the full model.

[0115] A model split point indicates one or more points at which an AI / ML training model separates into two or more subsets, each containing a different set of layers. A UE selects a model subset to run on the UE or within the network based on the model split point indication.

[0116] The inference unit information identifies the inference unit, and in one embodiment the package may be sent / received / forwarded by a dedicated package delivery / access that is responsible for routing the segment package according to this information and further information in the package.

[0117] The content segment number is a counter used to keep track of the processed segments. This number can be used to sort the processed segments in the network or the UE.

[0118] UE Identifier: The network may request a UE identifier to reassemble content segments output from network endpoint inferences for delivery to the identified UE.

[0119] The segment data payload type indicates whether the segment data is media content or intermediate data resulting from processing a subset of an AI / ML model.

[0120] Segment data payload compression profile. If the delivery function applies a compression technique before delivering the segment payload, this information may indicate the corresponding compression information, which includes the information necessary to encode / decode the data payload of the segment.

[0121] Content segmentation information has already been discussed. This information can be particularly useful when the UE decides to completely offload the segmentation process to the network.

[0122] The data payload information can be transmitted in different ways. For example, the information elements may already be known at the configuration stage. These may include the compression technique used between the UE and the network. The bitstream may contain the information elements in a specific header or in a separate timed information channel.

[0123] In one embodiment, the UE may package intermediate data for an internal UE inference unit, for example, if the inference unit is an independent process, VM, or hardware resource. Additionally, when the UE processes a first model subset, the UE may update the received information in the input segment package to generate a new, modified package that includes information needed to process the next subset to be sent to the next inference unit.

[0124] In one embodiment, the scalable, self-contained content segment package encapsulation includes result data information, such as a text indication of a recognized object, an output score, a bounding box, or extended media data information. If the result data information is media data, the segment data payload includes the processed media data, and the encapsulation includes result metadata useful for processing the media data.

[0125] In step S440, the UE delivers the self-contained content segment package to an inference unit within the UE or to a network. In one embodiment, the UE provides intermediate data to a local inference unit. The inference unit within the network may be located in the network or another remote endpoint, such as in another UE.

[0126] The UE and network provide a scalable inference unit capable of processing self-contained content segments. An inference unit may be an independent process that performs inference on a portion of a model for a given input segment. It may be an internal hardware device, such as an allocated TPU, GPU, CPU, or software process or virtual machine instance running on the UE or on the network. Such a hardware device may run multiple inference units.

[0127] The inference unit receives a self-contained content segment including relevant information of the model subset to be applied, performs inference of the model subset on the segment, and updates the self-contained segment information. For example, if the UE has just processed a subset, the UE may update information for applying the next subset.

[0128] Then, the inference unit sends the processed segment to the next inference unit according to the updated self-contained segment information.

[0129] In one embodiment, the inference unit receives or transmits the content directly.

[0130] In one embodiment, the inference unit receives or transmits content segments from a distribution or access function dedicated to brokering segments between inference units located in the UE or in the network. The access / distribution function can be considered as a network forwarder.

[0131] In another embodiment, for example, use case e) shown in Figure 3, when the UE receives intermediate data from the network (edge / cloud), the inference units on the network side send self-contained segments containing the intermediate data, so that they can send the segments to a distribution function in the network for encapsulation in an updated self-contained content package and transmission to the UE.

[0132] 3, use case b), when one or more model subsets are processed between nodes on the network side, e.g., between one edge node and one cloud node, the edge or cloud node may provide access / distribution functionality to send self-contained segments with intermediate data to other nodes. For example, in use case b), the edge node that processed segments from the first subset sends self-contained content segments with intermediate data to the cloud node for processing the next subset.

[0133] In step S450, the UE reorders and concatenates, as necessary, the processed media segments received from the UE or from different inference units belonging to one or more network endpoints.

[0134] In one embodiment, the UE receives, reorders, and concatenates the processed segments. The UE can use package information, such as segment numbers, to reassemble and reorder the processed segments from different network endpoints or possibly from different inference units provided by the UE's local inference unit.

[0135] In one embodiment, the network distribution function may reorder all or part of the processed segments within the network.

[0136] In one embodiment where the results are not media, partial results for different segments are rearranged within the UE.

[0137] 5 shows an exemplary system 500 in accordance with an embodiment of the present principles. The exemplary system includes a UE endpoint 510 and a network endpoint 520, although it will be understood that more endpoints of any type may be involved. In the exemplary system, modules are functional units that may be implemented in one or more processors.

[0138] UE endpoint 510 includes a content-aware pre-processing module 511, an application-level request module 512, a UE and network monitoring module 513, a local decision module 514, one or more UE partition inference modules 515, an intermediate data delivery module 516, and a result access module 517. Network endpoint 520 includes an intermediate data access and remote decision module 521, a network partition inference module 522, and a result delivery module 523.

[0139] The content-aware preprocessing module 511 is configured to preprocess the received content 530 to provide segments of media data and information on how to process the media segments. The information may include model information (which model to use according to the current environment), segment duration, and segment bit rate for a specific duration. The content-aware preprocessing module may receive information for input configuration from the local decision module 514, which requests segment characteristics.

[0140] The content-aware preprocessing module 511 can provide a flow of data segments, each of which can be viewed as a chunk of media data, media segment characteristics (e.g., duration, bit rate, data size), media-specific information and composition (e.g., video coding standard, full video segment or video slice), segmentation information (as already described), and segment dependency information (e.g., which other segments the current segment depends on).

[0141] The UE and network monitoring module 513 monitors UE capabilities and current network conditions and provides information about the processing power required to compute specific content and information about network congestion depending on initial configuration settings.

[0142] The application level request module 512 provides information about the processing power allocation to the AI / ML application. For example, when the application level starts or stops another process, the application level request module 512 can update the remaining processing power.

[0143] The UE Partition Inference Module 515 is an independent process that performs inference on a portion of the model for a given input segment. As mentioned above, the module may be an internal hardware module, such as an allocated TPU, GPU, CPU, or a software process, or an internal virtual machine instance running on the hardware. The module, which can be said to be stateless, receives content segments and information about which portion of the model to use for the input segment. The content segments and information can come from the local decision module 514. Once all or part of the model is complete, the module 515 sends the processed segments (i.e., output) to the intermediate data delivery module 516.

[0144] The intermediate data distribution module 516 encapsulates the information for each given independent segment into a data structure for one or more network endpoints to apply the remainder of the AI / ML model. When the entire model is processed by the UE 510, the data distribution function sends the processed segments to the result access function 517.

[0145] The intermediate data access and remote determination module 521 receives the output of the intermediate data delivery module 516 of the UE 510 and determines one or more network partition inference modules 522 for delivery of the processed segments (i.e., the received output).

[0146] The network partition inference module 522 processes the segments received from the intermediate data access and remote decision module 521 to obtain one or more results that are sent to the results delivery module 523 .

[0147] The results delivery module 523 collects the individual segment results, which may be sorted, before delivering them to the results access module 517 of the UE 510 .

[0148] The results access module 517 receives the segment results, sorts them if it has not already done so, and delivers the segment to the application. Note that the results access module 517 can receive the segment results from one or both of the intermediate data delivery module 516 and the results delivery module 523.

[0149] The local decision module 514 can compute input information from the content recognition preprocessing module 511, the application level request module 512, and the UE and network monitoring module 513, select a UE partition model configuration for each content segment, encapsulate the information for input to one or more UE partition reference modules 515 or one or more network partition inference modules 522, and mediate delivery of the segment to the UE partition reference module 515 or the intermediate data delivery module 516.

[0150] Figure 6 shows an example of inference according to one embodiment of the present principles. Video content is input to a content-aware pre-processing module, which outputs three segments: segments 1, 2, and 3. Based on information from the application-level request module and the UE and network monitoring module, a local decision function assigns segment 1 to inference module B, which indicates that it will process the model up to segmentation point B. Similarly, it assigns segment 2 to inference module C, which will process it up to segmentation point C. Then, for a given reason (e.g., required processing power), segment 3 is forwarded to a distribution encapsulation module for transmission to the network, which applies the entire model or a variant of the model to it.

[0151] The intermediate results from inference modules B and C are delivered to a distribution encapsulation module, which encapsulates the segment data in a metadata header that may include a timestamp. Additionally, each intermediate result and unprocessed segment may have a header (or other relevant information). The header information includes information for generating a stateless segment to process and may include information received from the inference module or the local decision function module, such as a UE ID, a content ID, a model ID, a segmentation point ID, and a segment number identifier. The distribution encapsulation module transmits the encapsulated data to the network side.

[0152] The encapsulated data includes segment data and metadata and is self-contained as it is processed in a stateless manner by the network inference module.

[0153] In the network, an intermediate data access and remote decision module receives the encapsulated data, decapsulates the header, and forwards the processing information to an associated network inference module to apply the remaining model portions starting from the received split point indication.

[0154] In this example, the output from UE inference module B is sent to network inference module B', the output from UE inference module C is sent to network inference module C', and segment 3 is sent to network inference module D.

[0155] Once completed, the network may rearrange the results before delivery to the UE (not shown).

[0156] The UE and the network endpoint may communicate via the control plane to agree on the configuration to use, e.g., one or more models to partition, one or more partitioned topologies to use, and for each model, in case of split inference, a list of split points available on the UE side, the identity of the UE or network endpoint sending or receiving data, and the identity of the content.

[0157] FIG. 7, which is comprised of FIGS. 7A-7C, illustrates one embodiment of the data flow for the example shown in FIG.

[0158] In step S702, the content pre-processing module receives media content, which may be captured by the UE.

[0159] In step S704, the content pre-processing module analyzes the received media content with respect to application, UE, and network constraints and segments the media content into segments, for example, depending on the application configuration. Segmentation points may depend on the media content itself, for example, when a camera shot changes or when an object is detected in an image. The content pre-processing module may enforce application constraints when segmenting segments based on size. The content pre-processing module may provide information about the content, such as light / dark, moving images, etc., to the local determination module.

[0160] In step S706, the content pre-processing module sends the first content segment Seg 1 and the segment information to the local decision module.

[0161] In step S708, the local decision module calculates the input information for Seg 1 as previously described and determines which model to apply and where to split the model for Seg 1. In this example, it is decided to assign the local inference module Inf B to perform inference on the subset of model M up to split point B that is applied to Seg 1.

[0162] In step S710, the local decision module transfers the segment data to Inf_B, which includes segment information and model information for inference.

[0163] In step S712, Inf B processes the AI / ML subset applying the input information received from the local decision module.

[0164] In step S714, the local decision module receives a second segment, Seg 2, from the content pre-processing module, possibly while still processing Seg 1.

[0165] In step S716, similar to step S708, the local decision module calculates the input information for Seg 2 and assigns it to a second local inference module Inf C to determine that inference will be performed on a subset of model M up to the split point C applied to Seg 2.

[0166] In step S718, similar to step S710, the segment data of Seg 2 and related information regarding how to process it are transferred to Inf C.

[0167] In step S720, independently of the processing of the previous segments Seg 1 and Seg 2, the local decision module receives a third segment Seg 3 from the content pre-processing module.

[0168] In step S722, Inf B processes the AI / ML subset applying the input information received from the local decision module. Generally, the processing of an inferer is independent of other inferers. For example, Inf B can buffer an incoming segment while waiting for the previous segment to finish.

[0169] In step S724, the local decision function calculates Seg 3 information and decides to forward Seg 3 to the network and apply the entire model or a variant of the model to it. Such a decision may be motivated by various conditions, such as: i) the local inference module is not available (in this example, Inf B and Inf C have not completed their inference), ii) edited information from Seg 3 indicates that processing may exceed the local remaining UE processing power to guarantee latency requirements, and iii) edited information indicates that using the partitioned model is inefficient.

[0170] In step S726, the local decision module sends Seg 3 to the intermediate data delivery module along with information for wrapping and encapsulating information (eg, segment information, model information) with the segment payload to the network side.

[0171] In step S728, the intermediate data delivery module encapsulates the Seg 3 segment data with additional information as previously described.

[0172] In step S730, the intermediate data delivery module transmits the encapsulated Seg 3 data to the intermediate access and remote decision module in the network.

[0173] In step S732, the intermediate access and remote decision module decapsulates the packet and calculates the input information to obtain model and segmentation point information for assigning the network inference module Inf D to process the entire model upon receiving Seg 3.

[0174] In step S734, the intermediate access remote determination module sends Seg 3 and additional information to Inf D.

[0175] In step S736, the network inference module Inf D processes the AI / ML subset applying input information received from the remote decision module to process the model M of Seg 3.

[0176] In step S738, Inf B, which has completed the processing of Seg 1, transmits information for wrapping and encapsulating useful information (segment information, model information) together with the segment intermediate data payload to the intermediate data delivery module.

[0177] In step S740, the intermediate data delivery module encapsulates the intermediate data associated with the segment Seg 1 and additional information including the type of the data (intermediate data).

[0178] In step S742, the intermediate data delivery module transmits the encapsulated Seg 1 to the network side.

[0179] In step S744, similar to step S728, the intermediate access and remote decision module assigns a network inference module Inf B' to handle the remaining work starting from division point B of model M on Seg 1.

[0180] In step S746, the intermediate access and remote decision module sends Seg 1 and additional information to Inf_B'.

[0181] In step S748, having finished executing the AI / ML model for Seg 3, Inf D sends the segment results to a result delivery module that is responsible for reassembling and sorting the processed segments and delivering the results to the UE.

[0182] In step S750, Inf C sends the Seg 2 intermediate data to the intermediate data distribution module.

[0183] In step S752, the intermediate data delivery module encapsulates Seg 2 and transmits it to the network side.

[0184] In step S754, the intermediate data delivery module transmits the encapsulated Seg 2 to the network side.

[0185] In step S756, Inf_B' processes the AI / ML subset applying the input information received from the remote decision module, starting the processing of the AI / ML model from segmentation point B of Seg 1.

[0186] In step S758, Inf B' sends the Seg 1 payload and information to the result delivery module.

[0187] In step S760, similar to step S740, the remote decision module assigns a network inference module Inf C' to Seg 2 to handle the remaining work starting from division point C of model M.

[0188] In step S762, the remote decision module sends Seg 2 and additional information to Inf_C'.

[0189] In step S764, Inf_C' processes the AI / ML subset from segmentation point C of Seg 2.

[0190] In step S766, Inf C' sends the Seg 2 payload and information to the result delivery module.

[0191] In steps S768, S770, S772, the result distribution module sends the results corresponding to the processed segments Seg 1, Seg 2, Seg 3 to the result access module in the UE, respectively.

[0192] conclusion Although features and elements are provided above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. The present disclosure is not limited in terms of the specific embodiments described herein; these embodiments are intended as illustrations of various aspects. It will be apparent to those skilled in the art that many modifications and variations may be made without departing from the spirit and scope of the present invention. No element, act, or instruction used in the description of the present application should be construed as critical or essential to the invention unless explicitly provided as such. Functionally equivalent methods and apparatuses within the scope of the present disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing description. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure should be limited only by the terms of the appended claims, and the full scope of equivalents to which such claims are entitled. It is understood that this disclosure is not limited to any particular method or system.

[0193] The foregoing embodiments are discussed with respect to the terminology and structure of infrared-enabled devices (i.e., infrared emitters and receivers) for simplicity, however, the discussed embodiments are not limited to these systems and may also be applied to other systems that use other forms of electromagnetic waves, or non-electromagnetic waves such as acoustic waves.

[0194] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the term “video” or “image” can mean either a snapshot, a single image, and / or multiple images displayed over time. As another example, when referred to herein, the term “user equipment” and its abbreviation “UE,” “remote,” and / or the term “head-mounted display” and its abbreviation “HMD” can mean or include (i) a wireless transmit and / or receive unit (WTRU), (ii) any of several embodiments of a WTRU, (iii) a wireless-enabled and / or wired-enabled (e.g., tetherable) device specifically configured to have some or all of the structure and functionality of a WTRU, (iii) a wireless-enabled and / or wired-enabled device configured to have less than all of the structure and functionality of a WTRU, or (iv) the like. Details of an exemplary WTRU, which may represent any WTRU listed herein, are provided herein with respect to FIGS. 1A-1D . As another example, various embodiments disclosed herein above and below are described as utilizing a head-mounted display. Those skilled in the art will recognize that devices other than head-mounted displays may be utilized and that some or all of the present disclosure and various disclosed embodiments may be modified accordingly without undue experimentation. Examples of such other devices may include drones or other devices configured to stream information to provide an adaptive reality experience.

[0195] Additionally, the methods provided herein may be implemented in a computer program, software, or firmware embodied in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

[0196] Modifications to the methods, apparatus, and systems provided above are possible without departing from the scope of the present invention. In view of the wide variety of possible embodiments, it should be understood that the illustrated embodiments are merely examples and should not be construed as limiting the scope of the following claims. For example, the embodiments provided herein include portable devices, which may include or be utilized with any suitable voltage source, such as a battery providing any suitable voltage.

[0197] Furthermore, in the above embodiments, it should be noted that processing platforms, computing systems, controllers, and other devices include processors. These devices may include at least one central processing unit (CPU) and memory. In accordance with the practices of those skilled in the art of computer programming, references to acts and symbolic representations of operations or instructions may be performed by various CPUs and memories. Such acts and operations or instructions may be referred to as being "executed," "executed by a computer," or "executed by a CPU."

[0198] Those skilled in the art will understand that the acts and symbolically represented operations or instructions include the manipulation of electrical signals by a CPU. The electrical system represents the data bits, which may cause the transformation or reduction of the resulting electrical signals and the retention of the data bits in memory locations within a memory system, thereby reconfiguring or altering the operation of the CPU and other processing of the signals. The memory locations where the data bits are maintained are physical locations with specific electrical, magnetic, optical, or organic properties that correspond to or represent the data bits. It should be understood that embodiments are not limited to the platforms or CPUs mentioned above, and that other platforms and CPUs may support the provided methods.

[0199] The data bits may also be maintained on a computer-readable medium, including magnetic disks, optical disks, and any other volatile (e.g., random access memory (RAM)) or non-volatile (e.g., read-only memory (ROM)) mass storage system readable by a CPU. The computer-readable medium may include cooperative or interconnected computer-readable media that reside exclusively on a processing system or that are distributed among multiple interconnected processing systems, which may be local or remote to a processing system. It should be understood that embodiments are not limited to the memories mentioned above, and that other platforms and memories may support the provided methods.

[0200] In an example embodiment, any of the operations, processes, etc. described herein may be implemented as computer-readable instructions stored on a computer-readable medium. The computer-readable instructions may be executed by a processor of a mobile unit, a network element, and / or any other computing device.

[0201] There is little difference between hardware and software implementations of aspects of the system. Using hardware or software is generally a design choice that represents a cost vs. efficiency tradeoff (although the choice between hardware and software can be significant in some circumstances). There may be a variety of vehicles (e.g., hardware, software, and / or firmware) in which the processes and / or systems and / or other technologies described herein may be affected, and the preferred vehicle may vary depending on the context in which the processes and / or systems and / or other technologies are deployed. For example, if an implementer determines that speed and accuracy are paramount, the implementer may select a primarily hardware and / or firmware vehicle. If flexibility is paramount, the implementer may select a primarily software implementation. Alternatively, the implementer may select some combination of hardware, software, and / or firmware.

[0202] The foregoing detailed description has illustrated various embodiments of devices and / or processes through the use of block diagrams, flowcharts, and / or examples. To the extent that such block diagrams, flowcharts, and / or examples include one or more functions and / or operations, it will be understood by those skilled in the art that each function and / or operation within such block diagrams, flowcharts, or examples can be individually and / or collectively implemented by a wide range of hardware, software, firmware, or substantially any combination thereof. In one embodiment, some portions of the subject matter described herein can be implemented via an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), and / or other integrated form. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein may equivalently be implemented, in whole or in part, as one or more computer programs running on one or more computers (e.g., one or more programs running on one or more computer systems), one or more programs running on one or more processors (e.g., one or more programs running on one or more microcomputers), firmware, or virtually any combination thereof, and that designing circuitry and / or writing code for software and / or firmware is within the skill of those skilled in the art in light of this disclosure. Additionally, those skilled in the art will understand that the mechanisms of the subject matter described herein may be distributed as a program product in various forms, and that exemplary embodiments of the subject matter described herein apply regardless of the particular type of signal-bearing medium used to actually accomplish the distribution. Examples of signal-bearing media include, but are not limited to, recordable-type media such as floppy disks, hard disk drives, CDs, DVDs, digital tape, computer memory, etc., and transmission-type media such as digital and / or analog communications media (e.g., fiber optic cables, wave guides, wired communications links, wireless communications links, etc.).

[0203] Those skilled in the art will recognize that it is common in the art to describe devices and / or processes in the manner described herein and then use engineering techniques to integrate such described devices and / or processes into a data processing system. That is, at least a portion of the devices and / or processes described herein can be integrated into a data processing system through a reasonable amount of experimentation. Those skilled in the art will recognize that a typical data processing system may generally include one or more of the following: a system unit housing; a video display device; memory, such as volatile and non-volatile memory; a processor, such as a microprocessor and a digital signal processor; computing entities, such as an operating system, drivers, a graphical user interface, and application programs; one or more interactive devices, such as a touchpad or screen; and / or a control system, including feedback loops and control motors (e.g., feedback for sensing position and / or velocity, control motors for moving and / or adjusting components and / or quantities). A typical data processing system may be implemented utilizing any suitable commercially available components, such as those typically found in data computing / communication systems and / or network computing / communication systems.

[0204] The subject matter described herein may illustrate different components that are contained within or connected to different other components. It should be understood that such depicted architectures are merely examples, and that in fact many other architectures that achieve the same functionality may be implemented. In a conceptual sense, an arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Thus, any two components herein that are combined to achieve a particular function may be considered to be "associated" with each other such that the desired functionality is achieved, regardless of the architecture or intervening components. Similarly, any two components so associated may also be considered to be "operably connected" or "operably coupled" to each other to achieve the desired functionality, and any two components that may be so associated may also be considered to be "operably coupleable" to each other to achieve the intended functionality. Specific examples of operably coupleable include, but are not limited to, physically coupleable and / or physically interacting components and / or wirelessly interacting and / or wirelessly interacting components and / or logically interacting and / or logically interacting components.

[0205] With respect to the use of virtually any plural and / or singular term herein, those skilled in the art will be able to convert from plural to singular and / or from singular to plural as appropriate to the context and / or application. Various singular / plural permutations may be expressly set forth herein for clarity.

[0206] In general, those skilled in the art will understand that the terms used in this specification, and particularly in the appended claims (e.g., the body of the appended claims), are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” and the term “includes” should be interpreted as “includes but is not limited to”). Where a specific number of introduced claim recitations is intended, such intention will be explicitly set forth in the claim; in the absence of such recitation, it will be further understood by those skilled in the art that no such intention exists. For example, where only one item is intended, the term “single” or similar language may be used. To assist in understanding, the following appended claims and / or description of this specification may include the use of the introductory phrases “at least one” and “one or more” to introduce claim recitations. However, the use of such phrases should not be construed as meaning that the introduction of a claim recitation with the indefinite article "a" or "an" limits any particular claim that includes such an introduced claim recitation to embodiments that include only that one recitation, even if the same claim also includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" should be interpreted to mean "at least one" or "one or more"). The same applies to the use of definite articles used to introduce claim recitations. Additionally, those skilled in the art will recognize that even when a specific number of recitations in an introduced claim are explicitly recited, such recitation should be interpreted to mean at least the recited number (e.g., the simple recitation "two recitations" without any other modifier means at least two recitations, or more than two recitations).Furthermore, when a rule similar to "at least one of A, B, and C, etc." is used, such a structure is generally intended in the sense that one of ordinary skill in the art would understand the rule (e.g., "a system having at least one of A, B, and C" includes, but is not limited to, systems having A only, B only, C only, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). When a rule similar to "at least one of A, B, or C, etc." is used, such a structure is generally intended in the sense that one of ordinary skill in the art would understand the rule (e.g., "a system having at least one of A, B, or C" includes, but is not limited to, systems having A only, B only, C only, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). Those skilled in the art will further understand that any disjunctive word and / or phrase, whether in the specification, claims, or drawings, substantially presenting two or more alternative terms, should be understood to contemplate the possibility of including either term, either other term, or both terms. For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B." Furthermore, as used herein, the term "any" followed by a list of multiple items and / or multiple categories of items is intended to include "any," "any combination," "any plurality," and / or "any combination of multiple items and / or categories of items, individually or in combination with other items and / or items from other categories." Furthermore, as used herein, the term "set" is intended to include any number of items, including zero. Furthermore, as used herein, the term "number" is intended to include any number, including zero. Also, as used herein, the term "multiple" is intended to be synonymous with "a plurality."

[0207] Additionally, where features or aspects of the disclosure are described in terms of a Markush group, those skilled in the art will recognize that the disclosure is also thereby described in terms of any individual member or subgroup of members of the Markush group.

[0208] As will be understood by those skilled in the art, for all purposes, including in terms of providing a written description, all ranges disclosed herein also encompass any possible subranges and combinations of subranges. Any recited range can be readily recognized as being sufficiently descriptive to allow the same range to be broken down into at least equal halves, thirds, quarters, fifths, tenths, etc. As a non-limiting example, each range discussed herein can be readily broken down into a lower third, middle third, upper third, etc. As will also be understood by those skilled in the art, all terms such as "up to," "at least," "greater than," "less than," etc., refer to ranges that are inclusive of the recited number and that can be further broken down into subranges as discussed above. Finally, as will be understood by those skilled in the art, a range includes each individual element. Thus, for example, a group having 1 to 3 cells refers to a group having 1, 2, or 3 cells. Similarly, a group having 1 to 5 cells refers to groups having 1, 2, 3, 4, or 5 cells, and so forth.

[0209] Furthermore, the claims should not be construed as limited to the order or elements provided unless expressly stated to that effect. Additionally, the use of the term "means for" in any claim is intended to invoke 35 U.S.C. 112, paragraph 6 or means-plus-function claim form, and a claim without the term "means for" is not so intended.

Claims

1. In the first device, Segmenting a unit of media data corresponding to a media content item into a plurality of media segments; For each media segment: determining whether a machine learning model, a machine learning unit, and inference using the machine learning unit are to be performed externally to the first device or in an inference unit of the first device; Transmitting content data corresponding to the media segment and information indicating the determined machine learning model for inference and machine learning unit; and obtaining processed data obtained by processing the content data using at least the machine learning unit.

2. said segmenting resulting in information associated with each media segment; The method of claim 1.

3. wherein said determining is based on at least one of information associated with each media segment, capabilities of the first device, capabilities of a network to which the first device is connected, current network conditions, capabilities of a reasoning unit, requirements of an application for which the media content item is intended, model characteristic information, user-specific requirements, and task-specific requirements. The method of claim 1.

4. The machine learning unit is a subset of the machine learning model. The method of claim 1.

5. concatenating result data corresponding to the output of the machine learning model processing the content data corresponding to the media segments; The method of claim 1 further comprising:

6. ordering the resulting data corresponding to the media segments prior to said concatenation; The method of claim 5 further comprising:

7. If the processed data includes intermediate data received from a first inference unit, The method of claim 1 , further comprising transmitting the intermediate data and information indicating the determined machine learning model, the information indicating the machine learning section to be used for inference in the second inference unit.

8. the first inference unit from which the processed data is received is internal to the first device, and the second inference unit is external to the first device; The method of claim 7.

9. the first inference unit from which the processed data is received is external to the first device, and the second inference unit is internal to the first device; The method of claim 7.

10. When the first device externally executes inference using the machine learning unit, the first device further transmits an identifier of the first device associated with the transmitted content data. The method of claim 1.

11. If the inference using the machine learning unit is performed by an internal inference unit, the sending includes providing the content data corresponding to the media segment and information indicating the determined machine learning model for inference and the machine learning unit to the internal inference unit. The method of claim 1.

12. A first device comprising at least one hardware processor, the at least one hardware processor comprising: Segmenting a unit of media data corresponding to a media content item into a plurality of media segments; For each media segment: determining whether a machine learning model, a machine learning unit, and inference using the machine learning unit are to be performed externally to the first device or in an inference unit of the first device; Transmitting content data corresponding to the media segment and information indicating the determined machine learning model for inference and machine learning unit; A first device configured to obtain processed data obtained by processing the content data using at least the machine learning unit.

13. Segmenting the units of media data results in information associated with each media segment; The first device of claim 12.

14. the at least one hardware processor is configured to determine whether a machine learning model, a machine learning unit, and inference using the machine learning unit should be performed external to the first device or in an inference unit of the first device based on at least one of information associated with each media segment, capabilities of the first device, capabilities of a network to which the first device is connected, current network conditions, capabilities of an inference unit, requirements of an application for which the media content item is intended, model characteristic information, user-specific requirements, and task-specific requirements. The first device of claim 12.

15. The machine learning unit is a subset of the machine learning model. The first device of claim 12.

16. the at least one hardware processor:

13. The first device of claim 12, further configured to concatenate result data corresponding to output of the machine learning model processing the content data corresponding to the media segments.

17. the at least one hardware processor: The first device of claim 16 , further configured to order the resulting data corresponding to the media segments prior to the concatenation.

18. When the processing data includes intermediate data received from a first inference unit, the at least one hardware processor The first device of claim 12, further configured to transmit the intermediate data and information indicating the determined machine learning model, the information indicating a machine learning section to be used for inference in a second inference unit.

19. the first inference unit from which the processed data is received is internal to the first device, and the second inference unit is external to the first device; 20. The first device of claim 18.

20. the first inference unit from which the processed data is received is external to the first device, and the second inference unit is internal to the first device; 20. The first device of claim 18.

21. the at least one hardware processor: The first device of claim 12, further configured to transmit an identifier of the first device associated with the transmitted content data when inference using the machine learning unit is performed externally.

22. the at least one hardware processor:

13. The first device of claim 12, further configured to, if inference using the machine learning unit is performed by an internal inference unit, provide the content data corresponding to the media segment and information indicating the determined machine learning model for inference and the machine learning unit to the internal inference unit.

Images