Dynamic feature size adaptation in partitionable deep neural networks

Abstract

The proposed approach deals with efficient transmission for distributed AI with provisions for switching between multiple bandwidths. During distributed inference in edge devices, each device needs to load a part of the AI ​​model only once, but the input / output features communicated between them can be flexibly configured according to the available transmission bandwidth by enabling / disabling connections between nodes in the Dynamic Feature Size Switch (DySw). Other parameters of the DNN remain the same when some nodes are connected or disconnected to achieve the desired compression factor. That is, the same DNN model is used for different compression factors, and there is no need to download a new DNN model to fit the compression factor or network bandwidth.

Description

[Technical field]

[0001] The present embodiments generally relate to dynamic feature size adaptation in separable Deep Neural Networks (DNNs). [Background technology]

[0002] Artificial intelligence is a key functional block in many technology areas today. This is due to the resurgence of neural networks in the form of deep neural networks (DNNs). Modern DNNs are often computationally intensive, and therefore it is difficult to perform DNN operations on mobile phones or other edge devices that have low processing power. This is often addressed by transferring data from the mobile device to a cloud server, where all computations are performed. Summary of the Invention

[0003] According to one embodiment, a device is presented, the device comprising a wireless transmit / receive unit (WTRU), the wireless transmit / receive unit (WTRU) comprising: a receiver configured to receive a portion of a deep neural network (DNN) model, the portion of the DNN model being before a division point of the DNN model and including a neural network for compressing features at the division point of the DNN model; one or more processors configured to obtain a compression factor of the neural network, determine which nodes in the neural network should be connected in response to the compression factor, and perform inference using the portion of the DNN model to configure the neural network and generate compressed features in response to the determination; and a transmitter configured to transmit the compressed features to another WTRU.

[0004] According to another embodiment, a device is presented, the device comprising: a wireless transmit / receive unit (WTRU), the wireless transmit / receive unit (WTRU) comprising: a receiver configured to receive a portion of a deep neural network (DNN) model, the portion of the DNN model being after a division point of the DNN model and including a neural network for expanding features at the division point of the DNN model, the receiver also configured to receive one or more features output from another WTRU; and one or more processors configured to obtain a compression factor of the neural network, determine which nodes in the neural network should be connected in response to the compression factor, configure the neural network in response to the determination, and perform inference using the portion of the DNN model using the one or more features output from another WTRU as inputs to the neural network.

[0005] According to another embodiment, a method is presented, including a method performed by a wireless transmit / receive unit (WTRU), the method including receiving a deep neural network (DNN) model, the portion of the DNN model being before a division point of the DNN model and including a neural network for compressing features at the division point of the DNN model; obtaining a compression factor for the neural network; determining which nodes in the neural network should be connected in response to the compression factor; configuring the neural network in response to the determination; performing inference using the portion of the DNN model to generate compressed features; and transmitting the compressed features to another WTRU.

[0006] According to another embodiment, a method is presented that includes receiving a deep neural network (DNN) model, the portion of the DNN model being after a division point of the DNN model and including a neural network for expanding features at the division point of the DNN model; receiving one or more features output from another WTRU; obtaining a compression factor for the neural network; determining which nodes in the neural network should be connected in response to the compression factor; configuring the neural network in response to the determination; and performing inference using the portion of the DNN model using the one or more features output from another WTRU as inputs to the neural network.

[0007] Further embodiments include systems configured to perform the methods described herein. Such systems may include a processor and a non-transitory computer storage medium storing instructions that, when executed on the processor, operate to perform the methods described herein. [Brief description of the drawings]

[0008] [Figure 1A] FIG. 1 is a system diagram illustrating an example communication system in which one or more disclosed embodiments may be implemented. [Figure 1B] 1A is a system diagram illustrating an example wireless transmit / receive unit (WTRU) that may be used within the communications system illustrated in FIG. 1A, according to one embodiment. [Diagram 2] We present a mechanism for distributed AI across two devices without feature size compression. [Figure 3A] We show DNNs with one, two, and three candidate splits for feature compression, respectively. [Figure 3B] We show DNNs with one, two, and three candidate splits for feature compression, respectively. [Figure 3C]We show DNNs with one, two, and three candidate splits for feature compression, respectively. [Figure 4] We present a DNN with a single split for feature compression. [Figure 5A] We show a feature size compression mechanism for distributed AI between two devices, device 1 and device 2, using a bandwidth-reducer (BWR) and a bandwidth-expander (BWE), where a single compression factor is supported. [Figure 5B] 1 illustrates a feature size compression mechanism where multiple compression factors are supported. [Figure 6A] 4 shows the total inference latency without BWR and BWE. [Figure 6B] 13 shows the BWR and the total inference latency with BWR, and the size of the intermediate data can be reduced. [Figure 7] 1 illustrates a process for dynamically switching between partitioning and compression factor (CF) configurations, according to one embodiment. [Figure 8A] Devices 1 and 2 are shown estimating their computational capabilities and transmission channels. [Figure 8B] Illustrates receipt of AI / ML models from each of the devices. [Figure 8C] 4 shows the inference time behavior of the device. [Figure 9] 1 illustrates a method involving a single split in a DNN for adaptive feature compression, according to one embodiment. [Figure 10] 1 shows an example DySw that can shrink and expand an input of size 4. [Figure 11] The connections of the DySw configuration shown in FIG. 9 are shown. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] 1A illustrates an example communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcasts, etc., to multiple wireless users. The communication system 100 may enable the multiple wireless users to access such content through sharing of system resources, including wireless bandwidth. For example, the communications system 100 may use one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tail unique-word DFT-Spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multicarrier (FBMC), etc.

[0010] 1A, the communications system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, a RAN 104, a CN 106, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, although it will be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d, any of which may be referred to as a “station” and / or “STA,” may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearables, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in an industrial and / or automated processing chain context), consumer electronics devices, devices operating in commercial and / or industrial wireless networks, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may be referred to interchangeably as a UE.

[0011] The communication system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106, the Internet 110, and / or other networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node B, an eNodeB, a Home Node B, a Home eNodeB, a gNB, an NR NodeB, a site controller, an access point (AP), a wireless router, etc. Although the base stations 114a, 114b are each shown as a single element, it will be understood that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0012] The base station 114a may be part of the RAN 104, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. The base station 114a and / or base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide wireless service coverage for a particular geographic area, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In one embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and may utilize multiple transceivers for each sector of the cell, for example, using beamforming to transmit and / or receive signals in a desired spatial direction.

[0013] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0014] More specifically, as noted above, the communications system 100 may be a multiple access system and may use one or more channel access schemes, such as, for example, CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base stations 114a of the RAN 104 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 116 using wideband CDMA (WCDMA). WCDMA may include communications protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​Uplink Packet Access (HSUPA).

[0015] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-Advanced, LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).

[0016] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR radio access, which may establish the air interface 116 using New Radio (NR).

[0017] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, e.g., using dual connectivity (DC) principles. Thus, the air interface utilized by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0018] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement a wireless technology such as IEEE 802.11 (i.e., Wireless Fidelity, WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access, WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), or the like.

[0019] 1A may be, for example, a wireless router, a Home NodeB, a Home eNodeB, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a localized area, such as a location of a business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, etc. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may establish a picocell or a femtocell using a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.). As shown in FIG. 1A, the base station 114b may have a direct connection to the Internet 110. Thus, the base station 114b may not need to access the Internet 110 through the CN 106.

[0020] The RAN 104 may communicate with the CN 106, which may be any type of network configured to provide voice, data, application, and / or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have various quality of service (QoS) requirements, such as, for example, different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 106 may provide call control, billing services, mobile location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions such as user authentication. Although not shown in FIG. 1A, it will be understood that the RAN 104 and / or the CN 106 may communicate directly or indirectly with other RANs that use the same RAT as the RAN 104 or a different RAT. For example, in addition to being connected to the RAN 104, which may utilize NR radio technology, the CN 106 may also communicate with another RAN (not shown) using GSM, UMTS, CDMA2000, WiMAX, E-UTRA or WiFi radio technologies.

[0021] The CN 106 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a public switched telephone network that provides plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP), and / or the internet protocol (IP) of the TCP / IP Internet protocol suite. The networks 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs that may use the same RAT as the RAN 104 or a different RAT.

[0022] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links). For example, the WTRU 102c shown in FIG. 1A may be configured to communicate with a base station 114a, which may use a cellular-based wireless technology, and a base station 114b, which may use an IEEE 802 wireless technology.

[0023] 1B is a system diagram illustrating an example WTRU 102. As shown in FIG. 1B, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be understood that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

[0024] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. Although FIG. 1B depicts the processor 118 and the transceiver 120 as separate components, it will be understood that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0025] The transmit / receive element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) over the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF and light signals. It will be appreciated that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0026] 1B as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may use MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0027] The transceiver 120 may be configured to modulate signals transmitted by the transmit / receive element 122 and demodulate signals received by the transmit / receive element 122. As mentioned above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs, such as, for example, NR and IEEE 802.11.

[0028] The processor 118 of the WTRU 102 may be coupled to and may receive user-entered data from a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light-emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from and store data in any type of suitable memory, such as non-removable memory 130 and / or removable memory 132. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, etc. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or home computer (not shown).

[0029] The processor 118 may receive power from the power source 134, but may be configured to distribute and / or control the power to other components in the WTRU 102. The power source 134 may be any suitable device for providing power to the WTRU 102. For example, the power source 134 may include one or more dry batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0030] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding a current location of the WTRU 102. In addition to or in lieu of information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 116 from a base station (e.g., base stations 114a, 114b) and / or determine its location based on the timing of signals being received from two or more nearby base stations. It will be understood that the WTRU 102 may obtain location information by way of any suitable location determination method while remaining consistent with an embodiment.

[0031] The processor 118 may further be coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, and the like. The peripherals 138 may include one or more sensors, which may be one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor, a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0032] The WTRU 102 may include a full-duplex radio where the transmission and reception of some or all of the signals (e.g., associated with a particular subframe for both the UL (e.g., for transmission) and the downlink (e.g., for reception)) may be parallel and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and or substantially eliminate self-interference via hardware (e.g., chokes) or signal processing via a processor (e.g., via a separate processor (not shown) or processor 118). In one embodiment, the WRTU 102 may include a half-duplex radio for the transmission and reception of either some or all of the signals (e.g., associated with a particular subframe for either the UL (e.g., for transmission) or the downlink (e.g., for reception)).

[0033] Although the WTRU is illustrated in FIGS. 1A-1B as a wireless terminal device, it is contemplated that in certain representative embodiments such a terminal device may use (e.g., temporarily or permanently) a wired communications interface with the communications network.

[0034] 1A-1B and the corresponding description thereof, one or more or all of the functions described herein with respect to one or more of the WTRUs 102a-d, base stations 114a-b, eNode-Bs 160a-c, MME 162, SGW 164, PGW 166, gNBs 180a-c, AMFs 182a-b, UPFs 184a-b, SMFs 183a-b, DNs 185a-b, and / or any other devices described herein may be performed by one or more emulation devices (not shown). The emulation devices may be one or more devices configured to emulate one or more or all of the functions described herein. For example, the emulation devices may be used to test other devices and / or to simulate network and / or WTRU functions.

[0035] The emulation devices may be designed to implement one or more tests of other devices in a lab environment and / or an operator network environment. For example, one or more emulation devices may perform one or more or all functions while fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices in the communication network. One or more emulation devices may perform one or more or all functions while temporarily implemented / deployed as part of a wired and / or wireless communication network. The emulation devices may be directly coupled to another device for testing purposes and / or may perform testing using terrestrial wireless communication.

[0036] The one or more emulation devices may perform one or more functions, inclusive, while not being implemented / deployed as part of a wired and / or wireless communication network. For example, the emulation devices may be utilized in test scenarios in a test lab and / or in an undeployed (e.g., test) wired and / or wireless communication network to implement testing of one or more components. The one or more emulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (which may include, for example, one or more antennas) may be used by the emulation devices to transmit and / or receive data.

[0037] As mentioned above, the execution of DNN operations is often addressed by transferring data from mobile devices to cloud servers, where all computations are performed. However, this requires bandwidth, is time intensive (due to transmission latency), and raises data privacy concerns. One way this can be solved is by performing all computations on the user device (e.g., mobile phone) via lightweight, low-precision DNNs. Another way is via high-precision DNNs, but by sharing the computations across single / multiple mobile devices and / or the cloud.

[0038] Flexible AI Method Model compression techniques are widely exploited to execute DNN models only on the user device. They reduce the model memory footprint and runtime, allowing it to adapt to a specific device. However, it may not be known in advance on which device a model will run, and even if the device is known, its available resources may change over time, for example due to other processes. To overcome these problems, a family of so-called flexible AI models has been recently proposed. These models can instantly adapt to available resources, for example through enabling early classification termination, adapting the model width (slimming), or enabling switchable model weight quantization.

[0039] Distributed AI method Some of the so-called distributed AI methods split the model between two or more devices (i.e., WTRUs) or between a device and the cloud / edge. For example, FIG. 2 shows a mechanism for distributed AI between two devices, device 1 and device 2, without feature size compression. In distributed AI, intermediate data (features), which may be of very high dimensionality, needs to be transmitted. This adds latency in processing and is not always possible due to bandwidth limitations of the corresponding transmission network. To overcome this problem, methods have been proposed to reduce feature size through bottlenecks. FIG. 3A shows a DNN with one candidate split for feature compression, where a1, a2, or a3 can be used as split points. FIG. 3B shows a DNN with two candidate splits for feature compression (e.g., a1 and a2). FIG. 3C shows a DNN with three candidate splits for feature compression (e.g., c1, c2, and c3).

[0040] Without introducing any limitations, a feature may be considered as an individual measurable characteristic or feature of data that can be used to represent a phenomenon. One or more features may be associated with the input and / or output of a machine learning algorithm, a neural network, and / or one of its layers. For example, features may be organized as a vector. For example, features associated with a wireless use case may include time, transmitter identity, and measurements on a Reference Signal (RS).

[0041] For example, features associated with algorithms used to process positioning information may include values ​​associated with measurements of the positioning RS (PRS), quantities such as Reference Signal Receive Power (RSRP), quantities such as Reference Signal Receive Quality (RSRQ), quantities related to the Received Signal Strength Indication (RSSI), quantities related to time difference measurements based on signals of separate sources (e.g., in the case of time-based positioning methods), quantities related to angle-of-arrival measurements, quantities related to beam quality, and / or values ​​associated with outputs from sensors (WTRU rotation, imaging from a camera, etc.).

[0042] For example, features associated with algorithms used to process Channel State Information (CSI) may include measurements of quantities associated with reception of a Channel State Reference Signal (CSI-RS), a Synchronization Signal Block (SSB), a Precoding Matrix Indication (PMI), a Rank Indicator (RI), a Channel Quality Indicator (CQI), RSRP, RSRQ, RSSI, etc.

[0043] For example, features associated with algorithms used to process beam management and selection may include quantities associated with CSI, Transmit / Receive Point (TRP) identity (ID), beam ID, and / or measurements similar to those for processing one or more parameters related to Beam Failure Detection (BFD) (e.g., threshold determination of sufficient beam quality).

[0044] Similarly, any method described herein may apply to or may further include specific parameter settings for a particular stage of an AI / ML process, such as hyperparameters used for a machine learning algorithm for training or inference.

[0045] FIG. 4 shows a DNN with a single partition (a2, b2, respectively) for feature compression, where the feature size is reduced (a) from 4 to 2 and (b) from 4 to 3. Specifically, (a3) ​​is a sub-network that realizes the feature size reduction from 4 to 2, and (b3) is a sub-network that realizes the reduction from 4 to 3. In existing works, the DNN is trained from scratch with a feature compressor and expander for each compression factor. Note that the compression factor is the ratio of the feature size at the output of the compressor to the feature size at the input to the compressor. This means that whenever the compression factor needs to be changed, the device and cloud server must adjust and newly download a new model from the cloud server. FIG. 5A shows a feature size compression mechanism for distributed AI between two devices, device 1 and device 2, using a bandwidth reducer (BWR, 510) and a bandwidth expander (BWE, 520), where a single compression factor is supported. However, these methods cannot adapt the bottleneck to different transmission network bandwidths.

[0046] To provide flexibility in the distributed AI paradigm, we introduce the Flexible and Distributed AI (FD-AI) method. The proposed method is distributed because the DNN can be split among two or more devices. Also, the proposed method is flexible because the split point can be selected among several possible split point candidates depending on the available resources in the device. In addition, the feature size transmitted at each split point can be compressed to fit the available network bandwidth for transmission.

[0047] In one embodiment, we propose a switchable bottleneck sub-network that is part of the DNN architecture. The bottleneck sub-network is switchable because it can adapt to different transmission network bandwidths during inference. In the proposed design, we have one bottleneck sub-network with layers for reducing feature size and another set of layers for restoring feature size to its original size. These bottleneck sub-networks can be incorporated into one or more split positions of any existing DNN. For simplicity, in the following description, we consider a DNN with a single split with one set of bottleneck sub-networks for feature size reduction and expansion.

[0048] In one embodiment, the first device may be either an edge device or a cloud server, and the second device may be either an edge device or a cloud server. More generally, the methods described herein may be applied to any device exchanging data over a communication link. Such devices may include processing of split neural networks or autoencoder functions. The methods described herein may be applicable to processing in devices, for example, for end user applications (e.g., audio, video, etc.) or for functions related to processing for transmission and / or reception of data. More generally, such devices may be mobile terminals, radio access network nodes such as gNBs, etc. Such communication links may be wireless links and / or interfaces, such as 3GPP Uu, 3GPP sidelink, or Wifi links.

[0049] The DNN layers up to the split point with the feature size reduction layer of the bottleneck sub-network are loaded on the first device. The remaining part, i.e. the bottleneck sub-network expander and the remainder of the DNN after the split point, is loaded on the second device. We refer to the bottleneck sub-network including the reducer and expander as the Dynamic feature size Switch (DySw). The features to be sent to the second device are extracted in the middle of the DySw. We call a DNN that achieves this a Dynamic Switchable Feature Size Network (DyFsNet). DyFsNet applies generally to any DNN architecture, such as convolutional neural networks (CNNs), and is novel in design and training. Inference in DyFsNet is simple and tunable (with respect to split position and available network bandwidth).

[0050] FIG. 5B illustrates an embodiment of a feature size compression mechanism that uses a bandwidth reducer (BWR) and a bandwidth expander (BWE) to support multiple compression factors for distributed AI between two devices, device-1 and device-2, where K1, K2, ..., K N specifies the compression factor within the trainable BWR (530) and BWE (540), which can be switched exclusively and dynamically during inference.

[0051] More specifically, device-1 and device-2, optionally together with the server, monitor the channel conditions and device states and select the compression factor and feature size at the split location. Device-1 receives the first part of the DNN model up to the split location, and device-2 receives the remaining part of the DNN model. Inference is performed at device-1 to compute features from the input, which are then compressed by the BWR. Different compression factors can be obtained by controlling which nodes are connected in the BWR (530), as described in more detail in connection with Figures 10 and 11. At device-2, the compressed features are received and expanded by the BWE (540). Similar to the BWR, the BWE can control the compression factor by controlling the node connections in the BWE. Device-2 then continues the inference and provides the final output.

[0052] Network bandwidth limitations introduce additional latency into the overall inference. Figure 6A shows the total inference latency without BWR and BWE. Figure 6B shows the total inference latency with BWR and BWR, where the size of the intermediate data can be reduced.

[0053] As mentioned above, we propose a method to reduce the intermediate data size at different locations in the DNN model to limit the throughput requirements on the communication network while roughly maintaining the accuracy of the prediction. Figure 7 shows the process of dynamically switching between partitioning / compression factor (CF) configurations, according to one embodiment.

[0054] During the model training and partitioning / CF estimation stage (710), the DyFsNet model is trained for different partitions and CFs. This can currently be done offline in a cloud server. The trained models are stored in the cloud server and are available for device download. An orchestrator (on the server side) manages the coordination of trained model selection and transmission to end devices based on requests. We assume that information about bandwidth is available. Based on this, the CF is estimated as the ratio of feature size to available bandwidth.

[0055] For example, the orchestrator or external control system determines the splitting position of the DNN based on the computational capabilities of the end devices (e.g., device-1 and device-2), which is communicated to the devices that load the DNN for processing according to the splitting information.

[0056] In the model deployment stage (720), the trained segmentation models are received by the device. Once received, they are loaded onto the device for inference.

[0057] The state of the network (e.g., bandwidth) and / or the devices (e.g., available processing power) is monitored (730). The devices monitor the network channels between them and coordinate CF between them. This is done without involving a server.

[0058] Once a consensus is reached among the devices, a CF selection (740) is made, thus influencing the feature size at the split location. Note that the available CF options depend on the number of channels in the filter of the DNN layer where the split is realized. Typically, the CF is selected to approximately match the available bandwidth, but not exactly.

[0059] The split model inference is performed on the first device and the second device (750). For example, the first device computes intermediate features using the DNN up to the split, compresses the features, and forwards the compressed features to the second device. The second device receives the compressed features, does not compress them, and continues the DNN inference. In an embodiment where the device is a wireless terminal device and / or the communication link of the device is a wireless air interface (e.g., NR Uu, sidelink, etc.), the device can perform at least one of the following: - Start adapting as proposed herein. For example, the device may adapt split processing points, feature dimensions, compression factor, inference latency, processing requirements, accuracy of features, or any other aspect proposed herein. A device may trigger such adaptation for AI processing upon determining at least one of the following in relation to L1 / physical (PHY) layer operations: o The device may determine that a change in radio characteristics has occurred, where such characteristics may affect the transmission data rate over the interface, e.g., a change in cell identity, a change in carrier frequency, a change in bandwidth part (BWP), a change in the BWP and / or number of physical resource blocks (PRBs) of the cell, a change in sub-carrier spacing (SCS), a change in the number of aggregated carriers available for transmission, a change in available transmit power, a change in a measurement quantity, etc. The device may determine that a change in operating conditions over the air interface has occurred, such as a change in control channel resources (CORESET) or identity, where a first identity may be associated with a first threshold and a second identity may be associated with a second threshold. The device may determine that the changes exceed certain possibly configured thresholds that indicate degradation of channel quality and may implement adaptations that reduce the data rate associated with AI processing. Conversely, the device may determine that radio conditions improve and may implement adaptations that may increase the data rate associated with AI processing. For example, this is applicable to device physical layer features such as CSI autoencoding. A device may trigger such adaptation for AI processing upon determining at least one of the following regarding L2 / Medium Access Control (MAC) layer operations: The device may determine that changes in data processing, information bearers (e.g., data radio bearers, signaling radio bearers) have occurred, and such characteristics may affect the transmission data rate over the interfaces available to AI processing, such as logical channel prioritization parameters (e.g., Packet Delay Budget (PDB), Prioritized Bit Rate (PBR), changes in TTI duration / numerology, changes in associated QoS flow IDs, mapping restrictions to sets of resources that allow different data rates, etc.). ○ The device may determine that the change exceeds a certain possibly configured threshold indicating a decrease in the data rate available for AI processing and may implement an adaptation that reduces the data rate associated with the AI ​​processing. Conversely, the device may determine an increase in the available data rate and may implement an adaptation that may increase the data rate associated with the AI ​​processing. For example, this can apply to system level functions such as the positioning function of a device, for example, this can be application to a particular data radio bearer (DRB) and / or DRB type, e.g., a DRB associated with a particular AI-enabled application such that a change in the DRB or its characteristics may trigger an adaptation of AI-based processing at the associated application layer. The device may trigger such adaptation for AI processing upon determining, with respect to L3 / Radio Resource Control (RRC) layer operation, at least one of the following: o The device may determine that a configuration change has occurred that affects one or more of the L1 / L2 configurations, such as, for example, the aspects described above that may change the available data rates. o The device may, for example, determine that it has received a reconfiguration message for mobility and / or should apply (e.g., for a conditional handover command), where the message may include an indication of AI processing and / or applicable data rates for its associated radio bearer. The device may determine that a radio link failure, such as a radio link failure (RLF), has occurred. o The device may determine that the change exceeds a certain possible configured threshold indicating a decrease in the data rate available for AI processing and may implement an adaptation that reduces the data rate associated with AI processing. Conversely, the device may determine an increase in the available data rate and implement an adaptation that may increase the data rate associated with AI processing. Alternatively, it may determine that the event itself may be associated with an increase (e.g., the addition of a cell to the device's configured connectivity, e.g., dual connectivity) or decrease (e.g., the removal of a cell from the RLF and / or device's configured connectivity) of the data rate available for AI processing. The device may trigger such adaptation for AI processing upon determining at least one of the following in relation to available processing resources: The device may determine that a change in available hardware processing has occurred, for example, based on a change in the number of instantiated and / or active AI processes, based on a change in dynamic device capabilities, or based on changes in processing requirements for AI processing (e.g., inference latency, precision). The device may determine that a change in the device's power state has occurred. For example, the device may determine that it has transitioned from a first state to a second state, where such state may relate to an RRC connectivity state (IDLE, INACTIVE, or CONNECTED), a DRX state (ACTIVE, INACTIVE), or different configurations thereof. ○ The device may determine that the change is above a particular possible configured threshold, indicating a decrease in available processing resources. Conversely, the device may determine an increase in available processing resources and implement an adaptation that may increase the data rate associated with the AI ​​processing. Similarly, a particular state may be associated with an associated particular AI processing level, split point configuration, and / or data rate. The device may trigger such adaptation for AI processing upon determining that it receives control signaling in accordance with at least one of the following: o The device may receive control information indicating either an increase or decrease in AI processing / data rate available for AI processing. This may be implicitly based on signaled values ​​and / or modification of control channel properties of values ​​as described above for L1, L2, L3 processing and / or power saving management, or may explicitly use an indication in a control message. Such control information may be received in L1 signaling, L1 messages, e.g. DCI on the PDCCH, in an L2 MAC control element, or in an RRC message. ○ Control information may include the specific split point configurations applied to a given AI process, hyperparameter settings, target resolution, target accuracy, target feature vectors, etc.

[0060] Figures 8A, 8B, and 8C provide another illustration of the process. Figure 8A shows that devices 1 and 2 (840, 860) estimate their computational capabilities and transmission channels (850). Those estimates are communicated (820, 830) to the operator / edge / cloud, which requests a suitable AI / ML model (810).

[0061] 8B shows receipt of the AI / ML models from each of the devices. The operator / cloud / edge makes a model selection and transmits the model over the network (830), and the requested model is received by devices 1 and 2.

[0062] FIG. 8C illustrates the inference time operation of the devices. Device-1 calculates the features and then sends the appropriate dimension feature size to device-2 based on the channel conditions. Device-1 performs inference on the input data (870). The input data can be one or many images from the device memory, or images captured live from the device's camera, or audio data on the device memory, or audio data captured live from the device microphone, or any other data that needs to be processed by the DNN. Device-1 outputs intermediate or initial output (880) processed by the DNN, such as in the case of MSDNet type DNNs. Information required for further processing of the features is also communicated to device-2 via the channel (850). Device-2 receives the features and continues inference and CF switching as necessary, providing a final output (890). Additionally, device-1 sends the features along with control information to device-2 for further processing of the features. Device-2 receives the features and control information and continues inference.

[0063] Figure 9 shows the proposed method using a single split in a DNN for feature compression. Figure 9(a) shows a jointly trained sub-network DySw (a3) ​​with no compression factor selected. Figure 9(b) shows a jointly trained sub-network (b3) with a selected feature compression factor of 4 vs. 2. Figure 9(c) shows a jointly trained sub-network (c3) with a selected feature compression factor of 4 vs. 3. Note that the DNNs in Figures 9(a), 9(b), and 9(c) are the same (single) DNN.

[0064] DySw can be trained together with the entire DNN. Alternatively, a DNN without DySw is pre-trained and the DySw sub-network is added. Note that in this alternative solution, the pre-trained DNN is augmented with the DySw (a3) ​​sub-network and the training is only on DySw while keeping the weights of the pre-trained DNN unchanged (i.e., fixed).

[0065] As shown in Fig. 9, DySw is reconfigurable to accommodate multiple compression factors. The reconfiguration is achieved through the connection details of the DySw nodes. For example, for a DySw sub-network as shown in Fig. 10, we can maintain a matrix of size 4 × 3 that specifies the node connections as shown in Fig. 11. Each element (E ij ) represents whether input node i is connected to output node j, with "0" representing disconnection and "1" representing connection. The matrices shown in Figures 11(a), 11(b), and 11(c) correspond to Figures 9(a), 9(b), and 9(c), respectively. Specifically, Figure 9(a) specifies that none of the input nodes are connected to any of the output nodes, Figure 9(b) specifies that only two of the output nodes (output node-2 and node-3) are connected to input nodes, and Figure 9(c) specifies that all nodes of the input are connected to the output. Figure 11 shows the connections on the reducer side, and the extender can maintain matrices corresponding to different compression factors. In one embodiment, the shape of the matrix on the extender side is transposed (with respect to that on the reducer side), but the number of all-zero rows remains the same.

[0066] As shown in Figure 8, the devices adjust the CF. In one embodiment, an orchestrator or external control system informs device-1 about the available bandwidth. Device-1 determines the CF to be used based on the information about the bandwidth. Device-1 then switches DySw to achieve the feature size compression corresponding to the determined CF. Device-1 can also communicate the CF it is using, so device-2 switches its side of the DNN to match the communicated information.

[0067] In one embodiment, after a CF is selected, device-1 determines which connections should be disabled between nodes to accommodate the selected CF, and device-2 also determines which connections should be correspondingly disabled to properly implement the expansion. The CF determines how many output nodes are connected to the input nodes, and how many are determined through learning.

[0068] As mentioned above, FIG. 10 shows an exemplary DySw that can shrink and expand an input of size 4. Note that while FIG. 10 shows a single-layer "reducer" block for simplicity, the reducer is not limited to a single layer. The illustrated DySw is capable of 4-to-3, 4-to-2, and 4-to-1 compressions, as well as corresponding expansions (i.e., 1-to-4, 2-to-4, and 3-to-4). The DySw design may have additional layers as needed, e.g., a BatchNorm layer for better training. Here, only the reducer (BWR shown to the left of the dotted line) and the expander (BWE shown to the right of the dotted line) are shown. The nonlinearity is inherent in the layers. The BatchNorm layer may be an optional layer required for efficient training, and is therefore not shown here.

[0069] More generally, a typical DySw comprises four types of layers: feature dimensionality reducer and expander layers, nonlinear layers, and batch normalization (BatchNorm) layers. Among these layers, the BatchNorm layer is optional. A simple DySw is shown in Figure 10.

[0070] DySw used in DNN classifiers can be trained using traditional task-specific losses, e.g., cross-entropy loss for classification tasks or mean squared error loss for regression tasks. DySw can be used for any task, i.e., classification, detection, or segmentation, and in any DNN architecture, i.e., CNN, GAN, autoencoder, etc. Training DySw involves learning the parameters of the reducer-expander layer weights and the batch normalization layer (also denoted as "BatchNorm"). BatchNorm is used for faster convergence of training.

[0071] DySw training allows for additional constraints on the loss objective. As an illustration, we show the addition of a reconstruction loss over DySw. The reconstruction loss penalizes the disparity between the input to DySw and the output to DySw. DySw is an auxiliary, optional entity that can be added to a trained DNN.

[0072] In DySw, the reduction coefficients can be switched on-the-fly during inference. In DyFsNet, training iterations are modified to co-learn the shared DySw weights with multiple reduction coefficients, as detailed further below.

[0073] Training of DySw can be offline or online, performed on cloud / operator / edge, or federated training on device. Here, we describe the architecture and training of split DNN in the case of single split between two devices with DySw. The training mechanism described herein can be extended to multiple split cases. In the following, we describe in detail the architecture of split DNN, the architecture of DySw layer and DyFsNet (DNN with DySw layer), as well as different loss functions and their training.

[0074] Consider a split at the end of the l-th layer, where device-1 processes up to layer-1 and device-2 processes from layer l+1 onwards. Let the part of the DNN in device-1 be h device1 Similarly, a part of the DNN in device-2 is device2 The input to the DNN can be any type of data, but here we define the input X as X∈R. {W×H×3} where W, H are the width and height, and 3 represents the number of color channels (e.g., RGB). The feature tensor (or simply features) in the segmentation is y l ∈R {M×N×C} where M, N, and C represent the width, the height, and the number of channels. l is sent to device-2 via the wireless network, and device-2 receives y l and produces output Y. Therefore, y l =h device1 (X) and Y=h device2 (y l ).

[0075] DySw is h DySw h DySw The parameters of θ DySw If we refer to the reducer (first part) and expander (second part) of DySw as BWR and BWE, example implementations of such reducer and expander can include a convolutional layer, a nonlinear layer (ReLu), and a batch normalization layer (BatchNorm), as summarized below.

[0076]

number

[0077] The DNN with DySw is called DyFsNet. Let DyFsNet be represented as h. θ is the parameter of h. The subnetwork of DyFsNet before the split point is

[0078]

number

[0079]

number

[0080] DySw switches between different compression factors (CF) of feature sizes. The CF switching is indexed by K. The intermediate output of the split of DyFsNet, indexed by K, is

[0081]

number

[0082]

number

[0083]

number

[0084]

number

[0085]

number

[0086] The setup provides two types of supervision: ground truth labels

[0087]

number

[0088]

number

[0089] DyFsNet trained from scratch:

[0090]

number

[0091] DyFsNet trained using pre-trained initialization:

[0092]

number

[0093] Multi-part DyFsNet trained from scratch:

[0094]

number

[0095] Multi-partition DyFsNet Trained from Pre-trained Initialization

[0096]

Number

[0097] DyFsNet Training Algorithm (X i , Y i ) ∈ D is used as the dataset, where X i and Y i are the data and its supervision respectively, i ∈ {0, 1,..., N} is the index, N is the number of training samples, and Num-of-epochs is the number of training epochs. Here, we provide a training algorithm for the classifier using the global loss, i.e., cross-entropy and KD. The KD-based loss can be of four types from the distillation of i) the output of DySw without compression (i.e., DySw with K = 1), ii) the output of DySw with the current lower compression factor (i.e., distillation from DySw with K = K1 to K = K2, where K1 < K2), iii) the affine combination of the non-compressed DySw output and the closest compressed DySw output / s, or iv) the output of a completely different DNN architecture that is well-trained for the same task.

[0098] The overall algorithm is as follows. a. Calculate the loss of DyFsNet for the non-compressed configuration of DySw. In our example, it is the cross-entropy loss, but it is not limited to that only. b. Perform backpropagation and accumulate the gradients for the non-compressed configuration of DySw. c. Select N r numbers of CF in the range of 1 to C, where 1 represents non-compression and C represents maximum compression. d. When CF = 2 to N r : i. Compute the loss of DyFsNet for distillation type (i), (ii), (iii), or (iv). ii. Do backpropagation and accumulate the gradients of DySw. e. Update the weights using the accumulated gradients.

[0099] In one embodiment, the following pseudocode is used:

[0100] KD from uncompressed (K=1) DySw output:

[0101] [Table 1]

[0102] KD from the output of DySw with K=K1 to DySw with K=K2, where K1 <K2である:

[0103] [Table 2]

[0104] KD from the output of DySw with K=K1 to DySw with K=K2, where K1 <K2である:

[0105] [Table 3]

[0106] We tested the proposed idea on an image classification task using the well-known MSDNet model. This model has several CNN blocks that can perform classification at the output of any block. We want to split this large network at the end of different blocks and send the corresponding features to a second device (or the cloud). The feature dimensions of MSDNet at the end of each block of the ImageNet dataset are shown in Table 1.

[0107] [Table 4]

[0108] Here, we demonstrate the usefulness of feature size reduction through an illustration of the data rate requirements in a typical DNN. The data rates required to transmit features corresponding to a single image of size 224x224x3 generated by a DNN used for image classification (MSDNet) range from 13Mbps to 0.5Gbps, a challenging data rate for transmission over wireless networks. In a preliminary implementation of our approach using the MSDNet model, we were able to reduce the feature size by 50% with a maximum loss of 1% in accuracy.

[0109] Below we describe our implementation of DySw on MSDNet for CIFAR-100, where the DNN is split at 7 positions and the feature size at each split position (each unit is 16 bits) is shown in Table 2. We achieved compression factors of 1, 2, 4 and 10.

[0110] [Table 5]

[0111] We investigate the effect of adding a bandwidth reducer-expander in MSDNet. We show the results in Table 3 for the baseline (without bandwidth reducer-expander) and with bandwidth reducer-expander for reduction factors of 1, 2, 4, and 10. Reduction factors of 1, 2, 4, and 10 correspond to 100%, 50%, 25%, and 10% of the original bandwidth, respectively. We can see that the accuracy of MSDNet with reduced bandwidth is almost the same as the baseline MSDNet without any reduction. Note that the accuracy is for the compressed implementation at the end of all six blocks (0-6) and all scales. In other words, by adding a new bandwidth reducer-expander at each split point, we can significantly reduce the features to support feature transmission, while the classification accuracy remains almost unchanged.

[0112] [Table 6]

[0113] There are methods of switchable precision networks that refer to the precision of CNN weights. There have also been studies on switchable multi-width CNNs. However, unlike them, we propose a switchable feature bandwidth network that can switch between different feature bandwidths at inference time. This switchability is useful to address the bandwidth constraints of communication channels between devices or device clouds or other combinations of them. For example, this mechanism can be used independently of CNN architectures, which can be seamlessly used with existing models that perform different machine learning tasks, such as ResNet, AlexNet, DenseNet, SoundNet, VGG, etc. This mechanism can also be used independently of other types of feature compression techniques, such as weight quantization.

[0114] The proposed approach deals with efficient bandwidth for transmission for distributed AI with provisions for switching between multiple feature bandwidths. During distributed inference in edge devices, each device needs to load a part of the AI ​​model only once, but the input / output features communicated between them can be flexibly configured according to the available transmission bandwidth by enabling / disabling connections between nodes in the DySw. Other parameters of the DNN remain the same when some nodes are connected or disconnected to achieve the desired compression factor. That is, the same DNN model is used for different compression factors, and there is no need to download a new DNN model to fit the compression factor or network bandwidth.

[0115] AI processing can be used, for example, but not limited to, on images taken with a basic mobile phone camera, or on images taken from a smart TV camera for UI interaction via gesture detection. The proposed approach can be used in various scenarios. For example, the AI ​​model can be split between the device and the cloud. Below we list some possible usage scenarios. 1. The AI ​​model is split between two devices: for example, a user wants to process data captured on a smartwatch, where part of the processing can be done on the watch and the rest on the user's mobile phone. 2. AI models are split between multiple devices and possibly the cloud: for example, a user might want to quickly process the feed from a smart CCTV camera on the camera itself, and have detailed processing handled on the cloud or a local server. 3. Similar to use case 3, but with speech / audio processing using a computationally enabled microphone instead of a CCTV camera. 4. Share medical data lab and cloud processing. 5. A terminal device capable of communicating via a wireless link, where the AI ​​processing relates to transmit and / or receive functions of the wireless processing chain (e.g., CSI compression, CSI auto-encoding, positioning determination, etc.). 6. A terminal device capable of communicating via a wireless link, where the AI ​​processing is related to scheduling or data processing functions, such as QoS processing (eg, user plane data rate adaptation, etc.).

[0116] In this application, various numerical values ​​are used. Specific values ​​are provided for illustrative purposes and the described aspects are not limited to these specific values.

[0117] Although the features and elements are described above in certain combinations, one skilled in the art will understand that each feature or element may be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware embodied in a computer-readable medium for execution by a computer or processor. Examples of non-transitory computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs). A processor in association with software may be used to implement a video encoder, video decoder, or both, a radio frequency transceiver for use in a UE, a WTRU, a terminal, a base station, an RNC, or any host computer.

[0118] Furthermore, in the above embodiments, processing platforms, computing systems, controllers, and other devices including processors are described. These devices may include at least one central processing unit ("Central Processing Unit (CPU") and memory. In accordance with the practices of those skilled in the art of computer programming, references to operations and symbolic representations of operations or instructions may be performed by various CPUs and memories. Such operations and operations or instructions may be referred to as being "executed," "computer-executed," or "CPU-executed."

[0119] Those of ordinary skill in the art will appreciate that the operations and symbolically represented operations or instructions include the manipulation of electrical signals by the CPU. The electrical system represents data bits that can cause a resultant transformation or reduction of electrical signals, and maintains the data bits in memory locations of the memory system, thereby reconfiguring or otherwise altering the operation of the CPU and the processing of other signals. The memory locations in which the data bits are maintained are physical locations that have particular electrical, magnetic, or optical properties that correspond to or represent the data bits. It should be appreciated that the exemplary embodiments are not limited to the platforms or CPUs mentioned above, and that other platforms and CPUs may support the methods provided.

[0120] The data bits may also be maintained on a computer readable medium, including magnetic disks, optical disks, and any other volatile (e.g., random access memory ("RAM")) or non-volatile (e.g., read only memory ("ROM")) mass storage system readable by a CPU. The computer readable medium may include computer readable media that resides exclusively on a processing system, or distributed, cooperative, or interconnected among multiple interconnected processing systems that may be local or remote to a processing system. It is understood that representative embodiments are not limited to the memories described above, and that other platforms and memories may support the methods described.

[0121] In an example embodiment, any of the operations, processes, etc. described herein may be implemented as computer-readable instructions stored on a computer-readable medium. The computer-readable instructions may be executed by a processor of a mobile unit, a network element, and / or any other computing device.

[0122] The use of hardware or software is generally (though not always, in certain circumstances the choice between hardware and software can be significant) a design choice that implies a trade-off between cost versus efficiency. There may be a variety of vehicles (e.g., hardware, software, and / or firmware) in which the processes and / or systems and / or other techniques described herein may be effective, and the preferred vehicle may vary depending on the context in which the processes and / or systems and / or other techniques are deployed. For example, if an implementer determines that speed and accuracy are paramount, the implementer may select a primarily hardware and / or firmware vehicle. If flexibility is paramount, the implementer may select a primarily software implementation. Alternatively, the implementer may select some combination of hardware, software, and / or firmware.

[0123] The foregoing detailed description has illustrated various embodiments of devices and / or processes through the use of block diagrams, flow charts, and / or examples. To the extent that such block diagrams, flow charts, and / or examples include one or more functions and / or operations, those skilled in the art will appreciate that each function and / or operation in such block diagrams, flow charts, or examples may be individually and / or collectively implemented by a wide variety of hardware, software, firmware, or substantially any combination thereof. Suitable processors include, by way of example, a GPU (graphics processing unit), a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and / or a state machine.

[0124] Although features and elements are provided above in specific combinations, those of ordinary skill in the art will understand that each feature or element can be used alone or in any combination with other features and elements. The present disclosure is not limited in terms of the specific embodiments described in this application, which are intended as examples of various aspects. As will be apparent to those skilled in the art, many modifications and variations can be made without departing from the spirit and scope of the present invention. No element, operation, or instruction used in the description of this application should be construed as critical or essential to the invention unless expressly set forth as such. In addition to those enumerated herein, functionally equivalent methods and apparatuses within the scope of the present disclosure will be apparent to those skilled in the art from the foregoing description. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure is to be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled. It is to be understood that the present disclosure is not limited to any particular method or system.

[0125] It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting.

[0126] In certain representative embodiments, some portions of the subject matter described herein may be implemented via application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), and / or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein may be equivalently implemented in whole or in part in an integrated circuit as a computer program running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as substantially any combination thereof, and that designing circuitry and / or writing software and / or firmware code is within the skill of those skilled in the art in light of this disclosure. In addition, those skilled in the art will recognize that the mechanisms of the subject matter described herein may be distributed as a program product in various forms, and that the exemplary embodiments of the subject matter described herein apply regardless of the particular type of signal-bearing medium used to actually effect the distribution. Examples of signal bearing media include, but are not limited to, recordable type media such as floppy disks, hard disk drives, CDs, DVDs, digital tape, computer memory, and transmission type media such as digital and / or analog communications media (e.g., fiber optic cables, wave guides, wired communications links, wireless communications links, etc.).

[0127] The subject matter described herein may in some cases depict different components that are included within or connected to different other components. It should be understood that such illustrated architectures are merely examples, and that in fact many other architectures that achieve the same functionality may be implemented. Conceptually, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality may be achieved. Thus, any two components combined herein to achieve a particular functionality may be viewed as "associated" with one another such that the desired functionality is achieved, regardless of the architecture or intermediate components. Similarly, any two components so associated may also be considered to be "operably connected" or "operably coupled" with one another to achieve the desired functionality, and any two components that may be so associated may also be considered to be "operably coupled" with one another to achieve the desired functionality. Examples of operably coupleable include, but are not limited to, physically matable and / or physically interacting components, and / or wirelessly interacting and / or wirelessly interacting components, and / or logically interacting and / or logically interacting components.

[0128] With respect to the use of substantially any plural and / or singular term herein, one of ordinary skill in the art can convert from plural to singular and / or from singular to plural as appropriate to the context and / or application. Various singular / plural permutations may be expressly set forth herein for purposes of clarity.

[0129] In general, those skilled in the art will understand that the terms used in this specification, and particularly in the appended claims (e.g., the body of the appended claims), are generally intended as "non-limiting" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," and the term "including" should be interpreted as "including but not limited to"). Furthermore, those skilled in the art will understand that where a specific number of recitations of an introduced claim are intended, such intent is expressly set forth in the claim, and in the absence of such recitation, no such intent exists. For example, where only one item is intended, the term "single" or similar language may be used. To aid in understanding, the following appended claims and / or description of this specification may include the use of the introductory phrases "at least one" and "one or more" to introduce the claim recitations. However, the use of such phrases should not be interpreted as meaning that the introduction of a claim recitation with the indefinite article "a" or "an" limits any particular claim that includes such an introduced claim recitation to an embodiment that includes only one such recitation, even if the same claim includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" should be interpreted to mean "at least one" or "one or more"). The same applies to the use of definite articles used to introduce claim recitations. In addition, those skilled in the art will recognize that even if a specific number of recitations of an introduced claim are explicitly recited, such recitation should be interpreted to mean at least the recited number (e.g., the simple recitation "two recitations" without other qualifiers means at least two recitations, or more than two recitations).Furthermore, when notation similar to "at least one of A, B, and C" is used, such structure is generally intended as a person of ordinary skill in the art would understand the notation (e.g., "a system having at least one of A, B, and C" includes, but is not limited to, systems having only A, only B, only C, A and B together, A and C together, B and C together, and / or A, B, and C together). When notation similar to "at least one of A, B, or C" is used, such structure is generally intended as a person of ordinary skill in the art would understand the notation (e.g., "a system having at least one of A, B, or C" includes, but is not limited to, systems having only A, only B, only C, A and B together, A and C together, B and C together, and / or A, B, and C together). Those skilled in the art will further appreciate that virtually any disjunctive word and / or phrase presenting two or more alternative terms, whether in the specification, claims, or drawings, should be understood to contemplate the possibility of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" should be understood to include the possibility of "A" or "B" or "A and B." Additionally, as used herein, the term "any of" followed by a list of items and / or a list of categories of items is intended to include "any of," "any combination of," "any more than," and / or "any more than" of the items and / or categories of items, individually or in combination with other items and / or categories of items. Additionally, as used herein, the term "set" or "group" is intended to include any number of items, including zero. Additionally, as used herein, the term "number" is intended to include any number, including zero.

[0130] In addition, where features or aspects of the disclosure are described in terms of a Markush group, those skilled in the art will recognize that the disclosure is also described thereby in terms of any individual members or subgroups of members of the Markush group.

[0131] As will be understood by those skilled in the art, for all purposes, including in terms of providing a written description, all ranges disclosed herein encompass any possible subranges and combinations of subranges thereof. Any recited range can be readily recognized as fully descriptive and allowing the same range to be broken down into at least equal halves, thirds, quarters, fifths, tenths, etc. As a non-limiting example, each range described herein can be readily broken down into a lower third, a middle third, an upper third, etc. As will also be understood by those skilled in the art, all words such as "up to," "at least," "greater than," "less than," etc., refer to a range that includes the recited number and that can be further broken down into subranges as described above. Finally, as will be understood by those skilled in the art, a range includes each individual element. Thus, for example, a group having 1 to 3 cells refers to a group having 1, 2, or 3 cells. Similarly, a group having 1 to 5 cells refers to a group having 1, 2, 3, 4, or 5 cells, and so on.

[0132] Moreover, the claims should not be read as limited to the provided order or to the provided elements unless specifically so recited. In addition, the use of the term "means for" in any claim is intended to be relied upon under 35 U.S.C. 112, paragraph 6, or means-plus-function claim format, and no claim without the term "means for" is so intended.

[0133] It is contemplated that the system may be implemented in software on a microprocessor / general purpose computer (not shown). In certain embodiments, one or more of the functions of the various components may be implemented in software controlling a general purpose computer.

[0134] In addition, although the invention is illustrated and described herein with reference to specific embodiments, the invention is not intended to be limited to the details shown. Rather, various modifications can be made in the details within the scope of the claims and equivalents thereof and without departing from the invention.

Claims

1. 1. A wireless transmit / receive unit (WTRU), comprising: a receiver configured to receive a portion of a deep neural network (DNN) model, the portion being before a division point of the DNN model, the portion of the DNN model including a neural network for compressing features at the division point of the DNN model; One or more processors, obtaining a compression factor for said neural network; determining which nodes in the neural network should be connected in response to the compression factor; configuring the neural network in response to said determining; performing inference using the portion of the DNN model to generate condensed features; and one or more processors configured to execute a transmitter configured to transmit the compressed features to another WTRU; A WTRU equipped with

2. The WTRU of claim 1 , wherein the transmitter is further configured to transmit an indication of the obtained compression factor to the other WTRU.

3. The WTRU of claim 1 , wherein the one or more processors are configured to determine which nodes in the neural network should be connected when the compression factor is adjusted.

4. 2. The WTRU of claim 1, wherein at least one of the splitting point and the compression factor is configured based on one or more of: (1) physical layer operation; (2) medium access control layer operation; (3) radio resource control layer operation; (4) available processing resources; and (5) control signaling.

5. The WTRU of claim 1 , wherein at least one of the split point and the compression factor is configured based on a transmission data rate.

6. 1. A method performed by a wireless transmit / receive unit (WTRU), comprising: receiving a portion of a deep neural network (DNN) model, the portion being prior to a division point of the DNN model, the portion of the DNN model including a neural network for compressing features at the division point of the DNN model; obtaining a compression factor for said neural network; determining which nodes in the neural network should be connected in response to the compression factor; configuring the neural network in response to said determining; performing inference using the portion of the DNN model to generate condensed features; and transmitting the compressed features to another WTRU; and The method includes:

7. The method of claim 6 , further comprising: transmitting an indication of the obtained compression factor to the other WTRU.

8. 7. The method of claim 6, wherein which nodes in the neural network should be connected are determined when the compression factor is adjusted.

9. The method of claim 6 , wherein only one DNN model for different compression factors is loaded into the WTRU.

10. 7. The method of claim 6, wherein at least one of the splitting point and the compression factor are configured based on one or more of: (1) physical layer operations; (2) medium access control layer operations; (3) radio resource control layer operations; (4) available processing resources; and (5) control signaling.

11. 1. A wireless transmit / receive unit (WTRU), comprising: a receiver configured to receive a portion of a deep neural network (DNN) model, the portion being after a division point of the DNN model, the portion of the DNN model including a neural network for expanding features at the division point of the DNN model, the receiver also configured to receive one or more features output from another WTRU; One or more processors, obtaining a compression factor for said neural network; determining which nodes in the neural network should be connected in response to the compression factor; configuring the neural network in response to said determining; performing inference using the portion of the DNN model using the one or more features output from another WTRU as input to the neural network; and one or more processors configured to execute A WTRU equipped with

12. The WTRU of claim 11 , wherein the receiver is further configured to receive a signal indicative of the compression factor.

13. The WTRU of claim 11 , wherein the one or more processors are configured to determine which nodes in the neural network should be connected when the compression factor is adjusted.

14. The WTRU of claim 11 , wherein only one DNN model for different compression factors is loaded into the WTRU.

15. 12. The WTRU of claim 11, wherein at least one of the splitting point and the compression factor is configured based on one or more of: (1) physical layer operation; (2) medium access control layer operation; (3) radio resource control layer operation; (4) available processing resources; and (5) control signaling.

16. 1. A method comprising: receiving a portion of a deep neural network (DNN) model, the portion being after a split point of the DNN model, the portion of the DNN model including a neural network for expanding features at the split point of the DNN model; receiving one or more characteristics output from the WTRU; obtaining a compression factor for said neural network; determining which nodes in the neural network should be connected in response to the compression factor; configuring the neural network in response to said determining; performing inference using the portion of the DNN model using the one or more features output from the WTRU as input to the neural network; The method includes:

17. The method of claim 16 , further comprising receiving a signal indicative of the compression factor.

18. 17. The method of claim 16, wherein which nodes in the neural network should be connected are determined when the compression factor is adjusted.

19. The method of claim 16 , wherein only one DNN model for different compression factors is loaded into the WTRU.

20. 17. The method of claim 16, wherein at least one of the splitting point and the compression factor are configured based on one or more of: (1) physical layer operations; (2) medium access control layer operations; (3) radio resource control layer operations; (4) available processing resources; and (5) control signaling.

Images