Partitioned Neural Network Computing

Abstract

The client device (104) and one or more such networked computing devices (112) coordinate to implement a client-initiated and device-initiated split-neural network computation scheme in which the neural network architecture of the neural network (116) is distributed, or "split," across the client device and one or more networked computing devices, whereby each device operates to perform at least one successive split-neural network portion (122, 124) of an overall neural network operation using input data (130), the intermediate result data (136) generated therefrom being subsequently communicated to the next device in sequence until a final result output (140) is generated by the final split-neural network portion of the sequence. The device generating the final result output can then take one or more actions based on the final result output or can forward the final result output to the device that initiated the split-neural network computation operation for further processing.

Description

[Background technology]

[0001] User equipment, wearable devices, and other client devices in cellular and other wireless networks are increasingly employing neural networks to perform specific operations at various protocol layers to improve performance efficiency or enhance user experience. For example, deep neural networks (DNNs) may be used in client devices to provide image analysis, transmitted environment awareness, radio frequency signaling, and the like. However, client devices are often resource-constrained, making it impractical to implement sufficiently complex neural networks to provide desired results. For example, user devices may have insufficient battery power, insufficient computational resources, or insufficient network capacity to perform DNN operations within the allotted time frame or with the intended complexity.

[0002] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings, in which: The use of the same reference symbols in different drawings indicates that items are similar or identical. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 illustrates an exemplary wireless system that uses a split neural network computation scheme to divide a neural network computational load between two or more devices, according to some embodiments. [Figure 2] FIG. 1 illustrates an example of a partitioned neural network configuration using a client-first implementation, according to some embodiments. [Figure 3] FIG. 10 illustrates another example of a partitioned neural network configuration using an interleaved configuration, according to some embodiments. [Figure 4]2 illustrates an exemplary hardware configuration of user equipment of the wireless system of FIG. 1, according to some embodiments. [Figure 5] 2 illustrates an exemplary hardware configuration of a server of the wireless system of FIG. 1, according to some embodiments. [Figure 6] FIG. 1 illustrates a machine learning (ML) module used in the split neural network portion, according to some embodiments. [Figure 7] FIG. 1 is a flow diagram illustrating an example method for partitioned neural network computing in a wireless system, according to some embodiments. [Figure 8] FIG. 1 is a flow diagram illustrating an example of a process for negotiating a split neural network configuration between at least two devices, according to some embodiments. [Figure 9] FIG. 1 is a flow diagram illustrating an example for performing machine learning operations using a neural network split across two or more devices, according to some embodiments. [Figure 10] 10 is a ladder signaling diagram illustrating an example operation of the method of FIGS. 7-9, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0004] Due to various resource constraints, client devices often avoid implementing complex DNNs or other neural networks. However, the networks to which they are connected often access servers or other computing devices that have significantly fewer resource constraints. Taking advantage of this, in at least one embodiment, the client device and one or more such networked computing devices are arranged to implement a split-neural network computation scheme in which the neural network architecture of a neural network is distributed or "split" across the client device and one or more networked computing devices, such that each device operates to perform at least one successive portion of an overall DNN operation (hereinafter, a "split-neural network portion" or simply a "split portion"), with intermediate result data generated therefrom subsequently communicated to the next device in sequence until a final result output is generated by the final split-neural network portion in the sequence. The device generating the final result output can then take one or more actions based on the final result output or can forward the final result output to the device that initiated the split-neural network computation operation for further processing.

[0005] In at least one embodiment, this split-neural network configuration is initiated by the client device. Thus, in response to initiating an operation involving use of the neural network, the client device or one of the networked computing devices determines a relevant set of current local conditions for the client device, such as its available computational resources (e.g., processor and memory), current battery conditions, current network conditions, current thermal conditions, etc. Additionally, in some implementations, there may be execution requirements associated with the operation, such as latency conditions indicating a maximum period allowed for generation of a final result output, quality of service (QoS) conditions indicating a maximum amount of time allowed to complete the operation, or accuracy conditions indicating a minimum level of accuracy or other performance level in the final result output. If the client device determines that it can meet the latency conditions, QoS conditions, accuracy conditions, or other such execution requirements given the client device's current local conditions, the client device may elect to execute the operation entirely locally, i.e., to execute the operation using an entirely local neural network architecture for the neural network.

[0006] However, if one or more specified execution requirements are deemed unlikely to be met by local-only execution, in at least some embodiments, either the client device or the networked computing device, or both, determine a partitioned neural network configuration, in which the neural network architecture of the neural network is allocated or “partitioned” between the client device and the networked computing device (or multiple networked computing devices). The particular partitioning between devices can be based on current local conditions, execution requirements, or a combination thereof. For example, if there is relatively low availability of local computing resources, low battery reserve, and / or a low latency requirement, the partitioned neural network configuration may allocate more layers of the neural network architecture to one or more networked computing devices and fewer layers to the client device; whereas, if there is relatively high availability of local computing resources, high battery reserve, and long latency, the partitioned neural network configuration may allocate more layers of the neural network to itself and fewer layers to one or more networked devices.

[0007] In some embodiments, the initiating device then initiates negotiations with other device(s) intended to be included in the partitioned neural network configuration by sending a partitioned computation request using the proposed partitioned neural network configuration. The other device(s) may then consider their own resource constraints and execution requirement(s) in determining whether to accept the proposed partitioned neural network configuration or counter-propose a different partitioned neural network configuration that is more compatible with their resource constraints or execution requirement(s). Once the client device and one or more networked computational resources settle on an agreed-upon partitioned neural network configuration, each of the client device and one or more networked computational resources implements its corresponding portion of the neural network architecture (i.e., its corresponding “partition”) in a corresponding machine learning (ML) module. Thereafter, the first device in the partitioned neural network configuration sequence receives input data for the operation, processes the input data in its ML module, and provides output as intermediate result data to the device having the next partitioned portion in the sequence of neural network architectures.

[0008] In some embodiments, the split is a two-way split, where one device (e.g., a networked computing device) has the first sequence or other initial portion (e.g., "first stage") of the layers of the neural network architecture, and the other device (e.g., a client device) has the last sequence or other final portion (e.g., "final stage") of the layers of the neural network architecture. In other embodiments, the split is an N-way split (N > 2), and each of the N stages is assigned to a corresponding device. In some embodiments, each stage is assigned to a different device, e.g., the first stage is assigned to one networked computing device, the intermediate stage is assigned to a second networked computing device, and the final stage is assigned to a client device. In other embodiments, the stages may be interleaved among a set of M devices (M < N). For example, the first stage and the final stage may be distributed to client devices, and the intermediate stages may be distributed to networked computing devices.

[0009] With this split-computation approach for performing neural network operations, the client device can utilize the resources of one or more available networked computing devices as appropriate to assist in the execution of neural network operations if the client device determines that its current local conditions may prevent local-only resolution from achieving one or more conditions or goals.

[0010] For ease of explanation, the systems and techniques are described herein in the exemplary context of a cellular network in which user equipment (UE) operates as a client device and one or more servers operate together as networked computing devices. However, it will be understood that these references are for illustrative purposes only, and that references to a UE, a server, or a cellular network also apply equally to other client devices, other networked computing devices, and other networks, respectively, unless otherwise noted.

[0011] Additionally, for ease of explanation and understanding, the techniques of this disclosure are described in exemplary implementations in which a UE or other client device is the initiator of neural network splitting via a split neural network configuration and, therefore, is the device that triggers the negotiation process. However, in other embodiments, it may be the networked computing device that initiates neural network splitting between the networked computing device and the client device (and, in some examples, between one or more additional networked computing devices). As such, it will be understood that references and descriptions to initiation or negotiation of a split neural network configuration by a client device instead apply to initiation or negotiation of a split neural network configuration by a networked computing device using the guidelines provided herein.

[0012] 1 illustrates an exemplary wireless communication network 100 employing a split-neural network computation scheme, according to some embodiments. In the illustrated example, the wireless communication network 100 is a cellular network including a network infrastructure 102 wirelessly connected to one or more client devices, such as a UE 104. The network infrastructure 102 includes a core network 106 coupled to one or more wide area networks (WANs) 108 or other packet data networks (PDNs), such as the Internet. The core network 106 is further connected to at least one base station (BS) 110. The BS 110 supports wireless communication with one or more wireless client devices, such as the UE 104, via radio frequency (RF) signaling using one or more applicable RATs defined by one or more communication protocols or standards. Thus, the BS 110 acts as a wireless interface between one or more wireless devices and various networks and services provided by the network infrastructure 102, such as packet-switched (PS) data services, circuit-switched (CS) services, etc.

[0013] The BS 110 may employ any of a variety of RATs, such as operating as a NodeB (or base transceiver station (BTS)) for the Universal Mobile Telecommunications System (UMTS) RAT (also known as “3G”), operating as an enhanced NodeB (eNodeB) for the Third Generation Partnership Project (3GPP®) Long Term Evolution (LTE) RAT, or operating as a 5G NodeB (“gNB”) for the 3GPP Fifth Generation (5G) New Radio (NR) RAT. The UE 104, in turn, represents any of a variety of client devices operable to communicate with the BS 110 via the appropriate RAT, including, for example, a mobile phone, a tablet or laptop computer, a desktop computer, a video game system, a server, a network-enabled consumer electronics appliance, a network-enabled automotive communication system, a network-enabled smart watch, or other wearable device. The core network 106 and / or WAN 108 include or have network access to one or more networked computing devices 112, such as one or more servers (or server farms), one or more workstations, etc., that provide computing resources to the core network, the WAN 108, and / or the UEs 104. In the examples described herein, the one or more networked computing devices 112 include one or more servers, and therefore the networked computing devices 112 are also referred to herein as “servers 112” for ease of reference.

[0014] In at least one embodiment, the UE 104 is configured to implement a machine learning (ML) module (see FIG. 6 ), which may be configured to implement one or more neural networks (or portions thereof) to provide neural network / machine learning capabilities to one or more software applications executing on the UE 104. Examples of such neural networks include deep neural networks (DNNs) such as convolutional neural networks (CNNs), artificial neural networks (ANNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), etc. For example, a software application on the UE 104 may use an ML module that implements a CNN to provide image detection / categorization or an ML module that implements an RNN to provide voice detection. However, depending on the complexity or computational resources required to implement the entire neural network, the UE 104 may have insufficient resources to locally perform neural network operations in a timely manner. For example, in the case of real-time voice detection and translation, the UE 104 may have insufficient computational resources to perform the voice detection and translation operations quickly enough to facilitate a satisfactory user experience. Alternatively or additionally, performing neural network operations entirely on the UE 104 may impose an unreasonable or unacceptable load on the UE 104. By way of example, performing a series of image classification operations using a DNN on the UE 104 may excessively drain the battery on the UE 104 or may induce a thermal load on the UE 104 that is difficult to manage.

[0015] Thus, in at least one embodiment, the UE 104 and the network infrastructure 102 together can implement a partitioned neural network computation scheme, in which a partitioned neural network configuration 114 is used to implement a neural network 116 that is distributed, or “partitioned,” across the UE 104 and one or more servers 112 or other networked computing devices of the network 100, whereby each device in the partitioned neural network configuration 114 operates to perform a portion of the overall neural network operation (i.e., a “partitioned neural network portion” or “partitioned portion”), the intermediate results generated therefrom being subsequently communicated to the next device in sequence until a final result output is generated by the last part in the sequence. The part assigned to a corresponding device can include any subset or partition of the elements of the neural network used by the partitioned neural network configuration 114. By way of example, many neural networks are organized as a sequence of layers, including an input layer, one or more hidden layers, and an output layer, each layer having one or more nodes (e.g., neurons and / or perceptrons). In such cases, the split neural network portions may include all of the nodes in one or more layers of the neural network, a portion of the nodes in one or more layers of the neural network, or a combination thereof. In certain neural networks, the layers / nodes are further organized, and the splitting of the neural network into corresponding split neural network portions may occur according to such organization. For example, a GAN is typically composed of a generator and a discriminator, and the splitting of the GAN across multiple devices may be done with this organization in mind, such as with layers and nodes associated with the generator assigned to one or more devices and layers and nodes associated with the discriminator assigned to one or more other devices.

[0016] In an embodiment, neural network partitioning is performed such that the partitioned neural network configuration 114 is implemented as a sequence of partitioned neural network portions representing the entire neural network, whereby each partitioned neural network portion receives as its input either an initial input or an output from a previous partitioned neural network portion in the sequence and provides its output as either an input to the next partitioned neural network portion in the sequence or as a final result output, depending on the partitioned neural network portion's position in the sequence. In a two-way partitioning, a first device may be assigned, for example, an input layer and one or more hidden layers following the input layer, while a second device may be assigned, for example, the remaining hidden layers and output layer. Which partitions are assigned to which devices may depend on various factors, such as the availability of comparative computational resources, the type of neural network operation being performed, and output usage. For example, in the case of an image classification operation with image classification results used by an application on the UE 104, the server 112 may be assigned an initial partition having most of the input layer and hidden layer, while the UE 104 may be assigned a final partition having the remaining hidden layer and output layer. In an N-way partition (N>2), each of M devices (M>1) is assigned a partition, and for N>M, at least one device is assigned multiple partitions.

[0017] As described below, in some embodiments, the UE 104 operates to initiate neural network partitioning for performing neural network operations and determines how to perform the partitioning either unilaterally or as a result of negotiation with the network infrastructure 102. The UE 104's decision to partition neural network operations between the UE 104 and one or more servers 112 (or other networked computing devices) can be based on a variety of factors, including the scope or extent of the operations to be performed, the UE 104's local resources available for performing the neural network operations, one or more execution requirements (e.g., latency, QoS, accuracy thresholds), network bandwidth or other network parameters, etc. As a general guideline for some implementations, the portion of the neural network that the UE 104 attempts to offload to one or more servers 112 is inversely proportional to the UE 104's available local resources and / or the stringency of the execution requirement(s), all else being equal. Note, however, that in other embodiments, the server 112 can instead initiate neural network partitioning using techniques similar to those described below.

[0018] When the UE 104 proposes / decides on this particular partitioning, the UE 104 can notify the server 112 of the assigned partitioned neural network portion(s) using any of a variety of techniques. In some embodiments, the server 112 (and the UE 104) can access a repository 118 containing representations 120 of neural networks, and the UE 104 can notify the server 112 of those partitioned neural network portion(s) assigned to the server 112, along with other relevant details such as initial weights, how the partitioned neural network portions are implemented at the server 112, etc. For example, each representation 120 may be identified by a corresponding identifier, which is subsequently referenced by the UE 104 and the server 112. Representation 120 may reference a particular implementation of a type of neural network (e.g., an RNN with a particular number of layers, nodes, weights, etc.), and UE 104 and server 112 can utilize additional information to implement the partitioning of this particular implementation into the determined partitioned portions, or representation 120 may represent a predetermined partitioning configuration template, such as an RNN with a particular number of layers, nodes, weights, etc., as well as a predetermined partitioning configuration template of these elements among multiple partitioned neural network portions. While shown as a separate component (e.g., a separate server within network infrastructure 102), in some embodiments, repository 118 may be implemented, in whole or in part, on server 112. In other embodiments, UE 104 may provide information to server 112 specifying how server 112 should implement the corresponding partitioned neural network portion, e.g., one or more data structures representing the structure of the partitioned neural network portion, its input format, its output format, weights or other parameters of different nodes within the structure, etc.

[0019] 1 shows an exemplary partitioned neural network configuration 114 of neural network 116, in which neural network 116 is partitioned into two portions, partitioned neural network portions 122 and 124, with partitioned neural network portion 122 assigned the input layer and the first five hidden layers of exemplary neural network 116 (and thus referred to as the "initial partitioned neural network portion 122") and partitioned neural network portion 124 assigned the last two hidden layers and the output layer of exemplary neural network 116 (and thus referred to as the "final partitioned neural network portion 124"). This configuration partitions neural network 116 into the sequence initial neural network portion 122 → final neural network portion 124. In this example, it is assumed that the UE 104 negotiates this partitioning with the server 112 based on the current local resources available to the UE 104, the execution requirements(s) of the operation to be performed (e.g., ML-assisted image classification, speech detection, etc.), and uses the techniques described below as a result of the negotiation with the server 112. Further, in this embodiment, the initial partitioned neural network portion 122 is assigned to the server 112, and the final partitioned neural network portion 124 is assigned to the UE 104. Thus, the initial partitioned neural network portion 122 is implemented in the server ML module 126 at the server 112, and the final partitioned neural network portion 124 is implemented in the UE 104 in the UE ML module 128.

[0020] To initiate an ML-assisted operation using this exemplary configuration, a software application on the UE 104 generates input data 130-1 for input to the neural network 116. For example, if the ML-assisted operation is image classification, the input data 130-1 may include, for example, images captured by a camera on the UE 104. As another example, if the ML-assisted operation is voice detection and translation, the input data 130-1 may include, for example, voice captured and digitized by a microphone on the UE 104. A radio frequency (RF) interface 132 on the UE 104 transmits the input data (as input data 130-2) to a corresponding RF interface 134 on the BS 110, which then transmits the input data (as input data 130-3) to the server 112. At the server 112, the input data 130-3 is provided as input to the server ML module 126, which processes the input data 130-3 at the input layer and the first five hidden layers of the neural network 116 according to its assigned initial partitioned neural network portion 122. Thus, the output 136 of the node in the fifth hidden layer serves as the intermediate result data 138-1 of the initial partitioned neural network portion 122 (and therefore the output of the server ML module 126). Thus, the server 112 transmits the intermediate result data (as intermediate result data 138-2) to the BS 110, which then wirelessly transmits the intermediate result data 138-2 to the UE 104.

[0021] The UE 104 provides the intermediate result data (as intermediate result data 138-3) as input to the UE ML module 128, which implements the final partitioned neural network portion 124. Accordingly, the output 136, represented by the intermediate result data 138-3, is provided as input to corresponding nodes in the sixth hidden layer of the neural network 116 and processed by the sixth, seventh, and eighth hidden layers and the output layer to generate a final result output 140 that represents the final result that the neural network 116 would output given the intermediate result data 138-3 as input. The software application of the UE 104 that initiated the ML-assisted operation and / or other software applications on the UE 104 may then perform one or more actions in response to the final result output 140. For example, if the ML-assisted operation is a speech detection and translation operation, the final result output 140 may represent a translated representation of the input speech, and thus the software application may operate the UE 104 to output an audio version of the translated representation via a speaker of the UE 104 and / or display a text representation via a display of the UE 104.

[0022] While Figure 1 illustrates a particular two-way partitioning of neural network 116, assigning an initial portion to server 112 and a final portion to UE 104, the partitioning of any given neural network can be performed in a variety of ways depending on the needs and context. For example, UE 104 can implement the initial portion of the partitioned neural network configuration, while server 112 implements the final portion. Figure 2 illustrates this example, with a partitioned neural network configuration 214 partitioning neural network 216 into an initial partitioned portion 202 implemented in UE ML module 228 of UE 104 and a final partitioned portion 204 implemented in server ML module 226 of server 112. In this approach, a software application on UE 104 provides input data 230 as input to UE ML module 228, which causes a first subset of the initial and hidden layers of neural network 216 to process the input data 230. An output 236 of the last hidden layer in the initial partition 202 is provided as an intermediate result output 238 that is transmitted to the server 112 via the base station 110. The server 112 provides the intermediate result output 238 as an input to the first hidden layer in the final partition 204, whereby this hidden layer, subsequent hidden layers, and an output layer process the intermediate result output 238 to generate a final result output 240. The final result output 240 may then be sent back to the UE 104 for use by one or more software applications at the UE and / or the final result output 240 may be used by the server 112, or some other component of the network infrastructure 102, to perform one or more remote actions 242. For example, the neural network 116 may be used to generate augmented display information for an augmented reality (AR) display; thus, the one or more remote actions 242 may be displaying the generated augmented display information on the AR display (one result of the final result output 240).As another example, the neural network 116 may be used to generate virtual reality (VR) content, such as VR display content or VR audio content (an example of a final result output 240), and the one or more remote actions 242 may be to display or output the generated VR content. As yet another example, the neural network 116 may be used to improve or enhance cellular connectivity between the UE 104 and the BS 110 (e.g., by controlling some aspects of the UE's RF antenna array based on input sensor data). Thus, the one or more remote actions 242 may include, for example, modifying the operation of the RF antenna array based on the final result output 240.

[0023] Further, as mentioned above, in some implementations, the partitioning configuration may be an N-way partition (N>2), with each of the M devices obtaining at least one partitioned neural network configuration. Figure 3 illustrates such an example, where the partitioned neural network configuration 314 partitions the neural network 316 into three partitions: an initial partition 302 implemented in the UE ML module 328 of the UE 104, an intermediate partition 304 implemented in the server ML module 326 of the server 112, and a final partition 306 implemented in the UE ML module 328 of the UE 104 (or a second instance of the UE ML module of the UE 104). In this example, a software application on the UE 104 provides input data 330 as input to the UE ML module 228 for processing by the initial partition 302. The output 336-1 of the last hidden layer in the initial partition 302 is provided as a first intermediate result output 338-1 transmitted to the server 112 via the base station 110. The server 112 provides a first intermediate result output 338-1 as input to a first hidden layer in the intermediate partitioning portion 304, whereby this hidden layer and subsequent hidden layers of the intermediate partitioning portion 304 process the first intermediate result output 338-1. An output 336-2 of the last hidden layer in the intermediate partitioning portion 304 is provided as a second intermediate result output 338-2 that is transmitted to the UE 104 via the base station 110. The UE 104 then provides the second intermediate result output 338-2 as input to a first hidden layer in the final partitioning portion 306, whereby this hidden layer, subsequent hidden layers, and the output layer in the final partitioning portion 306 process this input to generate a final result output 340. The final result output 340 may then be processed by one or more software applications at the UE 104, whereby one or more actions are taken at the UE 104 in response.

[0024] 4 illustrates an exemplary hardware configuration of a UE 104 (as a representative client device) according to some embodiments. Note that the illustrated hardware configuration represents the processing and communication components most directly related to the neural network-based processes described herein, and omits certain components that are well known to be frequently implemented in such electronic devices, such as displays, peripherals, external power supplies, etc.

[0025] In the depicted configuration, the UE 104 includes an RF interface 402 having one or more antennas 404 and one or more modems supporting one or more radio access technologies (RATs), such as a 3rd Generation Partnership Project (3GPP) Fourth Generation Long Term Evolution (4G LTE)-compliant RAT, a 3GPP Fifth Generation New Radio (5G NR)-compliant RAT, an Institute of Electrical and Electronics Engineers (IEEE) 802.11-compliant RAT, etc. The UE 104 further includes one or more processors 406, a set of sensors 408, a user interface (UI) 410, and one or more batteries 412 or other power sources. The one or more processors 406 may include, for example, one or more central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), or other application-specific integrated circuits (ASICs), etc. By way of example, the processor 406 may include an application processor (AP) utilized by the UE 104 to execute an operating system and various user-level software applications, as well as one or more processors utilized by a baseband processor of the modem or RF interface 402. The set of sensors 408 may include, for example, satellite positioning sensors such as global positioning system (GPS) sensors, global navigation satellite system (GNSS) sensors, internal measurement unit (IMU) sensors, visual odor sensors, accelerometers, gyroscopes, barometers, altimeters, tilt or other inclinometers, ultra-wideband (UWB)-based sensors, etc. Other examples of types of sensors in the sensor set 408 may include sensors for determining the current operating status of the UE 104, such as battery level sensors, temperature sensors, screen mode sensors, etc. The UI 410 includes components for interfacing with a user, such as a display, a keyboard or touch panel, buttons, a microphone, a speaker, etc.

[0026] The UE 104 further includes one or more computer-readable media 414, which may include any of a variety of media used by electronic devices to store data and / or executable instructions, such as random access memory (RAM), read-only memory (ROM), cache, flash memory, solid-state drive (SSD), or other mass storage device. For ease of explanation and brevity, the computer-readable media 414 is referred to herein as “memory 414” in recognition of the frequent use of system memory or other memory to store data and instructions for execution by the processor(s) 406, although references to “memory 414” will be understood to apply equally to other types of storage media unless otherwise noted. The one or more memories 414 of the UE 104 are used to store one or more sets of executable software instructions and associated data to operate the one or more processors 406 and other components of the UE 104 to perform the various functions described herein and attributed to the UE 104. The set of executable software instructions includes, for example, an operating system (OS) 416 and various drivers (not shown), as well as one or more user-level software applications 418, which operate or otherwise interact with the hardware of the UE 104 via the OS 416.

[0027] The set of executable software instructions further includes one or more of a neural network management module 420 and a partitioned computation management module 422. The neural network management module 420 implements one or more ML modules (e.g., ML module 128, FIG. 1 ) that utilize one or more neural networks (or partitioned portions thereof) in the UE 104, as described in detail below. The partitioned computation management module 422 operates to monitor the status of local resources in the UE 104, such as battery status, available memory status, available processor resources, network status, etc., and determines, based on these local resource statuses (and / or other considerations), whether to implement a neural network as a local-only neural network (i.e., entirely in the UE 104) or as a partitioned neural network configuration to perform one or more operations using the neural network. Additionally, if a split neural network configuration is adopted, the split computation management module 422 may operate to manage negotiations regarding implementation of the split portions with one or more servers 112 of the network infrastructure 102, as well as coordinate with the neural network management module 420 to implement the split portion(s) assigned to the UE 104 in the negotiated split neural network configuration.

[0028] To facilitate the operations of the UE 104 described herein, the one or more memories 414 of the UE 104 may further store data related to these operations. This data may include, for example, one or more neural network architecture configurations 424, as well as device data (not shown) including, for example, user data, multimedia data, beamforming codebooks, software application configuration information, etc. Each neural network architecture configuration 424 includes one or more data structures containing data and other information representing the corresponding architecture and / or parameter configuration used by the neural network management module 420 to form the corresponding neural network of the UE 104, or a portion thereof. The information included in the neural network architecture configuration 424 may include, for example, parameters specifying a fully connected layer neural network architecture, a convolutional layer neural network architecture, a recurrent neural network layer, the number of connected hidden neural network layers, the input layer architecture, the output layer architecture, the number of nodes used by the neural network, the coefficients (e.g., weights and biases) used by the neural network, kernel parameters, the number of filters used by the neural network, the stride / pooling configuration used by the neural network, the activation function for each neural network layer, the interconnections between neural network layers, which neural network layers to skip, etc. Thus, the neural network architecture configuration 424 may include any combination of neural network forming components (e.g., architectures and / or parameter configurations) that can be used to create a neural network architecture configuration (e.g., a combination of one or more neural network forming components) that defines and / or forms a DNN or other neural network.

[0029] 5 illustrates an exemplary hardware configuration of a server 112 (as a representative networked computing resource) according to some embodiments. It should be noted that the illustrated hardware configuration represents the processing and communication components most directly associated with the neural network-based processes of the embodiments described herein, omitting certain components that are well understood to be frequently implemented in such electronic devices. Furthermore, it should be noted that while the illustrated diagram depicts an implementation as a single server, functionality, and therefore hardware components, may instead be distributed across multiple servers or other networked computing devices in a manner that performs the functions described herein. Accordingly, references to functionality of a single server 112 may also apply to equivalent functionality across multiple servers 112 or other networked computing devices, unless otherwise noted.

[0030] In the illustrated configuration, the server 112 includes a network interface 502 for wired and / or wireless connectivity to the BS 110 and other components of the network infrastructure 102. The server 112 may further include one or more processors 506, such as, for example, one or more CPUs, GPUs, TPUs, or other ASICs. The server 112 also includes one or more computer-readable media 508, which may include any of a variety of media used by electronic devices to store data and / or executable instructions, such as RAM, ROM, cache, flash memory, SSD, or other mass storage device. Like the memory 414 of the UE 104, for ease of explanation and brevity, the computer-readable media 508 will be referred to herein as “memory 508” in light of the frequent use of system memory or other memory to store data and instructions for execution by the processor 506, although references to “memory 508” will be understood to apply equally to other types of storage media unless otherwise noted.

[0031] The one or more memories 508 of the server 112 are used to store one or more sets of executable software instructions and associated data that operate the one or more processors 506 and other components of the server 112 to perform the various functions described herein and attributed to the server 112, individually or collectively. The sets of executable software instructions include, for example, an OS 510 and various drivers (not shown), as well as various software applications. The sets of executable software instructions further include one or more of a neural network management module 512 and a partitioned computation management module 514. Similar to the neural network management module 420 and the partitioned computation management module 422 of the UE 104, these modules 512, 514 operate to implement a neural network or partition(s) thereof at the server 112. Specifically, the neural network management module 512 implements one or more ML modules (e.g., the ML module 126, FIG. 1 ) that utilize one or more neural networks (or partitions thereof) for the server 112, while the partition computation management module 514 operates to negotiate with the UE 104 regarding the implementation of the partitions at the server 112, as well as to coordinate with the neural network management module 512 to implement the partition(s) assigned to the server 112 in the negotiated partition neural network configuration.

[0032] The one or more memories 508 further store various information, such as one or more neural network architecture configurations 524 that represent trained neural network architecture configurations that can be used by an ML module (e.g., ML module 126, FIG. 1) of server 112. Thus, similar to neural network architecture configurations 424 of FIG. 4, each neural network architecture configuration 524 includes one or more data structures that include data and other information that represent a corresponding architecture and / or parameter configuration that is used by neural network management module 512 of server 112 to form a corresponding portion(s) or the entire neural network.

[0033] 6 illustrates an example machine learning (ML) module 600 for implementing a neural network or partition(s) thereof, according to some embodiments. As described herein, both the UE 104 and the server 112 each implement one or more ML modules to implement one or more partitioned neural network portions. Thus, the ML module 600 illustrates an example module for implementing one or more of these partitioned neural network portions (or the entire neural network).

[0034] In the illustrated example, the ML module 600 implements at least a portion 602 of a deep neural network (DNN) model having a group of connected nodes (e.g., neurons and / or perceptrons) organized into one or more layers. The nodes between layers can be configured in various ways, such as a partially connected configuration in which a first subset of nodes in a first layer are connected to a second subset of nodes in a second layer, or a fully connected configuration in which each node in the first layer is connected to each node in the second layer. Neurons process input data and generate continuous output values, such as any real number between 0 and 1. In some cases, the output value indicates how close the input data is to a desired category. Perceptrons perform linear classification, such as binary classification, on the input data. Nodes, whether neurons or perceptrons, can use various algorithms to generate output information based on adaptive learning. Using portion 602, ML module 600 performs a variety of different types of analysis, including single linear regression, multiple linear regression, logistic regression, stepwise regression, binary classification, multi-class classification, multivariate adaptive regression splines, local estimation scatterplot smoothing, and the like.

[0035] When portion 602 includes an entire DNN model, the two or more layers include an input layer, one or more hidden layers, and an output layer. When portion 602 includes a partitioned portion of a DNN model, the layers included in portion 602 depend on the position of the partitioned portion in the sequence of partitioned portions that together make up the DNN model. If portion 602 is an initial partitioned portion, portion 602 includes an initial layer and one or more adjacent hidden layers that immediately follow the initial layer. If portion 602 is a final partitioned portion, portion 602 includes an output layer and one or more adjacent hidden layers that immediately precede the output layer. If portion 602 is an intermediate partitioned portion, portion 602 includes one or more adjacent hidden layers.

[0036] For ease of explanation, the illustrated portion 602 of the DNN model includes three layers 604, 606, and 608, which may be implemented, for example, as an input layer and two adjacent hidden layers following the input layer in an initial partitioning portion implementation, or as three adjacent hidden layers in an intermediate partitioning portion implementation, or as an output layer and two adjacent hidden layers preceding the output layer in a final partitioning portion implementation. However, in actual implementations, the partitioning portion implemented by the ML module 600 typically has more layers. Each layer may have any number of nodes, and the number of nodes between layers may be the same or different. That is, an input layer may have the same and / or different number of nodes as an output layer, an output layer may have the same and / or different number of nodes as one or more hidden layers, etc. For example, node 610 corresponds to one of several nodes included in an input layer (which may be represented by layer 604), and the nodes perform separate and independent computations. As further described, the nodes receive input data and process the input data using one or more algorithms to generate output data. Typically, algorithms include weights and / or coefficients that change based on adaptive learning. Thus, the weights and / or coefficients reflect information learned by the neural network. Each node may, in some cases, determine whether to pass processed input data to one or more subsequent nodes. For example, after processing input data, node 610 may determine whether to pass the processed input data to one or both of nodes 612 and 614 in an adjacent hidden layer (layer 608 in this example). Alternatively or additionally, node 610 passes the processed input data to nodes based on the layer connection architecture. This process may be repeated across multiple layers until portion 602 generates output using a node (e.g., node 616) in the final layer (layer 608) of portion 602. To facilitate the input of received data and the output of generated output data, ML module 600 may further include an input interface 618 and an output interface 620.The input interface 618 operates to receive input data 622 and distribute individual data included in the input data 622 to corresponding nodes in the first layer 604 according to some predetermined format. Similarly, the output interface 620 operates to receive individual outputs of the nodes in the final layer 608 and provide them as output data 624 according to some predetermined format. The formats used for the input interface 618 and the output interface 620 depend on the neural network used and the layer of the neural network implemented in the portion 602. For example, in the case of an initial split portion, the input data 622 represents initial data input for processing by the neural network, and therefore the input interface 618 operates to distribute the initial data to the nodes in the input layer (layer 604) accordingly. However, for intermediate or final split portions, the input data 622 represents outputs from upstream hidden layers, and thus the input data 622 represents intermediate result data, and the input interface 618 is configured to distribute individual data of the intermediate result data as inputs to nodes of the first hidden layer (layer 604) according to the neural network architecture. Similarly, for the initial or intermediate split portions, the output data 624 is individual outputs of nodes of the last hidden layer (layer 608) in the ML module 600, and thus represents intermediate result data, while for the final split portion, the output data 624 is the final result output provided by the output layer (layer 608) of the neural network.

[0037] Neural networks can also use various architectures that determine which nodes in the neural network are connected, how data is advanced and / or retained within the neural network, what weights and coefficients are used to process input data, how data is processed, and so forth. These various elements collectively describe a neural network architectural configuration, such as the neural network architectural configuration briefly described above. By way of example, a recurrent neural network, such as a long short-term memory (LSTM) neural network, forms cycles between node connections to retain information from previous portions of an input data sequence. A recurrent neural network then uses the retained information for subsequent portions of the input data sequence. As another example, a feedforward neural network forwards information down the connections rather than forming cycles that retain information. While described in the context of node connections, it should be understood that a neural network architectural configuration can include various parameter configurations that affect how portion 602 or another neural network processes input data.

[0038] The neural network architecture configuration of a neural network can be characterized by various architecture and / or parameter configurations. To illustrate, consider an example in which portion 602 implements a portion or the entirety of a CNN. In general, a convolutional neural network corresponds to a type of DNN in which layers process data using convolution operations to filter input data. Thus, the CNN architecture configuration can be characterized by, for example, pooling parameter(s), kernel parameter(s), weights, and / or layer parameter(s).

[0039] Pooling parameters correspond to parameters that specify a pooling layer within a convolutional neural network that reduces the dimensionality of input data. For example, a pooling layer can connect the outputs of nodes in a first layer to the inputs of nodes in a second layer. Alternatively or additionally, pooling parameters specify how and where within a layer of data processing the neural network pools data. For example, a pooling parameter indicating "max pooling" configures the neural network to pool by selecting the maximum value from a group of data generated by nodes in the first layer, and uses that maximum value as input to a single node in the second layer. A pooling parameter indicating "average pooling" configures the neural network to generate an average value from a group of data generated by nodes in the first layer, and uses that average value as input to a single node in the second layer.

[0040] Kernel parameters indicate the filter size (e.g., width and height) used to process the input data. Alternatively or additionally, kernel parameters specify the type of kernel method used to filter and process the input data. Support vector machines, for example, correspond to kernel methods that identify and / or classify data using regression analysis. Other types of kernel methods include Gaussian processes, canonical correlation analysis, and spectral clustering methods. Thus, kernel parameters can indicate the filter size and / or type of kernel method to apply to the neural network. Weight parameters specify the weights and biases used by the algorithm in the node to classify the input data. In some implementations, the weights and biases are learned parameter configurations, such as parameter configurations generated from training data. Layer parameters specify layer connections and / or layer types, such as a fully connected layer type indicating that all nodes in the first layer (e.g., output layer 608) are connected to all nodes in the second layer (e.g., hidden layer 606), a partially connected layer type indicating which nodes in the first layer are disconnected from the second layer, and an activation layer type indicating which filters and / or layers to activate within the neural network. Alternatively or additionally, the layer parameters specify the type of the node layer, such as a normalization layer type, a convolutional layer type, and a pooling layer type.

[0041] Although described in the context of pooling parameters, kernel parameters, weight parameters, and layer parameters, it will be recognized that other parameter settings can be used to form a DNN consistent with the guidelines provided herein. Thus, a neural network architecture configuration can include any suitable type of configuration parameters applicable to a DNN that affect how the DNN processes input data and generates output data.

[0042] 7-10 , a method 700 for implementing a split-neural network configuration of a neural network used to perform ML operations on behalf of a software application or other process of a UE or other client device is shown in accordance with some embodiments. For illustrative purposes, method 700 and accompanying examples are described in the illustrative context of a wireless communications network 100 with a UE 104 operating as a client device and a server 112 representing one or more collaborative networked computing devices, although the method is not limited to the context of this particular implementation. Additionally, the ladder diagram 1000 of FIG. 10 is referenced below in-line with the description of method 700 of FIGS. 7-9 as an example of the operation of method 700 for an exemplary two-way split-neural network configuration to facilitate understanding. Furthermore, it should be noted that the order of operations described with reference to method 700 is for illustrative purposes only, and that operations may be performed in a different order, and further, one or more operations may be omitted, or one or more additional operations may be included in the illustrated method.

[0043] As described above, in an embodiment, a software application (e.g., user-level application 418 or OS 416 or other kernel / driver-level process) of UE 104 utilizes a neural network to perform corresponding operations, the results of which are utilized by that software application or other software applications to perform one or more actions. If UE 104 has sufficient resources to implement the neural network entirely locally while achieving corresponding goals (e.g., maximum latency goals or battery consumption goals), UE 104 may implement the entire neural network in an ML module of UE 104 and perform operations using this so-configured ML module. However, if UE 104 cannot implement the neural network entirely locally while meeting one or more specified goals, UE 104 may instead attempt to implement the neural network as a split-neural network configuration that leverages the computational resources of one or more servers 112 of network infrastructure 102.

[0044] Thus, method 700 begins at block 702, where a software application on the UE 104 begins executing an ML operation (block 1002, FIG. 10) that utilizes an identified neural network to generate an output result based on input data provided by or referenced by the software application. Using the example above, this operation may be an image classification operation using a particular CNN, with the input to the CNN being image data representing the image to be classified. Initiation of the ML operation may include, for example, the software application issuing a call to an application programming interface (API) of the OS 416 to support such an operation. Simultaneously, the partitioned computing management module 422 monitors the status of the UE 104's local resources, such as available memory, processor utilization, remaining battery capacity, current device usage, and other indications of current or future local resource availability (block 1004, FIG. 10). Thus, in response to initiating the ML operation, at block 704, the partitioned computing management module 422 determines the current condition of the UE 104 based on these monitoring operations. Additionally, the partitioned computation management module 422 identifies any execution requirements for the ML operations, such as execution latency limits, execution power consumption limits, and computational resource consumption limits.

[0045] Then, at block 706, the partitioned computation management module 422 determines whether a local-only implementation or partitioned configuration of the identified CNN is more suitable for the determined current conditions and any execution requirements. For example, if there are sufficient local power and computational resources available at the UE 104, a relatively long time limit for generating the final result output, and the wireless network bandwidth is constrained, the partitioned computation management module 422 may decide to implement the neural network entirely locally, and thus perform ML operations solely at the UE 104. However, if there are insufficient computational or power resources available at the UE 104, a strong wireless connection with the BS 110, and a relatively short time limit, the partitioned computation management module 422 may decide to implement the neural network as a partitioned configuration in which one or more portions of the neural network are assigned to the UE 104 and one or more other portions of the neural network are assigned to the server 112 (or multiple servers 112). This determination can be made in any of a variety of ways. For example, the determination may be implemented algorithmically (e.g., via a weighted sum equation), via a look-up table (LUT), via other less resource-intensive trained neural networks, etc. Illustratively, the UE 104 may model the performance of a proposed splitting configuration to determine whether it meets UE-side power consumption constraints while also meeting QoS (e.g., latency) requirements. If so, the UE 104 may select the proposed splitting configuration; if not, the UE 104 may select another proposed splitting configuration for similar modeling.

[0046] If the partitioned computation management module 422 determines that a local-only implementation of the neural network is appropriate or sufficient, then in block 708, the partitioned computation management module 422 instructs the neural network management module 420 to implement an ML module configured to implement the entire architectural configuration of the identified neural network, and the UE 104 uses this ML module to perform ML operations.

[0047] Returning to block 706, if the partitioning computation management module 422 determines that a partitioning configuration of the neural network is appropriate (block 1006, FIG. 10), then in block 710, the partitioning computation management module 422 performs a negotiation process to determine a partitioning of the neural network between the UE 104 and the server(s) 112 (or multiple servers 112) that is acceptable to the UE 104 and the server(s) 112 given their respective resource constraints.

[0048] 8, an exemplary implementation of the negotiation process of block 710 is shown according to some embodiments. In this exemplary approach, the negotiation process follows a decision to implement a partition configuration in block 802 using the partition computation management module 422, which determines or identifies a particular neural network architecture to be implemented to perform the ML operations. This includes determining the type of neural network (e.g., CNN, RNN, ANN, GAN, etc.) and parameters of the neural network type, such as the number of layers, the number of nodes in each layer, the operations performed at each node, the weights and other parameters of each node, and the connections between the nodes. Furthermore, while the negotiation process is described below for client-initiated partitioning, recall that in other embodiments, it may be the server 112 that initiates the partitioning, in which case a similar process may be implemented, but with the roles of the server 112 and the UE 104 reversed in terms of initiator and responder.

[0049] Once the neural network architecture is identified, at block 804, the partitioning computation management module 422 then determines a proposed partitioning configuration for the neural network architecture based on current UE conditions, performance requirements, etc. Similar to determining whether to partition the neural network, this proposed partitioning configuration may be determined or selected algorithmically, via a LUT or other selection structure, via a smaller neural network, etc. In some embodiments, multiple predetermined partitioning options are available, and the partitioning computation management module 422 selects one of the pre-determined partitioning options as the proposed partitioning configuration. For example, one partitioning option may be a two-way partition that divides the hidden layers evenly between the server 112 and the UE 104, while another partitioning option may be a two-way partition that allocates a larger proportion of the hidden layers to the server 112 than to the UE 104. Yet another partitioning option may be a three-way partition that allocates initial and final partitions with a small proportion of the hidden layers to the UE 104 and an intermediate partition with a larger proportion of the hidden layers to the server 112. In this example, the UE 104 may select the partitioning option that best suits the current UE conditions while satisfying the indicated performance requirements. In other embodiments, the partitioning computation management module 422 dynamically determines the proposed partitioning, such as by assigning the number of layers to the portions assigned to the server 112 inversely proportional to the available computational resources of the UE 104, using a simple example. Once the proposed partitioning is identified, the UE 104 then transmits a partitioning request 1008 ( FIG. 10 ) to the server 112 via the BS 110. The partitioning request 1008 may include a description or identifier of the proposed partitioning (e.g., a data structure identifying associated parameters sufficient to describe the partitioning portion the UE 104 is proposing for the server 112 to implement, or an identifier to a pre-defined partitioning option available from the repository 118 as the neural network representation 120). The partitioning request 1008 may also include information related to the requested partitioning proposal, such as the current conditions of the UE 104 and / or the performance requirements that resulted in the particular proposed partitioning.Additionally or alternatively, the server 112 may obtain some or all of the UE 104's current conditions from the most recent radio resource control (RRC) UE Capability Information message sent by the UE 104 to the BS 110 in response to an RRC UE Capability Inquiry message from the BS 110.

[0050] In block 806, the partitioning computation management module 514 of the server 112 (or BS 110, or other network component acting as an intermediary for the server 112) evaluates the proposed partitioning represented by the partitioning request 1008 to determine whether to accept the proposed partitioning or counter-propose with a modified proposed partitioning. The partitioning computation management module 514 may consider the server's own current resource constraints and the indicated performance requirements when determining whether to accept the proposed partitioning. For example, if the server 112 can dedicate the resources necessary to implement the partitioned neural network portion assigned to it based on the proposal and can meet the performance requirements indicated in the proposed partitioning, the partitioning computation management module 514 may accept the proposed partitioning and indicate this acceptance in block 808 by transmitting a partitioning computation acceptance message 1010 ( FIG. 10 ) to the UE 104 via the BS 110.

[0051] Returning to block 806, if the partitioning computation management module 514 determines that the server 112 cannot allocate the computational resources necessary to implement the proposed partitioning or that the proposed partitioning does not meet the indicated performance requirements, then at block 810, the partitioning computation management module 514 may determine a partitioning counterproposal that proposes a modified partitioning configuration of the neural network. For example, if the server 112 cannot dedicate sufficient resources to the original proposed partitioning, the partitioning computation management module 514 may counterproposal with a counterproposal partition that allocates fewer layers of the neural network architecture to the server 112. As another example, if the server 112 can allocate sufficient resources but the partitioning computation management module 514 determines that the original proposed partitioning cannot meet latency or other performance requirements, the partitioning computation management module 514 may counterproposal a modified partitioning that is more likely to meet the latency or other performance requirements. The server 112 then transmits a representation of the partitioning counterproposal to the UE 104. At block 812, the partitioning computation management module 422 of the UE 104 evaluates the counterproposal and, if acceptable, at block 814 sends an acceptance message to the server 112, in response to which the server 112 issues a partitioning computation allowance message 1010. If the counterproposal is not acceptable, at block 816, the UE 104 may terminate the negotiation and attempt to perform the ML operations entirely locally (i.e., by implementing the entire neural network in the ML module of the UE 104) or fall back by returning a no-op (NOP) or error message to the initiating software application to indicate that the ML operations cannot be performed under current conditions. Alternatively, in some embodiments, one or more rounds of counter-counterproposals for partitioning may be conducted between the UE 104 and the server 112 until a mutually acceptable partitioning is identified or until a threshold number of counterproposals have been sent.

[0052] Returning to FIG. 7 , with the server 112 and the UE 104 agreeing on the partition configuration proposal, the server 112 and the UE 104 proceed to implement their respective partition(s). However, to do this, the server 112 and the UE 104 need the neural network architecture details (e.g., layers, nodes, connections, weights, and other parameters) of each partition, which will be implemented as a corresponding ML module. In some examples, a negotiation process provides for the distribution of this information. For example, when the negotiation process results in mutual agreement to use a predetermined partition neural network configuration, e.g., from the repository 118, the partition implementation details may be explicitly included in the negotiation messaging, or may be subsequently accessible from the repository 118 by either or both of the server 112 or the UE 104 once the agreed-upon partition neural network configuration has been determined. Alternatively, if the partitioning neural network configuration is determined ad hoc by the UE 104, or if a counter-proposed partitioning from the server 112 is accepted by the UE 104, implementation details of the partitioning portion(s) to be implemented by the server 112 and / or the UE 104 may be included in the negotiation messaging (e.g., in the partitioning request 1008 or grant message 1010, respectively). However, in some embodiments, the negotiation may include negotiation of the overall partitioning, but not specific details such as weights or other parameters of particular nodes or particular layers. In such a case, as represented by block 712, the UE 104 sends one or more configuration messages to the server 112 via the BS 110, which include implementation details of the partitioning portion(s) of the neural network that may be implemented by the server 112 in accordance with the negotiated partitioning configuration (hereinafter, “server-side partitioning portion(s)”).With implementation details in place, in block 714, the UE 104 configures one or more of its ML modules (e.g., UE ML module 128, FIG. 1 ) to implement one or more partitions assigned to the UE 104 (hereinafter, the “UE-side partitions”), and in block 716, the server 112 configures one or more of its ML modules (e.g., server ML module 126, FIG. 1 ) to implement one or more server-side partitions. With the UE 104 and server 112 configured to implement their respective partitioned neural network portions of the agreed-upon partitioned neural network configuration for the neural network, in block 718, the server 112 and the UE 104 operate together to perform partitioned computational ML operations via the neural network distributed between the server 112 and the UE 104.

[0053] FIG. 9 illustrates an exemplary implementation of the partitioned computation ML operation execution process of block 718, according to some embodiments. As described above, a partitioned neural network configuration can include a sequence of two or more partitioned neural network portions, and the initial partitioned portion of the sequence can be assigned to either the UE 104 or the server 112, depending on the implementation, as can the final partitioned portion of the sequence. Accordingly, for purposes of describing the exemplary implementation of the partitioned computation ML operation execution process of FIG. 9 below, one of the UE 104 or the server 112 that implements the initial partitioned portion in a given partitioned configuration will be referred to herein as the “first device,” and the other of the UE 104 or the server 112 that implements the next partitioned portion following the initial partitioned portion of the sequence will be referred to as the “second device,” referring to the order of device participation in the sequence of the partitioned configuration. In the example of FIG. 10 illustrating this implementation, the server 112 is the first device, the UE 104 is the second device, the partitioned configuration is a two-way partitioned configuration, the server 112 implements the initial partitioned portion, and the UE 104 implements the final partitioned portion.

[0054] A typical neural network receives input data, processes the input data in various layers or other structures of the neural network, and then provides a final output result as a result of this processing of the input data. Depending on the purpose of the ML operation being performed and the origin of the input data for the neural network, which is implemented in a split configuration by the UE 104 and the server 112, the input data may be provided by the UE 104, by the server 112, by other components, or a combination thereof. Regardless of the source, to begin execution of the ML operation, the input data is provided to a first device implementing the initial split. Thus, in block 902, one or more sources of input data transmit or otherwise provide the first device access to the input data used for the ML operation. For example, in a split configuration in which the server 112 implements the initial split (i.e., the server 112 is the first device and the UE 104 is the second device), but the ML operation requires input data obtained by the UE 104 (e.g., an image captured by the UE 104), the UE 104 may transmit image data to the server 112 via the BS 110. As another example, the image data may be an image provided from a web page (e.g., the ML operation is an image match search), so rather than providing the image data directly, the UE 104 instead sends a web address or other pointer to the image and web page, and the server 112 uses the pointer to retrieve the image data from the web page.

[0055] It will be appreciated that in a cellular network or other network, such as the illustrated wireless communications network 100 of FIG. 1, the BS 110 and the UE 104 may use one or more resource allocation schemes to facilitate efficient and timely uplink or downlink transmission of data between the UE 104 and the BS 110. Such a scheme may be relevant when the UE 104 provides some or all of the input data to the server 112 for input in an initial layer of a neural network at the server 112, particularly when there are maximum latency requirements imposed on the ML operation. Thus, following the split computation negotiation process, and prior to transmitting the input data from the UE 104 to the server 112 via the BS 110, the UE 104 may inform the BS 110 of its uplink transmission needs for timely wireless transmission of the input data from the UE 104 to the BS 110. 10 to illustrate, following receipt of the split computing grant message 1010, the UE 104 identifies input data provided by the UE 104 to the server 112, determines uplink (UL) data requirements for transmitting the input data, and buffers the input data in one or more UL buffers in block 1012. In block 1012, the UE 104 transmits a UL transmission request for the input data to the BS 110 in the form of a UL buffer status report (BSR) message 1014 to inform the BS 110 of the amount of data in its UL buffer for UL transmission. In response to the UL BSR message 1014, the BS 110 transmits a UL grant to the UE 104, for example, in the form of a UL downlink control information (DCI) message 1016, which informs the UE 104 of parameters to use when transmitting the data in its UL buffer (including the input data) to the BS 110, such as physical layer resource allocations, power control commands, etc. In block 1018 , the UE 104 wirelessly transmits the input data as UL data to the BS 110 using the indicated parameters of the UL DCI message 1016 , and the BS 110 forwards the input data to the server 112 .

[0056] 9, input data is provided to the first device, whereby the first device receives the input data in block 904 and processes the input data in an initial partition implemented in the ML module of the first device (block 1020, FIG. 10) to generate output data representing intermediate results of the ML operation. In block 906, the intermediate results (block 1022, FIG. 10) are sent to the second device. In block 908, the second device receives the intermediate results and provides them as input data to the next partition implemented in the ML module of the second device, which processes the input data (block 1024, FIG. 10) to generate output data. If this next division is the final division in the sequence of divisions, as indicated by decision block 910, the output data represents the final result of the neural network and therefore the final result of the ML operation, and therefore either or both of the final results are transmitted to the first device in block 912 so that the first device can take one or more actions in response to the final result, or the second device can take one or more actions in response to the final result in block 914 (block 1026, FIG. 10). However, if this next division portion is an intermediate division portion in a sequence of division portions, the output data represents intermediate results, and therefore, at block 916, the intermediate results are sent to the first device (or to a third device if more than two devices are implemented in the division neural network configuration), whereby these intermediate results are provided as input data to the next division configuration in the sequence to generate output data, which may then be the final result of the neural network if this next division configuration is the last division configuration in the sequence and be processed accordingly, or if there are one or more additional division portions still remaining in the sequence, the output data may be provided to the next device for use as input data in the next division configuration in the sequence, and so on, until the final division configuration in the sequence outputs the final result of the neural network.

[0057] Embodiments of the present disclosure may also be better understood through consideration of the following non-limiting examples.

[0058] Example 1: A computer-implemented method comprising: at a first device, partitioning a neural network into partitioned neural network configurations for at least the first device and a second device based on a set of one or more current conditions of the first device, the partitioned neural network configuration specifying a distribution of elements of a neural network architecture of the neural network between at least a first neural network portion of the first device and a second neural network portion of the second device, the method further comprising: implementing the first neural network portion at the first device; transmitting a representation of the second neural network portion to the second device; and processing first data with the first neural network portion to generate a first output.

[0059] Example 2: The method of Example 1, wherein the first data includes input data of the neural network and the first output includes intermediate result data, and the method further includes transmitting the intermediate result data to the second device for processing by the second neural network portion.

[0060] Example 3: The method of example 2, further comprising receiving a final result output of the neural network from the second device; and performing at least one action at the first device in response to the final result output.

[0061] Example 4: The method of Example 1, further comprising receiving the first data from the second device, wherein the first data comprises intermediate result data generated by the second neural network portion at the second device, and the first output is a final result output of the neural network.

[0062] Example 5: The method of Example 4, further comprising performing at least one action at the first device based on the final result output of the neural network.

[0063] Example 6: The method of Example 4, further comprising transmitting input data from the first device to the second device for processing by the second neural network portion at the second device.

[0064] Example 7: A method as described in any of Examples 1 to 6, wherein the set of one or more current conditions includes at least one of a battery condition, a network condition, a thermal condition, availability of computational resources, or a specified quality of service condition.

[0065] Example 8: The method of any of Examples 1 to 7, further comprising: determining latency conditions for performing neural network operations using the neural network; and determining the split neural network configurations of at least the first device and the second device further based on the latency conditions.

[0066] Example 9: The method of Example 8, further comprising: determining whether to perform the neural network operation entirely on the client device or by using a split configuration of the neural network based on at least one of the set of one or more current conditions or the latency condition; and determining the split neural network configuration of at least the first device and the second device in response to determining to use the split configuration of the neural network.

[0067] Example 10: The method of any of Examples 1 to 9, further comprising sending a partitioned computation request to the second device, the partitioned computation request proposing the partitioned neural network configuration, and the first neural network portion being implemented on the first device in response to receiving approval of the partitioned computation request from the second device.

[0068] Example 11: A method according to any of Examples 1 to 9, wherein determining the partitioned neural network configuration of at least the first device and the second device includes sending a partitioned computation request to the second device, the partitioned computation request proposing a different partitioned neural network configuration based on the set of one or more current conditions, and the determining further includes receiving a partitioned computation counterproposal from the second device, the partitioned computation request counterproposal proposing the partitioned neural network configuration, and the first neural network portion is implemented on the first device in response to accepting the partitioned computation counterproposal based on the set of one or more current conditions.

[0069] Example 12: The method of one of Examples 10 or 11, wherein the partitioned computation request further indicates latency requirements for performing neural network operations using the partitioned neural network configuration.

[0070] Example 13: A method as described in any of Examples 1 to 12, wherein the representation of the second neural network portion includes at least one of data describing a neural network architecture of the second neural network portion, or an identifier of a predetermined split neural network configuration of the neural network represented in a repository accessible to the second device.

[0071] Example 14: A computer-implemented method, at a second device, comprising receiving, from a first device, instructions for a partitioned neural network configuration of at least a first device and the second device, the partitioned neural network configuration specifying a distribution of elements of a neural network architecture of a neural network between at least a first neural network portion of the first device and a second neural network portion of the second device, the method further comprising: implementing the second neural network portion at the second device; receiving first data from the first device; and processing the first data in the second neural network portion to generate a first output.

[0072] Example 15: The method described in Example 14, wherein the first data includes input data of the neural network and the first output includes intermediate result data, and the method further includes transmitting the intermediate result data to the first device for processing by the first neural network portion.

[0073] Example 16: The method described in Example 14, wherein the first data includes intermediate result data generated by the first neural network portion on the first device, and the first output is a final result output of the neural network.

[0074] Example 17: The method of Example 16, further comprising at least one of: performing at least one action on the second device in response to the final result output; or transmitting the final result output to the first device.

[0075] Example 18: A method according to any of Examples 14 to 17, wherein receiving the instruction for a split neural network configuration includes receiving a split computation request proposing the split neural network configuration, the method further including determining whether to grant the split computation request, and the second neural network portion is implemented on the second device in response to determining to grant the split computation request.

[0076] Example 19: The method of Example 18, further comprising receiving from the first device an indication of latency conditions for performing a neural network operation using the neural network, and determining whether to grant the split computation request is based on the latency conditions.

[0077] Example 20: A method as described in any of Examples 1 to 19, wherein the second neural network portion is implemented on the second device in response to receiving a representation of the second neural network portion from the first device, and the representation of the second neural network portion includes at least one of data representing a neural network architecture of the second neural network portion, or an identifier of a predetermined split neural network configuration of the neural network represented in a repository accessible to the second device.

[0078] Example 21: A method described in any of Examples 1 to 20, wherein the first device is a user equipment and the second device is one or more servers connected to a network wirelessly accessible by the user equipment.

[0079] Example 22: The method described in any of Examples 1 to 21, wherein the neural network is a deep neural network (DNN) model.

[0080] Example 23: A method described in any of Examples 1 to 22, wherein the split neural network configuration distributes a first set of successive layers of the neural network to the first neural network portion and a second set of successive layers of the neural network to the second neural network portion, the second set being adjacent to the first set.

[0081] Example 24: A method described in any of Examples 1 to 22, wherein the split neural network configuration includes a two-way split having an initial neural network portion and a final neural network portion, the initial neural network portion being assigned to one of the first device or the second device, and the final neural network portion being assigned to the other of the first device or the second device.

[0082] Example 25: A method described in any of Examples 1 to 22, wherein the split neural network configuration includes a three-way split having an initial neural network portion, an intermediate neural network portion, and a final neural network portion, the initial neural network portion and the final neural network portion being assigned to one of the first device or the second device, and the intermediate neural network portion being assigned to the other of the first device or the second device.

[0083] Example 26: A device comprising: a network interface; at least one processor coupled to the network interface; and a memory storing executable instructions, the executable instructions configured to operate the at least one processor to perform a method described in any one of Examples 1 to 25.

[0084] Example 27: The device described in Example 26, wherein the first device is user equipment of a cellular network and the second device is one or more servers of a network wirelessly accessible by the user equipment.

[0085] Example 28: The device of any of Examples 26 or 27, wherein the device is the first device.

[0086] Example 29: The device of any of Examples 26 or 27, wherein the device is the second device.

[0087] In some embodiments, certain aspects of the techniques described above may be implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored on, or otherwise tangibly embodied in, a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by, one or more processors, operate the one or more processors to perform one or more aspects of the techniques described above. Non-transitory computer-readable storage media may include, for example, magnetic or optical disk storage devices, solid-state storage devices such as flash memory, cache, random access memory (RAM), or other single or multiple non-volatile memory devices, and the like. The executable instructions stored on the non-transitory computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats that are interpreted or otherwise executable by one or more processors.

[0088] A computer-readable storage medium may include any storage medium, or combination of storage media, that is accessible by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tape, or magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS)-based storage media. A computer-readable storage medium may be incorporated into a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard drive), removably attached to a computing system (e.g., an optical disk or universal serial bus (USB)-based flash memory), or coupled to a computer system via a wired or wireless network (e.g., network-accessible storage (NAS)).

[0089] In addition to the above, it should be noted that not all activities or elements described above in the general description are required, and that some of the specific activities or devices may not be required, or that one or more additional activities may be performed, or that one or more additional elements may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will recognize that various modifications and variations can be made without departing from the scope of the present disclosure, as set forth in the claims below. Accordingly, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.

[0090] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, benefits, advantages, and solutions to problems, as well as any feature or features that may cause or make more pronounced any benefit, advantage, or solution, should not be construed as critical, necessary, or essential features of any or all claims. Moreover, the specific embodiments disclosed above are exemplary only, as the disclosed inventive subject matter may be modified and practiced in different but equivalent manners that will be apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as set forth in the claims below. It is therefore evident that the specific embodiments disclosed above may be altered or modified, and that all such variations are contemplated within the scope of the disclosed inventive subject matter. Accordingly, the protection sought herein is as set forth in the claims below.

Claims

1. 1. A computer-implemented method, comprising: partitioning a neural network into partitioned neural network configurations for at least the first device and a second device based on a set of one or more current conditions of the first device, the partitioned neural network configuration specifying a distribution of elements of a neural network architecture of the neural network between at least a first neural network portion of the first device and a second neural network portion of the second device, the method further comprising: implementing the first neural network portion on the first device; transmitting a representation of the second neural network portion to the second device; processing first data with the first neural network portion to generate a first output; A computer-implemented method comprising:

2. the first data includes input data for the neural network; the first output includes intermediate result data; The method further comprises: transmitting the intermediate result data to the second device for processing by the second neural network portion. Including, The method of claim 1.

3. receiving a final result output of the neural network from the second device; performing at least one action on the first device in response to the final result output; The method of claim 2 further comprising:

4. receiving the first data from the second device, the first data including intermediate result data generated by the second neural network portion at the second device; the first output is the final result output of the neural network; The method of claim 1.

5. performing at least one action at the first device based on the final result output of the neural network. The method of claim 4 further comprising:

6. transmitting input data from the first device to the second device for processing by the second neural network portion at the second device; The method of claim 4 further comprising:

7. The method of any of claims 1 to 6, wherein the set of one or more current conditions includes at least one of a battery condition, a network condition, a thermal condition, availability of computational resources, or a specified quality of service condition.

8. determining latency requirements for performing neural network operations using the neural network; determining the partitioned neural network configurations of at least the first device and the second device further based on the latency condition; The method of any one of claims 1 to 7, further comprising:

9. determining whether to perform the neural network operation entirely on the first device or by using a partitioned configuration of the neural network based on at least one of the set of one or more current conditions or the latency condition; determining a partitioned neural network configuration for at least the first device and the second device in response to determining to use a partitioned configuration of the neural network; The method of claim 8 further comprising:

10. sending a partitioned computation request to the second device, the partitioned computation request proposing the partitioned neural network configuration; the first neural network portion is implemented on the first device in response to receiving a grant of the split computation request from the second device; The method according to any one of claims 1 to 9.

11. Determining the partitioned neural network configuration of at least the first device and the second device includes: and transmitting a partitioned computation request to the second device, the partitioned computation request proposing a different partitioned neural network configuration based on the set of one or more current conditions, wherein the determining further comprises: receiving a partitioned computation counterproposal from the second device, the partitioned computation counterproposal proposing the partitioned neural network configuration; the first neural network portion is implemented on the first device in response to accepting the split computation counterproposal based on the set of one or more current conditions. The method according to any one of claims 1 to 9.

12. 12. The method of claim 10, wherein the partitioned computation request further indicates latency requirements for performing neural network operations using the partitioned neural network configuration.

13. 13. The method of claim 1, wherein the representation of the second neural network portion includes at least one of data describing a neural network architecture of the second neural network portion, or an identifier of a predetermined partitioned neural network configuration of the neural network represented in a repository accessible to the second device.

14. A computer-implemented method, comprising: receiving, from a first device, an indication of a partitioned neural network configuration of at least the first device and the second device, the partitioned neural network configuration specifying a distribution of elements of a neural network architecture of a neural network between at least a first neural network portion of the first device and a second neural network portion of the second device, the method further comprising: implementing the second neural network portion on the second device; receiving first data from the first device; processing the first data with the second neural network portion to generate a first output; A computer-implemented method comprising:

15. the first data includes input data for the neural network; the first output includes intermediate result data; The method further comprises: transmitting the intermediate result data to the first device for processing by the first neural network portion. Including, 15. The method of claim 14.

16. the first data includes intermediate result data generated by the first neural network portion at the first device; the first output is the final result output of the neural network; 15. The method of claim 14.

17. performing at least one action on the second device in response to the final result output; or sending the final result output to the first device; 17. The method of claim 16, further comprising at least one of:

18. receiving the indication of a partitioned neural network configuration includes receiving a partitioned computation request proposing the partitioned neural network configuration; The method further includes determining whether to grant the split computation request; the second neural network portion is implemented on the second device in response to determining to grant the split computation request. The method according to any one of claims 14 to 17.

19. receiving from the first device an indication of latency requirements for performing a neural network operation using the neural network; determining whether to grant the split computation request is based on the latency condition; 20. The method of claim 18.

20. 20. The method of claim 1, wherein the second neural network portion is implemented on the second device in response to receiving a representation of the second neural network portion from the first device, the representation of the second neural network portion including at least one of data representing a neural network architecture of the second neural network portion, or an identifier of a predetermined partitioned neural network configuration of the neural network represented in a repository accessible to the second device.

21. The method of any of claims 1 to 20, wherein the first device is a user equipment and the second device is one or more servers connected to a network wirelessly accessible by the user equipment.

22. The method according to any one of claims 1 to 21, wherein the neural network is a deep neural network (DNN) model.

23. 23. The method of claim 1, wherein the split neural network configuration distributes a first set of successive layers of the neural network to the first neural network portion and a second set of successive layers of the neural network to the second neural network portion, the second set being adjacent to the first set.

24. 23. The method of any of claims 1 to 22, wherein the split neural network configuration comprises a two-way split comprising an initial neural network portion and a final neural network portion, the initial neural network portion being assigned to one of the first device or the second device, and the final neural network portion being assigned to the other of the first device or the second device.

25. 23. The method of any of claims 1 to 22, wherein the partitioned neural network configuration comprises a three-way partition comprising an initial neural network portion, an intermediate neural network portion, and a final neural network portion, the initial neural network portion and the final neural network portion being assigned to one of the first device or the second device, and the intermediate neural network portion being assigned to the other of the first device or the second device.

26. A network interface; at least one processor coupled to the network interface; and a memory storing executable instructions, the executable instructions configured to operate the at least one processor to perform a method according to any one of claims 1 to 25. device.

27. 27. The device of claim 26, wherein the first device is user equipment of a cellular network and the second device is one or more servers of a network wirelessly accessible by the user equipment.

28. 28. The device of claim 26 or 27, wherein the device is the first device.

29. 28. The device of claim 26 or 27, wherein the device is the second device.

Images