Processing data using remote network computing resources

Abstract

Embodiments include a method for managing the processing of data and performing a computing task of a wireless device using computing resources of a remote network computing device. The wireless device and the network computing device may dynamically track factors affecting the round-trip latency of the computing task. The wireless device and the network computing device may generate and send metadata including the factors and a latency budget for the computing task. The wireless device and the network computing device may adjust a processing time for processing data associated with the computing task based on the received metadata and the latency budget.

Description

[Background technology]

[0001] Some computing tasks place high demands on the relatively limited processing capabilities of modern wireless devices. In some situations, wireless devices may take advantage of high-speed wireless communication and network computing resources to offload processor-intensive computing tasks to computing devices deeper in the network, such as server devices. However, offloading computing tasks to network computing devices introduces latency due to both the time it takes to perform remote processing and the information travel time across the communication link. This introduced latency may prove unacceptable for the implementation of some applications and services, especially those that affect health and safety. Summary of the Invention [Means for solving the problem]

[0002] Various aspects include systems and methods of wireless communications implemented by a processor of a wireless device to leverage remote computing resources. Various aspects may include determining factors affecting round trip latency of a computing task, generating first metadata including the determined factors and a latency budget for the computing task, sending the first metadata and data to a remote network computing device for processing as part of the computing task, receiving second metadata from the network computing device including processed data of the computing task and an indication of remaining time in the latency budget, and adjusting a processing time for post-processing the processed data based on the second metadata to complete post-processing of the processed data within the latency budget.

[0003] In some aspects, determining the factors affecting the round trip latency of the computing task may include determining one or more of a wireless device pre-processing time, a first communication time from the wireless device to a remote network computing device, a remote network computing device processing time, a second communication time from the remote network computing device to the wireless device, or a wireless device post-processing time.

[0004] In some aspects, adjusting the processing time for post-processing the processed data based on the second metadata to complete post-processing of the processed data within the latency budget may include adjusting a dynamic clock and voltage scaling (DCVS) and a dynamic task priority assignment of the processed data based on the time remaining in the latency budget. In some aspects, adjusting the processing time for post-processing the processed data based on the second metadata to complete post-processing of the processed data within the latency budget may include adjusting a DCVS and a task queue position of the processed data based on the time remaining in the latency budget.

[0005] In some aspects, the processed data of the computing task received from the remote network computing device may not be a completed work result. Some aspects may include adjusting a compression ratio of the data for processing as part of the computing task based on the determined factors and the latency budget.

[0006] Various aspects include systems and methods implemented by a network computing device for assisting a remote wireless device in processing data. Various aspects may include receiving metadata and data from the remote wireless device for processing as part of a computing task, where first metadata includes factors affecting round trip latency and a latency budget for the computing task, adjusting a processing time for processing the data based on the first metadata and the latency budget, generating second metadata including an indication of the time remaining in the latency budget, and sending the processed data and the second metadata to the remote wireless device in a format that enables post-processing by the remote wireless device.

[0007] In some aspects, receiving the first metadata and data from the wireless device for processing as part of the computing task may include receiving one or more of a remote wireless device pre-processing time, a first communication time from the remote wireless device to the network computing device, a network computing device processing time, a second communication time from the network computing device to the remote wireless device, or a remote wireless device post-processing time.

[0008] In various aspects, adjusting the processing time for processing the data based on the first metadata and the latency budget may include adjusting a dynamic clock and voltage scaling (DCVS) and a dynamic task priority assignment of the data based on the time remaining in the latency budget. In various aspects, adjusting the processing time for processing the data based on the first metadata and the latency budget may include adjusting a DCVS and a task queue position of the data based on the time remaining in the latency budget. Various aspects may include adjusting a compression ratio of the processed data based on factors affecting the round trip latency and the latency budget.

[0009] Further aspects include a wireless device having a processor configured to perform one or more operations of any of the methods summarized above. Further aspects include a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of the wireless device to perform the operations of any of the methods summarized above. Further aspects include a wireless device having means for performing the functions of any of the methods summarized above. Further aspects include a system-on-chip for use in a wireless device including a processor configured to perform one or more operations of any of the methods summarized above. Further aspects include a network computing device having a processor configured to perform one or more operations of any of the methods summarized above. Further aspects include a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of the network computing device to perform the operations of any of the methods summarized above. Further aspects include a network computing device having means for performing the functions of any of the methods summarized above. Further aspects include a system-on-chip for use in a network computing device including a processor configured to perform one or more operations of any of the methods summarized above.

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the claims and, together with the general description given above and the detailed description below, serve to explain the features of the claims. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a system block diagram illustrating an example communication system suitable for implementing any of the various embodiments. [Diagram 2]FIG. 1 is a component block diagram illustrating an exemplary computing and wireless modem system suitable for implementing any of the various embodiments. [Diagram 3] FIG. 1 is a component block diagram illustrating a software architecture including radio protocol stacks for user and control planes in wireless communications suitable for implementing any of the various embodiments. [Figure 4A] FIG. 1 is a component block diagram illustrating a system configured for managing information transmission for wireless communications implemented by a processor of a base station, in accordance with various embodiments. [Figure 4B] FIG. 1 is a component block diagram illustrating a system configured for managing information transmission for wireless communications implemented by a processor of a base station, in accordance with various embodiments. [Figure 5A] FIG. 1 is a conceptual diagram showing factors that affect overall latency. [Figure 5B] 1 is a timeline illustrating a latency budget model in accordance with various embodiments. [Figure 6] 1 is a block diagram illustrating a message according to various embodiments. [Figure 7] 7 is a process flow diagram illustrating a method 700 that may be implemented by a processor of a wireless device for processing data using computing resources of a remote networked computing device, according to various embodiments. [Figure 8] FIG. 1 is a process flow diagram illustrating operations that may be performed by a processor of a wireless device as part of a method for processing data using computing resources of a remote networked computing device, according to various embodiments. [Figure 9] FIG. 1 is a process flow diagram illustrating operations that may be performed by a processor of a wireless device as part of a method for processing data using computing resources of a remote networked computing device, according to various embodiments. [Figure 10]FIG. 1 is a process flow diagram illustrating operations that may be performed by a processor of a wireless device as part of a method for processing data using computing resources of a remote networked computing device, according to various embodiments. [Figure 11] FIG. 1 is a process flow diagram illustrating a method that may be implemented by a processor of a network computing device for processing data in assistance to a remote wireless device, according to various embodiments. [Figure 12] FIG. 1 is a process flow diagram illustrating operations that may be performed by a processor of a network computing device as part of a method for processing data with the assistance of a remote wireless device, according to various embodiments. [Figure 13] FIG. 1 is a process flow diagram illustrating operations that may be performed by a processor of a network computing device as part of a method for processing data with the assistance of a remote wireless device, according to various embodiments. [Figure 14] FIG. 1 is a process flow diagram illustrating operations that may be performed by a processor of a network computing device as part of a method for processing data with the assistance of a remote wireless device, according to various embodiments. [Figure 15] FIG. 1 is a component block diagram of a networked computing device suitable for use with the various embodiments. [Figure 16] FIG. 1 is a component block diagram of a wireless device suitable for use with the various embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Various embodiments are described in detail with reference to the accompanying drawings. Whenever possible, the same reference numbers are used throughout the drawings to refer to the same or like parts. References made to specific examples and embodiments are for illustrative purposes only and do not limit the scope of the claims.

[0013] Various embodiments include systems and methods for managing the offloading of computing tasks for remote processing of data for services or applications of a wireless device. The wireless device may offload resource-intensive computing tasks to a processing device in a communications network, such as a server device. Various embodiments enable managing the timing of the performance of the computing tasks within a latency budget to account for the time required to perform the remote processing and the latency incurred by the information travel time across the communications link.

[0014] The term "wireless device" is used herein to refer to any one or all of wireless router devices, wireless appliances, cellular telephones, smartphones, portable computing devices, personal or mobile multimedia players, laptop computers, tablet computers, smartbooks, ultrabooks, palmtop computers, wireless email receivers, multimedia Internet-enabled cellular telephones, medical devices and equipment, biometric sensors / devices, wearable devices including smart watches, smart clothing, smart glasses, smart wristbands, and smart jewelry (e.g., smart rings, smart bracelets, etc.), entertainment devices (e.g., wireless game controllers, music and video players, satellite radio, etc.), wireless network-enabled Internet of Things (IoT) devices including smart meters / sensors, industrial manufacturing equipment, and large and small machines and appliances for home and business use, wireless communication elements in autonomous and semi-autonomous vehicles, wireless devices fixed or embedded in various mobile platforms, global positioning system devices, and similar electronic devices including memory, wireless communication components, and programmable processors.

[0015] The term "system on chip" (SOC) is used herein to refer to a single integrated circuit (IC) chip that contains multiple resources and / or processors integrated on a single substrate. A single SOC may include circuitry for digital, analog, mixed signal, and radio frequency functions. A single SOC may also include any number of general purpose and / or special purpose processors (digital signal processors, modem processors, video processors, etc.), memory blocks (e.g., ROM, RAM, Flash, etc.), and resources (e.g., timers, voltage regulators, oscillators, etc.). A SOC may also include software for controlling the integrated resources and processors, as well as for controlling peripheral devices.

[0016] The term "system in package" (SIP) may be used herein to refer to a single module or package that contains multiple resources, computing units, cores, and / or processors on two or more IC chips, substrates, or SOCs. For example, a SIP may include a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may include one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged in a unifying substrate. A SIP may also include multiple independent SOCs packaged in close proximity, coupled to each other via high-speed communication circuitry, such as on a single motherboard or within a single wireless device. The proximity of the SOCs facilitates high-speed communication and sharing of memory and resources.

[0017] Some computing tasks place high demands on the relatively limited processing power of modern wireless devices. For example, computing tasks associated with virtual reality (VR), augmented reality (AR), or mixed reality (MR) applications (sometimes collectively referred to as extended reality (XR)) involve receiving sensor data, processing the sensor data, and generating images to be presented in real time on a display device (such as a head-mounted display device). The user experience associated with such presentation is highly latency sensitive. Similarly, many computing tasks required for autonomous and semi-autonomous vehicles, such as sensing the vehicle's environment, processing sensor data, and making steering and navigation decisions, are safety-critical and therefore also latency sensitive. Furthermore, remote robotic operations, such as remote surgery and other remote medical operations, also critically depend on processing latency. These and other similar applications have stringent low latency requirements in that they require latency to be below a threshold to provide a minimum acceptable performance.

[0018] Network computing devices, such as server devices, can provide computing support to wireless devices, enabling the wireless devices to leverage high-speed communications networks, such as 5G New Radio (NR) communications systems, to offload processing-intensive computing tasks to the network computing devices. Such systems seek to manage the time it takes to perform remote processing and the latency incurred by data travel time across any communications link.

[0019] However, traditional static solutions to complex processing problems are unacceptable or inefficient. For example, running a wireless device processor at a relatively high operating frequency may meet the latency requirements of an application, but may cause the processor to reach or exceed acceptable thermal limits and consume power at a rate that would deplete the battery within a short amount of time. Furthermore, latency across a communication network may be stochastic, making manual adjustment of the wireless device's processor speed to account for variations in network latency functionally impossible.

[0020] Various embodiments include a method for processing data using a computing resource of a remote network computing device that includes dynamically tracking latency in performing a computing task compared to a latency budget for the computing task. As used herein, the term "latency budget" refers to the time during which a computing task must be completed to meet a performance or quality threshold for the computing task. Remote processing of data for a computing task may require several operations, each of which requires some time, such as initial processing of the data by the wireless device, time for transmission of the data from the wireless device to the remote network computing device, processing time by the remote network computing device, time for transmission of the data from the remote network computing device to the wireless device, and further processing time by the wireless device to use the data received from the remote network computing device. The time required to complete all of the operations required in delegating a computing task to a remote computing resource is referred to herein as the "round trip latency" of the computing task. In various embodiments, the wireless device and the remote network computing device determine and report the time consumed by one or more of the remote processing operations so that a subsequent device (e.g., the wireless device or the remote network computing device) can adjust its processing time for processing data for a delegated computing task.

[0021] Various embodiments may include a method implemented by a processor of a wireless device for processing data using computing resources of a remote network computing device. Some embodiments may include determining factors affecting round trip latency of a computing task that may be delegated to a remote network computing device, generating first metadata including the determined factors and a latency budget for the computing task, sending the first metadata and the data to the remote network computing device for processing as part of the computing task, receiving second metadata from the network computing device including processed data of the computing task and an indication of the remaining time in the latency budget, and adjusting a processing time for receiving and / or post-processing the processed data based on the second metadata to complete the overall computing task within the latency budget.

[0022] Various embodiments may include a method implemented by a processor of a network computing device for assisting a remote wireless device (i.e., remote from the network computing device) in processing data. In some embodiments, the network computing device may receive first metadata and data from the remote wireless device for processing as part of a computing task. The first metadata may include factors affecting round trip latency and a latency budget for the computing task. The network computing device may adjust a processing time for processing the data based on the first metadata and the latency budget and may generate second metadata including an indication of the time remaining in the latency budget. The network computing device may send the processed data and the second metadata to the remote wireless device in a format that allows for post-processing and use by the remote wireless device to complete the computing task.

[0023] In some embodiments, the wireless device or network computing device may adjust the processing time by adjusting one or more of the operating frequency of one or more processors (such as a CPU, GPU, DSP, or another suitable processor) and / or the transfer rate of a memory component (such as a double data rate (DDR) memory), which may affect the processing power of the processor. In some embodiments, the wireless device or network computing device may adjust the processing time by adjusting a dynamic clock and voltage scaling (DCVS) of the processor and a dynamic task priority assignment of the data to be processed based on the time remaining in the latency budget. In some embodiments, the wireless device or network computing device may adjust the processing time by adjusting the DCVS and the position of the data to be processed in a processing or task queue ("task queue position"), or by changing the order in which the data is processed (processing order or sequence) based on the time remaining in the latency budget. In some embodiments, the wireless device may adjust the processing time by adjusting the level of detail in the computed output, such as the rendering quality of an image, for display as part of an XR application. In some embodiments, the wireless device may adjust processing time by selecting an appropriate algorithm for a computing task, such as a relatively complex or sophisticated algorithm (e.g., an algorithm that performs relatively many decisions or calculations or performs actions using a relatively large number of factors or criteria) or a relatively simple or lightweight algorithm (e.g., an algorithm that performs relatively few decisions or calculations or performs actions using a relatively small number of factors or criteria).

[0024] In some embodiments, the wireless device may receive data from the network computing device that has been processed by the network computing device but is not usable as received by the wireless device without additional processing by the wireless device (i.e., the data received from the wireless device is not a completed work result). In some embodiments, the wireless device or the network computing device may adjust a compression ratio of the data for transmission to the other device based on the determined factors and latency budget. For example, the wireless device or the network computing device may adjust a data compression ratio (e.g., by selecting a different data compression algorithm) to increase or decrease the size of the data portions (e.g., segments) based on one or more communication link conditions, such as a level of communication link congestion and / or signal noise, interference, throughput, bandwidth, or another suitable communication link condition.

[0025] 1 is a system block diagram illustrating an example communication system 100 suitable for implementing any of the various embodiments. The communication system 100 may be a 5G New Radio (NR) network or any other suitable network, such as a Long Term Evolution (LTE) network.

[0026] The communications system 100 may include a heterogeneous network architecture including a core network 140 and various wireless devices (shown in FIG. 1 as wireless devices 120a-120e). The communications system 100 may also include one or more network computing devices 125 that can communicate with the wireless devices 120a-120e. In some embodiments, the wireless devices 120a-120e may send data to the network computing devices 125 for processing as part of a computing task.

[0027] The communication system 100 may also include several base stations (denoted as BS 110a, BS 110b, BS 110c, and BS 110d) and other network entities. A base station is an entity that communicates with wireless devices and may also be called a Node B, a Node B, an LTE evolved Node B (eNB), an Access Point (AP), a radio head, a Transmit Receive Point (TRP), a New Radio Base Station (NR BS), a 5G Node B (NB), a Next Generation Node B (gNB), etc. Each base station may provide communication coverage for a particular geographic area. In 3GPP, the term "cell" can refer to a base station's coverage area, a base station subsystem serving this coverage area, or a combination thereof, depending on the context in which the term is used.

[0028] The base stations 110a-110d may provide communication coverage for a macro cell, a pico cell, a femto cell, another type of cell, or a combination thereof. A macro cell may cover a relatively large geographic area (e.g., a few kilometers in radius) and may allow unrestricted access by wireless devices with a service subscription. A pico cell may cover a relatively small geographic area and may allow unrestricted access by wireless devices with a service subscription. A femto cell may cover a relatively small geographic area (e.g., a home) and may allow restricted access by wireless devices that have an association with the femto cell (e.g., wireless devices in a Closed Subscriber Group (CSG)). A base station for a macro cell may be referred to as a Macro BS. A base station for a pico cell may be referred to as a Pico BS. A base station for a femto cell may be referred to as a Femto BS or a Home BS. 1, base station 110a may be a macro BS for macro cell 102a, base station 110b may be a pico BS for pico cell 102b, and base station 110c may be a femto BS for femto cell 102c. Base stations 110a-110d may support one or more (e.g., three) cells. The terms “eNB,” “base station,” “NR BS,” “gNB,” “TRP,” “AP,” “Node B,” “5G NB,” and “cell” may be used interchangeably herein.

[0029] In some examples, the cells may not be stationary and the geographic areas of the cells may move according to the location of the mobile base station. In some examples, the base stations 110a-110d may be interconnected to each other and to one or more other base stations or network nodes (not shown) in the communication system 100 through various types of backhaul interfaces, such as direct physical connections, virtual networks, or combinations thereof, using any suitable transport network.

[0030] The base stations 110a-110d may communicate with the core network 140 over wired or wireless communications links 126. The wireless devices 120a-120e may communicate with the base stations 110a-110d over wireless communications links 122.

[0031] The wired communications link 126 may use various wired networks (e.g., Ethernet, TV cable, telephony, fiber optics, and other forms of physical network connections) that may use one or more wired communications protocols, such as Ethernet, Point-to-Point Protocol, High-Level Data Link Control (HDLC), Advanced Data Communication Control Protocol (ADCCP), and Transmission Control Protocol / Internet Protocol (TCP / IP).

[0032] The communication system 100 may also include a relay station (e.g., relay BS 110d). A relay station is an entity that can receive a transmission of data from an upstream station (e.g., a base station or a wireless device) and transmit the data to a downstream station (e.g., a wireless device or a base station). A relay station may also be a wireless device that can relay a transmission for another wireless device. In the example shown in FIG. 1, relay station 110d may communicate with macro base station 110a and wireless device 120d to facilitate communication between base station 110a and wireless device 120d. A relay station may also be referred to as a relay base station, a relay base station, a repeater, etc.

[0033] The communications system 100 may be a heterogeneous network including different types of base stations, e.g., macro base stations, pico base stations, femto base stations, relay base stations, etc. These different types of base stations may have different transmit power levels, different coverage areas, and may have different impacts on interference in the communications system 100. For example, macro base stations may have high transmit power levels (e.g., 5-40 watts), whereas pico base stations, femto base stations, and relay base stations may have lower transmit power levels (e.g., 0.1-2 watts).

[0034] A network controller 130 may couple to a set of base stations and provide coordination and control for these base stations. The network controller 130 may communicate with the base stations via a backhaul. The base stations may also communicate with one another directly or indirectly, e.g., via wireless or wireline backhaul.

[0035] The wireless devices 120a, 120b, 120c may be dispersed throughout the communication system 100, and each wireless device may be fixed or mobile. A wireless device may also be referred to as an access terminal, a terminal, a mobile station, a subscriber unit, a station, etc.

[0036] The macro base station 110a may communicate with the communications network 140 over a wired or wireless communications link 126. The wireless devices 120a, 120b, 120c may communicate with the base stations 110a-110d over wireless communications links 122.

[0037] The wireless communication links 122, 124 may include multiple carrier signals, frequencies, or frequency bands, each of which may include multiple logical channels. The wireless communication links 122 and 124 may use one or more radio access technologies (RATs). Examples of RATs that may be used in the wireless communication links include 3GPP® LTE, 3G, 4G, 5G (e.g., NR), GSM, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA®), Worldwide Interoperability for Microwave Access (WiMAX), Time Division Multiple Access (TDMA), and other mobile telephony communication technology cellular RATs. Further examples of RATs that may be used in one or more of the various wireless communication links 122, 124 in the communication system 100 include medium-range protocols such as Wi-Fi, LTE-U, LTE-Direct, LAA, MuLTEfire, and relatively short-range RATs such as ZigBee, Bluetooth, and Bluetooth Low Energy (LE).

[0038] Some wireless networks (e.g., LTE) use orthogonal frequency division multiplexing (OFDM) on the downlink and single-carrier frequency division multiplexing (SC-FDM) on the uplink. OFDM and SC-FDM partition the system bandwidth into multiple (K) orthogonal subcarriers, also commonly referred to as tones, bins, etc. Each subcarrier may be modulated with data. In general, modulation symbols are sent in the frequency domain with OFDM and in the time domain with SC-FDM. The spacing between adjacent subcarriers may be fixed, and the total number of subcarriers (K) may be dependent on the system bandwidth. For example, the subcarrier spacing may be 15 kHz, and the minimum resource allocation (called a "resource block") may be 12 subcarriers (or 180 kHz). Thus, the nominal fast file transfer (FFT) size may be equal to 128, 256, 512, 1024, or 2048 for a system bandwidth of 1.25, 2.5, 5, 10, or 20 megahertz (MHz), respectively. The system bandwidth may also be partitioned into subbands. For example, a subband may cover 1.08 MHz (i.e., 6 resource blocks), and there may be 1, 2, 4, 8, or 16 subbands for a system bandwidth of 1.25, 2.5, 5, 10, or 20 MHz, respectively.

[0039] Although the description of some embodiments may use terminology and examples related to LTE technology, various embodiments may be applicable to other wireless communication systems, such as New Radio (NR) or 5G networks. NR may include support for half-duplex operation using OFDM with cyclic prefix (CP) on the uplink (UL) and downlink (DL) and using time division duplex (TDD). A single component carrier bandwidth of 100 MHz may be supported. An NR resource block may span 12 subcarriers with a subcarrier bandwidth of 75 kHz for a duration of 0.1 milliseconds (ms). Each radio frame may consist of 50 subframes having a length of 10 ms. As a result, each subframe may have a length of 0.2 ms. Each subframe may indicate a link direction (i.e., DL or UL) for data transmission, and the link direction per subframe may be dynamically switched. Each subframe may include DL / UL data as well as DL / UL control data. Beamforming may be supported, and the beam direction may be dynamically configured. Multiple-input multiple-output (MIMO) transmission with precoding may also be supported. MIMO configurations in the DL may support up to eight transmit antennas with multi-layer DL transmission of up to eight streams and up to two streams per wireless device. Multi-layer transmission with up to two streams per wireless device may be supported. Aggregation of multiple cells may be supported with up to eight serving cells. Alternatively, NR may support an air interface different from the OFDM-based air interface.

[0040] Some wireless devices may be considered as machine type communication (MTC) or evolved or extended machine type communication (eMTC) wireless devices. MTC and eMTC wireless devices include, for example, a robot, a drone, a remote device, a sensor, a meter, a monitor, a location tag, etc. that may communicate with a base station, another device (e.g., a remote device), or some other entity. A wireless node may provide, for example, connectivity for or to a network (e.g., a wide area network such as the Internet or a cellular network) via a wired or wireless communication link. Some wireless devices may be considered as Internet of Things (IoT) devices or may be implemented as NB-IoT (narrowband Internet of Things) devices. The wireless devices 120a-120e may be included within a housing that houses components of the wireless device, such as a processor component, a memory component, similar components, or a combination thereof.

[0041] In general, any number of communication systems and any number of wireless networks may be deployed in a given geographic area. Each communication system and wireless network may support a particular radio access technology (RAT) and may operate on one or more frequencies. The RAT may also be referred to as a radio technology, an air interface, etc. The frequencies may also be referred to as a carrier, a frequency channel, etc. Each frequency may support a single RAT in a given geographic area to avoid interference between communication systems of different RATs. In some cases, NR or 5G RAT networks may be deployed.

[0042] In some embodiments, two or more wireless devices 120a-120e (e.g., shown as wireless device 120a and wireless device 120e) may communicate directly (e.g., without using base station 110a-110d as an intermediary to communicate with each other) using one or more sidelink channels 124. For example, wireless devices 120a-120e may communicate using peer-to-peer (P2P) communication, device-to-device (D2D) communication, vehicle-to-everything (V2X) protocols (which may include vehicle-to-vehicle (V2V) protocols, vehicle-to-infrastructure (V2I) protocols, or similar protocols), mesh networks, or similar networks, or combinations thereof. In this case, wireless devices 120a-120e may perform scheduling operations, resource selection operations, and other operations described elsewhere herein as being performed by base station 110a.

[0043] 2 is a component block diagram illustrating an exemplary computing system and wireless modem system 200 suitable for implementing any of the various embodiments. The various embodiments may be implemented on a number of single-processor and multi-processor computer systems, including systems on a chip (SOC) or systems in a package (SIP).

[0044] 1 and 2, the illustrated exemplary computing system 200 (which may be a SIP in some embodiments) includes two SOCs 202, 204 coupled to a clock 206, a voltage regulator 208, and a wireless transceiver 266 configured to transmit and receive wireless communications to and from a wireless device, such as a base station 110a, via an antenna (not shown). In some embodiments, the first SOC 202 operates as a central processing unit (CPU) of the wireless device, implementing instructions of a software application program by performing arithmetic, logic, control, and input / output (I / O) operations specified by the instructions. In some embodiments, the second SOC 204 may operate as a dedicated processing unit. For example, the second SOC 204 may operate as a dedicated 5G processing unit responsible for managing high-capacity, high-speed (e.g., 5 Gbps, etc.), and / or very short wavelength (e.g., 28 GHz millimeter wave spectrum, etc.) communications.

[0045] The first SOC 202 may include a digital signal processor (DSP) 210, a modem processor 212, a graphics processor 214, an application processor 216, one or more coprocessors 218 (such as a vector coprocessor) connected to one or more of the processors, memory 220, custom circuitry 222, system components and resources 224, an interconnect / bus module 226, one or more temperature sensors 230, a thermal management unit 232, and a thermal power envelope (TPE) component 234. The second SOC 204 may include a 5G modem processor 252, a power management unit 254, an interconnect / bus module 264, multiple mmWave transceivers 256, memory 258, and various additional processors 260, such as application processors, packet processors, etc.

[0046] Each processor 210, 212, 214, 216, 218, 252, 260 may include one or more cores, and each processor / core may perform operations independent of the other processors / cores. For example, the first SOC 202 may include a processor that runs a first type of operating system (e.g., FreeBSD, LINUX, OS X, etc.) and a processor that runs a second type of operating system (e.g., Microsoft Windows 10). In addition, any or all of the processors 210, 212, 214, 216, 218, 252, 260 may be included as part of a processor cluster architecture (e.g., a synchronous processor cluster architecture, an asynchronous or heterogeneous processor cluster architecture, etc.).

[0047] The first SOC 202 and the second SOC 204 may include various system components, resources, and custom circuitry for managing sensor data, analog-to-digital conversion, wireless data transmission, and for performing other specialized operations, such as processing encoded audio and video signals for decoding data packets and rendering in a web browser. For example, the system components and resources 224 of the first SOC 202 may include power amplifiers, voltage regulators, oscillators, phase-locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, and other similar components used to support processors and software clients running on the wireless device. The system components and resources 224 and / or custom circuitry 222 may also include circuitry for interfacing with peripheral devices, such as cameras, electronic displays, wireless communication devices, external memory chips, etc.

[0048] The first SOC 202 and the second SOC 204 may communicate via an interconnect / bus module 250. The various processors 210, 212, 214, 216, 218 may be interconnected to one or more memory elements 220, system components and resources 224, as well as custom circuitry 222 and a thermal management unit 232 via an interconnect / bus module 226. Similarly, the processor 252 may be interconnected to a power management unit 254, a mmWave transceiver 256, a memory 258, and various additional processors 260 via an interconnect / bus module 264. The interconnect / bus modules 226, 250, 264 may include arrays of reconfigurable logic gates and / or implement a bus architecture (e.g., CoreConnect, AMBA, etc.). Communication may occur through advanced interconnects such as high-performance networks on chips (NoCs).

[0049] The first and / or second SOC 202, 204 may further include input / output modules (not shown) for communicating with resources external to the SOC, such as a clock 206 and a voltage regulator 208. Resources external to the SOC (e.g., clock 206, voltage regulator 208) may be shared by two or more of the internal SOC processors / cores.

[0050] In addition to the exemplary SIP 200 described above, various embodiments may be implemented in a wide variety of computing systems, which may include a single processor, multiple processors, multi-core processors, or any combination thereof.

[0051] FIG. 3 is a component block diagram illustrating a software architecture 300 including radio protocol stacks for user and control planes in wireless communications suitable for implementing any of the various embodiments. With reference to FIGS. 1-3, a wireless device 320 may implement the software architecture 300 to facilitate communication between the wireless device 320 (e.g., wireless devices 120a-120e, 200) and a base station 350 (e.g., base station 110a) of a communication system (e.g., 100). In various embodiments, layers in the software architecture 300 may form logical connections with corresponding layers in the software of the base station 350. The software architecture 300 may be distributed among one or more processors (e.g., processors 212, 214, 216, 218, 252, 260). Although illustrated with respect to one radio protocol stack in a multi-SIM (Subscriber Identity Module) wireless device, software architecture 300 may include multiple protocol stacks, each associated with a different SIM (such as two protocol stacks associated with two SIMs, respectively, in a dual-SIM wireless communication device). Although described below with respect to an LTE communication layer, software architecture 300 may support any of a variety of standards and protocols for wireless communications and / or may include additional protocol stacks supporting any of a variety of standards and protocols for wireless communications.

[0052] The software architecture 300 may include a non-access stratum (NAS) 302 and an access stratum (AS) 304. The NAS 302 may include functions and protocols for packet filtering, security management, mobility control, session management, and supporting traffic and signaling between a wireless device's SIM (e.g., SIM 204) and its core network 140. The AS 304 may include functions and protocols for supporting communications between a SIM (e.g., SIM 204) and supported access network entities (e.g., base stations). In particular, the AS 304 may include at least three layers (Layer 1, Layer 2, and Layer 3), each of which may include various sublayers.

[0053] In the user and control plane, Layer 1 (L1) of the AS 304 may be a physical layer (PHY) 306, which may oversee functions that enable transmission and / or reception over the air interface via a wireless transceiver (e.g., 256). Examples of such physical layer 306 functions may include cyclic redundancy check (CRC) attachment, coding blocks, scrambling and descrambling, modulation and demodulation, signal measurement, MIMO, etc. The physical layer may include various logical channels, including a physical downlink control channel (PDCCH) and a physical downlink shared channel (PDSCH).

[0054] In the user and control plane, Layer 2 (L2) of the AS 304 may carry the link between the wireless device 320 and the base station 350 on the physical layer 306. In various embodiments, Layer 2 may include a Medium Access Control (MAC) sublayer 308, a Radio Link Control (RLC) sublayer 310, and a Packet Data Convergence Protocol (PDCP) sublayer 312, each of which forms a logical connection that terminates at the base station 350.

[0055] In the control plane, Layer 3 (L3) of the AS 304 may include a Radio Resource Control (RRC) sublayer 3. Although not shown, the software architecture 300 may include additional Layer 3 sublayers, as well as various upper layers above Layer 3. In various embodiments, the RRC sublayer 313 may provide functions including broadcasting system information, paging, and establishing and releasing RRC signaling connections between the wireless device 320 and the base station 350.

[0056] In various embodiments, the PDCP sublayer 312 may provide uplink functions including multiplexing between different radio bearers and logical channels, sequence numbering, handover data processing, integrity protection, ciphering, and header compression. In the downlink, the PDCP sublayer 312 may provide functions including in-order delivery of data packets, duplicate data packet detection, integrity verification, decryption, and header recovery.

[0057] In the uplink, the RLC sublayer 310 may provide segmentation and concatenation of upper layer data packets, retransmission of lost data packets, and automatic repeat request (ARQ). In the downlink, the RLC sublayer 310 functions may include reordering of data packets to compensate for out-of-order reception, reassembly of upper layer data packets, and ARQ.

[0058] In the uplink, the MAC sublayer 308 may provide functions including multiplexing between logical and transport channels, random access procedures, logical channel priorities, and Hybrid ARQ (HARQ) operations. In the downlink, MAC layer functions may include channel mapping within a cell, demultiplexing, discontinuous reception (DRX), and HARQ operations.

[0059] While the software architecture 300 may provide functionality for transmitting data over a physical medium, the software architecture 300 may further include at least one host layer 314 for providing data transfer services to various applications in the wireless device 320. In some embodiments, the application-specific functionality provided by the at least one host layer 314 may provide an interface between the software architecture and the general-purpose processor 206.

[0060] In other embodiments, software architecture 300 may include one or more upper logical layers (e.g., transport, session, presentation, application, etc.) that provide host layer functionality. For example, in some embodiments, software architecture 300 may include a network layer (e.g., Internet Protocol (IP) layer) where logical connections terminate at a Packet Data Network (PDN) Gateway (PGW). In some embodiments, software architecture 300 may include an application layer where logical connections terminate at another device (e.g., an end user device, a server, etc.). In some embodiments, software architecture 300 may further include a hardware interface 316 between physical layer 306 and communications hardware (e.g., one or more radio frequency (RF) transceivers) within AS 304.

[0061] 4A and 4B are component block diagrams illustrating a system 400 configured for processing data using computing resources of a remote networked computing device, according to various embodiments. With reference to FIGS. 1-4B, the system 400 may include a wireless device 402 (e.g., 120a-120e, 200, 320) and a networked computing device 404 (e.g., 120a-120e, 200, 320). The wireless device 402 and the networked computing device 404 may communicate over a wireless communications network 424 (aspects of which are shown in FIG. 1).

[0062] 4A, the wireless device 402 may include one or more processors 428 coupled to electronic storage 426 and a wireless transceiver (e.g., 266). The wireless transceiver 266 may be configured to receive messages to be transmitted in uplink transmissions from the processor 428 and transmit such messages via an antenna (not shown) to the wireless communications network 424 for relay to the network computing device 404. Similarly, the wireless transceiver 266 may be configured to receive messages from the network computing device 404 in downlink transmissions from the wireless communications network 424 and pass the messages to the one or more processors 428 (e.g., via a modem (e.g., 252) that demodulates the messages).

[0063] The processor 428 may be configured with machine-readable instructions 406. The machine-readable instructions 406 may include one or more instruction modules. The instruction modules may include computer program modules. The instruction modules may include one or more of a factor determination module 408, a metadata module 410, a transmit and receive (TX / RX) module 412, a processing time adjustment module 414, or other instruction modules.

[0064] The factor determination module 408 may be configured to determine factors that affect the round trip latency of a computing task.

[0065] The metadata module 410 may be configured to generate second metadata that includes the determined factors and a latency budget for the computing task.

[0066] The TX / RX module 412 may be configured, for example, to send and receive messages to and from the network computing device 404. The TX / RX module 412 may be configured to send metadata and data to the remote network computing device 404 for processing as part of a computing task. The TX / RX module 412 may be configured to receive processed data of the computing task and second metadata from the remote network computing device 404, the second metadata including an indication of the time remaining in the latency budget.

[0067] The processing time adjustment module 414 may be configured to adjust the processing time for post-processing the processed data based on the second metadata, so as to complete the post-processing of the processed data within the latency budget.

[0068] 4B, the network computing device 404 may include one or more processors 432 coupled to electronic storage 430 and a wireless transceiver 406. The wireless transceiver 406 may be configured to receive messages to be transmitted in an uplink transmission from the processor 432 and transmit such messages via an antenna (not shown) to the wireless communications network 424 for relay to the wireless device 402. Similarly, the wireless transceiver 406 may be configured to receive messages from the wireless device 402 in a downlink transmission from the wireless communications network 424 and pass the messages to the one or more processors 432 (e.g., via a modem (e.g., 252) that demodulates the messages).

[0069] The processor 432 may be configured with machine-readable instructions 434. The machine-readable instructions 406 may include one or more instruction modules. The instruction modules may include computer program modules. The instruction modules may include one or more of a metadata module 436, a processing time adjustment module 438, a TX / RX module 440, or other instruction modules.

[0070] The metadata module 436 may be configured to receive first metadata and data from a remote wireless device for processing as part of a computing task, the first metadata including factors affecting round trip latency and a latency budget for the computing task, and generate second metadata including an indication of the time remaining in the latency budget.

[0071] The processing time adjustment module 438 may be configured to adjust the processing time for processing the data based on the metadata and the latency budget.

[0072] The TX / RX module 440 may be configured, for example, to send and receive messages to and from the wireless device 402. The TX / RX module 440 may be configured to send the processed data and the second metadata to the remote wireless device in a format that enables post-processing by the remote wireless device.

[0073] In some embodiments, the wireless device 402 and the network computing device 404 may be operatively linked via one or more electronic communications links. For example, such electronic communications links may be established at least in part via a network, such as the Internet and / or other networks. This is not intended to be limiting, and it will be appreciated that the scope of the present disclosure includes embodiments in which the wireless device 402 and the network computing device 404 may be operatively linked via some other communications medium.

[0074] The electronic storage 426, 430 may include non-transitory storage media that electronically store information. The electronic storage media of the electronic storage 426, 430 may include one or both of system storage that is integral (i.e., substantially non-removable) with the wireless device 402 or the networked computing device 404, and / or removable storage that is removably connectable to the wireless device 402 or the networked computing device 404, for example, via a port (e.g., a Universal Serial Bus (USB) port, a Firewire port, etc.) or a drive (e.g., a disk drive, etc.). The electronic storage 426, 430 may include one or more of an optically readable storage medium (e.g., an optical disk, etc.), a magnetically readable storage medium (e.g., a magnetic tape, a magnetic hard drive, a floppy drive, etc.), a charge-based storage medium (e.g., an EEPROM, a RAM, etc.), a solid-state storage medium (e.g., a flash drive, etc.), and / or other electronically readable storage medium. The electronic storage 426, 430 may include one or more virtual storage resources (e.g., cloud storage, virtual private networks, and / or other virtual storage resources). The electronic storage 426, 430 may store software algorithms, information determined by the processors 428, 432, information received from the wireless device 402 or network computing device 404, or other information that enables the wireless device 402 or network computing device 404 to function as described herein.

[0075] The processors 428, 432 may be configured to provide information processing capabilities within the wireless device 402 and the network computing device 404. As such, the processors 428, 432 may include one or more of a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and / or other mechanisms for electronically processing information. Although the processors 428, 432 are shown as a single entity, this is for illustrative purposes only. In some embodiments, the processors 428, 432 may include multiple processing units and / or processor cores. The processing units may be physically located within the same device, or the processors 428, 432 may represent the processing functions of multiple devices operating in cooperation. The processors 428, 432 may be configured to execute the modules 408-414 and modules 436-440 and / or other modules as software, hardware, firmware, some combination of software, hardware and / or firmware, and / or other mechanisms for configuring the processing capabilities on the processors 428, 432. As used herein, the term "module" may refer to any component or set of components that implements the functionality attributed to the module, which may include one or more physical processors in execution of processor-readable instructions, processor-readable instructions, circuitry, hardware, storage media, or any other component.

[0076] The description of the functionality provided by the various modules 408-414 and modules 436-440 described below is for purposes of illustration and is not intended to be limiting, as any of the modules 408-414 and modules 436-440 may provide more or less functionality than described. For example, one or more of the modules 408-414 and modules 436-440 may be excluded, with some or all of its functionality being provided by the other modules 408-414 and modules 436-440. As another example, the processors 428, 432 may be configured to execute one or more additional modules that may implement some or all of the functionality attributed to one of the following modules 408-414 and modules 436-440.

[0077] FIG. 5A is a conceptual diagram illustrating factors 500a that affect round trip latency. With reference to FIGS. 1-5A, the factors 500a may be monitored and / or adjusted by a processor of a wireless device (e.g., 120a-120e, 200, 320, 402) and / or a processor of a network computing device (e.g., 125, 200, 404). As discussed above, some aspects of the performance of a computing task may be stochastic and therefore difficult or impractical to adjust or control. In some embodiments, a processor of a wireless device and / or a network computing device may monitor one or more stochastic factors 520, such as communication delays within a communication network, communication delays outside a communication network (e.g., an intranet, the Internet, etc.), image rendering workload or task complexity (e.g., a 3D rendering workload, or the number of polygons in an XR task), and / or a processor workload in an autonomous or semi-autonomous vehicle. Examples of processor workloads affected by stochastic factors may include performing tasks such as image or object recognition, one or more artificial intelligence (AI) processes, pilot control processes, and other similar processes. In some embodiments, the processor of the wireless device and / or the networked computing device may communicate the monitored stochastic factors to another computing device (i.e., from the wireless device to the networked computing device or vice versa).

[0078] In some embodiments, a processor of a wireless device and / or network computing device may dynamically adjust one or more controllable factors 522 based at least in part on the stochastic factors 520. Examples of controllable factors 522 include the operating frequency of one or more processors (such as a CPU, GPU, DSP, or another suitable processor) and / or the transfer rate of a memory component, such as a DDR memory, which may affect the processing power of the processor.

[0079] FIG. 5B is a timeline illustrating a latency budget model 500b according to various embodiments. Referring to FIGS. 1-5B, the latency budget model 500b may be used by a processor of a wireless device (e.g., 120a-120e, 200, 320, 402) and / or a processor of a network computing device (e.g., 125, 200, 404). The budget model may include a latency budget 502 (which may be represented as T_budget) indicating a maximum latency for a computing task. The latency budget may include various factors that affect the round-trip latency of the computing task. In various embodiments, the factors that affect the round-trip latency of the computing task may include a wireless device pre-processing time 504, a first communication time 506 from the wireless device to a remote network computing device, a remote network computing device processing time 508, a second communication time 510 from the remote network computing device to the wireless device, and a wireless device post-processing time 512. In various embodiments, the latency budget model 500b may include one or more aspects of the factors that affect round-trip latency 500a (FIG. 5A), including stochastic factors 520 and controllable factors 522.

[0080] In some embodiments, the wireless device pre-processing time 504 (which may be represented as T_WD_sensing) may include the time for the wireless device processor to receive data from the sensors, perform some amount of processing of the received data, and / or prepare the data for transmission to the network computing device. The wireless device pre-processing time 504 may vary based on, for example, the complexity of the sensor data, the amount of sensor data, the processing capabilities of the wireless device, and other factors.

[0081] In some embodiments, a first communication time 506 (which may be represented as T_comm_TX) from a wireless device to a remote network computing device may include the time it takes for data sent by the wireless device to be conveyed by the communication system to the network computing device. The first communication time 506 may vary based on, for example, network congestion, one or more communication link conditions (such as signal noise, interference, throughput, bandwidth, or another suitable communication link condition).

[0082] In some embodiments, the remote network computing device processing time 508 (which may be represented as T_cloud_process) may include the time for a processor of the network computing device to receive the processed data, perform processing on it, and / or prepare it for transmission to the wireless device. The remote network computing device processing time 508 may vary based on, for example, the complexity of the sensor data, the amount of sensor data, the processing capabilities of the wireless device, and other factors. The remote network computing device processing time 508 may also vary based on the compression ratio of the data received from the wireless device.

[0083] A second communication time 510 (which may be represented as T_comm_RX) from the remote network computing device to the wireless device may include the time it takes for processed data sent by the network computing device to be conveyed by the communication system to the wireless device. The second communication time 510 may vary based on, for example, network congestion, one or more communication link conditions (such as signal noise, interference, throughput, bandwidth, or another suitable communication link condition).

[0084] The wireless device post-processing time 512 (which may be represented as T_WD_output) may include the time it takes for a processor of the wireless device to receive the processed data from the network computing device, decompress the data received from the network computing device (e.g., based on a compression ratio of the processed data), perform post-processing on the received data, and present the post-processed data (e.g., via an output device of the wireless device), as well as other factors.

[0085] 6 is a block diagram illustrating a message 600 according to various embodiments. Referring to FIGs. 1-6, the message 600 may be generated and sent by a processor of a wireless device (e.g., 120a-120e, 200, 320, 402) and / or a processor of a network computing device (e.g., 125, 320, 404).

[0086] The message 600 may be sent from the wireless device and / or the network computing device from time to time, for example, once per data transmission frame. In some embodiments, the message 600 may include metadata 602 and data 604 associated with the computing task, such as sensor data, image data, control commands, visual data (i.e., data associated with the XR application, such as posture data, image data, virtual images or text, etc.) and other suitable data associated with the computing task. In some embodiments, the metadata 602 may include a frame ID 606 to indicate an association with a particular data frame, a latency budget 502, and factors that affect the round trip latency of the computing task, including a wireless device pre-processing time 504, a first communication time from the wireless device to the remote network computing device 506, a remote network computing device processing time 508, a second communication time from the remote network computing device to the wireless device 510, and a wireless device post-processing time 512.

[0087] 7 is a process flow diagram illustrating a method 700 that may be performed by a processor of a wireless device for processing data using computing resources of a remote networked computing device, according to various embodiments. With reference to FIGS. 1-7, the method 700 may be implemented by a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 428, etc.) of a wireless device (e.g., 120a-120e, 200, 320, 402, etc.).

[0088] At block 702, a processor may determine factors that affect the round trip latency of the computing task. In some embodiments, the factors that affect the round trip latency of the computing task may include a wireless device pre-processing time 504, a first communication time 506 from the wireless device to a remote network computing device, a remote network computing device processing time 508, a second communication time 510 from the remote network computing device to the wireless device, and a wireless device post-processing time 512 (FIG. 5). Means for performing the functions of the operations at block 702 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 428).

[0089] At block 704, the processor may generate first metadata including the determined factors and a latency budget for the computing task. For example, the processor may generate the first metadata 602 (FIG. 6). The means for performing the functions of the operations at block 704 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 428).

[0090] At block 706, the processor may send the first metadata and data to a remote network computing device for processing as part of a computing task. For example, the processor may send message 600 (FIG. 6) to the remote network computing device. Means for performing the functions of the operations at block 706 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432) coupled to a wireless transceiver (e.g., 266).

[0091] At block 708, the processor may receive from the network computing device second metadata including the processed data of the computing task and an indication of the remaining time in the latency budget. In some embodiments, the wireless device may receive a message similar to message 600 (FIG. 6) from a remote network computing device. In some embodiments, the wireless device may receive data from the network computing device that has been processed by the network computing device but is not usable as received by the wireless device without additional processing by the wireless device (i.e., the data received from the wireless device is not a completed work result). Means for performing the functions of the operations at block 706 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432) coupled to a wireless transceiver (e.g., 266).

[0092] In block 710, the processor may adjust the processing time for post-processing the processed data based on the second metadata to complete the post-processing of the processed data within the latency budget. In some embodiments, the processor may dynamically adjust one or more of the operating frequency of one or more processors (such as a CPU, GPU, DSP, or another suitable processor) and / or the transfer rate of a memory component such as a DDR memory, which may affect the processing power of the processor. The means for performing the functions of the operations in block 706 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0093] The processor may, from time to time, re-perform the operations of blocks 702-710. In this manner, the wireless device may dynamically track the latency of performance of the computing tasks compared to the latency budget for the computing tasks, and dynamically adjust the timing of performance of the computing tasks to meet the latency budget.

[0094] 8-10 are process flow diagrams illustrating operations 800-1000 that may be performed by a processor of a wireless device as part of a method for processing data using computing resources of a remote networked computing device, according to various embodiments. With reference to FIGS. 1-10, operations 800, 900, 1000 may be implemented by a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 428) of a wireless device (e.g., 120a-120e, 200, 320, 402).

[0095] 8, following the operation of block 708 of method 700, the processor may adjust dynamic clock and voltage scaling (DCVS) and dynamic task priority assignment of the processed data based on the time remaining in the latency budget at block 802. In some embodiments, a relatively high task priority may cause the processor to process the processed data before data assigned a lower priority. Means for performing the functions of the operation at block 802 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0096] The processor may then perform the operations of block 702 of method 700 as described.

[0097] 9, following the operation of block 708 of method 700, the processor may adjust the DCVS and the task queue position of the processed data based on the time remaining in the latency budget at block 902. In some embodiments, the task queue position may affect the time the processor processes the processed data relative to other data for processing in the task queue. Means for performing the functions of the operation at block 902 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0098] The processor may then perform the operations of block 702 of method 700 as described.

[0099] 10, following the operations of block 704 of method 700, the processor may adjust the compression ratio of the data for processing as part of the computing task based on the determined factors and the latency budget at block 1002. Means for performing the functions of the operations at block 1002 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0100] The processor may then perform the operations of block 706 of method 700 as described.

[0101] 11 is a process flow diagram illustrating a method 1100 that may be performed by a processor of a network computing device for processing data with the assistance of a remote wireless device, according to various embodiments. With reference to FIGS. 1-11, the method 1100 may be implemented by a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 428) of a network computing device (e.g., 200, 125, 404).

[0102] At block 1102, a processor may receive first metadata and data from a remote wireless device for processing as part of a computing task, the first metadata including factors affecting round-trip latency and a latency budget for the computing task. For example, the processor may receive a message similar to message 600 (FIG. 6). In some embodiments, the factors affecting the round-trip latency of the computing task may include a wireless device pre-processing time 504, a first communication time 506 from the wireless device to a remote network computing device, a remote network computing device processing time 508, a second communication time 510 from the remote network computing device to the wireless device, and a wireless device post-processing time 512 (FIG. 5). Means for performing the functions of the operations at block 1102 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432) coupled to a wireless transceiver (e.g., 406).

[0103] In block 1104, the processor may adjust the processing time for processing the data based on the first metadata and the latency budget. The means for performing the functions of the operations in block 1104 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0104] At block 1106, the processor may generate second metadata including an indication of the time remaining in the latency budget. Means for performing functions of the operations at block 1106 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0105] In block 1108, the processor may send the processed data and the second metadata to the remote wireless device in a format that enables post-processing by the remote wireless device. Means for performing the functions of the operations in block 1108 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432) coupled to the wireless transceiver (e.g., 406).

[0106] The processor may re-perform the operations of blocks 1102-1108 from time to time. In this manner, the network computing device may dynamically track the latency of performance of a computing task compared to a latency budget for the computing task, and dynamically adjust the timing of performance of the computing task to meet the latency budget.

[0107] 12-14 are process flow diagrams illustrating operations 1200, 1300, 1400 that may be performed by a processor of a wireless device as part of a method for processing data using computational resources of a remote networked computing device, according to various embodiments. With reference to FIGS. 1-14, the operations 1200, 1300, 1400 may be implemented by a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 428) of a networked computing device (e.g., 125, 320, 404).

[0108] 12, following the operations of block 1102 of method 1100, a processor may adjust dynamic clock and voltage scaling (DCVS) and dynamic task priority of data based on the time remaining in the latency budget in block 1202. Means for performing the functions of the operations in block 1202 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0109] The processor may then perform the operations of block 1106 of method 1100 as described.

[0110] 13, following the operations of block 1102 of method 1100, a processor may adjust the DCVS and task queue position of the data based on the time remaining in the latency budget at block 1302. Means for performing the functions of the operations at block 1302 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0111] The processor may then perform the operations of block 1106 of method 1100 as described.

[0112] 14, following the operations of block 1106 of method 1100, the processor may adjust a compression ratio of the data for processing as part of the computing task based on the determined factors and the latency budget at block 1402. Means for performing the functions of the operations at block 1402 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0113] The processor may then perform the operations of block 1108 of method 1100 as described.

[0114] Various embodiments including the methods and operations 1100-1400 may be implemented in various network computing devices (e.g., in a server device), an example of which is shown in FIG. 15, which is a component block diagram of a network computing device 1500 suitable for use with various embodiments. Such a network computing device may include at least the components shown in FIG. 15. With reference to FIGS. 1-15, the network computing device 1500 may include a processor 1501 coupled to a volatile memory 1502 (e.g., 430) and a large capacity non-volatile memory such as a disk drive 1503. The network computing device 1500 may also include a peripheral memory access device, such as a floppy disk drive, compact disk (CD), or digital video disk (DVD) drive 1506, coupled to the processor 1501. The network computing device 1500 may also include a network access port 1504 (or interface) coupled to the processor 1501 for establishing a data connection with a network, such as the Internet and / or a local area network coupled to other system computers and servers. The network computing device 1500 may be connected to one or more antennas for sending and receiving electromagnetic radiation, which may be connected to a wireless communications link. The network computing device 1500 may include additional access ports, such as USB, Firewire, Thunderbolt, etc., for coupling to peripherals, external memory, or other devices.

[0115] Various embodiments, including the methods and operations 700, 800, 900, 1000, may be implemented in a variety of wireless devices (e.g., wireless devices 120a-120e, 200, 320, 402), an example of which is shown in FIG. 16, which is a component block diagram of a wireless device 1600 suitable for use with various embodiments. With reference to FIGS. 1-16, the wireless device 1600 may include a first SOC 202 (e.g., SOC-CPU) coupled to a second SOC 204 (e.g., a 5G-enabled SOC). The first SOC 202 and the second SOC 204 may be coupled to an internal memory 430, 1616, a display 1612, and a speaker 1614. Additionally, the wireless device 1600 may include an antenna 1604 for transmitting and receiving electromagnetic radiation, which may be connected to a wireless data link and / or cellular telephone transceiver 266 coupled to one or more processors in the first SOC 202 and / or second SOC 204. The wireless device 1600 may also include menu selection buttons or rocker switches 1620 for receiving user input.

[0116] The wireless device 1600 may also include a voice encoding / decoding (CODEC) circuit 1610 that digitizes sound received from a microphone into data packets suitable for wireless transmission and decodes the received sound data packets to generate analog signals that are provided to a speaker to generate sound. One or more of the processors, the wireless transceiver 266, and the CODEC 1610 in the first SOC 202 and the second SOC 204 may also include digital signal processor (DSP) circuitry (not separately shown).

[0117] The processors of the network computing device 1500 and the wireless device 1600 may be any programmable microprocessor, microcomputer, or one or more multi-processor chips that may be configured by software instructions (applications) to perform various functions, including those of the various embodiments described below. In some wireless devices, multiple processors may be provided, such as one processor in the SOC 204 dedicated to wireless communication functions and one processor in the SOC 202 dedicated to running other applications. Software applications may be stored in memory 426, 430, 1616 before being accessed and loaded into the processor. The processors may include sufficient internal memory to store application software instructions.

[0118] As used in this application, terms such as "component," "module," and "system" are intended to include, but are not limited to, computer-related entities, such as hardware, firmware, a combination of hardware and software, software, or software in execution, configured to perform certain operations or functions. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and / or a computer. By way of example, both an application running on a wireless device and the wireless device may be referred to as a component. One or more components may reside within a process and / or thread of execution, and a component may reside on one processor or core and / or be distributed among two or more processors or cores. Additionally, these components may execute from various non-transitory computer-readable media having various instructions and / or data structures stored thereon. Components may communicate by way of local and / or remote processes, function or procedure calls, electronic signals, data packets, memory reads / writes, and other known network, computer, processor, and / or process related communication methods.

[0119] A number of different cellular and mobile communication services and standards are available or are contemplated in the future, all of which may implement and benefit from the various embodiments. Such services and standards include, but are not limited to, Third Generation Partnership Project (3GPP®), Long Term Evolution (LTE) systems, third generation wireless mobile communication technology (3G), fourth generation wireless mobile communication technology (4G), fifth generation wireless mobile communication technology (5G), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), 3GSM, General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA) systems (cdmaOne, CDMA1020™, etc.), Enhanced Data Rates for GSM Evolution (EDGE), Advanced Mobile Phone System (AMPS), Digital AMPS (IS-136 / TDMA), Evolution Data Optimized (EV-DO), Digital Enhanced Cordless Telecommunications (DECT), and the like. Examples of common standards include wireless telecommunications, Worldwide Interoperability for Microwave Access (WiMAX), Wireless Local Area Networks (WLAN), Wi-Fi Protected Access I & II (WPA, WPA2), and Integrated Digital Enhanced Network (iDEN). Each of these technologies involves, for example, the transmission and reception of voice, data, signaling, and / or content messages. It should be understood that any reference to terminology and / or technical details relating to a particular telecommunications standard or technology is for illustrative purposes only and does not limit the scope of the claims to a particular communications system or communications technology unless specifically recited in the claim language.

[0120] The various embodiments shown and described are provided merely as examples to illustrate various features of the claims. However, features shown and described with respect to any given embodiment are not necessarily limited to the associated embodiment, and may be used with or combined with other embodiments shown and described. Furthermore, the claims are not intended to be limited by any one exemplary embodiment. For example, one or more of the operations of the methods disclosed above may be substituted for or combined with one or more operations of the methods disclosed above.

[0121] The above method descriptions and process flow diagrams are provided as illustrative examples only and do not require or imply that the operations of the various embodiments must be performed in the order presented. As will be appreciated by one of ordinary skill in the art, the order of operations in the above-described embodiments may be performed in any order. Terms such as "then," "then," and "next" do not limit the order of operations, but rather these terms are used to guide the reader through the method descriptions. Additionally, any reference to a claim element in the singular, for example using the articles "a," "an," or "the," should not be construed as limiting the element to the singular.

[0122] The various exemplary logic blocks, modules, components, circuits, and algorithmic operations described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various exemplary components, blocks, modules, circuits, and operations have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the claims.

[0123] The hardware used to implement the various example logic, logic blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed using general purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or combinations thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of receiver smart objects, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuitry specific to a given function.

[0124] In one or more embodiments, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a non-transitory computer-readable or processor-readable storage medium. The operations of the methods or algorithms disclosed herein may be embodied in a processor-executable software module or processor-executable instructions that may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium may be any storage medium that may be accessed by a computer or processor. By way of example and not limitation, such a non-transitory computer-readable or processor-readable storage medium may include RAM, ROM, EEPROM, FLASH memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage smart objects, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically and discs reproduce data optically using lasers. Combinations of the above are also included within the scope of non-transitory computer-readable media and non-transitory processor-readable media. Furthermore, operations of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transitory processor-readable storage medium and / or a non-transitory computer-readable storage medium, which may be embodied in a computer program product.

[0125] The foregoing description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the claims. Thus, the present disclosure is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein. [Explanation of symbols]

[0126] 100 Communication Systems 110 BS, base station 120 Wireless Devices 122 Wireless Communication Links 124 Wireless Communication Link, Sidelink Channel 125 Network Computing Devices 126 Wired or Wireless Communications Links 130 Network Controller 140 Core Network, Communication Network 200 Wireless modem system, computing system, SIP, wireless device 202 SOC 204 SOC, SIM 206 Clock 208 Voltage Regulator 210 Digital Signal Processor (DSP), Processor 212 Modem Processor, Processor 214 Graphics Processor, Processor 216 Application Processor, Processor 218 Coprocessor, processor 220 Memory, memory elements 222 Custom Circuit Configuration 224 System Components and Resources 226 Interconnect / Bus Modules 230 Temperature Sensor 232 Thermal Management Unit 234 Thermal Power Envelope (TPE) Components 250 Interconnect / Bus Modules 252 5G modem processor, processor 254 Power Management Unit 256 mmWave Transceiver 258 memory 260 additional processors, processors 264 Interconnect / Bus Modules 266 Wireless Transceiver, Cellular Telephone Transceiver 300 Software Architecture 320 Wireless Devices 350 base station 400 Systems 402 Wireless Devices 404 Network Computing Devices 406 Wireless Transceiver 408 Factor Judgment Module, Module 410 Metadata Module, Module 412 Transmit and receive (TX / RX) module, module 414 Processing time adjustment module, module 424 Wireless Communication Network 426 Electronic Storage, Memory 428 processor 430 Electronic storage, internal memory, memory 432 processor 436 Metadata Module, Module 438 Processing time adjustment module, module 440 TX / RX module, module 1500 Network Computing Devices 1501 Processor 1502 Volatile Memory 1503 Disk Drive 1504 Network Access Port 1506 Floppy disk drive, compact disk (CD), digital video disk (DVD) drive 1600 Wireless Devices 1604 Antenna 1610 Voice encoding / decoding (codec) circuit 1612 Display 1614 Speaker 1616 Internal memory, memory 1620 Rocker Switch

Claims

1. 1. A wireless device implemented method for processing data using computing resources of a remote networked computing device, comprising: determining an amount of time required to complete each of a plurality of operations required to complete a computing task; generating first metadata including the determined times and a latency budget for the computing task, the latency budget being a time for completion of the computing task; transmitting the first metadata, the data, and an amount of time elapsed since the start of the computing task to the remote network computing device for processing as part of the computing task; receiving second metadata from the remote network computing device, the second metadata including processed data for the computing task and an indication of the remaining time in the latency budget; and adjusting dynamic clock and voltage scaling (DCVS) of one or more processors of the wireless device for post-processing the processed data based on the second metadata to complete the post-processing of the processed data within the latency budget, the post-processing including one of the plurality of operations.

2. The method of claim 1, wherein the step of determining the multiple times includes determining one or more of a wireless device pre-processing time, a first communication time from the wireless device to the remote network computing device, a remote network computing device processing time, a second communication time from the remote network computing device to the wireless device, or a wireless device post-processing time.

3. The method of claim 1, further comprising adjusting a dynamic task priority assignment of the processed data based on time remaining within the latency budget.

4. The method of claim 1, further comprising adjusting a task queue position of the processed data based on time remaining in the latency budget.

5. The method of claim 1 , wherein the processed data of the computing task received from the remote networked computing device requires post-processing to complete the computing task.

6. The method of claim 1, further comprising adjusting a compression ratio of the data for processing as part of the computing task based on the plurality of times and the latency budget before transmitting the data to the remote network computing device.

7. 1. A wireless device comprising at least one processor, the at least one processor comprising: determining an amount of time required to complete each of a plurality of operations required to complete a computing task; generating first metadata comprising a plurality of times and a latency budget for the computing task, the latency budget being a time for completion of the computing task; transmitting the first metadata, the data, and the elapsed time since the start of the computing task to a remote networked computing device for processing as part of the computing task; receiving second metadata from the remote network computing device, the second metadata including processed data for the computing task and an indication of the remaining time in the latency budget; and adjusting dynamic clock and voltage scaling (DCVS) of one or more processors of the wireless device for post-processing the processed data based on the second metadata to complete the post-processing of the processed data within the latency budget, the post-processing including one of the plurality of operations.

8. The wireless device of claim 7, wherein the at least one processor is further configured with processor-executable instructions for performing the method of any one of claims 2 to 6.

9. 1. A method implemented by a network computing device for processing data with assistance to a remote wireless device, comprising: receiving first metadata and data from a remote wireless device for processing as part of a computing task, the first metadata including a time required to complete each of a plurality of operations required to complete the computing task, an elapsed time from start of the computing task, and a latency budget for the computing task, the latency budget being a time for completion of the computing task; adjusting dynamic clock and voltage scaling (DCVS) of one or more processors of the network computing device for processing the data based on the first metadata and the latency budget, where processing the data comprises one of the plurality of operations; generating second metadata including an indication of the time remaining in the latency budget; sending the processed data and the second metadata to the remote wireless device in a format that enables post-processing by the remote wireless device.

10. 10. The method of claim 9, wherein receiving first metadata and data from the remote wireless device for processing as part of a computing task includes receiving one or more of a remote wireless device pre-processing time, a first communication time from the remote wireless device to the network computing device, a network computing device processing time, a second communication time from the network computing device to the remote wireless device, or a remote wireless device post-processing time.

11. 10. The method of claim 9, wherein adjusting a processing time for processing the data based on the first metadata and the latency budget comprises adjusting a dynamic task priority assignment of the data based on time remaining within the latency budget.

12. 10. The method of claim 9, wherein adjusting a processing time for processing the data based on the first metadata and the latency budget further comprises adjusting a task queue position of the data based on time remaining within the latency budget.

13. The method of claim 9, further comprising adjusting a compression ratio of the processed data based on a plurality of times and the latency budget.

14. 1. A network computing device comprising at least one processor, the at least one processor comprising: receiving first metadata and data from a remote wireless device for processing as part of a computing task, the first metadata including a time required to complete each of a plurality of operations required for completion of the computing task, an elapsed time from a start of the computing task, and a latency budget for the computing task, the latency budget being a time for completion of the computing task; adjusting dynamic clock and voltage scaling (DCVS) of one or more processors of the network computing device for processing the data based on the first metadata and the latency budget; generating second metadata including an indication of time remaining in the latency budget; and sending the processed data and the second metadata to the remote wireless device in a format that enables post-processing by the remote wireless device.

15. The network computing device of claim 14, wherein the at least one processor is further configured with processor-executable instructions for performing the method of any one of claims 10 to 13.

Images