Processing data using remote network computing resources

Abstract

Embodiments include methods for processing of compute resource management data using a remote network computing device and performing a compute task of a wireless device. The wireless device and the network computing device can dynamically track factors that affect round-trip latency for the compute task. The wireless device and the network computing device can generate and send metadata including the factors and a latency budget for the compute task. The wireless device and the network computing device can adjust a processing time for processing data related to the compute task based on the received metadata and the latency budget.

Description

Background Technology

[0001] Some computational tasks place high demands on the relatively limited processing power of modern wireless devices. In some cases, wireless devices can leverage high-speed wireless communication and network computing resources to offload processor-intensive computational tasks to computing devices deeper within the network, such as server devices. However, offloading computational tasks to network computing devices introduces latency due to both the time required to perform remote processing and the time it takes for information to travel over the communication link. This introduced latency can prove unacceptable for the performance of certain applications and services, especially those impacting health and security. Summary of the Invention

[0002] The aspects include systems and methods for wireless communication executed by a processor of a wireless device to utilize remote computing resources. The aspects may include determining factors affecting the round-trip latency of a computing task; generating first metadata including the determined factors and a latency budget for the computing task; transmitting the first metadata and data for processing as part of the computing task to a remote network computing device; receiving processed data of the computing task from the network computing device and second metadata including an indication of the remaining time in the latency budget; and adjusting the processing time for post-processing the processed data based on the second metadata to complete the post-processing of the processed data within the latency budget.

[0003] In some respects, determining the factors affecting round-trip latency of a computing task may include determining one or more of the following: wireless device preprocessing time, first communication time from the wireless device to the remote network computing device, remote network computing device processing time, second communication time from the remote network computing device to the wireless device, or wireless device postprocessing time.

[0004] In some aspects, adjusting the processing time for post-processing of processed data based on second metadata to complete the post-processing of processed data within the delay budget may include: adjusting dynamic clock and voltage scaling (DCVS) and dynamic task priority assignment of processed data based on the remaining time in the delay budget. In some aspects, adjusting the processing time for post-processing of processed data based on second metadata to complete the post-processing of processed data within the delay budget may include: adjusting the DCVS and the task queue position of the processed data based on the remaining time in the delay budget.

[0005] In some respects, the processed data for computing tasks received from remote network computing devices may not be a completed work product. Some respects may include adjusting the compression ratio of the data used as part of a computing task based on defined factors and latency budgets.

[0006] The aspects include systems and methods executed by network computing devices for processing data supporting remote wireless devices. The aspects may include receiving metadata from the remote wireless device and data for processing as part of a computing task, the first metadata including factors affecting round-trip latency and a latency budget for the computing task; adjusting the processing time for processing the data based on the first metadata and the latency budget; generating second metadata including an indication of the remaining time in the latency budget; and transmitting the processed data and the second metadata to the remote wireless device in a format that enables post-processing by the remote wireless device.

[0007] In some aspects, receiving first metadata and data for processing as part of a computing task from a wireless device may include one or more of the following: receiving a preprocessing time from a remote wireless device, a first communication time from the remote wireless device to the network computing device, a processing time from the network computing device, a second communication time from the network computing device to the remote wireless device, or a postprocessing time from the remote wireless device.

[0008] In various aspects, adjusting the processing time for data processing based on the initial metadata and latency budget can include adjusting dynamic clock and voltage scaling (DCVS) and dynamic task prioritization of data based on the remaining time in the latency budget. In various aspects, adjusting the processing time for data processing based on the initial metadata and latency budget can include adjusting DCVS and the task queue position of data based on the remaining time in the latency budget. In various aspects, adjusting the compression ratio of processed data can include adjusting the compression ratio of processed data based on factors affecting round-trip latency and the latency budget.

[0009] Another aspect includes a wireless device having a processor configured to perform one or more operations of any of the methods outlined above. Another aspect includes a non-transitory processor-readable storage medium having processor-executable instructions stored thereon, the processor-executable instructions being configured to cause the processor of the wireless device to perform operations of any of the methods outlined above. Another aspect includes a wireless device having components having functionality for performing any of the methods outlined above. Another aspect includes a system-on-chip for use in a wireless device, the system-on-chip including a processor configured to perform one or more operations of any of the methods outlined above. Another aspect includes a network computing device having a processor configured to perform one or more operations of any of the methods outlined above. Another aspect includes a non-transitory processor-readable storage medium having processor-executable instructions stored thereon, the processor-executable instructions being configured to cause the processor of the network computing device to perform operations of any of the methods outlined above. Another aspect includes a network computing device having components having functionality for performing any of the methods outlined above. Another aspect includes a system-on-a-chip for use in a network computing device, the system-on-a-chip including a processor configured to perform one or more operations of any of the methods outlined above. Attached Figure Description

[0010] The accompanying drawings, which are incorporated herein and form part of this specification, illustrate exemplary embodiments of the claims and, together with the general description given above and the detailed description given below, serve to interpret the features of the claims.

[0011] Figure 1 This is a system block diagram illustrating an exemplary communication system suitable for implementing any of the various embodiments.

[0012] Figure 2 This is a block diagram illustrating an exemplary computing and wireless modem system suitable for implementing any of the various embodiments.

[0013] Figure 3 This is a component block diagram illustrating a software architecture including a radio protocol stack for user and control planes in wireless communication, which is adapted to implement any of the various embodiments.

[0014] Figure 4A and Figure 4B This is a block diagram of components of a system configured to manage information transmission of wireless communication performed by a processor of a base station, according to various embodiments.

[0015] Figure 5A This is a conceptual diagram illustrating the factors that affect total delay.

[0016] Figure 5B The diagram illustrates the timeline of the delay budget model according to various embodiments.

[0017] Figure 6 This is a block diagram illustrating messages according to various embodiments.

[0018] Figure 7 This is a flowchart illustrating a method 700 for processing data using computing resources of a remote network computing device, which can be executed by a processor of a wireless device according to various embodiments.

[0019] Figures 8 to 10 This is a flowchart illustrating operations that can be performed by a processor of a wireless device according to various embodiments, as part of a method for processing data using the computing resources of a remote network computing device.

[0020] Figure 11 This is a flowchart illustrating a method for processing data supporting a remote wireless device, which can be executed by a processor of a network computing device according to various embodiments.

[0021] Figures 12 to 14 This is a flowchart illustrating operations that can be performed by a processor of a network computing device according to various embodiments, as part of a method for processing data supporting remote wireless devices.

[0022] Figure 15 This is a component block diagram of a network computing device suitable for use with the various embodiments.

[0023] Figure 16 This is a component block diagram of a wireless device suitable for use with the various embodiments. Detailed Implementation

[0024] Various embodiments will be described in detail with reference to the accompanying drawings. Throughout the drawings, the same reference numerals are used wherever possible to denote the same or similar parts. References to specific examples and embodiments are for illustrative purposes and are not intended to limit the scope of the claims.

[0025] Various embodiments include systems and methods for managing the offloading of computational tasks for data used in services or applications for remote processing of wireless devices. Wireless devices can offload resource-intensive computational tasks to processing devices within a communication network, such as server devices. Various embodiments enable the timing of the execution of computational tasks to be managed within a latency budget, taking into account latency caused by the time required to perform remote processing and the time it takes for information to propagate through the communication link.

[0026] The term "wireless device" as used herein refers to any or all of the following: wireless router devices, wireless appliances, cellular phones, smartphones, portable computing devices, personal or mobile multimedia players, laptop computers, tablet computers, smartbooks, ultrabooks, handheld computers, wireless email receivers, cellular phones with multimedia internet capabilities, medical devices and equipment, biometric sensors / devices, wearable devices including smartwatches, smart clothing, smart glasses, smart wristbands, smart jewelry (e.g., smart rings, smart bracelets, etc.), entertainment devices (e.g., wireless game controllers, music and video players, satellite radios, etc.), Internet of Things (IoT) devices with wireless network capabilities including smart meters / sensors, industrial manufacturing equipment, large and small machines and appliances for home or business use, wireless communication elements in autonomous and semi-autonomous vehicles, wireless devices attached to or incorporated into various mobile platforms, GPS devices, and similar electronic devices including memory, wireless communication components, and programmable processors.

[0027] The term "System-on-a-Chip" (SOC) is used herein to refer to a single integrated circuit (IC) chip containing multiple resources and / or processors integrated on a single substrate. A single SOC may contain circuitry for digital, analog, mixed-signal, and radio frequency functions. A single SOC may also include any number of general-purpose and / or special-purpose processors (digital signal processors, modem processors, video processors, etc.), memory blocks (e.g., ROM, RAM, flash memory, etc.), and resources (e.g., timers, voltage regulators, oscillators, etc.). A SOC may also include software for controlling the integrated resources and processors, as well as software for controlling peripheral devices.

[0028] The term "System-in-Package" (SIP) may be used herein to refer to a single module or package containing multiple resources, computing units, cores, and / or processors on two or more IC chips, substrates, or SoCs. For example, a SIP may include a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may include one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged into a unified substrate. A SIP may also include multiple independent SoCs coupled together via high-speed communication circuitry and packaged in close proximity, such as on a single motherboard or in a single wireless device. Proximity of SoCs facilitates high-speed communication and the sharing of memory and resources.

[0029] Some computational tasks place high demands on the relatively limited processing power of modern wireless devices. For example, computational tasks associated with virtual reality (VR), augmented reality (AR), or mixed reality (MR) applications (sometimes collectively referred to as extended reality (XR)) involve receiving sensor data, processing sensor data, and generating images to be presented in real time on display devices such as head-mounted displays. The user experience associated with such presentations is highly latency-sensitive. Similarly, many computational tasks required for autonomous and semi-autonomous vehicles, such as sensing the vehicle's environment, processing sensor data, and making maneuvering and navigation decisions, are safety-critical and therefore also latency-sensitive. Furthermore, remote robotic operations, such as remote surgery and other telemedicine procedures, also heavily rely on processing latency. These and other similar applications have stringent low-latency requirements, as they require latency to be below a threshold to provide minimum acceptable performance.

[0030] Network computing devices, such as server equipment, can provide computing support to wireless devices, enabling wireless devices to offload processing-intensive computing tasks to network computing devices using, for example, high-speed communication networks (such as 5G New Radio (NR) communication systems). Such systems need to manage latency caused by the time required to perform remote processing and the data propagation time on any communication link.

[0031] However, conventional static solutions to complex processing challenges are unacceptable or inefficient. For example, running a wireless device processor at a relatively high operating frequency may meet the application's latency requirements, but it may push the processor to or beyond acceptable thermal limits and consume power at a rate that will drain the battery in a short time. Furthermore, latency on a communication network can be random, making it functionally impossible to manually adjust the processor speed of a wireless device to accommodate variations in network latency.

[0032] Various embodiments include methods for processing data using computing resources of a remote network computing device, including dynamically tracking latency during the execution of a computing task relative to a latency budget for the computing task. As used herein, the term "latency budget" refers to the time required for a computing task to be completed to meet a performance or quality threshold for that computing task. Remote processing of data for a computing task may involve several operations, each involving an amount of time, such as initial processing of data by a wireless device, transmission time of data from the wireless device to the remote network computing device, processing time of the remote network computing device, transmission time of data from the remote network computing device to the wireless device, and further processing time of the wireless device using data received from the remote network computing device. The time required to complete all operations involved in delegating a computing task to remote computing resources is referred to herein as the "round-trip latency" of the computing task. In various embodiments, the wireless device and the remote network computing device determine and report the time consumed by one or more of the remote processing operations, such that a next device (e.g., the wireless device or the remote network computing device) can adjust the processing time for processing the data for the delegated computing task.

[0033] Various embodiments may include a method executed by a processor of a wireless device for processing data using the computing resources of a remote network computing device. Some embodiments may include determining factors affecting the round-trip latency of a computing task that can be delegated to a remote network computing device, generating first metadata including the determined factors and a latency budget for the computing task, sending the first metadata and data to be processed as part of the computing task to the remote network computing device, receiving the processed data of the computing task and second metadata including an indication of the remaining time in the latency budget from the network computing device, and adjusting the processing time for receiving and / or post-processing the processed data based on the second metadata to complete the entire computing task within the latency budget.

[0034] Various embodiments may include methods executed by a process of a network computing device for processing data supporting a remote wireless device (i.e., remote from the network computing device). In some embodiments, the network computing device may receive first metadata and data for processing as part of a computing task from the remote wireless device. The first metadata may include factors affecting round-trip latency and a latency budget for the computing task. The network computing device may adjust the processing time for processing the data based on the first metadata and the latency budget, and may generate second metadata including an indication of the remaining time in the latency budget. The network computing device may send the processed data and the second metadata to the remote wireless device in a format that allows the remote wireless device to post-process and use it to complete the computing task.

[0035] In some embodiments, a wireless device or network computing device may adjust processing time by adjusting one or more of the operating frequencies of one or more processors (such as a CPU, GPU, DSP, or another suitable processor) and / or the transfer rates of memory components (such as double data rate (DDR) memory) that may affect the processor's processing power. In some embodiments, a wireless device or network computing device may adjust processing time by adjusting the processor's dynamic clock and voltage scaling (DCVS) based on the remaining time in the latency budget and the dynamic task priority assignment of the data to be processed. In some embodiments, a wireless device or network computing device may adjust processing time by adjusting the DCVS based on the remaining time in the latency budget and the position of the data to be processed in the processing queue or task queue ("task queue position"), or by changing the order of the data to be processed (processing order or sequence). In some embodiments, a wireless device may adjust processing time by adjusting the level of detail in the computed output, such as the rendering quality of an image used to be rendered as part of an XR application. In some embodiments, a wireless device can adjust processing time by selecting an appropriate algorithm for the computation task, such as a relatively complex or sophisticated algorithm (e.g., an algorithm that performs a relatively large number of determinations or computations, or uses a relatively large number of factors or criteria to perform operations), or a relatively simple or lightweight algorithm (e.g., an algorithm that performs a relatively small number of determinations or computations, or uses a relatively small number of factors or criteria to perform operations).

[0036] In some embodiments, a wireless device may receive data that has been processed by a network computing device, but if the wireless device has not performed additional processing, the data is unusable upon receipt by the wireless device (i.e., the data received from the wireless device is not a completed work product). In some embodiments, the wireless device or network computing device may adjust the compression ratio of data used for transmission to another device based on determined factors and latency budgets. For example, the wireless device or network computing device may adjust the data compression ratio (e.g., by selecting different data compression algorithms) to increase or decrease the size of data portions (e.g., fragments) based on the level of communication link congestion and / or one or more communication link conditions (such as signal noise, interference, throughput, bandwidth, or another suitable communication link condition).

[0037] Figure 1 This is a system block diagram illustrating an exemplary communication system 100 suitable for implementing any of the various embodiments. The communication system 100 may be a 5G New Radio (NR) network, or any other suitable network, such as a Long Term Evolution (LTE) network.

[0038] Communication system 100 may include a heterogeneous network architecture, which includes a core network 140 and various wireless devices (in... Figure 1 The diagram shows wireless devices 120a-120e. The communication system 100 may also include one or more network computing devices 125 that can communicate with the wireless devices 120a-120e. In some embodiments, the wireless devices 120a-120e may send data to the network computing devices(s)125 for processing as part of a computing task.

[0039] The communication system 100 may also include multiple base stations (illustrated as BS 110a, BS 110b, BS 110c, and BS 110d) and other network entities. A base station is an entity that communicates with wireless devices and may also be referred to as a NodeB, Node B, LTE Evolution NodeB (eNB), Access Point (AP), Radio Headend, Transmit / Receive Point (TRP), New Radio Base Station (NRBS), 5G NodeB (NB), Next Generation NodeB (gNB), etc. Each base station can provide communication coverage for a specific geographic area. In 3GPP, the term "cell" can refer to the coverage area of ​​a base station, a base station subsystem serving that coverage area, or a combination thereof, depending on the context in which the term is used.

[0040] Base stations 110a-110d can provide communication coverage for macrocells, picocells, femtocells, another type of cell, or a combination thereof. A macrocell can cover a relatively large geographic area (e.g., a radius of several kilometers) and can allow unrestricted access for wireless devices with service subscriptions. A picocell can cover a relatively small geographic area and can allow unrestricted access for wireless devices with service subscriptions. A femtocell can cover a relatively small geographic area (e.g., a home) and can allow restricted access for wireless devices associated with that femtocell (e.g., wireless devices in a Closed Subscriber Group (CSG)). A base station used for a macrocell can be referred to as a macro BS. A base station used for a picocell can be referred to as a pico BS. A base station used for a femtocell can be referred to as a femtocell BS or a home BS. Figure 1 In the example illustrated, base station 110a can be a macro BS for macro cell 102a, base station 110b can be a pico BS for pico cell 102b, and base station 110c can be a femto BS for femto cell 102c. Base stations 110a-110d can support one or more (e.g., three) cells. The terms “eNB,” “base station,” “NR BS,” “gNB,” “TRP,” “AP,” “Node B,” “5G NB,” and “cell” are used interchangeably herein.

[0041] In some examples, the cell may not be stationary, and the geographical area of ​​the cell may move depending on the location of the mobile base station. In some examples, base stations 110a-110d may be interconnected with each other and to one or more other base stations or network nodes (not shown) in the communication system 100 using any suitable transport network through various types of backhaul interfaces (such as direct physical connections, virtual networks, or combinations thereof).

[0042] Base stations 110a-110d can communicate with the core network 140 via wired or wireless communication link 126. Wireless devices 120a-120e can communicate with base stations 110a-110d via wireless communication link 122.

[0043] The wired communication link 126 can use various wired networks (e.g., Ethernet, TV cable, telephone, fiber optic and other forms of physical network connection) that can use one or more wired communication protocols, such as Ethernet, point-to-point protocol, advanced data link control (HDLC), advanced data communication control protocol (ADCCP) and transmission control protocol / Internet protocol (TCP / IP).

[0044] The communication system 100 may also include a relay station (e.g., relay BS 110d). A relay station is an entity that can receive data transmissions from an upstream station (e.g., a base station or wireless device) and transmit those data to a downstream station (e.g., a wireless device or base station). A relay station may also be a wireless device capable of relaying transmissions for other wireless devices. Figure 1 In the example illustrated, relay station 110d can communicate with macro base station 110a and wireless device 120d to facilitate communication between base station 110a and wireless device 120d. A relay station can also be referred to as a relay base station, repeater, etc.

[0045] The communication system 100 can be a heterogeneous network, which includes different types of base stations, such as macro base stations, pico base stations, femto base stations, relay base stations, etc. These different types of base stations can have different transmit power levels, different coverage areas, and different effects on interference in the communication system 100. For example, macro base stations can have high transmit power levels (e.g., 5 to 40 watts), while pico base stations, femto base stations, and relay base stations can have lower transmit power levels (e.g., 0.1 to 2 watts).

[0046] Network controller 130 can be coupled to a group of base stations and can provide coordination and control for these base stations. Network controller 130 can communicate with the base stations via backhaul. Base stations can also communicate with each other, for example, directly or indirectly via wireless or wired backhaul.

[0047] Wireless devices 120a, 120b, and 120c can be distributed throughout the communication system 100, and each wireless device can be stationary or mobile. Wireless devices can also be referred to as access terminals, terminals, mobile stations, subscriber units, stations, etc.

[0048] Macro base station 110a can communicate with communication network 140 via wired or wireless communication link 126. Wireless devices 120a, 120b, and 120c can communicate with base stations 110a-110d via wireless communication link 122.

[0049] Wireless communication links 122 and 124 may include multiple carrier signals, frequencies, or frequency bands, each of which may include multiple logical channels. Wireless communication links 122 and 124 may utilize one or more radio access technologies (RATs). Examples of RATs that can be used in wireless communication links include 3GPP LTE, 3G, 4G, 5G (e.g., NR), GSM, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Global System for Microwave Access (WiMAX), Time Division Multiple Access (TDMA), and other cellular RATs for mobile phone communication technologies. Further examples of RATs that can be used in one or more of the various wireless communication links 122 and 124 within the communication system 100 include mid-range protocols such as Wi-Fi, LTE-U, LTE-Direct, LAA, and MuLTEfire, and relatively short-range RATs such as ZigBee, Bluetooth, and Bluetooth Low Energy (LE).

[0050] Some wireless networks (e.g., LTE) utilize Orthogonal Frequency Division Multiplexing (OFDM) on the downlink and Single-Carrier Frequency Division Multiplexing (SC-FDM) on the uplink. OFDM and SC-FDM divide the system bandwidth into multiple (K) orthogonal subcarriers, often referred to as tones, bins, etc. Each subcarrier can be modulated with data. Typically, modulation symbols are transmitted using OFDM in the frequency domain and SC-FDM in the time domain. The spacing between adjacent subcarriers can be fixed, and the total number of subcarriers (K) can depend on the system bandwidth. For example, the subcarrier spacing could be 15 kHz, and the minimum resource allocation (called a "resource block") could be 12 subcarriers (or 180 kHz). Therefore, for system bandwidths of 1.25, 2.5, 5, 10, or 20 MHz, the nominal Fast Fourier Transform (FFT) size could be 128, 256, 512, 1024, or 2048, respectively. The system bandwidth can also be divided into subbands. For example, a subband can cover 1.08MHz (i.e., 6 resource blocks), and there can be 1, 2, 4, 8, or 16 subbands for system bandwidths of 1.25, 2.5, 5, 10, or 20MHz, respectively.

[0051] While some embodiments are described using terminology and examples associated with LTE technology, these embodiments are applicable to other wireless communication systems, such as New Radio (NR) or 5G networks. NR can utilize OFDM with a cyclic prefix (CP) on both the uplink (UL) and downlink (DL) and includes support for half-duplex operation using Time Division Duplex (TDD). A single component carrier bandwidth of 100 MHz can be supported. An NR resource block can span 12 subcarriers with a subcarrier bandwidth of 75 kHz over a duration of 0.1 milliseconds (ms). Each radio frame can include 50 subframes of 10 ms in length. Therefore, each subframe can have a length of 0.2 ms. Each subframe can indicate the link direction for data transmission (i.e., DL or UL), and the link direction of each subframe can be dynamically switched. Each subframe can include DL / UL data and DL / UL control data. Beamforming can be supported, and the beam direction can be dynamically configured. Multiple-input multiple-output (MIMO) transmission with pre-decoded encoding can also be supported. MIMO configurations in DL can support up to eight transmit antennas, with up to eight streams of multilayer DL transmission and up to two streams per radio device. Multilayer transmission with up to two streams per radio device can also be supported. Multiple cell aggregation can be supported using up to eight serving cells. Alternatively, NR can support different air interfaces in addition to OFDM-based air interfaces.

[0052] Some wireless devices can be considered Machine-Type Communication (MTC) or Evolved or Enhanced Machine-Type Communication (eMTC) wireless devices. MTC and eMTC wireless devices include, for example, robots, drones, remote devices, sensors, meters, monitors, location tags, etc., which can communicate with a base station, another device (e.g., a remote device), or some other entity. Wireless nodes can provide connectivity to or to a network (e.g., a wide area network, such as the Internet or a cellular network) via wired or wireless communication links, for example. Some wireless devices can be considered Internet of Things (IoT) devices, or can be implemented as NB-IoT (Narrowband Internet of Things) devices. Wireless devices 120a-120e can be included within a housing that houses components of the wireless device, such as processor components, memory components, similar components, or combinations thereof.

[0053] Typically, any number of communication systems and wireless networks can be deployed within a given geographical area. Each communication system and wireless network can support a specific Radio Access Technology (RAT) and can operate on one or more frequencies. A RAT can also be referred to as a radio technology, air interface, etc. A frequency can also be referred to as a carrier, frequency channel, etc. Each frequency can support a single RAT within a given geographical area to avoid interference between communication systems using different RATs. In some cases, NR or 5G RAT networks can be deployed.

[0054] In some embodiments, two or more wireless devices 120a-120e (e.g., illustrated as wireless device 120a and wireless device 120e) may communicate directly using one or more sidelink channels 124 (e.g., without using base stations 110a-110d as intermediaries to communicate with each other). For example, wireless devices 120a-120e may communicate using peer-to-peer (P2P) communication, device-to-device (D2D) communication, vehicle-to-everything (V2X) protocols (which may include vehicle-to-vehicle (V2V) protocols, vehicle-to-infrastructure (V2I) protocols, or similar protocols), mesh networks or similar networks, or combinations thereof. In this case, wireless devices 120a-120e may perform scheduling operations, resource selection operations, and other operations described elsewhere herein as being performed by base station 110a.

[0055] Figure 2 This is a component block diagram illustrating an exemplary computing and wireless modem system 200 suitable for implementing any of the various embodiments. The various embodiments can be implemented on multiple single-processor and multi-processor computer systems, including system-on-a-chip (SOC) or system-in-package (SIP).

[0056] refer to Figure 1 and Figure 2 The illustrated exemplary computing system 200 (which may be a SIP in some embodiments) includes two SOCs 202 and 204 coupled to a clock 206, a voltage regulator 208, and a wireless transceiver 266 configured to transmit and receive wireless communications to / from a wireless device such as base station 110a via an antenna (not shown). In some embodiments, the first SOC 202 operates as the central processing unit (CPU) of a wireless device that executes instructions of a software application by performing arithmetic, logic, control, and input / output (I / O) operations specified by instructions. In some embodiments, the second SOC 204 may operate as a dedicated processing unit. For example, the second SOC 204 may operate as a dedicated 5G processing unit responsible for managing high-capacity, high-speed (e.g., 5Gbps) and / or very high-frequency short-wavelength (e.g., 28GHz millimeter-wave spectrum) communications.

[0057] The first SOC 202 may include a digital signal processor (DSP) 210, a modem processor 212, a graphics processor 214, an application processor 216, one or more coprocessors 218 (e.g., vector coprocessors) connected to one or more of the processors, memory 220, custom circuitry 222, system components and resources 224, an interconnect / bus module 226, one or more temperature sensors 230, a thermal management unit 232, and a thermal power envelope (TPE) component 234. The second SOC 204 includes a 5G modem processor 252, a power management unit 254, an interconnect / bus module 264, multiple millimeter-wave transceivers 256, memory 258, and various additional processors 260, such as application processors, packet processors, etc.

[0058] Each processor 210, 212, 214, 216, 218, 252, 260 may include one or more cores, and each processor / core may perform operations independently of other processors / cores. For example, the first SOC 202 may include a processor running a first type of operating system (e.g., FreeBSD, LINUX, OS X, etc.) and a processor running a second type of operating system (e.g., MICROSOFT WINDOWS 10). Additionally, any one or all of processors 210, 212, 214, 216, 218, 252, 260 may be included as part of a processor cluster architecture (e.g., synchronous processor cluster architecture, asynchronous or heterogeneous processor cluster architecture, etc.).

[0059] The first SOC 202 and the second SOC 204 may include various system components, resources, and custom circuitry for managing sensor data, analog-to-digital conversion, wireless data transmission, and performing other specialized operations, such as decoding data packets and processing encoded audio and video signals for rendering in a web browser. For example, the system components and resources 224 of the first SOC 202 may include power amplifiers, voltage regulators, oscillators, phase-locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, and other similar components to support processors and software clients running on wireless devices. System components and resources 224 and / or custom circuitry 222 may also include circuitry for interfacing with peripheral devices such as cameras, electronic displays, wireless communication devices, and external memory chips.

[0060] The first SOC 202 and the second SOC 204 can communicate via interconnect / bus module 250. Various processors 210, 212, 214, 216, and 218 can be interconnected via interconnect / bus module 226 with one or more memory elements 220, system components and resources 224, as well as custom circuitry 222 and thermal management unit 232. Similarly, processor 252 can be interconnected via interconnect / bus module 264 with power management unit 254, millimeter-wave transceiver 256, memory 258, and various additional processors 260. Interconnect / bus modules 226, 250, and 264 may include reconfigurable gate arrays and / or implement bus architectures (e.g., CoreConnect, AMBA, etc.). Communication can be provided via advanced interconnects such as on-chip high-performance networks (NoC).

[0061] The first SOC 202 and the second SOC 204 may also include input / output modules (not shown) for communicating with external resources (such as clock 206 and voltage regulator 208). External resources (e.g., clock 206, voltage regulator 208) may be shared by two or more internal SOC processors / cores.

[0062] In addition to the example SIP 200 discussed above, various embodiments can be implemented in a variety of computing systems, which may include a single processor, multiple processors, multi-core processors, or any combination thereof.

[0063] Figure 3 This is a component block diagram illustrating a software architecture 300 including a radio protocol stack for user and control planes in wireless communication, which is adapted to implement any of the embodiments in the various embodiments. Reference Figures 1 to 3Wireless device 320 can implement software architecture 300 to facilitate communication between wireless device 320 (e.g., wireless devices 120a-120e, 200) and base station 350 (e.g., base station 110a) of a communication system (e.g., 100). In various embodiments, layers in software architecture 300 can form logical connections with corresponding layers in the software of base station 350. Software architecture 300 can be distributed among one or more processors (e.g., processors 212, 214, 216, 218, 252, 260). Although illustrated with respect to a single radio protocol stack, in a multi-SIM (Subscriber Identity Module) wireless device, software architecture 300 can include multiple protocol stacks, each of which can be associated with a different SIM (e.g., in a dual-SIM wireless communication device, the two protocol stacks are associated with two SIMs respectively). Although described below with reference to the LTE communication layer, software architecture 300 can support any of the various standards and protocols used for wireless communication, and / or can include additional protocol stacks supporting any of the various standard and protocol wireless communications.

[0064] Software architecture 300 may include a Non-Access Stratum (NAS) 302 and an Access Stratum (AS) 304. NAS 302 may include functions and protocols for supporting packet filtering, security management, mobility control, session management, and services and signaling between one or more SIMs (e.g., SIM 204) and their core network 140. AS 304 may include functions and protocols for supporting communication between one or more SIMs (e.g., one or more SIMs 204) and entities in the supported access network (e.g., base stations). In particular, AS 304 may include at least three layers (Layer 1, Layer 2, and Layer 3), each layer may include various sublayers.

[0065] In the user and control plane, Layer 1 (L1) of AS 304 can be Physical Layer (PHY) 306, which can monitor functions that enable transmission and / or reception via the air interface through a wireless transceiver (e.g., 256). Examples of such Physical Layer 306 functions may include Cyclic Redundancy Check (CRC) appending, decoding blocks, scrambling and descrambling, modulation and demodulation, signal measurement, MIMO, etc. The Physical Layer may include various logical channels, including the Physical Downlink Control Channel (PDCCH) and the Physical Downlink Shared Channel (PDSCH).

[0066] In the user and control plane, Layer 2 (L2) of AS 304 can be responsible for the link between wireless device 320 and base station 350 on physical layer 306. In various embodiments, Layer 2 may include a Medium Access Control (MAC) sublayer 308, a Radio Link Control (RLC) sublayer 310, and a Packet Data Convergence Protocol (PDCP) sublayer 312, each of which forms a logical connection that terminates at base station 350.

[0067] In the control plane, Layer 3 (L3) of AS 304 may include a Radio Resource Control (RRC) sublayer 3. Although not shown, software architecture 300 may include additional Layer 3 sublayers, as well as various upper layers above Layer 3. In various embodiments, RRC sublayer 313 may provide functions including broadcasting system information, paging, and establishing and releasing RRC signaling connections between radio device 320 and base station 350.

[0068] In various embodiments, PDCP sublayer 312 can provide uplink functions, including multiplexing between different radio bearers and logical channels, sequence numbering, handover data processing, integrity protection, encryption, and header compression. In the downlink, PDCP sublayer 312 can provide functions including sequential delivery of data packets, duplicate data packet detection, integrity verification, decryption, and header decompression.

[0069] In the uplink, RLC sublayer 310 can provide segmentation and concatenation of upper-layer data packets, retransmission of lost data packets, and Automatic Repeat Request (ARQ). In the downlink, the functions of RLC sublayer 310 can include reordering data packets to compensate for out-of-order reception, reassembly of upper-layer data packets, and ARQ.

[0070] In the uplink, MAC sublayer 308 can provide functions including multiplexing between logical and transport channels, random access procedures, logical channel prioritization, and hybrid ARQ (HARQ) operations. In the downlink, MAC layer functions can include intra-cell channel mapping, demultiplexing, discontinuous reception (DRX), and HARQ operations.

[0071] While the software architecture 300 can provide the ability to transmit data over a physical medium, it may also include at least one host layer 314 to provide data delivery services to various applications in the wireless device 320. In some embodiments, application-specific functions provided by at least one host layer 314 may provide an interface between the software architecture and the general-purpose processor 206.

[0072] In other embodiments, software architecture 300 may include one or more higher logical layers (e.g., transport, session, presentation, application, etc.) that provide host-level functionality. For example, in some embodiments, software architecture 300 may include a network layer (e.g., Internet Protocol (IP) layer) where logical connections terminate at a Packet Data Network (PDN) gateway (PGW). In some embodiments, software architecture 300 may include an application level where logical connections terminate at another device (e.g., end-user equipment, server, etc.). In some embodiments, software architecture 300 may also include a hardware interface 316 in AS 304 between physical layer 306 and communication hardware (e.g., one or more radio frequency (RF) transceivers).

[0073] Figure 4A and Figure 4B This is a component block diagram illustrating a system 400 configured to process data using the computing resources of a remote network computing device according to various embodiments. (Reference) Figures 1 to 4B System 400 may include wireless device 402 (e.g., 120a-120e, 200, 320) and network computing device 404 (e.g., 120a-120e, 200, 320). Wireless device 402 and network computing device 404 can communicate via wireless communication network 424 (…). Figure 1 The diagram illustrates its various aspects for communication.

[0074] refer to Figure 4A Wireless device 402 may include one or more processors 428 coupled to electronic storage 426 and a wireless transceiver (e.g., 266). Wireless transceiver 266 may be configured to receive messages from processor(s)428 to be transmitted in uplink transmissions and to transmit such messages via an antenna (not shown) to wireless communication network 424 for relay to network computing device 404. Similarly, wireless transceiver 266 may be configured to receive messages from network computing device 404 in downlink transmissions from wireless communication network 424 and (e.g., via a modem (e.g., 252) demodulating the messages) pass the messages to one or more processors 428.

[0075] One or more processors 428 may be configured by machine-readable instructions 406. Machine-readable instructions 406 may include one or more instruction modules. Instruction modules may include computer program modules. Instruction modules may include one or more of the following: factor determination module 408, metadata module 410, transmit and receive (TX / RX) module 412, processing time adjustment module 414, or other instruction modules.

[0076] The factor determination module 408 can be configured to determine the factors that affect the round-trip delay of a computation task.

[0077] Metadata module 410 can be configured to generate second metadata that includes identified factors and a delay budget for the computation task.

[0078] The TX / RX module 412 can be configured, for example, to send messages to and receive messages from the network computing device 404. The TX / RX module 412 can be configured to send metadata and data for processing as part of a computing task to the remote network computing device 404. The TX / RX module 412 can be configured to receive processed data of the computing task and a second metadata including an indication of the remaining time in the latency budget from the remote network computing device 404.

[0079] The processing time adjustment module 414 can be configured to adjust the processing time for post-processing the processed data based on the second metadata, so as to complete the post-processing of the processed data within the delay budget.

[0080] refer to Figure 4B The network computing device 404 may include one or more processors 432 coupled to electronic storage 430 and wireless transceiver 406. Wireless transceiver 406 may be configured to receive messages from processor(s)432(s) to be transmitted in uplink transmissions and to transmit such messages via an antenna (not shown) to wireless communication network 424 for relay to wireless device 402. Similarly, wireless transceiver 406 may be configured to receive messages from wireless device 402 in downlink transmissions from wireless communication network 424 and (e.g., via a modem (e.g., 252) demodulating the messages) pass the messages to one or more processors 432.

[0081] One or more processors 432 may be configured by machine-readable instructions 434. Machine-readable instructions 406 may include one or more instruction modules. Instruction modules may include computer program modules. Instruction modules may include one or more of the following: metadata module 436, processing time adjustment module 438, TX / RX module 440, or other instruction modules.

[0082] Metadata module 436 can be configured to receive first metadata and data for processing as part of a computation task from a remote wireless device. The first metadata includes factors affecting round-trip latency and a latency budget for the computation task. Metadata module 436 can be configured to generate second metadata that includes an indication of the remaining time in the latency budget.

[0083] The processing time adjustment module 438 can be configured to adjust the processing time used to process data based on metadata and delay budget.

[0084] The TX / RX module 440 can be configured, for example, to send messages to and receive messages from the wireless device 402. The TX / RX module 440 can be configured to send processed data and second metadata to a remote wireless device in a format that allows post-processing by the remote wireless device.

[0085] In some embodiments, wireless device 402 and network computing device 404 may be operatively linked via one or more electronic communication links. For example, such electronic communication links may be established at least partially via networks such as the Internet and / or other networks. It will be understood that this is not intended to be limiting, and the scope of this disclosure includes embodiments in which wireless device 402 and network computing device 404 may be operatively linked via some other communication medium.

[0086] Electronic storage devices 426 and 430 may include non-transitory storage media that electronically store information. The electronic storage media of electronic storage devices 426 and 430 may include one or both of system storage integrated with (i.e., substantially non-removable) wireless device 402 or network computing device 404, and / or removable storage removably connected to wireless device 402 or network computing device 404 via, for example, a port (e.g., a Universal Serial Bus (USB) port, a FireWire port, etc.) or a drive (e.g., a disk drive, etc.). Electronic storage devices 426 and 430 may include one or more of optically readable storage media (e.g., optical discs, etc.), magnetically readable storage media (e.g., magnetic tape, magnetic hard disk drives, floppy disk drives, etc.), charge-based storage media (e.g., EEPROM, RAM, etc.), solid-state storage media (e.g., flash drives, etc.), and / or other electronically readable storage media. Electronic storage devices 426 and 430 may include one or more virtual storage resources (e.g., cloud storage, virtual private networks, and / or other virtual storage resources). Electronic storage devices 426 and 430 may store software algorithms, information determined by one or more processors 428 and 432, information received from wireless device 402 or network computing device 404, or other information that enables wireless device 402 or network computing device 404 to function as described herein.

[0087] One or more processors 428, 432 may be configured to provide information processing capabilities in wireless device 402 and network computing device 404. Thus, one or more processors 428, 432 may include one or more of a digital processor, an analog processor, digital circuitry designed to process information, analog circuitry designed to process information, a state machine, and / or other mechanisms for electronically processing information. Although one or more processors 428, 432 are illustrated as a single entity, this is for illustrative purposes only. In some embodiments, one or more processors 428, 432 may include multiple processing units and / or processor cores. The processing units may be physically located within the same device, or one or more processors 428, 432 may represent the processing functions of multiple devices operating collaboratively. One or more processors 428, 432 may be configured to execute modules 408-414 and modules 436-440 and / or other modules via software; hardware; firmware; a combination of software, hardware and / or firmware; and / or other mechanisms for configuring the processing capabilities on one or more processors 428, 432. As used herein, the term "module" may mean any component or collection of components that performs the functions belonging to that module. This may include one or more physical processors, processor-readable instructions, circuitry, hardware, storage media, or any other component during the execution of processor-readable instructions.

[0088] The descriptions of the functionality provided by the various modules 408-414 and 436-440 described below are for illustrative purposes and not restrictive, as any of modules 408-414 and 436-440 may provide more or fewer functionality than described. For example, one or more of modules 408-414 and 436-440 may be removed, and some or all of their functionality may be provided by the other modules 408-414 and 436-440. As another example, processors(one or more) 428, 432 may be configured to execute one or more additional modules that perform some or all of the functionality categorized below as belonging to modules 408-414 and 436-440.

[0089] Figure 5A This is a conceptual diagram illustrating the factors affecting round-trip delay 500a. (Reference) Figures 1 to 5AFactor 500a can be monitored and / or adjusted by the processors of wireless devices (e.g., 120a-120e, 200, 320, 402) and / or network computing devices (e.g., 126, 200, 404). As mentioned above, certain aspects of the execution of computing tasks can be random and therefore difficult or impossible to adjust or control. In some embodiments, the processors of the wireless devices and / or network computing devices can monitor one or more random factors 520, such as communication latency in a communication network, communication latency outside the communication network (e.g., intranet, internet, etc.), the complexity of an image rendering workload or task (e.g., 3D rendering workload, or the number of polygons in an XR task), and / or the workload of the processor in an autonomous or semi-autonomous vehicle. Examples of processor workloads affected by random factors may include tasks such as performing image or object recognition, one or more artificial intelligence (AI) processes, manipulation control processes, and other similar processes. In some embodiments, the processor of a wireless device and / or a network computing device may communicate monitored random factors to another computing device (i.e., from the wireless device to the network computing device, or vice versa).

[0090] In some embodiments, the processor of a wireless device and / or a network computing device may dynamically adjust one or more controllable factors 522 based at least in part on a random factor 520. Examples of controllable factors 520 include the operating frequency of one or more processors (such as a CPU, GPU, DSP, or another suitable processor) and / or the transfer rate of memory components such as DDR memory that may affect the processor's processing power.

[0091] Figure 5B This diagram illustrates the timeline of delay budget model 500b according to various embodiments. (Reference) Figures 1 to 5B The latency budget model 500b can be used by the processor of a wireless device (e.g., 120a-120e, 200, 320, 402) and / or the processor of a network computing device (e.g., 126, 200, 404). The budget model may include a latency budget 502 (which may be represented as T_budget), indicating the maximum latency for a computational task. The latency budget may include various factors affecting the round-trip latency of the computational task. In various embodiments, factors affecting the round-trip latency of the computational task may include a wireless device preprocessing time 504, a first communication time 506 from the wireless device to the remote network computing device, a remote network computing device processing time 508, a second communication time 510 from the remote network computing device to the wireless device, and a wireless device postprocessing time 512. In various embodiments, the latency budget model 500b may include factors 500a (…) affecting the round-trip latency. Figure 5AOne or more aspects of ), including random factor 520 and controllable factor 522.

[0092] In some embodiments, the wireless device preprocessing time 504 (which may be denoted as T_WD_sensing) may include the time it takes for the wireless device processor to receive data from the sensor, perform a certain amount of processing on the received data, and / or prepare data for transmission to the network computing device. The wireless device preprocessing time 504 may vary based on factors such as the complexity of the sensor data, the amount of sensor data, the processing power of the wireless device, and other factors.

[0093] In some embodiments, the first communication time (which may be represented as T_comm_TX) from the wireless device to the remote network computing device 506 may include the time required for data sent by the wireless device to be transmitted by the communication system to the network computing device. The first communication time 506 may vary based on, for example, network congestion, one or more communication link conditions (such as signal noise, interference, throughput, bandwidth, or another suitable communication link condition).

[0094] In some embodiments, the remote network computing device processing time 508 (which may be denoted as T_cloud_process) may include the time taken for the network computing device's processor to receive, perform processing, and / or prepare processed data for transmission to the wireless device. The remote network computing device processing time 508 may vary based on factors such as the complexity of the sensor data, the amount of sensor data, the processing capabilities of the wireless device, and other factors. The remote network computing device processing time 508 may also vary based on the compression ratio of the data received from the wireless device.

[0095] The second communication time 510 (which may be represented as T_comm_RX) from the remote network computing device to the wireless device may include the time required for processed data sent by the network computing device to be transmitted by the communication system to the wireless device. The second communication time 510 may vary based on, for example, network congestion, one or more communication link conditions (such as signal noise, interference, throughput, bandwidth, or another suitable communication link condition).

[0096] The wireless device post-processing time 512 (which may be represented as T_WD_output) may include the time required for the wireless device's processor to receive processed data from the network computing device, decompress the data received from the network computing device (e.g., based on the compression ratio of the processed data), perform post-processing on the received data, present the post-processed data (e.g., via the wireless device's output device), and other factors.

[0097] Figure 6 This is a block diagram illustrating message 600 according to various embodiments. (See reference) Figures 1 to 6 Message 600 may be generated and sent by the processor of a wireless device (e.g., 120a-120e, 200, 320, 402) and / or the processor of a network computing device (e.g., 125, 320, 404).

[0098] Message 600 may be sent from the wireless device and / or the network computing device from time to time, for example, once per data transmission frame. In some embodiments, message 600 may include metadata 602 and data 604 related to the computing task, such as sensor data, image data, control commands, visual data (i.e., data related to XR applications, such as pose data, image data, virtual images, or text, etc.) and other suitable data related to the computing task. In some embodiments, metadata 602 may include frame ID 606 to indicate association with a particular data frame, latency budget 502, and factors affecting round-trip latency of the computing task, including wireless device preprocessing time 504, first communication time 506 from the wireless device to the remote network computing device, remote network computing device processing time 508, second communication time 510 from the remote network computing device to the wireless device, and wireless device postprocessing time 512.

[0099] Figure 7 This is a flowchart illustrating a method 700 for processing data using the computing resources of a remote network computing device, executable by a processor of a wireless device according to various embodiments. (See reference) Figures 1 to 7 Method 700 can be implemented by a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 428) of a wireless device (e.g., 120a-120e, 200, 320, 402).

[0100] In block 702, the processor can determine factors affecting the round-trip latency of a computational task. In some embodiments, factors affecting the round-trip latency of a computational task may include wireless device preprocessing time 504, a first communication time 506 from the wireless device to the remote network computing device, a remote network computing device processing time 508, a second communication time 510 from the remote network computing device to the wireless device, and wireless device postprocessing time 512 (FIG. 5). Components for performing the functions in block 702 may include processors (e.g., 210, 212, 214, 216, 218, 252, 260, 428).

[0101] In box 704, the processor can generate first metadata that includes determined factors and a latency budget for the computation task. For example, the processor can generate first metadata 602 ( Figure 6). Components used to perform the operations in block 704 may include processors (e.g., 210, 212, 214, 216, 218, 252, 260, 428).

[0102] In box 706, the processor may send first metadata and data for processing as part of a computing task to the remote network computing device. For example, the processor may send message 600 to the remote network computing device. Figure 6 ). Components for performing the operations in block 706 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432) coupled to a wireless transceiver (e.g., 266).

[0103] In block 708, the processor can receive processed data of a computing task and second metadata, including an indication of the remaining time in a latency budget, from the network computing device. In some embodiments, the wireless device can receive messages similar to 600 from a remote network computing device. Figure 6 The message is as follows. In some embodiments, the wireless device may receive data that has been processed by the network computing device, but if the wireless device has not performed additional processing, the data is unavailable when received by the wireless device (i.e., the data received from the wireless device is not a completed work product). Components for performing the functions in block 706 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432) coupled to the wireless transceiver (e.g., 266).

[0104] In block 710, the processor may adjust the processing time for post-processing the processed data based on second metadata to complete the post-processing of the processed data within a latency budget. In some embodiments, the processor may dynamically adjust one or more of the operating frequencies of one or more processors (such as a CPU, GPU, DSP, or another suitable processor) and / or the transfer rates of memory components (such as DDR memory) that may affect the processor's processing power. Components for performing the functions in block 706 may include processors (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0105] The processor can periodically re-execute the operations of boxes 702-710. In this way, the wireless device can dynamically track the latency of the computation task execution relative to a latency budget for the computation task, and dynamically adjust the timing of the computation task execution to meet the latency budget.

[0106] Figures 8 to 10This is a process flowchart illustrating operations 800-1000 that can be performed by a processor of a wireless device according to various embodiments, as part of a method for processing data using the computing resources of a remote network computing device. (Reference) Figures 1 to 10 Operations 800, 900, and 1000 can be implemented by the processors (e.g., 210, 212, 214, 216, 218, 252, 260, 428) of wireless devices (e.g., 120a-120e, 200, 320, 402).

[0107] refer to Figure 8 Following the operation at block 708 of method 700, in block 802, the processor may adjust the dynamic clock and voltage scaling (DCVS) and the dynamic task priority assignment of the processed data based on the remaining time in the delay budget. In some embodiments, a relatively high task priority may cause the processor to process the processed data before data assigned a lower priority. Components for performing the functions in block 802 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0108] The processor can then perform the operation of block 702 of method 700 as described.

[0109] refer to Figure 9 Following the operation at block 708 of method 700, in block 902, the processor may adjust the task queue position of the DCVS and the processed data based on the remaining time in the delay budget. In some embodiments, the task queue position may affect the time the processor spends processing the processed data relative to other data in the task queue for processing. Components for performing the functions in block 902 may include processors (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0110] The processor can then perform the operation of block 702 of method 700 as described.

[0111] refer to Figure 10 Following the operation at block 704 of method 700, in block 1002, the processor may adjust the compression ratio of the data to be processed as part of a computational task based on determined factors and a latency budget. Components for performing the operations in block 1002 may include processors (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0112] The processor can then perform the operation of block 706 of method 700 as described.

[0113] Figure 11 This is a flowchart illustrating a method 1100, executed by a processor of a network computing device according to various embodiments, for processing data supporting remote wireless devices. (See reference) Figures 1 to 11 Method 1100 can be implemented by a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 428) of a network computing device (e.g., 200, 125, 404).

[0114] In box 1102, the processor may receive first metadata and data for processing as part of a computational task from a remote wireless device. This first metadata includes factors affecting round-trip latency and a latency budget for the computational task. For example, the processor may receive message similar to 600. Figure 6 The message. In some embodiments, factors affecting the round-trip latency of the computing task may include wireless device preprocessing time 504, first communication time from the wireless device to the remote network computing device 506, remote network computing device processing time 508, second communication time from the remote network computing device to the wireless device 510, and wireless device postprocessing time 512 (FIG. 5). Components for performing the functions in block 1102 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432) coupled to the wireless transceiver (e.g., 406).

[0115] In box 1104, the processor may adjust the processing time for processing data based on the first metadata and a delay budget. Components for performing the functions in box 706 may include processors (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0116] In block 1106, the processor may generate second metadata including an indication of the remaining time in the delay budget. Components for performing the functions in block 706 may include processors (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0117] In block 1108, the processor can transmit processed data and second metadata to the remote wireless device in a format that enables post-processing by the remote wireless device. Components for performing the functions in block 706 may include a processor (e.g., 210, 212, 214, 216, 218, 252, 260, 432) coupled to the wireless transceiver (e.g., 406).

[0118] The processor can periodically re-execute the operations of boxes 1102-1108. In this way, the network computing device can dynamically track the execution latency of the computing task relative to the latency budget used for the computing task, and dynamically adjust the timing of the computing task execution to meet the latency budget.

[0119] Figures 12 to 14 This is a flowchart illustrating operations 1200, 1300, and 1400, which can be performed by a processor of a wireless device according to various embodiments, as part of a method for processing data using the computing resources of a remote network computing device. Reference Figures 1 to 14 Operations 1200, 1300, and 1400 can be implemented by the processors (e.g., 210, 212, 214, 216, 218, 252, 260, 428) of network computing devices (e.g., 125, 320, 404).

[0120] refer to Figure 12 Following the operation at block 1102 of method 1100, in block 1202, the processor may adjust dynamic clock and voltage scaling (DCVS) and dynamic task prioritization of data based on the remaining time in the delay budget. Components for performing the functions in block 802 may include processors (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0121] The processor can then perform the operation of block 1106 of method 1100 as described.

[0122] refer to Figure 13 Following the operation at block 1102 of method 1100, in block 1302, the processor may adjust the task queue positions of the DCVS and data based on the remaining time in the delay budget. Components for performing the operations in block 1302 may include processors (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0123] The processor can then perform the operation of block 1106 of method 1100 as described.

[0124] refer to Figure 14 Following the operation at block 1102 of method 1100, in block 1402, the processor may adjust the compression ratio of the data processed as part of a computational task based on determined factors and a latency budget. Components for performing the operations in block 1402 may include processors (e.g., 210, 212, 214, 216, 218, 252, 260, 432).

[0125] The processor can then perform the operation of block 1108 of method 1100 as described.

[0126] The various embodiments including methods and operations 1100-1400 can be executed in various network computing devices (e.g., in server devices). Figure 15 The diagram illustrates an example of a network computing device 1500 suitable for use with various embodiments. Such a network computing device may include at least... Figure 15 The components shown. (Reference) Figures 1 to 15 The network computing device 1500 may include a processor 1501 coupled to volatile memory 1502 (e.g., 430) and mass non-volatile memory such as a disk drive 1503. The network computing device 1500 may also include peripheral memory access devices coupled to the processor 1501, such as floppy disk drives, compact disc (CD) or digital video disc (DVD) drives 1506. The network computing device 1500 may also include a network access port 1504 (or interface) coupled to the processor 1501 for establishing data connections to networks (such as the Internet and / or local area networks) coupled to other system computers and servers. The network computing device 1500 may be connected to one or more antennas for transmitting and receiving electromagnetic radiation, and these antennas may be connected to wireless communication links. The network computing device 1500 may include additional access ports for coupling to peripheral devices, external memory, or other devices, such as USB, FireWire, Thunderbolt, etc.

[0127] The various embodiments including methods and operations 700, 800, 900, and 1000 can be implemented in various wireless devices (e.g., wireless devices 120a-120e, 200, 320, and 402). Figure 16 The diagram illustrates an example of a component block diagram of a wireless device 1600 suitable for use with various embodiments. Reference Figures 1 to 16 The wireless device 1600 may include a first SOC 202 (e.g., an SOC-CPU) coupled to a second SOC 204 (e.g., a 5G-capable SOC). The first SOC 202 and the second SOC 204 may be coupled to internal memories 430, 1616, a display 1612, and a speaker 1614. Additionally, the wireless device 1600 may include an antenna 1604 for transmitting and receiving electromagnetic radiation, which may be connected to a wireless data link and / or to a cellular transceiver 266 coupled to one or more processors in the first SOC 202 and / or the second SOC 204. The wireless device 1600 may also include menu selection buttons or a rocker switch 1620 for receiving user input.

[0128] The wireless device 1600 may also include a voice codec (CODEC) circuit 1610, which digitizes sound received from a microphone into data packets suitable for wireless transmission and decodes the received voice data packets to generate an analog signal provided to a speaker to produce sound. Furthermore, one or more processors among the first SOC 202, the second SOC 204, the wireless transceiver 266, and the CODEC 1610 may include digital signal processor (DSP) circuitry (not shown separately).

[0129] The processors of network computing device 1600 and wireless device 1600 can be any programmable microprocessor, microcomputer, or one or more multiprocessor chips that can be configured by software instructions (applications) to perform various functions (including those described in the various embodiments below). In some wireless devices, multiple processors may be provided, such as one processor within SOC 204 dedicated to wireless communication functions and one processor within SOC 202 dedicated to running other applications. Software applications may be stored in memories 426, 430, 1616 before being accessed and loaded into the processor. The processor may include internal memory sufficient to store application software instructions.

[0130] As used herein, the terms “component,” “module,” “system,” etc., are intended to include computer-related entities such as, but not limited to, hardware, firmware, combinations of hardware and software, software, or software in execution configured to perform a particular operation or function. For example, a component can be, but is not limited to, a process, processor, object, executable program, execution thread, program, and / or computer running on a processor. For illustration, both an application running on a wireless device and the wireless device itself can be referred to as a component. One or more components may reside within a process and / or execution thread, and components may reside on a single processor or core and / or be distributed across two or more processors or cores. Furthermore, these components may execute from various non-transitory computer-readable media on which various instructions and / or data structures are stored. Components may communicate via local and / or remote processes, function or procedure calls, electronic signals, data packets, memory read / write, and other known network, computer, processor, and / or process-related communication methods.

[0131] In the future, a variety of different cellular and mobile communication services and standards are available or anticipated, all of which can be implemented and benefit from various implementations. Such services and standards include, for example, the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE) systems, 3rd Generation Wireless (3G), 4th Generation Wireless (4G), 5th Generation Wireless (5G), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), 3GSM, Universal Packet Radio Service (GPRS), Code Division Multiple Access (CDMA) systems (e.g., cdmaOne, CDMA1020TM), Enhanced Data Rate Evolution of GSM (EDGE), Advanced Mobile Phone Systems (AMPS), Digital AMPS (IS-136 / TDMA), Evolved Data Optimized (EV-DO), Digital Enhanced Cordless Telecommunications (DECT), Global Microwave Access Interoperability (WiMAX), Wireless Local Area Networks (WLAN), Wi-Fi Protected Access I&II (WPA, WPA2), and Integrated Digital Enhanced Network (iDEN). Each of these technologies relates to, for example, the transmission and reception of voice, data, signaling, and / or content messages. It should be understood that any references to terms and / or technical details relating to individual telecommunications standards or technologies are for illustrative purposes only and are not intended to limit the scope of the claims to a particular communication system or technology, unless specifically stated in the language of the claims.

[0132] The various embodiments illustrated and described are provided merely as examples to illustrate the various features of the claims. However, the features shown and described with respect to any given embodiment are not necessarily limited to the associated embodiment, but can be used or combined with other embodiments shown and described. Furthermore, the claims are not intended to be limited to any one of the exemplary embodiments. For example, one or more operations of the above methods may be substituted for or combined with one or more operations of the above methods.

[0133] The foregoing method descriptions and process flowcharts are provided as illustrative examples only and are not intended to require or imply that the operations of the various embodiments must be performed in the presented order. As will be understood by those skilled in the art, the order of operations in the foregoing embodiments can be performed in any order. Words such as “after,” “then,” and “next” are not intended to limit the order of operations; these words are used to guide the reader in understanding the description of the method. Furthermore, any reference to a claim element in the singular form, such as the use of the terms “a,” “an,” or “the,” should not be construed as limiting the element to the singular.

[0134] The various illustrative logic blocks, modules, components, circuits, and algorithmic operations described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this hardware-software interchangeability, various illustrative components, blocks, modules, circuits, and operations have been described above in terms of their functionality. Whether such functionality is implemented in hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art can implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of these claims.

[0135] The hardware used to implement the various illustrative logics, logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed using a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of receiver intelligent objects, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuitry specific to a given function.

[0136] In one or more embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on a non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium. The operation of the methods or algorithms disclosed herein may be embodied in a processor-executable software module or processor-executable instructions, which may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium may be any storage medium accessible to a computer or processor. By way of example and not limitation, such non-transitory computer-readable or processor-readable storage media may include RAM, ROM, EEPROM, flash memory, CD-ROM or other optical disc storage, disk storage or other magnetic storage smart objects, or any other medium that can be used to store desired program code in the form of instructions or data structures and is accessible to a computer. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks generally reproduce data magnetically, while optical discs reproduce data optically using lasers. The aforementioned combinations also fall within the scope of non-transitory computer-readable and processor-readable media. Furthermore, the operation of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transitory processor-readable and / or computer-readable storage medium that can be incorporated into a computer program product.

[0137] The foregoing description of the disclosed embodiments is intended to enable any person skilled in the art to make or use the claims. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the claims. Therefore, this disclosure is not intended to be limited to the embodiments shown herein, but should be accorded the broadest scope consistent with the appended claims and the principles and novel features disclosed herein.

Claims

1. A method performed by a wireless device for processing data using computing resources of a remote network computing device, comprising: Determine the time required to complete each of the multiple operations involved in the computation task; Generate first metadata, which includes a plurality of determined times and a delay budget for the computing task, wherein the delay budget is the time to complete the computing task; The first metadata, the data, and the elapsed time since the start of the computing task are sent to the remote network computing device for processing as part of the computing task; Receive processed data and second metadata of the computing task from the remote network computing device, the second metadata including an indication of the remaining time in the delay budget; as well as Based on the second metadata, the dynamic clock and voltage scaling DCVS of one or more processors of the wireless device are adjusted to post-process the processed data to complete the post-processing of the processed data within the latency budget, wherein the post-processing includes one of the plurality of operations.

2. The method of claim 1, wherein determining the plurality of times includes determining one or more of the following: wireless device preprocessing time, a first communication time from the wireless device to the remote network computing device, a remote network computing device processing time, a second communication time from the remote network computing device to the wireless device, or a wireless device postprocessing time.

3. The method of claim 1 further includes dynamic task priority assignment of the processed data based on the remaining time in the delay budget.

4. The method of claim 1, further comprising adjusting the task queue position of the processed data based on the remaining time in the delay budget.

5. The method of claim 1, wherein the processed data of the computing task received from the remote network computing device requires post-processing to complete the computing task.

6. The method of claim 1, further comprising adjusting the compression ratio of the data to be processed as part of the computing task based on the plurality of times and the latency budget before sending the data to the remote network computing device.

7. A wireless device, comprising at least one processor, said at least one processor being configured with processor-executable instructions to: Determine the time required to complete each of the multiple operations involved in the computation task; Generate first metadata, the first metadata including the plurality of times and a delay budget for the computing task, the delay budget being the time to complete the computing task; Send the first metadata, data, and elapsed time since the start of the computing task to a remote network computing device for processing as part of the computing task; Receive processed data and second metadata of the computing task from the remote network computing device, the second metadata including an indication of the remaining time in the delay budget; as well as Based on the second metadata, the dynamic clock and voltage scaling DCVS of one or more processors of the wireless device are adjusted to post-process the processed data to complete the post-processing of the processed data within the latency budget, wherein the post-processing includes one of the plurality of operations.

8. The wireless device of claim 7, wherein the at least one processor is further configured with processor-executable instructions to determine one or more of the following: wireless device preprocessing time, a first communication time from the wireless device to the remote network computing device, a remote network computing device processing time, a second communication time from the remote network computing device to the wireless device, or a wireless device postprocessing time.

9. The wireless device of claim 7, wherein the at least one processor is further configured with processor-executable instructions to adjust dynamic task priority assignment of the processed data based on the remaining time in the latency budget.

10. The wireless device of claim 7, wherein the at least one processor is further configured with processor-executable instructions to adjust the task queue position of the processed data based on the remaining time in the latency budget.

11. The wireless device of claim 7, wherein the at least one processor is further configured with processor-executable instructions that cause the processed data of the computing task received from the remote network computing device to require post-processing to complete the computing task.

12. The wireless device of claim 7, wherein the at least one processor is further configured with processor-executable instructions to adjust the compression ratio of the data to be processed as part of the computing task based on the plurality of times and the latency budget before sending the data to the remote network computing device.

13. A method performed by a network computing device for processing data supporting a remote wireless device, comprising: Receive first metadata and data for processing as part of a computing task from a remote wireless device. The first metadata includes the time required to complete each of the various operations involved in completing the computing task, the elapsed time since the start of the computing task, and a delay budget for the computing task, the delay budget being the time to complete the computing task. The dynamic clock and voltage scaling (DCVS) of one or more processors of the network computing device are adjusted based on the first metadata and the latency budget to process the data, wherein processing the data includes one of the plurality of operations; Generate second metadata, which includes an indication of the remaining time in the delay budget; as well as This enables the processed data and the second metadata to be sent to the remote wireless device in a format that allows post-processing by the remote wireless device.

14. The method of claim 13, wherein receiving first metadata and data for processing as part of a computing task from the remote wireless device includes receiving one or more of the following: remote wireless device preprocessing time, a first communication time from the remote wireless device to the network computing device, network computing device processing time, a second communication time from the network computing device to the remote wireless device, or remote wireless device postprocessing time.

15. The method of claim 13, further comprising adjusting the dynamic task priority assignment of the data based on the remaining time in the delay budget.

16. The method of claim 13, further comprising adjusting the task queue position of the data based on the remaining time in the delay budget.

17. The method of claim 13, further comprising adjusting the compression ratio of the processed data based on the plurality of times and the delay budget.

18. A network computing device, comprising at least one processor, said at least one processor being configured with processor-executable instructions to: Receive first metadata and data for processing as part of a computing task from a remote wireless device. The first metadata includes the time required to complete each of the various operations involved in completing the computing task, the elapsed time since the start of the computing task, and a delay budget for the computing task, the delay budget being the time to complete the computing task. The dynamic clock and voltage scaling (DCVS) of one or more processors of the network computing device are adjusted based on the first metadata and the latency budget to process the data; Generate second metadata, which includes an indication of the remaining time in the delay budget; as well as This enables the processed data and the second metadata to be sent to the remote wireless device in a format that allows post-processing by the remote wireless device.

19. The network computing device of claim 18, wherein the at least one processor is further configured to have processor-executable instructions to receive one or more of the following: a remote wireless device preprocessing time, a first communication time from the remote wireless device to the network computing device, a network computing device processing time, a second communication time from the network computing device to the remote wireless device, or a remote wireless device postprocessing time.

20. The network computing device of claim 18, wherein the at least one processor is further configured with processor-executable instructions to adjust the dynamic task priority assignment of the data based on the remaining time in the latency budget.

21. The network computing device of claim 18, wherein the at least one processor is further configured with processor-executable instructions to adjust the task queue position of the data based on the remaining time in the latency budget.

22. The network computing device of claim 18, wherein the at least one processor is further configured with processor-executable instructions to adjust the compression ratio of the processed data based on the plurality of times and the latency budget.

23. An apparatus for processing data using the computing resources of a remote network computing device, the apparatus comprising components for performing the method of any one of claims 1-6.

24. A computer-readable medium having program code recorded thereon, wherein the program code is executable by one or more processors of a wireless device for processing data using the computing resources of a remote network computing device, thereby causing the processor to perform the method of any one of claims 1-6.

25. A computer program product comprising computer-readable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1-6.

26. An apparatus for processing data supporting a remote wireless device, the apparatus comprising components for performing the method of any one of claims 13-17.

27. A computer-readable medium having program code recorded thereon, wherein the program code is executable by one or more processors of a network computing device for processing data supporting a remote wireless device, thereby causing the processor to perform the method of any one of claims 13-17.

28. A computer program product comprising computer-readable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 13-17.

Images