Delivery of streaming video

Abstract

A network node (100) is configured to process video streams to send to client devices (200) over a network (32). The network node, which may be an eXtended Reality (XR) server node, for example, is configured to send bursts of XR video data in video frames to the client device. The data bursts are smaller in size than those sent by conventionally configured server nodes and are also sent with a higher frequency. In this manner, a network node configured according to this disclosure is configured to improve transport utilization without negatively affecting application latency.

Description

[0001] DELIVERY OF STREAMING VIDEO

[0002] TECHNICAL FIELD

[0003] This application relates generally to video streaming, and more particularly to the delivery of video streams, split rendering, network optimization, mixed reality, extended Reality (XR), and the like.

[0004] BACKGROUND

[0005] Extended Reality (XR) is an umbrella term that covers several related and rapidly growing technologies. These are Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR), and are considered subsets of XR technology. Generally, these technologies combine or mirror the physical world with a digital world to enable users to interact with it. More specifically, XR technology affects the human-machine interface by immersing the user in a virtual environment (i.e., VR), augmenting the user’s current environment (i.e., AR), or both (i.e., MR).

[0006] Conventionally, many XR applications render 3-Dimensional (3D) content in the cloud. The resulting video is then streamed to a client device (e.g., a computer display, a mobile device, a head-mounted display, etc.) under strict latency requirements. Such requirements are especially important for AR / MR applications, and therefore, work is being done on meeting those requirements. For example, video streaming usually involves sending large “bursts” of XR video data. Accordingly, it is worthwhile optimizing transmission of the resultant video, for example when sending over a radio interface.

[0007] SUMMARY

[0008] The present disclosure provides a method and corresponding apparatuses for streaming video to client devices over a network. In more detail, embodiments of the present disclosure configure a network node, such as an XR server node, for example, to send smaller bursts of XR video data with higher frequency to a client device. In this manner, the present embodiments improve transport utilization without negatively affecting application latency.

[0009] Accordingly, a first aspect of the present disclosure provides a method, implemented at a network node, for sending video to a client device. In this aspect, the method comprises the network node detecting a plurality of timed trigger signals within a video frame interval. Then, responsive to detecting each timer signal, the network node generates video data for display at the client device and sends the video data to the client device in a video frame.

[0010] A second aspect of the present disclosure provides a network node for sending video to a client device. In this aspect, the network node is configured to detect a plurality of timed trigger signals within a video frame interval, and responsive to detecting each timer signal, generate video data for display at the client device and send the video data to the client device in a video frame. In a third aspect, the present disclosure provides a network node that comprises processing circuitry and memory for sending video to a client device. In this aspect, the memory is configured to store instructions executable by the processing circuitry, whereby the network node is configured to detect a plurality of timed trigger signals within a video frame interval and, responsive to detecting each timer signal, generate video data for display at the client device and send the video data to the client device in a video frame.

[0011] In a fourth aspect, the present disclosure provides a computer program comprising instructions that, when executed on processing circuitry of a network node, causes the network analytics node to perform the method according to the first aspect.

[0012] In a fifth aspect, the present disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon. The computer program comprises executable instructions that, when executed by processing circuitry in a network node, causes the network node to perform the method according to the first aspect.

[0013] BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 illustrates a timeline showing the transmission timing for video streams at a network node (e.g., an XR server), the display timing of the received video streams at a client device, and the network transmission latency.

[0015] Figure 2 is a block diagram illustrating some components of a communication system configured according to embodiments of the present disclosure.

[0016] Figure 3 is a flow diagram illustrating a method for sending video streams from a network node to a client device according to embodiments of the present disclosure.

[0017] Figure 4 is a flow diagram illustrating a method for sending video streams from a network node to a client device according to an embodiment of the present disclosure.

[0018] Figure 5 is a flow diagram illustrating a method for receiving, at a client device, video streams from a network node according to an embodiment of the present disclosure.

[0019] Figure 6 is a functional block diagram illustrating some components of a network node configured to stream video to a client device according to embodiments of the present disclosure.

[0020] Figure 7 is a functional block diagram illustrating some components of a client device configured to receive video streams from a network node according to embodiments of the present disclosure.

[0021] Figure 8 is a functional block diagram illustrating an example of a communication system in accordance with some embodiments.

[0022] Figure 9 is a functional block diagram illustrating an example User Equipment (UE) in accordance with some embodiments.

[0023] Figure 10 is a functional block diagram illustrating an example network node in accordance with some embodiments. Figure 11 is a functional block diagram of a host, which may be an embodiment of the host illustrated in Figure 8, in accordance with some embodiments described herein.

[0024] Figure 12 is a functional block diagram illustrating a virtualization environment in which functions implemented by some embodiments of the present disclosure may be virtualized.

[0025] Figure 13 is a functional block diagram illustrating a communication diagram of a host communicating via a network node with a UE over a partially wireless connection in accordance with some embodiments.

[0026] DETAILED DESCRIPTION

[0027] The present disclosure provides a method and corresponding apparatuses for streaming video to client devices over a network. Particularly, embodiments of the present disclosure configure a network node, such as an XR server node, for example, to send smaller bursts of XR video data with higher frequency to a client device. In this manner, the present embodiments improve transport utilization without negatively affecting application latency.

[0028] As stated above, many XR applications render 3-Dimensional (3D) content in the cloud and then stream the video to a client device under strict latency requirements. Such requirements are especially important for AR / MR applications, and therefore, work is being done on meeting those requirements.

[0029] For example, some methods address issues such as latency in streaming video. U.S. Patent No. 10,446,119, for instance, addresses latency by using “graphic layers.” As defined in the ‘119 patent, a “graphic layer” is a rendered image that is associated with a selected set of virtual objects of a 3D scene. Expanding on this idea, the ‘119 patent also proposes using different Frame per Second (FPS) transmission rates for the different graphic layers when streaming the video from the Tenderer (e.g., an XR server node in the network) to a client device. Another publication, i.e., U.S. Patent Publication No. 2023 / 0039100, proposes using several graphic layers to support split rendering by associating rendering instructions to graphic layers.

[0030] In another example, The High Efficiency Video Coding standards (e.g., ITU-T H265 (08 / 2021)) define the use of slices in encoders. According to these standards, slices are used to encode parts of a picture with different encoding parameters. Encoded slices can then later be decoded individually on a per-slice basis. Further, as described in “Advanced Video Coding for Generic Audiovisual Services (ITU-T H264 (08 / 2021)), it is possible, in cases where the slices are encoded sequentially, to adjust the slice encoding parameters based on the bitstream (e.g., encoded slide / frame) statistics of previously encoded slices. This method helps support more precise rate control.

[0031] Additionally, according to “Self-Clocked Rate Adaptation for Multimedia, RFC 8298 (Dec. 2017), video streaming typically uses some form of rate control to adjust the target bitrate in case of transport congestion. RFC 8298 also defines “pacing.” Devices that stream video use pacing to add an artificial delay between selected video packets to reduce the size of transport packet bursts.

[0032] Other techniques, such as the “Interlaced Video” technique described in https: / / en.wikipedia.org / wiki / lnterlaced_video, for example, doubles the perceived frame rate of a video display without consuming extra bandwidth. With such techniques, one frame would contain all odd-numbered lines in an image, while the next frame would contain all even- numbered lines in the image.

[0033] On the client device side, U.S. Patent No. 10,595,000 describes “warping.” Such techniques enable the prediction of future positions of virtual objects based on the 2D video frames of a 3D scene and depth information. With this method, motion vectors for the virtual objects are derived from the 2D frames and then used for predicting the virtual object’s position and orientation.

[0034] Further, U.S. Patent Publication No. 2022 / 0038654 defines a method for interpolating a video frame based on a first and second sets of pixel data sampled from a first and second video frames, respectively, and on a set of forward pointing motion vectors from the first video frame to the second video frame.

[0035] Each of these techniques is useful; however, none adequately addresses the issue of latency. For example, video streaming is typically characterized by transmitting separate bursts of video data on low timescales. That is, encoded frames are sent to a client device as multiple bursts with one burst occurring once in every frame time (i.e., 1 / FPS, e.g., 16.6 ms on 60 FPS). Such bursts, however, require more network resources to transmit with given latency bounds than do more evenly distributed traffic.

[0036] Pacing attempts to reduce transmission bursts by introducing artificial delays between the data packets of a burst. This artificial delay, though, leads to increased transport latency, which is unwanted in AR applications.

[0037] Using slices in encoders can reduce the number of bytes to transmit in a burst, however, encoders typically encode all slices within a couple of milliseconds. Thus, even with the use of slices, transmission bursts still occur on a frame-time timescale (i.e., one burst per frame).

[0038] Some methods utilize various rate control techniques to adjust the video stream bitrate according to varying available network bandwidth. Particularly, video streamers typically receive rate control feedback on a per-frame basis or once in a given time interval. Encoders then consider the feedback when starting to encode a next video frame, or in some cases, during encoding before they start encoding a subsequent slice of the same video frame. Consequently, in a worst-case scenario, the use of the received feedback by the encoder may be delayed by a frame until the encoder can react to the feedback.

[0039] Typically, encoding takes a relatively small part of the frame time. For instance, conventional encoders require 4ms to perform the encoding function every 16.6ms. Thus, most of the additional latency remains even if the encoder uses slices. Additionally, the frame / slice that being encoded when the feedback arrives is not encoded according to the feedback. Rather, it is sent to the network with the previous target bitrate settings. Such approaches can overutilize the available network bandwidth, thereby leading the temporary instability of the network queues and longer transmission latency.

[0040] Sending graphic layers with different FPS can also be used to reduce the amount of traffic, but at the cost of introducing additional latency over the frame time. This is because a reduced update rate generally leads to less frequent updates and an increased amount of time in between the updates. Some AR / MR applications do not tolerate such increased latency. Interlaced transmission can be utilized to achieve better utilization, but it is defined on a per- pixel row basis. Further, how to apply interleaving on the virtual object level is not currently defined.

[0041] Accordingly, the present disclosure provides a method, implemented at a sending device such as a network node, for example, for sending video streams to a client device without increasing transport latency. Particularly, as described herein, a “sending device,” such as a network node (e.g., an XR server), for example, is configured to stream smaller bursts of XR video data with higher frequency to a client device. That is, the network node configured according to the present embodiments sends a greater number of bursts containing smaller amounts of XR data to the client device. This achieves better transport utilization without increasing application latency.

[0042] In one embodiment, sending the smaller bursts of XR video data is realized by assigning virtual objects to N graphic layers, which are then rendered and encoded. The rendered and encoded graphics layers are then distributed evenly throughout the timescale of a frame. The present embodiments also avoid increasing application latency by simulating 3D scenes of an image at a higher frequency as a function of the number (i.e., N) of graphics layers, and by compositing a video frame to be displayed in a client device based on the most up-to-date graphic layers.

[0043] The embodiments of the present disclosure provide benefits and advantages that current techniques cannot or do not provide. For example, a network node configured according to the present embodiments reacts much faster to received congestion feedback than do conventional devices. Thus, the present disclosure can decrease feedback loop latency such that an encoder associated with a network node configured according to the present embodiments can begin encoding a next frame / slice faster than conventionally configured encoders. These faster starts to the encoding process at the network node are due to an increase in the frequency with which a network node configured according to the present disclosure renders the video data. In at least one embodiment, the frequency with which the network node renders the video data is:

[0044] 1

[0045] N*FPS where: N is an integer value > 1 ; and FPS is Frames Per Second.

[0046] Slices could be used to decrease the response time in encoders. However, the encoding process in modern Graphics Processing Units (GPUs) typically takes approximately 2-5ms. Consequently, while video traffic bursts will still occur, they will not have an appreciable effect on latency as long as the encoding process is significantly faster than the frame time (e.g., 1 / FPS).

[0047] In another benefit, the present embodiments reduce the amount of peak burst size sent at a given time by sending, on average, N times less data in a burst. Additionally, according to the present disclosure, the transmission times of the video data packets are distributed more evenly during a frame interval.

[0048] Additionally, on average, the present embodiments will not increase the end-to-end (e2e) application latency, which as those of ordinary skill in the art will understand, is different from the latency related to a feedback loop. In fact, the present embodiments can actually decrease the e2e latency to some extent. For example, Figure 1 shows a graph 10 illustrating an example sequence of frames with sender device (e.g., an XR server) and client device timing. The upper time axis represents the sender device with frame / graphic layer transmission times O, I, while the lower time axis represents the display times D for the rendered / transmitted content at the client device. Particularly, “O” represents the transmission times for conventionally configured sender devices, and “I” represents the transmission times for sender devices configured according to the present disclosure. Additionally, the solid-line arrows 12, 14 represent conventional streaming (i.e., video streams sent by conventionally configured sender devices), while the dashed-line arrows 16, 18, 20, 22, 24, 26 represent the streaming of different graphic layers by a sender device configured according to the present embodiments. As can be seen in Figure 1 , the streaming times of the different graphic layers by a sender device configured according to the present embodiments are distributed across the frame time. Additionally, there is typically a continuous shift between the sender clock and the client clocks.

[0049] Figure 1 illustrates two factors that can contribute to latency in the present embodiments. First, because of the clock shifts, the residual time - i.e., the time between the arrival of a transmitted graphics layer at the client device and the display time D of the original frames (i.e., streams 12, 14), can be approximated by a uniform distribution. The same goes for each individual graphics layer transmitted in a stream represented by dashed-line arrows 16, 18, 20, 22, 24, 26. Hence distributing the frame to different graphic layers and distributing their sending from the sender device, as is done by the present embodiments, neither increases nor decreases frame-related latency.

[0050] Second, the transmission times will depend on the number of bytes that is transmitted at a time. Hence, the smaller the bitstream the faster the transmission. As seen in Figure 1 , the solid black arrows 12, 14 representing the transmission times at conventionally configured sender devices are less steep than are the dashed-line arrows 16, 18, 20, 22, 24, 26 representing the distributed transmission of the different graphics layers. Accordingly, the present embodiments are expected to decrease, at least slightly, the downstream streaming pipeline at least slightly.

[0051] Figure 2 is a block diagram illustrating some components of a communication system 30 configured according to embodiments of the present disclosure. As seen in Figure 2, system 30 comprises a communications network 32 communicatively interconnecting a network node 40 (e.g., an XR server) and a client device 50 (e.g., an XR client such as a head-mounted display, glasses, a mobile device, etc.). In this embodiment, network node 40 implements a physics simulation component (e.g., 3D simulator 42) and a Tenderer component 44. To support streaming to a remote client device, such as client device 50, network node 40 also includes an encoder 46 and a transport component 48. In this embodiment, the transport component 48 is a video transport component; however, those of ordinary skill in the art will readily appreciate that the use of video in describing the present embodiments is merely illustrative. As for the client device 50, it implements a warp component 52, a decoder 54, and a transport component 56.

[0052] In operation, the 3D simulator component 42 and the Tenderer component 44 at network node 40 process graphics layers according to the present disclosure. The encoder component 46 then encodes the graphics layers for transmission to client device 50 by transport component 48 via network 32. Upon receipt of the graphics layers by transport component 56, client device 50 decodes the graphic layers using decoder 54 and warps the decoded graphics layers using warp 52. As is known in the art, “warping” is the process of digitally manipulating an image such that any shapes that are in the image are significantly distorted. Warping may be used for almost any creative purpose, but in at least one embodiment, the warping process is utilized to correct image distortion.

[0053] Figure 3 is a flow diagram illustrating a method 60 for sending video streams from a network node to a client device (e.g., network node 40 and client device 50 of Figure 2) according to embodiments of the present disclosure. In the embodiment of Figure 3, method 60 is implemented at the sender device.

[0054] As seen in Figure 3, the sender device has a timer that fires at a frequency of - i -

[0055] N*FPS (box 62). As stated above, N is an integer value > 1 (e.g., 2 < N < 10) and FPS is Frames Per Second. Each time the timer fires, the sender device uses 3D simulator 42 to update the 3D scene (box 64). 3D simulators, 42, and their processes, are well-known in the art, and thus, not detailed here. The sender device then selects which of a plurality of graphics layers is scheduled to be rendered (box 66). For example, as stated above, there may be many different graphics layers, each with its own associated virtual objects. According to one embodiment, a sender device configured according to the present disclosure could select the graphics layers on a round-robin basis such that each graphics layer is rendered in a frame-time interval. The sender device then selects the virtual objects that are associated with the selected graphic layer based on information that defines the association between the virtual objects and the graphics layers (box 68). Particularly, the association maps virtual objects to their corresponding graphic layers. Such information changes slower than the rendering frequency. For example, a given virtual object can be mapped to a given graphic layer based on the depth (i.e., distance) of the virtual object from the camera that captured the image. Therefore, in this embodiment, the sending device is configured to read the given association between the virtual objects and the different graphic layers without modifying the association.

[0056] So selected, the sender device renders the selected objects to the selected graphic layer (box 70). In this embodiment, only the set of selected virtual objects are rendered. Once rendered, the sending device hands-off the graphic layer to an encoder, such as encoder 46, for example. The graphic layer includes rendered video frames. The sender device then selects an encoder configuration that is associated with the particular graphics layer (box 72) and changes the encoder configuration according to the selected configuration assigned to the current graphic layer. For example, in one embodiment of the present disclosure, the sender device may modify the reference frame so that a current picture will refer to a previous picture of the same graphic layer. Sender device then encodes the graphics layer using the selected encoder configuration (box 74) and sends the encoded frame to the transport component 48 for transmission to client device 50 (box 76).

[0057] Figure 4 is a flow diagram illustrating a method 80 for sending video streams from a sender device 40 (e.g., a network node) to a client device 50 according to an embodiment of the present disclosure. Method 80 of Figure 4 is implemented at the client device 50.

[0058] As seen in Figure 4, method 80 is triggered (box 82) by a timer having the same frequency as the display and that has a constant shift in-phase, which is configurable. The constant shift in-phase makes sure that method 80 concludes before the frame is swapped for display.

[0059] For each graphic layer, client device 50 searches for, and fetches, the latest decoded image of the graphic layer (box 84). The client device then reads the render time for the graphic layer and calculates a time difference between the time of display and the render time. The client device uses this time to warp / project the graphic layer (box 86). It should be noted here that the times for rendering the selected graphic layers may be different. Afterwards, the graphic layers that have been previously warped consistently to the same display time are composited together into a frame to be displayed. The composited frame can then be swapped for display.

[0060] Figure 5 is a flow diagram illustrating a method 90 for receiving, at client device 50, video streams from a network node, such as network node 40, according to an embodiment of the present disclosure. As seen in Figure 5, method 90 first detects each of a plurality of timed trigger signals within a video frame interval (box 92). Responsive to detecting each timer signal, the network node 40 generates video data for display at the client device (box 94) and sends the video data to a client device in a video frame (box 96).

[0061] In one embodiment, the trigger signals are timed to occur at a frequency of:

[0062] 1

[0063] N*FPS ’ where N is an integer value > 1 , and FPS is Frames Per Second.

[0064] In one embodiment, generating video data for display at the client device comprises updating one or more 3-Dimensional (3D) objects in a video scene using a 3-Dimensional (3D) simulator.

[0065] In one embodiment, generating video data for display at the client device further comprises selecting a graphics layer to render from among a plurality of graphics layers, selecting a set of one or more virtual objects associated with the graphics layer, and rendering the set of one or more virtual objects to the graphics layer.

[0066] In one embodiment, the graphics layer is selected round-robin such that each of the plurality of graphics layers is rendered within a single frame time interval.

[0067] In one embodiment, each virtual object in the set of one or more virtual objects is mapped to the graphics layer based on a distance of the virtual object from a camera that captured the video stream.

[0068] In one embodiment, generating video data for display at the client device further comprises encoding the graphics layer and the one or more virtual objects to generate an encoded video frame.

[0069] In one embodiment, encoding the graphics layer and the one or more virtual objects comprises selecting an encoder configuration based on a configuration assigned to the graphic layer, and encoding the graphics layer and the one or more virtual objects according to the encoder configuration.

[0070] In one embodiment, a reference frame is modified such that a current picture associated with the graphics layer refers to a previous picture associated with the same graphics layer.

[0071] In one embodiment, the encoded video frame is sent to the client device in a burst of video packets.

[0072] In one embodiment, the video frame is one of a plurality of video frames that are sent to the client device during the video frame interval.

[0073] In one embodiment, the plurality of video frames are distributed evenly throughout the video frame interval.

[0074] In one embodiment, the network node comprises an extended Reality server in a cloud network.

[0075] An apparatus can perform any of the methods herein described by implementing any functional means, modules, units, or circuitry. In one embodiment, for example, the apparatuses comprise respective circuits or circuitry configured to perform the steps shown in the method figures. The circuits or circuitry in this regard may comprise circuits dedicated to performing certain functional processing and / or one or more microprocessors in conjunction with memory. For instance, the circuitry may include one or more microprocessor or microcontrollers, as well as other digital hardware, which may include Digital Signal Processors (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory may include program instructions for executing one or more telecommunications and / or data communications protocols as well as instructions for carrying out one or more of the techniques described herein, in several embodiments. In embodiments that employ memory, the memory stores program code that, when executed by the one or more processors, carries out the techniques described herein.

[0076] Figure 6 is a functional block diagram illustrating some of the main functional components of a network node 100 configured to stream video to a client device according to embodiments of the present disclosure. As previously stated, network node 100 may be, for example, a server node, such as an XR server, operating in a communications network. However, those of ordinary skill in the art will readily appreciate that the present disclosure is not so limited. In other embodiments, for example, the network node 100 is some other type of computing node (e.g., an Application Server) operating in the communications network. Thus, the particular identification of the network node 100 as being an XR server is merely illustrative.

[0077] Regardless of its particular implementation, however, network node 100 in this embodiment comprises communication circuitry 102, processing circuitry 104, and memory 106 configured to store a computer program 108 thereon. The communication circuitry 102 comprises the hardware required for communicating with one or more other network nodes (e.g., core network nodes, etc.), as well as with one or more nodes in a radio access network (RAN), such as a base station, for example. In this regard, the communication circuitry 102 may comprise a Network Interface Circuit (NIC) (e.g., an ETHERNET or similar interface). In some embodiments, the network node 100 may be configured for wireless communications, and thus, communication circuitry 102 would comprise the radio frequency (RF) circuitry needed for transmitting and receiving signals over a wireless communication channel. In these cases, network node 100 may also be coupled to one or more antenna (not shown). In other embodiments, however, network node 100 is configured to communicate via a wire interface.

[0078] The processing circuitry 104 comprises one or more microprocessors, hardware, firmware, or a combination thereof that controls the overall operation of network node 100. In accordance with the present disclosure, the processing circuitry 104 can be configured by software to perform one or more of the methods herein described including any of methods 60 and 90 seen in Figures 3 and 5, respectively. Memory 106 comprises both volatile and non-volatile memory for storing computer program code and data needed by the processing circuitry 104 for operation. Memory 106 may comprise any tangible, non-transitory computer-readable storage medium for storing data including electronic, magnetic, optical, electromagnetic, or semiconductor data storage. Memory 106 stores a computer program 108 comprising executable instructions that configure the processing circuit 104 in network node 100 to perform one or more of the methods herein described including any of methods 60 and 90 seen in Figures 3 and 5, respectively. A computer program 108 in this regard may comprise one or more code modules corresponding to the means or units described above.

[0079] In general, computer program instructions and configuration information are stored in a non-volatile memory, such as a ROM, erasable programmable read only memory (EPROM) or flash memory. Temporary data generated during operation may be stored in a volatile memory, such as a random-access memory (RAM). In some embodiments, computer program 108 for configuring the processing circuitry 104 as herein described may be stored in a removable memory, such as a portable compact disc, portable digital video disc, or other removable media. The computer program 108 may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0080] Those skilled in the art will also appreciate that embodiments herein further include corresponding computer programs. A computer program comprises instructions which, when executed on at least one processor of an apparatus, cause the apparatus to carry out any of the respective processing described above. A computer program in this regard may comprise one or more code modules corresponding to the means or units described above.

[0081] Embodiments of the present disclosure further include a carrier containing such a computer program 108. This carrier may comprise one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0082] In this regard, embodiments herein also include a computer program product stored on a non-transitory computer readable (storage or recording) medium and comprising instructions that, when executed by a processor of an apparatus, cause the apparatus to perform as described above.

[0083] Embodiments further include a computer program product comprising program code portions for performing the steps of any of the embodiments herein when the computer program product is executed by network node 100. This computer program product may be stored on a computer readable recording medium.

[0084] Figure 7 is a functional block diagram illustrating some of the main functional components of a client device 200 configured to receive video streams from a network node according to embodiments of the present disclosure. In one embodiment, client device 200 is a head-mounted display configured to receive video streams processes by network node 100. However, as those of ordinary skill in the art will readily appreciate, this is merely for illustrative purposes. In other embodiments, for example, client device 200 is a user’s mobile device operating in the communications network. Thus, the particular identification of the client device 200 as being a head-mounted display (or any other specific device) is merely illustrative.

[0085] As seen in Figure 7, client device 200 in this embodiment comprises communication circuitry 202, processing circuitry 204, and memory 206 configured to store a computer program 208 thereon. In addition, although not specifically shown, the client device 200 may include one or more antennas and corresponding circuitry that configures the client device 200 to wirelessly connect client device 200 to one or more different nodes in the communications network, and to receive data and information from those nodes over an air interface. Such data and information includes, but is not limited to, the graphics layers processed and sent by network node 200, as previously described.

[0086] The communication circuitry 202 comprises the hardware required for communicating with one or more other network nodes (e.g., core network nodes, etc.), as well as with one or more nodes in a radio access network (RAN), such as a base station, for example. In this regard, the communication circuitry 202 may comprise a Network Interface Circuit (NIC) (e.g., an ETHERNET or similar interface). In some embodiments, the client device 200 may be configured for wireless communications, and thus, communication circuitry 202 would comprise the radio frequency (RF) circuitry needed for transmitting and receiving signals over a wireless communication channel. In these cases, client device 200 may also be coupled to one or more antenna (not shown). In other embodiments, however, client device 200 is configured to communicate via a wire interface.

[0087] The processing circuitry 204 comprises one or more microprocessors, hardware, firmware, or a combination thereof that controls the overall operation of client device 200. In accordance with the present disclosure, the processing circuitry 204 can be configured by software to perform one or more of the methods herein described including method 80 seen in Figure 4.

[0088] Memory 206 comprises both volatile and non-volatile memory for storing computer program code and data needed by the processing circuitry 204 for operation. Memory 206 may comprise any tangible, non-transitory computer-readable storage medium for storing data including electronic, magnetic, optical, electromagnetic, or semiconductor data storage. Memory 206 stores a computer program 208 comprising executable instructions that configure the processing circuit 204 in client device 200 to perform method 80 seen in Figure 4. A computer program 208 in this regard may comprise one or more code modules corresponding to the means or units described above.

[0089] In general, computer program instructions and configuration information are stored in a non-volatile memory, such as a ROM, erasable programmable read only memory (EPROM) or flash memory. Temporary data generated during operation may be stored in a volatile memory, such as a random-access memory (RAM). In some embodiments, computer program 208 for configuring the processing circuitry 204 as herein described may be stored in a removable memory, such as a portable compact disc, portable digital video disc, or other removable media. The computer program 208 may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0090] Those skilled in the art will also appreciate that embodiments herein further include corresponding computer programs. A computer program comprises instructions which, when executed on at least one processor of an apparatus, cause the apparatus to carry out any of the respective processing described above. A computer program in this regard may comprise one or more code modules corresponding to the means or units described above.

[0091] Embodiments of the present disclosure further include a carrier containing such a computer program 208. This carrier may comprise one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0092] In this regard, embodiments herein also include a computer program product stored on a non-transitory computer readable (storage or recording) medium and comprising instructions that, when executed by a processor of an apparatus, cause the apparatus to perform as described above.

[0093] Embodiments further include a computer program product comprising program code portions for performing the steps of any of the embodiments herein when the computer program product is executed by client device 200. This computer program product may be stored on a computer readable recording medium.

[0094] Additional embodiments will now be described. At least some of these embodiments may be described as applicable in certain contexts and / or wireless network types for illustrative purposes, but the embodiments are similarly applicable in other contexts and / or wireless network types not explicitly described.

[0095] Figure 8 shows an example of a communication system 1100 in accordance with some embodiments.

[0096] In the example, the communication system 1100 includes a telecommunication network 1102 that includes an access network 1104, such as a radio access network (RAN), and a core network 1106, which includes one or more core network nodes 1108. The access network 1104 includes one or more access network nodes, such as network nodes 1110a and 1110b (one or more of which may be generally referred to as network nodes 1110), or any other similar 3rd Generation Partnership Project (3GPP) access nodes or non-3GPP access points. Moreover, as will be appreciated by those of skill in the art, a network node is not necessarily limited to an implementation in which a radio portion and a baseband portion are supplied and integrated by a single vendor. Thus, it will be understood that network nodes include disaggregated implementations or portions thereof. For example, in some embodiments, the telecommunication network 1102 includes one or more Open-RAN (ORAN) network nodes. An ORAN network node is a node in the telecommunication network 1102 that supports an ORAN specification (e.g., a specification published by the O-RAN Alliance, or any similar organization) and may operate alone or together with other nodes to implement one or more functionalities of any node in the telecommunication network 1102, including one or more network nodes 1110 and / or core network nodes 1108.

[0097] Examples of an ORAN network node include an open radio unit (O-RU), an open distributed unit (O-DU), an open central unit (O-CU), including an O-CU control plane (O-CU- CP) or an O-CU user plane (O-CU-UP), a RAN intelligent controller (near-real time or non-real time) hosting software or software plug-ins, such as a near-real time control application (e.g., xApp) or a non-real time control application (e.g., rApp), or any combination thereof (the adjective “open” designating support of an ORAN specification). The network node may support a specification by, for example, supporting an interface defined by the ORAN specification, such as an A1 , F1 , W1 , E1 , E2, X2, Xn interface, an open fronthaul user plane interface, or an open fronthaul management plane interface. Moreover, an ORAN access node may be a logical node in a physical node. Furthermore, an ORAN network node may be implemented in a virtualization environment (described further below) in which one or more network functions are virtualized. For example, the virtualization environment may include an O-Cloud computing platform orchestrated by a Service Management and Orchestration Framework via an 0-2 interface defined by the O-RAN Alliance or comparable technologies. The network nodes 1110 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 1112A, 1112B, 1112C, and 1112D (one or more of which may be generally referred to as UEs 1112) to the core network 1106 over one or more wireless connections.

[0098] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 1100 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication system 1100 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0099] The UEs 1112 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 1110 and other communication devices. Similarly, the network nodes 1110 are arranged, capable, configured, and / or operable to communicate directly or indirectly with the UEs 1112 and / or with other network nodes or equipment in the telecommunication network 1102 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunication network 1102. In the depicted example, the core network 1106 connects the network nodes 1110 to one or more hosts, such as host 1116. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 1106 includes one more core network nodes (e.g., core network node 1108) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 1108. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0100] The host 1116 may be under the ownership or control of a service provider other than an operator or provider of the access network 1104 and / or the telecommunication network 1102, and may be operated by the service provider or on behalf of the service provider. The host 1116 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0101] As a whole, the communication system 1100 of Figure 8 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low- power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0102] In some examples, the telecommunication network 1102 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 1102 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 1102. For example, the telecommunications network 1102 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC)ZMassive loT services to yet further UEs.

[0103] In some examples, the UEs 1112 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 1104 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 1104. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).

[0104] In the example, the hub 1114 communicates with the access network 1104 to facilitate indirect communication between one or more UEs (e.g., UE 1112c and / or 1112D) and network nodes (e.g., network node 1110B). In some examples, the hub 1114 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 1114 may be a broadband router enabling access to the core network 1106 for the UEs. As another example, the hub 1114 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 1110, or by executable code, script, process, or other instructions in the hub 1114. As another example, the hub 1114 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 1114 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 1114 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 1114 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, the hub 1114 acts as a proxy server or orchestrator for the UEs, in particular if one or more of the UEs are low energy loT devices.

[0105] The hub 1114 may have a constant / persistent or intermittent connection to the network node 1110B. The hub 1114 may also allow for a different communication scheme and / or schedule between the hub 1114 and UEs (e.g., UE 1112C and / or 1112D), and between the hub 1114 and the core network 1106. In other examples, the hub 1114 is connected to the core network 1106 and / or one or more UEs via a wired connection. Moreover, the hub 1114 may be configured to connect to an M2M service provider over the access network 1104 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 1110 while still connected via the hub 1114 via a wired or wireless connection. In some embodiments, the hub 1114 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the UEs from / to the network node 1110B. In other embodiments, the hub 1114 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 1110B, but which is additionally capable of operating as a communication start and / or end point for certain data channels.

[0106] Figure 9 shows a UE 1200 in accordance with some embodiments. As used herein, a UE refers to a device capable, configured, arranged and / or operable to communicate wirelessly with network nodes and / or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle, vehicle-mounted or vehicle embedded / integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB- loT) UE, a machine type communication (MTC) UE, and / or an enhanced MTC (eMTC) UE.

[0107] A UE may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).

[0108] The UE 1200 includes processing circuitry 1202 that is operatively coupled via a bus 1204 to an input / output interface 1206, a power source 1208, a memory 1210, a communication interface 1212, and / or any other component, or any combination thereof. Certain UEs may utilize all or a subset of the components shown in Figure 9. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0109] The processing circuitry 1202 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory 1210. The processing circuitry 1202 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 1202 may include multiple central processing units (CPUs).

[0110] In the example, the input / output interface 1206 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into the UE 1200. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.

[0111] In some embodiments, the power source 1208 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 1208 may further include power circuitry for delivering power from the power source 1208 itself, and / or an external power source, to the various parts of the UE 1200 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 1208. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 1208 to make the power suitable for the respective components of the UE 1200 to which power is supplied.

[0112] The memory 1210 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 1210 includes one or more application programs 1214, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 1216. The memory 1210 may store, for use by the UE 1200, any of a variety of various operating systems or combinations of operating systems.

[0113] The memory 1210 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUlCC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’ The memory 1210 may allow the UE 1200 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 1210, which may be or comprise a device-readable storage medium.

[0114] The processing circuitry 1202 may be configured to communicate with an access network or other network using the communication interface 1212. The communication interface 1212 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 1222. The communication interface 1212 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network). Each transceiver may include a transmitter 1218 and / or a receiver 1220 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 1218 and receiver 1220 may be coupled to one or more antennas (e.g., antenna 1222) and may share circuit components, software or firmware, or alternatively be implemented separately.

[0115] In the illustrated embodiment, communication functions of the communication interface 1212 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in according to one or more communication protocols and / or standards, such as IEEE 802.11 , Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / internet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.

[0116] Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 1212, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).

[0117] As another example, a UE comprises a mobile device, such as phone or tablet having a depth sensor, Augmented Reality (AR) glasses, a Mixed Reality (MR) device, a Virtual Reality (VR) device, an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.

[0118] A UE, when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item-tracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an loT device comprises circuitry and / or software in dependence of the intended application of the loT device in addition to other components as described in relation to the UE 1200 shown in Figure 9.

[0119] As yet another specific example, in an loT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements, and transmits the results of such monitoring and / or measurements to another UE and / or a network node. The UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3GPP NB-loT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and / or reporting on its operational status or other functions associated with its operation.

[0120] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone’s speed. The first and / or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.

[0121] Figure 10 shows a network node 1300 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and / or operable to communicate directly or indirectly with a UE and / or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)), O-RAN nodes or components of an O-RAN node (e.g., O-RU, O-DU, O-CU).

[0122] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units, distributed units (e.g., in an O-RAN access node) and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0123] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs).

[0124] The network node 1300 includes a processing circuitry 1302, a memory 1304, a communication interface 1306, and a power source 1308. The network node 1300 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 1300 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 1300 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 1304 for different RATs) and some components may be reused (e.g., a same antenna 1310 may be shared by different RATs). The network node 1300 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 1300, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 1300.

[0125] The processing circuitry 1302 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 1300 components, such as the memory 1304, to provide network node 1300 functionality.

[0126] In some embodiments, the processing circuitry 1302 includes a system on a chip (SOC). In some embodiments, the processing circuitry 1302 includes one or more of radio frequency (RF) transceiver circuitry 1312 and baseband processing circuitry 1314. In some embodiments, the radio frequency (RF) transceiver circuitry 1312 and the baseband processing circuitry 1314 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 1312 and baseband processing circuitry 1314 may be on the same chip or set of chips, boards, or units.

[0127] The memory 1304 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by the processing circuitry 1302. The memory 1304 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions capable of being executed by the processing circuitry 1302 and utilized by the network node 1300. The memory 1304 may be used to store any calculations made by the processing circuitry 1302 and / or any data received via the communication interface 1306. In some embodiments, the processing circuitry 1302 and memory 1304 is integrated.

[0128] The communication interface 1306 is used in wired or wireless communication of signaling and / or data between a network node, access network, and / or UE. As illustrated, the communication interface 1306 comprises port(s) / terminal(s) 1316 to send and receive data, for example to and from a network over a wired connection. The communication interface 1306 also includes radio front-end circuitry 1318 that may be coupled to, or in certain embodiments a part of, the antenna 1310. Radio front-end circuitry 1318 comprises filters 1320 and amplifiers 1322. The radio front-end circuitry 1318 may be connected to an antenna 1310 and processing circuitry 1302. The radio front-end circuitry may be configured to condition signals communicated between antenna 1310 and processing circuitry 1302. The radio front-end circuitry 1318 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 1318 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 1320 and / or amplifiers 1322. The radio signal may then be transmitted via the antenna 1310. Similarly, when receiving data, the antenna 1310 may collect radio signals which are then converted into digital data by the radio front-end circuitry 1318. The digital data may be passed to the processing circuitry 1302. In other embodiments, the communication interface may comprise different components and / or different combinations of components.

[0129] In certain alternative embodiments, the network node 1300 does not include separate radio front-end circuitry 1318, instead, the processing circuitry 1302 includes radio front-end circuitry and is connected to the antenna 1310. Similarly, in some embodiments, all or some of the RF transceiver circuitry 1312 is part of the communication interface 1306. In still other embodiments, the communication interface 1306 includes one or more ports or terminals 1316, the radio front-end circuitry 1318, and the RF transceiver circuitry 1312, as part of a radio unit (not shown), and the communication interface 1306 communicates with the baseband processing circuitry 1314, which is part of a digital unit (not shown).

[0130] The antenna 1310 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. The antenna 1310 may be coupled to the radio front-end circuitry 1318 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, the antenna 1310 is separate from the network node 1300 and connectable to the network node 1300 through an interface or port.

[0131] The antenna 1310, communication interface 1306, and / or the processing circuitry 1302 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node. Any information, data and / or signals may be received from a UE, another network node and / or any other network equipment. Similarly, the antenna 1310, the communication interface 1306, and / or the processing circuitry 1302 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node and / or any other network equipment.

[0132] The power source 1308 provides power to the various components of network node 1300 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 1308 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 1300 with power for performing the functionality described herein. For example, the network node 1300 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 1308. As a further example, the power source 1308 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[0133] Embodiments of the network node 1300 may include additional components beyond those shown in Figure 10 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, the network node 1300 may include user interface equipment to allow input of information into the network node 1300 and to allow output of information from the network node 1300. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 1300.

[0134] Figure 11 is a block diagram of a host 1400, which may be an embodiment of the host 1116 of Figure 8, in accordance with various aspects described herein. As used herein, the host 1400 may be or comprise various combinations hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. The host 1400 may provide one or more services to one or more UEs.

[0135] The host 1400 includes processing circuitry 1402 that is operatively coupled via a bus 1404 to an input / output interface 1406, a network interface 1408, a power source 1410, and a memory 1412. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 18 and 19, such that the descriptions thereof are generally applicable to the corresponding components of host 1400.

[0136] The memory 1412 may include one or more computer programs including one or more host application programs 1414 and data 1416, which may include user data, e.g., data generated by a UE for the host 1400 or data generated by the host 1400 for a UE. Embodiments of the host 1400 may utilize only a subset or all of the components shown. The host application programs 1414 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (WC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application programs 1414 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, the host 1400 may select and / or indicate a different host for over-the-top services for a UE. The host application programs 1414 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[0137] Figure 12 is a block diagram illustrating a virtualization environment 1500 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1500 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized. In some embodiments, the virtualization environment 1500 includes components defined by the O-RAN Alliance, such as an O-Cloud environment orchestrated by a Service Management and Orchestration Framework via an O-2 interface.

[0138] Applications 1502 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment Q400 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.

[0139] Hardware 1504 includes processing circuitry, memory that stores software and / or instructions executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1506 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1508A and 1508B (one or more of which may be generally referred to as VMs 1508), and / or perform any of the functions, features and / or benefits described in relation with some embodiments described herein. The virtualization layer 1506 may present a virtual operating platform that appears like networking hardware to the VMs 1508.

[0140] The VMs 1508 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1506. Different embodiments of the instance of a virtual appliance 1502 may be implemented on one or more of VMs 1508, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

[0141] In the context of NFV, a VM 1508 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each of the VMs 1508, and that part of hardware 1504 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1508 on top of the hardware 1504 and corresponds to the application 1502.

[0142] Hardware 1504 may be implemented in a standalone network node with generic or specific components. Hardware 1504 may implement some functions via virtualization. Alternatively, hardware 1504 may be part of a larger cluster of hardware (e.g. such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1510, which, among others, oversees lifecycle management of applications 1502. In some embodiments, hardware 1504 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 1512 which may alternatively be used for communication between hardware nodes and radio units.

[0143] Figure 13 shows a communication diagram of a host 1602 communicating via a network node 1604 with a UE 1606 over a partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a UE 1112A of Figure 8 and / or UE 1200 of Figure 9), network node (such as network node 1110A of Figure 8 and / or network node 1300 of Figure 10), and host (such as host 1116 of Figure 8 and / or host 1400 of Figure 11) discussed in the preceding paragraphs will now be described with reference to Figure 13.

[0144] Like host 1400, embodiments of host 1602 include hardware, such as a communication interface, processing circuitry, and memory. The host 1602 also includes software, which is stored in or accessible by the host 1602 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 1606 connecting via an over-the-top (OTT) connection 1650 extending between the UE 1606 and host 1602. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 1650.

[0145] The network node 1604 includes hardware enabling it to communicate with the host 1602 and UE 1606. The connection 1660 may be direct or pass through a core network (like core network 1106 of Figure 8) and / or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.

[0146] The UE 1606 includes hardware and software, which is stored in or accessible by UE 1606 and executable by the UE’s processing circuitry. The software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 1606 with the support of the host 1602. In the host 1602, an executing host application may communicate with the executing client application via the OTT connection 1650 terminating at the UE 1606 and host 1602. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 1650 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 1650.

[0147] The OTT connection 1650 may extend via a connection 1660 between the host 1602 and the network node 1604 and via a wireless connection 1670 between the network node 1604 and the UE 1606 to provide the connection between the host 1602 and the UE 1606. The connection 1660 and wireless connection 1670, over which the OTT connection 1650 may be provided, have been drawn abstractly to illustrate the communication between the host 1602 and the UE 1606 via the network node 1604, without explicit reference to any intermediary devices and the precise routing of messages via these devices.

[0148] As an example of transmitting data via the OTT connection 1650, in step 1608, the host 1602 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 1606. In other embodiments, the user data is associated with a UE 1606 that shares data with the host 1602 without explicit human interaction. In step 1610, the host 1602 initiates a transmission carrying the user data towards the UE 1606. The host 1602 may initiate the transmission responsive to a request transmitted by the UE 1606. The request may be caused by human interaction with the UE 1606 or by operation of the client application executing on the UE 1606. The transmission may pass via the network node 1604, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1612, the network node 1604 transmits to the UE 1606 the user data that was carried in the transmission that the host 1602 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1614, the UE 1606 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 1606 associated with the host application executed by the host 1602.

[0149] In some examples, the UE 1606 executes a client application which provides user data to the host 1602. The user data may be provided in reaction or response to the data received from the host 1602. Accordingly, in step 1616, the UE 1606 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input / output interface of the UE 1606. Regardless of the specific manner in which the user data was provided, the UE 1606 initiates, in step 1618, transmission of the user data towards the host 1602 via the network node 1604. In step 1620, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 1604 receives user data from the UE 1606 and initiates transmission of the received user data towards the host 1602. In step 1622, the host 1602 receives the user data carried in the transmission initiated by the UE 1606.

[0150] One or more of the various embodiments improve the performance of OTT services provided to the UE 1606 using the OTT connection 1650, in which the wireless connection 1670 forms the last segment. More precisely, the teachings of these embodiments provide the capability to react much faster to received congestion feedback than do conventional systems. This decreases feedback loop latency such that an encoder can begin encoding a next frame / slice faster than conventionally configured encoders.

[0151] The teachings of these embodiments also provides the capability to reduce the amount of peak burst size sent at a given time by sending, on average, N times less data in a burst. Additionally, the transmission times of the video data packets are distributed more evenly during a frame interval, and the present embodiments will not increase the end-to-end (e2e) application latency.

[0152] In an example scenario, factory status information may be collected and analyzed by the host 1602. As another example, the host 1602 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, the host 1602 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 1602 may store surveillance video uploaded by a UE. As another example, the host 1602 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 1602 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and / or transmitting data.

[0153] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 1650 between the host 1602 and UE 1606, in response to variations in the measurement results. The measurement procedure and / or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 1602 and / or UE 1606. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 1650 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 1650 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 1604. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 1602. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or ‘dummy’ messages, using the OTT connection 1650 while monitoring propagation times, errors, etc.

[0154] Although the computing devices described herein (e.g., UEs, network nodes, hosts) may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and / or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and / or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.

[0155] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer- readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer- readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and / or by end users and a wireless network generally.

[0156] The present embodiments may, of course, be carried out in other ways than those specifically set forth herein without departing from characteristics described herein. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, and all changes coming within the meaning and equivalency range of the appended claims are intended to be embraced therein.

Claims

CLAIMS1 . A method (90), implemented at a network node (100), for sending video to a client device (200), the method comprising: detecting (92) a plurality of timed trigger signals within a video frame interval; and responsive to detecting each timer signal: generating (94) video data for display at the client device; and sending (96) the video data to the client device in a video frame.

2. The method of claim 1 , wherein the trigger signals are timed to occur at a frequency of:1NxFPS ’ wherein:N is an integer value > 1 ; andFPS is Frames Per Second3. The method of any of claims 1 or 2, wherein generating video data for display at the client device comprises updating (64) one or more 3-Dimensional (3D) objects in a video scene using a 3-Dimensional (3D) simulator.

4. The method of any of claims 1-3, wherein generating video data for display at the client device further comprises: selecting (66) a graphics layer to render from among a plurality of graphics layers; selecting (68) a set of one or more virtual objects associated with the graphics layer; and rendering (70) the set of one or more virtual objects to the graphics layer.

5. The method of claim 4, wherein the graphics layer is selected round-robin such that each of the plurality of graphics layers is rendered within a single frame time interval.

6. The method of claim 4, wherein each virtual object in the set of one or more virtual objects is mapped to the graphics layer based on a distance of the virtual object from a camera that captured a video stream.

7. The method of any of claims 1-6, wherein generating video data for display at the client device further comprises encoding (74) the graphics layer and the one or more virtual objects to generate an encoded video frame.

8. The method of claim 7, wherein encoding the graphics layer and the one or more virtual objects comprises: selecting (72) an encoder configuration based on a configuration assigned to the graphic layer; and encoding (74) the graphics layer and the one or more virtual objects according to the encoder configuration.

9. The method of claim 8, wherein a reference frame is modified such that a current picture associated with the graphics layer refers to a previous picture associated with the same graphics layer.

10. The method of any of claims 1 -9, wherein the encoded video frame is sent to the client device in a burst of video packets.

11. The method of any of claims 1-10, wherein the video frame is one of a plurality of video frames that are sent to the client device during the video frame interval.

12. The method of claim 11 , wherein the plurality of video frames are distributed evenly throughout the video frame interval.

13. The method of any of claims 1-12, wherein the network node comprises an extended Reality (XR) server in a cloud network.

14. A network node (100) for sending video to a client device (200), the network node configured to: detect (92) a plurality of timed trigger signals within a video frame interval; and responsive to detecting each timer signal: generate (94) video data for display at the client device; and send (96) the video data to the client device in a video frame.

15. The network node of claim 14, wherein the network node is further configured to perform the method according to any one of claims 2-13.

16. A network node (100) for sending video to a client device (200), the network node comprising: processing circuitry (104); and memory (106) configured to store instructions (108) executable by the processing circuitry, whereby the network node is configured to:detect (92) a plurality of timed trigger signals within a video frame interval; and responsive to detecting each timer signal: generate (94) video data for display at the client device; and send (96) the video data to the client device in a video frame.

17. The network node of claim 16, wherein the network node is further configured to perform the method according to any one of claims 2-13.

18. A computer program (108) comprising instructions that, when executed on processing circuitry (102) of a network node (100), cause the network node to perform the method according to any of claims 2-13.

19. A carrier containing the computer program of claim 18, wherein the carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

20. A non-transitory computer-readable storage medium (106) comprising a computer program (108) stored thereon, the computer program comprising executable instructions that, when executed by processing circuitry (102) in a network node (100), causes the network node to perform the method of any one of claims 1-13.

Images