Remote data processing migration between servers

Abstract

A gateway system for use as a gateway between a client device and a remote server is described. The client device can stream a data stream to the remote server via the gateway system for remote processing. The gateway system can receive the data stream and a marker identifying respective blocks of data packets in the data stream, where each identified block of data packets is independently processable. In response to an instruction or configuration to effectuate a migration of the remote processing from the remote server to another remote server, the gateway system can determine a block of data packets in the data stream from which the other remote server is to begin processing the data stream in order to effectuate or initialize the migration, and stream the determined block of data packets and subsequent blocks of data packets of the data stream to the other remote server as received.

Description

Technical Field

[0001] This invention relates to remotely processing data streams from client devices via remote servers, such as cloud servers or edge nodes in telecommunications networks. The invention further relates to migrating remote processing from one remote server to another. The invention further relates to a gateway system for acting as a gateway between a client device and a remote server, and to a computer-implemented method for providing corresponding gateway functionality. The invention further relates to the client device and a corresponding computer-implemented method, to the remote server and a corresponding computer-implemented method, and to an orchestration system and computer-implemented method for orchestrating remote processing migrations. The invention further relates to a computer-readable medium including a computer program, and to a computer-readable medium including data packets comprising data streams. Background Technology

[0002] Multi-access edge computing (MEC)[1] (or simply edge computing) is a computing paradigm in which computing nodes can be located relatively close to client devices (e.g., at the edge of a mobile network). Therefore, such computing nodes can also be called edge nodes, and their location enables high-bandwidth and low-latency connections between client devices and edge nodes. Thus, edge computing can be seen as a enabler for applications that have high demands on computing and bandwidth resources, including real-time data processing and high-definition extended (e.g., augmented or virtual) reality streaming applications.

[0003] A typical use case for edge computing could be that client devices can send continuous, high-bandwidth data streams to edge nodes, where the edge nodes process the data streams to obtain output (e.g., analytics results), and then transmit the output to a cloud server or other receiving entity. For example, a lightly compressed high-definition video stream can be uploaded to an edge node; the edge node can then analyze the video stream to extract metadata tags from it, and potentially forward the metadata tags along with a heavily compressed version of the video stream to a cloud server or other receiving entity, thereby imposing lighter networking requirements on the backhaul network upstream of the edge node.

[0004] While the location of edge nodes near client devices is often a primary advantage from a performance perspective, it's possible that edge processing functionalities for a particular application might need to be migrated to another edge node. For example, edge migration might be triggered by client mobility (e.g., another edge node could provide better connectivity to client devices) or by resource optimization by the edge provider (e.g., by reducing the load on heavily loaded edge nodes by moving parts of the processing to another edge node, or by merging processing on multiple lightly loaded edge nodes onto a single node). Migration can occur between edge clusters located in physically different locations or between servers within a cluster.

[0005] However, when performing migrations between edge nodes, applications that rely on a continuous stream of data from client devices to the edge nodes may experience processing interruptions. Such interruptions may also be undesirable in other cases where remote servers, in addition to edge nodes, also process the data streams from client devices, such as in cloud computing where the data streams from client devices can be processed by one or more cloud servers. These servers and their processing may also be simply referred to below as remote servers and remote processing, and include the aforementioned edge processing performed by edge nodes and cloud processing performed by cloud servers.

[0006] It may be desirable to enable more seamless migration between edge nodes, or between remote servers that typically handle data streams from client devices.

[0007] Govindaraj et al. [2] described a redundant migration scheme. It is said that, starting from the time instance that triggers the migration, the client launches a stream of packets to the destination server. The destination server buffers these packets and forwards them to the source server. Meanwhile, while the service to be migrated continues to run on the source server, the source server begins iteratively launching files and checkpoints associated with that service to the destination service. The destination server recovers the checkpoints provided by the source server. Once the service is up and running, the destination server begins replaying all the buffered packets, which involves precisely replaying the packets from the checkpoints to catch up with the state of the source server. Once caught up, the outputs of the two servers are compared, and if they are identical, the destination server declares itself ready to take over.

[0008] Disadvantageously, in many applications, packet buffering during migration and subsequent processing at higher speeds for catch-up ('replay') prior to final takeover ('switchover phase', see Figure 2 in [2]) may be infeasible. For example, in video processing, the computational complexity of processing at speeds higher than real-time may be (too) high. Therefore, catch-up may take a long time or may require a significant or even excessive temporary increase in computing power.

[0009] In [3], Govindaraj et al. described a non-disruptive hot backup migration based on a redundant migration scheme, but combined with the concept of redundant servers. That is, both the source server and the destination server are now redundantly instantiated. During the migration, a snapshot of the secondary source server is created while the primary source server continues to serve the client devices without interruption. The snapshots are restored simultaneously on both destination servers. After the application is received on the destination servers, the two destination servers replay the packets buffered by the SDN switch. After replay, the outputs of all four servers are compared, and if they are the same, the destination server takes over the service.

[0010] [3] He himself admitted that the drawback is that the resource overhead of redundant servers is very high because the resources are actually doubled.

[0011] References

[0012] [1] ETSI White Paper No. 28 - MEC in 5G networks; First Edition – June 2018, ISBN 979-10-92620-22-1.

[0013] [2] Govindaraj, K. and Artemenko, A. Container live migration for latency-critical industrial applications on edge computing. IEEE International Conference on Emerging Technologies and Factory Automation, Vol. 1, pp. 83-90, 23rd, 2018.

[0014] [3] Govindaraj, K., Saha, M., Artemenko, A., and Kirstaedter, A. Investigation of Uninterrupted Service Live Migration Using Software-Defined Networking. International Conference on Networking Systems 2019, pp. 1-6. Summary of the Invention

[0015] It may be desirable to enable remote processing of client data streams from one remote server to another in a way that avoids or at least reduces disruptions in data stream processing, while addressing at least one of the drawbacks of [2] and [3], such as the need to catch up with faster processing and / or the requirement to instantiate all remote servers twice.

[0016] According to a first aspect of the present invention, a gateway system can be provided for use as a gateway between a client device and a remote server, wherein the client device can be configured to stream data to the remote server via the gateway system for remote processing. The gateway system may include:

[0017] - A network interface for receiving the data stream and tags that identify corresponding data packet blocks in the data stream, wherein each identified data packet block may be independently processable;

[0018] - A processor subsystem that can be configured to use a network interface to perform the following operations:

[0019] - In response to instructions or configurations for migrating the remote processing from one remote server to another, a data packet block is identified in the data stream, from which the other remote server begins processing the data stream to implement or initialize the migration;

[0020] - When the identified data packet block is received, the identified data packet block and subsequent data packet blocks of the data stream are streamed to the other remote server.

[0021] According to another aspect of the invention, a client device can be provided and configured to stream data to a remote server for remote processing. The client device may include:

[0022] -The network interface to the network;

[0023] - A processor subsystem that can be configured to use a network interface to perform the following operations:

[0024] - Before or simultaneously with streaming the data stream, generate tags for the data stream that identify the corresponding data packet blocks in the data stream, wherein each identified data packet block can be processed independently;

[0025] - Stream the data stream along with these tags so that the gateway receiving the data stream can identify the packet blocks in the data stream, starting from which another remote server will begin processing the data stream to implement or initiate a migration from the first remote server to the second remote server.

[0026] According to another aspect of the invention, an orchestration system can be provided for orchestrating the migration of a data stream from one remote server to another, wherein the data stream can be streamed from a client device to a gateway system, and then streamed from the gateway system to the remote server prior to the migration. The orchestration system may include:

[0027] -The network interface to the network;

[0028] - A processor subsystem that can be configured to use a network interface to perform the following operations:

[0029] - Configure the other remote server to perform the remote processing;

[0030] - Send an instruction message to the gateway system to implement or initiate the migration of remote processing from one remote server to another by streaming the data stream to the other remote server, wherein the instruction message may instruct the gateway system to determine when to start streaming the data stream to the other remote server based on tags that mark the data stream, wherein each tag may identify a corresponding data packet block in the data stream, and each identified data packet block is independently processable.

[0031] According to another aspect of the present invention, a server for processing data streams transmitted by client devices can be provided. The server may include:

[0032] - A network interface used to receive data packets from this data stream;

[0033] - A processor subsystem configured to perform the following operations via the network interface:

[0034] - In operating mode, the data stream is processed to generate output for use by the client device or another entity, wherein the data stream comprises data packet blocks, wherein each data packet block is independently processable, and wherein the processor subsystem is configured to process the data stream by at least initially processing these data packet blocks independently; and

[0035] - Before invoking this operation mode, the output is not generated in response to instructions or configurations that indicate the initiation of processing of the data stream.

[0036] According to another aspect of the present invention, a computer-implemented method can be provided for providing gateway functionality between a client device and a remote server, wherein the client device can be configured to stream data to the remote server via the gateway functionality for remote processing. The method may include:

[0037] - Receive the data stream and the tags that identify the corresponding data packets in the data stream, wherein each identified data packet can be processed independently;

[0038] - In response to instructions or configurations for migrating the remote processing from one remote server to another, a data packet block is identified in the data stream, from which the other remote server begins processing the data stream to implement or initialize the migration;

[0039] - When the identified data packet block is received, the identified data packet block and subsequent data packet blocks of the data stream are streamed to the other remote server.

[0040] According to another aspect of the present invention, a computer-implemented method can be provided for streaming data to a remote server for remote processing. The method may include:

[0041] - Before or simultaneously with streaming the data stream, generate tags for the data stream that identify the corresponding data packet blocks in the data stream, wherein each identified data packet block can be processed independently;

[0042] - Stream the data stream along with these tags so that the gateway receiving the data stream can identify the packet blocks in the data stream, starting from which another remote server will begin processing the data stream to implement or initiate a migration from the first remote server to the second remote server.

[0043] According to another aspect of the invention, a computer-implemented method can be provided for orchestrating the migration of a data stream from one remote server to another, wherein the data stream can be streamed from a client device to a gateway system and then streamed from the gateway system to the remote server prior to the migration. The method may include:

[0044] - Configure the other remote server to perform the remote processing;

[0045] - Send an instruction message to the gateway system to implement or initiate the migration of the remote processing from the remote server to the other remote server by streaming the data stream to the other remote server, wherein the instruction message may instruct the gateway system to determine when to start streaming the data stream to the other remote server based on tags that mark the data stream, wherein each tag may identify a corresponding data packet block in the data stream, and each identified data packet block may be independently processable.

[0046] According to another aspect of the present invention, a computer-implemented method can be provided for processing a data stream streamed by a client device. The method may include:

[0047] - Receive the data packets of this data stream;

[0048] - Process the data stream to generate output for use by the client device or another entity, wherein the data stream includes data packet blocks, wherein each data packet block may be independently processable, and wherein the processing may include at least initially processing these data packet blocks independently; and

[0049] - The output is not generated before the data stream is processed to generate the output, in response to an instruction or configuration that instructs the processing of the data stream to be initialized.

[0050] According to another aspect of the invention, a computer-readable medium may include transient or non-transitory data representing a computer program. The computer program may include instructions for causing a processor system to perform any computer-implemented method described herein.

[0051] The above measures provide a client device that can transmit a data stream for remote processing. The data transmitted via the data stream may, but does not need to, be acquired by the client device itself. For example, the client device may use multiple sensors to capture sensor data and encode the sensor data into a data stream in real-time or near real-time and transmit it. In other examples, the client device may transmit a data stream that is not captured in real-time and / or is not captured by the client device itself, but rather, for example, forwarded or retrieved from a storage device. Typically, for example, applications that manipulate the data stream may prefer or require processing the data stream in a substantially uninterrupted and continuous manner.

[0052] A remote server can be provided to process the data stream, for example, by applying image or video analysis algorithms to the data stream, by transcoding the data stream, by enhancing the data stream, or in any other way. The output of the remote server can also be another data stream, such as a processed version of the original data stream (e.g., transcoded, enhanced, etc.) or a data stream containing processing results (e.g., analysis results). However, the output of the remote server can also be in a non-streaming form, such as a single output instance. Examples of remote servers include, but are not limited to, edge nodes and cloud servers. It should be noted that the term 'remote' can refer to a server being a separate entity from the client device and typically located upstream of the client device in the network, and therefore 'remote' from the client device. However, there is no limitation on the (minimum) physical or network distance between the client device and the server. The term 'remote processing' can simply refer to processing performed by a remote server.

[0053] For example, due to client mobility, overload of the first remote server, or the expectation of cross-server load balancing, it may be decided at some point to migrate the processing of a data stream that could previously be handled remotely by the first remote server to another remote server. These two servers may also be referred to as 'the' and 'the other' or 'another' remote server, or as source server and destination server, or as source edge node and destination edge node, etc.

[0054] To orchestrate remote processing migrations, an orchestration system can be provided that includes known functionalities for orchestrating migrations from one remote server to another. For example, the orchestration system can be configured to schedule the transfer of an application and its state from one remote server to another, the application being used to process data streams. The orchestration system can also be configured to determine when to initiate a migration, for example, based on external data such as data characterizing client mobility or server load data.

[0055] Based on the above measures, data streams can be sent to remote servers via a gateway system. Client devices can provide tags to the data streams, which, once indicated or configured to be used, the gateway system can use to send data streams to and / or stop sending data streams to other remote servers. Tags can be generated by the client devices to identify corresponding data packet blocks in the data stream, where each identified data packet block can be processed independently by the remote server. Here, the term 'independently processable' can mean that there are no dependencies between data packet blocks in at least some aspects of the processing by the remote server. For example, data packet blocks can be independently decoded and processed by at least a first processing step (e.g., initial or low-level processing). However, it is not excluded that subsequent processing steps (e.g., higher-level) performed by the remote server can be applied together to the corresponding outputs of the first processing of several blocks, thereby (re)establishing dependencies between consecutive blocks.

[0056] The above measures can be based on the insight that one of the main challenges in migrating remote processing between remote servers may be the potential for time dependencies in processing, meaning that the current state of the processing characterizing packets and therefore the output of that processing may depend on previous packets. This may mean that the destination server cannot simply assume responsibility from the source server at any given moment, for example, from any packet to the next. Instead, it may be necessary to ensure that the processing state is identical between the source and destination servers so that the destination server can take over. In practice, [2] and [3] monitor the output of the source server and (multiple) destination servers to ensure that (multiple) destination servers have taken into account the time dependencies in processing, for example, by processing a sufficient number of consecutive packets before finally completing the migration to (multiple) destination servers.

[0057] Based on the above measures, markers are generated for the data stream. These markers can indicate points in the data stream, i.e., points between blocks, where the destination server can begin processing the data stream without time dependence relative to packets from previous blocks (which may not have been received by the destination server). Therefore, the markers can indicate from which point packets can be processed independently by at least a first (e.g., initial or low-level) processing step performed by the destination server. These markers can be available to the gateway system as part of the data stream or in any other way, enabling the gateway system to determine when to begin sending a data stream to another remote server. For example, if the gateway system is instructed or configured to implement migration, it can monitor the markers to determine when the next block of packets arrives and begin sending packets to the destination server from at least that point for remote processing.

[0058] Therefore, the above measures can avoid the need for buffered packets, as the destination server can be pre-configured for migration and, once configured, is only provided with packets of the data stream. Specifically, based on the flag, the entities involved in the migration know that once the destination server begins processing packets for a new block, the destination server's output can be used to replace the source server's output, since the outputs of the two servers can be identical in at least the first processing step. This flag can also be used to delay the source server's determination of processing until the new block has been at least partially or completely sent to the destination server.

[0059] The advantage of using tags is that the use of these tags avoids the need to buffer packets at or for the destination server, thus avoiding the need to 'catch up' by having to process previously buffered packets, as in [2]. Furthermore, by observing the tags in the data stream, the gateway system can determine that the destination server is ready to take over and therefore stops sending packets to the source server, which then processes the packets. This avoids the need to monitor and compare the outputs of the two servers to determine when the destination server is ready to take over, as in [2] and [3]. That is, such comparisons might need to be performed over a long period to have the same sufficient confidence in the outputs; for example, a comparison of a single output instance might not be sufficient. Moreover, when the gateway system fully redirects the data stream to the destination server at the beginning of a new block, it may not even be necessary to send packets to both servers simultaneously and process them concurrently, as the destination server can take over from the source server after the redirection. This reduces network allocation and the need for duplicate processing. It may also be unnecessary to use duplicate servers as in [3].

[0060] Another advantage is that when migrating between remote servers, state information can be reduced or eliminated from transmission, as it might otherwise be necessary to reflect dependencies in the packet processing of the data stream. In other words, if a migration were to begin at a packet that has a processing dependency on a previous packet, state information might need to be transmitted between remote servers to reflect that dependency. By starting the migration at a block boundary, another remote server can immediately begin processing the block without such state information, since there might not be any dependency on previous blocks, at least during decoding and the first processing steps.

[0061] Another advantage is that, since the gateway system can infer block boundaries based on tags associated with data flows, and thus infer appropriate migration cuts, there may be less or no timed message exchange regarding traffic redirection between another remote server, the gateway system, and / or the orchestration system. Given the speed at which packets are processed in the network, using signals to orchestrate traffic redirection may not be effective. Latencies in transmitting and processing signals can cause packets to be routed to the wrong remote server, leading to data processing corruption and other problems.

[0062] Another advantage of migration based on tags indicating which block a packet belongs to is that this migration can be flexible in terms of packet reordering. For example, packets with older 'pre-migration' tags can still be forwarded to the source remote server, ensuring that the source remote server has all the data from the data block available for processing. Such packets may not be forwarded to the destination remote server, which ensures that the destination remote server edge only receives the packets it should receive from the data block.

[0063] In an embodiment, the gateway system's processor subsystem can be configured to either stop streaming the data stream to a remote server when a determined block of data packets is being streamed to another remote server, or temporarily continue streaming the data stream repeatedly to the remote server. By stopping streaming the data stream to the remote server starting from the determined block of data packets, the gateway system can effectively redirect the data stream from that block and including that block to another remote server. This can also be referred to as 'direct migration' and may involve the gateway system terminating the forwarding of data packets to the remote server when it begins forwarding them. This avoids duplicate traffic to the remote server and the need for processing by the remote server. In contrast, by temporarily continuing to stream the data stream repeatedly to the remote server, for example for a duration indicated by the orchestration system, the remote server can temporarily continue processing the data stream and producing output. This can be advantageous if higher-level processing steps do exhibit dependencies between blocks, in which case the duration can be chosen to ensure that the other remote server has processed a sufficient number of blocks before switching to it. This allows another remote server to have already initialized its entire processing chain from the determined block, while it can continue receiving actual output from the remote server temporarily. Meanwhile, packet copying and parallel routing to both remote servers may only need to be performed as necessary. When the other remote server has received enough data to take over, markers in the data stream can indicate when and where packet copying and routing to the remote server can be stopped.

[0064] In an embodiment, the gateway system's processor subsystem can be configured to initiate streaming of data packets to another remote server in response to an instruction or configuration prior to a determined data packet block, enabling the other remote server to initialize its processing of the data stream. This can also be referred to as 'dirty migration', and may, for example, involve the gateway system immediately initiating forwarding packets of the current block to the other remote server. Although it may be known or anticipated that the other remote server may not yet be able to produce accurate output based on these immediately forwarded packets, such forwarded packets may already allow the other remote server to set up its processing, for example, in the case of a video stream, by inferring which video codec to use from the forwarded packets and subsequently initializing the appropriate decoder pipeline. In other words, the gateway system may have forwarded a portion of the block to allow the other remote server to set up its processing to match the data stream being transmitted.

[0065] In an embodiment, instructions can be received as instruction messages by the gateway system from an orchestration system configured to orchestrate migrations for remote processing, and the gateway system's processor subsystem can be configured to send a response message indicating that the migration has been implemented or initialized to the orchestration system. Instructions for implementing the migration can be received from the orchestration system. To confirm that the migration was implemented or initialized by the gateway system, the gateway system can send a response message to the orchestration system. For example, the gateway system can send a response message once it begins forwarding packets for a defined block to another remote server, or when it stops forwarding packets to a remote server. This response can allow the orchestration system to take further action, such as releasing resources allocated on the remote server for processing.

[0066] In an embodiment, the processor subsystem of the gateway system can be configured to send the data stream to the other remote server via at least one of the following:

[0067] - Forward the data packets of this data stream;

[0068] - Create new data packets using the network address of this other remote server;

[0069] - Modify the network address field in these data packets received from the client device to identify the other remote server, and then send the modified data packets to the other remote server; and

[0070] - If these packets are encapsulated packets containing data to be processed remotely and include these tags in the header of these encapsulated packets, then these encapsulated packets are unpacked and the data to be processed remotely is sent as the data stream to the other remote server.

[0071] Based on the above measures, the gateway system can modify or create new data packets instead of simply retransmitting them, enabling it to send the data stream to another remote server. For example, this allows the gateway system to redirect data packets to another remote server, even if they were originally addressed to that server. Specifically, the gateway system can extract tags from the data stream, such as tags from the headers encapsulating the data packets, and send the data contained within the data stream in a repackaged form to another remote server, since the other remote server may not need these tags.

[0072] In one embodiment, a mobile network may be provided, the mobile network comprising:

[0073] - A first base station, and a first gateway system and a first remote server for serving client devices connected to the first base station;

[0074] - A second base station, and a second gateway system and a second remote server for serving client devices connected to the second base station;

[0075] The first gateway system and the second gateway system are each examples of the gateway system according to any one of claims 1 to 4;

[0076] -The first gateway system is configured as follows:

[0077] - Receive data streams from client devices connected to the first base station and stream those data streams to the first remote server.

[0078] - In response to an instruction or configuration for migrating the remote processing from the first remote server to the second remote server when the client device is expected to switch to the second base station, the data stream is streamed to the second remote server via the second gateway system;

[0079] -The second gateway system is configured to take over the role of the gateway system for the client device from the first gateway system after the client device is switched to the second base station.

[0080] In this embodiment, the processor subsystem of the client device can be configured to include tags in the data stream. The client device can therefore include the tags within the data stream itself, for example, as metadata. This avoids the need to send the tags separately to the gateway system, such as 'out-of-band'.

[0081] In an embodiment, the processor subsystem of the client device can be configured to include a tag in each data packet of the data stream, wherein the tag may include a block identifier that identifies which data packet belongs to which data packet block. By including the block identifier in the data packet, the gateway system can determine which data packet belongs to which block. This, in turn, allows the gateway system to detect changes in the block identifier and thus determine when a newly received data packet belongs to a new block. For example, the tag can be generated as a pseudo-random identifier or by sequentially numbering the blocks.

[0082] In an embodiment, the processor subsystem of the client device may be configured to include these tags in at least one of the following:

[0083] - The header of the transport protocol used to transmit this data stream;

[0084] - The header of the encapsulated data packet that encapsulates the data to be processed remotely;

[0085] - A tunnel established between the client device and the entity receiving the data stream, wherein these tags are included in the encapsulated data packets that form the tunnel.

[0086] The above describes various advantageous methods for providing tags to the gateway system when transmitting separate data streams without requiring another transmission mechanism to set tags to the gateway system.

[0087] The advantage of using packet encapsulation is that it allows the use of non-application-specific ('generic') gateway systems, and the gateway system may not need to access the application data contained in the data stream. Therefore, the gateway system can process packets more efficiently (e.g., without having to parse application data or transport protocols), and can encrypt the application data when necessary.

[0088] The advantage of including tags in the transport protocol is that the gateway system can be implemented using existing programming libraries, and the gateway system can serve multiple applications, provided that they use the same transport protocol and consistently tag packets (e.g., using the same header fields in HTTP requests).

[0089] In an embodiment, the processor subsystem of the orchestration system can be configured to generate instruction messages indicating a block of data packets in the data stream from which the gateway system streams the data stream to the other remote server. For example, the instruction message can indicate an absolute block or a relative block. An example of the latter is an instruction message indicating the 'next' block to be streamed to another remote server. The advantage of indicating a relative block is that neither the orchestration system nor the gateway system may need to know the exact value of the block identifier. Instead, the gateway system can simply monitor or count transitions in the block identifier.

[0090] In an embodiment, the processor subsystem of the orchestration system may be configured to include at least one of the following in the instruction message:

[0091] - The network address of the other remote server;

[0092] - Migration type, which indicates whether to temporarily continue streaming the data stream to the other remote server after the streaming of the data stream has started;

[0093] - Migration duration, which indicates the number of data packets that need to be temporarily and repeatedly streamed to the other remote server after the data stream has begun to be streamed; and

[0094] - Information that informs the gateway system how to identify these tags in the data stream or in the metadata associated with the data stream.

[0095] In addition to instructing a migration to be performed, the orchestration system may provide further information in its instructions to the gateway system, such as the network address of the other remote server, the migration type (e.g., the aforementioned 'direct' or 'dirty' migration type, or the 'load' migration type as described elsewhere in this specification), for some migration types, the number of packets to be repeatedly sent to the two remote servers, and / or information from the gateway system regarding how to identify tags in the data stream or in the metadata associated with the data stream. The latter type of information may take the form of, for example, a 'tag filter' by which the gateway system can filter out tags from the data stream or from its associated metadata. The advantage of using a tag filter is that when the gateway system can handle different data streams, their tags are located at different positions within the packets; the tag filter allows the gateway system to find the tags in any such data stream.

[0096] In an embodiment, the (remote) server may be, for example, a telecommunications network and / or an edge node or edge node system within a telecommunications network.

[0097] Those skilled in the art will understand that two or more of the embodiments, implementations and / or aspects of the invention mentioned above can be combined in any manner that they deem useful.

[0098] Those skilled in the art can make modifications and variations to any entity described herein, such as any system, server, device, method, or computer program, based on this specification, and these modifications and variations correspond to the modifications and variations described in another of these entities. Attached Figure Description

[0099] These and other aspects of the invention will be understood and illustrated with reference to the embodiments described below. In the accompanying drawings:

[0100] Figure 1A The diagram illustrates how a user equipment streams data to a first edge node via a gateway system, and how an orchestration system instructs the gateway system to migrate remote processing to a second edge node.

[0101] Figure 1B This illustrates how a gateway system responds to instructions from an orchestration system by sending data packets of a data stream to a second edge node.

[0102] Figure 1C The diagram illustrates how a gateway system completes migration to a second edge node by ceasing to send data packets of the data stream to the first edge node.

[0103] Figure 2A A data stream comprising a sequence of data packets is shown, which is illustrated as being sent from the gateway system to the first edge node;

[0104] Figure 2B The data flow is shown to consist of data packet blocks, and the gateway system redirects the data flow to a second edge node starting from the block boundary between the first and second blocks;

[0105] Figure 2C Showing Figure 2B An alternative scheme is shown, in which the gateway system sends a given data packet block to the second edge node while continuing to repeatedly send the given data packet block to the first edge node, and stops sending data packets to the first edge node only from the beginning of the subsequent data packet block;

[0106] Figure 2D Showing Figure 2B and Figure 2C An alternative approach is shown, demonstrating that the gateway system has begun partially forwarding a given packet block in response to an instruction to migrate remote processing to a second edge node;

[0107] Figures 3A to 3D The physical movement of the client device and its switch from the first base station to the second base station are shown, along with the traffic flow associated with the data flow from the client device to the corresponding edge node.

[0108] Figure 4 An example of a processor system is shown, which may be a gateway system, an orchestration system, or a remote server.

[0109] Figure 5 An example of a processor system, which may be a client device, is shown, wherein the processor system includes a camera interface to a camera;

[0110] Figure 6 A computer-readable medium including non-transient data is shown;

[0111] Figure 7 An exemplary data processing system is shown.

[0112] It should be noted that items with the same reference numerals in different figures have the same structural features and the same function, or the same signal. Since the function and / or structure of such items have already been explained, it is unnecessary to repeat the explanation in the specific implementation.

[0113] List of reference numerals

[0114] The following list of reference numerals and abbreviations is provided to facilitate the interpretation of the drawings and should not be construed as limiting the claims.

[0115] BS(X) base station (X)

[0116] EN(X) edge node (X)

[0117] GW(X) Gateway System (X)

[0118] OC orchestration system

[0119] UE User Equipment

[0120] 100 data streams to the first edge node

[0121] The most recently instantiated data stream from node 110 flows to the second edge node.

[0122] 120 data streams to the second edge node

[0123] 150 Command Message

[0124] 160 Response Message

[0125] 200 data stream packets

[0126] 210 data packet block

[0127] 212 Incomplete data packet blocks

[0128] 230 Block boundaries indicated by markers

[0129] 240 Next data packet

[0130] 300 data streams to the first edge node

[0131] 310 The most recently instantiated data stream flows to the second edge node.

[0132] 320 Data stream to the second edge node

[0133] 350 client devices mobile

[0134] 400 indicates the processor system of the gateway system, orchestration system, or remote server.

[0135] 410 Network Interface

[0136] 420 Processor Subsystem

[0137] 430 Data storage device

[0138] 500 indicates the processor system of the client device.

[0139] 510 Network Interface

[0140] 520 Processor Subsystem

[0141] 530 Data storage device

[0142] 540 camera interface

[0143] 550 camera

[0144] 600 Computer-readable media

[0145] 610 Non-transient data

[0146] 1000 Exemplary Data Processing Systems

[0147] 1002 processor

[0148] 1004 Memory Element

[0149] 1006 System Bus

[0150] 1008 Local Memory

[0151] 1010 High-capacity storage device

[0152] 1012 Input Devices

[0153] 1014 Output Device

[0154] 1016 Network Adapter

[0155] 1018 Application Detailed Implementation

[0156] The following embodiments relate to a client device streaming data to a remote server for remote processing, which may be done via a gateway system. As an example, it is assumed that the remote server is an edge node or edge node system that can be configured to provide application services (also referred to as 'edge application services' or simply 'edge services') to client devices, and that the remote processing of the data stream is performed by the application service. However, the concepts and mechanisms described in this specification are equally applicable to any other type of remote server and any other type of remote processing, such as a cloud server providing cloud processing services to clients, and other examples of remote servers are given in this specification.

[0157] Furthermore, as an example, the client device in the following examples is a so-called User Equipment (UE) of a mobile network conforming to one or more 3GPP standards. This device may also be referred to hereinafter as a 'mobile device' due to its connectivity to the mobile network. It should be noted that although the UE is described as mobile and is actually shown as mobile in the following examples, this is not a limitation, as the described measures also apply to stationary or non-mobile devices that have connectivity to the mobile network (and therefore are 'mobile devices'). It should be noted that the concepts and mechanisms described in this specification also apply to any other type of client device, including mobile devices used with other types of mobile network infrastructure, such as non-3GPP types (e.g., Wi-Fi-based or satellite-based mobile network infrastructure), and client devices used with non-mobile, fixed-line network infrastructure.

[0158] Figure 1A A client device in the form of a user equipment (UE) is shown, which streams data stream 100 to a first edge node EN1 via a gateway system GW. This streaming can be performed via a network (not explicitly shown), such as a mobile network with its mobile network infrastructure. The first edge node EN1 can process data stream 100 to generate output. For example, the output may include a processed version of data stream 100, such as an enhanced or improved version, or any other type of output result, such as analysis results generated by analyzing data stream 100. The streaming of data stream 100 can be performed via the gateway system GW, since the gateway system GW can receive data packets of the data stream from the client equipment UE and subsequently send data packets to the first edge node EN1. This sending may include forwarding data packets in an unmodified or substantially unmodified manner, but may also include modifying data packets, for example, by changing the packet header, or by repackaging the data content of the received data packets into a new data packet to be sent to the first edge node EN1. It should be understood that, based on the fact that the data streams contain substantially the same data content, both the data stream received by the gateway system GW and the data stream sent by the gateway system GW can be referred to as 'the same' data stream. It should also be noted that the functionality of the entities described in this paragraph can be considered as known.

[0159] A gateway system (GW) can be configured to assist in remotely processing a migration from a first edge node EN1 to a second edge node EN2. This migration can be orchestrated by an orchestration system (OC). The orchestration system (OC) can be configured to determine when to initiate this migration, for example, based on various known considerations and criteria, and instruct the gateway system (GW) to assist in the migration via instruction message 150, i.e., by redirecting the data stream of the client device. Here, the term 'implementation' can refer to the gateway system (GW) taking action to implement the migration. However, it should be understood that other entities, such as the orchestration system (OC), can also typically play a significant role in the overall migration. The instruction message 150 sent to the gateway system (GW) can instruct the gateway system (GW) to begin sending a data stream to the second edge node EN2 and to stop sending a data stream to the first edge node EN1 at a given time. Therefore, as... Figure 1B As shown, the gateway system GW can begin sending data packets to the second edge node EN2, thereby initiating the streaming of data stream 110 to the second edge node EN2, while in this example, data stream 100 continues to be streamed to the first edge node EN1. During this period, the second edge node EN2 can also be referred to as the 'loading' edge node, while the first edge node EN1 can continue to serve the client device UE and is therefore referred to as the 'serving' edge node. Figure 1C The diagram illustrates how the gateway system (GW) completes its portion of the migration to the second edge node (EN2) by ceasing to send data packets to the first edge node (EN1). The gateway system (GW) can then acknowledge the completion of the migration to the orchestration system (OC) by sending a response message 160 to the orchestration system (OC).

[0160] exist Figures 1A to 1C In examples of other diagrams, the data stream may contain packet blocks that can be processed independently by the corresponding edge nodes EN1, EN2, because such blocks can be decoded at least independently, and in some examples, can be processed according to a first or a first set(s) of processing steps. The client device UE can be configured to generate tags identifying such corresponding packet blocks, for example, by assigning the same block identifier to the corresponding packets of the packet blocks. Such tags can be made available to the gateway system GW. For example, tags can be included in the data stream sent by the client device UE to the gateway system GW. The gateway system GW can then determine, based on the tags associated with the data stream, when to begin sending data stream packets to the second edge node EN2, and / or when to stop sending data stream packets to the first edge node EN1. It should be noted that the tags generated by the client device may also be referred to 'migration tags' below, as they can be generated by the client device UE for the specific purpose of migrating the remote processing of the data stream.

[0161] Continue to refer to Figures 1A to 1CSpecific examples may involve the following. Here and elsewhere, a remote server may also be referred to as 'edge' rather than 'edge node' to include embodiments where the remote server is a single edge node and embodiments where the remote server is embodied by an edge node system.

[0162] 1. Applications on the client device (UE) can be served by processing functions running on the first edge EN1. The client device (UE) can connect to the first edge EN1 via the gateway system (GW).

[0163] 2. When it is desired to migrate to another edge, the orchestration system OC can instantiate the processing functions on the second edge EN2 and can wait for the processing functions to be installed and / or booted on the second edge EN2.

[0164] 3. The orchestration system OC can, for example, instruct the gateway system GW to assist in the migration from the first edge EN1 to the second edge EN2 via the aforementioned instruction message 150.

[0165] 4. Upon receiving instruction message 150, the gateway system (GW) can begin monitoring the data stream received from the client device (UE) to locate migration markers, and can take one of the following actions:

[0166] a. Direct migration: When receiving a data packet belonging to the next data block, the gateway system GW can stop forwarding the data stream to the first edge EN1 and start forwarding the data stream to the second edge EN2;

[0167] b. Load migration: When receiving a data packet belonging to the next data block, the gateway system GW can start copying the data stream to the second edge EN2, and stop forwarding the data stream to the first edge EN1 after receiving the next block or usually several further data blocks;

[0168] c. "Dirty" Migration: The Gateway System (GW) can directly begin copying the data stream to the second edge EN2 and thereby forwarding packet blocks, where (multiple) first packets may have already been sent to the first edge EN1 and therefore may no longer be available to the second edge EN2. In fact, the Gateway System (GW) can begin forwarding currently received packets regardless of their position within a block (e.g., beginning, middle, end). After receiving one or more data blocks, or after receiving packets belonging to the next data block, the Gateway System (GW) can stop forwarding the data stream to the first edge EN1.

[0169] Action type can also be referred to as 'migration type', and its selection may depend on the application associated with the data flow, as certain types of applications may require or prefer specific migration types.

[0170] 5. The gateway system (GW) can, for example, use the aforementioned response message 160 to notify the orchestration system (OC) that the migration has been completed.

[0171] 6. The second edge EN2 can continue to serve as a service edge, that is, to serve the edge provided by the client device UE by remotely processing its data stream.

[0172] 7. The orchestration system OC can terminate and clear instances of processing functions on the first edge EN1.

[0173] Figures 2A to 2D The aforementioned migration types are illustrated. Typically, depending on the migration type, data blocks may be copied without copying any, some, or all, and routed to both edges. The migration type can be chosen based on application requirements. Applications that do not require any initialization after receiving data blocks can use direct migration, which may be the most efficient in terms of network resource allocation. Other applications may benefit from temporary copying of traffic.

[0174] Figure 2A A data stream including packet sequence 200 is shown, which is illustrated as being sent from gateway system GW to first edge EN1.

[0175] Figure 2B The diagram illustrates two consecutive blocks 210 and 240 of data packets in a data stream, and how the gateway system redirects the data stream to a second edge EN2 starting from block boundary 230 between the first block 210 and the second block 240. This type of migration can correspond to the aforementioned 'direct' migration type.

[0176] Figure 2C Showing Figure 2B An alternative scheme is shown, in which the gateway system GW sends the first data packet block 210 to the second edge EN2 while continuing to repeatedly send the first block 210 to the first edge EN1, and stops sending data packets to the first edge EN1 only from the second (or later) block 240.

[0177] Figure 2D Showing Figure 2B and Figure 2C An alternative approach is shown whereby the gateway system GW, in response to a command to migrate remote processing, immediately begins forwarding packets of the first block 210 to the second edge EN2 by rapidly forwarding the current packet of the first block 210. Here, the term 'immediately' can mean without waiting for forwarding at the block boundary. Since this current packet may not be the first packet of the first block 210, this could result in an incomplete version 212 of the first block 210 being transmitted to the second edge EN2. This type of migration can correspond to the aforementioned 'dirty' migration type.

[0178] It should be understood that although 'dirty' migration may seem inefficient, because incomplete data blocks may be emitted to the second edge EN2, the second edge EN2 may have already used this data during initial processing. For example, in the case of video streaming, the second edge EN2 can use partial data blocks to infer the video / audio codec and initialize the appropriate decoder pipeline. Once a new data block that is fully decodeable and processable is received, the decoder pipeline can be set up, thereby reducing the startup before output is produced.

[0179] The following describes various embodiments of the gateway system (including distributed and non-distributed embodiments) and various embodiments of migration tags. Here, the remote server continues to be referred to as an edge node or edge.

[0180] Gateway system

[0181] Depending on the type of connectivity between the client device and the edge, gateway systems can be implemented in various forms.

[0182] For example, the transport protocol between the client device and the edge can be stateful (e.g., when using TCP) or stateless (e.g., when using UDP), and to some extent, this is also true when using short-lived TCP sessions. In some examples, the transport protocol can be session-based, and the gateway system can act as a proxy server or forwarding unit. In this case, the gateway system can disconnect the client device from the edge, allowing the client device to connect to the gateway system, the gateway system to connect to the edge, and the gateway system to forward traffic from one connection to another. When the gateway system receives instructions from the orchestration system to perform a migration, the gateway system can establish a second connection with the second edge, thus now forwarding incoming data to both edge connections. In other examples, the transport protocol can be stateless. In this case, the gateway system can also act as a proxy server, recreating packets with the same data and sending them to the destination edges(s).

[0183] In other examples, gateway systems can be implemented by network-specific elements (i.e., network components), and forwarding can be based on routing and handover principles. Network-level replication may mean replicating actual network packets, potentially rewriting the network-level address in the header. In fixed networks, the network component that can be used to form a gateway system could be a switch supporting Software-Defined Networking (SDN). In this case, the orchestration system can instruct the network controller to perform migrations. In 5G networks and Service-Based Architecture (SBA) scenarios, the orchestration system, embodied as an Application Function (AF), can request new rules regarding User Plane Functions (UPFs) via Policy Control Functions (PCFs) and Session Management Functions (SMFs). In this example, functions representing the gateway system's packet monitoring and migration capabilities can be added.

[0184] From the perspective of the client device, edge addressing can be transparent. This means that the client device may not need to select different edges by using different network addresses, but can address the edge via a single network address and network, while the gateway system can handle routing to the appropriate edge. This transparent addressing can be established in various ways.

[0185] For example, a physical edge instance (or a virtual edge instance on top of real hardware) can have different network addresses, such as IP addresses. These network addresses can be masked, and in some examples even reside in a private IP address space, and are known to the orchestration and gateway systems, but not to the client devices. The gateway system's network address can be communicated to the client devices. The client devices' packets can then reach the gateway system, which can forward the packets to the edge(s), and thus to the new network address. Depending on the type of gateway system, the gateway system can create new packets with the edge's destination network address, or the gateway system can perform Network Address Translation (NAT) to change the destination network address in the packets before forwarding them.

[0186] Another example is that edge instances and gateway systems can share a single network address, which can be a network address that can be communicated to client devices. The network address can be, for example, a reused IPv4 address or an IPv6 selective broadcast address. To prevent packet misdirection, network organization and routing can be based on regions (sometimes called segments), where the gateway system is the point from which packets from one region can enter other regions. In a configuration with two edges, at least three regions can be identified: between the client device and the gateway system, between the gateway system and the first edge instance, and between the gateway system and the second edge instance. Network regions can be separated from each other, meaning that a route to a network address may only be available to one destination, depending on the region, i.e., either the gateway system or the edge. The gateway system can have multiple (virtual) network interfaces to different network regions.

[0187] Centralized gateway system and distributed gateway system

[0188] A gateway system can be a single instance in a network located between a client device and two (source and destination) edges. This may be a relevant implementation when client mobility is minimal or nonexistent but migration is still required. For example, a single gateway node can be used to optimize resources, scale computing resources up or down. In some examples, a gateway system can facilitate migration between edges. In other examples, a gateway system can facilitate migration between compute nodes in an edge cluster. In still other examples, a gateway system can facilitate migration from a cloud node to an edge node, or from an edge node to a cloud node. In yet another example, a gateway system can facilitate migration between cloud nodes or any other remote server.

[0189] Continue to refer to Figures 1A to 1C This example uses a gateway system (GW) to illustrate migration between two edges, EN1 and EN2. Initially, traffic can be routed to the first edge EN1 (e.g., the current serving edge). When migration is desired, and in accordance with instructions from the orchestrator system (OC), the gateway system GW can replicate the traffic to the second edge EN2. Once the second edge EN2 is ready and the gateway system GW has inferred the state of the second edge EN2 based on migration markers in the data stream, traffic routing to the first edge EN1 can cease, and the second edge EN2 can continue to function as a serving node. Here, the phrase 'infer state based on migration markers' can refer to the gateway system GW's ability to determine, based on markers in the data stream, when multiple complete data blocks have been sent to the second edge EN1 and when the second edge processing has determined whether the state of the second edge EN2 is (sufficiently) matched that of the first edge EN1, and therefore when the second edge EN2 is ready to take over remote processing tasks from the first edge EN1.

[0190] Figures 3A to 3D This demonstrates that gateway systems can be instantiated multiple times. For example, a first gateway system GW1 can be located between a first base station BS1 and a first edge EN1, while a second gateway system GW2 can be located between a second base station BS2 and a second edge EN2. Gateway systems can together form a gateway supersystem. Two or more gateway systems can exist (directly connected or indirectly connected, for example via a third (central) gateway system). Figure 3A As shown, the client device UE can initially connect to the first base station BS1 and send its data stream 300 to the first edge EN1 via the first gateway system GW1. Then, the client device UE can move, as... Figures 3A to 3C As indicated by arrow 350. Figure 3B As shown, when a handover to the second base station BS2 is anticipated, the first gateway system GW1 can copy the data stream and route the copied data stream 310 to the second gateway system GW2. This routing to the second gateway system GW2 can also be performed if a handover to the second base station BS2 has already occurred but the first gateway system GW1 is still attached to the client device UE, for example, when a UPF reselection has not yet occurred. The second gateway system can then forward the data stream 310 to the second edge EN2. Figure 3C As shown, when a handover from the client device to the second base station BS2 has occurred and the second gateway system GW2 takes over the role of the first gateway system GW1, the second gateway system GW2 can forward traffic to the first gateway system GW1, while the first edge EN1 can continue to act as a serving edge node. Figure 3D As shown, once the second edge EN2 is loaded, the second gateway system GW2 can stop forwarding data flow 300 from the block boundary to the first gateway system GW1 based on the migration marker in the data flow, while the second edge EN2 can become a serving edge node. In the example of a 5G network where the gateway system is embodied by a UPF (User Plane Function), the above functionality can be obtained by modifying the SSC mode 2 or 3PDU session anchor relocation procedure to incorporate the detection and use of migration markers.

[0191] When using a single gateway system, for example, Figures 1A to 1C As shown, the use of session-based and stateless transport protocols might seem readily available because the connection between the client device and the gateway system is unlikely to change. However, it may involve more complex scenarios using multiple gateway systems (e.g., such as...). Figures 3A to 3C(As shown). For example, when using stateless protocols and when forwarding network packets, the handover from a first gateway system to a second gateway system can be transparent. However, when using session-based transport protocols, the session may be corrupted after the handover. The serving gateway system may have to notify the client device that it may need to re-establish the connection (e.g., using TCP RST), or must use techniques such as those described in A SDN Based Method of TCP Connection Handover (ICT EurAsia, Vol. 9357, 2015) to facilitate the TCP handover. However, it should be noted that re-establishing the connection or performing a TCP handover is relatively quick and may occur even without edge migration of gateway systems in its path.

[0192] Regarding addressing, in the case of several gateway systems, additional rules regarding routing and network area fencing can be defined. When different network addresses are used for the edges, routing from gateway systems may need to be limited to only one edge. For example, a second gateway system GW2 can directly address edge EN2, but the network can be configured not to route traffic directly to edge EN1. Instead, the network can be configured such that traffic to the first edge EN1 is routed through the first gateway system GW1. Similarly, when using a single network address and fencing between different areas within the network, the second gateway system GW2 can use a network address on one of its network interfaces (areas) to address the second edge EN2, while addressing the first gateway system GW1 on another network interface (area). By ensuring that all involved gateway systems process the same packets, gateway systems can be configured to remain in the same state and can perform their parts of the migration independently of each other.

[0193] Data stream tags

[0194] Client devices can tag packets in a data stream to allow packets to be assigned to appropriate blocks. In some examples, client applications running on the client device can tag or control the tagging of packets because the application can know the relationships between packets within a data block, and possibly the relationships between data blocks themselves. Depending on the type of tag, the client application can include the tag in the packet or provide tagging information to the operating system (OS) so that the OS can perform tagging on behalf of the client application.

[0195] Message-based tagging

[0196] In some examples, client applications can format application data for remote processing, allowing messages (e.g., self-contained units of data) to be identified, and data blocks can consist of several messages. Each message can contain a tag for the associated data block. The gateway system can act as an application-level proxy server, forwarding messages to edge nodes. The tag can be part of the application data structure or integrated into the transport protocol.

[0197] Example 1: Below is an example of tags being included in a data structure formatted using a common JSON format.

[0198]

[0199] Using this type of tag, messages can be forwarded as is to (multiple) destination edges. Messages with tags included in data structures can be included in UDP packets, or as part of a TCP stream, or can be transmitted using network protocols such as WebSockets. Gateway systems can be application-specific.

[0200] Example 2: Below is an example of a tag being included in a transport protocol; this tag is a header field in an HTTP POST request.

[0201] POST / message HTTP / 1.1

[0202] Host: example.org

[0203] Content-Type:application / j son

[0204] Content-Length: 148

[0205] X-Marker:aGVsbG8taS1hbS1hLW1hcmtlcg==

[0206] <application data>

[0207] While the examples above use text-based protocols, message-based tagging can also be applied to binary messages. It should be noted that with message-based tagging, the gateway system may need to be trusted by the client application, as it may act as a man-in-the-middle. Application data, including data block tags, can be encrypted between the client application and the gateway system. However, to perform the migration, the gateway system may need access to the decrypted tags, and thus, access to other application data.

[0208] Packet encapsulation-based tagging

[0209] In some examples, application data to be processed remotely can be encapsulated in tagged packets before being transmitted, effectively adding an extra layer to the TCP / IP or UDP / IP stack. This can be performed by a non-application-specific gateway system ('general' because it can serve multiple applications) that does not need to access the application data. For example, the gateway system can be implemented as a proxy server or using software-defined networking (SDN) technology.

[0210] In some examples, client applications can perform encapsulation before providing packets to the OS. Client applications can be configured to prevent packet fragmentation (e.g., by not exceeding the maximum packet size) and avoid packets being combined into streams (e.g., by disabling the Nagle algorithm when using TCP). In other examples, the OS can provide a tag-as-a-service to client applications, such as one accessible via an API. In addition to providing low-level primitives that allow opening TCP / UDP sockets and reading from / writing to them, the OS can also provide tagged versions of these TCP / UDP sockets and associated functionality. For example, the OS can provide write primitives where client applications can provide a file description, data, and the length of the data to a tagged socket, along with tags for the packets used to deliver the provided data. The OS can ensure that all data is encapsulated in tagged packets, enabling network-level implementations of gateway systems.

[0211] The OS of the gateway system and edge nodes can provide similar functionality for reading from tagged versions of sockets. When reading from a tagged socket, the OS can provide the application with the data and its associated tags. The OS can ensure that read calls can provide only data belonging to a single data block, thus allowing the implementation of a proxy-server-based gateway system.

[0212] Tunnel-based marking

[0213] Tunnel-based tagging can be viewed as a special form of encapsulation-based tagging. In some examples, a client device can establish a tunnel with a gateway system, and the tag can be part of the encapsulated data packet that forms the tunnel. In some examples, the tunnel can be a VPN (Virtual Private Network) connection between the client device and the gateway system (or can be part of it). In other examples, the tunnel can be part of an LTE / 5G network connection, such as a GTP (GPRS Tunneling Protocol) for PDU (Packet Data Unit) sessions, and the gateway system is represented as a UPF.

[0214] When using tunneling, the OS can provide a virtual network interface to the application, and packets written to that network interface will be tunneled to that virtual network interface. In some examples, the tag may resemble an encapsulation-based tag, where the OS may provide a write primitive to the API that includes tag parameters. In another embodiment, the OS can provide the ability to configure the network interface, allowing the application to set a tag for all outgoing packets until the tag is cleared or a new tag is set. At the gateway system, the tunnel can be terminated, and the gateway system can unpack the packets before forwarding them to one or more edge servers.

[0215] Migration instructions

[0216] The orchestration system can instruct the gateway system to perform a migration. Depending on the gateway system implementation, this instruction can take different forms, including but not limited to SDN rules, PCF rules in 5G networks, API calls, etc. In some examples, the instruction can be represented by an appropriate configuration of the gateway system. Independent of the implementation of the instruction, the instruction may contain fields such as the following:

[0217] • Destination Edge: The address of the destination edge during migration. For example, the address can be a URL, hostname, IP address, network zone identifier, or network interface identifier. The URL, hostname, or IP address may include or may include a (UDP / TCP) port number.

[0218] • Migration type: This can indicate the migration type, such as direct migration, load migration, or dirty migration mentioned above.

[0219] • Migration Duration: In the case of a load migration or a dirty migration, the gateway system can send at least one data block to both edges, but can replicate more data blocks if needed. The duration can be the number of data blocks that can be sent to both edges during the migration before the destination edge can take over and the gateway system can stop forwarding packets to the first edge.

[0220] • Tag Filter: Instructions or formats from the gateway system on how to find tags. Filters can, for example, describe the data type, byte offset, and length of a data packet. Alternatively, filters can be keys / paths to JSON-formatted documents, or regular expressions for other text-based messages.

[0221] Figure 4A processor system 400 is shown, which may be an example of a gateway system, orchestration system, or remote server described in this specification, meaning that the processor system 400 may embody any of these entities. The processor system 400 may include a network interface 410 for data communication via a network (not shown itself). The network interface 410 may be, for example, a wired communication interface, such as an Ethernet interface or a fiber-optic-based interface. The network may be, for example, the Internet or a mobile network, wherein the processor system 400 is part of and connected to a fixed portion of the mobile network. Alternatively, the network interface 410 may be, for example, the following for… Figure 5 The client device 500 describes a wireless communication interface of the type described. In other examples, the processor system 400 may be a subsystem of a larger system (e.g., a supersystem). In this case, the network interface 410 may be an internal interface of the supersystem, such as a virtual, software-based network interface.

[0222] Processor system 400 may further include processor subsystem 420, which may be configured, for example by hardware design or software, to perform the operations described herein concerning an entity embodied by the processor system (e.g., a gateway system, orchestration system, or remote server). Typically, processor subsystem 420 may be embodied by a single central processing unit (CPU), such as an x86 or ARM-based CPU, but may also be embodied by a combination or system of such CPUs and / or other types of processing units. In embodiments where processor system 400 is distributed across different entities (e.g., across different servers), processor subsystem 420 may also be distributed across, for example, the CPUs of such different servers. Similarly, as... Figure 4 As shown, the processor system 400 may include a data storage device 430, such as a hard disk drive, a solid-state drive, or an array of such hard disk drives and / or solid-state drives, which can be used to store data.

[0223] In some examples, processor system 400 may be implemented by a network node or a system of a network node. In some examples, processor system 400 may be configured to implement a gateway system as a User Plane Function (UPF). In some examples, processor system 400 may be configured to implement an orchestration system as an open-source NFV Management and Orchestration (MANO) system. In some examples, processor system 400 as an orchestration system may be part of a client device or distributed across different client devices.

[0224] Figure 5A processor system 500 is illustrated, which may be an example of the client device described in this specification, and may also be simply referred to as client device 500 below. Client device 500 may include a network interface 510 to a network to enable communication with a gateway system via data communication. Network interface 510 may be, for example, a wireless communication interface, which may also be referred to as a radio interface and may be configured to connect to mobile network infrastructure. In some examples, network interface 510 may include a radio and an antenna, or include a radio and antenna connection. In specific examples, network interface 510 may be a 4G or 5G radio interface for connecting to a 4G or 5G mobile network conforming to one or more 3GPP standards, or it may be a Wi-Fi communication interface for connecting to Wi-Fi network infrastructure, etc. In other examples, network interface 510 may be, for example, for... Figure 4 The processor system 400 describes a wired communication interface of the type described. It should be noted that data communication between the client device 500 and the gateway system and / or remote server may involve multiple networks. For example, the client device 500 may be connected to the infrastructure of a mobile network via a radio access network and connected to the Internet via the infrastructure of the mobile network, wherein the remote server is a server also connected to the Internet. In other examples, the gateway and / or remote server may be part of or connected to the same (mobile) network to which the client device 500 may connect.

[0225] Client device 500 may further include a camera interface 540 through which processor system 500 can acquire camera data, for example, in near real-time, and then stream it to a remote server for processing, in near real-time. In other examples, client device 500 may include a camera as an internal component. In still other examples, in addition to or instead of the camera interface, client device 500 may include a sensor data interface to external or internal sensors and may be configured to stream the acquired sensor data to a remote server for processing. In still other examples, client device 500 may retrieve the data to be streamed from elsewhere (e.g., from internal storage, another network connection entity, etc.).

[0226] Client device 500 may further include a processor subsystem 520, which may be configured, for example by hardware design or software, to perform the operations described herein concerning the client device or UE, or generally concerning client-side streaming of data streams and the generation of tags for the data stream. Typically, processor subsystem 520 may be embodied by a single central processing unit (CPU), such as an x86 or ARM-based CPU, but may also be embodied by a combination or system of such a CPU and / or other types of processing units, such as a graphics processing unit (GPU). Client device 500 is further shown as including a data storage device 530, which may have... Figure 4 The processor system 400 is of the same or similar type as described, and the data storage device can be used by the client device 500 to store data, such as storing and / or buffering data streams to be sent to a remote server, and / or generating tags for the data streams.

[0227] although Figure 5 While not explicitly shown, client device 500 may further include a display output for outputting display data to a display. This enables the client device to display data, such as acquired camera data or data streams received (returned) from a remote server. Such received data streams may, for example, be a processed version of a data stream sent to a remote server. In a particular example, client device 500 may acquire a camera data stream and send it to a remote server, which may render the camera data stream and other camera data streams in a virtual environment and send a data stream representing the rendered virtual environment back to client device 500 for display thereon.

[0228] Typically, client device 500 can be represented by a (single) device or apparatus, such as a smartphone, personal computer, laptop computer, tablet device, game console, set-top box, television, monitor, projector, smartwatch, smart glasses, media player, media recorder, etc. In some examples, client device 500 can be a so-called user equipment (UE) of a mobile telecommunications network, such as a 5G or next-generation mobile network. In some examples, client device 500 can be represented by a distributed system of devices, or it can represent a virtual, software-based client.

[0229] Typically, each entity described in this specification may be embodied as a device or apparatus, or embodied in a device or apparatus. The device or apparatus may include one or more (micro)processors executing appropriate software. The processor(s) of a corresponding entity may be embodied by one or more of these (micro)processors. The software implementing the functionality of the corresponding entity may have been downloaded and / or stored in the corresponding one or more memories, for example, in volatile memory such as RAM or non-volatile memory such as flash memory. Alternatively, the processor(s) of the corresponding entity may be implemented in the device or apparatus as programmable logic, for example, as a field-programmable gate array (FPGA). Any input and / or output interfaces may be implemented by the corresponding interfaces of the device or apparatus. Typically, each functional unit of the corresponding entity may be implemented as a circuit or circuit system. The corresponding entities may also be implemented in a distributed manner, for example, involving different devices or apparatuses.

[0230] Note that any method described in this specification, such as any method described in any claim, can be implemented on a computer as a computer-implemented method, dedicated hardware, or a combination of both. Instructions for a computer (e.g., executable code) can be stored, for example... Figure 6 The computer-readable medium 600 shown may contain, for example, a series of machine-readable physical marks 610 and / or a series of elements having different electrical (e.g., magnetic) or optical properties or values. Executable code may be stored in a transient or non-transient manner. Examples of computer-readable media include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Figure 6 An optical storage device 600 is illustrated by way of example.

[0231] In alternative embodiments of the computer-readable medium 600, the computer-readable medium 600 may include transient or non-transient data 610 in the form of a data structure representing the markers described in this specification.

[0232] Figure 7This is a block diagram illustrating an exemplary data processing system 1000 that can be used in the embodiments described herein. Such a data processing system includes the data processing entities described herein, including but not limited to gateway systems, orchestration systems, remote servers, and client devices. The data processing system 1000 may include at least one processor 1002 coupled to a memory element 1004 via a system bus 1006. Thus, the data processing system can store program code within the memory element 1004. Furthermore, the processor 1002 can execute program code accessed from the memory element 1004 via the system bus 1006. In one aspect, the data processing system may be implemented as a computer suitable for storing and / or executing program code. However, it is understood that the data processing system 1000 may be implemented in the form of any system including a processor and memory capable of performing the functions described herein.

[0233] Memory element 1004 may include one or more physical memory devices, such as local memory 1008 and one or more mass storage devices 1010. Local memory may refer to random access memory or other non-persistent memory(s) typically used during the actual execution of the program code. Mass storage devices may be implemented as hard disk drives, solid-state drives, or other persistent data storage devices. Data processing system 1000 may also include one or more cache memories (not shown) that provide temporary storage for at least some program code to reduce the number of times program code is otherwise retrieved from mass storage device 1010 during execution.

[0234] Input / output (I / O) devices, depicted as input device 1012 and output device 1014, may optionally be coupled to the data processing system. Examples of input devices include, but are not limited to, microphones, keyboards, pointing devices such as mice, game controllers, Bluetooth controllers, VR controllers, and gesture-based input devices. Examples of output devices include, but are not limited to, monitors or displays, speakers, etc. Input and / or output devices may be coupled to the data processing system directly or via an intermediate I / O controller. Network adapter 1016 may also be coupled to the data processing system to enable it to be coupled to other systems, computer systems, remote network devices, and / or remote storage devices via an intermediate private or public network. The network adapter may include a data receiver for receiving data transmitted to the data processing system by the system, device, and / or network, and a data transmitter for transmitting data to the system, device, and / or network. Modems, cable modems, and Ethernet cards are examples of different types of network adapters that can be used with the data processing system 1000.

[0235] like Figure 7As shown, memory element 1004 can store application 1018. It should be understood that data processing system 1000 can further execute an operating system (not shown) capable of facilitating the execution of the application. The application, implemented in the form of executable program code, can be executed by data processing system 1000 (e.g., by processor 1002). In response to executing the application, the data processing system can be configured to perform one or more operations, which will be described in further detail herein.

[0236] For example, data processing system 1000 may represent a gateway system as described in this specification. In this case, application 1018 may represent an application that, when executed, configures data processing system 1000 to perform the functions described with reference to the gateway system. In another example, data processing system 1000 may represent an orchestration system as described in this specification. In this case, application 1018 may represent an application that, when executed, configures data processing system 1000 to perform the functions described with reference to the orchestration system. In yet another example, data processing system 1000 may represent a remote server as described in this specification. In this case, application 1018 may represent an application that, when executed, configures data processing system 1000 to perform the functions described with reference to the remote server. Data processing system 1000 may also represent a (client) device as described in this specification. In this case, application 1018 may represent an application that, when executed, configures data processing system 1000 to perform the functions described with reference to the client device.

[0237] It should be noted that the embodiments mentioned above are illustrative and not limiting of the invention, and those skilled in the art will be able to devise many alternative embodiments without departing from the scope of the appended claims.

[0238] In the claims, any reference numerals placed between parentheses should not be construed as limiting the claims. The use of the verb "comprising" and its variations does not exclude the presence of elements or levels other than those stated in the claims. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Expressions such as "at least one" preceding a list or group of elements indicate a selection of all elements or any subset of elements from the list or group. For example, the expression "at least one of A, B, and C" should be understood to include only A, only B, only C, both A and B, both A and C, both B and C, or all of A, B, and C. The invention can be embodied by hardware comprising several different elements and by a suitably programmed computer. In an apparatus claim enumerating several means, several of these means can be embodied by the same hardware. The fact that certain measures are stated in mutually different dependent claims does not imply that combinations of these measures cannot be advantageously used.

Claims

1. A gateway system, wherein the gateway system serves as a gateway between a client device and a remote server, wherein, The client device is configured to stream data to the remote server via the gateway system for remote processing. The gateway system includes: - A network interface for receiving the data stream and tags that identify corresponding data packet blocks in the data stream, wherein each identified data packet block is independently processable; - A processor subsystem configured to use the network interface to perform the following operations: - In response to instructions or configurations for migrating the remote processing from one remote server to another, a data packet block is identified in the data stream, from which the other remote server begins processing the data stream to implement or initialize the migration; - When the identified data packet block is received, the identified data packet block and subsequent data packet blocks of the data stream are streamed to the other remote server.

2. The gateway system according to claim 1, wherein, The processor subsystem is configured to perform the following operations when the determined data packet blocks are streamed to the other remote server: - Stop streaming this data stream to the remote server; or - Temporarily continue to repeatedly stream the data to the remote server.

3. The gateway system according to claim 1 or 2, wherein, The processor subsystem is configured to initiate streaming of data packets to the other remote server in response to the instruction or before the determined data packet block, so that the other remote server can initialize its processing of the data stream.

4. The gateway system according to any one of claims 1 to 2, wherein: - This instruction is received as an instruction message from an orchestration system configured to orchestrate the migration of the remote processing; - The processor subsystem is configured to send a response message to the orchestration system indicating that the migration has been implemented or initialized.

5. The gateway system according to any one of claims 1 to 2, wherein, The processor subsystem is configured to send the data stream to the other remote server via at least one of the following: - Forward the data packets of this data stream; - Create new data packets using the network address of this other remote server; - Modify the network address field in these packets received from the client device to identify the other remote server, and then send the modified packets to the other remote server; - If these packets are encapsulated packets containing data to be processed remotely and include these tags in the header of these encapsulated packets, then these encapsulated packets are unpacked and the data to be processed remotely is sent as the data stream to the other remote server.

6. A mobile network, comprising: - A first base station, and a first gateway system and a first remote server for serving client devices connected to the first base station; - A second base station, and a second gateway system and a second remote server for serving client devices connected to the second base station; The first gateway system and the second gateway system are each examples of the gateway system according to any one of claims 1 to 4; - The first gateway system is configured as follows: - Receive data streams from client devices connected to the first base station and stream those data streams to the first remote server. - In response to an instruction or configuration for migrating the remote processing from the first remote server to the second remote server when the client device is expected to switch to the second base station, the data stream is streamed to the second remote server via the second gateway system; - The second gateway system is configured to take over the role of the gateway system for the client device from the first gateway system after the client device is switched to the second base station.

7. A client device configured to stream data to a remote server for remote processing, the client device comprising: - The network interface to the network; - A processor subsystem configured to use the network interface to perform the following operations: - Before or simultaneously with streaming the data stream, generate tags for the data stream that identify the corresponding data packet blocks in the data stream, wherein each identified data packet block is independently processable; - Stream the data stream along with these tags so that the gateway receiving the data stream can identify the packet blocks in the data stream, starting from which another remote server will begin processing the data stream to implement or initiate a migration from that remote server to that other remote server.

8. The client device according to claim 7, wherein, The processor subsystem is configured to include these tags in the data stream.

9. The client device according to claim 8, wherein, The processor subsystem is configured to include a tag in each packet of the data stream, wherein the tag includes a block identifier that identifies which packet block the corresponding packet belongs to.

10. The client device according to any one of claims 7 to 9, wherein, The processor subsystem is configured to generate these tags either as pseudo-random identifiers or by sequentially numbering these data packets.

11. The client device according to claim 7, wherein, The processor subsystem is configured to include these tags in at least one of the following: - The header of the transport protocol used to transmit this data stream; - The header of the encapsulated data packet that encapsulates the data to be processed remotely; - A tunnel established between the client device and the entity receiving the data stream, wherein these tags are included in the encapsulated data packets that form the tunnel.

12. An orchestration system for orchestrating the migration of a data stream from one remote server to another, wherein, The data stream is streamed from the client device to the gateway system and then streamed from the gateway system to the remote server prior to the migration. The orchestration system includes: - The network interface to the network; - A processor subsystem configured to use the network interface to perform the following operations: - Configure the other remote server to perform the remote processing; - Send an instruction message to the gateway system to implement or initiate the migration of remote processing from the remote server to the other remote server by streaming the data stream to the other remote server, wherein the instruction message instructs the gateway system to determine when to start streaming the data stream to the other remote server based on tags that mark the data stream, each tag identifying a corresponding data packet block in the data stream, and each identified data packet block being independently processable.

13. The orchestration system according to claim 12, wherein, The processor subsystem is configured to generate the instruction message indicating a data packet block in the data stream, from which the gateway system streams the data stream to the other remote server.

14. The orchestration system according to claim 12 or 13, wherein, The processor subsystem is configured to include at least one of the following in the instruction message: - The network address of the other remote server; - Migration type, which indicates whether to temporarily continue streaming the data stream to the other remote server after the streaming of the data stream has started; - Migration duration, which indicates the number of data packet blocks that will be temporarily continued to be streamed to the other remote server after the data stream has been started; - Information that informs the gateway system how to identify these tags in the data stream or in the metadata associated with the data stream.

15. A server for processing data streams streamed by client devices, the server comprising: - A network interface used to receive data packets from the data stream; - A processor subsystem configured to perform the following operations via the network interface: - In operating mode, the data stream is processed to generate output for use by the client device or another entity, wherein the data stream comprises data packet blocks, wherein each data packet block is independently processable, and wherein the processor subsystem is configured to process the data stream by at least initially processing these data packet blocks independently; and - Before invoking this operation mode, the output is not generated in response to instructions or configurations that indicate the initiation of processing of the data stream's packets.

16. The server according to claim 15, wherein, The server is an edge node or edge node system in a telecommunications network.

17. A computer-implemented remote processing method for communication between a client device and a remote server, the method being applied to a gateway system to provide gateway functionality, wherein, The client device is configured to stream data to the remote server via the gateway function for remote processing, and the method implemented by the computer includes: - Receive the data stream and tags that identify the corresponding data packet blocks in the data stream, wherein each identified data packet block is independently processable; - In response to instructions or configurations for migrating the remote processing from one remote server to another, a data packet block is identified in the data stream, from which the other remote server begins processing the data stream to implement or initialize the migration; - When the identified data packet block is received, the identified data packet block and subsequent data packet blocks of the data stream are streamed to the other remote server.

18. A computer-implemented remote processing method for streaming data to a remote server for remote processing, the computer-implemented method comprising: - Before or simultaneously with streaming the data stream, generate tags for the data stream that identify the corresponding data packet blocks in the data stream, wherein each identified data packet block is independently processable; - Stream the data stream along with these tags so that the gateway receiving the data stream can identify the packet blocks in the data stream, starting from which another remote server will begin processing the data stream to implement or initiate a migration from that remote server to that other remote server.

19. A computer-implemented method for orchestrating the migration of a data stream from one remote server to another, wherein, The data stream is streamed from the client device to the gateway system and then streamed from the gateway system to the remote server prior to the migration. The method implemented by the computer includes: - Configure the other remote server to perform the remote processing; - Send an instruction message to the gateway system to implement or initiate the migration of remote processing from the remote server to the other remote server by streaming the data stream to the other remote server, wherein the instruction message instructs the gateway system to determine when to start streaming the data stream to the other remote server based on tags that mark the data stream, each tag identifying a corresponding data packet block in the data stream, and each identified data packet block being independently processable.

20. A computer-implemented method for processing a data stream streamed by a client device, the computer-implemented method comprising: - Receive the data packets from this data stream; - Process the data stream to generate output for use by the client device or another entity, wherein the data stream comprises data packet blocks, wherein each data packet block is independently processable, and wherein the processing includes at least initially processing these data packet blocks independently; and - Before processing the data stream to generate the output, in response to an instruction or configuration indicating the initialization of the processing of the data stream, the data packets of the data stream are processed without generating the output.

21. A computer-readable medium comprising transient or non-transient data representing a computer program, the computer program comprising instructions for causing a processor system to perform the method according to any one of claims 17 to 20.

22. A computer program product comprising transient or non-transient data representing a computer program, the computer program comprising instructions for causing a processor system to perform the method according to any one of claims 17 to 20.

Images