Method for managing connection loss for distributed system

Abstract

The invention relates to a method for managing connection loss for a distributed system, comprising:-deploying a support module on at least two transmitters, on at least one receiver and on at least one orchestrator, the support module being designed to provide transmission of signals on additional communication channels, wherein the orchestrator is designed to manage periodic communications between the at least two transmitter runtime and the at least one receiver runtime,-analysing an additional communication channel between the at least two transmitter runtime and the orchestrator, in order to detect a loss of connection of at least one of the at least two transmitters during operation on the basis of the detection of the signal transmitted on the additional communication channel,-taking a countermeasure in the event of the detection of the loss of connection, comprising stopping, by means of a support module, periodic communication between at least one transmitter runtime and at least one receiver runtime for which the loss of connection is detected, in order to manage the loss of connection in the distributed system.

Description

Technical Field

[0001] The invention relates to a method for managing connection losses for a distributed system. The invention furthermore relates to a computer program, a device and a storage medium for this purpose. Background Art

[0002] In recent decades, the availability and processing power of cloud and edge resources have increased dramatically. This trend is likely driven in large part by the multiple advantages of monolithic applications on a (dynamically changing) collection of heterogeneous resource nodes - such as better resource utilization, fault tolerance, and application availability.

[0003] Cloud servers can provide huge computing power and almost unlimited elasticity, while transferring data between devices and the cloud may require relatively long and strongly varying transmission times, i.e., the transmission time to and from the cloud server may have large latency and significant jitter. Therefore, efforts for distributing applications may initially focus on timing-insensitive applications from fields such as enterprise, scientific computing, or machine learning.

[0004] Meanwhile, real-time and safety-critical CPS (cyber-physische Systeme) have been used in particular as monoliths, where the processing part of the application (responsible for control / decision-making) as well as the sensor and actuator parts are all deployed on the same device at the deployment site. However, with the increasing distribution of edge resources and high-bandwidth communications, it is now possible to achieve high-speed communications between the deployed devices and efficient edge nodes (especially with runtimes in the millisecond range), making it possible to offload the processing part of the CPS application to more efficient computing nodes - thereby effectively leveraging the distribution advantages briefly described above - while the deployed devices can only manage the sensor and actuator functionality.

[0005] Use cases where time-sensitive CPS applications are distributed across a network of edge resources may have been successfully deployed in the lab to demonstrate the advantages of distribution. However, the natural unreliability of the communication network at the deployment site, where individual network nodes may be susceptible to periodic outages, may still be a significant problem for actual distributed deployments of real-time critical CPS systems, which may be particularly sensitive to packet losses resulting from disconnections.

[0006] Cloud orchestration frameworks such as Kubernetes can use more traditional solutions for fault tolerance because they can use general-purpose protocols such as Paxos or Raft that have very powerful properties. However, this design decision may introduce significant overhead in terms of messages and the number of replicas required. For example, etcd (Kubernetes' very persistent distributed key-value store, which is used, among other things, to maintain the state of the cluster) may have significant network requirements, such as 1GbE for a typical deployment. Although this may not be surprising, as cloud orchestration is designed for data centers with significant networking and computing, such requirements may not be well matched to resource-poor distributed embedded systems. These protocols may be overkill for strongly synchronized systems such as industrial automation systems or automotive control devices.

[0007] Existing middleware solutions, in particular, lack the ability to identify resource disconnections quickly enough to enable failover within milliseconds and manage new connections to resources in a manner that is transparent to applications. Most existing solutions may focus on connection failures in data centers and may have significant network and data processing requirements. Some existing efforts to enable faster failover times may rely on additional mechanisms provided by (software defined) networks. Summary of the invention

[0008] According to aspects of the invention, a method having the features of claim 1, a computer program having the features of claim 8, a data processing device having the features of claim 9 and a computer-readable storage medium having the features of claim 10 are provided. Further features and details of the invention are disclosed in the corresponding dependent claims, the description and the drawings. Features and details described in the context of the method according to the invention also correspond to the computer program according to the invention, the data processing device according to the invention and the computer-readable storage medium according to the invention, and vice versa.

[0009] According to one aspect of the present invention, a method for managing connection loss for a distributed system is provided. The method comprises the following steps:

[0010] - deploying at least one support module on at least two sender runtimes, at least one receiver runtime, and at least one orchestrator of the distributed system, wherein the at least one support module is designed to provide for transmission of signals on an additional communication channel, wherein the orchestrator is designed to manage periodic communications between the at least two sender runtimes and the at least one receiver runtime,

[0011] - analyzing an additional communication channel between the at least two sender runtimes and the orchestrator in order to detect a loss of connection of at least one of the at least two sender runtimes, respectively based on the detection of a signal transmitted on the additional communication channel,

[0012] - taking at least one countermeasure when a connection loss is detected, wherein the at least one countermeasure comprises stopping, with the aid of at least one supporting module, periodic communication between at least one sender runtime that detected the connection loss and at least one receiver runtime among the at least two sender runtimes, so as to manage the connection loss in a distributed system.

[0013] The distributed system may be a distributed information technology system, such as a cloud system. The sender runtime may be a runtime that includes or hosts a sender module. The receiver runtime may be a runtime that includes or hosts a receiver module. In one example, an application may be run on the receiver runtime, and the application requires data from the sender runtime, providing the application with periodic communication between the sender runtime and the receiver runtime. The orchestrator may be a decision-making entity that has a global view of the distributed system, and may be able to deploy and withdraw modules and control the communication behavior of the modules on the runtime. Analyzing the additional communication channel may include periodically verifying by means of detection to verify whether the transmitted signal exists or exists within a defined time period. The transmitted signal may be a digital signal, such as a string or a bit sequence. When a connection loss is detected, the stop of the periodic communication may be initialized by at least one of the at least two sender runtimes. Simply put, in one example, the support module of the sender runtime may detect that a signal has not been sent by the support module of the orchestrator within a defined time period, so that the sender runtime may stop the periodic communication. The additional communication channel by means of at least one support module may be advantageous because the additional communication channel may be dedicated to detecting a connection loss and at least one of the at least two sender runtimes may detect the connection loss so that the at least one sender runtime may stop regular communication. This may advantageously prevent the receiver runtime from being flooded with data from the sender runtime when the connection was lost after a reconnection.

[0014] Possibly, the method comprises the following steps:

[0015] - starting, within the interval, the transmission of said signal on an additional communication channel by means of at least one supporting module,

[0016] - Analyzing the presence of transmitted signals in order to detect a loss of connection, wherein the presence of transmitted signals is analyzed by at least two sender runtimes and / or an orchestrator.

[0017] It is possible to transmit and detect the signal with the aid of at least one supporting module. In one example, the supporting module can be provided on the transmitter runtime and transmit the signal to other supporting modules provided on the orchestrator, which can then analyze the presence of the transmitted signal and thereby detect a connection loss, or vice versa. The expression "presence" can indicate that the transmitted signal is present at a given point in time or is present or detected within a defined period of time.

[0018] Furthermore, it is possible that the method further comprises at least one of the following steps:

[0019] - defining intervals in which said signals are transmitted, wherein said intervals are defined manually or automatically, preferably according to at least one requirement,

[0020] - defining a time period for the transmitted signal after which a loss of connection is detected.

[0021] The possibility of defining intervals can be advantageous because greater flexibility is guaranteed. For example, in more critical applications where a connection loss must be detected quickly, the interval can be defined to be very short, such as ten milliseconds. In the case of less critical applications and in order to save data processing power or network capacity, the interval can be defined to be longer, such as one second. It is possible to define different intervals for different transmission directions of the signal. In one example, the interval of the signal transmitted from the orchestrator to one of the at least two sender runtimes is different from the interval from one of the at least two sender runtimes to the orchestrator. Different intervals can advantageously enable a compromise between the response time and the communication overhead of the signal. The definition can also occur automatically and dynamically, for example in order to achieve degradation in a scenario where the available network bandwidth is reduced due to a connection or node failure. Therefore, at least one requirement can, for example, be how time-critical the application is or how much bandwidth or data processing power is provided. In the case of an alternative, the time period for the transmitted signal after which the connection loss is detected can be a multiple of the interval in which the signal is transmitted. If, for example, the interval in which the signal is transmitted is 10ms, the time period can be 30ms. Therefore, if no signal exists after 30ms, the connection loss can be detected in this case. It is also possible to analyze the pattern of the transmitted signals to detect a connection loss.In one example, a connection loss is detected when two signals or m of k signals are not detected in sequence.

[0022] In another example, at least one of the following three types of support modules is deployed on the at least two sender runtimes, the at least one receiver runtime, and / or the at least one orchestrator:

[0023] - a transmitter, wherein said transmitter transmits a signal,

[0024] - a detector, wherein said detector detects a signal transmitted by said transmitter,

[0025] - a mute switch, wherein the mute switch stops regular communication between at least one of the at least two transmitter runtimes and at least one receiver runtime.

[0026] A mute switch may be deployed on at least two transmitter runtimes to stop periodic communication. This may be advantageous because the at least two transmitter runtimes may then stop periodic communication without requiring intervention by an orchestrator. In one example, a transmitter and a detector may be deployed on at least two transmitter runtimes, at least one receiver runtime, and at least one orchestrator, respectively, so that additional communication channels may be analyzed with respect to, for example, a corresponding transmitter on one of the at least two transmitter runtimes and, for example, a corresponding detector on at least one orchestrator.

[0027] In another example, at least one countermeasure further comprises activating temporary periodic communication between at least one of the at least two sender runtimes in which the connection loss is not detected and the at least one receiver runtime. In other words, the periodic communication between at least one of the at least two sender runtimes in which the connection loss is detected can be temporarily replaced by temporary periodic communication between another of the at least two sender runtimes in which the connection loss is not detected, preferably until at least one of the at least two sender runtimes in which the connection loss is detected is connected again to the at least one receiver runtime. Thus, once the connection is reestablished, the periodic communication can be reactivated and the temporary periodic communication can be deactivated.

[0028] Furthermore, it is possible for the orchestrator to activate a temporary periodic communication between at least one of the at least two sender runtimes in which no connection loss is detected and the at least one receiver runtime. This can be advantageous because the orchestrator can have a global view of all sender runtimes and receiver runtimes that may be present in the distributed system and in particular also about possible connection losses of the respective runtimes.

[0029] The method may further comprise the following steps:

[0030] - analyzing an additional communication channel between the at least two transmitter runtimes in order to detect a loss of connection of at least one of the at least two transmitter runtimes based on the transmission of signals between the at least two transmitter runtimes.

[0031] This can be advantageous because the sender runtimes can then detect a possible connection loss each independently and without an orchestrator. It can be provided that if a connection loss is detected at one of the sender runtimes, at least one other sender runtime starts providing regular communication with at least one receiver runtime.

[0032] In another aspect of the invention, a computer program, in particular a computer program product, may be provided, the computer program comprising instructions which, when executed by a computer, cause the computer to perform the method according to the invention. Thus, the computer program according to the invention may have the same advantages as described in detail with reference to the method according to the invention.

[0033] In another aspect of the invention, a device for data processing can be provided, which is designed to perform the method according to the invention. As such a device, for example, a computer can be provided that executes a computer program according to the invention. The computer can include at least one processor that can be used to execute the computer program. In addition, a non-volatile data memory can be provided, in which the computer program can be stored and from which the processor can read the computer program for execution.

[0034] According to another aspect of the present invention, a computer readable storage medium may be provided, comprising a computer program and / or instructions according to the present invention, which, when executed by a computer, cause the computer to perform the steps of the method according to the present invention. The storage medium may be configured as a data storage device, such as a hard disk and / or a non-volatile memory and / or a memory card and / or a semiconductor drive. For example, the storage medium may be integrated into a computer.

[0035] Furthermore, the method according to the present invention may be realized as a computer-implemented method. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Further advantages, features and details of the invention are apparent from the following description, in which embodiments of the invention are described in detail with reference to the accompanying drawings. In this context, the features mentioned in the claims and the description may be essential to the invention individually or in any combination.

[0037] Figure 1 The method, computer program, storage medium and device according to the embodiments of the present invention are shown.

[0038] Figure 2 A distributed system according to the prior art is shown,

[0039] Figure 3A distributed system according to the prior art is shown,

[0040] Figure 4 A distributed system according to the prior art is shown,

[0041] Figure 5 A distributed system according to an embodiment of the present invention is shown.

[0042] Figure 6 A distributed system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0043] Figure 1 A computer program 20, a storage medium 15 and a device 10 according to an embodiment of the invention are shown.

[0044] Figure 1 Furthermore, a method 100 for managing connection loss for a distributed system 1 according to an embodiment of the present invention is shown. In a first step 101, at least one support module 5 can be deployed on at least two sender runtimes 2, at least one receiver runtime 3, and at least one orchestrator 4 of the distributed system 1. The at least one support module 5 is preferably designed to provide for the transmission of signals on an additional communication channel. The orchestrator 4 is especially designed to manage periodic communication between at least two sender runtimes 2 and at least one receiver runtime 3. In a second step 102, the additional communication channels between the at least two sender runtimes 2 and the orchestrator 4 can be analyzed respectively, so as to detect the connection loss of at least one of the at least two sender runtimes 2 based on the detection of the signal transmitted on the additional communication channel. In a third step 103, at least one countermeasure can be taken in the case of detecting the connection loss. At least one countermeasure can include stopping the periodic communication between at least one sender runtime and at least one receiver runtime 3 in the at least two sender runtimes 2 that detect the connection loss by means of at least one support module 5, so as to manage the connection loss 1 in the distributed system.

[0045] The present invention is particularly focused on a failover mechanism for connection failures that occur at intervals in a distributed system 1. It can be assumed that the system includes a collection of distributed (computing) nodes that can be connected via a communication network, and it is assumed that a so-called runtime can be deployed on each node. The runtime can be a program that can access the resources of the node on which the runtime is deployed in order to start an executable file - the executable file is called a module. In addition to starting the module and monitoring its execution, the runtime can also manage the communication between modules. In order to transmit a message generated by a sending module, the runtime hosting the module can intercept the message and then inject the message into the communication network so that the message is transmitted to the runtime hosting the receiving module. The runtime hosting the receiving module can receive the message and provide the message to the receiving module.

[0046] Applications provided as a collection of deployable modules may be deployed by an orchestrator 4. The orchestrator 4 may be a decision-making entity having a global view of the system and being able to deploy and revoke modules at runtime, as well as control the communication behavior of said modules. In particular, the orchestrator 4 may "mute" a module by configuring its runtime host to no longer transmit messages generated by the module. The orchestrator 4 may determine the mapping from modules to the runtime of the deployed application, monitor its execution and enable failover mechanisms in order to shield the deployed application from dynamic failures (e.g., connection failures or node failures).

[0047] Figure 2 A failure scenario solved by an embodiment of the present invention is shown. In a distributed system 1 that may include multiple runtimes 2, 3 and may be managed by an orchestrator 4, an application may be deployed by distributing its modules over the available runtimes 2, 3. For simplicity, this example uses in particular a minimal application comprising a transmitter module T (also referred to as a speaker) that produces data and a listener-receiver module L (also referred to as a listener) that consumes these data. For example, these modules may represent a control task (speaker) that calculates a control signal that is then sent to an actuator task (listener). The right part of the figure in particular emphasizes the disconnectability of this deployment: once RT 1 is disconnected, the listener may not be able to receive the received data and the application may fail.

[0048] Figure 31 shows how hot standby deployment of task replicas can harden an application relative to a runtime disconnect. As explained above, the orchestrator 4 can selectively deactivate the communication capabilities of individual modules. This capability can be used to deploy a "silent" replica of a sender module T on RT 3 as a "backup" for a "primary" sender module T communicating on RT 1. Since the orchestrator 4 can detect a disconnect (which can be difficult, as briefly described below), the orchestrator 4 can respond by activating the communication capabilities of the backup sender module T, which can then generate the data required by the receiver module L, so that the application may not be affected by the disconnect and can continue its execution.

[0049] However, this approach may not be sufficient to handle the scenario of disconnections that occur at intervals (which may be a very frequent failure in large-area networks), where RT 1 rejoins the network after it has been disconnected for a time interval, such as in Figure 4 As soon as RT 1 joins the network again, the receiver module L may start receiving replicated messages, which may cause the application to fail if the application is not specifically written in a way that allows the application to handle message replication - a capability that is typically missing for most applications. It should be noted that the ability of the orchestrator 4 to mute a module and unmute it again may not be sufficient to address this scenario. Until RT 1 disconnects, the transmitter module T can act as a failover master device on RT 1 and generate messages. During the time that RT 1 is disconnected, it may not be possible for the orchestrator 4 to mute the module because the orchestrator cannot communicate with RT 1. The orchestrator 4 can be implemented to mute one of the transmitter modules T as soon as the orchestrator 4 detects a new connection, and it may not be possible for the orchestrator to prevent that the receiver module L can receive replicated messages within a certain time interval, which may have catastrophic consequences for the application.

[0050] In the explanations so far, it has been assumed in particular that the orchestrator 4 is able to detect runtime disconnections with sufficient speed to perform failover, i.e., activate a backup, before the application fails. For real-time critical CPS applications that may be particularly sensitive to packet loss, the complete middleware response chain - (a) detecting the disconnection, (b) making a decision for determining a response, and (c) making the response effective - is particularly completed within very short time intervals (in the range of milliseconds), thereby requiring, for example, a detection mechanism that is able to detect the disconnection in even shorter times. Most mechanisms for detecting disconnections in a distributed system 1 can be based on some type of periodic heartbeat, i.e., a scheme that periodically sends messages between a trusted reliable node (e.g., the node on which the orchestrator 4 is deployed) and the monitored node. As long as the heartbeat is received regularly, the monitored node can be considered connected, and the absence of a heartbeat message can signal a disconnection. Most existing middleware and communication protocols are equipped with built-in heartbeat functionality, and typically may not be able to configure the following heartbeat period, which will be short enough to enable disconnection detection at a sufficient speed to provide failover for the CPS application. The smallest possible configurable time period may typically be of the order of one second. A possible reason for this would be that reducing the heartbeat period may increase the volume of messages sent over the network, possibly to the point where the interference / network load of these messages may affect the deployed application.

[0051] The present invention may provide such a failover mechanism. The present invention may be used to harden real-time critical applications with respect to disconnection during operation after intermittent network connection failures. The present invention may require neither adaptation of the application or specific assumptions about the application nor any fault detection or recovery mechanisms supplied by the underlying communication protocol. The application may be treated as a black box and may be completely agnostic about its distributed deployment.

[0052] Compared with existing solutions, the present invention can provide at least the following advantages. Detection of disconnection and reconnection situations within a few milliseconds of their occurrence, without having to rely on mechanisms supplied by the underlying communication protocol. Mechanisms for triggering low-latency failover when using hot standby deployment. Mechanisms for managing modules on a disconnected runtime during the time when the runtime is disconnected from the network. Mechanisms for managing a previously disconnected runtime when the previously disconnected runtime reconnects to the network. All of the advantages listed above can be provided in a manner that is completely transparent to the ongoing application and may not require any adaptation of the application code or application behavior.

[0053] The object of the proposed invention may be to harden a distributed real-time application against disconnections that occur at intervals. The mechanism may be based on the idea that in addition to the application modules, the orchestrator 4 may deploy so-called "sidecar" modules. A "sidecar" may be a specialized orchestration module that may be deployed on a runtime together with the modules of the application. In particular, the orchestration module implements error handling criteria in order to (a) detect disconnections of the runtime, (b) manage and / or monitor the communication behavior of the application modules, (c) observe and / or manage the runtime: whether the runtime will be disconnected from the network and therefore from the orchestrator 4, (d) detect when the runtime is reconnected to the network, and / or (e) cooperate with the orchestrator 4 to integrate the reconnected runtime into the system again. In the context of the present invention, the sidecar module may be synonymously referred to as a support module 5. For this deployment, the orchestrator 4 may take into account the computing and networking requirements of the application and the sidecar module in order to ensure that the selected deployment meets the resource requirements and real-time constraints of the application. This "sidecar" mechanism allows for extremely fast failover and does not rely on failure detection or response mechanisms provided by the underlying communication protocol.

[0054] Furthermore, the mechanism may not require any adaptation of the application code, so that even the mechanism may be used to "build-in" error handling functionality to existing applications "after the fact." By bringing the orchestration closer to the processing, the introduced sidecar mechanism may thus extend the temporal and spatial locality of the recovery logic of the distributed system 1.

[0055] In addition to the general mechanism of using the sidecar (i.e., support module 5), a solution for managing disconnection-reconnection scenarios according to an embodiment of the present invention is described below for orchestration tasks. The solution is particularly based on the dynamically deployed support module 5 for generating runtime heartbeats and managing the communication behavior of application modules, wherein the application is transparently shielded from (a) runtime disconnections due to (transient) connection failures and (b) potential interference caused by reconnecting to a previously disconnected runtime.

[0056] Figure 5 A number of possible components of the inventive solution for solving the scenario of disconnections occurring at intervals are visually shown.As explained in the previous paragraph, the solution may be based on the use of a sidecar, ie the support module 5 . Figure 5The scheme depicted in the embodiment uses, in particular, three different types of support modules 5: (a) a transmitter E, which, after being deployed on the runtime, can publish or transmit signals or in particular heartbeat messages in a periodic manner, the period of which can be dynamically configured (preferably through the orchestrator 4), (b) a detector D, which can be responsible for listening to a specific heartbeat, detecting the absence of the heartbeat, i.e. detecting a disconnection, and triggering a disconnection response of the orchestrator 4 and / or other support modules 5, and (c) a mute switch S, which, after receiving a trigger signal from the detector D, can configure the runtime on which the detector is hosted so as to deactivate the communication of other modules hosted through the runtime. In order to ensure robustness against false positive results, it is possible to flexibly configure what exactly indicates the absence of a heartbeat, such as the omission of two heartbeats in sequence or the receipt of less than m out of k heartbeats. As Figure 5 As shown in , orchestrator 4 may deploy a pair of emitters E and detectors D to (a) monitor the connectivity status of the runtimes and (b) enable the runtimes to detect when they are disconnected.

[0057] Implementing the heartbeat mechanism through the sidecar module or support module 5 allows fine-grained control over the heartbeat period: different heartbeat periods can be configured for different runtimes. Runtimes on which more important or sensitive tasks are running can be designed to send heartbeats more frequently. In addition, different heartbeat directions can be configured. There may be different periods for (a) heartbeats sent from the orchestrator 4 to the runtime and (b) heartbeats sent from the runtime to the orchestrator 4. The above enables a trade-off between response time and the communication overhead of the heartbeat. Reconfiguration of the heartbeat period can even occur dynamically, for example to enable slight degradation in scenarios where the available network bandwidth is reduced due to connection failures or node failures.

[0058] The support modules 5 are preferably deployed by an orchestrator 4, which may have a global view of the system and may jointly consider the computing and networking requirements of the application and support modules 5. Figure 5As shown on the right, the use of support module 5 can not only allow very fast detection of disconnection, but also enable orchestrator 4 to manage the runtime when it is disconnected. In the depicted example, detector D deployed on RT 1 detects the disconnection and notifies (a) transmitter E to stop sending heartbeats and notifies (b) mute switch S to deactivate the communication capabilities of application task T. This response, which as already mentioned cannot be directly caused by orchestrator 4 being disconnected from RT 1 at this point in time, ensures that neither messages, i.e. heartbeats nor application messages, are sent through RT 1. Therefore, the runtime can now safely join the network again without flooding the network with messages. In contrast, detector D deployed on a node of orchestrator 4 can detect the disconnection of RT 1 and trigger a response of orchestrator 4, in which (a) the communication capabilities of the backup task on RT3 are activated, which are set to original, and (b) the mute switch S for transmitter module T is deployed on RT3 if RT3 is disconnected.

[0059] at last, Figure 6 In particular, a secure reconnection is shown by using the support module 5. The sequence of events may unfold as follows: (1) As soon as RT 1 is connected to the network again, the detector D of RT 1 may start receiving heartbeats sent by the transmitter E on the node of the orchestrator 4. (2) The detector D may inform the transmitter E of the reconnection, which may start sending heartbeats again on its side. (3) The detector D responsible for Res 1 on the node of the orchestrator 4 may receive the heartbeat of the orchestrator 4 and may inform the orchestrator 4, which preferably takes over the control and management of RT 1 again from this point on.

[0060] The entire process according to one embodiment is described below. In normal operating mode, the orchestrator 4 and the runtime can exchange periodic heartbeats. After disconnection, the detector D on the runtime can detect the absence of the heartbeat of the orchestrator 4, and can deactivate the runtime heartbeat and mute the application modules hosted there. When the detector D on the orchestrator 4 detects the absence of the runtime heartbeat, the detector preferably activates failover by unmuting the backup transmitter module T. Since the orchestrator 4 may continue to send its heartbeat, the detector D on the runtime can detect the heartbeat after reconnection and can reactivate the transmitter E on the runtime. When the detector D of the orchestrator 4 receives the runtime heartbeat again, the orchestrator 4 can be notified so that the orchestrator can take over control of the runtime again.

[0061] Possible alternatives to the described scheme are described below, since the use of a sidecar can generally provide great flexibility for these and other failure / management scenarios. A possible alternative restriction would be, for example, to configure the detector D to listen to application messages instead of generating additional heartbeats. In case this is easily possible with communication schemes such as PubSub, this can reduce the network bandwidth overhead introduced by the mechanism. When, for example, a motion control application comprising sensor and actuator modules is hardened, the detector D can be deployed on the runtime hosting the actuator modules and periodic sensor signals can be listened there, thereby enabling a fast disconnect failover when the runtime hosting the sensor modules is disconnected, without any overhead related to the network load.

[0062] In the case of another alternative, a maximum absence time interval can be defined for the application module. In the case of a disconnection, for example, the middleware and the watchdog support module 5 on the disconnected resource can monitor how long the resource has been disconnected. Once the absence time of a module is exceeded, the module can be (a) removed from the disconnected resource by the resource watchdog and can (b) be deployed again on the connected resource by the middleware watchdog.

[0063] In case of another alternative, the logger support module 5 may log messages of the application modules during the disconnected time, instead of silencing the application modules. Various schemes may be used for this, such as log all messages, log the last n messages, etc. After reconnection, the messages may be injected into the network in an organized manner coordinated with the middleware. This may be another example of functionality implemented on a disconnected runtime, which may be achieved through the sidecar concept.

[0064] In another example, the presented scheme can also be extended for use with other failover mechanisms, such as cold backup or redeployment of a module whose runtime has been disconnected on another (connected) runtime.

[0065] Furthermore, the mute switch S can additionally monitor messages sent by the application modules when they are connected in order to detect "babbling idiot" erroneous behavior, mute the module and trigger a failover to the backup before the module can affect the application by flooding the network. "Babbling idiot" erroneous behavior can be sending messages out of sync with the network at random times or sending incorrect message content.

[0066] Furthermore, the possibility for managing disconnected runtimes can also be used to ensure a higher degree of autonomy with respect to the orchestrator 4, for example by deploying support modules 5 which can take over the management of the runtimes in case of a disconnection / abnormal termination of the orchestrator 4 or which can autonomously migrate modules between runtimes in order to balance the computing load / network load.

[0067] The above explanation of the embodiments describes the invention in the context of examples. Of course, the individual features of the embodiments can be freely combined with one another, as long as this is technically appropriate, without departing from the scope of protection of the present invention.

Claims

1. A method (100) for managing connection loss for a distributed system (1), comprising the following steps: - deploying (101) at least one support module (5) on at least two sender runtimes (2), at least one receiver runtime (3) and at least one orchestrator (4) of the distributed system (1), wherein the at least one support module (5) is designed to provide for transmission of signals on an additional communication channel, wherein the orchestrator (4) is designed to manage periodic communications between the at least two sender runtimes (2) and the at least one receiver runtime (3), - analyzing (102) an additional communication channel between the at least two sender runtimes (2) and the orchestrator (4) in order to detect a loss of connection of at least one of the at least two sender runtimes (2) based on the detection of a signal transmitted on the additional communication channel, - taking (103) at least one countermeasure in the event of a detected connection loss, wherein the at least one countermeasure comprises stopping, by means of the at least one support module (5), periodic communication between at least one of the at least two sender runtimes (2) that detected the connection loss and the at least one receiver runtime (3) in order to manage the connection loss in the distributed system (1).

2. The method (100) according to claim 1, characterized in that: The method (100) further comprises the following steps: - starting the transmission of said signal on said additional communication channel in an interval by means of said at least one supporting module (5), - analyzing the presence of the transmitted signal in order to detect a loss of connection, wherein the presence of the transmitted signal is analyzed by the at least two transmitter runtimes (2) and / or by the orchestrator (4).

3. The method (100) according to claim 2, characterized in that: The method (100) further comprises the following steps: - defining intervals in which said signals are transmitted, wherein said intervals are defined manually or automatically, preferably according to at least one requirement, - defining a time period for the transmitted signal after which a loss of connection is detected.

4. The method (100) according to any one of the preceding claims, characterized in that At least one of the following three types of support modules (5) is deployed on the at least two sender runtimes (2), the at least one receiver runtime (3) and / or the at least one orchestrator (4): - a transmitter (E) wherein said transmitter (E) transmits said signal, - a detector (D), wherein said detector (D) detects a signal transmitted by said emitter (E), - a mute switch (S), wherein the mute switch (S) stops regular communication between at least one of the at least two transmitter runtimes (2) and the at least one receiver runtime (3).

5. The method (100) according to any one of the preceding claims, characterized in that The at least one countermeasure further comprises activating a temporary periodic communication between at least one of the at least two sender runtimes (2) in which no connection loss is detected and the at least one receiver runtime (3).

6. The method (100) according to claim 5, characterized in that: Temporary periodic communication between at least one of the at least two sender runtimes (2) in which no connection loss is detected and the at least one receiver runtime (3) is activated by the orchestrator (4).

7. The method (100) according to any one of the preceding claims, characterized in that The method (100) further comprises the following steps: - analyzing an additional communication channel between the at least two transmitter runtimes (2) in order to detect a loss of connection of at least one of the at least two transmitter runtimes (2) based on the transmission of signals between the at least two transmitter runtimes (2).

8. A computer program (20) comprising instructions which, when executed by a computer (10), cause the computer (10) to perform the method (100) according to any one of the preceding claims.

9. A data processing device (10) comprising means for executing the method (100) according to any one of claims 1 to 7.

10. A computer-readable storage medium (15) comprising instructions which, when executed by a computer (10), cause the computer (10) to perform the steps of the method (100) according to any one of claims 1 to 7.

Images