Method for transmitting a data stream in a publish-subscribe system and data stream producer

By dynamically negotiating side channels and using adapted protocols for data streams, the method addresses the challenge of transmitting high-frequency data streams in IoT systems, ensuring efficient and real-time communication in decentralized IoT environments.

WO2024240535A1PCT designated stage expired Publication Date: 2024-11-28SIEMENS AG
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2024/063144
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-25
Filing Date
2024-05-14
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Decentralized IoT systems face challenges in transmitting high-frequency, large-scale continuous data streams using publish-subscribe systems, as existing protocols are not designed for real-time or quasi-real-time communication and can lead to network congestion and latency due to increased load on brokers.

Method used

The method involves advertising and accepting offers for side channels using publish-subscribe messages, dynamically negotiating separate communication channels between senders and receivers, and transmitting data streams using protocols adapted for the specific data type, such as video or audio, to ensure optimal efficiency and meet real-time requirements without affecting the publish-subscribe system's fundamental advantages.

Benefits of technology

This approach allows for efficient, real-time transmission of data streams by decoupling them from the publish-subscribe system, reducing latency and network congestion, and enabling flexible negotiation of data rate and quality of service, thus supporting time-critical applications in industrial IoT environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2024063144_28112024_PF_FP_ABST
    Figure EP2024063144_28112024_PF_FP_ABST
Patent Text Reader

Abstract

In the method for transmitting a data stream in a publish-subscribe system, offers for one or more side-channels for transmitting the data stream by means of publish-subscribe messages are tendered and offers from side-channels by means of publish-subscribe messages are received, wherein the offers undergo a check and side-channels are used to transmit the data stream depending on a result of the check. The data stream producer comprises in particular a processor, designed to execute such a method.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Description

[0002] Method for transmitting a data stream in a publish-subscribe system and data stream producer

[0003] The invention relates to a method for transmitting a data stream in a publish-subscribe system and a data stream producer.

[0004] In decentralized IoT systems, there is a need for methods for transmitting data streams by means of which autonomous, loosely coupled system participants, such as IoT devices, exchange messages using publish-subscribe communication.

[0005] This need exists particularly in the context of Industry 4.0, i.e., flexible and / or autonomous production and logistics and their end-to-end digitalization with technologies from the areas of IoT, AI, edge, cloud, mobile, and distributed computing, as well as from the areas of new communication technologies, e.g., 5G and 6G. In this area, there is increasing demand for the transmission of data streams that form continuous data sets and that are continuously provided by IoT system components from the IT and OT sectors at high frequency and / or on a large scale. Such data streams are regularly transmitted to consuming system components in industrial work environments for the purposes of data analysis, further processing, and storage.

[0006] Typical examples of the application of such data streams are continuously running telemetric process data from a manufacturing or work process, which is processed in functional pipelines in a serverless edge cloud infrastructure, and / or high-frequency and / or large-volume field data from field devices and sensor systems, such as ultrasound, infrared, or tomographic images and / or recorded or real-time audio and / or video streams, or an upload or download of files, or an export or import of data sets from or to external systems. Processing of such data streams is often required in time-critical application contexts or (quasi-)real-time systems.

[0007] The object of the present invention is to enable such data streams by means of publish-subscribe messages.

[0008] This object of the invention is achieved by a method having the features specified in claim 1 and by a data stream producer having the features specified in claim 9. Preferred developments of the invention are specified in the associated subclaims, the following description and the drawing.

[0009] In the method according to the invention for transmitting a data stream in a publish-subscribe system, offers for one or more side channels for transmitting the data stream are advertised by means of publish-subscribe messages and offers from side channels are received by means of publish-subscribe messages, wherein the offers are subjected to a check and one or more side channels are used to transmit the data stream depending on the result of the check.

[0010] In the method according to the invention, data streams can nevertheless be transmitted by means of publish-subscribe messages, which themselves are not set up to transmit continuous data streams. Publish-subscribe messages themselves cannot generally guarantee such a transmission of data streams, since the transport protocols of publish-subscribe messages are designed to disseminate discrete, complete and self-contained information, such as in particular signals and / or events and / or status messages, in the form of messages with a limited payload, which can be coded and interpreted in isolation, i.e. independently of other messages. This does not necessarily apply to the aforementioned continuous data streams.Furthermore, publish-subscribe messages cannot take real-time or quasi-real-time requirements into account, since specialised real-time communication-capable transport protocols are used for this purpose via a point-to-point connection, such as the WebRTC transport protocol for audio streaming and / or video streaming.

[0011] In broker-based publish-subscribe systems, where communication follows established standards such as the MQTT standard, routing a continuous data stream through the broker—in the form of individual messages with fragmented data blocks—would lead to increased load on the broker and increased latency in message delivery, which negatively impacts the throughput of the publish-subscribe system and can cause network congestion. In the method according to the invention, however, data streams are transmitted on communication channels separate from the publish-subscribe system. These channels are negotiated dynamically between senders and receivers at runtime and only when needed via the publish-subscribe system in such a way that optimal transmission efficiency between sender and receiver can be achieved.In the context of the present invention, transmitters are also referred to as data stream producers, while receivers are referred to as data stream consumers.

[0012] With the method according to the invention, offers for one or more side channels can be advertised and offers from side channels can be received in the usual way using publish-subscribe messages. In this respect, the communication processes of publish-subscribe systems remain unaffected, and the method according to the invention advantageously requires no special adaptations.

[0013] An industrial network which already uses a publish-subscribe system for the communication of components of the publish-subscribe system can be advantageously and easily expanded by new types of data streams and their transmission protocols by means of the method according to the invention, since only the affected data stream production components and data stream consumption components have to be changed to implement the side channels.

[0014] According to the invention, fundamental advantages of a publish-subscribe system, such as enabling latency times and throughput, are therefore retained in principle, since only an agreement of side channels and / or their parameters takes place by means of publish-subscribe messages, but not the transmission of data streams.

[0015] According to the method according to the invention, the data stream is preferably not transmitted via the one or more side channels themselves as or by means of publish-subscribe messages. According to the invention, the offers for the one or more side channels for transmitting the data stream are advertised by means of publish-subscribe messages and offers from side channels are received by means of publish-subscribe messages and the offers are checked, but the use of the one or more side channels themselves depending on a result of the check for transmitting the data stream, and thus the transmission of the data stream via the side channel or channels themselves, expediently does not take place as or by means of publish-subscribe messages, but preferably as or by means of messages of a different protocol.

[0016] In a preferred development of the method according to the invention, the offers for the one or more side channels contain parameters for transmitting the data stream and the check takes the parameters into account and the transmission of the data stream takes place according to the parameters.

[0017] In the method according to the invention, the parameters expediently comprise an endpoint address of the side channel. In the method according to the invention, the parameters preferably comprise one or more transport protocols for the side channel SK and / or a content type of the data stream, in particular video and / or audio, and / or an encoding, in particular an MPEG-4 format, of the data stream.

[0018] In this further development of the invention, the respective data stream type can advantageously be transmitted using a transmission protocol adapted for it, which corresponds to the respective state of the art. Corresponding advantages arise analogously for other communication parameters via the respective side channel.

[0019] An industrial network which already uses a publish-subscribe system for the communication of components of the publish-subscribe system can advantageously be easily expanded by new types of data streams and their transmission protocols by means of the method according to the invention, since only the affected data stream production components and data stream consumption components have to be changed to implement the side channels.

[0020] In a preferred development of the method according to the invention, the parameters comprise a data rate and / or a quality of service of the data stream.

[0021] In this further development of the invention, publish-subscribe messages advantageously enable the flexible negotiation of optimal transmission efficiency for data streams between senders and receivers, meeting the requirements of both the sender and receiver. In particular, this approach can avoid problems that arise when, for example, senders send a data stream at a higher data rate than the receiver can receive or process.

[0022] In the method according to the invention, the data stream is constructed, in particular, from a data stream producer to a data stream consumer. By means of the publish-subscribe messages used according to the invention, senders and receivers of data streams can be easily loosely coupled for the transmission of data streams. In particular, data stream producers are the senders of the data stream and data stream consumers are the receivers of the respective data stream. For transmission, data stream producers do not need any prior knowledge of the data stream consumer and, in particular, do not initially need to know connection data such as its endpoint address. According to the invention, a system configuration of data stream producers is therefore not specifically required, which enables ad-hoc transmission of data streams and significantly reduces the effort required to configure and commission data stream producers and data stream consumers.

[0023] Particularly preferably, in the method according to the invention, the data stream consumer(s) and / or the data stream producer(s) is / are component(s) of a, in particular industrial, Internet of Things system.

[0024] The data stream producer according to the invention comprises in particular a processor and is designed to carry out a method according to the invention as described above.

[0025] The data stream producer according to the invention is particularly configured and designed to implement the preferred developments of the method according to the invention. The data stream producer according to the invention offers the same advantages as those already mentioned for the method according to the invention.

[0026] Preferably, in the data stream producer according to the invention, the processor is configured to execute the method according to the invention as described above.

[0027] The data stream producer according to the invention is expediently an industrial Internet of Things device and / or comprises a camera and / or a microphone that is expediently designed to generate the data stream. The invention is explained in more detail below with reference to an exemplary embodiment illustrated in the drawing.

[0028] Figure 1 shows an Internet of Things network, which will also be referred to below as an IoT network (IOTN). In the illustrated embodiment, the IoT network (IOTN) comprises a data stream consumer (DSK) and a data stream producer (DSP). In the illustrated embodiment, the data stream consumer (DSK) and the data stream producer (DSP) are implemented as wirelessly networked components of a production plant. The components of the production plant are connected to one another via a publish-subscribe system (PUBSUB). The data stream consumers (DSK) are decentralized control units that receive continuous measurement signals in the form of a video feed. The video feed is provided by the data stream producer (DSP), which is formed by a computer-controlled surveillance camera. Using the video feed, the data stream consumers, i.e. the control units, can monitor process parameters and control variables for controlling the process parameters.Each of the decentralized control units receives a copy of the entire video feed. The data stream producer (DSP) transmits the data stream DS to each of the data stream consumers (DSK).

[0029] In the IoT network IOTN shown, the data streams DS are transmitted via a side channel SK, i.e., via a separate communication channel separated from the publish-subscribe system PUBSUB, using a specific transport protocol suitable for the respective type of data stream and which, in this example, is not designed to transmit data streams as publish-subscribe messages. The side channel SK is negotiated dynamically at runtime, as needed, between the data stream producer DSP and the data stream consumer DSK via the existing publish-subscribe system with the aim of achieving an optimal balance between the requirements of the data stream producer and those of the data stream consumer. For this purpose, the data stream producer DSP is loosely coupled to all data stream consumers DSK via the publish-subscribe system PUBSUB.A data stream producer DSP sends a request ANF to transmit a data stream DS to all interested data stream consumers DSK via the publish-subscribe system PUBSUB, receives their offers AGB, sets up side channels SK to the offer creators and transmits the data stream DS on each individual side channel SK to the respective data stream consumers DSK.

[0030] The method according to the invention comprises the steps described below:

[0031] Steps S1, S2, and S3 concern a comparison procedure for establishing side channels SK using publish-subscribe communication between the data stream producer DSP and the data stream consumers DSK, which submit offers of terms and conditions for consuming the data stream. Steps S4, S5, and S6, on the other hand, concern the subsequent transmission of the data stream via the established side channels SK from the data stream producer DSP to the data stream consumers DSK.

[0032] In the first step S1, which constitutes a request step, the data stream producer DSP establishes a subscription to the topic "Side Channel Offer" by specifying a unique request identifier and then publishes a message on the topic "Side Channel Request" by specifying this request identifier and specific request parameters in its payload. Such specific request parameters specify the available transport protocols for the side channel SK and further characteristics of the data stream DS to be transmitted, such as a content type of the data stream DS, in this case video, and an encoding, in this case an MPEG-4 format, of the data stream DS. Alternatively, in further embodiments, the content type may specify an audio signal, wherein the video feed in these further embodiments is replaced by an audio feed from a monitoring microphone.Furthermore, further characteristics of the data stream DS include, if applicable, predetermined data compression methods, a minimum possible and a maximum possible data rate, a desired data rate and guarantees regarding the quality of service, also referred to as quality of service.

[0033] In the second step S2, which constitutes an offer step, the data stream consumers DSK have established a permanent subscription to the topic "Side Channel Request" with any request identifier as a wildcard. Upon receipt of a message on this topic, a check is carried out to determine whether the requirements of the respective data stream consumer DSK for the data stream to be transmitted DS can be matched with the requirements in the payload. If this is not possible, no offer is submitted by the respective data stream consumer DSK. Otherwise, an offer is created and published as a payload on the topic "Side Channel Offer" with the correlated request identifier specified. The offer comprises the request parameters of the data stream consumer DSK and additional specific connection parameters, including an endpoint address that enables the data stream producer DSP to establish a side channel SK to the data stream consumer.

[0034] In a third step S3, which constitutes an offer selection step, the data stream producer DSP receives offer messages from the offer creators on the subscription established in the first step S1 and collects them until a certain offer period has elapsed. After that, it cancels the subscription so that late-arriving offers are no longer delivered. The data stream producer DSP then compares each received offer with its requirement parameters specified in the ANF request. All offers that satisfy both its requirement parameters and the requirement parameters of the respective offer creator are selected for establishing a side channel and transmission via this respective side channel.

[0035] In a fourth step S4, which constitutes a setup step, the data stream producer DSP opens a side channel SK to each selected offer creator, i.e., each selected data stream consumer DSK, by establishing a connection via a transport protocol that satisfies the common requirements with the matched request parameters to the endpoint address specified in the offer.

[0036] In a fifth step S5, the data stream producer (DSP) encodes and transmits the data stream simultaneously via the open side channels (SK). Simultaneously, the data stream consumers (DSK) decode the transmitted data and forward it to further processing components.

[0037] In a sixth step S6, a termination step, the data stream DS per side channel SK is completely transmitted, or the data stream DS is no longer to be consumed by the respective data stream consumer DSK. In step S6, the side channel SK is then closed. This sixth step S6 can usually be initiated both on the sender side and the receiver side, i.e., by both the data stream producer DSP and the data stream consumer DSK.

[0038] The previously described steps of the method according to the invention, which relate to the comparison of the sender- and receiver-side requirements, i.e., the request parameters, for the transmission, aim to find an optimal transmission protocol with optimal transmission parameters that both the data stream producer DSP and the data stream consumer DSK technically support. A data stream consumer DSK is only considered in the described method if at least one of the transport protocols in its offer matches the transport protocols specified in the request, hereinafter also referred to as the protocol. If the data stream consumer DSK and the data stream producer DSP specify maximum data rates, the sender ensures that the selected transmission rate does not exceed the minimum of the maximum data rates of the data stream consumer DSK and the data stream producer DSP, i.e.that both maximum data rates are met simultaneously. If the data stream consumer (DSK) and the data stream producer (DSP) each specify minimum data rates, the sender ensures that the selected transmission rate does not fall below the maximum of the two minimum data rates, i.e., that both minimum data rates are met simultaneously. This comparison also applies to all other requirement parameters.

[0039] The side channel SK is operated in the client-server model, with the data stream producer DSP taking on the role of the client that requests the transmission service from the data stream consumer DSK, who takes on the role of the server. Since the server services assigned to the data stream consumers DSK are permanently in operation and listen for the endpoint addresses specified in the offers T&Cs, a client can establish a connection to a server via a side channel SK without first informing the data stream consumer DSK. This means that the data stream producer DSP does not have to send confirmation messages to the offer creators indicating whether their offers T&Cs have been accepted. Such an additional step in the described procedure would only be necessary if the server service operated by the offer creator is to be started up with a delay, i.e. only when needed, once its submitted offer T&Cs has been accepted.

Claims

Patent claims 1. Method for transmitting a data stream (DS) in a publish-subscribe system (PUBSUB), in which offers for one or more side channels (SK) for transmitting the data stream (DS) are advertised by means of publish-subscribe messages and offers (AGB) from side channels (SK) are received by means of publish-subscribe messages, wherein the offers (AGB) are subjected to a check and one or more side channels (SK) are used to transmit the data stream (DS) depending on the result of the check.

2. Method according to the preceding claim, in which transmission of the data stream (DS) via the one or more side channels (SK) itself does not take place as or by means of publish-subscribe messages.

3. Method according to the preceding claim, in which the offers for the one or more side channels (SK) contain parameters for transmitting the data stream (DS) and the check takes the parameters into account and the transmission of the data stream (DS) takes place in accordance with the parameters.

4. Method according to one of the preceding claims, wherein the parameters comprise an endpoint address of the side channel (SK).

5. Method according to one of the preceding claims, in which the parameters comprise one or more transport protocols for the side channel (SK) and / or a content type of the data stream (DS), in particular video and / or audio, and / or a coding, in particular an MPEG-4 format, of the data stream (DS).

6. Method according to one of the preceding claims, wherein the parameters comprise a data rate and / or a quality of service of the data stream (DS).

7. Method according to one of the preceding claims, in which the data stream (DS) is constructed from a data stream producer (DSP) to a data stream consumer (DSK).

8. Method according to one of the preceding claims, in which the data stream consumer(s) (DSK) and / or the data stream producer(s) (DSP) is / are component(s) of a, in particular industrial, Internet of Things (IOTN) system.

9. Data stream producer, in particular comprising a processor designed to carry out a method according to one of the preceding claims.

10. Data stream producer according to the preceding claim, wherein the processor is configured to execute a method according to one of claims 1 to 8.

11. A data stream producer according to any one of the preceding claims, which is an industrial Internet of Things device and / or comprises a camera and / or a microphone.

Citation Information

Patent Citations

  • System and Method for Querying and Updating a Live Video Stream Using a Structured Query Language

    US20190297131A1