Scalable distributed data architecture

Abstract

Computer-implemented method (600) for processing sensor data by a first endpoint device (102), comprising: Transmitting (604) a first message to other endpoint devices (102) in a network (130), the first message indicating that the first endpoint device (102) is providing a first service; Receiving respective second messages from a plurality of endpoint devices (102), wherein the respective second messages indicate that the plurality of endpoint devices (102) also provides the first service; Forming (606) a private communication network with the multitude of endpoint devices (102); Receiving (616) sensor data via a sensor; Performing (618) a first operation to extract a first feature from the sensor data; and Transmitting (620) a third message containing data associated with the first feature to the plurality of endpoint devices (102).

Description

CROSS-REFERENCE TO RELATED REGISTRATIONS

[0001] This application claims the benefit of the provisional US patent application entitled “Scalable Distributed Data Architecture”, filed on January 9, 2019, and application number 62 / 790,425. The subject matter of that related application is hereby incorporated by reference. GENERAL STATE OF THE ART Area of ​​embodiments

[0002] Embodiments of the present invention generally relate to computing devices and, in particular, a scalable distributed data architecture. Description of the state of the art

[0003] In a distributed system, such as an Internet of Things (IoT) architecture, computing devices equipped with one or more sensors are integrated into everyday devices such as video cameras, thermostats, kitchen appliances, and the like. These computing devices, also referred to here as "endpoint devices," communicate via a wired or wireless communication network, enabling them to transmit and receive data to other computing devices reachable through the network. More precisely, each endpoint device continuously receives data from one or more associated sensors and transmits the sensor data to a gateway device. Generally, the endpoint devices transmit raw sensor data without performing any analysis or processing of the sensor data prior to transmission.The endpoint devices can also receive data, such as configuration data, via the gateway device.

[0004] Traditionally, the network over which the endpoint devices and the gateway device communicate is compatible with communication using the User Datagram Protocol (UDP) or the Transmission Control Protocol (TCP). In a UDP- and / or TCP-compatible network, each endpoint device and each gateway device is configured as either a client or a server. The gateway device, in turn, forwards the sensor data to other computing devices, such as data aggregators and / or cloud servers. These data aggregators and / or cloud servers process the sensor data and initiate various operations based on it. For example, a data aggregator or cloud server might process sensor data received from a video camera to determine the presence of an intruder.In response, the data aggregator or cloud server transmits a notification to a security monitoring company or a local police unit for further action.

[0005] One drawback of the traditional distributed system, however, is that the amount of sensor data transmitted over the network can increase significantly as endpoint devices and associated sensors improve their performance and resolution over time. For example, a distributed system for a conference room might include multiple video cameras to track one or more people in the room and to direct highly directional speakers toward each person as they move around the room. A typical high-resolution video camera might have a frame rate of 1080 pixels per line by 720 lines, a bit depth of 24 bits per pixel, and a frame rate of 24 frames per second (fps).A high-resolution video camera of this type can transmit a compressed video stream of pixel data, requiring a bandwidth of approximately 447 megabits per second (Mbps). If four such video cameras are needed to cover the entire area of ​​the conference room, the total amount of video data transmitted by the four cameras would be 4 x 447 Mbps, or 1788 MBps. Furthermore, the amount of video data transmitted over the network increases linearly as more video cameras are added to the distributed system.

[0006] In general, this volume of data consumes a significant portion of the available bandwidth of a typical communications network. As the number of video cameras on the network increases, the latency between when an endpoint device transmits video data and when the data aggregator or cloud server receives it can increase considerably. Furthermore, the amount of processing resources required by the data aggregator or cloud server can be substantial, further increasing the latency between when an endpoint device transmits video data and when the data aggregator or cloud server takes action in response. Consequently, the performance of the distributed system can become unacceptably low, thus reducing its suitability for people tracking.

[0007] As illustrated by the above, there is a need in the field for more effective techniques for processing sensor data in a distributed system.

[0008] DE 10 2018 107 744 A1 describes control algorithms and system architectures for managing the operation of networked control systems and devices, including vehicles with an on-board network of electronic control units (ECUs) and control logic for regulating the sleep and activation of these ECUs.A method for managing the in-vehicle network of ECUs in a motor vehicle includes: determining status vectors for a group of ECUs, each status vector indicating whether the corresponding ECU is active or inactive; determining the device roles for these ECUs – slave or master; determining an assigned hierarchy for selecting the ECUs as master devices; receiving a mode change signal indicating that an ECU intends to enter the sleep or active state; and, in response, modifying the respective device role for one ECU from master to slave and the respective device role for another ECU from slave to master based on the assigned hierarchy and the status vectors for the ECUs. SUMMARY

[0009] The problem underlying the invention is solved by the subject matter of the independent claims. Further advantageous embodiments are specified in the dependent claims.

[0010] Various embodiments of the present disclosure describe a computer-implemented method for processing sensor data by a first endpoint device of a plurality of endpoint devices. The endpoint device is also referred to herein as a distributed message handler (DMH). The method includes receiving first sensor data via a first sensor. The method further includes receiving a command from a first coordinator to perform a first operation, the first coordinator being selected from the plurality of endpoint devices comprising the first endpoint device. The method further includes performing the first operation in response to receiving the command to extract a first feature from the first sensor data.The procedure further involves transmitting an initial message containing data related to the first feature to the multitude of endpoint devices.

[0011] Other embodiments include, among others, a first endpoint device or other system that implements one or more aspects of the disclosed techniques, and a computer-readable medium that contains instructions for performing one or more aspects of the disclosed techniques.

[0012] At least one technical advantage of the disclosed techniques compared to the prior art is that the endpoint devices in a distributed system transmit less sensor data compared to conventional approaches. Instead of transmitting raw sensor data to a remote data aggregator or cloud server, each endpoint device extracts one or more features represented by the sensor data. Each endpoint device then transmits messages containing only data related to the extracted features to the other endpoint devices in the local communication network. As a result, the amount of data transmitted over the communication network can be significantly reduced. Likewise, the processing power required to handle the messages can be significantly lower than the processing power required to handle the raw sensor data.Furthermore, the endpoint device, designated as the coordinator, determines the type of analysis that the endpoint devices perform at any given time, resulting in reduced processing requirements for the endpoint devices. Consequently, the latency of the distributed system is shorter and its performance is higher compared to traditional approaches.

[0013] A further advantage of the disclosed techniques is that if the endpoint device designated as the coordinator fails, the remaining endpoint devices automatically select a new coordinator from among themselves. As a result, the distributed system is more stable with respect to the failure of an endpoint device, leading to a more robust distributed system compared to previous approaches. These technical advantages represent one or more technological advances over prior art approaches. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] To enable a detailed understanding of the aforementioned features of the various embodiments, a more precise description of the inventive concepts, which were briefly summarized above, can be given by reference to various embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings only illustrate typical embodiments of the inventive concepts and are therefore not to be considered limiting in any way, and that other equally effective embodiments exist. Fig. Figure 1 illustrates a distributed system configured to implement one or more aspects of the present disclosure; Fig. Figure 2 is a more detailed illustration of the distributed message handler device of the Fig. 1 according to various embodiments; Fig. Figure 3 illustrates a data flow diagram for selecting a distributed message handler device as an initial coordinator according to various embodiments; Fig. Figure 4 illustrates a data flow diagram for distributing event messages between a set of distributed message handler devices according to different embodiments; Fig. Figure 5 illustrates a data flow diagram for selecting a distributed message handler device as a subsequent coordinator according to various embodiments; Fig. 6A-6B present a flowchart of procedural steps for managing a set of distributed message-handler devices included in an Internet of Things architecture according to various embodiments; and Fig. Figure 7 is a flowchart for process steps for selecting a subsequent coordinator in an Internet of Things architecture according to different implementations. DETAILED DESCRIPTION

[0015] The following description sets out numerous specific details to enable a more thorough understanding of certain specific embodiments. However, it is evident to those skilled in the art that other embodiments may be implemented in practice without one or more of these specific details or without additional specific details.

[0016] As further described herein, several endpoint devices, such as message-handler devices of the apparatus, are equipped with sensors to acquire sensor data. The endpoint devices communicate with each other over a distributed domain, referred to herein as a "private communication network," to transmit, receive, and process sensor-related data in a scalable manner. The endpoint devices communicate with each other by passing data as messages or objects over the private communication network, with the messages or objects being agnostic to the type of endpoint device or corresponding sensor. The set of endpoint devices is self-contained and does not require a dedicated central server to process the sensor data.More precisely, the endpoint devices form a private communication network, with each endpoint device connected to the private communication network capable of acting as a coordinator for the analysis and / or detection performed by the endpoint devices. Furthermore, if the current coordinator fails or otherwise ceases communication with the private communication network, the remaining endpoint devices can select one of the remaining endpoint devices as the successor coordinator. The successor coordinator then assumes the activities of the previous coordinator without interrupting service. System overview

[0017] In the descriptions of the Fig. Sections 1-7 describe embodiments of the distributed system in general terms with regard to a camera-based surveillance system in the context of a conference room. However, it is understood that the various embodiments of the distributed system can be used in other environments of interest, such as at home, in an office, in an industrial setting, and / or the like, which may include sensors other than the camera-based sensors or additional sensors to them.

[0018] Fig. Figure 1 illustrates a distributed system 100 configured to implement one or more aspects of this disclosure. The distributed system 100 may be an Internet of Things (IoT) architecture, a distributed data architecture, and / or the like. As shown, the distributed system 100 includes, among other things, distributed message-control program or message-handler (DMH) devices 102(1), 102(2), 102(3), and 102(4), a gateway device 104, and an external device 106 communicating with each other via communication networks 130 and 132. The distributed message-handler devices 102(1), 102(2), 102(3), and 102(4) are collectively referred to herein as distributed message-handler devices 102.Communication networks 130 and 132 can be any suitable environment to enable communication between remote and local computer systems and computing devices, including, but not limited to, Bluetooth communication channels, wireless and wired local area networks (LANs), internet-based wide area networks (WANs), mobile networks and / or the like.

[0019] Each of the distributed message-handler devices 102 includes, among other things, a computing device, which may be an IoT device, a PC, a personal digital assistant, a tablet computer, a mobile phone, a mobile device, or any other device suitable for implementing one or more aspects of the present invention. For illustration, the distributed message-handler devices 102(1), 102(2), 102(3), and 102(4) are connected to the communication network 130 via communication links 112, 114, 116, and 118, respectively.

[0020] In operation, the distributed message handler devices 102, which offer a specific service, form a private communication network and exchange service-related data in a scalable manner. The distributed message handler devices 102 constitute the private communication network as a subset of the communication network 130. The distributed message handler devices 102 execute a process to select one of the distributed message handler devices 102 as the coordinator. Once selected, the coordinator transmits trigger events to the distributed message handler devices 102 to control the type of analysis and recognition that each of the distributed message handler devices 102 performs within the private communication network. This type of analysis and recognition includes, but is not limited to, gesture recognition, object recognition, location recognition, and / or similar activities.In response, each distributed message handler 102 receives sensor data from the corresponding sensors, performs analysis and / or recognition on the sensor data, and extracts certain features from the sensor data according to the trigger events received by the coordinator. The distributed message handler devices 102 that recognize one or more features transmit messages to all distributed message handler devices 102, including the coordinator. The messages contain data that identifies the recognized features.

[0021] The distributed message handler devices 102, other than the coordinator (referred to herein as non-coordinators), store the received messages in a data store without performing any action. The coordinator also stores the received messages, performs an analysis on them, and performs one or more actions in response. These actions include, but are not limited to, transmitting trigger events to the distributed message handler devices 102 to start, stop, or modify the analysis and / or recognition performed by them.The response actions also include, among other things, transmitting messages to one or more distributed message-handler devices 102 to change a configuration, operating mode, or other aspect of the distributed message-handler devices 102, such as the zoom level or pan direction of a camera associated with a distributed message-handler device 102. The response actions also include, among other things, transmitting messages to one or more external devices, such as the external device 106, to change a configuration, operating mode, or other aspect of the external devices.

[0022] Since each distributed message handler device 102 has previously recorded messages transmitted by the other distributed message handler devices 102, all distributed message handler devices 102 are capable of assuming the role of coordinator. If the coordinator ceases communication with the distributed message handler devices 102, for example, if the coordinator experiences a system failure, the remaining distributed message handler devices 102 select one of the remaining distributed message handler devices 102 to be the successor coordinator. The successor coordinator retrieves the previously stored messages and begins performing the operations associated with the coordinator. As a result, the remaining distributed message handler devices 102 continue to provide service even if the coordinator fails.

[0023] The gateway device 104 includes, among other things, a computing device that can be a router, a switch, a gateway, a firewall, an access point, or any other device suitable for implementing one or more aspects of the present disclosure. For illustration, the gateway device 104 communicates with the distributed message-handler devices 102 via the communication network 130 using the communication link 120. Likewise, the gateway device 104 communicates with the external device 106 via the communication network 132 using the communication link 122.

[0024] In operation, the gateway device 104 provides a connection between the distributed message handler devices 102 and the external device 106. As described herein, the distributed message handler device 102, selected as the coordinator, performs response actions based on messages received by the distributed message handler devices 102. These response actions may include transmitting messages to the external device 106 to change a configuration, operating mode, or other aspect of the external device 106. In such cases, the coordinator transmits a message received by the gateway device 104 over the communication network 130. The gateway device 104 then transmits the message to the external device 106 over the communication network 132.Similarly, the external device 106 can transmit messages to the coordinator, with the messages containing status information, configuration data, and the like. In such cases, the external device 106 transmits a message that is received by the gateway device 104 via the communication network 132. The gateway device 104, in turn, transmits the message to the coordinator via the communication network 130.

[0025] The external device 106 includes, among other things, a computing device, which may be a single server, a cluster or a "farm" of servers, one or more network applications, or any other device suitable for implementing one or more aspects of the present disclosure. For illustration, the external device 106 communicates via the communication network 132 using the communication link 122.

[0026] During operation, the external device 106 receives messages from the distributed message handler device 102, which has been selected as the coordinator. The external device can be any device that can be controlled via a communication network, such as communication networks 130 and 132. The messages instruct the external device 106 to perform various functions. For example, the external device 106 can be a highly directional loudspeaker (HDL). Such a highly directional loudspeaker can be configured to produce a directional sound beam with a specific beam width and direction, referred to here as the optimal point or "sweet spot." The coordinator can transmit messages to the highly directional loudspeaker to increase or decrease the beam width.Furthermore, the coordinator can transmit messages to the loudspeaker with high directional power to change the direction of the sound beam. In another example, the external device 106 can be a microphone. The coordinator can transmit messages to the microphone to turn it on or off. Furthermore, the coordinator can transmit messages to the microphone to increase or decrease its sensitivity.

[0027] The external device 106 can also transmit messages to the distributed message handler device 102, which has been selected as the coordinator. These messages can include confirmation that the external device 106 has executed a command contained in the previous message. Furthermore, these messages can contain status information regarding the external device 106.

[0028] It will be understood that the system shown herein is for illustrative purposes only and that variations and modifications are possible. For example, within the scope of this disclosure, the distributed system 100 can include any technically feasible number and / or type of distributed message-handler devices 102, gateway devices 104, and external devices 106. In another example, the distributed message-handler devices 102 can have different capabilities for the same service. In such cases, each of the distributed message-handler devices 102 can provide a basic set of capabilities for the service, while a subset of the distributed message-handler devices 102 can provide one or more extended capabilities.In yet another example, the distributed message handler devices 102, the gateway device 104, and the external device 106 are illustrated with a specific communication network configuration that includes communication networks 130 and 132. However, the distributed message handler devices 102, the gateway device 104, and the external device 106 can be configured in any technically feasible way and can communicate over any number of communication networks. In this respect, the distributed message handler devices 102 and the external device 106 can be connected to the same communication network. In such cases, the message handler devices 102 can communicate with the external device 106 without transmitting messages via the gateway device 104.The various functions and features of the distributed message handler devices 102 will now be described. Processes of the distributed system

[0029] Fig. Figure 2 is a more detailed illustration of the distributed message handler device 102. Fig. 1 according to various embodiments. As shown, the distributed message handler device 102 includes, among other things, a processor 202, a memory 204, an I / O device interface 206, a network interface 208, an intermediate connection 210 and a system memory 212.

[0030] The processor 202 retrieves and executes programming instructions stored in system memory 212. Likewise, the processor 202 stores and retrieves application data stored in system memory 212. The intermediate connection 210 enables the transfer of data, such as programming instructions and application data, between the processor 202, the I / O device interface 206, the memory 204, the network interface 208, and the system memory 212. The I / O device interface 206 is configured to receive input data from and transmit data to I / O devices 222. Examples of I / O devices 222 include one or more buttons, a keyboard, a mouse, or other pointing devices, and / or the like.The I / O device interface 206 can also include an audio output unit configured to generate an electrical audio output signal, and the I / O devices 222 can further include a loudspeaker configured to generate an audible output in response to the electrical audio output signal. Another example of an I / O device 222 is a display device, which is generally any technically feasible device for generating an image for display. For example, the display device could be a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, or a digital light processing (DLP) display. The display device could be a television that includes a transmitter or cable tuner for receiving digital or analog television signals. The display device could be included in a VR / AR headset or a heads-up display (HUD).Furthermore, the display device can project an image onto one or more surfaces, such as walls, projection screens, or a vehicle's windshield. Additionally or alternatively, an image can be projected directly onto a user's eyes (e.g., via retinal projection).

[0031] The I / O devices 222 can further include various sensors that generate sensor data corresponding to one or more objects within a currently monitored area. For example, the sensors can include, but are not limited to, visual sensors (e.g., RGB cameras, infrared cameras, etc.), distance measurement sensors (e.g., LiDAR, radar), acoustic sensors (e.g., microphones), and / or environmental sensors (e.g., thermometers, etc.). Data from these sensors can be aggregated and analyzed together to generate meaningful information, alerts, recommendations, and so on.

[0032] The processor 202 is representative of a single central processing unit (CPU), multiple CPUs, a single CPU with multiple processor cores, and the like. System memory 212 is generally representative of random-access memory. Memory 204 can be a storage device with a disk drive. Although shown as a single unit, memory 204 can be a combination of fixed and / or removable storage devices, such as hard disk drives, floppy disk drives, tape drives, removable memory cards or optical storage, network-attached storage (NAS), or a storage area network (SAN).The processor 202 communicates with other computing devices and systems via the network interface 208, the network interface 208 being configured to transmit and receive data over a communication network.

[0033] The system memory 212 includes, among other things, an operating system 232, a module 234 of the distributed message handler (DMH), and a data memory 242. When executed by the processor 202, the operating system 232 and the distributed message handler module 234 perform one or more operations in connection with the distributed message handler device 102. Fig. 1, as further described herein. When performing the operations in connection with the distributed message handler device 102, the operating system 232 and the distributed message handler module 234 can store data in and retrieve data from the data store 242.

[0034] During operation, the operating system 232 provides various support functions for the distributed message handler device 102. The processor 202 is generally under the control of the operating system 232. Examples of operating systems include the UNIX operating system, versions of the Microsoft Windows operating system, and distributions of the Linux operating system. (UNIX is a registered trademark of The Open Group in the United States and other countries. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.) More generally, any operating system that supports the functions disclosed herein may be used.

[0035] During operation, the distributed message handler module 234 exchanges data with the distributed message handler device 102 in the form of messages. The messages can be formatted as shown below: DMH service Message type {(Sender: <kennung>) (Data: <werte>)}

[0036] "DMH Service" is an identifier that specifies the service associated with the message, such as "Conference Room Service." "Message Type" is an identifier that specifies the nature of the message. The message type could be a trigger event to instruct the Distributed Message Handler 102 to detect the presence of people or other objects, detect the positions of people or objects, or detect gestures made by a person. The message type could be a message containing data regarding detected people or other objects, the positions of such people or objects, or gestures made by a person. The "Sender" field contains an identifier that specifies the Distributed Message Handler 102 transmitting the message. The "Data" field contains corresponding data values ​​based on the message type.The data can be in any technically feasible format suitable for transmitting messages over a communication network. For example, the message data can be formatted in JavaScript Object Notation (JSON). Additionally or alternatively, the messages can be in any technically feasible format. Generally, the disclosed messages are significantly shorter than the original sensor data. Since the message format is standardized across all distributed message handler devices, these devices are also able to process the messages regardless of the specific format of the original sensor data.

[0037] In some embodiments, the data in a message can have multiple parts and can be nested in any technically feasible way. For example, the message can be an object detection message. A first section of the data can identify that two people and three chairs have been detected. A second section of the data can list each individual person, with the data for each person including certain characteristics of the person, such as hair color, whether the person is wearing glasses, and whether the person is sitting or standing. A third section of the data can list each individual chair, with the data for each chair including certain characteristics of the chair, such as the type of chair, the number of legs, and whether the chair is adjustable.

[0038] The distributed message handler module 234 begins execution when the distributed message handler device 102 is powered on, when the distributed message handler device 102 completes a boot sequence, when the operating system 232 starts the distributed message handler module 234, or similar events. The distributed message handler module 234 transmits a selection message in the form of a sent message to all devices communicating with the communication network 130. The sent message specifies or "announces" a particular service that the message handler device 102 is configured to provide. For example, the service might be a conference room service that tracks people or other objects in a conference room over a full 360-degree field of view.The service can adjust the beamwidth and sweet spot for one or more highly directional loudspeakers to direct certain audio signals toward specific individuals in the conference room. The service can activate microphones in areas of the conference room where people are detected, deactivate microphones in areas where no people are detected, and adjust microphone sensitivity based on the distance between the microphones and the people in the conference room. Additionally or alternatively, the service could track people or objects over a narrower field of view, such as 90 degrees, 180 degrees, or 270 degrees. Additionally or alternatively, the service could track people or objects in an open space, such as an auditorium or arena.

[0039] The distributed message handler module 234 receives messages transmitted by other distributed message handler devices 102 that advertise the same service as the distributed message handler module 234. The distributed message handler module 234 then determines which distributed message handler device 102 offering the service should be selected as the coordinator for the distributed message handler devices 102. The distributed message handler device 102 to be the coordinator can be selected using any technically feasible technique. These techniques include, but are not limited to, the distributed message handler device 102 with the longest uptime since being powered on, the highest or lowest media access control (MAC) address, the highest or lowest Internet Protocol (IP) address, and / or similar criteria. These techniques could be used in combination.For example, the distributed message handler 102 with the longest uptime could be selected as the coordinator. Using this technique, it's possible that two distributed message handlers 102 might have exactly the same uptime, while the uptimes for all other distributed message handlers 102 are shorter. To resolve a tie, the distributed message handler 102 with the higher MAC address could be selected as the coordinator. Since all distributed message handlers 102 offering the same service use the same technique to select a coordinator, they will all choose the same coordinator.

[0040] The distributed message handler module 234 forms a private communication network with other distributed message handler devices 102 that provide the same service. The distributed message handler module 234 communicates with these other distributed message handler devices 102 via one-to-many messages, also referred to here as group or multicast messages. Through multicast notification, the distributed message handler module 234 transmits multicast messages that are received by the other distributed message handler devices 102 located within the private communication network. Devices outside the private communication network do not receive the multicast messages.

[0041] When the distributed message handler module 234 is running on the distributed message handler device 102, which has been selected as the coordinator, the distributed message handler module 234 transmits messages to control the activities of the distributed message handler devices 102 within the private communication network. More specifically, the distributed message handler module 234 transmits a multipoint message in the form of a trigger event to instruct the distributed message handler devices 102 to perform one or more types of analysis, detection, and / or other appropriate actions on sensor data received from one or more sensors of the respective distributed message handler device 102. For example, the distributed message handler module 234 transmits a trigger event to instruct the distributed message handler devices 102 to perform gesture detection, object detection, or position detection.

[0042] The distributed message handler module 234 receives sensor data from one or more sensors connected via the I / O device interface 206, such as a still image or video camera. The distributed message handler module 234 extracts various features from the sensor data. For example, it can detect one or more people or other objects, such as tables or chairs, present in the camera image. Furthermore, it can detect additional features of a detected object, such as the number of legs on a detected chair or whether a detected chair is adjustable. The distributed message handler module 234 extracts features that identify the number and type of detected objects.Similarly, the distributed message handler module 234 can detect the position of one or more people or other objects, such as chairs or tables, relative to the camera. The distributed message handler module 234 extracts features that identify the position of the detected objects. The distributed message handler module 234 can detect that a person in the camera image is performing a specific gesture. The distributed message handler module 234 extracts a feature that identifies the gesture performed by the person. The distributed message handler module 234 then formats the extracted features into one or more messages and transmits the messages to the distributed message handler devices 102 in the private communication network.

[0043] When the distributed message handler module 234 is running on the distributed message handler device 102 selected as the coordinator, the distributed message handler module 234 receives messages from one or more distributed message handler devices 102 in the private communication network and stores the received messages in the data store 242. If the distributed message handler module 234 anticipates that multiple distributed message handler devices 102 might be transmitting messages, it waits a specified duration to allow all distributed message handler devices 102 sufficient time to acquire the sensor data, such as a single image, process the sensor data, and transmit the messages that identify features extracted from the sensor data.After receiving messages from one or more distributed message handler devices 102, the distributed message handler module 234 processes the received messages to determine one or more actions in response. For example, the distributed message handler module 234 can determine that the distributed message handler devices 102 should stop performing a current analysis mode, detection, or other action and begin performing a different analysis mode, detection, or other action. In response, the distributed message handler module 234 can transmit one or more messages to instruct the distributed message handler devices 102 to stop performing a current analysis mode, detection, or other action and begin performing a different analysis mode, detection, or other action.The distributed message handler module 234 can determine that certain actions should be performed by one or more external devices, such as the external device 106. The distributed message handler module 234 can transmit one or more messages to a set of high-frequency loudspeakers. The messages can instruct the high-frequency loudspeakers to increase or decrease a lobe width or to change the direction of a lobe emitted by the high-frequency loudspeakers. Likewise, the distributed message handler module 234 can transmit one or more messages to a set of microphones. The messages can instruct the high-frequency loudspeakers to increase or decrease the sensitivity of the microphones or to turn certain microphones on or off.

[0044] When the distributed message handler module 234 is running on the distributed message handler device 102 that has been selected as the coordinator, the distributed message handler module 234 also periodically transmits so-called heartbeat messages to inform the other distributed message handler devices 102 in the private communication network that the coordinator is still in operation.

[0045] When the distributed message handler module 234 is running on the distributed message handler device 102 that is not selected as the coordinator, the distributed message handler module 234 receives messages from one or more distributed message handler devices 102 in the private communication network and stores the received messages in the data store 242. The distributed message handler module 234 does not parse the received messages or transmit any messages in response. Instead, the distributed message handler module 234 manages the storage of the messages in case the current coordinator ceases communication with the private communication network.

[0046] In this regard, if it does not receive any heartbeat messages from the coordinator for a threshold period, the distributed message handler module 234 transmits a selection message to initiate a further coordinator selection process. Likewise, all other distributed message handler devices 102 in the private communication network that remain operational also transmit a selection message. The distributed message handler module 234 then performs a selection process together with the other distributed message handler devices 102 to choose a successor coordinator. Because the distributed message handler module 234 and each of the other distributed message handler devices 102 maintain separate data stores of received messages, each distributed message handler device 102 is able to assume the role of successor coordinator without interrupting service.

[0047] The exchange of messages between the distributed message handler devices 102 and the external device 106 is now described.

[0048] Fig. Figure 3 illustrates a data flow diagram 300 for selecting a distributed message handler device as an initial coordinator according to various embodiments. The data flow diagram 300 can be implemented using the distributed message handler module 234, which is based on a set of distributed message handler devices 102. Fig. 2 is executed, is implemented. Data flow diagram 300 illustrates messages exchanged between distributed message handler devices 102(1), 102(2), 102(3), and 102(4) and the external device 106. The messages are transferred chronologically from the top of data flow diagram 300 to the bottom of data flow diagram 300.

[0049] Each of the distributed message handler devices 102(1), 102(2), 102(3), and 102(4) begins execution when distributed message handler device 102 is powered on, when distributed message handler device 102 completes a boot sequence, when the operating system 232 starts distributed message handler module 234, or the like. Distributed message handler device 102(3) transmits a selection message 302 as a disseminated message to all devices within the communication network, including, but not limited to, distributed message handler devices 102(1), 102(2), and 102(4). The selection message 302 may include an uptime, which specifies the duration for which distributed message handler device 102(3) was powered on. Additionally or alternatively, the selection message 302 can contain a MAC address associated with the distributed message handler device 102(3).Additionally or alternatively, the 302 selection message can contain an IP address associated with the distributed message handler device 102(3). Additionally or alternatively, the 302 selection message can contain any other data associated with the distributed message handler device 102(3) that is relevant for selecting a coordinator. The 302 selection message can be formatted as follows: Conference Room Service Selection Message {(Sender: DMH Device 102(3)) (Data: Uptime:. <wert>MAC Address: <wert>, IP address: <wert>)}

[0050] The selection message 302 indicates that the DMH service is a conference room service and that the message type is a selection message. The selection message 302 also indicates that the sender is the distributed message handler device 102(3) and that the data is uptime, MAC address, and IP address associated with the distributed message handler device 102(3).

[0051] Similarly, the distributed message handler devices 102(1), 102(4), and 102(2) transmit selection messages 304, 306, and 308, respectively. The format and content of selection messages 304, 306, and 308 are functionally identical to selection message 302. After all distributed message handler devices 102 have transmitted the selection messages, each device determines which other distributed message handler device 102 is to be selected as the coordinator. Since all distributed message handler devices 102 select the coordinator using the same technique, they all select the same distributed message handler device 102 as the coordinator. As shown, distributed message handler device 102(3) is selected as the coordinator.Accordingly, the distributed message handler device 102(3) transmits a trigger event message 310 to instruct all distributed message handler devices 102(1), 102(2), 102(3), and 102(4) to perform a specific type of analysis, detection, or other action. For example, the distributed message handler device 102(3) can transmit a trigger event message 310 to instruct the distributed message handler devices 102(1), 102(2), 102(3), and 102(4) to perform gesture detection in order to extract a feature from the sensor data that corresponds to a specific gesture of interest. The trigger event message 310 can be formatted as follows: Conference Room Service Gesture Recognition Trigger {(Sender: DMH Coordinator Device 102(3)) (Data: Perform Gesture Recognition: . <ein>)}

[0052] Trigger event message 310 indicates that the DMH service is a conference room service and that the message type is a gesture recognition trigger message. Trigger event message 310 indicates that the sender is the distributed message handler device 102(3) and that the distributed message handler device 102(3) is the current coordinator. Trigger event message 310 instructs the distributed message handler devices 102 to perform gesture recognition.

[0053] Each of the distributed message handler devices 102(1), 102(2), 102(3), and 102(4) stores the trigger event message 310 in a local data store. The distributed message handler devices 102(1), 102(2), 102(3), and 102(4) then exchange additional messages associated with the service, as described below.

[0054] Fig. Figure 4 illustrates a data flow diagram 400 for distributing event messages between a set of distributed message handler devices according to various embodiments. The data flow diagram 400 can be implemented by the distributed message handler module 234, which is based on a set of distributed message handler devices 102. Fig. 2 is executed, is implemented. Data flow diagram 400 illustrates messages exchanged between distributed message handler devices 102(1), 102(2), 102(3), and 102(4) and the external device 106. The messages are transferred chronologically from the top of data flow diagram 400 to the bottom of data flow diagram 400.

[0055] As in connection with Fig. As described in Section 3, the distributed message handler devices 102(1), 102(2), 102(3), and 102(4) perform a specific type of analysis, detection, or other action as specified by the trigger event message 310. As described in Fig. As shown in Figure 4, the distributed message handler 102(1) can transmit an extraction result message 402 containing data about one or more features that the distributed message handler 102(1) extracts from the image data. The distributed message handler 102(1) can transmit an extraction result message 402 to the distributed message handler devices 102(2), 102(3), and 102(4). Each of the distributed message handler devices 102(1), 102(2), 102(3), and 102(4) stores the extraction result message 402 in a local data store. The extraction result message 402 can contain data about a gesture feature that the distributed message handler 102(1) detects in an image. The extraction result message 402 can be formatted as follows: Conference Room Service Extraction Result Gesture Recognition {(Sender: DMH Device 102(1)) (Data: Gesture Recognition: Gesture A Detected)}

[0056] Extraction result message 402 indicates that the DMH service is a conference room service and that the message type is a gesture recognition extraction result. Extraction result message 402 indicates that the sender is the distributed message handler device 102(1). Extraction result message 402 indicates that the distributed message handler device 102(1) has recognized a gesture of type A.

[0057] In response, distributed message handler devices 102(1), 102(2), 102(3), and 102(4) store the extraction result message 402. Distributed message handler device 102(3) determines that the gesture A identified by extraction result message 402 is a gesture to perform object and / or location detection. Therefore, distributed message handler device 102(3) transmits a trigger event message 404 to instruct all distributed message handler devices 102(1), 102(2), 102(3), and 102(4) to perform a specific type of analysis, detection, or other action.For example, the distributed message handler device 102(3) can transmit a trigger event message 404 to instruct the distributed message handler devices 102(1), 102(2), 102(3), and 102(4) to perform object and / or position detection to extract a feature from the sensor data that corresponds to a specific object of interest. The trigger event message 404 can be formatted as follows: Conference Room Service Object / Position Detection Trigger {(Sender: DMH Coordinator Device 102(3)) (Data: Perform Object / Position Detection: . <ein>)}

[0058] The trigger event message 404 indicates that the DMH service is a conference room service and that the message type is an object / position detection trigger message. Trigger event message 404 indicates that the sender is distributed message handler device 102(3) and that distributed message handler device 102(3) is the current coordinator. Trigger event message 404 instructs distributed message handler devices 102 to perform object and / or position detection. Each of the distributed message handler devices 102(1), 102(2), 102(3), and 102(4) stores trigger event message 404 in a local data store.

[0059] Distributed message handler 102(4) can transmit an extraction result message 402 containing data about one or more features that distributed message handler 102(1) extracts from the image data. Distributed message handler 102(4) can transmit an extraction result message 402 to distributed message handler 102(1), 102(2), and 102(3). Each of these distributed message handler 102(1), 102(2), 102(3), and 102(4) stores the extraction result message 406 in a local data store. The extraction result message 406 can contain data about an object feature that distributed message handler 102(1) detects in an image. Extraction result message 406 can be formatted as follows: Conference Room Service Extraction Result Object / Viewpoint Detection {(Sender: DMH Device 102(4)) (Data: Object Detection: <person>, Object viewpoint: <wert>)}

[0060] Extraction result message 406 indicates that the DMH service is a conference room service and that the message type is an object / position detection extraction result. Extraction result message 406 indicates that the sender is the distributed message handler device 102(4). Extraction result message 406 indicates that the distributed message handler device 102(4) has detected an object of interest and specifies the object's position relative to the distributed message handler device 102(4).

[0061] In response, distributed message handler devices 102(1), 102(2), 102(3), and 102(4) store the extraction result message 406. Distributed message handler device 102(3) combines data from the extraction result message 406 with extraction result messages received from other distributed message handler devices 102 to determine the location of a person present in the room. In response, distributed message handler device 102(3) transmits a control message 408 for the external device to the external device 106.

[0062] In one example, the external device 106 can be a high-power loudspeaker. The distributed message handler device 102(3) can transmit the control message 408 for the external device to the high-power loudspeaker to adjust the lobe width and direction of an audio signal so that the detected person can hear the audio signal while other people cannot. In this example, the control message 408 for the external device can be formatted as follows: Control message for external device (Data: Lobe width: <wert>, Direction: <wert>)

[0063] Control message 408 for the external device indicates that the message type is a control message for the external device. Control message 408 for the external device instructs the external device 106 to set the lobe width and direction of an audio signal to specific values.

[0064] In another example, the external device 106 can be a microphone. If the microphone is in an area of ​​the room where no person is detected, then the distributed message handler device 102(3) can transmit the external device control message 408 to the microphone to turn the microphone off. Conversely, if the microphone is in an area of ​​the room where a person is detected, then the distributed message handler device 102(3) can transmit the external device control message 408 to the microphone to turn the microphone on. In the latter case, the external device control message 408 can further adjust the microphone's sensitivity based on the distance between the microphone and the detected person. The external device control message 408 can be formatted as follows: External Device Control Message (Data: Microphone: <ein>, Sensitivity: <wert>)

[0065] Control message 408 for the external device indicates that the message type is a control message for the external device. Control message 408 for the external device commands the external device 106 to turn on the microphone and set the microphone sensitivity to a specific value.

[0066] Subsequently, the distributed message handler 102(2) can transmit an extraction result message 410 containing data regarding one or more features that the distributed message handler 102(2) extracts from the image data. The distributed message handler 102(2) can transmit an extraction result message 410 to the distributed message handler devices 102(1), 102(3), and 102(4). Each of the distributed message handler devices 102(1), 102(2), 102(3), and 102(4) stores the extraction result message 410 in a local data store. The extraction result message 410 can contain data regarding a gesture feature that the distributed message handler 102(2) detects in an image. Extraction result message 410 can be formatted as follows: Conference Room Service Extraction Result Gesture Recognition {(Sender: DMH Device 102(2)) (Data: Gesture Recognition: Gesture B Detected)}

[0067] Extraction result message 410 indicates that the DMH service is a conference room service and that the message type is a gesture recognition extraction result. Extraction result message 410 indicates that the sender is the distributed message handler device 102(2). Extraction result message 410 indicates that the distributed message handler device 102(2) has recognized a gesture of interest B.

[0068] In response, distributed message handler devices 102(1), 102(2), 102(3), and 102(4) store the extraction result message 410. Distributed message handler device 102(3) determines that the gesture identified by extraction result message 410 is a gesture to stop performing object and / or location detection. Therefore, distributed message handler device 102(3) transmits a trigger event message 412 to instruct all distributed message handler devices 102(1), 102(2), 102(3), and 102(4) to stop performing a specific type of analysis, detection, or other action. For example, the distributed message handler device 102(3) can transmit a trigger event message 412 to instruct the distributed message handler devices 102(1), 102(2), 102(3) and 102(4) to stop performing object and / or location detection.The trigger event message 412 can be formatted as follows: Conference Room Service Object / Position Detection Trigger {(Sender: DMH Coordinator Device 102(3)) (Data: Perform Object / Position Detection: . <aus>)}

[0069] Trigger event message 412 indicates that the DMH service is a conference room service and that the message type is an object / position detection trigger message. Trigger event message 412 indicates that the sender is distributed message handler device 102(3) and that distributed message handler device 102(3) is the current coordinator. Trigger event message 412 commands distributed message handler devices 102 to stop performing object and / or position detection. Each of the distributed message handler devices 102(1), 102(2), 102(3), and 102(4) stores trigger event message 404 in a local data store. The distributed message handler devices 102(1), 102(2), 102(3) and 102(4) then stop performing object and / or position detection.

[0070] Fig. Figure 5 illustrates a data flow diagram 500 for selecting a distributed message handler device as a subsequent coordinator according to various embodiments. The data flow diagram 500 can be implemented using the distributed message handler module 234, which is based on a set of distributed message handler devices 102. Fig. 2 is executed, is implemented. Data flow diagram 500 illustrates messages exchanged between distributed message handler devices 102(1), 102(2), 102(3), and 102(4) and the external device 106. The messages are transferred chronologically from the top of data flow diagram 500 to the bottom of data flow diagram 500.

[0071] As shown, the distributed message handler device 102(3) is the current coordinator. Because the distributed message handler device 102(3) is the current coordinator, it transmits heartbeat messages, such as a heartbeat message 502, at periodic intervals. The heartbeat message 502 can be formatted as follows: Conference Room Service Heartbeat Message {(Sender: DMH Coordinator Device 102(3)) (Data: Heartbeat: <ein>)}

[0072] Heartbeat message 502 indicates that the DMH service is a conference room service and that the message type is a heartbeat message. Heartbeat message 502 indicates that the sender is distributed message handler device 102(3) and that distributed message handler device 102(3) is the current coordinator. Sometimes, after transmitting heartbeat message 502, distributed message handler device 102(3) ceases communication with the other distributed message handler devices 102. Subsequently, distributed message handler devices 102(1), 102(2), and 102(4) determine that distributed message handler device 102(3) will no longer transmit heartbeat messages. The distributed message handler devices 102(1), 102(2) and 102(4) can determine that the distributed message handler device 102(3) has not transmitted a heartbeat message for a specified period of time.Additionally or alternatively, the distributed message handler devices 102(1), 102(2) and 102(4) can determine that the distributed message handler device 102(3) was unable to transmit a heartbeat message for a specified number of times.

[0073] After determining that distributed message handler device 102(3) is no longer transmitting heartbeat messages, distributed message handler devices 102(1), 102(2), and 102(4) initiate a selection process to choose a subsequent coordinator. Distributed message handler devices 102(1), 102(4), and 102(2) transmit selection messages 504, 506, and 508, respectively. The format and content of selection messages 504, 506, and 508 are functionally identical to selection messages 302, 304, 306, and 308. Distributed message handler devices 102(1), 102(4), and 102(2) perform a selection process, as described herein, to choose a subsequent coordinator. As shown, the distributed message handler device 102(1) is selected as the coordinator.Accordingly, distributed message handler device 102(1) transmits a trigger event 510 to instruct all remaining distributed message handler devices 102(1), 102(2), and 102(4) to perform a specific type of analysis, detection, or other action. Distributed message handler devices 102(1), 102(2), and 102(4) then continue to provide the conference room service without interruption.

[0074] It will be understood that the data flow diagrams shown herein are for illustrative purposes only and that variations and modifications are possible. In this respect, the various messages can be transmitted in any technically feasible chronological order. For example, the distributed message handler devices 102(3), 102(1), 102(4), and 102(2), as shown in Fig. Figure 3 shows that the selection messages 302, 304, 306, and 308 can be transmitted in any technically feasible chronological order. Likewise, the distributed message handler devices 102(1), 102(4), and 102(2), as shown in Fig. Figure 5 shows that selection messages 504, 506, and 508 are transmitted in any technically feasible chronological order. In another example, as shown in Fig. As shown in Figure 4, the distributed message handler 102(3) can transmit the control message 408 for the external device to the external device 106 after the distributed message handler 102(2) has transmitted the extraction result message 410 to the distributed message handler devices 102(1), 102(3) and 102(4) and after the distributed message handler 102(3) has transmitted the trigger event message 412 to the distributed message handler devices 102(1), 102(2), 102(3) and 102(4).

[0075] The Fig. Sections 6A-6B present a flowchart of procedural steps for managing a set of distributed message-handler devices included in an Internet of Things architecture, according to various embodiments. Even though the procedural steps are related to the systems of Fig. From the descriptions in Figures 1-5, the person skilled in the art understands that any system configured to perform the process steps in any order is within the scope of this disclosure.

[0076] As shown, a procedure 600 begins at step 602, where an application program running on multiple endpoint devices initiates the operation. The application program can be the distributed message handler module 234 running on one of the distributed message handler devices 102 of the Fig. 1 is executed. A distributed message handler device 102 can initiate the process when the distributed message handler device 102 is turned on, when the distributed message handler device 102 completes a boot sequence, when the operating system 232 starts the distributed message handler module 234, or the like.

[0077] At step 604, the distributed message handler devices 102 announce their ability to offer an initial service. The initial service can be a conference room service that tracks people or other objects in a conference room over a full 360-degree field of view. The distributed message handler devices 102 announce the initial service by sending a message to all devices in the communications network.

[0078] In step 606, the distributed message handler devices 102 that offer the first service form a private communication network. More precisely, each distributed message handler device 102 identifies the set of distributed message handler devices 102 that announce the first service. The distributed message handler devices 102 then transmit messages to each other via multicast messages, which are sent to the distributed message handler devices 102 in the private communication network that offer the first service, but are not sent to other devices. The messages are functionally similar to messages 302, 304, 306, and 308 of the Fig. 3.

[0079] In step 608, the distributed message handler devices 102 select an initial coordinator. The distributed message handler device 102 selected as the initial coordinator can be the distributed message handler device 102 with the longest uptime since power-on, the highest or lowest Media Access Control (MAC) address, the highest or lowest Internet Protocol (IP) address, and so on.

[0080] In step 610, the coordinator specifies a type of analysis or recognition for the distributed message handler devices 102 to perform. This type of analysis or recognition includes, among other things, gesture recognition, object recognition, or position recognition. In step 612, the coordinator transmits a trigger event to the distributed message handler devices 102 to extract certain features from the sensor data available to the distributed message handler devices 102. The trigger event is functionally similar to trigger event message 310 of the Fig. 3 and the trigger events 404 and 412 of the Fig. 4.

[0081] At step 614, the distributed message handler devices 102 receive the trigger event from the coordinator.

[0082] In step 616, the distributed message handler devices 102 receive sensor data from their respective sensors. This sensor data can include, among other things, image data from still and / or video cameras. In step 618, the distributed message handler devices 102 extract one or more features from the sensor data. In one example, the feature could identify a gesture performed by a person. In another example, the feature could identify an object present in the image data and / or the object's position relative to the camera.

[0083] In step 620, each distributed message handler device 102 that has detected one or more features transmits messages containing the corresponding data to all distributed message handler devices 102, including the coordinator. These messages are functionally similar to the extraction result messages 402, 406, and 410 of the Fig. 4. In step 622, the distributed message-handler devices 102, which are not coordinator devices, store the received messages. In step 624, the coordinator processes the received messages. In one example, the coordinator receives object and / or location detection messages from multiple distributed message-handler devices 102. In response, the coordinator determines the location of one or more people or other objects within an area. In step 636, the coordinator performs one or more actions in response. The response actions may include instructing the distributed message-handler devices 102 to stop performing one type of analysis or detection and to begin performing another type of analysis or detection.Additionally or alternatively, the response actions can include transmitting messages to one or more external devices, such as external device 106, to modify control features or properties of the external devices. The messages are functionally similar to the control message 408 for the external device. Fig. 4.

[0084] Procedure 600 then ends. Alternatively, procedure 600 continues to the previously described step 610, in which the distributed message handler devices 102 perform additional processing of sensor data.

[0085] Fig. Figure 7 is a flowchart for the process steps for selecting a subsequent coordinator in an Internet of Things architecture according to various implementations. Even if the process steps are related to the systems of Fig. From the descriptions in Figures 1-5, the person skilled in the art understands that any system configured to perform the process steps in any order is within the scope of this disclosure.

[0086] As shown, a procedure 700 begins at step 702, in which an application program running on an endpoint device currently selected as the coordinator periodically transmits heartbeat messages to the other endpoint devices. The messages are functionally similar to heartbeat message 502 of the Fig. 5. The application program can be the distributed message handler module 234, which runs on one of the distributed message handler devices 102 of the Fig. Step 1 is executed. At step 704, the coordinator stops transmitting heartbeat messages. For example, the coordinator may stop communication with message handler devices 102 of the device due to power loss, system failure, and the like.

[0087] In step 706, the remaining distributed message handler devices 102 determine that the coordinator is not transmitting heartbeat messages. The distributed message handler devices 102 can determine that the coordinator has not transmitted a heartbeat message for a specified period of time. Additionally or alternatively, the distributed message handler devices 102 can determine that the coordinator has been unable to transmit a heartbeat message for a specified number of times.

[0088] In step 708, the remaining distributed message handler devices 102 select a subsequent coordinator. These devices can then exchange messages that are functionally similar to selection messages 504, 506, and 508. Fig. 5. The distributed message handler device 102 selected as the subsequent coordinator can be the distributed message handler device 102 with the longest uptime since power-on, the highest or lowest MAC address, the highest or lowest IP address, and so on. In step 710, the subsequent coordinator retrieves the stored messages received from other distributed message handler devices 102. The stored messages provide the context necessary for the selected distributed message handler device 102 to assume the role of coordinator. In step 712, the distributed message handler device 102 selected as the subsequent coordinator takes over the operations of the previous coordinator. The subsequent coordinator transmits messages that are functionally similar to the trigger event message 510 of the Fig. 5 is. Procedure 700 then ends.

[0089] In summary, several endpoint devices, such as message-handler devices, are equipped with sensors to collect sensor data. These endpoint devices communicate with each other over a private communication network to transmit, receive, and process sensor-related data in a scalable manner. The endpoint devices communicate by passing data as messages or objects over the private communication network, with the messages or objects being agnostic to the type of endpoint device or corresponding sensor. The set of endpoint devices is self-contained and does not require a dedicated central server to process the sensor data.More precisely, the endpoint devices form a private communication network, with each endpoint device connected to the private communication network capable of acting as a coordinator for the analysis and / or detection performed by the endpoint devices and all subsequent actions. Furthermore, if the current coordinator fails or otherwise ceases communication with the private communication network, the remaining endpoint devices can select one of the remaining endpoint devices as the successor coordinator. The successor coordinator assumes the activities of the previous coordinator without interrupting service.

[0090] At least one technical advantage of the disclosed techniques compared to the prior art is that the endpoint devices in a distributed system transmit less sensor data compared to conventional approaches. Instead of transmitting raw sensor data to a remote data aggregator or cloud server, each endpoint device extracts one or more features represented by the sensor data. Each endpoint device then transmits messages containing data related to the extracted features, rather than the raw sensor data, to the other endpoint devices in the local communication network. As a result, the amount of data transmitted over the communication network can be significantly reduced. Likewise, the processing power required to handle the messages can be significantly lower than the processing power required to handle the raw sensor data.Furthermore, the endpoint device, designated as the coordinator, determines the type of analysis that the endpoint devices perform at any given time, resulting in reduced processing requirements for the endpoint devices. Consequently, the latency of the distributed system is shorter and its performance is higher compared to traditional approaches.

[0091] A further advantage of the disclosed techniques is that if the endpoint device designated as the coordinator fails, the remaining endpoint devices automatically select a new coordinator from among themselves. As a result, the distributed system is more stable with respect to the failure of an endpoint device, leading to a more robust distributed system compared to previous approaches. These technical advantages represent one or more technological advances over prior art approaches. 1. In some embodiments, a computer-implemented method for processing sensor data by a first endpoint device of a plurality of endpoint devices comprises: receiving first sensor data via a first sensor; receiving a command from a first coordinator to perform a first operation, the first coordinator being selected from the plurality of endpoint devices comprising the first endpoint device; in response to receiving the command, performing the first operation to extract a first feature from the first sensor data; and transmitting a first message containing data related to the first feature to the plurality of endpoint devices. 2. Computer-implemented method according to paragraph 1, wherein the first operation comprises gesture recognition and further comprises: recognizing a first gesture within a first image contained in the first sensor data, wherein the data associated with the first feature indicate that the first gesture has been recognized. 3. Computer-implemented method according to paragraph 1 or paragraph 2, further comprising, in response to the transmission of the first message, receiving a second command from the first coordinator to perform a second operation to extract a second feature from the first sensor data. 4. Computer-implemented method according to any one of paragraphs 1-3, wherein the second operation comprises at least one of detecting the presence of an object within a second image contained in the first sensor data or detecting a viewpoint in relation to the object within the second image. 5. Computer-implemented method according to any one of numbers 1-4, wherein the first operation comprises detecting the presence of an object and further comprises: detecting a first object within a first image contained in the first sensor data, wherein the data relating to the first feature indicate that the first object has been detected. 6. Computer-implemented method according to any one of numbers 1-5, wherein the first operation comprises recognizing a position associated with an object and further comprises: recognizing a first position of an object within a first image contained in the first sensor data, wherein the data associated with the first feature indicate that the first position has been recognized. 7. Computer-implemented method according to one of the items 1-6, further comprising: Determine that no coordinator has been selected from the plurality of endpoint devices; transmit a second message specifying a first parameter of the first endpoint device; receive a third message from a second endpoint device included in the plurality of endpoint devices, specifying a second parameter of the second endpoint device; and determine, based on the first parameter and the second parameter, whether the first endpoint device is the first coordinator. 8. Computer-implemented method according to any one of paragraphs 1-7, wherein the first parameter includes at least one of a time period since the first endpoint device was switched on, a media access control address (MAC address) associated with the first endpoint device or an Internet Protocol address (IP address) associated with the first endpoint device. 9. Computer-implemented method according to one of the numbers 1-8, further comprising: Determine, by the first endpoint device, that the first parameter and the second parameter are equal; and compare a third parameter of the first endpoint device with a fourth parameter of the second endpoint device by the first endpoint device, further determining whether the first endpoint device is the first coordinator based on the third parameter and the fourth parameter. 10. Computer-implemented method according to one of the numbers 1-9, wherein the first endpoint device is the first coordinator. 11. Computer-implemented method according to any one of the numbers 1-10, further comprising: in response to determining that the first coordinator has not transmitted a heartbeat message for a threshold time period, transmitting a fourth message specifying a third parameter of the first endpoint device; receiving a fifth message from a third endpoint device, which is included in the plurality of endpoint devices, specifying a fourth parameter of the third endpoint device; and determining, based on the third parameter and the fourth parameter, whether the first endpoint device is a second coordinator. 12. In some embodiments, one or more computer-readable storage media contain instructions which, when executed by one or more processors, cause the one or more processors to process sensor data through a first endpoint device of a plurality of endpoint devices by performing the following steps: receiving first sensor data via a first sensor; receiving an instruction from a first coordinator to perform a first operation, the first coordinator being selected from the plurality of endpoint devices comprising the first endpoint device; in response to receiving the instruction, performing the first operation to extract a first feature from the first sensor data; and transmitting a first message comprising data related to the first feature to the plurality of endpoint devices, the first message comprising a multipoint message. 13. One or more computer-readable storage media according to paragraph 12, wherein the first operation comprises gesture recognition and wherein the instructions further cause the one or more processors to perform the following step: recognizing a first gesture within a first image contained in the first sensor data, wherein the data associated with the first feature indicate that the first gesture has been recognized. 14. One or more computer-readable storage media according to paragraph 12 or paragraph 13, wherein the instructions further cause the one or more processors to perform the following step: in response to the transmission of the first message, receiving a second instruction from the first coordinator, to perform a second operation to extract a second feature from the first sensor data. 15. One or more computer-readable storage media according to any of items 12-14, wherein the second operation comprises at least one of detecting the presence of an object within a second image contained in the first sensor data or detecting a viewpoint in relation to the object within the second image. 16. One or more computer-readable storage media of one of the numbers 12-15, wherein the first operation comprises detecting the presence of an object and wherein the instructions further cause the one or more processors to perform the following step: detecting a first object within a first image contained in the first sensor data, wherein the data relating to the first feature indicate that the first object has been detected. 17. One or more computer-readable storage media of one of the digits 12-16, wherein the first operation comprises recognizing a position associated with an object and wherein the instructions further cause the one or more processors to perform the following step: recognizing a first position of an object within a first image contained in the first sensor data, wherein the data associated with the first feature indicate that the first position has been recognized. 18. One or more computer-readable storage media of one of the numbers 12-17, wherein the instructions further cause the one or more processors to perform the following steps: Determine that no coordinator has been selected from the plurality of endpoint devices; Transmit a second message specifying a first parameter of the first endpoint device; Receive a third message from a second endpoint device included in the plurality of endpoint devices, specifying a second parameter of the second endpoint device; and Determine, based on the first parameter and the second parameter, whether the first endpoint device is the first coordinator. 19. One or more computer-readable storage media according to any of items 12-18, wherein the first parameter includes at least one of a time period since the first endpoint device was switched on, a media access control address (MAC address) associated with the first endpoint device or an Internet Protocol address (IP address) associated with the first endpoint device. 20. In some embodiments, a first endpoint device comprises: a memory containing instructions; and a processor coupled to the memory which, upon execution of the instructions: receives first sensor data via a first sensor; receives an instruction from a first coordinator to perform a first operation, the first coordinator being selected from a plurality of endpoint devices comprising the first endpoint device, all endpoint devices comprising the plurality of endpoint devices communicating via a communication network; in response to receiving the instruction, performs the first operation to extract a first feature from the first sensor data; and transmits a first message comprising data relating to the first feature to the plurality of endpoint devices.

[0092] Any and all combinations of any claim elements presented in any of the claims and / or any elements described in this application are in any way within the intended scope of the present disclosure and its scope of protection.

[0093] The descriptions of the various embodiments are provided for illustrative purposes only and are not intended to be exhaustive or limited to the disclosed embodiments. Numerous modifications and variations are apparent to the person skilled in the art without deviating from the scope and spirit of the described embodiments.

[0094] Aspects of the present embodiments may be implemented as a system, a method, or a computer program product. Accordingly, aspects of the present disclosure may take the form of an embodiment entirely contained in hardware, an embodiment entirely contained in software (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware, all of which are generally referred to herein as a "module" or "system." Furthermore, aspects of the present disclosure may take the form of a computer program product executed on one or more computer-readable media comprising computer-readable program code executed thereon.

[0095] Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signaling medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, a device of such type, or any suitable combination thereof.More specific examples of computer-readable storage media (a non-exhaustive list) would include: an electrical connection with one or more wires, a portable computer disk, a hard disk, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only storage device (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, computer-readable storage media can be any physical medium capable of containing or storing a program for use by or in conjunction with an instruction execution system or device.

[0096] Aspects of the present disclosure are described above with reference to flowchart illustrations and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the disclosure. It is understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer program instructions.These computer program instructions can be provided to a processor of a general-purpose computer, a specialized computer, or another programmable data processing device for the manufacture of a machine, such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in the flowchart and / or block(s) of the block diagram. Such processors can be, among others, general-purpose processors, specialized processors, application-specific processors, or field-programmable.

[0097] The flowcharts and block diagrams in the figures illustrate the architecture, functions, and operation of possible implementations of systems, procedures, and computer program products according to various embodiments of the present disclosure. In this respect, each block in the flowcharts or block diagrams can represent a module, segment, or section of code comprising one or more executable instructions for implementing the specified logical function(s). It should also be noted that the functions specified in the block may appear in a different order than shown in the figures in some alternative implementations. For example, two consecutive blocks shown may actually be executed essentially simultaneously, or the blocks may sometimes be executed in reverse order, depending on the specific function.It should also be noted that each block of the block diagram and / or flowchart representation and combinations of blocks of the block diagram and / or flowchart representation can be implemented by special hardware-based systems that perform the specified functions or actions, or by combinations of special hardware and computer instructions.

[0098] Although the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure can be developed without deviation from its basic scope, and their scope is determined by the following claims.< / ein> < / aus> < / wert> < / ein> < / wert> < / wert> < / wert> < / person> < / ein> < / ein> < / wert> < / wert> < / wert> < / werte> < / kennung>

Claims

[1] Computer-implemented method (600) for processing sensor data by a first endpoint device (102), comprising: Transmitting (604) a first message to other endpoint devices (102) in a network (130), the first message indicating that the first endpoint device (102) is providing a first service; Receiving respective second messages from a plurality of endpoint devices (102), wherein the respective second messages indicate that the plurality of endpoint devices (102) also provides the first service; Forming (606) a private communication network with the multitude of endpoint devices (102); Receiving (616) sensor data via a sensor; Performing (618) a first operation to extract a first feature from the sensor data; and Transmitting (620) a third message containing data associated with the first feature to the plurality of endpoint devices (102). [2] Computer-implemented method (600) according to claim 1, further comprising receiving (614) a command from a coordinator to perform the first operation. [3] Computer-implemented method (600) according to claim 2, wherein the coordinator is one of several endpoint devices (102) in the private communication network. [4] Computer-implemented method (600) according to claim 2, further comprising receiving a second instruction from the coordinator to extract a second feature from the sensor data. [5] Computer-implemented method (600) according to claim 1, wherein the first operation comprises recognizing a gesture in an image that is included in the sensor data. [6] Computer-implemented method (600) according to claim 1, wherein the first operation comprises detecting the presence of an object within an image contained in the sensor data. [7] Computer-implemented method (600) according to claim 1, wherein the first operation comprises detecting an angle associated with an object detected within an image contained in the sensor data. [8] Computer-implemented method (600) according to claim 1, further comprising: Determine that no coordinator has been selected from the multitude of endpoint devices (102) in the private communications network; Transmitting a fourth message specifying a first parameter of the first endpoint device (102); Receiving a fifth message from a second endpoint device (102) included in the plurality of endpoint devices (102), which specifies a second parameter of the second endpoint device (102); and Determine, based on the first parameter and the second parameter, whether the first endpoint device (102) should be a coordinator. [9] Computer-implemented method (600) according to claim 8, wherein the first parameter comprises at least one of a time period since the first endpoint device (102) was switched on, a media access control MAC address assigned to the first endpoint device (102), or an Internet protocol IP address assigned to the first endpoint device (102). [10] Computer-implemented method (600) according to claim 8, further comprising: Determine, by means of the first endpoint device (102), that the first parameter and the second parameter are equal; and Comparing a third parameter of the first endpoint device (102) with a fourth parameter of the second endpoint device (102) by the first endpoint device, furthermore, determining whether the first endpoint device (102) should be the coordinator is based on the third parameter and the fourth parameter. [11] Computer-implemented method (600) according to claim 1, further comprising: In response to determining that a coordinator for the private communications network has not transmitted a heartbeat message (502) for a threshold duration, transmit a fourth message specifying a first parameter of the first endpoint device (102); Receiving a fifth message from a second endpoint device (102) included in the plurality of endpoint devices (102), which specifies a second parameter of the second endpoint device (102); and Determine, based on the first parameter and the second parameter, whether the first endpoint device (102) should become the coordinator for the private communication network. [12] Computer-implemented method (600) according to claim 1, further comprising: Receiving a fourth message from a second endpoint device (102) in the plurality of endpoint devices (102), wherein the fourth message contains data that is associated with the first feature extracted from the second endpoint device (102); and Saving the fourth message. [13] Computer-implemented method (600) according to claim 1, further comprising receiving a command from a coordinator in response to the transmission of the third message to the coordinator, wherein the command specifies that the first endpoint device (102) should stop performing the first operation. [14] One or more computer-readable data storage media containing instructions which, when executed by one or more processors, cause the one or more processors to carry out the method according to any one of claims 1-13. [15] First endpoint device (102), comprising: a memory (212) containing instructions; and a processor (202) that is coupled to the memory (212) and when executed the instructions carry out the procedure according to one of claims 1-13.

Images