Sensor data integration and event detection

Abstract

A method for time alignment of sensor data is described. A plurality of sensors used in sensor data integration is determined. A reference signal with a time stamp is provided to each of the sensors. An output signal in response to the reference signal is identified and used to determine a response time delay for each of the sensors. These time delays are stored and time corrections are applied to the sensor data during sensor data integration. This can be used for event detection. An associated method of detecting events from sensor data from a plurality of sensors is also described. A threshold is determined for each of the sensors such that a signal exceeding the threshold is identified as a potential sensor event. Potential sensor events from each of the sensors are detected and when potential sensor events from two or more sensors fall within a predetermined time window, this is identified as a possible actual event. This is followed by performing time alignment of the sensor data for sensor data integration to determine if the potential sensor events correspond to an actual event. If so, a representation of the actual event is provided from the integrated sensor data. A suitable computing device for performing such a method is described.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to integration of data obtained from multiple sensors and detection of events. BACKGROUND

[0002] Control processes respond to input variables, which often include sensed values. In complex environments, there will often be multiple different types of sensors. As shown in Figure 1 This complex environment can for example be a geographical location 1 or a group of associated buildings, where a wide variety of different sensors 3 can be used to determine behaviour within the environment, for example in a control system used to predict power management, to anticipate and respond to future demand. A control system 5 is shown remote from location 1, which uses data from sensors 3 to determine behaviour of a power supply system 4 to power users 2 in location 1.

[0003] Different types of sensors will have different performance characteristics, including in particular time related characteristics. There are two main types of such characteristics: reaction time, i.e. the time delay between an event occurring and a sensor producing a measurement or report; and changes in responsiveness over time as components age or are affected by environmental or manufacturing differences. Integration of data from different sensor types presents several challenges. In particular, when using sensors to determine when an event of interest has occurred, differences in responsiveness between sensors can seriously distort the result, making the association of sensor data with a given event less accurate, and in extreme cases even impossible. It is desirable to more effectively manage the integration of data from a combination of mixed sensors, particularly but not exclusively in the case of event detection. SUMMARY

[0004] In a first aspect, the present disclosure provides a method for time alignment of sensor data, the method comprising: determining a plurality of sensors for use in sensor data integration; providing a reference signal with a time stamp to each of the sensors in the plurality of sensors; detecting an output signal from each of the sensors in response to the reference signal, and determining a response time delay for each of the sensors; and storing the time delay for each sensor for application of a time correction to the plurality of sensors during sensor data integration.

[0005] Using this method, differences in response between different sensors can be eliminated by providing each sensor with a time correction factor, where these time correction factors are used to provide time alignment. Variations in this time delay can be caused by differences in buffering and processing stages within a single sensor. Eliminating these differences allows sensor data to be effectively integrated as a representative of the overall system, the individual sensors being part of the overall system, and also allows accurate correlation of sensor signals with a specified time event.

[0006] In embodiments, a plurality of sensors are identified from a sensor inventory, and the time delay for each sensor can be stored by updating the sensor inventory. Time correction can also (or in other embodiments instead) be stored in or associated with the sensor itself.

[0007] In certain embodiments, the reference signal is provided as a pulsed signal or a periodic signal. In other embodiments, the sensors are channelized, such that each sensor is identified as a separate channel. These channels can then be multiplexed within a single physical channel, such that (for example) multiple sensors can use the same channel with different time slots. This can be accomplished by adding a periodic preamble to the sensor that includes a low autocorrelation code, where detecting the output signal includes performing a cross-correlation to determine the presence of the signal. Such channelization - using, for example, Gold codes or other codes with appropriate cross-correlation properties, like the Kasami codes and Kasami codes in CDMA - can be a particularly effective method of managing data in complex sensor assemblies.

[0008] The process of detecting the output signal for a sensor can include identifying a protocol for reading the sensor, and performing a sensor read operation according to the protocol. Sensors that are unable to use any protocol to effectively read are then discarded.

[0009] In a second aspect, the disclosure provides a method of detecting events from sensor data from a plurality of sensors, the method comprising: determining a threshold value for each of the sensors, whereby a signal that exceeds the threshold value is identified as a potential sensor event; receiving the sensor data, and detecting potential sensor events from each of the sensors; identifying when potential sensor events from two or more sensors fall within a predetermined time window; performing time alignment of the sensor data for sensor data integration to determine whether the potential sensor events correspond to one or more actual events, and providing a representation of the actual events from the integrated sensor data.

[0010] This detection action extends to inferring events based on fusion and analysis or two or more sensor outputs. For example, positive outputs from sensors indicating high humidity and low pressure can infer that it is raining.

[0011] Using this method, sensor data can be effectively used to identify and characterize actual events without excessive computation when potential events are detected. This has a number of practical benefits, including energy savings and conservation of network bandwidth.

[0012] The time alignment can comprise finding a preferred timing offset for each sensor. This can involve averaging over multiple readings, for example by a windowed average over the last N readings, for one or more of the sensors.

[0013] The time alignment can use the time delays of the plurality of sensors established using the method of the first aspect of the disclosure. This is a particularly effective way in which to manage the integration of sensor data of a combination of sensors in a particular environment, for example a building or group of buildings, and to use the integrated sensor data in event detection.

[0014] In embodiments, the representation of the actual event can comprise an identification of the sensor that detected the actual event. It can also comprise an identification of the sensor pattern used to detect the actual event.

[0015] In a third aspect, the disclosure comprises a computing device comprising a processor and a memory, wherein the processor is programmed to perform the method of either or both of the first and second aspects, and wherein the computing device is connected to receive sensor data from the plurality of sensors used in the method.

[0016] In the case where the processor is programmed to perform the method of the second aspect, the computing device can be adapted to provide a representation of the actual event to a remote computing system. This is particularly effective when the first and second aspects are performed by a computing device located in association with a particular environment, but where it needs to be provided elsewhere, for example to a management system addressing a plurality of different environments, for analysis and management of the environmental sensor data.

[0017] In a fourth aspect, the disclosure provides a system for detecting events, the system comprising: a plurality of sensors, wherein each of the sensors has a stored response time delay and a threshold for detecting a potential sensor event; and a programmed processor adapted to identify potential sensor events from the plurality of sensors and to determine whether the identified potential sensor events correspond to actual events using the stored time delays of the relevant sensors from the plurality of sensors.

[0018] The programmed processor can be adapted to provide a representation of the event to a remote computing system. The programmed processor can be adapted to characterise the determined actual event from data received from the relevant sensors of the plurality of sensors.

[0019] In a fifth aspect, the present disclosure provides a control system comprising a programmed processor, wherein: the control system is adapted to receive a representation of actual events obtained according to the method of the second aspect; and the programmed processor is adapted to analyse the system using the received representation of actual events, and thereby determine a control operation or control strategy.

[0020] Such a control system can be a power management system for an environment, and the actual events can be events associated with the environment. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings in which:

[0022] Figure 1 A control system is shown which uses multiple sensors to determine actions based on detected events;

[0023] Figure 2 An architecture for implementing the control system as Figure 1 shown is shown;

[0024] Figure 3a and Figure 3b Sensor readings and time-aligned sensor readings are shown respectively;

[0025] Figure 4 A first embodiment of a method of providing time-alignment for readings from a combination of sensors is shown;

[0026] Figure 5 A second embodiment of a method of providing time-alignment for readings from a combination of sensors is shown; and

[0027] Figure 6 Embodiments of a method of event detection following time-alignment as provided by the methods shown in Figure 5 and Figure 6 are shown.

[0028] As described above in Figure 1 Embodiments of the present disclosure can involve sensors deployed in a complex environment, such as a geographical location 1 or a set of associated buildings. In embodiments of the present disclosure, a wide variety of different sensors 3 are used to determine behaviour within the environment. This has several possible purposes. One purpose is for a control system for power management to predict and respond to future demand. A control system 5 is shown remote from location 1, which uses data from sensors 3 to determine behaviour of a power supply system 4 to power users 2 in location 1 (both detecting existing behaviour and providing control). Power supply metering and control are not the only possible applications for this type of system.

[0029] Figure 2 A computing architecture 20 suitable for use in such an arrangement is shown in the middle. The computation of power management behavior can require complex calculations, and it can involve information received from multiple different environments - typically this will be performed in a distributed environment ("the cloud"), a local computing environment, a mobile edge computing environment, or by a suitable remote computing environment (here denoted the cloud environment 24). However, it is generally desirable to perform important computations locally - in particular event detection, so that it is not necessary to forward the complete data stream from each sensor to the cloud environment 24. This can be performed by an edge computing unit 21, there can be many such edge computing units 21 in a complex environment (for example, one for managing each building in an environment), and these can be linked by a local data network 22. Each of the edge computing units 21 here connects to a combination of sensors 23, such as the sensors 23.1, 23.2,..., 23.n shown as being associated with the edge computing unit 21.1. Here, the edge computing unit 21 will have a suitably programmed processor and access to memory, as well as the network or other connections required to receive sensor data and to be able to communicate with the cloud environment 24.

[0030] In embodiments of the present disclosure, data integration and event detection involve two processes together. One process is the time alignment of data from a combination of sensors. The other process is the use of the time-aligned data to detect events. These two events can in principle occur at more than one point in the computing architecture 20, but a particularly suitable location is the edge computing unit 21 for detecting events from a combination of sensors 23 associated with that edge computing unit 21.

[0031] Figure 3a and Figure 3b The problem is illustrated generally. As perceived by the edge computing unit 21, the signals from the sensors 23 appear to occur at different times, even when they originate from a common event. There can be many reasons for this - delays between the event and its sensor detection, the time response of the sensors themselves, and the time delay between the sensors and the edge computing unit are three reasons that can need to be considered. It is desirable to correct the position at the edge computing unit to Figure 3b the position shown, rather than Figure 3a the position shown, so that it is possible to determine when two or more sensor signals are related to the same event. It should be noted that the actual position can be slightly more complex than Figure 3a and Figure 3b the position shown, because there can be some variation in the time taken for sensor readings from a common event to arrive at the edge computing unit, which means that it is necessary to consider the corrected sensor values as a distribution rather than a point reading when considering event detection.

[0032] Figure 4 The main steps in a first embodiment of a method of time alignment of sensor readings according to the present disclosure are shown.

[0033] First, it is necessary to discover 41 the combination of sensors, typically in the form of a list of sensors with sensor identifiers (e.g. addresses). These are the sensors that are relevant to the detection event, as if the same event is captured at some combination of sensors, that combination can help characterize the event or distinguish it from other events in a way that will be used in the downstream analysis model. For example, the sensors can relate to a common geographical environment (e.g. the sensors can form a set of sensors contained within a particular building), but this is not a requirement. Typically, this step can comprise obtaining 411 a library of sensors from a data storage 211 associated with the edge computing unit 21, but in principle can comprise other procedures such as device discovery.

[0034] After this, a test input signal is provided 42 to characterize the latency associated with the sensors. This is typically a known reference signal emitted at a known time (e.g. a delta function at a defined time zero, or a pulse period signal as shown in the example). The sensors are connected 421 to the signal input, a time stamp 422 is applied to establish a time frame, and the test signal is activated 423. This test input signal is used to activate each sensor so that the sensor will produce a characteristic output that can be used for event detection. Figure 4

[0035] After the edge computing unit 21 is connected 43 to the sensors, the sensor data is read out for the reference signal sensor events. The communication of the sensor data to the edge computing unit can be through one of a number of different protocols, which can be a protocol that is particularly suitable for a particular sensor type, and the path from the sensor to the edge computing unit can also be associated with a particular protocol (e.g. the communication with a particular type of sensor can be through a particular bus that uses a particular protocol). Exemplary protocols are Serial Peripheral Interface (SPI), Inter-Integrated Circuit (I2C) and I3C as specified by the Mobile Industry Processor Interface (MIPI) Alliance. In the method shown, when the connection is made, it is determined 431 which protocol to use, and the protocol is processed 432 in a list, and the reading is taken 433 using the first appropriate protocol - if none of the available protocols are valid, then an unsupported sensor type is reported 434, and the sensor is omitted from the time alignment process. Figure 4

[0036] ​​When the edge computing unit 21 has received the sensor signal, it determines 44 the time delay - this can be determined from the sensor event data and the known reference signal timestamp 441. Local time sources or network based time sources can be used. Possibilities include Network Time Protocol (NTP), IEEE 1588 Precision Time Protocol (PTP) and local or satellite timing based on atomic clock timing sources such as Global Positioning Service - GPS, Global Navigation Satellite System - GLONASS, Galileo, Beidou Navigation System. When this has been determined, the data store 211 can be updated 442 with the time delay of the sensor. Information can also be logged in the sensor itself (or in a local store associated with the sensor itself).

[0037] The process is repeated 45 until completion. There are two aspects to this repetition. One aspect is that the combination of sensors can be addressed sequentially or in parallel (as the steps indicated above can be performed on multiple sensors in the combination at the same time). Whichever approach is taken, the process needs to be repeated until all sensors in the combination have been evaluated and provided with a time delay (or rejected from the process). However, in some cases, for the reasons discussed above, a sensor can provide a variable delay. In this case, it is desirable to perform multiple readings on a given sensor until a sufficient level of confidence in the sensor time delay is reached - this can involve simply reaching a sufficient level of confidence in the mean or median value, or it can involve recording a distribution of possible values for the sensor time delay. Again, information will be logged in the data store 211 for each sensor, or possibly in or with the sensor itself.

[0038] Figure 5 A modified method of establishing sensor time delays is shown in Figure 5. As in Figure 4 a list of sensors is established 51, but rather than simply providing a test signal in relation to the reference time in Figure 4 a periodic known preamble sequence is added 52 to the sensor payload. This is preferably provided 521 as a code with low auto-correlation properties to allow efficient resolution of the sensor signal - the Barker sequence is the most complete solution, but practical approaches are to use Gold codes (as is commonly done in Code Division Multiple Access (CDMA) channel access for telecommunications). Thus, other codes with appropriate properties, such as Zero-Correlation Codes - Kasami codes and Hadamard codes are other possible candidate codes. The code and associated timestamp can then be written 522 to the sensor synchronisation register.

[0039] Connection 53 of the sensor to the edge computing unit 21 is made using an appropriate protocol as previously described, but Figure 5The method differs in how the signals are received. The sensor data is channelized 54, where each sensor is labeled as a channel 541. Time division can be used for multiplexing - multiple sensors can use a single channel, each sensor having a different time slot A, then a cross correlation process is performed 55 using a code - if the cross correlation threshold is met 551, it is determined if a preamble exists 552 - if not, the process continues, but if so, the time delay is measured 553 and, as before, the time stamp is applied 554 to determine the time delay of the sensor, after which the data store 211 is updated 555.

[0040] Once the time delays have been established for a combination of sensors, they are used for event detection. Figure 6 An embodiment of an event detection process using time aligned sensor data is shown in FIG. 6.

[0041] The first step is to determine 61 if a signal of interest exists - this criterion can be, for example, that two or more sensors record an event, but other criteria can be used in different arrangements. This is achieved by receiving a stream of maximum sensor values - absolute sensor values - 611, then determining if a potential event exists for any given sensor by a threshold detection 612 step, to determine if the sensor output value meets or exceeds a predetermined threshold or watermark level (which can be configurable). If the threshold is met, the sensor value is converted 613 to a simple binary value. This is followed by a logical AND operation 614 between different sensor values to determine if a potential event exists, as this AND operation finds an event detected by at least two sensors. The sensors that have "detected" a potential event are corrected 615 for timing offset to determine if the event truly occurred at the same point in time after time alignment - if not 616, the process stops here, otherwise it continues.

[0042] It can be seen that, using this method, full time alignment is not required before an event is detected. In principle, all sensors could be time aligned before it is determined if an event has been detected. However, this would require more computation - avoiding this computation would allow efficient use of computational resources, and bring attendant benefits such as energy savings.

[0043] However, once an event is detected, the best time delay offset is found 62 for the combination of sensors. This best time delay offset is the correction value that most accurately results in all recorded sensor data measurements being aligned. This can include, for example, determining a windowed average based on a set of values derived from one or more reference tests - here a sliding time windowing method is used, where the latest N samples are used 621, and an average is calculated 622, all with reference to data held in the data store 211.

[0044] A delay correction factor 63 is then calculated - this can be done by ordering 631 the sensor measurements chronologically (e.g. with the oldest first) and selecting 632 the oldest measurement as the new time alignment point for all sensor measurements. The time stamps are recovered 633 and a time offset is determined 634 for each sensor, which allows the delay correction 64 to take place. The time offset is subtracted 65 from the measurement time stamp, and the process continues until all sensor measurements are compensated 66. At this point, time alignment is complete, and it can be established which sensors have detected or otherwise responded to the event, and the process can stop 67.

[0045] The method allows the determination at the edge computing unit 21 that an event of interest has occurred, and which of the sensors have detected the event of interest. This can allow the event to be characterised, and a determination to be made as to what information relating to the event should be provided from the edge computing unit to a remote computing environment that performs analysis and power management. This allows a stream of information from the edge computing unit 21 that includes a selected set of data relating to the detected event, rather than an undifferentiated stream of data that requires further analysis. This limits the computational burden on the remote computing system, and reduces the amount of data transmitted between the edge computing unit and the remote computing system, improving overall efficiency.

[0046] The act of detecting an event can be extended to infer an event based on fusion and analysis or two or more sensor outputs. For example, positive outputs from sensors indicating high humidity and low pressure can infer that it is raining. In embodiments, such event characterisation can take place at the edge computing unit 21, and in other cases the edge computing unit 21 can simply determine that an event has occurred that needs to be considered, with the remote computing environment providing the event characterisation - event characterisation can also be split between the two. It should be noted that event characterisation can be determined not only by the combination of sensors involved, but also by the pattern of sensors used.

[0047] Using the method, in a scenario with Figure 2 The computing architecture shown Figure 1In the illustrated environment, an event is detected by the edge computing unit 21 from the readings given by the sensors 3, 23. The edge computing unit 21 uses the stored time delays to determine whether the potential event sensed by the sensors is an actual event, and can also provide a characterization of the event based on what sensors or patterns of sensor responses are involved - although such characterization can in embodiments remain with the power management system itself located in the cloud environment 24, with the edge computing unit 21 simply providing sufficient information such as sensor readings from the relevant sensors for the relevant time period to indicate an actual event. The power management system uses the actual event and its characterization or indication to perform a power management control operation or power management control strategy on the environment. In other embodiments, the control system can not be a power management system, but another control system.

[0048] As will be appreciated by persons skilled in the art, the embodiments described above are exemplary, and further embodiments falling within the spirit and scope of the present disclosure can be developed by persons skilled in the art from the principles and embodiments set forth above.

Claims

1. A method for detecting an event from sensor data from multiple sensors, the method comprising: A threshold is determined for each of the sensors, whereby signals exceeding the threshold are identified as potential sensor events. Receive the sensor data and detect potential sensor events from each of the sensors; Identify when potential sensor events from two or more sensors fall within a predetermined time window; Perform time alignment of the sensor data for sensor data integration to determine whether the potential sensor event corresponds to an actual event, and Provide a representation of the actual event from integrated sensor data; The timing alignment process includes finding a preferred timing offset for each sensor.

2. The method of claim 1, wherein for one or more of the sensors, the preferred timing offset is obtained by averaging multiple readings.

3. The method according to claim 2, wherein the preferred timing offset is the windowed average of the last N readings.

4. The method according to any one of claims 1 to 3, wherein time alignment includes using a time delay of the plurality of sensors established according to a method comprising: Identify the multiple sensors used in sensor data integration; Provide a timestamped reference signal to each of the plurality of sensors; The output signal from each of the sensors is detected in response to the reference signal, and the response time delay of each of the sensors is determined; as well as The time delay of each sensor is stored for applying time correction to the plurality of sensors during sensor data integration.

5. The method of claim 4, wherein the plurality of sensors are identified from a sensor inventory, and wherein the time delay of each sensor is stored by updating the sensor inventory.

6. The method of claim 4, wherein the reference signal is provided as a pulse signal or a periodic signal.

7. The method of claim 4, further comprising channelizing the plurality of sensors such that each sensor is identified as a separate channel.

8. The method of claim 7, wherein the reference signal is provided by adding a periodic preamble comprising a low autocorrelation code to the sensor, wherein detecting the output signal comprises performing cross-correlation to determine the presence of the signal.

9. The method of claim 4, wherein the process of detecting the output signal for the sensor includes identifying a protocol for reading the sensor and performing a sensor reading operation according to the protocol.

10. The method of claim 1, wherein the representation of the actual event includes the identification of the sensor that has detected the actual event.

11. A computing device comprising a processor and a memory, wherein the processor is programmed to perform a method for detecting an event from sensor data from a plurality of sensors by providing a representation of the actual event from the integrated sensor data, according to any one of claims 1 to 10, and wherein the computing device is connected to receive sensor data from the plurality of sensors used in the method.

12. The computing device of claim 11, wherein the computing device is adapted to provide the representation of the actual event to a remote computing system.

13. A system for detecting events, the system comprising: The computing device of claim 11 or 12, wherein the computing device is adapted to perform as a system for detecting events to provide a representation of the actual events from integrated sensor data; as well as The control system includes a programming processor, wherein: The control system is adapted to receive a representation of the actual event from the system that detects the event; and The programming processor is adapted to analyze the system using representations of the received actual events, and thereby determine control operations or control strategies.

14. The system of claim 13, wherein the control system is a power management system for an environment, and the actual event is an event associated with the environment.

Images