Hardware-Based Sensor Analytics

Abstract

A method is provided for monitoring messages from a sensor using an integrated circuit, the messages including data measured by the sensor, the method including: reading a first message from an interconnect circuit of the integrated circuit, the interconnect circuit connecting the sensor to one or more core devices configured to process the message, calculating a first hash value for the first message, comparing the first hash value to one or more previous hash values ​​stored in a hash store, each previous hash value corresponding to a message read from the interconnect circuit prior to the first message, and implementing corrective action if a difference between the first hash value and at least one of the previous hash values ​​stored in the hash store is greater than a predetermined threshold.

Description

[Technical Field]

[0001] The present disclosure relates to monitoring messages from sensors using monitoring circuitry within a SoC (System on Chip) or MCM (Multi-Chip Module).

[0002] background Integrated circuit chips can be used to monitor data messages recorded by sensors. The recorded data messages can be used to detect sensor failures, which can result in the sensor being unable to refresh its recorded messages, either in whole or in part. Such sensor failures can result in the sensor being unable to record problems occurring within the system in which the sensor is implemented. To detect sensor failures in known integrated circuit chip devices, data messages recorded by the sensor are monitored by first writing the messages to a memory connected to the integrated circuit chip. Once the messages are written to memory, they can then be read from the memory and processed by software contained within one or more processing modules on the chip.

[0003] One example of a sensor that can be used to indicate a problem occurring within a system is an image sensor, such as a camera. If the sensor is an image sensor, each data message recorded by the sensor is, for example, an image frame, or a line of pixels, or other subdivision of an image frame, captured in a given time window. Image frames captured over subsequent time windows are compared by the monitoring software to determine whether a problem is occurring. Small differences in the image frames recorded by the sensor over time can be used to indicate a fault in the entity being monitored by the sensor.

[0004] The need for integrated circuit chips to store data messages in memory before they can be processed by monitoring software introduces a significant time delay in detecting faults in the system being monitored by the sensor. This is problematic for implementations that rely heavily on sensor data to identify problems in the system in which the sensor is implemented so that responses to those problems can be implemented with minimal time delay. For example, if the sensor is an image sensor, the sensor may be required to recognize upcoming external objects and trigger a response to the proximity of those objects.

[0005] It would also be desirable to be able to use such a system to monitor changes within the monitored entity that may indicate a critical situation. For example, when monitoring a nuclear reactor, the integrity of the reactor's structure is critical to ensuring its safety, and therefore, knowing when a deformation of the reactor structure occurs is absolutely essential. If such a fault were detected using sensors in known integrated circuit chip devices, the data messages that would be recorded by the sensors would suffer from time delay problems similar to those described above. This would lead to the additional problem that events where the timeliness of any notification could make the difference between being able to take corrective action may be inadequately monitored or, in the worst case, missed altogether.

[0006] There is a need to reduce the time it takes to monitor data messages recorded by sensors, especially when examining entities for time-critical changes.

[0007] overview According to a first aspect, there is provided a method for monitoring messages from a sensor using an integrated circuit, the message including data measured by the sensor, the method including: reading a first message from interconnect circuitry of the integrated circuit, the interconnect circuitry connecting the sensor to one or more core devices configured to process the message; calculating a first hash value for the first message; comparing the first hash value to one or more previous hash values stored in a hash store, each previous hash value corresponding to a message read from the interconnect circuitry before the first message; and implementing corrective action if a difference between the first hash value and at least one of the previous hash values stored in the hash store exceeds a predetermined threshold.

[0008] The processes of observing a problem, processing the data, and notifying that a problem has been observed are carried out within nanosecond timescales based on the processing power of integrated circuits, and therefore by comparing any differences in hash values with a predetermined threshold, the method can be used to monitor messages from systems designed to observe entities in real time, and to effectively issue alerts when changes occur in such time-critical entities.

[0009] The method can further include storing the first hash value in a hash store, where the first hash value can be stored at the same time the first hash value is compared to the previous hash value.

[0010] The method may further include buffering the first message before the first message is read by the one or more core devices, wherein the size of the buffer may correspond to the number of data values used to calculate the data hash including the first hash value.

[0011] The method can further include comparing the first hash value with M previous hash values and performing a corrective action when the difference between the first hash value and P previous hash values exceeds a predetermined threshold, provided that 1 < P < M. When this is implemented, the method can further include deriving a confidence value corresponding to the P previous hash values such that the difference between the value of each of the P previous hash values and the first hash value exceeds the predetermined threshold. M can be dynamically set and changed.

[0012] A corrective action can be performed when the first hash value is not the same as at least one of the previous hash values stored in the hash store.

[0013] The method can further include calculating a plurality of hash values for a first message, wherein each of the plurality of hash values represents a portion of the first message, and each of the plurality of hash values for the first message is compared with one or more previous hash values corresponding to the portion of the first message. The plurality of hash values calculated for the first message can represent overlapping portions of the first message. Alternatively, the plurality of hash values calculated for the first message can represent non-overlapping portions of the first message.

[0014] The corrective action can be one or more of notifying the user that the sensor data is unreliable, deactivating components of an integrated circuit that depend on data obtained from the sensor, and activating a mechanism configured to repair the sensor.

[0015] The corrective action can include outputting a warning signal to another component of the integrated circuit.

[0016] Corrective action may include outputting a hardware event, outputting an interrupt signal, or outputting a message to an external core device.

[0017] The sensor may be an image sensor, and each message read from the interconnect circuitry may be at least a portion of a frame captured by the image sensor.

[0018] The sensor may be an image sensor, and each message read out from the interconnect circuit may be a frame captured by the image sensor.

[0019] The integrated circuit may be a system on a chip. The interconnect circuit may be a data bus.

[0020] In a second aspect, an integrated circuit chip is provided for monitoring messages from a sensor, the messages including data measured by the sensor. The integrated circuit chip includes: interconnection circuitry connecting the sensor to a core device configured to process the messages; a monitoring device configured to read a first message from the interconnection circuitry; a hash store for storing a plurality of previous hash values, each hash value corresponding to a message read from the interconnection circuitry prior to the first message; and a device configured to calculate the first hash value for the first message and compare the first hash value with one or more of the previous hash values stored in the hash store. The integrated circuit is configured to implement corrective action if a difference between the first hash value and at least one of the previous hash values stored in the hash store exceeds a predetermined threshold.

[0021] The device may be a processor or a set of finite state machines.

[0022] The present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a schematic diagram of an exemplary integrated circuit chip device. [Figure 2] 1 is a schematic diagram of an exemplary system for monitoring data messages using an integrated circuit chip device. [Figure 3] 3 illustrates a detailed embodiment of a monitoring device for the integrated circuit chip device shown in FIG. 2. [Figure 4] FIG. 1 is a schematic diagram of an alternative exemplary system for monitoring data messages using an integrated circuit chip device. [Figure 5] 5 is a flowchart of an exemplary method for monitoring messages from a sensor using the integrated circuit chip device shown in FIG. 2 or FIG. 4. [Figure 6] 5 is a flowchart of an alternative exemplary method for monitoring messages from a sensor using the integrated circuit chip device shown in FIG. 2 or FIG. 4. [Figure 7] FIG. 1 is a schematic diagram of an exemplary implementation of an embodiment.

[0024] Detailed Description 1-4 are schematic diagrams of exemplary system architectures and components within the system architectures. These figures present the structure in terms of functional blocks. Some functional blocks for performing functions well known in the art are omitted from these figures. FIGS. 5 and 6 are flowcharts illustrating a method for monitoring sensor messages using the system architectures of FIGS. 1-4. Each flowchart illustrates an order in which the method of that flowchart may be performed. However, these flowcharts are not intended to limit the described methods to implementation in the order shown. Method steps may be performed in an order alternate to that shown in the flowchart.

[0025] 1 shows the general structure of an exemplary monitoring network for an SoC 100. Monitoring circuitry 104 is arranged to monitor system circuitry 102.

[0026] 2 illustrates an exemplary system 200 including an integrated circuit chip 100, such as an SoC 200, for monitoring data messages received from a sensor 202. The messages are referred to as data messages because they include one or more distinct data values recorded by the sensor. The sensor 202 may be any type of sensor capable of recording data periodically. That is, the sensor 202 may record data continuously or periodically over periodic time windows. Each time window may alternatively be referred to as a number of clock cycles of the integrated circuit chip. In one example, the sensor 202 is an image sensor, such as a camera, and each data message recorded by the sensor is an image frame.

[0027] Like the sensor 202, the SoC 100 is also connected to the memory 204. Thus, the sensor 202 is connected to the memory 204 by the SoC 100. In FIG. 2, the memory 204 is external memory. An external memory is memory that is not included within the SoC 100. In an alternative example, the memory 204 can be internal memory and therefore included within the SoC 100. The memory 204 is configured to store data messages recorded by the sensor 202 and transmitted to the memory 204 via the SoC 100.

[0028] 1, the SoC of FIG. 2 also includes a system circuit 102 and a monitoring circuit 104. The system circuit 102 includes an interconnect circuit 206 and one or more core devices 208, 210, and 212. The interconnect circuit 206 connects the sensors 202 and memory 204 to the core devices 208, 210, and 212, and also connects the core devices 208, 210, and 212 to each other. Thus, the interconnect circuit 206 enables the transfer of data between the sensors 202, the memory 204, and the core devices 208, 210, and 212. In one example, the core devices 208, 210, and 212 are master devices. In an alternative example, the core devices 208, 210, and 212 are combined master and slave devices. One or more of the core devices 208, 210, and 212 are configured to process data messages received from the sensors 202. To enable processing of data, one or more core units 208, 210, 212 include appropriate system software. While the SoC is illustrated in Figure 2 as including three core units, any number of core units may be suitably incorporated into the system circuitry. The SoC interconnect forms a communications backbone for the SoC through which the core units can communicate with each other. These communications are bidirectional.

[0029] A master device is a device that initiates traffic such as read / write requests in a network. Examples of master devices are processors such as DSPs (digital signal processors), video processors, application processors, CPUs (central processing units), and GPUs (graphics processing units). Any programmable processor can be a master device. Other examples of master devices are devices with DMA (direct memory access) capabilities, such as traditional DMA for moving data from one location to another, autonomous coprocessors with DMA capabilities (such as encryption engines), and peripherals with DMA capabilities (such as Ethernet controllers).

[0030] A slave device is a device that responds to the commands of a master device. Examples of slave devices are on-chip memory, memory controllers for off-chip memory (such as DRAM), and peripheral units.

[0031] The topology of the interconnect circuitry 206 is SoC dependent. For example, the topology of the interconnect circuitry 206 may include any one or combination of the following types of networks for carrying communications around the system circuitry: a bus network, a ring network, a tree network, or a mesh network. In one example, the interconnect circuitry is a data bus.

[0032] As described above, the core devices 208, 210, and 212 of the system circuit 102 are configured to process data messages recorded by the sensor 202. The core devices 208, 210, and 212 use appropriate system software to perform this data processing. To process a data message, the data message is first written to memory 204. Once written to memory 204, the data messages can be read by one or more of the core devices 208, 210, and 212 for processing. Software contained within one or more of the core devices 208, 210, and 212 can process the data by comparing newly acquired data values with previous data values stored in memory 204. In one example, dissimilarity between data values acquired from successive data messages indicates the occurrence of a change in the entity being monitored by the sensor 202. That is, the sensor 202 may record multiple different data values, indicating a change in conditions in the system in which the sensor is implemented. In examples where sensor 202 is an image sensor, the detection of multiple distinct data values recorded by the sensor can indicate that the image sensor is monitoring an entity for a change that is currently occurring. The sensor may not refresh the messages it records identically; for example, all or a portion of the recorded image frames may no longer be identical; instead, the sensor may record a message that is different from the previous message. This indicates a change within the monitored entity that may indicate a critical situation. For example, when monitoring a nuclear reactor, the integrity of the reactor's structure is important to ensuring its safety, and therefore, knowing when deformation of the reactor's structure occurs is absolutely essential.

[0033] A drawback of using the system circuitry 102 to process data messages recorded by the sensors 202 is that it results in a significant time delay between a change in the message from the sensor and the detection of a change in the entity being monitored. That is, the sensor data must be written into memory 204 before it can be read and processed by software contained within the core devices 208, 210, and 212. The time taken to store and process the data message can be on the order of tens to hundreds of milliseconds. This time is too slow for some embedded systems, which necessitates a faster fault detection mechanism to be implemented to allow an adequate amount of time to react to changes in the entity.

[0034] To provide a faster method of monitoring data messages recorded by the sensors 202, the monitoring circuit 104 includes one or more monitoring devices 214, 216, and 218. The monitoring devices 214, 216, and 218 are connected to the interconnection circuit 206. The monitoring devices 214, 216, and 218 are configured to non-intrusively read data messages from the interconnection circuit 206. The data messages from the interconnection circuit 206 are messages sent by the sensors 202 through the interconnection circuit 206 before they reach the memory 204. These data messages are read by the core devices 208, 210, and 212 without being extracted from the interconnection circuit 206. Instead of using system software used by the core devices 208, 210, and 212, the monitoring devices 214, 216, and 218 are configured to read and subsequently process the data messages using hardware component devices.

[0035] The monitoring devices 214, 216, and 218 can be configured to selectively read only data messages transmitted by the sensor 202. That is, the monitoring devices 214, 216, and 218 can be configured to filter out any messages transmitted via the interconnection circuitry 206 that are not data messages from the sensor, if any such messages exist. The monitoring devices 214, 216, and 218 can further be configured to autonomously read these data messages without any software intervention. In one example, messages transmitted via the interconnection circuitry 206 can be accompanied by a flag identifying the module from which the messages originated. The monitoring devices 214, 216, and 218 can be configured to observe this identifying flag to filter out messages that are not data messages transmitted by the sensor 202. In other words, the monitoring devices 214, 216, and 218 can selectively read only data messages that include a flag identifying the sensor 202 as the module from which the data messages originated. In another example, messages sent over interconnect circuitry 206 are accompanied by flags indicating the modules to which those messages are to be written. Thus, monitors 214, 216, 218 can selectively read only those data messages that include a flag identifying the address in memory 204 to which the data message is to be written.

[0036] 2 illustrates the SoC as including three monitors, any number of monitors may be suitably incorporated into the integrated circuit chip 100. Each monitor 214, 216, 218 may be connected to the interconnect circuit 206 by a single communication link. Alternatively, one or more of the equipment units 214, 216, 218 may be connected to multiple communication links. The monitors 214, 216, 218 are further configured to monitor data messages read from the interconnect circuit 206 to determine whether these messages contain data values indicative of a change in the entity being monitored by the sensor 202.

[0037] In addition to the interconnect circuit 206, the monitoring devices 214, 216, and 218 are also connected to an output module 220. The output module 220 is configured to implement corrective action when one or more of the monitoring devices 214, 216, and 218 determine that the data values included in the data message indicate that a sensor has detected a change in the entity. That is, when the monitoring devices 214, 216, and 218 determine that the data values included in the data message indicate that a sensor has detected a change in the entity, the monitoring devices 214, 216, and 218 are configured to instruct the output module 220 to implement the corrective action. Although FIG. 2 shows the system including one output module 220, the circuit may include any number of output modules 220. Although the output module 220 in FIG. 2 is illustrated as being external to the integrated circuit chip 100, the output module 220 may alternatively be included within the integrated circuit chip. If the circuit includes more than one output module 220, each output module can be configured to implement a different corrective action in response to a determination from one or more monitoring devices 214, 216, 218 that the read data indicates a change in the entity.

[0038] Figure 3 is a detailed embodiment of the integrated circuit chip monitor 214 shown in Figures 1 and 2. Monitor 214 is able to read and monitor data messages from interconnect circuitry 206 without any system software intervention using a device of hardware components. These hardware components include a processor 302 connected to both a data store 304 and a communicator 306.

[0039] The processor 302 of the monitoring device 214 is configured to calculate a first hash value for a first message received from the interconnect circuitry 102. The hash store 304 is memory storage configured to store a plurality of previous hash values, each corresponding to a message read from the interconnect circuitry prior to the first message. The processor 302 is further configured to compare the at least one hash value calculated for the first message with one or more of the previous hash values stored in the hash store 304. If a difference between the first hash value and at least one of the previous hash values stored in the hash store exceeds a predetermined threshold identified by the processor 302, the processor 302 calculates a hash value for the first message. above If the processor 302 determines that the motor is rotating, the processor outputs a signal to the communicator 306. Upon receiving this signal, the communicator 306 is then configured to instruct the output module 220, which is connected to the integrated circuit, to take corrective action.

[0040] In this embodiment, the processor 302 of the monitoring device 214 further includes a data buffer 308 connected to a hash calculator 310. In addition to the data buffer 308, the hash calculator 310 is also connected to both a current hash selector 312 and a comparator 314. Both the current hash selector 312 and the comparator 314 are connected to a hash store 304. The comparator 314 is further connected to a communicator 306. Alternatively, the present disclosure can be implemented entirely in suitable hardware, such as a DNA machine encompassed by or coupled to multiple finite state machines. For example, the finite state machines can be configured to perform the following steps: obtain image data, calculate a hash value, obtain the hash value from memory, compare the hash values, and output a trigger event if there is a difference. For example, an SDL state machine may be used. The finite state machine can include a programmable logic device, a programmable logic controller, logic gates, and flip-flops or relays.

[0041] The data buffer 308 is configured to receive data messages read from the interconnect circuitry 206 and store these received data messages for a short period of time. The data buffer 308 then forwards the received data messages to the hash calculator 310. As outlined above, the first data message received by the data buffer 308 is the data message read from the interconnect circuitry 206 before it is received by and written to the memory 204 connected to the integrated circuit chip 100. Thus, the data buffer 308 receives the first data message before it is read from the memory 204 by one or more of the core devices 208, 210, 212.

[0042] The data buffer 308 may be a predetermined size. In one example, the size of the data buffer 308 is the same as the size of a data message received by the buffer. In other words, the size of the data buffer 308 is the same as the size of a first message received by the buffer. Thus, the data buffer 308 is configured to store only the data message recorded in one clock cycle. In an alternative example, the size of the data buffer 308 is the size of only a subset of the data values contained in the first data message. In this alternative example, the size of the buffer corresponds to the size of the hash calculated by the hash calculator 310. Thus, the data buffer 308 is configured to store only a subset of the data values contained in one data message in any one clock cycle. Hash size is described in more detail below.

[0043] Hash calculator 310 is configured to calculate one or more data hashes or hash values for each data message received from data buffer 308. Each data hash includes a hash value representing a data value from the message used to calculate the hash. As noted above, each data message received from data buffer 308 includes one or more distinct data values. Thus, each data message may include multiple data values. In one example, hash calculator 310 is configured to calculate one data hash for all data values of a received data message. In an alternative example, hash calculator 310 is configured to calculate multiple data hashes for each received data message. In this alternative example, for a first data message, each hash value is calculated for a subset of the data values of that data message. The subset may include one or more data values from the first data message. That is, each hash value calculated for a first data message represents a portion of that data message.

[0044] Each data hash calculated by hash calculator 310 has a predetermined hash size, where the hash size indicates the number of data values used to calculate the hash. The hash size is configurable by an operator of the integrated circuit chip. In one example, the hash size is configurable by an operator at the time of instantiation of integrated circuit chip 100. In an alternative example, the hash size is dynamically configurable, i.e., the hash size is configurable during operation of integrated circuit chip 100.

[0045] In an example where the sensor is an image sensor, each data message read from the interconnection circuit 206 is an image frame, a pixel line, or other subdivision of an image frame. In other words, each data message is at least a part of an image frame. Therefore, each data value of a given message may be a pixel of the image frame. Thus, in one example, the hash calculator 310 is configured to calculate a data hash for all pixels of the first image frame. In an alternative example, the hash calculator 310 is configured to calculate a data hash for each of a predetermined subset of pixels of the first image frame. The subset of pixels can be determined by the transfer method used by the image sensor to transfer pixel data off-chip. For example, in the case of an image frame containing n×m pixels, a subset of n×1 or y×x can be transmitted, where y < n and x < m. This depends on the size of the data packet that can be handled by the integrated chip.

[0046] In an example where the hash calculator 310 is configured to calculate two or more data hashes for each data message received from the data buffer 308, the hash calculator 310 can form a rolling input or a non-overlapping input using the data values of the data message to form these hash values. A rolling input is an input in which there are significant overlaps in the data values used to calculate a plurality of consecutive hashes. That is, the hash calculated for each data message represents an overlapping subset or part of the data of that message. An example of a series of data hashes obtained from a rolling input can be H1 = f(X1, X2 ··· X N ) H2 = f(X2, X3 ··· X N+1 ) H3 = f(X3, X4 ··· X N+2 ) expressed as follows.

[0047] In the above representation, H1, H2, and H3 represent the data values X1, X2, . . . X in the first data message Y1. N+2 Each represents a data hash value obtained from a subset of H. In the above example, the input to data hash H1 differs from the input to data hash H2 by one data value. Similarly, the input to data hash H2 differs from the input to data hash H3 by one data value. In an alternative example, the inputs between consecutive data hashes may differ by a greater number of data values.

[0048] A non-overlapping input is an input such that there is no overlap between the data values used to form multiple successive hash values. That is, the hash value calculated for each data message represents the non-overlapping portions of that message. An example of a sequence of data hashes obtained from a non-overlapping input is: H1=f(X1,X2...X N ) H2=f(X N+1 ,X N+2 X 2N ) H3=f(X 2N+1 ,X 2N+2 X 3N ) It can be expressed as follows.

[0049] In the above representation, H1, H2, and H3 represent the data values X1, X2, . . . X in the first data message Y1. 3N Each represents a data hash value obtained from a subset of

[0050] Hash store 304 is a memory store configured to store multiple previous hash values calculated by hash calculator 310 prior to receipt of the first data message. Hash store 304 includes multiple entries 316, 318, 320 for storing previous hash values calculated by hash calculator 310.

[0051] Each previous hash value stored in hash store 304 corresponds to a data message read from interconnect 102 by monitor 214 prior to the first data message being received by data buffer 308. For each hash value calculated for the first data message, the hash store stores previous hash values that were calculated for the same subset of data values as used to calculate the one or more hash values for the data message, but in a previous clock cycle. For example, the first data message recorded by sensor 202 in the first clock cycle T may be denoted by reference Y. 1,T The first hash value calculated by the hash calculator 310 from this first data message can be represented by the reference H 1,T Therefore, entries 316, 318, and 320 represent the first data message Y 1,T Data message Y recorded before 1,T-1 ,Y 1,T-2 ,···Y 1,T-N We can store the previous hash values corresponding to H. 1,T-1 ,H 1,T-2 ,···H 1,T-N For example, the first previous hash value H 1,T-1 may be stored in the first entry 316. The hash store may store hash values for any number of clock cycles prior to the data being received.

[0052] In an alternative example, the first hash value H 1,T is calculated by the hash calculator 310 from the first message Y 1,T The data value contained within X 1,T ,X 2,T ,···X N,T Note that it may be calculated for only a subset of .

[0053] First data message Y 1,T The first hash value H for 1,T is calculated by the hash calculator 310, this first hash value H 1,T is transmitted to both the current hash selector 312 and the comparator 314. The current hash selector 312 is configured to receive a hash value for the data hash calculated by the hash calculator 310 and use that hash value to look up a corresponding previous hash value stored in the hash store 304. The corresponding previous hash value is a hash value calculated for the first data message or a subset of data used to form part of the first data message that was used to calculate the first hash value, but in a different, previous time window. Thus, for the first data message Y 1,T The first hash value H calculated for 1,T For, the corresponding previous hash value is H 1,T-1 ,H 1,T-2 and so on. That is, one or more previous hash values H 1,T-1 ,H 1,T-2 etc., the first hash H 1,T The first message Y used to generate 1,T Once the corresponding previous hash values in hash store 304 are identified, these values are sent to comparator 314.

[0054] The comparator 314 is connected to both the hash store 304 and the hash calculator 310. Thus, the comparator receives the first hash value H calculated by the hash calculator 310. 1,T and the corresponding previous hash value from hash store 304. Comparator 314 is further configured to receive the first hash value H received from hash calculator 310. 1,T is configured to compare the first hash value H with one or more corresponding previous hash values stored in the hash store 304. 1,Tis the first data message Y in the first clock cycle T 1,T When expressing a hash value for a 1,T-1 Then, H 1,T the previous hash value H 1,T-2 From H 1,T-M where M is the first hash value H 1,T M is the total number of previous hash values to be compared with the first hash value. The number M of previous hash values to be compared with the first hash value is configurable by an operator of the integrated circuit chip. In one example, M is configurable at the time of instantiation of the integrated circuit chip 100. In an alternative example, M is dynamically configurable, i.e., M is configurable during operation of the integrated circuit chip 100.

[0055] The comparator 314 calculates the first data hash value H 1,T hash value of and each previous hash value H 1,T-2 H 1,T-M The comparator 314 is configured to compare the difference between the first hash value H and the second hash value H with a predetermined threshold W1. The predetermined threshold value may be stored within the monitor 214. In one example, the predetermined threshold value is stored in additional memory (not shown) of the monitor. Alternatively, this additional memory may be included in the hash store 304 or in the memory of the comparator 314. Furthermore, the predetermined threshold value may be stored in an alternative location within the processor 302, or may be stored generally within the monitor 214. The comparator 314 further compares the difference between the first hash value H 1,T and the previous hash value H stored in the hash store. 1,T-2 H 1,T-M The difference between at least one of the above That is, the comparator is configured to determine whether |H1-H 1,T-Z | > W1, where Z=1, 2, M.

[0056] In one example, the monitoring device 214 is configured to perform a corrective action when it is determined that the difference between the first hash value H 1,T and one previous hash value exceeds a predetermined threshold value W1. In this example, the first hash value is compared only with the hash value immediately preceding it. |H1 - H 1,T-1 | >W is 1, a corrective action is performed. When it is determined that the hash value exceeds the predetermined threshold value W1, the sensor 202 can detect a change in the entity being monitored by the sensor 202, thereby enabling an alarm, a corrective action, or other notification to be made.

[0057] In an alternative example, the monitoring device 214 is configured to perform a corrective action when it is determined that the difference between the first hash value H 1,T and a plurality of previous hash values exceeds a predetermined threshold value W1. In this example, the number P of previous hash values such that the difference between the first hash value and each previous hash value exceeds the predetermined threshold value is predefined by the monitoring device 214. The number P of previous hash values is defined as a fraction of M, where M is the total number of previous hash values to be compared with the first hash value H 1,T . Thus, 1 < P < M. As an example, P may be equal to M / 2. Thus, when the number of previous hash values such that |H1 - (H 1,T-Z )| > W1 is more than M / 2, a corrective action is performed.

[0058] The first hash value H 1,TIf it is determined that the difference between the hash value W1 and at least one of the previous hash values stored in the hash store 304 exceeds a predetermined threshold W1, the comparator 314 outputs a signal to the communicator 306. The communicator 306 is configured to communicate with one or more output modules 220 external to the monitoring device 214. As such, the communicator 306 is responsible for sending instructions to the output modules. The instructions indicate corrective actions or notifications to be implemented by the system.

[0059] The corrective actions commanded by the communicator 306 may include one or more of the following non-exhaustive list of actions: notifying a user that the entity has changed (certain changes may result in specific notifications); activating a mechanism configured to repair the entity; outputting a warning signal to another component of the integrated circuit 100; outputting a hardware event; outputting an interrupt signal; and outputting a message to an external core device 208, 210, 212.

[0060] The current hash selector 312 further selects the first hash value H 1,T into the hash store 304. Once inserted into the hash store 304, the first hash value becomes the previous hash value. The previous hash value can then be used for comparison with future hash values calculated from subsequent data messages read from the interconnect circuitry 206. In one example, the first hash value H 1,T is compared by the comparator 314 with this first hash value H 1,T is inserted into the hash store by the current hash selector 312 while being compared with the corresponding previous hash value of 1,T This first hash value H 1,T , and stores the hash value in hash store 304. In an alternative example, the first hash value H 1,Tis processed by the comparator 314 and then stored in the hash store. In other words, the comparison and storage of the first hash value may be performed in the same clock cycle or may be performed in multiple consecutive clock cycles.

[0061] The integrated circuit chip shown in Figure 2 includes three monitors 214, 216, and 218. As noted above, a chip may alternatively include any number of monitors. In the case of a chip including multiple monitors, each of these monitors may receive a first data message Y 1,T , X1, X2, . . . X2. N The first hash value H 1,T The second monitoring device can calculate and compare the subsets X2, X3, X N+1 the second hash value H 2,T The third monitoring device can calculate and compare the subsets X3, X4, X N+2 the first hash value H 1,T can be calculated and compared. In this manner, each monitoring device can simultaneously process data values for data messages for its respective subset of monitoring devices, further reducing the latency associated with processing sensor data.

[0062] FIG. 4 is a schematic diagram of an alternative exemplary system for monitoring data messages using an integrated circuit chip device. Similar to FIG. 2, the system includes a sensor 402 connected to an external memory 404 by an SoC 100. The SoC further includes a system circuit 102 and a monitoring circuit 104 configured to monitor the system circuit 102. The system circuit 102 includes core devices 406, 408, 410, 418, and 420, which correspond to core devices 208, 210, and 212 of FIG. 2. The monitoring circuit includes monitoring devices 422, 424, and 426, which correspond to monitoring devices 214, 216, and 218 of FIG. 2. The core device 420 is an output module and will be described in more detail below. The core device 418 is internal or on-chip memory. In one example, the on-chip memory 418 can be used to store data messages recorded by the sensor 402, in addition to the external memory 404. In another example, data messages from the sensor 402 may be written to only one of the external memory 404 or the on-chip memory 418. The memory to which the data message is written may be selected depending on the characteristics of that memory. Examples of characteristics that may determine whether a data message is written to a given memory include the amount of storage that the memory contains, the latency associated with writing the data message to that memory (which depends on how far the memory is from the sensor), and how much bandwidth the memory supports.

[0063] 4, the interconnect circuitry of the system circuit 102 further includes a memory controller 412 and a sensor controller 416. The sensor controller 416 is connected to both the sensor 402 and the SoC interconnect 414. The sensor controller 416 is responsible for sending instructions to the sensor 402 and controlling the performance of the sensor. The sensor controller 416 is also responsible for receiving data messages recorded by the sensor 402 and transmitting those messages to other components of the system circuit 102 via the SoC interconnect 414. The memory controller 412 is connected to both the memory 404 and the SoC interconnect 414. The memory controller 412 is responsible for both receiving data messages from the SoC interconnect to be written to or stored in memory and transmitting the stored data messages to the core device via the SoC interconnect 414.

[0064] The SoC interconnect 414 is connected to the core devices 406, 408, 410, 418, and 420, the sensor controller 416, and the memory controller 412. The SoC interconnect 414 is therefore responsible for transferring data messages between the sensors 402 (via the sensor controller 416), the memory 404, and the core devices. The topology of the SoC interconnect is SoC dependent but may include any one or combination of the following types of networks for carrying communications around the system circuitry: a bus network, a ring network, a tree network, or a mesh network. In one example, the SoC interconnect 414 is a data bus.

[0065] Both the system circuit 102 and the monitoring circuit 104 shown in FIG. 4 include one or more output modules 420, 432, and 436. The output modules are configured to implement corrective actions in response to signals received by either the core devices 406, 408, and 410 or the monitoring devices 422, 424, and 426. The output module 420 is a safety controller. The safety controller 420 is configured to implement one or more actions to ensure the safety of a system in which the integrated circuit chip is implemented. For example, if the integrated circuit chip 100 is implemented inside a moving vehicle, the safety controller can be configured to apply brakes to the system. The output module 432 is an analysis CPU. The analysis CPU is configured to perform further analysis on data output by either the core devices 406, 408, and 410 or the monitoring devices 422, 424, and 426 to determine why a sensor failure occurred.

[0066] The analysis CPU is connected to one or more monitoring devices 422, 424, 426 by a message engine 428 and an internal communicator 430. The message engine 428 is configured to connect all components of the monitoring circuit together and send messages between these components. Like the SoC interconnect 414, the topology of the message engine is also SoC-dependent but can include any one or combination of the following types of networks for conveying communications around the system circuitry: bus network, ring network, tree network, or mesh network. In one example, the message engine 428 is a data bus. The internal communicator 430 is configured to relay messages between the analysis CPU and the message engine 428.

[0067] The output module 436 is an external debugger. The external debugger 436 is used to troubleshoot defects or problems within the integrated circuit chip 100 or within the surrounding system that are causing the sensor to fail. Strategies used by the external debugger 436 may include interactive debugging, control flow analysis, unit testing, integration testing, log file analysis, application-level or system-level monitoring, memory dumps, and / or profiling. The external debugger 436 is connected to the message engine 428 by an external communicator 434. The external communicator 434 is configured to relay messages between the external debugger 436 and the message engine 428.

[0068] 5 is a flowchart of an exemplary method for monitoring messages from sensors using an integrated circuit chip device such as that shown in FIG. 2 or 4. As discussed above, the data messages include measured data from sensors 202, 204 connected to the integrated circuit chip 100. The method begins at step 502, where a first data message Y is received from the interconnect circuitry 206 of the integrated circuit using one or more monitoring devices 214, 216, 218. 1,T The first data message Y 1,T is recorded by the sensor 202 in the first clock cycle T. The first data message Y 1,T is received by a data buffer 308 located within the monitoring device's processor 302. In one example, the data message Y 1,T contains only one subset of the data values X1. In an alternative example, the data message contains multiple subsets of the data values.

[0069] In step 504, the hash calculator 310 of the processor 302 of the monitoring device 214 calculates the hash value of the first data message Y 1,T hash value H for 1,T As outlined above, the first hash value H 1,Tmay represent all of the data values in the received data message, or alternatively, may represent only a subset of the data values contained in the data message. 1,T Once calculated, the hash value is sent to the current hash selector 312. The current hash selector 312 selects the first hash value H 1,T is used to look up the corresponding previous hash values stored in hash store 304. These corresponding previous hash values are read from hash store 304 by comparator 314.

[0070] In step 506, the comparator 314 compares the first hash value H 1,T , by one or more previous hash values H stored in the hash store 304. 1,T-1 H 1,T-M As described above, each previous hash value stored in the hash store corresponds to a data message read from the interconnect circuitry 206 prior to the first data message. In one example, the first hash value is compared to one previous hash value H 1,T-1 In an alternative example, the first hash value is compared only with multiple consecutive previous hash values H 1,T-1 ~H 1,T-M Then, the first hash value H 1,T and one or more previous hash values H 1,T-1 H 1,T-M The difference between is compared to a predetermined threshold stored within the monitor 214 .

[0071] In step 508, the comparator 314 compares the first hash value H 1,T and one or more previous hash values H 1,T-1 ,H 1,T-2 H 1,T-MThe difference between the first hash value and at least one of the one or more previous hash values is compared with a predetermined threshold W1. If the comparator 314 determines that the difference between the first hash value and at least one of the one or more previous hash values is greater than the predetermined threshold W1, the method proceeds to step 510, where the communicator 306 commands the output module 220 of the integrated circuit to implement corrective action. In response to this command, the output module 220 implements the corrective action. If the comparator 314 determines that the difference between the first hash value and one of the one or more previous hash values is less than or equal to the predetermined threshold W1 (i.e., the difference is greater than or equal to W1), no command is sent by the communicator 306. Thus, the one or more output modules 220 do not implement corrective action. The method instead skips step 510 and proceeds to step 512.

[0072] If, in step 510, the comparator 314 determines that the difference between the first hash value and at least one of the previous hash values exceeds a predetermined threshold, the comparator outputs a signal to the communicator 306. Upon receiving this signal, the communicator 306 commands the implementation of corrective action. That is, the communicator 306 transmits one or more instructions to one or more modules external to the monitoring device 214. The instructions indicate the corrective action to be implemented by the system.

[0073] In step 512, the first hash value H 1,T is stored in hash store 304. Thus, the first hash value becomes the previous hash value and can be compared with subsequent successive hash values to determine whether those successive hash values indicate that a change has occurred in the entity being monitored by sensor 202. The method then proceeds to step 514, where Y 1,T is used as an input to step 502. 1,T+1 That is, Y 1,TThe data message recorded by the sensor in the clock cycle following the clock cycle when Y was recorded is received by the data buffer 308. In step 504, the subsequent data message Y 1,T+1 is used by hash calculator 310 to generate a new hash value H 1,T+1 is calculated and the new hash value is compared to the previous hash value H 1,T ,H 1,T-1 ,H 1,T-2 H 1,T-M It is compared with etc.

[0074] The method illustrated in FIG. 5 is suitable when hash values calculated by hash calculator 310 are similar for similar data values. If hash values for similar data values are similar, a threshold W1 can be used to determine the extent to which the hash values change over successive clock cycles. FIG. 6 is a flowchart illustrating an alternative exemplary method of monitoring messages from a sensor using an integrated circuit chip device such as that shown in FIG. 2 or FIG. 4, in which similar hash values are not generated for similar data values. That is, hash values calculated by hash calculator 310 for similar data values may be significantly different from each other, in which case the difference between the hash values is unrelated to the difference between the data values themselves. In this example, a consistent threshold indicating the extent to which hash values change over successive clock cycles is not possible. Instead, identification of a change in an entity monitored by a sensor can be determined by observing the similarity between hash values. That is, a first hash value may differ from one or more previous hash values, indicating that a change in the entity has occurred.

[0075] Steps 602-606 of Figure 6 correspond to steps 502-506 of Figure 5. That is, in step 602, a first data message Y is received from the interconnect circuit 206 of the integrated circuit using one or more monitoring devices 208, 210, 212. 1,T In step 604, the hash calculator 310 of the processor 302 of the monitoring device 214 reads the first data message Y 1,T The first hash value H for 1,T Calculate.

[0076] In step 608, the comparator 314 compares the first hash value H 1,T and the previous hash value H 1,T-Z Compare the difference between H, where Z = 1, 2...M. 1,T H 1,T-Z is not equal to, or in other words |H 1,T -H 1,T-Z If the comparator 314 determines that |≠0, the method proceeds to step 610, where the communicator 306 commands the output module 220 of the integrated circuit to take corrective action. In response to this command, the output module 220 takes corrective action. H 1,T H 1,T-Z or in other words |H 1,T -H 1,T-Z If the comparator 314 determines that |=0, then no instruction is sent by the communicator 306. Thus, no corrective action is taken by the one or more output modules 220. The method instead skips step 610 and proceeds to step 612.

[0077] Steps 610-614 in Figure 6 correspond to steps 510-514 in Figure 5. That is, in step 610, the communicator 306 instructs the output module 220 of the integrated circuit to take corrective action. In step 612, the first hash value H 1,T is stored in the hash store 304. In step 614, Y 1,T is used as an input to step 602.1,T+1 can be replaced by

[0078] As mentioned above, in one example, the comparator 314 determines whether a difference between the first hash value and each previous hash value over a predetermined number P of previous hash values is satisfied, where P is defined as a fraction of M. M is the number of times the comparator 314 compares the first hash value H 1,T In a further implementation of this example, the comparator 314 compares |H1-H 1,T-Z |>W1 is defined as a number of hash values P1, P2, P N Each predetermined number P1 corresponds to a confidence value that indicates how certain the monitoring device is that a sensor failure has occurred. That is, the comparator 314 can derive a first predetermined number P1, where P1=M / 2. Therefore, |H1-H 1,T-Z If the number of previous hash values where |>W1 is greater than P1, the comparator can determine that it is "highly confident" that a change has occurred in the entity monitored by the sensor. Comparator 314 can further derive a second predetermined number P2, where P2=M / 4. Thus, |H1-H 1,T-Z If the number of previous hash values where |>W1 is less than P1 but more than P2, then comparator 314 can determine that it is "somewhat confident" that a change has occurred in the entity being monitored by the sensor. Comparator 314 can be used to determine whether a change has occurred in any amount of predetermined numbers P1, P2, P N where each predetermined number is associated with a confidence value indicating a different probability that a change in the entity being monitored by the sensor has occurred. The predetermined numbers and corresponding confidence levels may be stored in a memory within the monitoring device 214.

[0079] As mentioned above, in one example, the sensor connected to the integrated circuit chip is a camera, and each data message read by one or more monitoring devices of the integrated circuit chip is an image frame or a portion thereof captured in a first clock cycle T. That is, a first hash value H1 and at least one corresponding previous hash value H stored in a hash store. 1,T-1 ,H 1,T-2 If the comparator 314 of the processor 302 of the monitoring device 214 determines that the difference between H1, H2, etc., exceeds a predetermined threshold W1, then this determination indicates where or what change occurred in the monitored entity in the portion of the frame used to calculate H1.

[0080] In one implementation of the example where the sensor is a camera, the integrated circuit chip described above can be inserted into an industrial environment, such as a production line. In this example, the one or more output modules 220 include a manufacturing safety controller that is alerted in addition to or instead of an operator when a change in the operation or output of the production line occurs. Thus, one or more corrective actions that can be commanded by the communicator 306 of the monitoring device 214 can include notifying an operator that an error has been detected in the production line, shutting down a subsystem that relies on a particular output of the production line to operate, activating an alarm, or notifying the manufacturing safety controller of the change so that it can determine the specific action to be taken.

[0081] 7 is a schematic diagram of an exemplary implementation of one embodiment. A reactor chamber 701 is monitored by four image cameras 702a, 702b, 702c, and 702d, which are positioned to observe a floor 703 of the reactor chamber 701. Each camera 702a, 702b, 702c, and 702d monitors a designated area of the floor 703 for any indications of a problem with the integrity of the floor 703, and potentially the integrity of the reactor itself. During normal use, each camera 702a, 702b, 702c, and 702d relays image frames that are substantially identical. Thus, as described above with respect to FIG. 5, a first hash value H 1,T and one or more previous hash values H 1,T-1 ,H 1,T-2 H 1,T-M When the difference between the measured values of the floor 703 and the measured values of the rod covers 705 is compared with a predetermined threshold W1 for each camera 702a, 702b, 702c, and 702d, the difference is found to be well below the predetermined threshold W1. However, as part of the monitoring process, one camera 702a of the cameras picks up a crack 704 between the rod covers 705 in an area R of the floor 703 assigned to that camera 702a (as highlighted in the drawing). In the nuclear industry, the risk to safety from reactor problems is so high that notification of a problem on a very short time scale of microseconds or less is required. The first image frame captured by camera 702a shows an expected image of the floor 703, while a second, subsequent image frame shows the early stages of the crack appearing. The hash value H for the first image frame 1,T is the hash value H 1,T One or more previous hash values H to be compared with 1,T-1 ,H 1,T-2 H 1,T-M while the hash value H for the second image frame is 1,T+1 are different and the stored hash value H 1,T-1 ,H 1,T-2 H 1,T-Mexceeds a predetermined threshold W1, which causes the comparator 314 to output a signal to the communicator 306, which in turn can send one or more commands to one or more modules external to the camera 702a, thereby generating an alarm.

[0082] In one example, the sensor is an image sensor, but alternatively, any type of sensor capable of recording continuous data may be connected to the integrated circuit chip. Examples of alternative sensors from which messages may be monitored include temperature sensors, proximity sensors, accelerometers, pressure sensors, flow sensors, humidity sensors, or touch sensors. For example, the sensor may be a thermostat, and the integrated circuit chip may be used to monitor the temperature recorded by the thermostat.

[0083] The systems and methods described herein significantly reduce the amount of time it takes to identify a change in an entity when compared to corresponding systems that rely on software implementations. That is, the calculation and comparison of hash values by a monitoring device can be performed in just a few hardware clock cycles. Once a change in an entity is identified by one or more monitoring devices, these devices can ensure that an appropriate output module is instructed to take corrective action using a low-latency mechanism to further reduce the time delay.

[0084] The above-described SoC is suitably incorporated into a computing-based device. The computing-based device may be an electronic device. Suitably, the computing-based device includes one or more processors for processing computer-executable instructions for controlling the operation of the device to implement the methods described herein. The computer-executable instructions may be provided using any computer-readable medium, such as a memory. Some of the methods described herein may be implemented by software in machine-readable form on a tangible storage medium. Software may be provided to the computing-based device to implement some of the methods described herein.

[0085] The above description describes the system circuitry and the monitoring circuitry as being included on the same SoC. In alternative implementations, the system circuitry and the monitoring circuitry are included across two or more integrated circuit chips of an MCM. In an MCM, multiple integrated circuit chips may be stacked or positioned side-by-side on an interposer substrate. Some system circuits may be located on one integrated circuit chip of the MCM, and other system circuits may be located on different integrated circuit chips of the MCM. Similarly, the monitoring circuitry may be distributed across two or more integrated circuit chips of the MCM. Thus, the methods and apparatus described above in the context of an SoC also apply in the context of an MCM.

[0086] The applicant discloses each individual feature described herein alone, and any combination of two or more such features to the extent that it can be realized as a whole based on the present specification in light of the general knowledge of those skilled in the art, without limiting the scope of the claims, regardless of whether such feature or combination of features solves the problems disclosed herein. The applicant indicates that any such individual feature or combination of features can be included in an aspect of the present disclosure. In view of the foregoing description, it will be apparent to those skilled in the art that various modifications may be made within the scope of the present disclosure.

Claims

1. 1. A method of monitoring messages from a sensor using an integrated circuit, comprising: the message includes data measured by the sensor; The method comprises: reading a first message from interconnect circuitry of the integrated circuit, the interconnect circuitry connecting the sensor to one or more core devices configured to process the first message; calculating a first hash value for the first message; comparing the first hash value to one or more previous hash values stored in a hash store, each previous hash value corresponding to a message read from the interconnect circuitry prior to the first message; performing corrective action if a difference between the first hash value and at least one of the previous hash values stored in the hash store exceeds a predetermined threshold; Including, method.

2. The method of claim 1 , further comprising storing the first hash value in the hash store.

3. The method of claim 2 , wherein the first hash value is stored in the hash store at the same time that the first hash value is compared to the previous hash value.

4. The method of claim 1 , further comprising buffering the first message before it is read by the one or more core devices.

5. 5. The method of claim 4, wherein the size of the buffer corresponds to the number of data values used to calculate a data hash that includes the first hash value.

6. comparing the first hash value with M previous hash values; taking corrective action if a difference between the first hash value and the P previous hash values exceeds the predetermined threshold; where 1<P<M.

6. The method according to any one of claims 1 to 5.

7. 7. The method of claim 6, further comprising deriving confidence values corresponding to the P previous hash values where a difference between the first hash value and each of the P previous hash values exceeds the predetermined threshold.

8. The method of claim 6 , wherein M is dynamically configurable.

9. 9. The method of claim 1, wherein corrective action is implemented if the first hash value is not the same as at least one of the previous hash values stored in the hash store.

10. further comprising: computing a plurality of hash values for the first message, each hash value of the plurality of hash values representing a portion of the first message; each hash value of the plurality of hash values for the first message is compared to one or more previous hash values corresponding to the portion of the first message; 10. The method according to any one of claims 1 to 9.

11. 11. The method of claim 10, wherein the multiple hash values calculated for the first message represent overlapping portions of the first message.

12. The method of claim 10 , wherein the multiple hash values calculated for the first message represent non-overlapping portions of the first message.

13. The corrective action is: notifying a user that the sensor data is unreliable; and deactivating components of the integrated circuit that rely on data obtained from the sensor; activating a mechanism configured to repair the sensor; one or more of 13. The method according to any one of claims 1 to 12.

14. 14. The method of claim 1, wherein the corrective action comprises outputting a warning signal to another component of the integrated circuit.

15. The corrective action is: outputting hardware events; outputting an interrupt signal, or outputting a message to an external core device; 15. The method of any one of claims 1 to 14, comprising:

16. the sensor is an image sensor; each message read from the interconnect circuitry is at least a portion of a frame captured by the image sensor; 16. The method according to any one of claims 1 to 15.

17. the sensor is an image sensor; each message read from the interconnect circuit is a frame captured by the image sensor; 16. The method according to any one of claims 1 to 15.

18. 18. The method of claim 1, wherein the integrated circuit is a system on a chip.

19. 19. The method of claim 1, wherein the interconnect circuit is a data bus.

20. 1. An integrated circuit chip for monitoring messages from a sensor, comprising: the message includes data measured by the sensor; The integrated circuit chip comprises: an interconnection circuit connecting the sensor and a core device configured to process the first message; a monitoring device configured to read the first message from the interconnect circuit; a hash store for storing a plurality of previous hash values, each hash value corresponding to a message read from the interconnect circuitry prior to the first message; an apparatus configured to calculate a first hash value for the first message and compare the first hash value to one or more of the previous hash values stored in the hash store; Including, the integrated circuit is configured to perform a corrective action if a difference between the first hash value and at least one of the previous hash values stored in the hash store exceeds a predetermined threshold. Integrated circuit chip.

21. 21. The integrated circuit chip of claim 20, wherein the device is a processor or a set of finite state machines.

Images