Hardware-based sensor analysis

Abstract

A method for monitoring messages from a sensor using an integrated circuit, the messages including data measured by the sensor, the method comprising: reading a first message from an interconnect circuit of the integrated circuit connecting the sensor to one or more core devices configured to process the messages; calculating a first hash value for the first message; comparing the first hash value with one or more previous hash values ​​stored in a hash memory, each previous hash value corresponding to a message read from the interconnect circuit prior to the first message; and performing a correction action if the difference between the first hash value and at least one previous hash value stored in the hash memory is less than a predetermined threshold.

Description

Technical Field

[0001] This invention relates to using monitoring circuitry within a System-on-Chip (SoC) or Multi-chip Module (MCM) to monitor messages from sensors. Background Technology

[0002] Integrated circuit chips can be used to monitor data messages recorded by sensors. The recorded data messages can be used to detect sensor malfunctions that prevent the sensor from fully or partially updating its records. Such sensor malfunctions result in problems that cannot be recorded within the system (where the sensor is used). To detect sensor malfunctions in a known integrated circuit chip layout, these messages are monitored by first writing the data messages recorded by the sensor to a memory connected to the integrated circuit chip. Once the messages have been written to the memory, they can be read from the memory and processed by software included in one or more processing modules on the chip.

[0003] An example of a sensor that can be used to indicate a problem within a system is an image sensor, such as a camera. When the sensor is an image sensor, each data message recorded by it is an image frame captured within a given time window. Whether a problem has occurred is determined by comparing image frames captured in subsequent time windows with monitoring software. Strong similarities among image frames recorded by the sensor over time can be used to indicate sensor failure.

[0004] The requirement to store data messages in memory before they can be processed by monitoring software introduces a significant time delay to integrated circuit chips when detecting sensor malfunctions. This is problematic for implementations that heavily rely on sensor data to identify problems within the system (in which the sensor is used) and thus to execute responses to those problems with minimal time delay. For example, in the case of an image sensor, it might be necessary to identify an approaching external object and trigger a response to its proximity.

[0005] There is a need to reduce the time spent monitoring data messages recorded by sensors. Summary of the Invention

[0006] According to a first aspect, a method is provided for monitoring messages from a sensor using an integrated circuit, the messages including data measured by the sensor, the method comprising: reading a first message from an interconnect circuit of the integrated circuit connecting the sensor to one or more core devices configured to process the message; calculating a first hash value for the first message; comparing the first hash value with one or more previous hash values ​​stored in a hash memory, each previous hash value corresponding to a message read from the interconnect circuit prior to the first message; and performing a correction action if the difference between the first hash value and at least one previous hash value stored in the hash memory is less than a predetermined threshold.

[0007] The method may also include storing the first hash value in a hash memory.

[0008] The first hash value can be stored in the hash memory while comparing it with the previous hash value.

[0009] The method may also include buffering the first message before it is read by one or more core devices.

[0010] The size of the buffer can correspond to the number of data values ​​used to calculate the data hash including the first hash value.

[0011] The method may further include comparing a first hash value with M previous hash values, and performing a correction action if the difference between the first hash value and P previous hash values ​​is less than a predetermined threshold, where 1 < P < M.

[0012] The method may further include deriving confidence values ​​corresponding to P previous hash values, wherein the difference between the first hash value and each of the P previous hash values ​​is less than a predetermined threshold.

[0013] M can be dynamically configured.

[0014] If the first hash value is the same as at least one previous hash value stored in the hash memory, a correction action can be performed.

[0015] The method may further include calculating a plurality of hash values ​​for a first message, each of the plurality of hash values ​​representing a portion of the first message, and each of the plurality of hash values ​​of the first message may be compared with one or more previous hash values ​​corresponding to that portion of the first message.

[0016] Multiple hash values ​​calculated for the first message can represent the overlapping parts of the first message.

[0017] Multiple hash values ​​calculated for the first message can represent the non-overlapping parts of the first message.

[0018] Interconnect circuits can also connect sensors to memory, and the memory can store messages transmitted by the sensors.

[0019] The calibration action can be one or more of the following: informing the user that the sensor data is unreliable, disabling components of the integrated circuit that depend on the data obtained from the sensor, and activating a mechanism configured to repair the sensor.

[0020] The calibration action may include outputting an alarm signal to another component of the integrated circuit.

[0021] The calibration action may include outputting hardware events, outputting interrupt signals, or outputting messages to external core devices.

[0022] The sensor can be an image sensor, and each message read from the interconnect circuit can be a frame captured by the image sensor.

[0023] Integrated circuits can be systems-on-a-chip.

[0024] The interconnect circuit can be a data bus.

[0025] According to a second aspect, an integrated circuit chip is provided for monitoring messages from a sensor, the messages including data measured by the sensor, the integrated circuit chip comprising: an interconnect circuit connecting the sensor to a core device configured to process the messages; a monitoring device configured to read a first message from the interconnect circuit; a hash memory for storing a plurality of previous hash values, each hash value corresponding to a message read from the interconnect circuit prior to the first message; and a processor configured to: calculate a first hash value for the first message; and compare the first hash value with one or more previous hash values ​​stored in the hash memory; wherein the integrated circuit is configured to: perform a correction action if the difference between the first hash value and at least one previous hash value stored in the hash memory is less than a predetermined threshold. Attached Figure Description

[0026] The invention will now be described by way of example with reference to the accompanying drawings. In the drawings:

[0027] Figure 1 This is a schematic diagram of an exemplary integrated circuit chip device;

[0028] Figure 2 This is a schematic diagram of an exemplary system that uses an integrated circuit chip device to monitor data messages;

[0029] Figure 3 yes Figure 2 Detailed embodiment of the monitoring device for the integrated circuit chip device shown;

[0030] Figure 4This is a schematic diagram of an alternative exemplary system for monitoring data messages using an integrated circuit chip device;

[0031] Figure 5 Is it like this? Figure 2 or Figure 4 A flowchart illustrating an exemplary method for monitoring messages from sensors using an integrated circuit chip device; and

[0032] Figure 6 Is it like this? Figure 2 or Figure 4 The flowchart illustrates an alternative exemplary method for monitoring messages from sensors using an integrated circuit chip device. Detailed Implementation

[0033] Figures 1 to 4 These are schematic diagrams of an exemplary system architecture and the components within that architecture. The structure is presented in the form of functional blocks. Some functional blocks used to perform functions well-known in the art are omitted from these diagrams. Figure 5 and Figure 6 It shows the use Figures 1 to 4 The system architecture described is illustrated with flowcharts illustrating methods for monitoring sensor messages. Each flowchart describes the sequence in which the methods of that flowchart can be executed. However, these flowcharts are not intended to restrict the described methods to implementation in the described order. The steps of these methods may be executed in a different order than that described in the flowcharts.

[0034] Figure 1 A general structure of an exemplary monitoring network for SoC 100 is shown. Monitoring circuitry 104 is arranged to monitor system circuitry 102. For example, it is used to detect improper operation of core devices related to security or safety issues.

[0035] Figure 2 An exemplary system 200 is illustrated, comprising an integrated circuit chip 100, such as a SoC, for monitoring data messages received from a sensor 202. These messages are referred to as data messages because they include one or more distinct data values ​​recorded by the sensor. The sensor 202 can be any type of sensor capable of recording data periodically. That is, the sensor 202 can be capable of recording data continuously in regular time windows. Each time window may also be referred to as a clock cycle of the integrated circuit chip. In one example, the sensor 202 is, for example, an image sensor of a camera, and each data message recorded by the sensor is an image frame.

[0036] Like sensor 202, SoC 100 is also connected to memory 204. Therefore, sensor 202 is connected to memory 204 via SoC 100. Figure 2In this context, memory 204 is external memory. External memory is memory that is not included within the SoC 100. In an alternative example, memory 204 may be internal memory and therefore may be included within the SoC 100. Memory 204 is configured to store data messages recorded by sensor 202 and transferred to memory 204 via the SoC 100.

[0037] and Figure 1 Same, Figure 2 The SoC includes system circuitry 102 and monitoring circuitry 104. System circuitry 102 includes interconnect circuitry 206 and one or more core devices 208, 210, 212. Interconnect circuitry 206 connects sensor 202 and memory 204 to core devices 208, 210, 212, and also connects core devices 208, 210, 212 to each other. Therefore, interconnect circuitry 206 enables data transfer between sensor 202, memory 204, and core devices 208, 210, 212. In one example, core devices 208, 210, 212 are master devices. In an alternative example, core devices 208, 210, 212 are a combination of master and slave devices. One or more of core devices 208, 210, 212 are configured to process data messages received from sensor 202. To be able to process data, one or more core devices 208, 210, 212 include appropriate system software. Although in Figure 2 The SoC shown includes three core devices, but any number of core devices can be appropriately integrated into the system circuitry. The SoC interconnects form the SoC's communication backbone, through which the core devices can communicate with each other. This communication is bidirectional.

[0038] A master device is one that initiates requests for the flow of information, such as read / write operations on a network. Examples of master devices are processors, such as DSPs (Digital Signal Processors), video processors, application processors, CPUs (Central Processing Units), and GPUs (Graphics Processing Units). Any programmable processor can be a master device. Other examples of master devices are devices with DMA (Direct Memory Access) capabilities, such as conventional DMA for moving data from one location to another, autonomous coprocessors with DMA capabilities (e.g., encryption engines), and peripheral devices with DMA capabilities (e.g., Ethernet controllers).

[0039] Slave devices are those devices that respond to commands from a master device. Examples of slave devices are on-chip memory, memory controllers for off-chip memory (such as DRAM), and peripheral units.

[0040] The topology of interconnect circuit 206 is dependent on the SoC. For example, the interconnect circuit may include any or a combination of the following types of networks to transmit communication around the system circuitry: bus network, ring network, tree network, or mesh network. In one example, the interconnect circuit is a data bus.

[0041] As described above, the core devices 208, 210, and 212 of system circuit 102 are configured to process data messages recorded by sensor 202. The core devices 208, 210, and 212 use appropriate system software to perform this data processing. To process the data messages, they are first written to memory 204. That is, once the data messages have been written to memory 204, they can be read and processed by one or more core devices 208, 210, and 212. The data can be processed by software included in one or more core devices 208, 210, and 212 by comparing the most recently acquired data value with the previous data values ​​stored in memory 204. In one example, similarity between data values ​​obtained from consecutive data messages indicates a sensor malfunction. That is, sensor 202 may record duplicate data values ​​when the state of the system (on which the sensor is applied) changes. In the example where sensor 202 is an image sensor, detecting duplicate data values ​​recorded by the sensor may indicate that the image sensor includes defective or stagnant pixels.

[0042] The drawback of using system circuitry 102 to process data messages recorded by sensor 202 is that it results in a significant time delay from the time a sensor malfunctions to its detection. Specifically, sensor data must be written into memory 204 before it can be read and processed by software included in core devices 208, 210, and 212. The time spent storing and processing the data messages can be on the order of tens to hundreds of milliseconds. This is too slow for some integrated systems that require faster fault detection mechanisms to allow for a timely response to sensor malfunctions.

[0043] To provide a faster method for monitoring data messages recorded by sensor 202, monitoring circuitry 104 includes one or more monitoring devices 214, 216, and 218. Monitoring devices 214, 216, and 218 are connected to interconnect circuitry 206. Monitoring devices 214, 216, and 218 are configured to read data messages from interconnect circuitry 206. Data messages from interconnect circuitry 206 are messages transmitted by sensor 202 through interconnect circuitry 206 before reaching memory 204. These data messages are read by core devices 208, 210, and 212, rather than extracted from interconnect circuitry 206. Instead of using the system software used by core devices 208, 210, and 212, monitoring devices 214, 216, and 218 are configured to use an arrangement of hardware components to read and subsequently process the data messages.

[0044] Monitoring devices 214, 216, and 218 can be configured to selectively read only data messages transmitted by sensor 202. That is, monitoring devices 214, 216, and 218 can be configured to filter out all messages that are not data messages from the sensor, and all messages transmitted through interconnect circuit 206 (if such messages exist). Monitoring devices 214, 216, and 218 are also configured to autonomously read these data messages without any software intervention. In one example, a message transmitted through interconnect circuit 206 may be accompanied by an identifier indicating which module from which the message was transmitted. Monitoring devices 214, 216, and 218 can be configured to filter out messages that are not data messages transmitted by sensor 202 by observing this identifier. In other words, monitoring devices 214, 216, and 218 can selectively read only these data messages, which include an identifier marking sensor 202 as the module from which these data messages were transmitted. In another example, messages transmitted through interconnect circuit 206 are accompanied by a marker indicating the module from which these messages will be written. Therefore, monitoring devices 214, 216, and 218 can selectively read only data messages that include an identifier marking the address to be written in memory 204.

[0045] Despite Figure 2 The SoC is shown as including three monitoring devices, but any number of monitoring devices can be appropriately integrated into the integrated circuit chip 100. Each monitoring device 214, 216, 218 can be connected to the interconnect circuit 206 via a single communication link. Alternatively, one or more device units 214, 216, 218 can be connected to multiple communication links. The monitoring devices 214, 216, 218 are also configured to monitor data messages read from the interconnect circuit 206 and determine whether these messages include data values ​​indicating a sensor fault in the system being monitored by sensor 202.

[0046] In addition to interconnect circuitry 206, monitoring devices 214, 216, and 218 are connected to output module 220. Output module 220 is configured to perform a calibration action if one or more monitoring devices 214, 216, and 218 determine that a data value included in a data message indicates a sensor fault. That is, monitoring devices 214, 216, and 218 are configured to instruct output module 220 to perform a calibration action if they determine that a data value included in a data message indicates a sensor fault. Figure 2 In this system, an output module 220 is included; however, the circuitry may include any number of output modules 220. Although Figure 2 The output module 220 is shown outside the integrated circuit chip 100, but alternatively, the output module 220 may be included within the integrated circuit chip. In cases where the circuit includes more than one output module 220, each output module may be configured to perform a different correction action in response to a determination from one or more monitoring devices 214, 216, 218 that the read data indicates a sensor malfunction.

[0047] Figure 3 yes Figure 1 and 2 A detailed embodiment of the monitoring device 214 for the integrated circuit chip is shown below. The monitoring device 214 is capable of reading and monitoring data messages from the interconnect circuit 206 using an arrangement of hardware components without the intervention of any system software. These hardware components include a processor 302 connected to both the data memory 304 and the communicator 306.

[0048] The processor 302 of the monitoring device 214 is configured to calculate a first hash value for a first message received from the interconnect circuit 102. The hash memory 304 is a memory configured to store a plurality of previous hash values, each stored hash value corresponding to a message read from the interconnect circuit prior to the first message. The processor 302 is also configured to compare at least one hash value calculated for the first message with one or more previous hash values ​​stored in the hash memory 304. If the processor 302 determines that the difference between the first hash value and at least one previous hash value stored in the hash memory is less than a predetermined threshold identified by the processor 302, the processor outputs a signal to the communicator 306. Upon receiving this signal, the communicator 306 is then configured to instruct the output module 220 connected to the integrated circuit to perform a correction action.

[0049] The processor 302 of the monitoring device 214 also includes a data buffer 308 connected to the hash calculator 310. In addition to the data buffer 308, the hash calculator 310 is connected to a current hash selector 312 and a comparator 314. Both the current hash selector 312 and the comparator 314 are connected to a hash memory 304. The comparator 314 is also connected to a communicator 306.

[0050] Data buffer 308 is configured to receive data messages read from interconnect circuit 206 and store these received data messages for a short period of time. Then, data buffer 308 transmits the received data messages to hash calculator 310. As summarized above, the first data message received by data buffer 308 is a data message read from interconnect circuit 206 before being received and written to memory 204 connected to integrated circuit chip 100. Therefore, data buffer 308 receives the first data message before one or more core devices 208, 210, 212 read data messages from memory 204.

[0051] Data buffer 308 can be of a predetermined size. In one example, the size of data buffer 308 is the same as the size of the data message received by the buffer. In other words, the size of data buffer 308 is the same as the size of the first message received by the buffer. Therefore, data buffer 308 is configured to store only the data message recorded in one clock cycle. In an alternative example, the size of data buffer 308 is the size of only a subset of the data values ​​included in the first data message. In this alternative example, the size of the buffer corresponds to the size of the hash calculated by hash calculator 310. Therefore, data buffer 308 is configured to store only a subset of the data values ​​included in a data message in any given clock cycle. The hash size is described in further detail below.

[0052] The hash calculator 310 is configured to calculate one or more data hashes for each data message received from the data buffer 308. Each data hash includes a hash value representing a data value from the message used to calculate the hash. As mentioned above, each data message received from the data buffer 308 includes one or more distinct data values. Therefore, each data message may include multiple data values. In one example, the hash calculator 310 is configured to calculate a data hash for all data values ​​of a received data message. In an alternative example, the hash calculator 310 is configured to calculate multiple data hashes for each received data message. In this alternative example, for a first data message, each hash value is calculated for a subset of the data values ​​of that data message. This subset may include one or more data values ​​from the first data message. That is, each hash value calculated for the first data message represents a portion of that data message.

[0053] Each data hash calculated by hash calculator 310 has a predetermined hash size, where the hash size indicates the number of data values ​​used to calculate the hash. The hash size can be configured by the operator of the integrated circuit chip. In one example, the hash size can be configured by the operator at the instantiation time of integrated circuit chip 100. In an alternative example, the hash size is dynamically configurable; that is, the hash size is configurable during the operation of integrated circuit chip 100.

[0054] In the example where the sensor is an image sensor, each data message read from interconnect circuit 206 is an image frame. Therefore, each data value of a given message can be a pixel of the image frame. Thus, in one example, hash calculator 310 is configured to calculate a data hash over all pixels of the first image frame. In an alternative example, hash calculator 310 is configured to calculate a data hash for each pixel in a defined subset of the first image frame.

[0055] In an example where the hash calculator 310 is configured to calculate more than one data hash for each data message received from the data buffer 308, the data values ​​of the data messages can be used to form either rolling inputs or non-overlapping inputs to form these hash values. A rolling input is an input in which there is significant overlap in the data values ​​used to calculate consecutive hashes. That is, the hash calculated for each data message represents an overlapping subset or overlapping portion of the message's data. An example of a set of data hashes obtained from the rolling inputs is shown below:

[0056] H1 = f(X1, X2, ..., X) N )

[0057] H2 = f(X2, X3, ..., X) N+1 )

[0058] H3 = f(X3, X4, ..., X) N+2 )

[0059] In the above description, H1, H2, and H3 each represent the data values ​​X1, X2, ... X from data message Y1. N+2 The values ​​of the data hashes obtained from a subset of the hashes. In the example provided above, the inputs to data hash H1 and data hash H2 differ by one data value. Similarly, the inputs to data hash H2 and data hash H3 differ by one data value. In alternative examples, the inputs between consecutive data hashes can differ by a greater number of data values.

[0060] Non-overlapping input is input where the data values ​​used to form consecutive hash values ​​do not overlap. That is, the hash value calculated for each data message represents the non-overlapping portion of that message. An example of a set of data hashes obtained from non-overlapping input is shown below:

[0061] H1 = f(X1, X2, ..., X) N )

[0062] H2=f(X N+1 X N+2 , ...X 2N )

[0063] H3=f(X 2N+1 X 2N+2 , ...X 3N )

[0064] In the above description, H1, H2, and H3 each represent the data values ​​X1, X2, ... X from data message Y1. N+2 The hash value of the data obtained from the subset.

[0065] Hash memory 304 is a memory configured to store multiple previously calculated hash values ​​by hash calculator 310 before receiving the first data message. Hash memory 304 includes multiple entries 316, 318, and 320 for storing the previously calculated hash values ​​by hash calculator 310.

[0066] Each previous hash value stored in hash memory 304 corresponds to a data message that is read from interconnect 102 by monitoring device 214 before the first data message received by data buffer 308. For each hash value calculated for the first data message, hash memory stores a previous hash value that has been calculated for a subset of data values ​​that are the same as those used to calculate one or more hash values ​​for the data message (but calculated for a previous clock cycle). For example, the first data message recorded by sensor 202 within the first clock cycle T can be represented by tag Y. 1,T The first hash value calculated by hash calculator 310 from the first data message can be represented by the tag H. 1,T Therefore, entries 316, 318, and 320 can be stored in conjunction with data messages preceding the first data message Y. 1,T Recorded data message Y 1,T-1 Y 1,T-2 ...Y 1,T-N The corresponding previous hash values. These hash values ​​can be represented as H. 1,T-1 H 1,T-2 ...H 1,T-N For example, the first previous hash value H 1,T-1 can be stored in the first entry 316. The hash memory can store the hash value of any number of previous clock cycles of received data.

[0067] Note that in an alternative example, the first hash value H 1,T The hash calculator 310 can be used only for messages contained in the first message Y. 1,T The data value X in 1,T X 2,T ...X N,T Subset computation.

[0068] Once hash calculator 310 matches the first data message Y 1,T The first hash value H was calculated. 1,T The hash value will then be passed to the current hash selector 312 and comparator 314. The current hash selector 312 is configured to receive the hash value of the data hash calculated by the hash calculator 310 and use this hash value to look up the corresponding previous hash value stored in the hash memory 304. The corresponding previous hash value is a hash value that has been calculated for a subset of data used to form the first data message or for a part of the first data message used to calculate the first hash value (but calculated for a different previous window). Therefore, for the first data message Y... 1,T The first hash value H calculated 1,T The corresponding previous hash value is H. 1,T-1 H 1,T-2 That is, one or more previous hash values ​​H. 1,T-1 H 1,T-2 The order corresponds to the hash value H used to generate the first hash value. 1,T First news Y1,T The previous hash value. Once the corresponding previous hash value in hash memory 304 is identified, these values ​​are passed to comparator 314.

[0069] Comparator 314 is connected to both hash memory 304 and hash calculator 310. Therefore, the comparator is configured to receive the first hash value H calculated by hash calculator 310. 1,T And the corresponding previous hash value from hash memory 304. Comparator 314 is also configured to receive the first hash value H from hash calculator 310. 1,T Compare with one or more corresponding previous hash values ​​stored in hash memory 304. If the first hash value H... 1,T This indicates the first data message Y in the first clock cycle T. 1,T If the hash value is H, then the hash value will first be compared with the previous hash value H. 1,T-1 Compare them. Then you can use H. 1,T Compared with the previous hash value H 1,T-2is compared until it is the same as the previous hash value H 1,T-M is compared, where M is the total number of previous hash values to be compared with the first hash value H 1,T The number M of previous hash values to be compared with the first hash value can be configured by the operator of the integrated circuit chip. In one example, M is configurable at the instantiation time of the integrated circuit chip 100. In an alternative example, M is dynamically configurable. That is, M is configurable during the operation of the integrated circuit chip 100.

[0070] Comparator 314 is configured to compare the hash value of the first data hash H 1,T and each previous hash value H 1,T-2 ……H 1,T-M with a predetermined threshold W1. The value of the predetermined threshold can be stored in the monitoring device 214. In one example, the value of the predetermined threshold is stored in an additional memory (not shown) of the monitoring device. Alternatively, this additional memory can be included in the hash memory 304 or in the memory of the comparator 314. In addition, the predetermined threshold can generally be stored in an alternative location within the processor 302 or the monitoring device 214. Comparator 314 is also configured to determine whether the difference between the first hash value H 1,T and at least one of the previous hash values H 1,T-2 ……H 1,T-M stored in the hash memory is lower than the predetermined threshold W1. That is, the comparator is configured to determine whether |H1 - H 1，T-Z | < W1, where Z = 1, 2... M.

[0071] In one example, the monitoring device 214 is configured to perform a correction action if it is determined that the difference between the first hash value H ) 1,T and one of the previous hash values is lower than the predetermined threshold W1. In this example, the first hash value is only compared with the hash value immediately preceding it. If |H1 - H 1，T-1 | < W1, a correction action is performed. and one of the previous hash values is lower than the predetermined threshold W1. In this example, the first hash value is only compared with the hash value immediately preceding it. If |H1 - H

[0072] In an alternative example, the monitoring device 214 is configured to perform a correction action if it is determined that the difference between the first hash value H 1,T and a plurality of previous hash values is lower than the predetermined threshold W1. In this example, the number P of previous hash values for which the difference between the first hash value and each previous hash value is lower than the predetermined threshold is predefined by the monitoring device 214. The number P of previous hash values is defined as a part of M, where M is the total number of previous hash values to be compared with the first hash value H 1,T by the comparator 314. Therefore, 1 < P < M. For example, P can be equal to M / 2. Thus, if |H1 - H 1，T-ZIf the number of previous hash values ​​of W1 is greater than M / 2, then a correction action is performed.

[0073] If the first hash value H is determined 1,T If the difference between the comparator 314 and at least one previous hash value stored in hash memory 304, as defined above, is less than a predetermined threshold W1, then the comparator 314 outputs a signal to the communicator 306. The communicator 306 is configured to communicate with one or more output modules 220 external to the monitoring device 214. Therefore, the communicator 306 is responsible for transmitting instructions to the output modules. These instructions indicate the correction action to be performed by the system.

[0074] The correction action indicated by communicator 306 may include one or more of the following actions:

[0075] Inform users that sensor data is unreliable;

[0076] Components of the integrated circuit 100 that rely on data obtained from sensor 202 are deactivated;

[0077] Activate the mechanism configured to repair sensor 202;

[0078] Output an alarm signal to another component of integrated circuit 100;

[0079] Output hardware events;

[0080] Output interrupt signal; and

[0081] Output messages to external core devices 208, 210, and 212.

[0082] The current hash selector 312 is also configured to select the first hash value H. 1,T The hash value is inserted into hash memory 304. Once inserted into hash memory 304, the first hash value becomes the previous hash value. This previous hash value can then be used for comparison with a future hash value calculated on the input data message read from interconnect circuit 206. In one example, the first hash value H... 1,T The first hash value is inserted into the hash memory by the current hash selector 312, and simultaneously compared by the comparator 314 with the previous hash value corresponding to the first hash value. That is, the first hash value H... 1,T The comparison with the previous hash value corresponding to the first hash value and the storage of the hash value in the hash memory 304 occur simultaneously. In an alternative example, the first hash value H... 1,T After being processed by comparator 314, the hash value is stored in the hash memory. In other words, the comparison and storage of the first hash value can occur in the same clock cycle or in consecutive clock cycles.

[0083] exist Figure 2The integrated circuit chip shown includes three monitoring devices 214, 216, and 218. As mentioned above, the chip can optionally include any number of monitoring devices. For a chip including multiple monitoring devices, each of these devices can be responsible for monitoring data from the first data message Y. 1,T The first monitoring device calculates and compares hash values ​​for different subsets of the acquired data values. In other words, it can analyze and compare data values ​​X1, X2, ... X... N The first hash value H of the subset 1,T The second monitoring device can perform calculations and comparisons on subsets X2, X3, ... X. N+1 The second hash value H 2,T The third monitoring device can perform calculations and comparisons on subsets X3, X4, ... X. N+2 The first hash value H 1,T Calculations and comparisons are performed. In this way, each monitoring device can process a subset of the data values ​​from its data messages simultaneously. This will further reduce the latency associated with sensor data processing.

[0084] Figure 4 This is a schematic diagram of an alternative exemplary system for monitoring data messages using an integrated circuit chip device. For example... Figure 2 As shown, the system includes a sensor 402 connected to external memory 404 via a SoC 100. The SoC also includes system circuitry 102 and monitoring circuitry 104 configured to monitor system circuitry 102. System circuitry 102 includes components corresponding to… Figure 2 The core devices 208, 210, and 212 are core devices 406, 408, 410, 418, and 420. The monitoring circuit includes components corresponding to... Figure 2 Monitoring devices 214, 216, and 218, and monitoring devices 422, 424, and 426. Core device 420 is an output module and is described in further detail below. Core device 418 is internal memory or on-chip memory. In one example, in addition to external memory 404, on-chip memory 418 can also be used to store data messages recorded by sensor 402. In another example, data messages from sensor 402 can be written to either external memory 404 or on-chip memory 418. The memory to which data messages are written can be selected based on its characteristics. Examples of characteristics that can be used to determine whether data messages are written to a given memory include: the amount of storage included in the memory, the latency associated with writing data messages to the memory (which is a function of the distance of the memory from the sensor), and the bandwidth supported by the memory.

[0085] like Figure 4As shown, the interconnect circuitry of system circuit 102 also includes a memory controller 412 and a sensor controller 416. Sensor controller 416 is connected to both sensor 402 and SoC interconnect 414. Sensor controller 416 is responsible for transmitting instructions to sensor 402 and controlling the sensor's performance. Sensor controller 416 is also responsible for receiving data messages recorded by sensor 402 and transmitting these messages to other components of system circuit 102 via SoC interconnect 414. Memory controller 412 is connected to both memory 404 and SoC interconnect 414. Memory controller 412 is responsible for receiving data messages from the SoC interconnect that will be written to or stored in memory, and for transmitting the stored data messages to the core device via SoC interconnect 414.

[0086] SoC interconnect 414 is connected to core devices 406, 408, 410, 418, 420, sensor controller 416, and memory controller 412. Therefore, SoC interconnect 414 is responsible for transmitting data messages between sensor 402 (via sensor controller 416), memory 404, and core devices. The topology of the SoC interconnect is SoC-dependent but can include any or a combination of the following network types to transmit communication around system circuitry: bus network, ring network, tree network, or mesh network. In one example, SoC interconnect 414 is a data bus.

[0087] exist Figure 4 The system circuitry 102 and monitoring circuitry 104 shown both include one or more output modules 420, 432, and 436. These output modules are configured to perform corrective actions in response to signals received by the core devices 406, 408, and 410 or the monitoring devices 422, 424, and 426. Output module 420 is a safety controller. The safety controller 420 is configured to perform one or more actions to ensure the safety of the system (in which the integrated circuit chip is applied). For example, if the integrated circuit chip 100 is used in a moving vehicle, the safety controller can be configured to apply brakes to the system. Output module 432 is an analysis CPU. This analysis CPU is configured to perform further analysis on the data output by the core devices 406, 408, and 410 or the monitoring devices 422, 424, and 426 to determine why a sensor malfunction occurred.

[0088] The analysis CPU is connected to one or more monitoring devices 422, 424, 426 via message engine 428 and internal communicator 430. Message engine 428 connects all components of the monitoring circuitry and is configured to transmit messages between these components. Similar to SoC interconnect 414, the topology of the message engine is SoC-dependent but can include any or a combination of the following network types for transmitting communication around the system circuitry: bus network, ring network, tree network, or mesh network. In one example, message engine 428 is a data bus. Internal communicator 430 is configured to relay messages between the analysis CPU and message engine 428.

[0089] Output module 436 is an external debugger. External debugger 436 is used to resolve defects or problems within the integrated circuit chip 100 or in the surrounding system responsible for sensor failures. Strategies used by external debugger 436 may include interactive debugging, control flow analysis, unit testing, integration testing, log file analysis, application or system-level monitoring, memory dumping, and / or profiling. External debugger 436 is connected to message engine 428 via external communicator 434. External communicator 434 is configured to relay messages between external debugger 436 and message engine 428.

[0090] Figure 5 Is it used as Figure 2 or Figure 4 The flowchart illustrates an exemplary method for monitoring messages from sensors using an integrated circuit chip device. As described above, the data messages include data measured from sensors 202 and 204 connected to the integrated circuit chip 100. The method begins at step 502, where a first data message Y is read from the interconnect circuitry 206 of the integrated circuit using one or more monitoring devices 214, 216, and 218. 1,T The sensor 202 records the first data message Y during the first clock cycle T. 1,T First data message Y 1,T The data is received by the data buffer 308 located within the processor 302 of the monitoring device. In one example, the data message Y... 1,T It includes only a subset of the data value X1. In an alternative example, the data message includes multiple subsets of the data values.

[0091] In step 504, the hash calculator 310 of the processor 302 of the monitoring device 214 processes the first data message Y. 1,T Calculate the hash value H 1,T The first data hash. As summarized above, the first hash value H 1,TIt can represent all data values ​​in the received data message, or alternatively, only a subset of the data values ​​included in the data message. Once the first hash value H is calculated... 1,T The hash value is then passed to the current hash selector 312. The current hash selector 312 uses the first hash value H. 1,T The corresponding previous hash value is retrieved from the hash memory 304 by the comparator 314.

[0092] In step 506, comparator 314 converts the first hash value H... 1,T Compared with one or more previous hash values ​​H stored in hash memory 304 1,T-1 ...H 1,T-M A comparison is made. As described above, each previous hash value stored in the hash memory corresponds to a data message read from interconnect circuit 206 prior to the first data message. In one example, the first hash value is compared with only one previous hash value H. 1,T-1 The comparison is performed. In an alternative example, the first hash value is compared with several consecutive previous hash values ​​H. 1,T-1 To H 1,T-M Compare them. Then, use the first hash value H. 1,T With one or more previous hash values ​​H 1,T-1 ...H 1,T-M The difference between them is compared with a predetermined threshold stored in the monitoring device 214.

[0093] In step 508, comparator 314 converts the first hash value H... 1,T and one or more previous hash values ​​H 1,T-1 H 1,T-2 ...H 1,T-M The difference between the first hash value and one or more previous hash values ​​is compared with a predetermined threshold W1. If comparator 314 determines that the difference between the first hash value and at least one of the previous hash values ​​is less than the predetermined threshold W1, the method proceeds to step 510, where communicator 306 instructs the output module 220 of the integrated circuit to perform a correction action. In response to this instruction, output module 220 performs the correction action. If comparator 314 determines that the difference between the first hash value and one of the previous hash values ​​is not less than the predetermined threshold W1 (i.e., the difference is greater than or equal to W1), communicator 306 does not transmit an instruction. Therefore, one or more output modules 220 do not perform the correction action. The method alternatively skips step 510 and proceeds to step 512.

[0094] In step 510, if comparator 314 determines that the difference between the first hash value and at least one previous hash value is less than a predetermined threshold, the comparator outputs a signal to communicator 306. Upon receiving this signal, communicator 306 instructs the execution of a correction action. That is, communicator 306 transmits one or more instructions to one or more modules outside of monitoring device 214. These instructions indicate the correction action to be performed by the system.

[0095] In step 512, the first hash value H 1,T The hash value is stored in hash memory 304. Therefore, the first hash value becomes the previous hash value and can be compared with subsequent input hash values ​​to determine whether these subsequent hash values ​​indicate the occurrence of a sensor malfunction. The method then proceeds to step 514, where Y... 1,T+1 Replace Y 1,T As input to step 502. That is, in recording Y 1,T In the subsequent clock cycle of the clock cycle, the data message recorded by the sensor is received by the data buffer 308. In step 504, the hash calculator 310 uses the subsequent data message Y 1,T+1 To calculate the new hash value H 1,T+1 And compare the new hash value with the previous hash value H. 1,T H 1,T-1 H 1,T-2 ...H 1,T-M Compare them.

[0096] When the hash value calculated by hash calculator 310 is similar for similar data values, Figure 5 The method shown is appropriate. When similar data values ​​have similar hash values, a threshold W1 can be used to determine how much the hash value changes over consecutive clock cycles. Figure 6 It shows, as Figure 2 or Figure 4 The flowchart shown illustrates an alternative exemplary method for monitoring messages from sensors using an integrated circuit chip device, wherein similar hash values ​​are not generated for similar data values. That is, hash values ​​calculated by hash calculator 310 for similar data values ​​can have significantly different values, where the difference between hash values ​​is independent of the difference between the data values ​​themselves. In this example, it is impossible to use a consistent threshold to indicate the extent to which hash values ​​change over consecutive clock cycles. Sensor fault identification can instead be determined by observing the equality between hash values. That is, if a first hash value is the same as one or more previous hash values, it can be determined that a sensor fault has occurred.

[0097] Figure 6 Steps 602 to 604 correspond to Figure 5Steps 502 to 506. That is, in step 602, one or more monitoring devices 208, 210, 212 are used to read the first data message Y from the interconnect circuit 206 of the integrated circuit. 1,T In step 604, the hash calculator 310 of the processor 302 of the monitoring device 214 processes the first data message Y. 1,T Calculate the first hash value H 1,T .

[0098] In step 608, comparator 314 compares the first hash value H. 1,T And the previous hash value H 1,T-Z The difference between them, where Z = 1, 2, ..., M. If comparator 314 determines H 1,T equals H 1,T-Z Or in other words |H 1，T -H 1，T-Z If |=0, the method proceeds to step 610, where communicator 306 instructs the output module 220 of the integrated circuit to perform a correction operation. In response to this instruction, output module 220 performs the correction operation. If comparator 314 determines H… 1,T Not equal to H 1,T-Z , or in other words |H 1，T -H 1，T-Z If | > 0, then communicator 306 does not transmit an instruction. Therefore, one or more output modules 220 do not perform the correction action. The method instead skips step 610 and proceeds to step 612.

[0099] Figure 6 Steps 610 to 614 correspond to Figure 5 Steps 510 to 514. That is, in step 610, the communicator 306 instructs the output module 220 of the integrated circuit to perform a correction operation. In step 612, the first hash value H... 1,T It is stored in hash memory 304. In step 614, Y is used... 1,T+1 Replace Y 1,T As input for step 602.

[0100] As described above, in one example, comparator 314 determines whether the difference between the first hash value and each of the predetermined number of previous hash values ​​P has been satisfied, where P is defined as a part of M. M is to be determined by comparator 314 and the first hash value H. 1,T The total number of previous hash values ​​compared. In another implementation of this example, comparator 314 can be the number of previous hash values ​​(where |H1-H2)... 1，T-Z |<W1) and multiple predetermined numbers of hash values ​​P1, P2, ... P NThe comparison is performed. Each predefined number P1 corresponds to a confidence value indicating the degree of certainty that a sensor malfunction has occurred. That is, comparator 314 can derive a first predefined number P1, where P1 = M / 2. Therefore, if |H1-H 1，T-Z If the number of previous hash values ​​of |H1-H1 is greater than P1, the comparator can determine with "high confidence" that a sensor malfunction has occurred. Comparator 314 can also derive a second predetermined number P2, where P2 = M / 4. Therefore, if |H1-H1| > 1, the comparator can determine with "high confidence" that a sensor malfunction has occurred. 1，T-Z If the number of previous hash values ​​of W1 is less than P1 but greater than P2, then comparator 314 can determine with some certainty that a sensor failure has occurred. Comparator 314 can access any number of predetermined numbers P1, P2, ... P N Each predefined number is associated with a confidence value indicating a different probability that a sensor malfunction has occurred. The predefined numbers and corresponding confidence levels can be stored in the memory within the monitoring device 214.

[0101] As described above, in one example, the sensor connected to the integrated circuit chip is a camera, and each data message read by one or more monitoring devices of the integrated circuit chip is an image frame captured in the first clock cycle T. In this example, the integrated circuit chip, and in particular one or more monitoring devices, can be used to monitor the presence of stagnant pixels in the frame. That is, if the comparator 314 of the processor 302 of the monitor 214 determines that the first hash value H1 matches at least one corresponding previous hash value H stored in the hash memory... 1,T-1 H 1,T-2 If the difference between the two is less than a predetermined threshold W1, then this determination indicates that a portion of the frame used to calculate H1 includes stagnant pixels.

[0102] In one application where the sensor is a camera, the aforementioned integrated circuit chip can be inserted into a vehicle, such as an autonomous vehicle. In this example, one or more output modules 220 include an automotive safety controller. One or more correction actions, which can be instructed by the communicator 306 of the monitoring device 214, can therefore include the following:

[0103] The driver is warned that the camera view is unreliable.

[0104] Discontinue the use of car subsystems that rely on camera assistance for operation;

[0105] Activate the camera cleaner, which can remove dirt from the camera that causes malfunctions;

[0106] The fault is reported to the vehicle's safety controller, which then determines the specific action to be taken.

[0107] While the sensor in this example is an image sensor, any type of sensor capable of recording continuous data can alternatively be connected to an integrated circuit chip. Examples of alternative sensors that can monitor messages include temperature sensors, proximity sensors, accelerometers, pressure sensors, flow sensors, humidity sensors, or touch sensors. For example, the sensor could be a thermostat, and the integrated circuit chip could be used to monitor the temperature recorded by the thermostat.

[0108] Compared to corresponding systems that rely on software implementation, the system and method described herein significantly reduce the amount of time spent identifying sensor faults. Specifically, the calculation and comparison of hash values ​​by the monitoring devices can be performed in just a few hardware clock cycles. Once one or more monitoring devices identify a sensor fault, these devices can ensure that the appropriate output module is instructed to perform corrective actions using a low-latency mechanism to further reduce time delays.

[0109] The described SoC is suitably integrated into a computing-based device. This computing-based device can be an electronic device. Suitably, the computing-based device includes one or more processors for processing computer-executable instructions to control the operation of the device, thereby implementing the methods described herein. The computer-executable instructions can be provided using any computer-readable medium, such as memory. Some of the methods described herein can be performed by software in a machine-readable form on a tangible storage medium. Software can be provided at the computing-based device to implement some of the methods described herein.

[0110] The above description portrays system circuitry and monitoring circuitry as being included on the same SoC. In an alternative implementation, system circuitry and monitoring circuitry are included on two or more integrated circuit chips of the MCM. In an MCM, integrated circuit chips are typically stacked or located adjacently on an interpolator substrate. Some system circuitry may reside on one integrated circuit chip, while other system circuitry may reside on different integrated circuit chips of the MCM. Similarly, monitoring circuitry may be distributed across more than one integrated circuit chip of the MCM. Therefore, the methods and apparatus described above in the context of SoC are also applicable in the context of MCM.

[0111] The applicant discloses, in isolation, each individual feature described herein, and any combination of two or more such features, to the extent that such features or combinations can be implemented based on this specification as a whole, according to common general knowledge of those skilled in the art, regardless of whether such features or combinations of features solve any problem disclosed herein, and not to limit the scope of the claims. The applicant notes that aspects of the invention may include any such individual feature or combination of features. In light of the foregoing description, it will be apparent to those skilled in the art that various modifications can be made within the scope of this invention.

Claims

1. A method for monitoring messages from a sensor using an integrated circuit, the messages including data measured by the sensor, the method comprising: a) A hardware monitoring device reads a first message from the interconnect circuit of the integrated circuit, the interconnect circuit connecting the sensor to one or more core devices configured to process the message, the interconnect circuit connecting the sensor to a memory, and the memory storing a message transmitted by the sensor, the first message being transmitted by the sensor through the interconnect circuit and read by the hardware monitoring device before reaching the memory; b) The hardware monitoring device calculates a first hash value for the first message; c) The hardware monitoring device compares the first hash value with one or more previous hash values ​​stored in the hash memory, each previous hash value corresponding to a message read from the interconnect circuit prior to the first message; as well as d) If the difference between the first hash value and at least one of the previous hash values ​​stored in the hash memory is less than a predetermined threshold, a correction action is performed.

2. The method according to claim 1, further comprising storing the first hash value in the hash memory.

3. The method according to claim 2, wherein, While comparing the first hash value with the previous hash value, the first hash value is stored in the hash memory.

4. The method of claim 3, further comprising buffering the first message before the first message is read by the one or more core devices.

5. The method according to claim 4, wherein, The size of the buffer in the hardware monitoring device corresponds to the number of data values ​​used to calculate the data hash including the first hash value.

6. The method of claim 5, further comprising comparing the first hash value with M previous hash values, and performing a correction action if the difference between the first hash value and P previous hash values ​​is less than the predetermined threshold, wherein, 1 <P<M。 7. The method of claim 6, further comprising deriving confidence values ​​corresponding to the P previous hash values, wherein, The difference between the first hash value and the P previous hash values ​​is lower than the predetermined threshold.

8. The method according to claim 6, wherein, M is dynamically configurable.

9. The method according to claim 8, wherein, If the first hash value is the same as at least one of the previous hash values ​​stored in the hash memory, a correction action is performed.

10. The method of claim 9, further comprising calculating a plurality of hash values ​​for the first message, each of the plurality of hash values ​​representing a portion of the first message, and wherein, Each of the plurality of hash values ​​of the first message is compared with one or more previous hash values ​​corresponding to that portion of the first message.

11. The method according to claim 10, wherein, The plurality of hash values ​​calculated for the first message represent the overlapping portion of the first message.

12. The method according to claim 10, wherein, The plurality of hash values ​​calculated for the first message represent the non-overlapping portions of the first message.

13. The method of claim 12, wherein the correction action is one or more of the following: informing the user that the sensor data is unreliable, disabling components of the integrated circuit that rely on data obtained from the sensor, and activating a mechanism configured to repair the sensor.

14. The method according to claim 13, wherein, The correction action includes outputting an alarm signal to another component of the integrated circuit.

15. The method according to claim 14, wherein, The correction action includes: outputting a hardware event, outputting an interrupt signal, or outputting a message to an external core device.

16. The method according to claim 15, wherein, The sensor is an image sensor, and each message read from the interconnect circuit is a frame captured by the image sensor.

17. The method according to claim 16, wherein, The integrated circuit is a system-on-a-chip.

18. The method according to claim 17, wherein, The interconnect circuit is a data bus.

19. An integrated circuit chip for monitoring messages from a sensor, the messages including data measured by the sensor, the integrated circuit chip comprising: Interconnect circuitry that connects the sensor to a core device configured to process the message; The interconnect circuit connects the sensor to the memory, and the memory stores messages transmitted by the sensor. A hardware monitoring device configured to read a first message from the interconnect circuit, the first message being transmitted by the sensor through the interconnect circuit and read by the hardware monitoring device before reaching the memory; And a hash memory for storing a plurality of previous hash values, each hash value corresponding to a message read from the interconnect circuit prior to the first message; The hardware monitoring device is also configured to: Calculate the first hash value for the first message; as well as The first hash value is compared with one or more of the previous hash values ​​stored in the hash memory; The integrated circuit is configured to perform a correction action if the difference between the first hash value and at least one of the previous hash values ​​stored in the hash memory is less than a predetermined threshold.

Images