Systems and methods for sequential anomaly detection in IVNs using a graph-based state-space approach

Abstract

A method for operating an anomaly detection system includes receiving training message sequences corresponding to messages transmitted in an in-vehicle network (IVN), constructing a model containing multiple states corresponding to observed signal values ​​in the training message sequences and state transitions between the respective states of the multiple states, based on the training message sequences, training the model by feeding it first message sequences corresponding to the training message sequences and second message sequences not included in the training message sequences, and executing the model to identify anomalous message sequences transmitted in the IVN using the anomaly detection system by receiving an IVN message sequence and outputting a value based on state transitions between signal states.the value contained in the IVN message sequence is based on the model and outputs an indication of whether the IVN message sequence contains an anomalous message sequence, based on the value.

Description

Technical area

[0001] The present disclosure relates to detecting anomalies in in-vehicle networks (IVNs). background

[0002] The background description provided herein serves to generally illustrate the context of the disclosure. Work by the inventors identified herein, to the extent that work is described in this background section, as well as aspects of the description that do not otherwise qualify as prior art at the time of filing, are neither explicitly nor implicitly admitted as prior art to the present disclosure.

[0003] Vehicles such as passenger cars, trucks, off-road vehicles, SUVs, minivans, or other suitable vehicles include a propulsion system, a braking system, a steering system, and the like. Such vehicle systems may be manually controlled (e.g., by a vehicle operator) and / or may be autonomously or semi-autonomously controlled (e.g., by one or more autonomous or semi-autonomous vehicle controllers). According to some examples, autonomous or semi-autonomous vehicle controllers rely on or utilize various machine learning models for vehicle control decision-making.

[0004] The central communication network in several modern systems, such as automotive systems, aeronautical systems, industrial systems, etc., is a Controller Area Network (CAN) bus. For example, electronic control units (ECUs) control multiple vehicle subsystems via the CAN bus and can communicate with each other to ensure the proper functioning of one or more of these subsystems. Some nodes on the CAN bus, and more generally on the vehicle's internal network, may be equipped with remote interfaces. Such interfaces can often be used to enable over-the-air updates, offer additional services, measure application usage, and measure the use of specific functions in electronic control units (ECUs) to enable maintenance services, etc. Summary

[0005] A method of operating an anomaly detection system includes receiving training message sequences corresponding to messages transmitted in an in-vehicle network (IVN), constructing a model including a plurality of states corresponding to observed signal values ​​in the training message sequences and state transitions between respective states of the plurality of states based on the training message sequences, training the model by supplying first message sequences corresponding to the training message sequences and second message sequences not included in the training message sequences to the model, and executing the model to identify anomalous message sequences transmitted in the IVN using the anomaly detection system by receiving an IVN message sequence, outputting a value based on state transitions between states of signals,contained in the IVN message sequence, from the model and outputting an indication of whether the IVN message sequence contains an anomalous message sequence based on the value.

[0006] According to other features, the method further includes decoding the training message sequences and constructing the model based on the decoded training message sequences. Each of the plurality of states in the model corresponds to values ​​of two or more correlated signals included in the training message sequences. Each of the plurality of states includes at least two values ​​for each of the two or more correlated signals. The model identifies probabilities of each of the state transitions between the respective states of the plurality of states.Outputting the value from the model includes calculating a distance between a first state of the plurality of states corresponding to a first message in the IVN message sequence and a second state corresponding to a second message in the IVN message sequence, and calculating the value based on the distance and / or calculating the value using a heuristic probabilistic method. The distance corresponds to a number of state transitions in the model required to traverse between the first state and the second state. The second state does not correspond to any message included in the training message sequences.

[0007] An anomaly detection system includes model constructor circuitry configured to receive training message sequences corresponding to messages transmitted in an in-vehicle network (IVN) and to construct a model including a plurality of states corresponding to observed signal values ​​in the training message sequences and state transitions between respective states of the plurality of states based on the training message sequences; sequence generator circuitry configured to train the model by supplying first message sequences corresponding to the training message sequences and second message sequences not included in the training message sequences to the model; model circuitry configured to execute the model to identify anomalous message sequences transmitted in the IVN;by receiving an IVN message sequence and outputting a value based on state transitions between states of signals contained in the IVN message sequence from the model, and model output classifier circuitry configured to output an indication of whether the IVN message sequence contains an anomalous message sequence based on the value.

[0008] According to other features, the anomaly detection system further includes a decoder configured to decode the training message sequences. The model constructor circuitry is configured to construct the model based on the decoded training message sequences. Each of the plurality of states in the model corresponds to values ​​of two or more correlated signals included in the training message sequences. Each of the plurality of states includes at least two values ​​for each of the two or more correlated signals. The model circuitry is configured to implement the model to identify probabilities of each of the state transitions between the respective states of the plurality of states.The model circuitry is configured to implement the model to calculate a distance between a first state of the plurality of states corresponding to a first message in the IVN message sequence and a second state corresponding to a second message in the IVN message sequence, and to calculate the value based on the distance and / or to calculate the value using a heuristic probabilistic method. The distance corresponds to a number of state transitions in the model required to traverse between the first state and the second state. The second state does not correspond to any message included in the training message sequences.

[0009] A computing device configured to implement an anomaly detection system for a vehicle includes a processing device configured to execute instructions stored in memory to cause the anomaly detection system to receive an in-vehicle network (IVN) message sequence corresponding to messages transmitted between electronic control units (ECUs) of the vehicle, provide the IVN message sequence to a model including a plurality of states corresponding to observed signal values ​​in previous IVN message sequences and state transitions between respective states of the plurality of states, output from the model a value based on state transitions between states of signals included in the IVN message sequence, and output, based on the value, an indication of whether the IVN message sequence includes an anomalous message sequence.

[0010] According to other features, the processing device is further configured to execute instructions stored in memory to cause the anomaly detection system to decode the IVN message sequence and to provide the decoded IVN message sequence to the model. Each of the plurality of states in the model corresponds to values ​​of two or more correlated signals included in the decoded IVN message sequence, wherein each of the plurality of states includes at least two values ​​for each of the two or more correlated signals, and wherein the model identifies probabilities of each of the state transitions between the respective states of the plurality of states.Furthermore, the processing device is for executing instructions stored in the memory to cause the anomaly detection system to calculate a distance between a first state of the plurality of states corresponding to a first message in the IVN message sequence and a second state corresponding to a second message in the IVN message sequence and to calculate the value based on the distance, wherein the distance corresponds to a number of state transitions in the model required to traverse between the first state and the second state, and / or to calculate the value using a heuristic probabilistic method. Brief description of the drawings Fig. 1 is an exemplary computing device according to the present disclosure. Fig. 2 is an exemplary anomaly detection system according to the present disclosure. Fig. 3A is an exemplary state-based model implemented by an anomaly detection system according to the present disclosure. Fig. 3B is an example classification process configured to classify IVN signal sequences based on an output of a model, according to the present disclosure. Fig. 3C are tables illustrating exemplary anomaly detection performed using distance measurements between previous states (source states) and next states (target states) in a model for a sequence of normal messages and for a sequence of anomalous messages, according to the present disclosure. Fig. 3D illustrates an exemplary heuristic distance method as applied to a portion of a model in accordance with the present disclosure. Fig. 4 illustrates steps of an exemplary method for constructing and implementing a model of an anomaly detection system according to the present disclosure. Fig. 5 shows a schematic representation of an interaction between a computer-controlled machine and a control system according to the principles of the present disclosure. Fig. Figure 6 shows a schematic representation of the control system of Fig. 5 configured to control a vehicle, which may be a semi-autonomous vehicle, a fully autonomous vehicle, a semi-autonomous robot, or a fully autonomous robot, according to the principles of the present disclosure. Fig. Figure 7 shows a schematic representation of the control system of Fig. 5, which is configured to control a manufacturing machine such as a die cutter, a cutting machine or a spindle drill of a manufacturing system such as part of a production line. Fig. Figure 8 shows a schematic representation of the control system of Fig. 5, which is configured to control a power tool such as an electric drill or an electric screwdriver, which has at least one semi-autonomous operating mode. Fig. 9 shows a schematic representation of the control system of Fig. 5, which is configured to control an automated personal assistant. Fig. 10 shows a schematic representation of the control system of Fig. 5, which is configured to control a control system such as a control access system or a monitoring system. Fig. 11 shows a schematic representation of the control system of Fig. 5, which is configured to control an imaging system, e.g. an MRI device, an X-ray imaging device or an ultrasound device.

[0011] Reference numerals may be reused in the drawings to identify similar and / or identical elements. Detailed description

[0012] Embodiments of the present disclosure are described herein. It should be understood, however, that the disclosed embodiments are merely examples, and other embodiments may take various and alternative forms. The figures are not necessarily to scale; some features may be exaggerated or minimized to show details of particular components. Thus, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching those skilled in the art to variously utilize the embodiments.As will be understood by one of ordinary skill in the art, various features illustrated and described with respect to any of the figures may be combined with features illustrated in one or more other figures to produce embodiments not explicitly illustrated or described. The combinations of illustrated features provide representative embodiments for typical applications. However, various combinations and modifications of the features consistent with the teachings of the disclosure may be desired for particular applications or implementations.

[0013] Vehicle systems can be vulnerable to external attacks at multiple levels or layers, such as a physical layer (e.g., sensors, actuators, etc.), an in-vehicle network (IVN) layer, an application layer, and various external network interfaces (e.g., cellular, Bluetooth, Wi-Fi, etc.). Automotive IVNs (e.g., Controller Area Networks (CANs)) connect various electronic control units (ECUs) in a vehicle. ECUs broadcast messages over IVNs to communicate information about various system parameters, such as sensor values ​​(e.g., engine speed, throttle position, etc.). IVNs can be vulnerable to malicious attacks / intrusions due to the large number of electrical control units (ECUs), the ECUs' connectivity to an external network, the ECUs' increased dependence on software, etc.An attacker can compromise an ECU via external interfaces, inject spoofed signal values, prevent critical signals from being transmitted to the IVN, etc. Anomaly detection systems that operate based on IVN signals observe traffic in the network and attempt to distinguish normal traffic from anomalous traffic patterns that occur when one or more ECUs have been compromised.

[0014] Accordingly, detecting network intrusion events is an integral part of automotive security to prevent or at least mitigate the impact of such intrusion events. To improve automotive security, various methodologies can be implemented, including, but not limited to, firewalls, whitelisting, message blacklisting, and dedicated intrusion detection systems (IDSs) implemented in one or more of the ECUs (e.g., gateway ECUs). An IDS is a system that monitors predefined systems and / or networks and identifies intrusion attempts based on the detection of feigned, malicious, or undefined system behavior. The operation of the IDS is based on the prerequisite that an intrusion attempt different from regular system behavior is detectable.

[0015] As one example, the IDS may implement fingerprinting techniques and methods to compare unique electrical or other operating characteristics of an ECU with detected behavior. However, fingerprinting methods are unable to detect attacks at higher operational layers in automotive systems. As another example, entropy-based methods and methods that monitor parameters such as ECU message frequency, detect injection or denial-of-service (DoS) attacks, may be ineffective against attacks whose signatures are unknown. As yet another example, machine learning (ML) and deep learning (DL)-based methods may rely on training the system with a large number of examples of normal IVN traffic and flag deviations from normal patterns.The above methods can process raw IVN message data without considering the message content. Furthermore, the methods implement models that analyze individual ECU message streams. However, the respective contexts of message streams emitted by other ECUs simultaneously may not be available for analysis, which can utilize intersignal correlation effects to detect anomalous behavior.

[0016] Anomaly detection systems and anomaly detection methods according to the present disclosure are configured to detect attacks on the IVN layer, such as frame injection, frame modification, frame erasure, and DoS attacks, which are caused by one or more ECUs being compromised and used as a starting point for such attacks. For example, an anomaly detection system based on ECU signals and one or more heuristic methods implements a state-space representation of the IVN to distinguish between normal and anomalous IVN traffic. While other methods based on ML / DL techniques and Hidden Markov Model work on raw IVN signals / raw messages (i.e., coded IVN signals / messages) from individual ECUs and on non-transparent system states, the anomaly detection system of the present disclosure works on actual (i.e.,decoded) signal content of IVN messages from multiple ECUs based on correlations between the signals. Using the state-space model and heuristics, the anomaly detection system significantly reduces the occurrence of false positives while maintaining high accuracy (i.e., the detection of true positives).

[0017] Accordingly, the anomaly detection system is configured to access decoded IVN messages (e.g., using a supplier-provided database (DBC file) for CAN messages) in real time. The anomaly detection system may include and / or implement, among other things, the state-space model, a set of heuristics, and a classifier (e.g., implemented or executed by circuitry such as one or more processing devices or processors). The state-space model may be based on a signal-based state space (e.g., represented in a graph data structure) generated using a training set of IVN message sequences. Using the state-space model and the set of heuristics, state transitions induced by ECU signals emitted during vehicle operation may be evaluated.In other words, transitions between states of the model are evaluated with state transition scores. The state transition scores indicate whether the sequences of IVN messages are normal or anomalous. The state-space model is configured to use measures computed by the heuristics to better distinguish between true anomalous sequences and sequences that may not have been observed during model construction but are nevertheless "normal." Accordingly, the false positive rate is improved while maintaining high accuracy. Examples of heuristics include, but are not limited to, (i) a distance measure that computes a "distance" between two states in a state transition, and (ii) probability bounds on a change in signal value or the rate of change of the signal value associated with a state transition.

[0018] The classifier in the anomaly detection system is configured to classify sequences of IVN messages (e.g., with a fixed length) as normal or anomalous. The classifier can be trained with a set of scores (e.g., an output from the state-space model and the various heuristics) associated with the message sequences (e.g., in the training set), along with classification labels (e.g., ground-truth labels) for the message sequences. Various ML or DL ​​methods, such as a support vector machine (SVM), random forests, neural networks, etc., can be used to build the classifier.

[0019] Accordingly, the anomaly detection systems and anomaly detection methods according to the present disclosure implement a state-space representation of a system of ECUs modeled as a graph based on observable signals transmitted by the ECUs in the IVN. The use of correlated signals and the inclusion of the memory of recent signals in the state-space model provide more context for anomaly detection, and the use of various heuristics along with the state-space model to output scores for signal sequences enables the classifier to better distinguish between normal and anomalous sequences. The model can be updated / improved by incrementally learning (e.g., via online learning) from new data, which is more efficient compared to training models using ML / DL methods, which must be retrained as new data becomes available.The classification tasks of these systems and methods are more efficient than ML or DL ​​methods, which require complex hardware acceleration to meet real-time constraints. Furthermore, the graph-based model described here enables the representation of anomaly predictions for analysis, compared to various methods such as DL and hidden Markov models.

[0020] Fig. 1 shows a block diagram of an exemplary computing device 100 and associated circuitry configured to implement the anomaly detection systems and anomaly detection methods according to some embodiments of the present disclosure. Computing device 100 may include a controller 105, which may be, for example, a central unit processor (CPU), a chip, or any suitable computer or computing device, an operating system 115, a memory 120, executable code 125, a storage server system 130, input devices 135, and output devices 140. Controller 105 (or one or more controllers or processors, possibly across multiple units or devices) may be configured to perform methods described herein and / or to execute or act as the various modules, units, etc.More than one of the computing devices 100 may be included, and one or more of the computing devices 100 may act as the components of a system according to embodiments of the disclosure.

[0021] The operating system 115 may be, be, or include any code segment (e.g., similar to the executable code 125 described herein) that is determined and / or configured to perform tasks that involve coordinating, scheduling, arbitrating, monitoring, controlling, or otherwise managing the operation of the computing device 100, e.g., scheduling the execution of software programs or software tasks or enabling software programs or other hardware modules or hardware units to communicate. The operating system 115 may be a commercial operating system. It is noted that the operating system 115 may be an optional component; for example, according to some embodiments, a system may include a computing device that does not require or include the operating system 115.For example, a computer system may be or include a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a network controller (e.g., a CAN bus controller), an associated transceiver, a system-on-chip (SOC), and / or any combination thereof that can be used without an operating system.

[0022] Memory 120 may be or include, for example, random access memory (RAM), read-only memory (ROM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), a double data rate memory (DDR) chip, flash memory, volatile memory, non-volatile memory, cache memory, a buffer, a short-term storage device, a long-term storage device, or other suitable storage or storage devices. Memory 120 may be or include multiple, possibly different, storage devices. Memory 120 may be a non-transitory computer- or processor-readable medium or a non-transitory computer storage medium, e.g., RAM.

[0023] The executable code 125 may be any executable code, e.g., an application, a program, a process, a task, or a script. The executable code 125 may be executed by the controller 105, possibly under the control of the operating system 115. The executable code 125 may, for example, be an application that, as further described herein, enforces security in a vehicle and, for example, detects or prevents cyberattacks in an in-vehicle network. Although in Fig. 1, a single unit of executable code 125 is shown for clarity, a system according to some embodiments of the disclosure may include multiple executable code segments similar to executable code 125 that may be loaded into memory 120 and cause controller 105 to execute methods described herein. Where applicable, the terms "process" and "executable code" may mean the same thing and may be used interchangeably. For example, verifying, validating, and / or authenticating a process may mean verifying, validating, and / or authenticating executable code.

[0024] The storage system 130 may be or include, for example, flash memory as known in the art, memory internal to or embedded in a microcontroller or chip as known in the art, a hard disk drive, a CD-Recordable Drive (CD-R Drive), a Blu-Ray Disc (BD), a Universal Serial Bus (USB) device, or any other suitable removable and / or fixed storage device. Content may be stored in the storage system 130 and loaded from the storage system 130 into the memory 120, where it may be processed by the controller 105. According to some embodiments, some of the Fig. 1 may be omitted. For example, memory 120 may be non-volatile memory with the storage capacity of storage system 130. Accordingly, storage system 130, although shown as a separate component, may be embedded in or included with memory 120.

[0025] Input devices 135 may be or include any suitable input devices, components, or systems, e.g., physical sensors such as accelerometers, tachometers, thermometers, microphones, analog-to-digital converters, etc., a detachable keyboard or keypad, a mouse, and the like. Output devices 140 may include one or more (possibly detachable) displays or monitors, motors, servo motors, speakers, and / or any other suitable output devices. As shown by blocks 135 and 140, any applicable input / output (I / O) devices may be connected to computing device 100.For example, input devices 135 and / or output devices 140 may include a wired or wireless network interface card (NIC), a universal serial bus (USB) device, a JTAG interface, or an external hard disk drive. It will be appreciated that, as shown by blocks 135 and 140, any suitable number of input devices 135 and output devices 140 may be operatively connected to computing device 100. For example, input devices 135 and output devices 140 may be used by a technician or engineer to connect to computing device 100, update software, and the like. Input and / or output devices or input and / or output components 135 and 140 may be used to interface or communicate with the controller or other units in a vehicle, e.g.,Input and / or output devices, or components 135 and 140 may include ports that enable computing device 100 to communicate with an engine control unit or engine control module, a suspension control unit or suspension control module, a traction control unit or traction control module, and the like.

[0026] Embodiments may include an article such as a non-transitory computer- or processor-readable medium or a non-transitory computer- or processor-readable storage medium such as, for example, memory, a disk drive, or USB flash memory, coding that includes or stores instructions (e.g., computer-executable instructions that, when executed by a processor or controller, perform methods disclosed herein), a storage medium such as memory 120, computer-executable instructions such as executable code 125, and a controller such as controller 105.

[0027] The storage medium may include, but is not limited to, any type of disk, including magneto-optical disks, semiconductor devices such as read-only memories (ROMs), random access memories (RAMs) such as dynamic RAM (DRAM), erasable programmable read-only memories (EPROMs), flash memory, electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, or any type of media suitable for storing electronic instructions, including programmable storage devices.

[0028] Embodiments of the disclosure may include components such as, but not limited to, multiple central processing units (CPUs) or any other suitable general-purpose or special-purpose processors or general-purpose or special-purpose controllers (e.g., controllers similar to controller 105), multiple input units, multiple output units, multiple memory units, and multiple storage units, etc. A system may additionally include other suitable hardware components and / or software components. According to some embodiments, a system may include or be, for example, a personal computer, a desktop computer, a mobile computer, a laptop computer, a notebook computer, a terminal, a workstation, a server computer, a personal digital assistant (PDA) device, a tablet computer, a network device, or any other suitable computing device.

[0029] For example, according to some embodiments, a system may include or be multiple circuitry components, each including multiple central processing units, e.g., multiple CPUs as described, multiple CPUs embedded in an on-board or in-vehicle system or network, multiple chips, FPGAs or SOCs, microprocessors, transceivers, microcontrollers, multiple computers or network devices, any other suitable computing device, and / or any combination thereof. For example, a system as described herein may include one or more devices such as computing device 100.

[0030] Fig. 2 shows one embodiment of an anomaly detection system 200 and a vehicle 204 that may implement an IDS 208 according to the present disclosure. Portions of the IDS 208 are implemented in the vehicle 204, while other portions of the IDS 208 may be implemented in one or more remote servers (which may be implemented in a cloud computing system or cloud backend, a vehicle safety operations center (VSOC), etc.). The anomaly detection system 200 according to the present disclosure may include and / or be implemented using circuitry such as a computing device, one or more processing devices, etc., as described above and elsewhere herein.For example, the anomaly detection system 200 includes circuitry configured to store and / or implement one or more state-space models (of the model 212), and further includes a model output classifier 216 configured to classify outputs of the model 212 as normal or anomalous, as described in more detail below. For example only, the model output classifier 216 may be implemented using classifier circuitry, such as a computing device, a processing device configured to execute instructions to classify outputs of the model 212, etc.

[0031] According to some example implementations, a device such as a CAN bus logger 220 may communicate with vehicle systems via an on-board diagnostic port 224 (e.g., an OBD-II port) of the vehicle 204 and record and / or output CAN messages (e.g., IVN signals provided as traces 228), including safety and vehicle control-related CAN messages. For example, the device communicates with one or more ECUs 232 of the vehicle 204 via a CAN bus 236. One or more of the ECUs 232 correspond to a gateway ECU. The stored CAN messages are available for visualization and further processing and may be accessed by external services such as model construction and model training performed using the anomaly detection system 200.For example, the curves 228 may be provided to one or more model constructors 240 configured to construct the models 224 based on data contained in the curves 228, as described in more detail below. As an example, the model constructor 224 may be implemented using circuitry of computing devices, such as any computing device described herein.

[0032] The curves 228 may further be provided to a sequence generator 240. The sequence generator 240 (e.g., sequence generator circuitry such as circuitry implemented by a computing device, a processing device, etc., and configured to generate / output CAN message sequences) provides CAN message sequences to the models 224. For example, the sequence generator 240 feeds (e.g., passes without modification) a normal CAN message sequence constructed based on the curves 228 and generates a synthetic anomalous CAN message sequence. In other words, the normal CAN message sequence corresponds to actual CAN message sequences observed on the CAN bus 236 and recorded by the CAN bus recorder 220. Conversely, the synthetic anomalous CAN message sequence is generated by the sequence generator 240.For example, the synthetic anomalous CAN message sequence includes message sequences that are not observed on the CAN bus 236 but may occur during operation (such as, for example, after an intrusion into the IVN). The sequence generator 240 may implement one or more algorithms that modify the normal CAN message sequences to simulate the effects of various known attacks on IVNs. In other words, the synthetic anomalous CAN message sequence represents possible normal CAN bus behaviors that the anomaly detection system 200, components of the IDS 208, etc., have nevertheless not seen or observed during normal operation.

[0033] In this way, the model 212 is constructed and trained to distinguish between (i) new or previously unseen "normal" CAN message sequences that do not correspond to an attack or intrusion, and (ii) CAN message sequences that do correspond to an attack or intrusion. As an example, the model 212 is constructed based on unmodified curves (e.g., curves 228) provided to the model constructor 240 and then trained based on both the normal CAN message sequences and the synthetic anomalous CAN message sequences received from the sequence generator 244. The normal CAN message sequences act as training sequences used to identify states (nodes) and transitions (edges) between the states that comprise the graph-based state-space model described below.After the model has been built from the training sequences, the model output classifier 216 is trained based on the model output to distinguish between normal and anomalous sequences. Once the model 212 and the model output classifier 216 are trained, a test set of sequences can be passed through the model 212 and the model output classifier 216 to determine precision and recall metrics of the anomaly detection system 200.

[0034] Typically, IVN signals / messages transmitted on CAN bus 236 are encoded. Accordingly, the anomaly detection system 200 according to the present disclosure communicates with (or includes, according to some examples), a decoder or decoder circuitry 248 configured to decode the IVN signals / messages (e.g., using a CAN database (CAN DBC) file). The decoder 248 provides, as the curves 228, an actual (i.e., decoded) signal content of the IVN messages from the multiple ECUs 232 to the relevant components of the anomaly detection system 200. For example, using longer sequences of decoded IVN signals received from decoder 248, sequence generator 244 generates shorter sequences of the decoded IVN signals (e.g., sequences with a predetermined or adjustable length). The decoded IVN messages represent actual signal values ​​(e.g.,commanded signals or control signals, sensed values ​​or measurements, etc.) transmitted by the ECUs 232 and may include strings or sequences of signal values, each with a corresponding timestamp. Accordingly, correlations between the actual signal values ​​can be observed by the model constructor 240 and other components of the anomaly detection system 200.

[0035] Typically, the ECUs 232 are connected to various sensors and actuators and broadcast signals in the network (e.g., the CAN bus 236). These signals may have continuous ranges within some intervals (e.g., engine speed, throttle position, vehicle speed, etc.) or may have discrete values ​​(e.g., transmission gear position (neutral, reverse, 1st, 2nd, 3rd, 4th, 5th, etc.), brake pedal status (depressed, not depressed, etc.). The values ​​of each signal type in the network may correspond to a time series. The time series may be examined and analyzed in isolation or together to detect anomalies (i.e., deviations from normal patterns). The state-space-based model of the present disclosure is configured to evaluate characteristics of IVN signals to better predict anomalies, as described below.

[0036] As one example, certain signals such as vehicle speed and throttle signals (e.g., a measured vehicle speed and a commanded or sensed throttle position) may be directly or indirectly correlated because throttle position controls speed. However, coded signals corresponding to vehicle speed and throttle, as communicated on CAN bus 236, may not exhibit any correlation. Because IVN messages are decoded according to the present disclosure, detection of anomalous behavior by anomaly detection system 200 is facilitated. For example, anomaly detection system 200 may determine to inject spoofed signal values ​​into the system if decoded IVN messages corresponding to two correlated signals do not correlate as expected (e.g., vehicle speed increases while throttle position decreases).More specifically, model 212 is constructed partly based on expected correlations between IVN signals.

[0037] As another example, a recent history of the signal, along with correlations with other signal values, may provide anomaly detection system 200 with additional context for processing message sequences. For example, if recent values ​​of a vehicle speed signal increased along with corresponding values ​​of an accelerator pedal position signal, future values ​​of a vehicle speed signal may be expected to continue increasing as the vehicle accelerates. A deviation from this pattern (e.g., a vehicle speed that decreases while acceleration increases) may indicate an anomaly. Accordingly, anomaly detection system 200 of the present disclosure incorporates a "memory" of recent signal values ​​into states of the state-space-based model.Each state encodes not only the most recently observed values ​​of signals, but also values ​​from a recent history of signals to facilitate the identification of signal patterns.

[0038] Fig. 3A shows an exemplary signal, graph, and state-space-based model 300 implemented by the anomaly detection system 200 according to the present disclosure. The model 300 is represented as a graph G = (V, E), where V, a set of nodes in the graph, represents states of the system, and E, a set of directed edges, represents transitions between the states. Each state S, corresponding to a node in the graph, encodes the values ​​of signals from a set of correlated signals (e.g., as signal snapshots). A signal snapshot is an n-tuple of signal values. Each signal snapshot specifies signal values ​​that characterize the state of the system at a time. Each state may contain more than one signal snapshot. For example, a set of signal snapshots in a state encodes the most recent signal values ​​(e.g., in chronological order).

[0039] For example only, the model 300 shown corresponds to a system with only two signals: ThrottlePosition and VehicleSpeed. Since these signals have continuous ranges, the values ​​can be discretized into ranges according to assigned criteria. According to this example, the ranges are divided into intervals with a length of five units. Each state encodes two signal snapshots. For example, state S i two signal snapshots: ((ThrottlePosition = (5, 10), VehicleSpeed ​​= 20, 25)) and (ThrottlePosition = (5, 10), VehicleSpeed ​​= (20, 25)). Each signal snapshot contains values ​​of the two signals at a time. As in Fig. As shown in Figure 3A, a most recent signal snapshot for each of the states (e.g., as shown at 302) is listed at the top, while the previous snapshot (e.g., as shown at 304) is listed at the bottom. Together, the two signal snapshots for each state in model 300 represent a storage capacity of two.

[0040] Edges (ie, transitions) between states represent state transitions generated by observations of a signal in the IVN. For example, an edge 306 between the states S i and S j represents a transition induced by a signal value of (25, 30) for VehicleSpeed ​​observed in the network. This transition causes the system to S j where the most recent signal snapshot (ThrottlePosition = (5, 10), VehicleSpeed ​​= (25, 30)) reflects a change in VehicleSpeed ​​from the previous signal snapshot. The most recent signal snapshot in state Si the previous (older) signal snapshot is in state S j .

[0041] Furthermore, each edge / transition is assigned a transition probability. For example, edge 306, which connects the states S i and S j connects, a transition probability p ij The transition probabilities can be estimated empirically from curves 228. For example, the transition from S i to S j assigned transition probability p ij = 0.8. In other words, of all outward transitions from S i 80% of the transitions correspond to transitions from S i to S j (e.g., and correspondingly the remaining 20% ​​transitions from S i to S k). Accordingly, each transition probability assigned to each corresponds to a probability of a transition between the two states. In this way, the model 300 is constructed (e.g., by the model constructor 240 in response to decoded IVN signals received from the decoder 248) based on the actually observed (i.e., "normal") behavior on the CAN bus 236.

[0042] For example, corresponding states (e.g., each corresponding to two signal snapshots of correlated signals) are added to the model 300 as CAN message sequences are received. Similarly, edges / transitions between the states are added to the model 300 as signals transition between states. All transitions between states are counted over time to calculate representative probabilities of transitions between any two of the states. Accordingly, the model constructor 240 (i) adds states to the model 300, (ii) adds transitions between states to the model 300, and (iii) counts and updates transitions between states of the model 300 to calculate transition probabilities for each transition as the CAN message sequences are received.

[0043] Fig. 3B shows an example classification process 310 (e.g., implemented by the model output classifier 216) configured to classify IVN signal sequences based on an output 312 of the model 212 / 300. For example, the output 312 of the model 300 contains values ​​based on the assigned transition probabilities for transitions between different states for one or more signals, as described above. The model output classifier 216 analyzes sequences of messages to predict whether the sequences are normal or anomalous (i.e., contain anomalies indicating attacks or intrusion attempts). For example, each signal input to the model 300 (e.g., from the sequence generator 244) may correspond to a state transition. Some of the state transitions may already be present in the model 300 (i.e.,where they have already been observed by the model constructor 240), while others may not be included in the model 300 because, for example, (i) the training data used to construct the model 300 did not contain the missing state transition or (ii) the state transition is invalid and potentially an anomaly.

[0044] As in Fig. As shown in Figure 3B, the output 312 of the model 300 assigns a value v to each state transition associated with a sequence of input signals. The output 312 represents n state transitions associated with n signals observed in the IVN. The transition from each state S i to a corresponding next state or successor state S j is a value v ijAccordingly, the output 312 of the model 300 for the sequence is a vector of the values ​​associated with the state transitions. The model output classifier 216 receives the vector of values ​​for the state transitions and outputs 0 in response to a determination that the values ​​correspond to a normal message sequence, and outputs 1 in response to a determination that the values ​​correspond to anomalous message sequences. Various types of classification techniques, including, but not limited to, a threshold classifier, an ML classifier (e.g., implementing a support vector machine (SVM) algorithm, a random forest algorithm, etc.), and a DL classifier based on neural networks, may be implemented by the model output classifier.

[0045] Various methods, including various heuristic methods, may be used to assign values ​​to the state transitions (i.e., to evaluate the state transitions). As one example, each value assigned to a state transition may correspond to a transition probability associated with the corresponding edge in the model 300. As another example, each value assigned to a state transition may be a measure of a distance of a next state from a previous state in a transition. For example, the distance is 1 if the previous state and the next state in the model 300 form an edge (i.e., they are directly connected by an edge / transition). Conversely, the distance is greater than 1 (such asestimated using heuristic or other methods described in more detail below) if the previous state and the next state are not directly connected. As yet another example, each value associated with a state transition may be a measure of a probability of observing a specific signal change associated with the state transition (e.g., a probability that the specific transition could have occurred).

[0046] Fig. 3C illustrates exemplary anomaly detection performed using distance measurements between the previous state (source state) and the next state (target state) in the model 300 for a normal message sequence (in a first table 320) and an anomalous message sequence (in a second table 324). According to this example, two signals are considered: ThrottlePosition (e.g., received from a first ECU (ECU1) of the ECUs 232) and VehicleSpeed ​​(e.g., received from a second ECU (ECU2) of the ECUs 232). Each state of a corresponding model (such as a model similar to the model 300) encodes the two most recent signal snapshots containing signal values ​​for the two signals. In the tables 320, 324, a signal column contains a most recent signal value of each signal, and a source state column and a target state column contain a source state (previous state) and a target state column, respectively.a target state (next state) associated with a transition generated by the signal value in the signal column. A distance measure column contains a distance measure associated with the state transition or a distance value (e.g., a heuristic distance measure) associated with the state transition. Accordingly, each row represents a transition (e.g., labeled 1-6) from a source state to a target state.

[0047] The first table 320 represents signal values ​​and transitions of a normal message sequence. For example, transitions 1, 2, 3, and 6 may correspond to transitions included in the model (i.e., corresponding to directly known, previously observed transitions between states in the model) and may thus have a distance measure of 0 (i.e., "normal"). Conversely, transitions 4 and 5 have non-zero values ​​associated with them. For example, transitions 4 and 5 may not have been previously observed and added to the model (e.g., in response to training sequences), but they may not necessarily indicate an anomaly. Rather, the relatively low distance values ​​(2 and 1 for transitions 2 and 1, respectively) may instead indicate that the states correspond to unprecedented but nevertheless normal behavior.For example, the model output for the message sequence corresponding to transitions 1-6 of the first table 320 is a vector [0, 0, 0, 2, 1, 0]. As an example, a classification method implemented by the model output classifier 216 (e.g., a threshold classifier) ​​may be configured to compare an L2 or Euclidean norm or other computed value (e.g., a measure of the average distance of the transitions) of the vector to a predetermined threshold (e.g., a threshold chosen during training / construction of the model output classifier 216). If the computed value of the vector is less than (or, according to some examples, less than or equal to) the predetermined threshold, the message sequence may be classified as normal (e.g., it may be assigned a value of 0).

[0048] Conversely, the second table 324 is an exemplary representation of an anomalous message sequence. According to this example, a third ECU (ECU3) of the ECUs 232 represents an ECU that has been compromised and used to spoof VehicleSpeed ​​signal values ​​(i.e., to send VehicleSpeed ​​values ​​as if the signals originated from ECU2). For example, the transitions shown in rows 3 and 4 may contain spoofed values ​​initiated by ECU3. Accordingly, in response to receiving the message sequence shown in the second table 324, the model outputs a vector [0, 0, 4, 5, 5, 6] with larger distance measures corresponding to transitions 3, 4, 5, and 6. In other words, the distance measures associated with the state transitions resulting from the spoofed signals initiated by ECU3 have larger values. Thus, the model output classifier 216 (which e.g.implemented the same threshold classifier described above) assumes that the message sequence is anomalous. For example, the computed value of the vector (e.g., the L2 or Euclidean norm) is greater than the specified threshold.

[0049] In this manner, the anomaly detection system 200 implements a model configured to output vectors that can be identified (e.g., using the model output classifier 216) as either normal or anomalous.

[0050] To distinguish between normal and anomalous message sequences, the anomaly detection system 200 may implement various heuristic techniques or methods, such as a heuristic distance method (as described above) or a heuristic signal change probability method. The heuristic methods are configured to assign higher scores (e.g., values ​​of the vectors output by the model) to state transitions not included in the model. In this way, the model output classifier 216 is better able to distinguish between normal and anomalous message sequences, resulting in a lower incidence of false positives and improved accuracy during classification.

[0051] Fig. 3D illustrates an exemplary heuristic distance method as applied to a portion of a model 340. According to this example, A, B, and C correspond to signals received in each signal snapshot. A most recent state transition is shown between states S and S'. An edge 344 between states S and S' is shown with a dashed line to indicate that the edge 344 does not correspond to an edge added during training of the model 340 (i.e., from training messages / training message sequences). Accordingly, the anomaly detection system 300 analyzes the edge 344 to determine whether the corresponding message sequence is normal (but not observed during training) or anomalous.

[0052] To assign a value to a state transition corresponding to edge 344, a breadth-first search (BFS) traversal of nodes (i.e., states) (e.g., starting from state S and up to a predetermined or fixed distance) is performed in model 340. During the BFS traversal, a state in model 340 that is most similar to state S' among the nodes is identified (e.g., in accordance with a defined similarity measure, such as described in more detail below). For example, any states that are candidates for being a most similar state to state S' must agree on a pivot signal value, such as the signal value that caused the state transition from S to S' (e.g., as shown, a value A = (10, 15) for signal A). When the most similar state is identified (e.g.,S''), a first distance from state S to state S'' is calculated in model 340, and a second distance from S to S' is subtracted from the first distance. The difference between the first distance and the second distance corresponds to a "bonus" distance that must be traveled to transition from state S to reach a state S'' that is most similar to the last observed state S'. For example, a larger bonus distance can be assumed to more likely correspond to an anomalous message sequence. In other words, a bonus distance value of 0 corresponds to transitions between states that have already been observed and are included in model 340, while distances of 1 or greater correspond to transitions to a next state that have not been previously observed.The model output classifier 216 is trained with the vectors output by the model 340 for a plurality of normal and anomalous message sequences to learn to distinguish between the normal and anomalous message sequences based on distance measures.

[0053] As in Fig. As shown in Figure 3D, of all the states detected during the BFS run, state S'' is determined to be the most similar state to S'. Only one other state (S y ) agrees with respect to the pivot signal A. However, according to a similarity measure (described in more detail below), the state S'' has a similarity relative to the state S y, which has a similarity score of sim = 1, (since S'' also matches with respect to signal C) has a higher similarity score of sim = 2. Thus, the distance measure (d) associated with the transition from S to S' is given by a distance from S to S'' minus a distance from S to S' (i.e., d(S, S') = dist(G, S, S'') - dist(G, S, S') = 2 - 1 = 1). Conversely, the distance measure associated with the transition would be d(S, S u ) = dist(G, S, S u ) - dist(G, S, S u ) = 0 if S'' is the next state after S instead of S'.

[0054] The similarity (e.g., a similarity score) between two states can be determined using various methods. For two states S i and S j , the signal snapshots (s i1 , s i2 , ...) or (s j1 , s j2 , ...) encode, set variable s mnin the signal snapshots represents an n-th signal value in the m-th signal snapshot. As an example, a similarity score between S i and S j using an L1 or Manhattan norm of the vector (s i1 - s j1 , s i2 - s j2 , ...). For example, smaller L1 norms can correspond to states with signal values ​​(e.g., scaled signal values ​​as captured in respective signal snapshots) that are similar to each other.

[0055] According to one example, each iteration of a BFS traversal, starting from a node corresponding to the previous state, returns a next node of the model 340. The BFS traversal may be bounded by a fixed search distance (e.g., 3 nodes ahead, 4 nodes ahead, etc.).

[0056] If the search distance is exceeded, the BFS traversal does not return a valid state (e.g., it returns a NIL value). The pivot signal used for the BFS traversal is the most recent signal seen in the received message sequence (i.e., the signal value that caused the state transition). During each iteration, the state corresponding to the next node in the BFS traversal is extracted, and a similarity score of that state relative to the next state is determined. If the score exceeds the previous highest similarity score, the corresponding distance measure is updated. After all nodes within the allowed search distance have been processed and evaluated for respective similarity scores and distance measures, a distance measure value corresponding to the state with the highest similarity score is selected.If no similar node is found within the search distance of the BFS pass, the transition can be assigned a large constant value (e.g., a distance value of 10 or greater). Using the heuristic distance algorithm described above, each state transition associated with a signal value in a message sequence can be assigned a score, and the resulting output vector for the entire signal sequence can then be input to the model output classifier 216.

[0057] According to another example, the anomaly detection system implements a probabilistic heuristic signal change method configured to evaluate a change in the signal value that caused the transition. Using the training samples, a mean (µ) and a variance (σ 2) of the distribution of the difference between two consecutive values ​​of a signal. For a random variable X, which represents the change in the value of a signal in a sequence, a value k corresponds to a difference between X and the mean µ. From the Chebyshev inequality, a probabilistic inequality can be written as P{IX - µ| ≥ k} ≤ σ 2 / k 2 be defined. The value σ 2 / k 2 represents an upper bound on the probability that the actual signal value change can deviate from the mean (signal change) by more than k. Each state transition corresponding to a signal change in a signal sequence can then be assigned this probability measure. A lower value of the measure indicates that the magnitude of the signal change is unlikely and thus suggests an anomaly. Conversely, a higher value indicates that the signal change is very likely and thus normal.

[0058] In this way, the anomaly detection system 200 may be configured to implement one or more heuristic methods for evaluating state transitions corresponding to received message sequences.

[0059] Fig. 4 illustrates steps of an exemplary method 400 for constructing and implementing a model of an anomaly detection system in accordance with the present disclosure. The method 400 is implemented, for example, by one or more computing devices or computing systems and / or associated circuitry such as the anomaly detection system 200, a computing device (e.g., the computing device 100) configured to implement the anomaly detection system 200, etc.

[0060] At 404, the method 400 receives (e.g., using a CAN bus logger, decoder, etc., as described above) decoded CAN bus messages associated with messages transmitted between ECUs of a vehicle (e.g., training messages). For example, the CAN bus messages may be received from an IVN (i.e., from a diagnostic port of a vehicle), from a test device, etc. At 408, a model (e.g., a signal-, graph-, and state-space-based model such as model 212, 300, 340, etc.) is constructed using the received CAN bus messages. As described above, the model may consist of various observed signal states and observed transitions / edges between the signal states. Each node or state may be a sequence of one or more signal values ​​for one or more signals (e.g.,n different values ​​of m different signals, where n and m are each integers with a value of 1 or greater).

[0061] At 412, the model 400 (e.g., the sequence generator 244) trains the model by feeding both normal and synthetic anomalous message sequences to the model. For example, the normal message sequences correspond to message sequences included in the training messages.

[0062] Conversely, the synthetic anomalous message sequences contain message sequences that are not included in the training messages, but, as described above, may still occur during operation. In other words, the model is trained to recognize messages that were not included in the training messages but do not correspond to an intrusion or attack as "normal" messages. The model can be tested and / or updated by further feeding synthetic anomalous message sequences to estimate the accuracy of the model's outputs.

[0063] At 416, the trained model is implemented in a system such as an IVN of a vehicle. At 420, the trained model receives sequences of decoded messages (e.g., an IVN message sequence). At 424, the model outputs a vector corresponding to a value assigned to a state transition between states of one or more signals. For example, the model calculates the vector as described above using a heuristic distance method. At 428, the method 400 classifies the output as normal (e.g., outputs as "1") or anomalous (e.g., outputs as "0") based on the vector. At 432, the method 400 (e.g., the anomaly detection system 200, the IDS 208, etc.) optionally takes one or more corrective actions in response to a determination that a message sequence is anomalous.The corrective actions include, but are not limited to, disabling one or more vehicle systems or functions, warning the driver, instructing the driver to pull the vehicle over or turn it off, operating in a limited-function mode, etc. Conversely, in response to a determination that a message sequence is normal, method 400 proceeds to 420 for further monitoring of the decoded messages.

[0064] Fig. 5-11 illustrate exemplary systems and devices that may implement anomaly detection systems and anomaly detection methods according to the present disclosure. Fig. 5 shows a schematic representation of an interaction between a computer-controlled machine 500 and a control system 502. The computer-controlled machine 500 includes an actuator 504 and a sensor 506. The actuator 504 may include one or more actuators, and the sensor 506 may include one or more sensors. The sensor 506 is configured to detect a state of the computer-controlled machine 500. The sensor 506 may be configured to encode the detected state into sensor signals 508 and send the sensor signals 508 to a control system 502. Non-limiting examples of the sensor 506 include video, radar, LiDAR, ultrasonic, and motion sensors. According to some embodiments, the sensor 506 is an optical sensor configured to detect optical images of an environment proximate the computer-controlled machine 500.

[0065] The control system 502 is configured to receive sensor signals 508 from the computer-controlled machine 500. As set forth below, the control system 502 may be further configured to calculate actuator control commands 510 depending on the sensor signals and to send actuator control commands 510 to the actuator 504 of the computer-controlled machine 500.

[0066] As in Fig. 5, the control system 502 includes a receiving unit 512. The receiving unit 512 may be configured to receive sensor signals 508 from the sensor 506 and to convert the sensor signals 508 into input signals x. According to an alternative embodiment, the sensor signals 508 are received directly as input signals x without a receiving unit 512. Each input signal x may be a part of each sensor signal 508. The receiving unit 512 may be configured to process each sensor signal 508 to generate each input signal x. The input signal x may include data corresponding to an image recorded by the sensor 506.

[0067] The control system 502 includes a classifier 514. The classifier 514 may be configured to classify input signals x into one or more labels using a machine learning (ML) algorithm, such as a neural network. The classifier 514 is configured to be parameterized by parameters such as those described above (e.g., parameter θ). The parameters θ may be stored in and provided by the non-volatile storage 516. The classifier 514 is configured to determine output signals y from input signals x. Each output signal y includes information that assigns one or more labels to each input signal x. The classifier 514 may send output signals y to the conversion unit 518. The conversion unit 518 is configured to convert output signals y into actuator control commands 510.The control system 502 is configured to send actuator control commands 510 to the actuator 504, which is configured to actuate the computer-controlled machine 500 in response to the actuator control commands 510. According to some embodiments, the actuator 504 is configured to actuate the computer-controlled machine 500 directly based on output signals y.

[0068] The actuator 504 is configured to perform an action corresponding to the associated actuator control command 510 upon receipt of the actuator control commands 510 by the actuator 504. The actuator 504 may include control logic configured to convert the actuator control commands 510 into a second actuator control command used to control the actuator 504. According to one or more embodiments, the actuator control commands 510 may be used to control a display instead of, or in addition to, an actuator.

[0069] According to some embodiments, instead of or in addition to the computer-controlled machine 500 including a sensor 506, the control system 502 includes a sensor 506. Furthermore, instead of or in addition to the computer-controlled machine 500 including the actuator 504, the control system 502 may also include an actuator 504.

[0070] As in Fig. 5, control system 502 also includes a processor 520 and a memory 522. Processor 520 may include one or more processors. Memory 522 may include one or more storage devices. Classifier 514 (e.g., ML algorithms) of one or more embodiments may be implemented by control system 502, which includes non-volatile storage 516, processor 520, and memory 522.

[0071] Non-volatile storage 516 may include one or more persistent data storage devices, such as a hard disk drive, an optical drive, a tape drive, a non-volatile solid-state device, cloud storage, or any other device capable of persistently storing information. Processor 520 may include one or more devices selected from high-performance computing (HPC) systems containing high-performance cores, microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, field-programmable logic arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other devices that manipulate signals (analog or digital) based on computer-executable instructions residing in memory 522.The memory 522 may include a single memory device or a number of memory devices, including, but not limited to, random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache memory, or any other device capable of storing information.

[0072] Processor 520 may be configured to read into memory 522 and execute computer-executable instructions located in non-volatile storage 516 and embodying one or more anomaly detection methodologies of one or more embodiments. Non-volatile storage 516 may include one or more operating systems and applications. Non-volatile storage 516 may store compiled and / or interpreted computer programs generated using a variety of programming languages ​​and / or technologies, including, without limitation, and either alone or in combination, Java, C, C++, C#, Objective-C, Fortran, Pascal, Java Script, Python, Perl, and PL / SQL.

[0073] When executed by processor 520, the computer-executable instructions of non-volatile storage 516 may cause control system 502 to implement one or more of the anomaly detection methodologies disclosed herein. Non-volatile storage 516 may also contain data supporting the functions, features, and processes of one or more embodiments described herein.

[0074] The program code embodying the algorithms and / or methodologies described herein is capable of being distributed individually or collectively as a program product in a variety of different forms. The program code may be distributed using a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to perform aspects of one or more embodiments. Computer-readable storage media, which are inherently non-transitory, may include volatile and non-volatile, and removable and non-removable tangible media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data.Furthermore, the computer-readable storage media may include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technology, portable compact disc read-only memory (CD-ROM) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be read by a computer. Computer-readable program instructions may be downloaded from a computer-readable storage medium into a computer, another type of programmable data processing apparatus or device, or over a network to an external computer or external storage device.

[0075] Computer-readable program instructions stored on a computer-readable medium can be used to instruct a computer, other types of programmable computing devices, or other apparatus to act in a particular manner so that the instructions stored on the computer-readable medium, including instructions that implement the functions, acts, and / or operations specified in the flowcharts or diagrams, produce an article of manufacture. According to certain alternative embodiments, the functions, acts, and / or operations specified in the flowcharts and diagrams can be rearranged, processed sequentially, and / or processed concurrently in accordance with one or more embodiments.Moreover, any of the flowcharts and / or illustrations may include more or fewer nodes or blocks than those illustrated in accordance with one or more embodiments.

[0076] The processes, methods or algorithms may be embodied in whole or in part using suitable hardware components such as application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), state machines, controllers or other hardware components or devices, or a combination of hardware, software and firmware components.

[0077] Fig. 6 shows a schematic representation of the control system 502 configured to control a vehicle 600, which may be an at least partially autonomous vehicle or an at least partially autonomous robot. The vehicle 600 includes an actuator 504 and a sensor 506. The sensor 506 may include one or more video sensors, cameras, radar sensors, ultrasonic sensors, LiDAR sensors, and / or position sensors (e.g., GPS). One or more of the one or more specific sensors may be integrated into the vehicle 600. Alternatively, or in addition to one or more specific sensors identified above, the sensor 506 may include a software module configured, when executed, to determine a state of the actuator 504.A non-limiting example of a software module includes a weather information software module configured to determine a current or future state of the weather in the vicinity of the vehicle 600 or at another location.

[0078] The classifier 514 of the control system 502 of the vehicle 600 may be configured to detect objects in the vicinity of the vehicle 600 depending on input signals x. According to such an embodiment, the output signal y may contain information characterizing the proximity of an object to the vehicle 600. The actuator control command 510 may be determined according to this information. The actuator control command 510 may be used to avoid collisions with the detected objects.

[0079] According to some embodiments, if the vehicle 600 is an at least partially autonomous vehicle, the actuator 504 may be embodied in a brake, a propulsion system, a prime mover, a drivetrain, or a steering system of the vehicle 600. The actuator control commands 510 may be determined such that the actuator 504 is controlled such that the vehicle 600 avoids collisions with detected objects. Detected objects may also be classified according to what the classifier 514 considers them most likely to be, such as pedestrians or trees. The actuator control commands 510 may be determined depending on the classification. According to a scenario in which a hostile attack may occur, the system described above may be further trained to better detect objects or to identify a change in lighting conditions or angle for a sensor or camera on the vehicle 600.

[0080] According to some embodiments in which the vehicle 600 is an at least partially autonomous robot, the vehicle 600 may be a mobile robot configured to perform one or more functions, such as flying, swimming, diving, and walking. The mobile robot may be an at least partially autonomous lawnmower or an at least partially autonomous cleaning robot. According to such embodiments, the actuator control command 510 may be determined such that a propulsion unit, a steering unit, and / or a braking unit of the mobile robot can be controlled such that the mobile robot can avoid collisions with identified objects.

[0081] According to some embodiments, the vehicle 600 is an at least partially autonomous robot in the form of a gardening robot. According to such an embodiment, the vehicle 600 may use an optical sensor as sensor 506 to determine a condition of plants in an environment near the vehicle 600. The actuator 504 may be a nozzle configured to spray chemicals. Depending on an identified type and / or condition of the plants, it may be determined that the actuator control command 510 causes the actuator 504 to spray the plants with an appropriate amount of suitable chemicals.

[0082] The vehicle 600 may be an at least partially autonomous robot in the form of a household appliance. Non-limiting examples of household appliances include a washing machine, a stove, an oven, a microwave, or a dishwasher. In such a vehicle 600, the sensor 506 may be an optical sensor configured to detect a state of an object undergoing processing by the household appliance. For example, in the case of a washing machine, the sensor 506 may detect a state of the laundry within the washing machine. Based on the detected state of the laundry, an actuator control command 510 may be determined.

[0083] Fig. 7 shows a schematic representation of a control system 502 configured to control a system 700 (e.g., a manufacturing machine), such as a die cutter, a cutting machine, or a spindle drill of the manufacturing system 702, such as a portion of a production line. The control system 502 may be configured to control the actuator 504 configured to control the system 700 (e.g., the manufacturing machine).

[0084] The sensor 506 of the system 700 (e.g., a manufacturing machine) may be an optical sensor configured to detect one or more characteristics of the manufactured product 704. The classifier 514 may be configured to determine a state of the manufactured product 704 from one or more of the detected characteristics. The actuator 504 may be configured to control the system 700 (e.g., a manufacturing machine) depending on the determined state of the manufactured product 704 for a subsequent manufacturing step of the manufactured product 704. The actuator 504 may be configured to control functions of the system 700 (e.g., a manufacturing machine) on a subsequently manufactured product 706 of the system 700 (e.g., the manufacturing machine) depending on the determined state of the manufactured product 704.

[0085] Fig. Figure 8 shows a schematic representation of a control system 502 configured to control a power tool 800, such as an electric drill or an electric screwdriver, that has an at least partially autonomous operating mode. The control system 502 may be configured to control the actuator 504, which is configured to control the power tool 800.

[0086] The sensor 506 of the power tool 800 may be an optical sensor configured to detect one or more characteristics of the work surface 802 and / or a fastener 804 being driven into the work surface 802. The classifier 514 may be configured to determine a condition of the work surface 802 and / or the fastener 804 relative to the work surface 802 from one or more of the detected surfaces. The condition may be that the fastener 804 is flush with the work surface 802. Alternatively, the condition may be the hardness of the work surface 802. The actuator 504 may be configured to control the power tool 800 such that the driving function of the power tool 800 is adjusted depending on the determined condition of the fastener 804 relative to the work surface 802 or one or more detected characteristics of the work surface 802.For example, the actuator 504 may interrupt the driving function if the condition of the fastener 804 is flush relative to the work surface 802. As another non-limiting example, the actuator 504 may apply additional or lesser torque depending on the hardness of the work surface 802.

[0087] Fig. Figure 9 shows a schematic diagram of a control system 502 configured to control an automated personal assistant 900. The control system 502 may be configured to control the actuator 504 configured to control the automated personal assistant 900. The automated personal assistant 900 may be configured to control a household appliance such as a washing machine, a stove, an oven, a microwave, or a dishwasher.

[0088] Sensor 506 may be an optical sensor and / or an audio sensor. The optical sensor may be configured to receive video images of gestures 904 from user 902. The audio sensor may be configured to receive a voice command from user 902.

[0089] The control system 502 of the automated personal assistant 900 may be configured to determine actuator control commands 510 configured to control the system 502. The control system 502 may be configured to determine actuator control commands 510 in accordance with sensor signals 508 from the sensor 506. The automated personal assistant 900 is configured to send sensor signals 508 to control the system 502. The classifier 514 of the control system 502 may be configured to execute a gesture recognition algorithm to identify a gesture 904 performed by the user 902, determine actuator control commands 510, and send the actuator control commands 510 to the actuator 504.The classifier 514 may be configured to read information from a non-volatile storage memory in response to a gesture 904 and to output the read information in a form suitable for receipt by the user 902.

[0090] Fig. 10 shows a schematic diagram of a control system 502 configured to control a control system 1000. The control system 1000 may be configured to physically control access through the door 1002. The sensor 506 may be configured to detect a scene relevant to determining whether access is granted. The sensor 506 may be an optical sensor configured to generate and transmit image and / or video data. Such data may be used by the control system 502 to detect a person's face.

[0091] The classifier 514 of the control system 502 of the control system 1000 may be configured to interpret the image and / or video data by matching it to the identities of known people stored in the non-volatile storage 516, thereby determining an identity of a person. The classifier 514 may be configured to generate an actuator control command 510 in response to the interpretation of the image and / or video data. The control system 502 is configured to send the actuator control command 510 to the actuator 504. According to this embodiment, the actuator 504 may be configured to lock or unlock the door 1002 in response to the actuator control command 510. According to some embodiments, non-physical logical access control is also possible.

[0092] The control system 1000 may also be a surveillance system. According to such an embodiment, the sensor 506 may be an optical sensor configured to detect a scene under surveillance, and the control system 502 is configured to control the display 1004. The classifier 514 is configured to determine a classification of a scene, e.g., whether the scene detected by the sensor 506 is suspicious. The control system 502 is configured to send an actuator control command 510 to the display 1004 in response to the classification. The display 1004 may be configured to adjust the displayed content in response to the actuator control command 510. For example, the display 1004 may highlight an object deemed suspicious by the classifier 514.The monitoring system can predict objects that will appear at certain times in the future using an embodiment of the disclosed system.

[0093] Fig.11 shows a schematic representation of a control system 502 configured to control an imaging system 1100, e.g., an MRI device, an x-ray imaging device, or an ultrasound device. Sensor 506 may, for example, be an imaging sensor. Classifier 514 may be configured to determine a classification of all or a portion of the acquired image. Classifier 514 may be configured to determine or select an actuator control command 510 in response to the classification obtained by the trained neural network. For example, classifier 514 may interpret a region of an acquired image as potentially anomalous. In this case, actuator control command 510 may be determined or selected to cause display 1102 to display the imaging and highlight the potentially anomalous region.

[0094] Although exemplary embodiments are described above, these embodiments are not intended to describe all possible forms encompassed by the claims. The words used in the specification are words of description rather than limitation, and it is to be understood that various changes may be made without departing from the spirit and scope of the disclosure. As previously described, the features of various embodiments may be combined to form further embodiments of the invention that may not be explicitly described or illustrated.Although various embodiments may be described as providing advantages or being preferred over other embodiments or prior art implementations with respect to one or more desired characteristics, one of ordinary skill in the art will recognize that compromises may be made with respect to one or more features or characteristics to achieve desired overall system attributes depending on the specific application and implementation. These attributes may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, serviceability, weight, manufacturability, ease of assembly, etc.Thus, to the extent that any embodiments are described as less desirable than other embodiments or prior art implementations with respect to one or more characteristics, these embodiments are not outside the scope of the disclosure and may be desirable for certain applications.

Claims

[1] A method of operating an anomaly detection system, the method comprising: Receiving training message sequences corresponding to messages transmitted in an in-vehicle network (IVN); Constructing a model based on the training message sequences, the model including a plurality of states corresponding to observed signal values in the training message sequences and state transitions between respective states of the plurality of states; Training the model by supplying (i) first message sequences corresponding to the training message sequences and (ii) second message sequences not included in the training message sequences to the model; and Executing the model to identify anomalous message sequences transmitted in the IVN using the anomaly detection system by (i) receiving an IVN message sequence, (ii) outputting a value based on state transitions between states of signals included in the IVN message sequence from the model, and (iii) outputting an indication of whether the IVN message sequence contains an anomalous message sequence based on the value. [2] The method of claim 1, further comprising decoding the training message sequences and constructing the model based on the decoded training message sequences. [3] The method of claim 1, wherein each of the plurality of states in the model corresponds to values of two or more correlated signals included in the training message sequences. [4] The method of claim 3, wherein each of the plurality of states includes at least two values for each of the two or more correlated signals. [5] The method of claim 4, wherein the model identifies probabilities of each of the state transitions between the respective states of the plurality of states. [6] The method of claim 1, wherein outputting the value from the model comprises: (i) calculating a distance between a first state of the plurality of states corresponding to a first message in the IVN message sequence and a second state corresponding to a second message in the IVN message sequence, and calculating the value based on the distance; and / or (ii) Calculating the value using a heuristic probability method. [7] The method of claim 6, wherein the distance corresponds to a number of state transitions in the model required to traverse between the first state and the second state. [8] The method of claim 7, wherein the second state does not correspond to any message included in the training message sequences. [9] Anomaly detection system comprising: a model constructor circuitry configured to (i) receive training message sequences corresponding to messages transmitted in an in-vehicle network (IVN), and (ii) construct a model based on the training message sequences, the model including a plurality of states corresponding to observed signal values in the training message sequences and state transitions between respective states of the plurality of states; a sequence generator circuit configured to train the model by supplying (i) first message sequences corresponding to the training message sequences and (ii) second message sequences not included in the training message sequences to the model; a model circuitry configured to execute the model to identify anomalous message sequences transmitted in the IVN by (i) receiving an IVN message sequence and (ii) outputting a value based on state transitions between states of signals included in the IVN message sequence from the model; and model output classifier circuitry configured to output an indication of whether the IVN message sequence contains an anomalous message sequence based on the value. [10] The anomaly detection system of claim 9, further comprising a decoder configured to decode the training message sequences, wherein the model constructor circuitry is configured to construct the model based on the decoded training message sequences. [11] The anomaly detection system of claim 9, wherein each of the plurality of states in the model corresponds to values of two or more correlated signals included in the training message sequences. [12] The anomaly detection system of claim 11, wherein each of the plurality of states includes at least two values for each of the two or more correlated signals. [13] The anomaly detection system of claim 12, wherein the model circuitry is configured to implement the model to identify probabilities of each of the state transitions between the respective states of the plurality of states. [14] The anomaly detection system of claim 9, wherein the model circuitry is configured to implement the model for: (i) calculating a distance between a first state of the plurality of states corresponding to a first message in the IVN message sequence and a second state corresponding to a second message in the IVN message sequence, and calculating the value based on the distance; and / or (ii) Calculating the value using a heuristic probability method. [15] The anomaly detection system of claim 14, wherein the distance corresponds to a number of state transitions in the model required to traverse between the first state and the second state. [16] The anomaly detection system of claim 15, wherein the second state does not correspond to any message included in the training message sequences. [17] A computing device configured to implement an anomaly detection system for a vehicle, the computing device including a processing device configured to execute instructions stored in memory to cause the anomaly detection system to: receives an in-vehicle network (IVN) message sequence corresponding to messages transmitted between electronic control units (ECUs) of the vehicle; providing the IVN message sequence to a model, the model including (i) a plurality of states corresponding to observed signal values in previous IVN message sequences, and (ii) state transitions between respective states of the plurality of states; outputs a value from the model based on state transitions between states of signals contained in the IVN message sequence; and based on the value, returns an indication as to whether the IVN message sequence contains an anomalous message sequence. [18] The computing device of claim 17, wherein the processing device is further configured to execute instructions stored in memory to cause the anomaly detection system to decode the IVN message sequence and to provide the decoded IVN message sequence to the model. [19] The computing device of claim 18, wherein each of the plurality of states in the model corresponds to values of two or more correlated signals included in the decoded IVN message sequence, each of the plurality of states including at least two values for each of the two or more correlated signals, and wherein the model identifies probabilities of each of the state transitions between the respective states of the plurality of states. [20] The computing device of claim 17, wherein the processing device is further configured to execute instructions stored in memory to cause the anomaly detection system to perform the following: (i) calculating a distance between a first state of the plurality of states corresponding to a first message in the IVN message sequence and a second state corresponding to a second message in the IVN message sequence, and calculating the value based on the distance, wherein the distance corresponds to a number of state transitions in the model required to traverse between the first state and the second state; and / or (ii) Calculating the value using a heuristic probability method.

Images