Systems and methods for the sequential detection of anomalies in IVN networks using a graph-based state-space approach

Abstract

Title: Systems and methods for sequential anomaly detection in IVN networks using a graph-based state-space approach. The method for operating an anomaly detection system consists of receiving training message sequences transmitted in an embedded network (IVN), constructing a model that includes a set of states corresponding to signal values ​​observed in the training message sequences and state transitions, training the model with elements from the first message sequences corresponding to the training message sequences and second message sequences not present in the training message sequences, and running the model to identify abnormal message sequences transmitted in the IVN network. Figure 1

Description

Title of the invention: Systems and methods for sequential anomaly detection in IVN networks using a graph-based state-space approach Technical field

[0001] The present invention relates to the detection of anomalies in on-board networks (IVN). STATE OF THE ART

[0002] The purpose of the description of the prior art is to present in general terms the background of the invention. The work of the currently named inventors, to the extent described in this section, as well as aspects of the description which might not qualify as prior art at the time of filing, are neither expressly nor implicitly admitted as prior art with respect to the present invention.

[0003] Vehicles, such as cars, trucks, utility vehicles, sport utility vehicles, crossover vehicles, minivans, or other vehicles, have a propulsion system, a braking system, a steering system, etc. The systems of these vehicles may be controlled manually (e.g., by the driver) and / or autonomously or semi-autonomously (e.g., by one or more autonomous or semi-autonomous vehicle controls). In some examples, the autonomous or semi-autonomous vehicle controls rely on various machine learning models for decision-making regarding vehicle control or use.

[0004] A CAN (Controller Area Network) bus is the central communication network of many modern systems such as automotive systems, aerospace systems, industrial systems, etc. For example, electronic control units (ECUs) control several subsystems of vehicles and can communicate with each other via the CAN bus to ensure the proper functioning of one or more of the subsystems. Some nodes of the CAN bus, and more generally of the internal network of the vehicle, are equipped with remote interfaces. These interfaces are often used for over-the-air updates, offering additional services, measuring the use of applications and measuring the use of certain functions in the electronic control units (ECUs) for maintenance services, etc.

[0005] The method of operating an anomaly detection system comprises receiving sequences of training messages corresponding to messages transmitted in an in-vehicle network (IVN), constructing, on the basis of the sequences of training messages, a model that includes a set of states corresponding to signal values ​​in the training message sequences and state transitions between respective states of the set of states, training the model by providing to the model, the following of the first message sequences corresponding to the training message sequences and the second message sequences not in the training message sequences and, using the anomaly detection system, executing the model to identify abnormal message sequences transmitted in the IVN network by receiving an IVN message sequence, outputting, from the model, a value based on the state transitions between the states of the signals contained in the IVN message sequence and outputting, based on the value, an indication that the IVN message sequence includes an abnormal message sequence.

[0006] According to other features, the method comprises decoding the training message sequences and constructing the model based on the decoded training message sequences. Each of the states of the model corresponds to the values ​​of at least two correlated signals contained in the training message sequences. Each of the states comprises at least two values ​​for each of the two or more correlated signals. The model identifies the probabilities of each of the state transitions between the respective states of the set of states. Outputting the value of the model comprises calculating a distance between a first state of the set of states corresponding to a first message of the IVN message sequence and a second state corresponding to a second message of the IVN message sequence and calculating the value based on the distance, using a probability heuristic method.The distance is the number of state transitions in the model required to move from the first to the second state. The second state does not correspond to any message contained in the training message sequences.

[0007] The anomaly detection system comprises a model construction circuit for receiving training message sequences corresponding to messages transmitted in an in-vehicle network (IVN) and constructing, based on the training message sequences, a model comprising a set of states corresponding to signal values ​​observed in the training message sequences and state transitions between respective states of the set of states, a sequence generation circuit for training the model by providing thereto, first message sequences corresponding to the training message sequences and second message sequences not contained in the training message sequences, a model circuit for executing the model to identify abnormal message sequences transmitted in the IVN by receiving an IVN message sequence and outputting, from the model, a value based on the 。 state transitions between states of signals contained in the IVN message sequence, and a model output classifier circuit for outputting, based on the value, an indication that the IVN message sequence comprises an abnormal sequence of messages.

[0008] According to other features, the anomaly detection system comprises a decoder configured to decode the training message sequences. The model building circuit is configured to build the model based on the decoded training message sequences. Each of the states of the model corresponds to the values ​​of at least two correlated signals contained in the training message sequences. Each of the states comprises at least two values ​​for each of the two or more correlated signals. The model circuit implements the model to identify the probabilities of each of the state transitions between the respective states of the set of states.The modeling circuit implements the model to calculate the distance between a first state of the set of states corresponding to a first message of the IVN message sequence and a second state corresponding to a second message of the IVN message sequence, and to calculate the value based on the distance, using a probabilistic heuristic method. The distance corresponds to the number of state transitions in the model, necessary to go from the first state to the second state. The second state does not correspond to any message contained in the training message sequences.

[0009] The computing device configured to implement the anomaly detection system of a vehicle comprises a processing device for executing instructions stored in the memory so that the anomaly detection system receives a sequence of messages from the in-vehicle network (IVN) corresponding to messages exchanged between electronic control units (ECUs) of the vehicle, provides the IVN message sequence to a model that includes a set of states corresponding to signal values, observed in prior IVN message sequences and state transitions between respective states of the set of states, outputs, from the model, a value based on the state transitions between the states of the signals contained in the IVN message sequence, and outputs, based on the value, an indication that the IVN message sequence includes an abnormal sequence of messages.

[0010] According to other features, the processing device executes instructions stored in the memory so that the anomaly detection system decodes the IVN message sequence and provides the decoded IVN message sequence to the model. Each of the states of the model corresponds to the values ​​of at least two correlated signals contained in the decoded IVN message sequence, each of the states comprises at least two values ​​for each of the two correlated signals or more, and the model identifies the probabilities of each of the state transitions between the respective states of the set of states. The processing device executes instructions stored in the memory so that the anomaly detection system can at least calculate the distance between a first state of the set of states corresponding to a first message of the IVN message sequence and a second state corresponding to a second message of the IVN message sequence, and calculate the value based on the distance, corresponding to the number of state transitions in the model required to go from the first state to the second state, and calculate the value of a probability-based heuristic method. Brief description of the drawings

[0011] [Fig-1] example of a computer device according to the present invention,

[0012] [Fig.2] example of anomaly detection system according to the present invention,

[0013] [Fig.3A] Example of a space-based state model implemented by an anomaly detection system according to the present invention,

[0014] [Fig.3B] exemplary classification process configured to classify IVN signal sequences based on an output of a model according to the present invention,

[0015] [Fig.3C] tables illustrating an example of anomaly detection using distance measurements between the previous (source) and next (target) states in a model for a normal message sequence and an abnormal message sequence according to the present invention,

[0016] [Fig.3D] example of a heuristic distance method applied to a part of a model according to the present invention,

[0017] [Fig.4] steps of an example of a method of construction and implementation of a anomaly detection system model according to the present invention,

[0018] [Fig.5] Schematic diagram of an interaction between a machine controlled by computer and a control system according to the present invention,

[0019] [Fig.6] diagram of the control system of [Fig.5] configured to control a vehicle, a partially autonomous vehicle, a fully autonomous vehicle, a partially autonomous robot or a fully autonomous robot, according to the present invention,

[0020] [Fig.7] Schematic diagram of the control system of [Fig.5] for control a manufacturing machine, such as a punch press, milling machine, or hand drill, a manufacturing system, such as part of a production line,

[0021] [Fig.8] diagram of the control system of [Fig.5] for controlling a power tool, such as a drill or screwdriver, in at least partially autonomous mode,

[0022] [Fig.9] diagram of the control system of [Fig.5] for controlling an assistant automated personnel,

[0023] [Fig. 10] diagram of the control system of [Fig.5] for controlling a surveillance system, such as an access control system or a surveillance system,

[0024] [Fig. 11] shows a diagram of the control system of [Fig.5] configured to control an imaging system, for example an MRI machine, an X-ray imaging machine or an ultrasound machine.

[0025] In the drawings, references identify similar and / or identical elements. DETAILED DESCRIPTION

[0026] Examples of the present disclosure are given herein. It is understood, however, that the disclosed embodiments are only examples and that other embodiments may take various and alternative forms. The figures are not necessarily to scale; certain features may be increased or decreased to show details of particular components. Therefore, the specific structural and functional details described are not limiting, but are merely an example for those skilled in the art in using the embodiments in a variety of ways. The various features illustrated and described with reference to the figures may be combined with features of other figures to obtain embodiments not explicitly illustrated or described. The illustrated combinations of features correspond to exemplary embodiments.Various combinations and modifications of the features of this disclosure may be contemplated for particular applications or implementations.

[0027] Vehicle systems may be vulnerable to external attacks at multiple levels or layers, such as a physical layer (e.g., sensors, actuators, etc.), an in-vehicle network (IVN) layer, an application layer, etc. through various external network interfaces (e.g., cellular, Bluetooth, Wi-Fi, etc.). Automotive IVNs (e.g., controller area networks (CANs)) connect various electronic control units (ECUs) of a vehicle. ECUs broadcast messages over the IVNs to communicate information to each other about various system parameters, such as sensor data (e.g., engine speed, throttle position, etc.). IVNs are vulnerable to malicious attacks / intrusions due to the large number of electrical control units (ECUs), the connectivity of the ECUs with the external network, the increased dependence of the ECUs on software, etc.An attacker can compromise a UCE unit through external interfaces, inject spurious signal values, prevent the transmission of critical signals on the IVN network, etc. etc. Anomaly detection systems that operate based on IVN network signals observe network traffic and attempt to distinguish normal traffic from abnormal traffic patterns that appear when one or more ECUs have been compromised.

[0028] Accordingly, network intrusion detection is an integral part of automotive security in order to prevent or at least reduce the impact of such intrusions. Various methods can be implemented to improve automotive security, including, but not limited to, firewalls, whitelists, message blacklists, and dedicated intrusion detection systems (IDS) implemented on one or more ECUs (e.g., on gateway ECUs). An intrusion detection system is a system that monitors predefined systems and / or networks and identifies intrusion attempts by detecting suspicious, malicious, or undefined system behavior. The operation of the IDS system relies on a prerequisite: the intrusion attempt must be detected as being different from the normal behavior of the system.

[0029] For example, the IDS system implements fingerprinting techniques and methods to compare electrical or other unique operating characteristics of an ECU to detected behavior. However, fingerprinting methods are not effective in detecting attacks at higher levels of operation in automotive systems. As another example, methods using entropy that monitor parameters such as the frequency of ECU messages can detect injection or denial of service (DoS) attacks, but are not effective against attacks with unknown signatures. As yet another example, machine learning (ML) and deep learning (DL) based methods rely on training the system with a large number of examples of normal IVN network traffic; they flag deviations from normal patterns.The above methods can use raw IVN message data without considering the message content. They also implement models that analyze individual message streams from the ECU. However, the respective contexts of message streams issued simultaneously by other ECUs may not be available for analysis that can use inter-signal correlation effects to detect abnormal behavior.

[0030] The anomaly detection systems and methods of the present invention detect attacks at the IVN layer, such as frame injection, frame modification, frame deletion, and DoS attacks caused by one or more compromised ECUs used as fulcrums for such attacks. For example, an anomaly detection system implements a state space representation of the IVN network based on the ECU signals and one or more heuristic methods for distinguishing normal IVN traffic from abnormal traffic. While other methods using ML / DL techniques and Hidden Markov Models, operate on raw (i.e., encoded) IVN signals / messages from a single ECU and opaque system states, the anomaly detection system according to the present invention operates on the actual (i.e., decoded) content of IVN messages from multiple ECUs, based on correlations between the signals. With the state space model and heuristics, the anomaly detection system significantly reduces the occurrence of false positives while maintaining high accuracy (i.e., detection of true positives).

[0031] Accordingly, the anomaly detection system is configured to access the decoded IVN messages (e.g., using a vendor database (DBC file) for CAN messages) in real time. The anomaly detection system may comprise and / or implement, among other things, the state space model, a set of heuristic methods, and a classifier (e.g., implemented or executed by circuitry such as one or more processing devices or processors). The state space model may be based on a signal-based state space (e.g., represented in a graphical data structure) generated using a training set of IVN message sequences. With the state space model and the set of heuristic methods, state transitions induced by the ECU signals emitted during vehicle operation are noted.In other words, transitions between model states are scored using state transition scores. The state transition scores indicate whether the IVN message sequences are normal or abnormal. The state-space model is configured to use the metrics calculated by the heuristic methods to better distinguish between truly abnormal sequences and sequences that may not have been observed during model construction but are nevertheless "normal." As a result, the false positive rate is improved (i.e., the false positive rate is decreased) while maintaining high accuracy. Examples of heuristic methods include, but are not limited to, .

[0032] - (i) a distance measure that calculates a "distance" between two states in a state transition, and

[0033] - (ii) probability limits of a change in the signal value or the rate of change in the value of the signal associated with a state transition.

[0034] The classifier of the anomaly detection system classifies IVN message sequences (e.g., of fixed length) as normal or abnormal. The classifier may be trained using a set of scores (e.g., produced by the state space model and the various heuristic methods) associated to the message sequences (e.g., in the training set), as well as classification labels (e.g., ground truth labels) for the message sequences. Various ML or DL ​​methods can be used to build the classifier, such as a support vector machine (SVM), random forests, neural networks, etc.

[0035] Accordingly, the anomaly detection systems and methods of the present invention implement a representation of the state space of an ECU system, modeled as a graph, based on the observable signals transmitted by the ECUs over the IVN network. The use of correlated signals and the incorporation of memory of recent signals into the state space model provide better context for anomaly detection, and the use of various heuristics, along with the state space model, to produce scores for the signal sequences allows the classifier to better distinguish between normal and abnormal sequences.The model can be updated / improved by learning incrementally (e.g., via online learning) from new data, which is more efficient than training models that use ML / DL methods that need to be relearned when new data becomes available. The classification tasks of these systems and methods are more efficient than ML or DL ​​methods that require expensive hardware acceleration to meet real-time constraints. Furthermore, compared to various methods such as DL models and hidden Markov models, the graph-based model described here facilitates the representation of anomaly predictions for analysis.

[0036] [Fig.l] shows a block diagram of an exemplary computing device 100 and associated circuitry, configured to implement the anomaly detection systems and methods according to embodiments of the present invention. The computing device 100 includes a controller 105, e.g., a central processing unit (CPU) processor, chip, or other suitable computing or computing device, an operating system 115, memory 120, executable code 125, a storage server system 130, input devices 135, and output devices 140. The controller 105 (or one or more controllers or processors, possibly distributed across multiple units or devices) may be configured to perform the described methods, and / or to perform or operate as the various modules, units, etc.Multiple computing devices 100 may be included, and one or more computing devices 100 may act as components of a system according to the invention.

[0037] The operating system 115 may be or include any code segment (e.g., a segment similar to the executable code 125) designed and / or configured to perform tasks involving coordination, scheduling, arbitration, supervising, controlling, or managing the operation of the computing device 100, for example, scheduling the execution of software programs or tasks or activating software programs or other hardware modules or units to communicate. The operating system 115 may be a commercial operating system or an optional component; for example, in some embodiments, the system comprises a computing device that does not require or include the operating system 115. For example, a computing system is, or includes, a microcontroller, an application-specific circuit (ASIC), a field-programmable gate array (FPGA), a network controller (e.g., a CAN bus controller), an associated transceiver, a system-on-chip (SOC), and / or any combination thereof used without an operating system.

[0038] The memory 120 is or comprises, for example, a RAM, a ROM, a dynamic random access memory (DRAM), a synchronous random access memory (SD-RAM), a double data rate (DDR) memory chip, a flash memory, a volatile memory, a non-volatile memory, a cache memory, a buffer, a short-term memory unit, a long-term memory unit, or other suitable memory or storage units. The memory 120 is or comprises a set of memory units, possibly different. The memory 120 may be a non-transitory computer or processor readable medium or a non-transitory computer storage medium, for example a RAM.

[0039] The executable code 125 is any executable code, for example, an application, a program, a process, a task, or a script. The executable code 125 is executed by the controller 105, possibly under the control of the operating system 115. For example, the executable code 125 is an application that enhances security in a vehicle, and which, for example, detects or prevents cyberattacks on on-board networks. Although, for the sake of clarity, only one element of the executable code 125 is shown in [Fig. 1], a system according to some embodiments of the invention includes a set of executable code segments, similar to the executable code 125, which can be loaded into the memory 120 and cause the controller 105 to execute the described methods. Where appropriate, the terms "process" and "executable code" may have the same meaning and be used interchangeably in this description.For example, verification, validation, and / or authentication of a process may mean verification, validation, and / or authentication of executable code.

[0040] The storage system 130 is or includes, for example, flash memory, internal or microcontroller or chip-integrated memory, a hard disk drive, a CD-R disk drive, a Blue-ray disc (BD), a universal serial bus (USB) device, or any other suitable removable and / or fixed storage unit. The content may be stored in the storage system 130 and loaded from the storage system 130 into the memory 120 for processing by the controller 105. In some cases, some of the components of [Fig. 1] are removed. For example, the memory 120 may be non-volatile memory having the storage capacity of the storage system 130. Therefore, although shown as a separate component, the storage system 130 may be integrated or included in the memory 120.

[0041] Input devices 135 are or include any suitable input device, component, or system, for example, physical sensors such as accelerometers, tachometers, thermometers, microphones, analog-to-digital converters, etc. Output devices 140 include one or more displays or monitors (possibly detachable), motors, servomotors, speakers, and / or any other suitable output device. Any applicable input / output (I / O) devices may be connected to computing device 100 as indicated by blocks 135 and 140. For example, a wired or wireless network interface card (NIC), a universal serial bus (USB) device, a JTAG interface, or an external hard drive may be included in input devices 135 and / or output devices 140.Any number of input devices 135 and output devices 140 may be connected to cooperate with the computing device 100, (see blocks 135 and 140). For example, the input devices 135 and the output devices 140 are used by a technician or engineer to connect to the computing device 100, update software, etc. The input and / or output devices or components 135 and 140 are adapted to interface or communicate with control units or other units of a vehicle, for example, the input and / or output devices or components 135 and 140 may include ports allowing the computing device 100 to communicate with an engine control unit or module, a suspension control unit or module, a traction control unit or module.

[0042] Embodiments may include an article such as a computer- or processor-readable medium, or a storage medium not transferable by a computer or processor, such as, for example, memory, a disk drive, or a USB flash drive, encoding, including, or storing instructions (e.g., computer-executable instructions that, when executed by a processor or controller, implement the methods described herein), a storage medium such as memory 120, computer-executable instructions such as executable code 125, and a controller such as controller 105.

[0043] The storage medium includes, but is not limited to, any type of disk, including magneto-optical disks, semiconductor devices such as read-only memory (ROM), random access memory (RAM), such as dynamic memory (DRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, or any type of medium suitable for storing electronic instructions, including programmable storage devices.

[0044] Embodiments of the invention include, but are not limited to, components such as a set of central processing units (CPUs) or any general-purpose or specific processor or controller (e.g., controllers similar to controller 105), a set of input units, a set of output units, a set of memory units, and a set of storage units, etc. A system may also include other suitable hardware and / or software components. In some embodiments, the system comprises or is, for example, a personal computer, a desktop computer, a mobile computer, a laptop computer, a notebook computer, a terminal, a workstation, a server computer, a personal digital assistant (PDA), a tablet computer, a network device, or any other suitable computing device.

[0045] In some embodiments, the system includes or is, for example, a set of circuit components with a respective set of central processing units, e.g., a set of central processing units, a set of central processing units, integrated into an on-board or vehicle system or network, a set of chips, FPGAs or SOCs, microprocessors, transceivers, microcontrollers, a set of computers or network devices, any other suitable computing device, and / or any combination thereof. For example, a system includes one or more devices such as computing device 100.

[0046] [Fig. 2] shows an exemplary anomaly detection system 200 and a vehicle 204 implementing an IDS system 208 according to the present invention. Some portions of the IDS system 208 are implemented in the vehicle 204 while other portions of the IDS system 208 are implemented on one or more remote servers (which may be implemented in a cloud computing system or a cloud server, a vehicle security operations center (VSOC), etc.). The anomaly detection system 200 according to the present invention comprises and / or implements using circuitry such as a computing device, one or more processing devices, etc. as described herein.For example, the anomaly detection system 200 includes circuitry configured to store and / or implement one or more state space models (model 212) and further includes a model output classifier 216 configured to . classify the outputs of the model 212 as normal or abnormal, as described in more detail below. For example only, the model output classifier 216 may be implemented using a classification circuit, such as a computing device, a processing device configured to execute instructions to classify the outputs of the model 212.

[0047] In some exemplary implementations, a device such as a CAN bus recorder 220 communicates with vehicle systems through an on-board diagnostic port 224 (e.g., an OBD-II port) of the vehicle 204 and records and / or outputs CAN messages (e.g., IVN signals provided as traces 228), including CAN messages related to vehicle safety and control. For example, the device communicates with one or more ECUs 232 of the vehicle 204 via a CAN bus 236. One or more ECUs 232 correspond to a gateway ECU. The stored CAN messages are available for viewing and further processing and may be accessible for external services, such as model building and training performed using the anomaly detection system 200.For example, the traces 228 may be provided to one or more model builders 240 configured to build the models 224 based on the data contained in the traces 228, as described in more detail below. For example, the model builder 224 may be implemented using circuitry of a computing device such as that described herein.

[0048] The traces 228 are provided to a sequence generator 240. The sequence generator 240 (e.g., the sequence generator circuit, such as the circuit implemented by a computing device, a processing device, etc. and configured to generate / output CAN message sequences) provides CAN message sequences to the models 224. For example, the sequence generator 240 provides (e.g., without modification) a normal CAN message sequence constructed based on the traces 228 and generates a synthetic abnormal CAN message sequence. In other words, the normal CAN message sequence corresponds to the actual CAN message sequences observed on the CAN bus 236 and recorded by the CAN bus recorder 220. Conversely, the synthetic abnormal CAN message sequence is generated by the sequence generator 240.For example, the anomalous sequence of synthetic CAN messages includes sequences of messages that are not observed on the CAN bus 236 but that may occur during operation (e.g., as a result of an intrusion into the IVN network). The sequence generator 240 implements one or more algorithms that modify the normal sequences of CAN messages to simulate the effects of various known attacks on the IVN networks. In other words, the anomalous sequence of synthetic CAN messages represents a possible . normal CAN bus behavior that the fault detection system 200, IDS system components 208, etc., nevertheless did not see or observe during normal operation.

[0049] In this way, the model 212 is built and trained to differentiate:

[0050] - (i) the "normal" sequences of CAN messages, new or not yet seen, which do not correspond to an attack or intrusion, and

[0051] - (ii) CAN message sequences that correspond to an attack or a intrusion. For example, model 212 is built based on unmodified traces (e.g., traces 228) provided to model builder 240 and then trained based on the normal CAN message sequences and the anomalous synthetic CAN message sequences received from sequence generator 244. The normal CAN message sequences serve as training sequences used to identify the states (nodes) and transitions (edges) between states that comprise the graph-based state-space model described below. Once the model is built from the training sequences, model output classifier 216 is trained to distinguish between normal sequences and anomalous sequences based on the model output.Once the model 212 and the model output classifier 216 are trained, a set of test sequences are applied by the model 212 and the model output classifier 216 to determine the precision and recall measures of the anomaly detection system 200.

[0052] Typically, the IVN signals / messages communicated over the CAN bus 236 are encoded. Accordingly, the anomaly detection system 200 of the present invention communicates with or, in some examples, includes a decoder or decoding circuit 248 configured to decode the IVN signals / messages (e.g., using a CAN database (DBC) file). The decoder 248 provides, as traces 228, the actual (i.e., decoded) contents of the IVN message signals from the multiple computers 232 to the relevant components of the anomaly detection system 200. For example, the sequence generator 244 generates shorter sequences of decoded IVN signals (e.g., sequences having a predetermined or adjustable length) using longer sequences of decoded IVN signals received from the decoder 248.The decoded IVN messages indicate the actual signal values ​​(e.g., command or control signals, sensed or measured values, etc.) communicated by the computers 232 and which may include strings or sequences of signal values, each with a corresponding timestamp. Accordingly, correlations between the actual signal values ​​may be observed by the model builder 240 and other components of the anomaly detection system 200.

[0053] Typically, the computers 232 are connected to various sensors and actuators and broadcast signals over the network (e.g., the CAN bus 236). These signals may have continuous domains in certain intervals (e.g., engine speed, throttle position, vehicle speed, etc.) or discrete values ​​(e.g., transmission gear position (neutral, reverse, 1st, 2nd, 3rd, 4th, 5th, etc.), brake pedal state (depressed, not depressed, etc.). The values ​​of each type of signal over the network correspond to a time series. The time series are examined and analyzed in isolation or together to detect anomalies (i.e., deviations from normal patterns). The state-space-based model of the present invention is configured to evaluate the characteristics of the IVN signals to better predict anomalies, as described in more detail below.

[0054] For example, certain signals such as vehicle speed and throttle signals (e.g., measured vehicle speed and commanded or sensed throttle position) may be directly or indirectly correlated since throttle position controls speed. However, encoded signals corresponding to vehicle speed and throttle, as communicated over CAN bus 236, may not be correlated. The decoded IVN messages according to the present invention facilitate the detection of abnormal behavior by the anomaly detection system 200. For example, if the decoded IVN messages corresponding to two correlated signals are not correlated as expected (e.g., vehicle speed increases while throttle position decreases), the anomaly detection system 200 may determine that spurious signal values ​​are being injected into the system.More specifically, model 212 is built partly on the basis of expected correlations between IVN signals.

[0055] As another example, the recent history of a signal and correlations with other signal values ​​may provide the anomaly detection system 200 with additional context for processing message sequences. For example, if the most recent values ​​of a vehicle speed signal increased concurrently with corresponding values ​​of an accelerator pedal position signal, future values ​​of the vehicle speed signal may be expected to continue to increase as the vehicle accelerates. A deviation from this pattern (e.g., vehicle speed decreases as acceleration increases) may indicate an anomaly. Accordingly, the anomaly detection system 200 of the present invention incorporates the "memory" of recent signal values ​​into the state-space-based model states.Each state encodes not only the most recent observed values ​​of the signals, but also values ​​from the recent history of the signals, to facilitate the identification of signal patterns.

[0056] [Fig.3A] shows an example of a signal, graph, and state-space based model 300 implemented by the anomaly detection system 200 according to the present invention. The model 300 is represented as a graph G = (V, E) in which V, is a set of nodes in the graph representing the states of the system, and E, is a set of directed edges, representing the transitions between the states. Each state S, corresponding to a node of the graph, encodes the signal values ​​of a set of correlated signals (e.g., as signal snapshots). A signal snapshot is a tuple of signal values. Each signal snapshot indicates the signal values ​​that characterize the state of the system at a given time. Each state may have more than one signal snapshot. For example, a set of signal snapshots in a state codifies the most recent signal values ​​(e.g., in chronological order).

[0057] By way of example only, the illustrated model 300 corresponds to a system having only two signals: Accelerator Position and Vehicle Speed. Since these signals have continuous domains, the values ​​can be separated into domains according to associated criteria. In this example, the domains are subdivided into intervals of a length of five units. Each state encodes two snapshots of the signal. For example, state S( has two signal snapshots:

[0058] - ((Accelerator Position=(5,10), Vehicle Speed=(20,25)), and

[0059] - (Accelerator Position=(5,10), Vehicle Speed=(20,25)).

[0060] Each signal snapshot incorporates the values ​​of both signals at a given time. As shown in [Fig.3A], the most recent signal snapshot for each of the states is listed at the top (e.g., as shown at 302) while the previous snapshot is listed at the bottom (e.g., as shown at 304). Together, the two signal snapshots represent a memory capacity of two for each state of the model 300.

[0061] Edges (i.e., transitions) between states represent state transitions induced by observations of a signal on the IVN network. For example, an edge 306 between states Sf and Sj represents a transition induced by observing on the network a signal value of (25,30) for vehicle speed. This transition moves the system to state Sj in which the most recent signal snapshot (Accelerator Position=(5,10), Vehicle Speed=(25,30)) reflects a change in vehicle speed relative to the previous signal snapshot. The most recent signal snapshot in state S; becomes the previous (older) signal snapshot in state S j.

[0062] In addition, a transition probability is assigned to each edge / transition. For example, a transition probability Pq is assigned to the edge 306 connecting the states ■S,- and Sj. The transition probabilities can be estimated empirically from the traces 228. For example only, the transition probability associated with the transition from to Sj is Pÿ = 0.8. In other words, over the set of outgoing transitions from Si 80% of the transitions correspond to transitions from S; to Sj (for example), and the remaining 20% ​​correspond to transitions from to Sk. Thus, each transition probability assigned to a state corresponds to a transition probability between the two states. In this way, the model 300 is constructed (for example, by the model builder 240, in response to the decoded IVN signals received from the decoder 248) based on the actual observed (i.e., "normal") behavior on the CAN bus 236.

[0063] For example, as CAN message sequences are received, the corresponding states (e.g., each corresponding to two correlated signal snapshots) are added to the model 300. Similarly, as the signals transition from one state to another, edges / transitions between the states are added to the model 300. All transitions between the states are counted over time to calculate the respective probabilities of the transitions between any two states. Thus, as CAN message sequences are received, the model builder 240,

[0064] - (i) adds states to the model 300,

[0065] - (ii) adds transitions between states to the model 300, and

[0066] - (iii) counts and updates the transitions between the states of the model 300 in order to calculate the transition probabilities for each transition.

[0067] [Fig. 3B] shows an example classification process 310 (e.g., implemented by the model output classifier 216) configured to classify the IVN signal sequences based on an output 312 of the model 212 / 300. For example, the output 312 of the model 300 includes values ​​based on the assigned transition probabilities for transitions between various states for one or more signals as described above. The model output classifier 216 analyzes the message sequences to predict whether the sequences are normal or abnormal (i.e., have anomalies indicating attacks or intrusion attempts). For example, each signal transmitted to the model 300 (e.g., by the sequence generator 244) corresponds to a state transition.Some of the state transitions may already exist in model 300 (i.e., have already been observed by model builder 240) while others may not be included in model 300 because, for example, .

[0068] - (i) the training data used to build the model 300 does not did not understand the missing state transition, or

[0069] - (ii) the state transition is illegal and potentially constitutes an anomaly.

[0070] As shown in Figure 3B, the output 312 of the model 300 associates a value v with each state transition associated with a sequence of input signals. The output 312 represents n state transitions associated with n signals observed on the IVN network. The transition from each state $ to a corresponding next or successor state Sj is associated with a value Vÿ. Therefore, the output 312 of the model 300 for the sequence is a vector of the values ​​associated with the state transitions. The model output classifier 216 receives the vector of values ​​for the state transitions and outputs the value 0 in response to determining that the values ​​correspond to a normal sequence of messages and outputs the value 1 in response to determining that the values ​​correspond to an abnormal sequence of messages.Different types of classification techniques can be implemented by the model output classifier, including, but not limited to, a threshold classifier, an ML classifier (e.g., implementing a support vector machine (SVM) algorithm, a random forest algorithm, etc.).

[0071] Various methods may be used to assign values ​​to (i.e., score) state transitions, including various heuristic methods. For example, each value associated with a state transition corresponds to the transition probability associated with the corresponding edge in the model 300. As another example, each value associated with a state transition is a measure of the distance between the next state and the previous state in a transition. For example, if the previous state and the next state form an edge in the model 300 (i.e., are directly connected by an edge / transition), the distance is 1. Conversely, if the previous state and the next state are not directly connected, the distance is greater than 1 (e.g., estimated using a heuristic or other method, described in more detail below).Another example: each value associated with a state transition is a measure of the probability of observing a specific signal change associated with the state transition (e.g., a probability that the specific transition could have occurred).

[0072] [Fig.3C] illustrates an example of anomaly detection performed using distance measurements between the previous (source) and next (target) states in the model 300 for a normal sequence of messages (in a first table 320) and an abnormal sequence of messages (in a second table 324). In this example, two signals are considered: Throttle Position (e.g., received from a first ECU (ECU1) of the ECUs 232) and Vehicle Speed ​​(e.g., received from a second ECU (ECU2) of the ECUs 232). Each state of a corresponding model (such as a model similar to the model 300) encodes the two most recent signal snapshots containing signal values ​​for both signals. In the tables 320, 324, the signal column contains the signal value the most recent of each signal, and the source state column and target state column contain, respectively, the source (previous) state and target (next) state associated with a transition induced by the signal value in the signal column. The distance measure column contains the distance measure or value associated with the state transition (e.g., a heuristic distance measure). Accordingly, each row illustrates a transition (e.g., labeled 1 to 6) from a source state to a target state.

[0073] The first table 320 represents the signal values ​​and transitions of a normal sequence of messages. For example, transitions 1, 2, 3, and 6 correspond to transitions included in the model (i.e., corresponding directly to known and previously observed transitions between states of the model) and therefore have a distance measure of 0 (i.e., "normal"). Conversely, transitions 4 and 5 are associated with non-zero values. For example, transitions 4 and 5 were not previously observed and were added to the model (e.g., in response to training sequences), but they are not necessarily indicative of an anomaly. On the contrary, the relatively low distance values ​​(2 and 1, respectively, for transitions 2 and 1) may indicate that the transitions correspond to previously unobserved, but nonetheless normal, behavior.For example, the model output for the message sequence corresponding to transitions 1-6 of the first table 320 is a vector [0, 0, 0, 2, 1, 0]. For example, a classification method implemented by the model output classifier 216 (e.g., a threshold classifier) ​​may be configured to compare an L2 or Euclidean norm or other calculated value (e.g., an average distance measure of the transitions) of the vector to a predetermined threshold (e.g., a threshold selected when training / building the model output classifier 216). If the calculated value of the vector is less than (or, in some examples, less than or equal to) the predetermined threshold, the message sequence may be classified as normal (e.g., assigned the value 0).

[0074] Conversely, the second table 324 is an example of a representation of an abnormal message sequence. In this example, a third ECU (ECU3) of the ECUs 232 represents an ECU that has been compromised and used to falsify vehicle speed signal values ​​(i.e., transmit vehicle speed values ​​as if they originated from ECU2). For example, the transitions illustrated in lines 3 and 4 include spurious values ​​injected by ECU3. Accordingly, in response to receiving the message sequence shown in the second table 324, the model produces a vector [0, 0, 4, 5, 5, 6] with larger distance measurements corresponding to transitions 3, 4, 5, and 6. In other words, the distance measurements associated with transitions of states resulting from the spurious signals injected by the ECU3 unit have higher values. Therefore, the model output classifier 216 (e.g., the same threshold classifier described above) predicts that the message sequence is abnormal. For example, the calculated value of the vector (e.g., the L2 or Euclidean norm) is greater than the predetermined threshold.

[0075] In this manner, the anomaly detection system 200 implements a model configured to produce vectors that can be identified as normal or abnormal (e.g., using the model output classifier 216).

[0076] The anomaly detection system 200 implements various heuristic techniques or methods to distinguish between normal message sequences and abnormal sequences, such as a distance heuristic method (described above) or a signal change probability heuristic method. The heuristic methods are configured to associate higher scores (e.g., values ​​of vectors produced by the model) with state transitions that are not incorporated in the model. In this way, the model output classifier 216 can better distinguish between normal and abnormal message sequences, thereby reducing the incidence of false positives and improving classification accuracy.

[0077] Figure 3D illustrates an example of a distance heuristic method applied to a portion of a model 340. In this example, A, B, and C correspond to the signals received in each signal snapshot. The most recent state transition is shown between states S and S'. The edge 344 between states $ and 5” is shown by a dotted line to indicate that the edge 344 does not correspond to an edge added during training of the model 340 (i.e., from training messages / training message sequences). Accordingly, the anomaly detection system 300 analyzes the edge 344 to determine whether the corresponding message sequence is normal (but not observed during training) or anomalous.

[0078] To assign a value to the state transition corresponding to edge 344, a first breadth search (BFS) of the nodes (i.e., states) of the model 340 is performed (e.g., from state S and up to a predetermined or fixed distance). During the BFS traversal, the state of the model 340 that is most similar to state S' is identified among the nodes (e.g., based on a similarity measure defined as below). For example, all candidate states to be the state most similar to state S' must correspond to a pivot signal value, such as the value of the signal that caused the transition from state S to S' (e.g., as shown, a value of A=(10,15) for signal A). When the most similar state is identified (e.g., S”), a first distance between state S and state S" is calculated in the model 340 and a second distance between S and S" is subtracted from the first distance.The difference between the first and second distance is an "extra" distance. to travel to go from state S to the state S" closest to the last observed state S'. For example, it is assumed that a larger additional distance is more likely to correspond to an abnormal sequence of messages. In other words, an additional distance value of 0 corresponds to transitions between states already observed and contained in the model 340, while distances of 1 or more correspond to transitions to a next state that has not been previously observed. The model output classifier 216 is trained with the output vectors of the model 340 for several normal and abnormal message sequences to learn to distinguish between normal and abnormal sequences of messages based on the distance metric values.

[0079] As shown in Figure 3D, among all the states encountered during the BFS crossing from state S, state S” is determined to be the state most similar to S'. Only one other state (Sy), is consistent with the pivot signal A. However, according to a similarity measure described in more detail below, state S" has a higher similarity score sim=2 (since S” is also consistent with signal C) compared to state S.v, which has a similarity score sim=1. Therefore, the distance measure (d) associated with the transition from S to S' is given by a distance from S to S” minus a distance from $ to S' (i.e., d(S,S') = dist(G, S, S”) - dist(G, S, S') = 2-1 = 1). Conversely, if instead of state the next state after $ is S„the measure of distance associated with the transition would be d($, = dist(G, S, Su) - dist(G, S, = 0.

[0080] The similarity (e.g., a similarity score) between two states may be determined using various methods. For two states Sj and Sj that encode signal snapshots (s«i, s«2...) and (SJ\ SJ2...\ respectively, the variables sm» in the signal snapshots represent an nth signal value in the same signal snapshot. For example, a similarity score between Sj and Sj may be calculated using an L1 or Manhattan norm of the vector (5n - sJl, si2 - sj2,...). For example, smaller L1 norms may correspond to states whose signal values ​​(e.g., the scaled signal values ​​as captured in the respective signal snapshots) are similar to each other.

[0081] In one example, each iteration of a BFS traversal returns a next node of the model 340 from a node corresponding to the previous state. The BFS traversal may be limited by a fixed search distance (e.g., 3 nodes ahead, 4 nodes ahead, etc.). If the search distance is exceeded, the BFS traversal does not return a valid state (e.g., it returns a NIL value). The pivot signal used for the BFS traversal is the most recent signal seen in the sequence of received messages (i.e., the value of the signal that caused the state transition). At each iteration, the state corresponding to the next node in the BFS traversal is extracted and a determination is made. a similarity score of this state to the next state. If the score exceeds the highest previous similarity score, the corresponding distance measure is updated. Once all nodes within the allowed search distance have been processed and evaluated according to the respective similarity scores and distance measures, the distance measure value corresponding to the state with the highest similarity score is selected. If no similar nodes are found within the search distance of the BFS traversal, the transition is assigned a large constant value (e.g., a distance value of 10 or higher).With the distance heuristic algorithm described above, each state transition associated with a signal value in a message sequence is associated with a score and the resulting output vector for the entire signal sequence is then input into the model output classifier 216.

[0082] According to another example, the anomaly detection system applies a signal change probability heuristic method, configured to evaluate the change in the signal value causing the transition. Using the training samples, the mean (p) and the variance (<72) of the distribution of the difference between two successive values ​​of a signal are calculated. For a random variable X that represents the change in the value of a signal in a sequence, the value k corresponds to the difference between X and the mean p. From the Chebyshev inequality, a probabilistic inequality is defined as follows p [ | y _ | > £} < a21 k2-The value ct-TIc represents the upper limit to the probability that the actual variation in the signal value deviates from the mean (signal variation) by more than k.Each state transition corresponding to a signal change in a signal sequence can then be associated with this probability measure. A low value of the measure indicates that the magnitude of the signal change is improbable and therefore suggests an anomaly. Conversely, a high value indicates that the signal change is very likely and therefore normal.

[0083] In this manner, the anomaly detection system 200 may be configured to implement one or more heuristic methods to evaluate the state transitions corresponding to the received message sequences.

[0084] [Fig. 4] illustrates the steps of an exemplary method 400 for constructing and implementing a model of an anomaly detection system according to the present invention. For example, the method 400 is implemented by one or more computing devices or systems and / or associated circuitry, such as the anomaly detection system 200, a computing device (e.g., the computing device 100) configured to implement the anomaly detection system 200, etc.

[0085] In step 404, the method 400 (using a CAN bus recorder, or decoder) receives decoded CAN bus messages, associated with messages exchanged between the computers of a vehicle, for example, training messages. The CAN bus messages are received from an IVN network, i.e., a vehicle diagnostic port, a testing device, etc. In step 408, a signal, graph, and state space based model, such as model 212, 300, 340, etc., is composed of various observed signal states and transitions / paths, observed between the signal states. Each node or state comprises a sequence of one or more signal values ​​for one or more signals (e.g., n different values ​​of m different signals, where n and m are each integers with a value equal to or greater than 1.

[0086] In step 412, the method 400 with the sequence generator 244 trains the model by providing it with normal message sequences and anomalous synthesis sequences. The normal message sequences correspond to the message sequences contained in the training messages. Conversely, the anomalous synthesis message sequences include message sequences that are not contained in the training messages but that may still occur during the operation described above. In other words, the model is trained to recognize as "normal" messages that were not included in the training messages but that are not an intrusion or an attack. The model is tested and / or updated by continuing to provide anomalous synthesis message sequences to evaluate the accuracy of the model's results.

[0087] At step 416, the trained model is implemented in a system, such as a vehicle's IVN network. At step 420, the trained model receives sequences of decoded messages; for example, a sequence of IVN messages. At step 424, the model produces a vector corresponding to a value assigned to a state transition between the states of one or more signals. For example, the model calculates the vector using a distance heuristic method. At 428, method 400 classifies the output of the model as normal, for example, outputting a value "1" or abnormal, for example, outputting a value "0" based on the vector.

[0088] At step 432, the method 400, for example, the anomaly detection system 200, the IDS 208, takes one or more corrective actions in response to determining an abnormal sequence of messages. The corrective actions include, but are not limited to: disabling one or more systems or functions of the vehicle, alerting the driver, instructing the driver to pull over or stop the vehicle, operating in a limited mode, etc. Conversely, if a normal sequence of messages is determined, the method 400 proceeds to step 420 to continue monitoring for decoded messages.

[0089] Figures 5-11 depict example systems and devices implementing anomaly detection systems and methods according to the present invention. [Fig. 5] depicts a schematic diagram of an interaction between a computer-controlled machine 500 and the control system 502. The computer-controlled machine 500 includes an actuator 504 and a sensor 506. The actuator 504 includes one or more actuators and the sensor 506 includes one or more sensors. The sensor 506 is configured to detect the state of the computer-controlled machine 500. The sensor 506 is configured to encode the detected state into sensor signals 508 and to transmit the sensor signals 508 to the control system 502. Non-limiting examples of sensors 506 are: video sensors, radar, lidar, ultrasonic sensors, and motion sensors.In some cases, the sensor 506 is an optical sensor configured to detect optical images of an environment near the computer-controlled machine 500.

[0090] The control system 502 is configured to receive sensor signals 508 from the computer-controlled machine 500. The control system 502 is configured to calculate the actuator commands 510 based on the sensor signals and to transmit the commands 510 to the actuator 504 of the computer-controlled machine 500.

[0091] According to [Fig. 5], the control system 502 comprises a receiving unit 512. The receiving unit 512 is configured to receive the signals 508 from the sensor 506 and to transform these sensor signals 508 into input signals (x). In another embodiment, the sensor signals 508 are received directly as input signals x without the receiving unit 512. Each input signal (x) is a part of each sensor signal 508. The receiving unit 512 is configured to process each sensor signal 508 to produce the respective input signal (x). The input signal x includes data corresponding to an image recorded by the sensor 506.

[0092] The control system 502 includes a classifier 514. The classifier 514 is configured to classify the input signals (x) into one or more labels using a machine learning (ML) algorithm, such as a neural network. The classifier 514 is configured and parameterized by parameters, such as those described above, for example, parameter 0. The 0 parameters are stored and provided by the non-volatile memory 516. The classifier 514 is configured to determine the output signals (y) from the input signals (x). Each output signal (y) includes information that assigns one or more labels to each input signal (x). The classifier 514 transmits the output signals (y) to the conversion unit 518. The conversion unit 518 is configured to transform the output signals (y) into control commands for the actuators 510.The control system 502 transmits the control commands 510 to the actuator 504, configured to activate. the computer-controlled machine 500 in response to the actuator control commands 510. In some embodiments, the actuator 504 is configured to activate the computer-controlled machine 500 based directly on the output signals (y).

[0093] Upon receipt of the actuator 510 control commands by the actuator 504, the latter performs an action corresponding to the actuator 510 control command. The actuator 504 has control logic to transform the actuator 510 control commands into a second actuator control command, used to control the actuator 504. In one or more embodiments, the actuator 510 control commands are used to control a display instead of or in addition to an actuator.

[0094] In some embodiments, the control system 502 includes the sensor 506 instead of or in addition to the computer-controlled machine 500 including the sensor 506. The control system 502 may also have the actuator 504 instead of or in addition to the computer-controlled machine 500 with the actuator 504.

[0095] As shown in [Fig. 5], the control system 502 also includes a processor 520 and a memory 522. The processor 520 may be one or more processors. The memory 522 may be one or more memory devices. The classifier 514, for example, ML algorithms; of one or more embodiments may be implemented by the control system 502, including the non-volatile memory 516, the processor 520, and the memory 522.

[0096] Non-volatile memory 516 includes one or more non-volatile data storage devices such as a hard disk drive, optical drive, tape drive, non-volatile semiconductor device, cloud storage, or any other device capable of persistently storing information. Processor 520 includes one or more devices selected from high-performance computing (HPC) systems, including high-performance cores, microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, programmable gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other device that processes analog or digital signals with computer-executable instructions, residing in memory 522.The memory 522 comprises a single memory device or a number of memory devices, including, but not limited to, random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory. (DRAM), flash memory, cache memory, or any other device capable of storing information.

[0097] The processor 520 is configured to read the memory 522 and execute computer-executable instructions residing in the non-volatile memory 516 and incorporating one or more anomaly detection methodologies of one or more embodiments. The non-volatile memory 516 includes one or more operating systems and applications. The non-volatile memory 516 may store compiled and / or interpreted computer programs created with various programming languages ​​and / or technologies, including, but not limited to, alone or in combination, the languages: Java, C, C++, C#, Objective C, Fortran, Pascal, Java Script, Python, Perl, and PL / SQL.

[0098] When executed by the processor 520, the computer-executable instructions in the non-volatile memory 516 implement the control system 502 with one or more anomaly detection methodologies as described above. The non-volatile memory 516 may also include data supporting the functions, features, and processes of one or more described embodiments.

[0099] Program code incorporating the described algorithms and / or methodologies may be distributed individually or collectively as a program product in a variety of different forms. The program code is distributed using a computer-readable storage medium containing computer-readable program instructions and enabling a processor to execute certain aspects of one or more embodiments. Computer-readable storage media, which are inherently non-transitory, are tangible, volatile and non-volatile, removable and non-removable media implemented with any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data.Computer-readable storage media also include: RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, portable compact disc read-only memory (CD-ROM) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage device, or any other medium that can be used to store desired information and can be read by a computer. Computer-readable program instructions are downloaded to a computer, other type of programmable data processing device, or other device from a computer-readable storage medium. computer or to an external computer or external storage device via a network.

[0100] Computer-readable program instructions stored on a computer-readable medium are used to control a computer, other types of programmable data processing apparatus, or other devices, to operate in a particular manner, such that the instructions stored on the computer-readable medium produce an article of manufacture, including instructions that implement the functions, acts, and / or operations specified in the flowcharts or diagrams. In certain other embodiments, the functions, acts, and / or operations specified in the flowcharts and diagrams may be reordered, processed serially, and / or processed concurrently, in accordance with one or more embodiments. Further, the flowcharts and / or diagrams include more or fewer nodes or blocks in accordance with one or more embodiments.

[0101] The processes, methods or algorithms are implemented in whole or in part using suitable hardware components, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers or other hardware components or devices, or a combination of hardware, software and firmware components.

[0102] [Fig. 6] shows a schematic diagram of the control system 502 for controlling the vehicle 600, which is an at least partially autonomous vehicle or an at least partially autonomous robot. The vehicle 600 includes the actuator 504 and the sensor 506. The sensor 506 represents one or more video sensors, cameras, radar sensors, ultrasonic sensors, lidar sensors and / or position sensors, for example, GPS sensors. One or more specific sensors are integrated with the vehicle 600. Alternatively or in addition to one or more specific sensors identified above, the sensor 506 includes a software module for, at runtime, determining the state of the actuator 504. A non-limiting example of a software module includes a weather information software module for determining the present or future state of the weather in the vicinity of the vehicle 600 or another location.

[0103] The classifier 514 of the control system 502 of the vehicle 600 is configured to detect objects in the vicinity of the vehicle 600 based on the input signals (x). In such an embodiment, the output signal (y) includes information characterizing the proximity of the objects to the vehicle 600. The control command of the actuator 510 is determined based on this information. The control command of the actuator 510 is used to avoid collisions with the detected objects.

[0104] In some embodiments, the vehicle 600 is an at least partially autonomous vehicle; the actuator 504 is incorporated into a brake, propulsion system, engine, powertrain, or steering of the vehicle 600. Control commands for the actuator 510 are determined to control the actuator 504 so that the vehicle 600 avoids collisions with the detected objects. The detected objects are also classified based on what the classifier 514 believes they are most likely to be; pedestrians or trees. Control commands for the actuators 510 are determined based on the classification. In an adversary attack scenario, the system described above is further trained to better detect objects or identify a change in lighting conditions or an angle for a sensor or camera on the vehicle 600.

[0105] In some embodiments, the vehicle 600 is an at least partially autonomous robot; the vehicle 600 may be a mobile robot, configured to perform one or more functions, such as flying, swimming, diving, and walking. The mobile robot may be an at least partially autonomous lawn mower or an at least partially autonomous cleaning robot. In such embodiments, the control command of the actuator 510 is determined to control a propulsion unit, a steering unit, and / or a braking unit of the mobile robot so that the mobile robot avoids collisions with identified objects.

[0106] In some cases, the vehicle 600 is an at least partially autonomous robot in the form of a gardening robot. In this embodiment, the vehicle 600 uses an optical sensor as sensor 506 to determine the condition of plants in an environment near the vehicle 600. The actuator 504 may be a nozzle configured to spray chemicals. Based on an identified species and / or an identified condition of the plants, the actuator control command 510 commands the actuator 504 to spray the plants with an appropriate amount of appropriate chemicals.

[0107] The vehicle 600 may be an at least partially autonomous robot in the form of a household appliance. Non-limiting examples of household appliances are a washing machine, a stove, an oven, a microwave oven or a dishwasher. In such a vehicle 600, the sensor 506 is an optical sensor for detecting the state of an object to be treated by the household appliance. For example, in the case where the household appliance is a washing machine, the sensor 506 detects the state of the laundry inside the washing machine. The control order of the actuator 510 is determined on the basis of the detected state of the laundry.

[0108] [Fig.7] shows a schematic diagram of the control system 502 configured to control the system 700, for example, a manufacturing machine, such such as a punch, a milling cutter, or a gun drill, of the manufacturing system 702, such as part of a production line. The control system 502 controls the actuator 504, which activates the system 700, for example, the manufacturing machine.

[0109] The sensor 506 of the system 700, e.g., the manufacturing machine, may be an optical sensor configured to capture one or more properties of the manufactured product 704. The classifier 514 determines the state of the manufactured product 704 from one or more of the captured properties. The actuator 504 is configured to control the system 700, e.g., the manufacturing machine, based on the determined state of the manufactured product 704 for a subsequent manufacturing step of the manufactured product 704. The actuator 504 may be configured to control functions of the system 700, e.g., the manufacturing machine, by the manufactured product 706 of the system 700, e.g., the manufacturing machine, based on the determined state of the manufactured product 704.

[0110] [Fig. 8] shows the diagram of the control system 502 configured to control the power tool 800, such as a drill or an electric screwdriver, which has an at least partially autonomous mode. The control system 502 is configured to control the actuator 504, which controls the power tool 800.

[0111] The sensor 506 of the tool 800 is an optical sensor for capturing one or more properties of the work surface 802 and / or the fastener 804 engaged in the work surface 802. The classifier 514 is configured to determine the state of the work surface 802 and / or the fastener 804 relative to the work surface 802 from one or more of the captured properties. The state is the flushness of the fastener 804 with the work surface 802. The state may also be the hardness of the work surface 802. The actuator 504 is configured to control the power tool 800 to adjust the driving function of the power tool 800 based on the determined state of the fastener 804 relative to the work surface 802 or one or more captured properties of the work surface 802. For example, the actuator 504 stops driving if the state of the fastener 804 is flush with the work surface 802.Another non-limiting example: the actuator 504 applies a more or less significant torque depending on the hardness of the working surface 802.

[0112] [Fig.9] shows a diagram of the control system 502 for controlling an automated personal assistant 900. The control system 502 may be configured to control the actuator 504, which is configured to control the automated personal assistant 900. The automated personal assistant 900 is configured to control a home appliance, such as a washing machine, stove, oven, microwave, or dishwasher.

[0113] The sensor 506 is an optical sensor and / or an audio sensor. The optical sensor is configured to receive video images of the gestures 904 of the user 902. The audio sensor may be configured to receive a voice command from the user 902.

[0114] The control system 502 of the automated personal assistant 900 determines the control commands of the actuator 510 configured for the control system 502. The control system 502 may be configured to determine the control commands of the actuator 510 based on the signals 508 from the sensor 506. The automated personal assistant 900 transmits the sensor signals 508 to the control system 502. The classifier 514 of the control system 502 executes a gesture recognition algorithm to identify the gesture 904 performed by the user 902, determine the control commands of the actuator 510, and transmit the control commands 510 to the actuator 504. The classifier 514 is configured to retrieve information from a non-volatile memory in response to the gesture 904 and to render the retrieved information in a form suitable for receipt by the user 902.

[0115] [Fig. 10] shows a diagram of the control system 502 configured to control the surveillance system 1000. The surveillance system 1000 may be configured to physically control access through the door 1002. The sensor 506 may be configured to detect a relevant scene to decide whether access is allowed. The sensor 506 is an optical sensor for generating and transmitting image and / or video data. This data is used by the control system 502 to detect a person's face.

[0116] The classifier 514 of the control system 502 of the surveillance system 1000 is configured to interpret the image and / or video data by matching the identities of known persons stored in the non-volatile memory 516, thereby determining the identity of a person. The classifier 514 may be configured to generate an actuator command 510 in response to the interpretation of the image and / or video data. The control system 502 is configured to transmit the actuator control command 510 to the actuator 504. In this embodiment, the actuator 504 may be configured to lock or unlock the door 1002 in response to the actuator control command 510. In some embodiments, logical, non-physical access control is also possible.

[0117] The surveillance system 1000 may also be a surveillance system with an optical sensor 506 for detecting a scene under surveillance and the control system 502 is configured to control the screen 1004. The classifier 514 determines the classification of a scene, for example whether the scene detected by the sensor 506 is suspicious. The control system 502 is configured to transmit a actuator command 510 to the screen 1004 in response to the classification. The screen 1004 may be configured to adjust the displayed content in response to the actuation command 510. For example, the screen 1004 may highlight an object deemed suspicious by the classifier 514. Using one embodiment of the described system, the monitoring system predicts the appearance of objects at certain times.

[0118] [Fig. 11] shows the diagram of the control system 502 configured to control the imaging system 1100, for example, an MRI device, an X-ray imaging device or an ultrasound device. The sensor 506 may, for example, be an imaging sensor. The classifier 514 may be configured to determine a classification of all or part of the captured image. The classifier 514 is configured to determine or select a control command of the actuator 510 in response to the classification obtained by the trained neural network. For example, the classifier 514 interprets a region of a detected image as being potentially abnormal. In this case, the control command of the actuator 510 is determined or selected so that the screen 1102 displays the imagery and highlights the potentially abnormal region.

[0119] Although exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms covered by the claims. The terms used in the specification are terms of description rather than limitation, and various modifications may be made without departing from the spirit and scope of the invention. As previously described, features of the various embodiments may be combined to form other embodiments of the invention which may not be explicitly described or illustrated.Although various embodiments may have been described as providing advantages or being preferable to other embodiments or prior implementations with respect to one or more desired characteristics, those of ordinary skill in the art recognize that one or more characteristics may be compromised to achieve the desired overall attributes of the system, which depend on the specific application and implementation. Such attributes may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, ease of maintenance, weight, manufacturability, ease of assembly, etc.Thus, to the extent that certain embodiments are described as being less desirable than other embodiments or prior art implementations with respect to one or more features, such embodiments do not depart from the scope of the invention and may be desirable for particular applications.

Claims

Claims

1. A method of operating an anomaly detection system, comprising steps of: - receiving training message sequences corresponding to messages transmitted in an in-vehicle network (IVN), - building a model based on the training message sequences, * the model comprising a set of states corresponding to signal values, observed in the training message sequences and to state transitions between the respective states of the set of states, - training the model by providing it with, * (i) first message sequences corresponding to the training message sequences, and * (ii) second message sequences not contained in the training message sequences, and - executing with the anomaly detection system, the model to identify abnormal message sequences, transmitted in the IVN network, by * (i) receiving an IVN message sequence, * (ii) transmitting,from the model, a value based on the transitions between the states of the signals contained in the IVN message sequence, and * (iii) issuing, based on this value, an indication that the IVN message sequence comprises an abnormal sequence of messages.,

2. The method of claim 1, further comprising decoding the training message sequences and building the model based on the decoded training message sequences.

3. The method of claim 1, wherein each of the states of the model corresponds to the values ​​of at least two correlated signals contained in the training message sequences.

4. A method according to claim 3, wherein each state of the state set includes at least two values ​​for each state of at least two correlated signals.

5. The method of claim 4, wherein the model identifies the probabilities of each of the state transitions between the respective states of the set of states.

6. The method of claim 1, wherein outputting the value from the model comprises: (i) calculating the distance between a first state of the set of states corresponding to a first message of the IVN message sequence and a second state corresponding to a second message of the IVN message sequence and calculating the value with this distance, and (ii) calculating the value by a probability-based heuristic method.

7. The method of claim 6, wherein the distance corresponds to a number of state transitions in the model required to move from the first state to the second state.

8. The method of claim 7, wherein the second state does not correspond to any message contained in the training message sequences.

9. An anomaly detection system comprising: a model building circuit configured to, * (i) receive training message sequences corresponding to messages transmitted in an in-vehicle network (IVN), and * (ii) build a model based on the training message sequences, - the model comprising a set of states corresponding to signal values ​​observed in the training message sequences and state transitions between respective states of the set of states, - a sequence generator circuit for training the model by providing it with, * (i) sequences of first messages corresponding to the training message sequences, and * (ii) sequences of second messages not contained in the training message sequences, - a modeling circuit for executing the model to identify abnormal sequences of messages transmitted in the IVR network, by * (i) receiving a sequence of IVR messages, and * (ii) outputting, from the model, a value based on the transitions between the states of the signals contained in the sequence of IVR messages, and - a model output classification circuit for indicating, based on the value, whether the sequence of IVR messages comprises an abnormal sequence of messages.

10. An anomaly detection system according to claim 9, further comprising: a decoder for decoding the training message sequences, * said model building circuit being configured to build the model based on the decoded training message sequences.

11. An anomaly detection system according to claim 9, wherein each of the states of the model corresponds to the values ​​of at least two correlated signals contained in the training message sequences.

12. The anomaly detection system of claim 11, wherein each state of the set of states comprises at least two values ​​for each state of at least two correlated signals.

13. The anomaly detection system of claim 12, wherein the modeling circuit implements the model to identify the probabilities of each of the state transitions between the respective states of the set of states.

14. The anomaly detection system of claim 9, wherein the modeling circuitry applies the model by at least one of the following steps: * (i) calculating the distance between a first state of the set of states corresponding to a first message of the IVN message sequence and a second state corresponding to a second message of the IVN message sequence and calculating the value with this distance, and * (ii) calculating the value with a probability heuristic method.

15. The anomaly detection system of claim 14, wherein the distance corresponds to the number of state transitions in the model required to move from the first state to the second state.

16. An anomaly detection system according to claim 15, wherein the second state does not correspond to any message contained in the training message sequences.

17. A computing device for implementing a vehicle anomaly detection system, comprising: - a processing device for executing instructions stored in memory for the anomaly detection system, - receives a sequence of messages from the on-board network (IVN) corresponding to messages exchanged between the electronic control units (ECUs) of the vehicle, - provides the IVN message sequence to a model, * the model comprising: ** (i) a set of states corresponding to signal values ​​observed in previous IVN message sequences, and ** (ii) state transitions between the respective states of the set of states, - produces, from the model, a value based on the transitions between the signal states of this IVN message sequence, and - produces, based on this value, an indication that the IVN message sequence comprises an abnormal sequence of messages.

18. A computing device according to claim 17, wherein the processing device further executes instructions stored in the memory for the detection system anomalies decodes the IVN message sequence and provides the decoded IVN message sequence to the model.

19. The computing device of claim 18, wherein each state of the set of states of the model corresponds to the values ​​of at least two correlated signals contained in the decoded sequence of IVN messages, * each state of the set of states comprises at least two values ​​for each signal of at least two correlated signals, and * the model identifies the probabilities of each of the state transitions between the respective states of the set of states.

20. A computing device according to claim 17, wherein - the processing device further executes instructions stored in the memory for the anomaly detection system to perform at least one of the following steps: * (i) calculating a distance between a first state of the set of states corresponding to a first message of the IVN message sequence and a second state corresponding to a second message of the IVN message sequence, with calculation of the value with this distance, corresponding to the number of state transitions in the model, necessary to go from the first state to the second state, and * (ii) calculating the value by a probability heuristic method.

Images