System and computer-based method for simulating human-like control behavior in an environmental context

Abstract

A computer-based method for simulating human-like decisions in an environmental context, comprising: - capturing environmental data with at least one sensor, - implementing a computer-based method for achieving a bidirectional compression of the data by compressing high-dimensional data into a low-dimensional mapping, - if environmental data is captured during a learning phase of the computer-based model, assessing the quality of the mapping by the good degree of separation of data with different characteristics through the mapping of the compressed data, the captured data corresponding to known pre-recorded data that has been pre-assessed, - if environmental data is captured after the learning phase, adding new points to the compressed data and generating a signal indicating which human-like decision to use to correspond to the state of the operator.

Description

BACKGROUND

[0001] The present invention relates to a computer-based method for simulating human-like control behavior in an environmental context, and in particular to such a system and method comprising a computer-based method for implementing bidirectional compression of data.

[0002] Today, assistance systems are used for many applications in almost any kind of industry. In particular, the automotive industry or the aviation industry as well as the computing industry assistance systems are commonly used to provide an operating system for a user. In the automotive industry, widely used are known systems like ESC (Electronic Stability Control) which improves the safety of a vehicle by detecting and minimizing skidding, or EBA (Emergency Brake Assist) which ensures maximum braking force by resolving the speed and force with which the brake pedal is pushed in emergency situations. In the aviation industry, active autopilots or so-called wired flight systems are commonly used in modern civil and military aircraft, wherein electronic interfaces replace manual flight control devices by converting the movements of the flight control devices into electronic signals and without the need for human intervention to activate the flight control devices.

[0003] Furthermore, the number of sensors and actuators integrated in systems like vehicles or aircraft is rapidly increasing with the aim to electronically monitor and / or control each component in these systems.

[0004] One problem arising from the increasing number of sensors and actuators is the amount of data that needs to be processed and / or analyzed for identifying a specific situation or operating state of the system.

[0005] The intention of modern assistance systems is to provide a robust, sensitive and real-time monitoring of human intentions while operating a machine in a dynamically changing environment. Such modern assistance systems enable an efficient and safe operation of the machine, optimized for the actual human intention operation. Such a machine can be any kind of system operated by a human, like for example a vehicle, an aircraft, a ship, a machine, etc. as well as a computing system like for example a computer game. In the following, the term "system" is to be understood as any of these types.

[0006] Assistance systems commonly use virtual development methods. In the automotive development process, these methods are rapidly gaining importance to address important business challenges: shortening product time-to-market, optimizing product quality, performance and value, reducing production and development costs, and complying with increasingly stringent emission and safety regulations. One particular challenge in the field of engines and powertrains, for example, is the upcoming European Real Driving Emissions (RDE) regulation. RDE compliance requires controlling vehicle exhaust emissions under a variety of operating conditions. This severely impacts the traditional engine calibration process, which is based on fixed driving cycles.

[0007] The virtualization part of the calibration process allows to evaluate the vehicle and engine behavior under various real driving conditions by running a physical or grey-box vehicle model in a virtual driving environment operated by a virtual driver model.

[0008] Perception / cognition architectures with artificial (i.e. implemented on a machine, or in a computer or as a computer implementation, or in a tool or as a tool implementation) memory systems are used as such assistance systems. They are particularly suitable to understand and control the dynamic behavior of a tool in response to a human operator, especially for real-time safe interaction of the operator and the tool in a dynamically changing environment.

[0009] Document WO 2014 009031 relates to a method or system or architecture, in particular a perception / cognition architecture with an artificial memory system, comprising:

[0010] - at least one first node adapted to store and call (e.g. long) input sequences, such as (e.g. long) input sequences modeled as events;

[0011] - at least one second node adapted to extract and save prototypes from raw data, each prototype representing a chunk of data characterizing real-world operations of the tool (in its environment) with similar features.

[0012] In the architecture disclosed herein, the second node provides input to the first node. The raw data is typically data obtained from sensors, e.g. sensors for sensing the activity of actuators or control elements of a machine or tool.

[0013] The artificial memory system is typically composed of a plurality of nodes, each node solving one specific task like for example one of the following: spatial pooling, signal quantification, temporal and forced temporal pooling, events and forced events pooling. The functionality of the whole system is achieved by connecting these nodes with different types of directed connections, e.g.: feedforward (FF) input / output, feedback (FB) input / output, state consistency check, etc.

[0014] The memory prediction framework (MPF) described in this document is, in particular embodiments, an "artificial memory arousal system" inspired by the functioning of the neocortex of mammals, capable of discriminating, storing, recognizing spatial / temporal patterns of sensory parameters, capable of classifying exposed events, and making predictions over time. A MPF network is composed of a hierarchical set of nodes that can contain an adaptive memory system similar to a neurogas, capable of analyzing different spatio-temporal inputs in parallel, then communicating across the whole hierarchy in order to discover higher level behaviors and properties.

[0015] The three main purposes of a node with inputs and outputs are to learn and recognize particular input patterns, to associate them with a number of external (possibly unknown) causes or events, and to eventually predict the next input pattern.

[0016] To achieve these goals, two different phases are usually distinguished: a training phase or learning phase, for learning the input patterns that affect the internal "memory" of the node, and an inference phase, which is the actual operating phase of the node, during which the input patterns are compared with the learned patterns and the FF and FB outputs are generated.

[0017] In practice, there is nothing that prevents a node from operating in a "continuous learning" fashion, where training and inference are done together. Another possibility is to have a very short initial training phase to overcome the "cold start" problem, followed by a continuous learning phase.

[0018] In any case, the purpose of the inference phase is to evaluate the current input pattern in order to associate it with one or more learned causes. This leads to the generation of a FF output signal that represents the likelihood that the current input pattern belongs to one of the learned causes. This process is usually also influenced by a FB input signal that represents the prediction of the current FF output signal provided by the parent node. This last feature is to allow the node to disambiguate between different causes that can share a common input pattern: this is usually referred to as "focus of attention". After generating the FF output, the node should also generate a prediction of the next input pattern to send a signal to the child node via its FB output: this can require waiting for all the network to complete the FF update to propagate the prediction from top to bottom nodes.

[0019] It is clear that the exchange of information between nodes, thanks to the FF and FB connections, is at the core of the Memory Prediction Framework (MPF), such as the one disclosed in the above mentioned WO 2014 009031.

[0020] Therefore, the efficiency of the communication between nodes affects the whole system both in terms of number of bits that can be exchanged and in terms of amount of computation within the nodes. In fact, the higher the efficiency, the more bits can be exchanged between two nodes and the less amount of computation is required within the nodes to process the same amount of data.

[0021] The bidirectional mapping is a dimensionality reduction technique that avoids the typical pitfalls of common techniques such as Principal Component Analysis (PCA) and usual manifold learning algorithms. In particular, it avoids linearity, which constrains the learned manifold to low-accuracy mappings, and directionality, which fosters the impossibility of recovering the original high-dimensional points from their low-dimensional representation.

[0022] A well-known computer-based method named t-distributed Stochastic Neighbor Embedding (t-SNE) is disclosed in the article titled "Visualizing Data using t-SNE" by Laurens van der Maaten and Geoffrey Hinton published in the Machine Learning Research journal, volume 1 (2008) pages 1-48.

[0023] The t-SNE method allows to visualize high-dimensional data sets with a reduced dimensionality preserving proximities without any possible inference on the visualization. SUMMARY

[0024] The present invention aims at improving the global efficiency of a memory prediction framework by performing an invertible dimensionality reduction on the inter-node communication, which allows a bidirectional mapping of data between a high-dimensional space and a low-dimensional space.

[0025] A computer-based method for achieving a bidirectional compression of data by compressing high-dimensional data into a lower-dimensional mapping, for example a two- or three-dimensional mapping, achieves this goal, the method comprising: receiving data having a first dimension greater than one dimension, for example data captured and transmitted by at least one sensor; and compressing said data or properties derived from said data, such as GPS data segments or data segments of an on-board electronic control unit of a vehicle, into compressed data having a second dimension lower than the first dimension.

[0026] According to the general features of this computer-based method, for two metric spaces X = (x, μ x ) and Y = (y, μ y ), where x is a first space of dimension greater than or equal to the dimension of a second space y, μ x is a first metric on said first space x, μ y is a metric on said second space y, P is a group of a first n points belonging to said first space x, a point being an element of a space, the compression is achieved by finding a group Q of a second n points belonging to said second space y by the following equation:

[0027]

[0028] where d(.,.) is a symmetric distance function on the space of square matrices, and A1 and A2 are two arbitrary functions having the following properties, A being either of the A1 and A2 functions:

[0029] - A: R + → [0,1],

[0030] - A(0) = 1,

[0031] - limx→∞ A(z) = 0,

[0032] A(z1) > A(z2) if and only if z1 < z2.

[0033] A1 and A2 can be any function with the said properties, including:

[0034] When s > 0, or when s > 0, or when s > 0 or when s > 0

[0035] Once equation 1 is solved for each of the n points of the group P of the first n points, the correspondence between the points of the two groups P and Q of n points, called BiMap, BiMap = (P, Q) can be used as a transformation atlas to transform the points of the first space x into points of the second space y and vice versa by means of the following equations 2 and 3.

[0036] The first space x and the second space y are general spaces with the dimension of y being less than the dimension of x. The computer-based method creates a correspondence between the points in x and the points in y. Thus, the compression is achieved by making the points of y less dimensional than the points of x. This compression method is defined as bidirectional because it allows both the compression of data and the decompression of data, contrary to other manifold learning algorithms that only allow the compression of data without allowing the decompression of data.

[0037] The two main differences between this computer-based method and the computer-based method of t-SNE are the possibility given here to both add new points to the mapping without the need to recalculate the whole mapping and to create the inverse correspondence from metric y to metric x to recover the higher dimensional points from their lower dimensional representation.

[0038] In fact, t-SNE does not create any mapping and the addition of any new point restricts the recalculation of all data to visualize it.

[0039] The second difference with t-SNE is also the main difference with standard manifold learning computer-based methods disclosed in the article "Iterative non-linear dimensionality reduction with manifold sculpting" by Gashler et al. published in NIPS (2007) volume 8 pages 513-520 and in the article "Diffusion maps" by Coifman et al. published in the journal "Applied and computational harmonic analysis" volume 21 number 1 pages 5-30 in July 2006.

[0040] Moreover, depending on the choice of the functions A1 and A2, this computer-based method can accurately convey the proximity relationships from P to Q and / or can reveal self-clustering features and / or can help highlighting hidden patterns in the data. For example, if A1 and A2 are the same function, then the proximity relationships are accurately conveyed, whereas when A2 is a function with a fatter tail than A1, then points that are less close on x will be spread on the projection on y, thus highlighting the neighborhood structure of the origin in x, i.e. the clusters. l With a fatter tail, then points that are less close on x will be spread on the projection on y, thus highlighting the neighborhood structure of the origin in x, i.e. the clusters.

[0041] In a first variant of this computer-based method, the method further comprises extracting properties from at least one data before initializing the bi-directional compression.

[0042] Extracting properties from each data has made it possible to pick up elements of each data that are considered relevant for future operations and thus has achieved a first compression of the data used by keeping only the relevant parts (i.e. the properties) of the bi-directional compression into a two or three dimensional mapping.

[0043] Moreover, the extraction of properties such as spatio-temporal properties has facilitated the training of the computer-based model using the bi-directional mapping compression.

[0044] In a second variant of this computer-based method, the compression method comprises compressing a new point m belonging to metric x on metric y by solving the following equation:

[0045]

[0046] and decompressing a new point t belonging to metric y on metric x by solving the following equation:

[0047]

[0048] where μx (P,m) and μ y (Q,t) is the distance vector between the points of BiMap and the new points in the corresponding respective space, BiMap being the correspondence between the points of x and the points of y given by equation 2 (compression) and equation (3) (decompression).

[0049] Moreover, when a new point m or t needs to be mapped in Y or X respectively, A1(μ x (P,m)) or A2(μ y (Q,t)) can be used to evaluate the objective mapping distortion index (MDI). Indeed, the mapping accuracy obtained by this computer-based method depends on the proximity with respect to the points in the atlas BiMap = (P,Q). The relevance is expressed by searching for the point with the minimum activation distortion from the learned points (i.e. solving equation 2 or equation 3). The possibility to evaluate the MDI causes a partial solution of the points in the X space in order to have a more accurate atlas and an intrinsic anomaly detection mechanism to support or implement the background check within the MPF.

[0050] Another object of the application is to provide a computer-based method for simulating human-like decisions in an environmental context, said method comprising:

[0051] - capturing environmental data with at least one sensor,

[0052] - implementing a computer-based method for implementing a bidirectional compression of data by compressing high-dimensional data into a lower-dimensional mapping defined above,

[0053] - if environmental data are captured during a learning phase of the computer-based model, evaluating the quality of said mapping by the good degree of separation of data with different characteristics by compressing said mapping of data, said captured data corresponding to known pre-recorded data that have been pre-evaluated,

[0054] - if environmental data are captured after said learning phase, adding new points to said compressed data and generating a signal indicating which human-like decision to use to correspond to the state of the operator.

[0055] Depending on the environmental context and the use of the method, the simulated human-like decisions can be human judgments about their environment taking into account some first type of environmental data, for example sound, or human-like control behavior predictions of what a human (for example a driver) would do knowing some second type of environmental data like the position, speed and gear of a car, or other kinds of human-like decisions.

[0056] Thus, in an architecture comprising a memory prediction framework, the computer-based method can act in three different ways: first as input data pre-processing, second as inter-node feedforward and feedback data compression, third as node output classifier.

[0057] In the first configuration, i.e. input data pre-processing, the role played by the computer-based compression method with bidirectional mapping is to make the network process data from domains of smaller dimension, thus saving storage and computing power.

[0058] In the second configuration, i.e. inter-node feedforward and feedback data compression, the computer-based compression method with bidirectional mapping compresses the feedforward output from the lowest layer nodes and decompresses the feedback output from the highest layer nodes.

[0059] In the third configuration, i.e. node output classifier, the computer-based compression method with bidirectional mapping helps to visualize the learning dynamics within the nodes, thus facilitating the qualitative interpretation by humans of the information stored therein. In this case, the compression method acts as a clustering-based classifier.

[0060] In the second configuration, during the inference task which occurs after the end of the learning phase, according to WO 2014 009031, the MPF node computes the distance between the current input and all its stored coincidences with respect to a defined metric, so its output consists of a feedforward signal constituted by the distance-related probability distribution of said input as a sample of known stored coincidences.

[0061] Within the node, by using the coincidences as reference points, the space of the coincidences is coupled with a metric which indicates both that the coincidences have a naturally defined spatial concept within the node input space and that the feedforward signal is an encoding of the input in terms of its position within the input space.

[0062] In other words, for any new input m, the output of the MPF node N is the vector A N (μ N (C,m)) where A N (.) is the activation function of the node N, μ N (.) is its metric, and C is its set of coincidences.

[0063] Thus, the output of any node is the first half of equation 2, while symmetrically, the feedback output is the second half of equation 3.

[0064] In particular, the dimensionality of the output of each node increases with the memory size of the node itself. During the use of MPF in the real world, i.e. during the inference phase, a node can have an output dimensionality of hundreds or thousands of probabilities. Since the nodes in the above-mentioned layers have to learn from the output of the nodes in the lower layers, the need for dimensionality reduction is important when stacking nodes in a hierarchy of layers such as MPF.

[0065] In particular, the smaller dimensionality of the input means a reduction of the computational power needed to perform all the MPF capabilities.

[0066] This problem is even more relevant when a node of a higher layer receives as input the output of several nodes of lower layers, so that the dimensionality of its input space increases accordingly to the sum of the coincident number of its input nodes.

[0067] In feedback connections, back-projection is certainly needed, since it is mandatory for many MPF features.

[0068] Moreover, whenever the order of C exceeds the dimensionality of the input space N by at least one dimension, there is a one-to-one correspondence between the position of the input m in the node input space and the space of the feed-forward signal in the output, which means that any further correspondence increases the redundancy of A N (μ N (C,m)), which can be removed by the compression method.

[0069] Nowadays, for example when buying a new car, the sound quality of a car is a very important aspect to be considered by the consumer. Since sound quality is mainly a subjective aspect, there is no reliable way to objectively evaluate the sound quality. Therefore, the evaluation step in the car development process usually relies on the judgment of experts, which often gives a relative scale between different cars.

[0070] Therefore, this approach of the computer-based method enables to objectify the subjective judgment of the car sound, in other words, it enables to objectify the process of evaluating the car sound.

[0071] In a first aspect of the computer-based method for simulating human-like decisions in an environmental context, the computer-based method comprises pre-processing the environmental data with at least one filter before implementing a computer-based method for implementing a bidirectional compression of data, the computer-based method for implementing a bidirectional compression of data being implemented as input filtered environmental data.

[0072] This additional step enables, particularly in the application of automotive sound processing, the environmental data to be filtered through a series of filters such as band-pass filters and pre-processing steps that help increase and diversify the amount of information processed. For example, in the application of automotive sound processing, the pre-processing steps can include de-noising, convolution or any possible audio processing. In general, it can be any type of data manipulation or feature extraction.

[0073] The MPF architecture is mainly unsupervised and data-driven. Therefore, filtering the data can help to exclude useless or misleading information, helping the MPF to focus on more important features.

[0074] In a second variant of the computer-based method for simulating human-like decisions in an environmental context, the environmental data comprises at least one sample of raw sound recorded in a car cabin with a noise sensor, and the method further comprises: inputting the environmental data into an auditory model to deliver a plurality of different versions of the environmental data, before pre-processing the data output by the auditory model with at least one filter, the pre-processing being implemented on each version of the environmental data.

[0075] This variant of the computer-based method thus enables to objectify the subjective judgment of car sound, in other words, it enables to objectify the process of evaluating car sound.

[0076] The raw sound can be recorded by using a high-fidelity microphone synchronized with CAN bus data, i.e. Controller Area Network.

[0077] The auditory model is a biomimetic technique that converts acoustic data into psychoacoustic data. The correspondence between physical sound properties and human perception is complex and far from being linear. Several operations such as integration, time-frequency masking are performed. From a computational point of view, the auditory model can be seen as a set of filters whose task is to replicate the operations performed by the human ear. This approach is widely used in speech recognition but rarely considered in the automotive field.

[0078] The raw recorded sound is processed using auditory models proposed in the scientific literature. The output of this step is a set of quantities of energy in determined frequency bands, taking into account the effects of different auditory models such as frequency selectivity, time-frequency masking.

[0079] A set of different auditory models is used to generate a plurality of "psychoacoustic versions" of the same raw sound to be processed in the following steps.

[0080] Another object of the invention is to propose a system for simulating human-like behavior in an environmental context comprising at least one sensor for capturing environmental data, and an architecture with memory for interacting with the dynamic behavior of tools and operators.

[0081] According to the general features of the system for simulating human-like behavior in an environmental context, the architecture is configured to process the steps of the computer-based method for simulating human-like decisions in the environmental context defined above.

[0082] Moreover, all the features of the application WO 2014 009031 are integrated in the present specification.

[0083] In a first solution of the system for simulating human-like behavior in an environmental context, the memory of the architecture is an artificial memory, the architecture is a first neural network with structures and mechanisms for abstraction, generalization and learning, the implementation of the first neural network comprises an artificial hierarchical memory system comprising: a receiving port configured to receive data produced by sensors (on tools); one or more first nodes configured to learn and recognize frequent occurring input attributes and sequences received from the receiving port, the one or more first nodes form a second neural network comprising neurons and edges connecting two or more of the neurons in a graph of components; and an output port configured to output data built by the architecture and associated with the behavior, whereby each or essentially all of the one or more first nodes are adapted for time series analysis and comprise connected components in a topological or temporal graph.

[0084] In a second solution of the system for simulating human-like behavior in an environmental context, the system comprises a pre-processing filter to apply a set of different filters to the environmental data captured by at least one sensor.

[0085] In a third solution of the system for simulating human-like behavior in an environmental context, the environmental data comprises at least one sample of raw sound recorded within a car cockpit with a noise sensor, and the system further comprises an auditory model configured to be applied to the recorded environmental data to output a plurality of different versions of the environmental data before the filtering pre-processing filter, the filtering pre-processing being implemented on each version of the environmental data.

[0086] Another object of the application is to propose a motor vehicle comprising an electronic control unit comprising a system for simulating human-like behavior in an environmental context. DETAILED DESCRIPTION

[0087] The application will be described with reference to particular embodiments and with reference to certain drawings but the application is not limited thereto but only by the claims. The drawings described are only schematic and are non-limiting. In the drawings, the size of some of the elements can be exaggerated relative to other elements for illustrating the concept. Whenever appropriate, like reference numerals designate like structural elements in the accompanying drawings. The use of the terms "including", "comprising", "having" and "with" are meant to encompass the presence of something missed or of anything included but not necessarily limited to what is directly associated with the associated word.

[0088] The term "comprising", used in the claims, should not be interpreted as being restricted to the means listed thereafter; it does not exclude other elements or steps. Thus, the statement "device comprises means A and B" does not exclude some other elements or steps. It merely means that the only relevant components of the device are A and B as far as the person claiming the application is concerned.

[0089] Moreover, the terms "first", "second", "third", etc. are used herein for distinguishing between similar elements in a limited sense and are not necessarily used to describe a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the application described herein are capable of functioning in other sequences than the one described or illustrated herein.

[0090] The system for simulating human-like behavior in an environmental context according to embodiments of the application comprises two sensors configured to capture environmental data. In the present embodiment, the sensors are two high-fidelity microphones placed in two different locations within the cockpit of a car.

[0091] The simulation system further comprises an architecture with an artificial memory system configured to interact with the dynamic behavior of the tool and the operator. The architecture with an artificial memory system is constituted by a first neural network with structure and mechanisms of abstraction, generalization and learning.

[0092] The implementation of the first neural network comprises an artificial hierarchical memory system comprising a receiving port configured to receive data generated by the microphones, one or more first nodes configured to learn and recognize frequently occurring input attributes and sequences received from the receiving port, and one or more second nodes. The one or more first nodes form a second neural network comprising neurons and edges connecting two or more of said neurons in a graph of components, and an output port configured to output data built by the architecture and associated with the behavior, whereby each or essentially all of the one or more first nodes are adapted for time series analysis and comprise connected components in a topological or temporal graph.

[0093] In embodiments of the invention, reference is made to an operator, such as a driver, a skipper, a pilot, a player. Such a person can be "autonomous". For example, operator commands can be pre-given in the form of commands stored on a suitable storage medium readable by the system. For example, if the operator is a driver and the tool is a vehicle such as a car, the commands can be a destination, an expected time of arrival, an expected fuel consumption, whether to drive actively, sportily, calmly. The actual driver can also intentionally switch on the system and can provide commands to drive the driver to the destination in the manner of the normal behavior or other behavior of the driver, acceleration, braking force, speed, fuel consumption, etc. In the absence of commands, the system can also take over. For example, if it is detected that the driver is falling asleep, the system will safely stop the vehicle. Whether the driver has fallen asleep can be detected by a consciousness check.

[0094] Embodiments of the invention are based on the Memory Prediction Framework (MPF) and comprise an artificial hierarchical memory system. The artificial hierarchical memory system can be an enhanced Hierarchical Temporal Memory (HTM) with improved capabilities for the scene recognition / "classification" and / or pre-processing phase. The system and method have been adapted to be able to provide any suitable number of samples required such as 10-15 samples or for example around half a second depending on the sampling frequency, the prediction with respect to input operator commands due to the introduction of a time event management and the above-mentioned pre-processing phase.

[0095] The architecture of the embodiment of figure 1 corresponds to the architecture comprising an enhanced hierarchical memory described in the document WO 2014 009031.

[0096] The Memory Prediction Framework is suitable as a basic model in that it foresees almost all the structures and mechanisms necessary for good learning, abstraction and generalization, for example the flow of feedforward and feedback signals within the network, their correspondence as a measure of the degree of understanding of the situation, which is a prerequisite for assessing the generalization capabilities of the network. The MPF computation method and its learning algorithm have been shown to work well with different sensory channels, but still require pre-processing or conditioning specific to the sensory. Once established, i.e. adapted to a specific sensor modality, no pre-processing is in principle required. However, it can be advantageously applied from time to time as for example the so-called primitives mentioned earlier. Embodiments of the invention can be seen as instances of suitable parts of the MPF.

[0097] The computer-based method for simulating a human-like decision in an environmental context is implemented by nodes within an artificial hierarchical memory of the simulation system.

[0098] In this example, the human-like decision is the behavior of a driver of a car, the environmental context considered corresponding to the sounds around him.

[0099] In the first step of the simulation method, several high-fidelity microphones synchronized with CAN bus data record the raw sound inside the car's cockpit. The recorded raw sound is in a high-dimensional configuration, such as a three-dimensional configuration.

[0100] In the second step of the simulation method, electronic signals corresponding to the recorded original sounds are sent to an architecture with the artificial memory system and preprocessed by a set of auditory models to generate multiple “psychoacoustic versions” of each recorded original sound, which will be input into the artificial memory system after a filtering step.

[0101] In the third step of the simulation method, the electronic signals corresponding to multiple psychoacoustic versions of each recorded original sound are preprocessed using a set of bandpass filters.

[0102] In a subsequent step of the simulation method, the filtered sound is sent to the node to be processed using a computer-based method for bidirectional compression of the data by compressing the filtered data into a two-dimensional map.

[0103] In the steps of the simulation method and the first step of the compression method, the filtered sound is received by other nodes or the same nodes.

[0104] In the second step of the compression method, node operations extract attributes from each filtered raw sound in order to select elements of each data that are considered relevant to the method used to simulate human-like control behavior.

[0105] The extracted attributes are, for example, spatiotemporal attributes, to simplify the training of computer-based methods for simulating human-like control behavior.

[0106] In the third step of the compression method, node operations compress the attributes extracted from the filtered original sound.

[0107] The compression is achieved by finding the set Q of n points belonging to the metric y using the following equation:

[0108]

[0109] X=(x,μ x ) and Y = (y, μ y Let y be two metric spaces, where x is the first space and its dimension is greater than or equal to the dimension of the second space y, and μ is the second space. x μ is the first metric on space x. y Let P be the second metric in space y, P be the group of the first n points in the first space x, and d(.,.) be the symmetric distance function in the matrix space, while A1 and A2 are any two functions that preserve the following properties:

[0110] -A:R + → [0, 1], that is, A is a function defined for all positive real numbers and produces values between 0 and 1 including 0 and 1;

[0111] -A(0) = 1, that is, A is a function where the independent variable 0 will generate the value 1,

[0112] -lim x→∞ A(z) = 0, that is, A is a function with a limit value of 0 when the independent variable approaches growth,

[0113] -A(z1) > A(z2) if and only if z1 < z2, that is, A is a strictly decreasing function.

[0114] Once the equation 1 is solved for each of the n points of the group P of the first n points, the correspondence between the points of the two groups P and Q of n points, that is, BiMap = (P, Q) (refer to BiMap) can be used as a transformation atlas to convert the points of the first space x into the points of the second space y by means of the following equation (2) and equation (3), and vice versa.

[0115] By such steps, new points can thus be added to the mapping without having to recalculate the entire mapping, and a reverse correspondence from space y to space x can be created to recover higher-dimensional points from their lower-dimensional representations.

[0116] Therefore, by solving the following equations, any new point m belonging to space x can be compressed on space y:

[0117] And any new point t belonging to space y can be decompressed on space x by solving the following equations:

[0118]

[0119] where μ x (P, m) and μ y ​​​​​​​​​In the inference phase, after the learning phase, it can act as an inter-node feed-forward (FF) and feedback (FB) data compression or as a node output classifier.

[0123] When acting as an inter-node FF and FB data compression, the compression method is used to compress the FF output from lower layer nodes and to decompress the feedback output from higher layer nodes.

[0124] When acting as a node output classifier, the compression method helps to visualize learning dynamics within the node to help human qualitative interpretation of the information stored therein. In this case, the compression method acts as a clustering-based classifier.

[0125] If the architecture is in the learning phase, in the next step 24 of the computer-based method for simulating human-like decisions in the context of an environment, the mapping of compressed data is evaluated by comparing said compressed data with different references of previous memories and labelling the mapping of compressed data with the label associated with one of said references.

[0126] If the architecture is in the inference phase, instead of step 24, in step 26 any new properties extracted from the derivation of the new recorded raw sound are added to the mapping of compressed data and a human state is generated indicating which human-like decision to use corresponding to the use of the system of the computer-based method for simulating human-like decisions in the context of an environment.

[0127] Thus, during the inference phase, the method for simulating similar decisions can make the architecture perform several operations, such as:

[0128] - evaluate new sounds by going through the process described here without modifying the learned memories;

[0129] - infer the common characteristics of the sounds that lead to similar judgments by analyzing the low-dimensional clusters performed by BiMap;

[0130] - improve the reliability of human-like judgments by highlighting unusual behavior on the mapping;

[0131] - generate sounds from desired judgments; since all steps are reversible, it is possible to start from the position on the generated BiMap to synthesize new sounds with the expected judgment.

[0132] Below is another example of the use of the present invention, which explains in detail certain solutions of the invention.

[0133] Virtual development methods are rapidly gaining traction in the automotive development process to address important business challenges: shortening product time-to-market, optimizing product quality, performance and value, reducing production and development costs, and complying with increasingly stringent emission and safety regulations. One particular challenge in the engine and powertrain domain is the upcoming European Real Driving Emissions (RDE) regulation. RDE compliance requires controlling vehicle exhaust emissions under a variety of operating conditions. This severely impacts the engine calibration process, which is traditionally based on fixed drive cycles.

[0134] The virtualization part of the calibration process allows evaluating vehicle and engine behavior under a variety of real driving conditions by running a physical or grey-box vehicle model in a virtual driving environment operated by a virtual driver model.

[0135] The invention is in this example used to develop a new external driver model that handles acceleration / braking, clutch and shifting in a human-like way, i.e. a method for simulating human-like control behavior.

[0136] The method of the invention is based on machine learning, as it allows the model to learn from recorded data and replicate the driver behavior on any arbitrary route, making it suitable for RDE simulation.

[0137] The proposed method consists of three main stages, in which the nodes of the architecture set out to establish a relationship between the recorded data in the test and the simulation environment.

[0138] In the first stage, called the extraction stage, the data captured by sensors such as sensors 110 is sent to a mapped attribute extractor. The mapped attribute extractor can either take as input recorded Electronic Control Unit (ECU) data together with Global Positioning System (GPS) locations, or only GPS points as input, to produce attributes of the route connecting them. The first case is to prepare the input in order to learn real driver behavior from the data, while the second case requires to infer the learned behavior through new routes.

[0139] In the second stage, called the learning stage, the captured data is input into the model to learn real driver behavior on a driving route and generalize it in order to replicate said behavior on new routes. In this stage, BiMap is used to learn real driver behavior in the environment by sensing ECU and GPS data. In the reasoning stage, BiMap produces expected human behavior in new environments.

[0140] In a third module, called the reasoning phase, Functional Mock-up Unit (FMU) controllers, which are function models in closed loop with the simulated environment, use the constructed models during the learning phase to implement the driver longitudinal behavior by controlling (shift and pedals) on the car in a human feasible way.

[0141] This example shows an artificial driver that can learn real driving behavior on a recorded route and generalize it to arbitrary routes. The driver model exhibits human-like driving behavior on both vehicle pedals and gear selection, so it is suitable for use in simulated emissions.

[0142] The present application can be implemented on a processor system. The processing system can include a computing device or processing engine, such as a microprocessor or FPGA. Any of the methods described above according to embodiments of the present application or claims can be implemented in the processing system. Such a processing system can include at least one customizable or programmable processor coupled to a storage subsystem including at least one form of memory, such as RAM, ROM, etc. It should be noted that the one or more processors can be general or special purpose processors and can be contained in a device, such as a chip having other components that perform other functions. Thus, one or more aspects of the methods according to embodiments of the present application can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations thereof. The processing system can include a storage subsystem having at least one of a disk drive and / or a CD-ROM drive and / or a DVD drive. In some implementations, a display system, a keyboard, and a pointing device can be included as part of a user interface subsystem to provide for manual input of information, such as parameter values, by a user. More elements, such as network connections to various devices, interfaces, etc., that allow connection to different types of physical sensors can be included. The various elements of the processing system can be coupled in various manners, including via a bus subsystem, as a single bus for simplicity, but one of skill in the art will appreciate that a system including at least one bus is contemplated. The storage of the storage subsystem can hold some or all of a set of instructions that implement the steps of the method embodiments described in the present document at some time while executing on the processing system.

[0143] The present invention also includes a computer program product having software according to the present invention, which when executed on a processing engine can contain code segments providing a perceptual / cognitive architecture with artificial memory for interacting with the dynamic behavior of the tool and operator, wherein the perceptual / cognitive architecture is based on an implementation of a memory prediction framework with structures and mechanisms for abstraction and generalization and selective learning, the implementation of the memory prediction framework including an enhanced artificial hierarchical memory system. The software can be adapted to cooperate with a pre-processing stage, the software when executed can be adapted to provide predictions about input operator commands based on time event management, and to describe input data from physical sensors in terms of primitives and cyclic patterns.

[0144] The present invention also includes a computer program product having software according to the present invention, which when executed on a processing engine can contain code segments providing a perceptual / cognitive architecture with artificial memory for interacting with the dynamic behavior of the tool and operator, wherein the perceptual / cognitive architecture is based on an implementation of a memory prediction framework with structures and mechanisms for abstraction and generalization and selective learning, the implementation of the memory prediction framework including an enhanced artificial hierarchical memory system. The software can be adapted to cooperate with a pre-processing stage, the software when executed can be adapted to provide predictions about input operator commands based on time event management, and to describe input data from physical sensors in terms of primitives and cyclic patterns.

[0145] The present invention also includes a computer program product having software according to the present invention, which when executed on a processing engine can contain code segments providing a perceptual / cognitive architecture with artificial memory for interacting with the dynamic behavior of the tool and operator, wherein the perceptual / cognitive architecture is based on an implementation of a memory prediction framework with structures and mechanisms for abstraction and generalization and selective learning, the implementation of the memory prediction framework including an enhanced artificial hierarchical memory system. The software can be adapted to cooperate with a pre-processing stage, the software when executed can be adapted to provide predictions about input operator commands based on time event management, and to describe input data from physical sensors in terms of primitives and cyclic patterns.

[0146] The above software can be adapted for the physical I / O layer, the enhanced hierarchical memory, the temporal master, the background check, the supervisory gating, the system control unit, and the user control I / O layer. The software can be adapted to translate the physical sensor input without modifying the information content to a format manageable by the enhanced hierarchical memory. The software can be adapted to be taught and recognize, and predict sequences of input patterns received from the physical I / O layer. The software can be adapted to output compressed higher layer information based on the physical sensor input received by the physical I / O layer.

[0147] While the application has been illustrated and described in detail in the drawings and foregoing description, such illustration and description is to be considered illustrative or exemplary and not restrictive; the application is not limited to the disclosed embodiments.

[0148] Other modifications that are within the scope of the claimed application can occur to those skilled in the art upon studying the drawings, the disclosure and the appended claims.

[0149] For example, a single unit can implement the functionality of several items detailed in the claims. Any reference signs in the claims should not be construed as limiting the scope.

[0150] For example, in embodiments of the application, the modeling action can be a "path" or "sequence" employing a Markov chain, whereby such embodiments of the HTM can "follow" only one path.

[0151] The present application further provides an architecture developed in compliance with the theory of the Memory Prediction Framework (MPF), and particularly relates to a system developed based on the hMPF to handle specific classification / prediction problems, providing an artificial brain as its main processing element.

[0152] The artificial brain in turn can comprise many processing nodes organized in layers and communicating with each other through feedforward (FF), feedback (FB), and prediction signals. These signals are essentially probability density functions (PDFs) representing the likelihood that a particular input pattern is the result of a particular world cause. The system as a whole can further check the state of the hierarchical network with the help of a background check signal also provided as output to the network.

Claims

1. A computer-based method for achieving bidirectional compression of high-dimensional data by compressing it into a lower-dimensional mapping on a system for generating signals indicative of human-like decision-making in an environmental context, the method comprising: - Receive environmental data with a first dimension, which is greater than the dimension of the data captured and transmitted by at least one sensor; The environmental data or attributes derived from the environmental data are compressed into compressed data with a second dimension that is lower than the first dimension. - During the learning phase, the quality of the compressed data mapping is determined by assessing the degree to which pre-evaluated known received environmental data with different characteristics are separated through the mapping of the compressed data, in order to evaluate the mapping of the compressed data. - In the reasoning phase, signals are generated to guide human-like decision-making, and this is the actual operation phase of the nodes. During this phase, the input patterns of the environmental data are compared with the patterns memorized in the learning phase, and feedforward (FF) and feedback (FB) outputs are generated. Its characteristic is that, for two metric spaces X = (x, μ) x ) and Y = (y, μ y ), where x is the first space with a dimension greater than or equal to the dimension of the second space y, and μ x It is the first metric on the first space x, μ y It is a second metric in the second space y, and the compression is achieved by bidirectional mapping between a group P of first n points of the environmental data belonging to the first space x and a group Q of second n points of the compressed data belonging to the second space y, wherein each point in the group Q of the second n points is paired with the group P of the first points by the following equation: Where μ x (P) is the first symmetric distance matrix between points in P, μ y Let d(t) be the second symmetric distance matrix between points in Q, and d(.,.) be a symmetric distance function in the matrix space, while A1 and A2 are any two functions with the following properties: -A:R + → [0,1], -A(0)＝1, -lim x→∞ A(z)=0, -A(z1)>A(z2) if and only if z1 <z2, The method further includes: compressing a new point m belonging to the first space x into the second space y by solving the following equation: And by solving the following equation, a new point t belonging to the second space y is decompressed onto the first space x: Where μ x (P,m) and μ y (Q,t) is the distance vector between a point in BiMap and a new point in the corresponding space. BiMap is the correspondence between points in the first space x and points in the second space y, represented by equations 2 and 3. Point Q is calculated from point P by solving equation 1. The system includes an architecture with a memory system, which is configured to interact with the dynamic behavior of tools and operators. The memory in the architecture is artificial memory, and the architecture is a first neural network with structures and mechanisms for abstraction, generalization, and learning. An implementation of the first neural network defines an architecture including an artificial hierarchical memory system, which includes: a receiving port configured to receive environmental data generated by sensors; one or more first nodes configured to learn and recognize frequently occurring input attributes and sequences received from the receiving port, the one or more first nodes forming a second neural network, the second neural network including neurons as components and edges connecting two or more of the neurons in a connection graph; and an output port configured to output environmental data constructed by the architecture and associated with the behavior, whereby each or essentially all of the one or more first nodes are suitable for time series analysis and include components connected in a topology graph or time graph.

2. The computer-based method of claim 1 further includes extracting attributes from at least one environmental data prior to initializing the bidirectional compression.

3. The computer-based method according to claim 1 or 2, characterized in that, Before implementing a computer-based method for bidirectional data compression, the environmental data is preprocessed using at least one filter, the computer-based method for bidirectional data compression receiving the filtered environmental data as input.

4. The computer-based method of claim 1 or 2, wherein the environmental data includes at least one sample of raw sound recorded in a vehicle cockpit using at least one microphone, and the method further includes inputting the environmental data into the auditory model before preprocessing the data output by the auditory model using at least one filter to deliver multiple different versions of the environmental data, the preprocessing being performed on each version of the environmental data.

5. A system comprising: At least one sensor to capture environmental data, and an architecture with memory for interacting with the dynamic behavior of tools and operators. The feature is that the architecture is configured to process the steps of the computer-based method according to any one of claims 1 to 4.

6. The system of claim 5 further includes a preprocessing filter for applying a set of different filters to the environmental data captured by the at least one sensor.

7. The system of claim 5, wherein the environmental data includes at least one sample of raw sound recorded in the cockpit of a vehicle using a noise sensor, and the system further includes an auditory model configured to be applied to the recorded environmental data prior to filtering preprocessing to output multiple different versions of the environmental data, the filtering preprocessing being performed on each version of the environmental data.

8. A computer program product having software for implementing the computer-based method according to any one of claims 1 to 4, the computer program product, when executed on a processing engine, being capable of including code segments that provide a perceptual / cognitive architecture with artificial memory for interacting with the dynamic behavior of tools and operators, wherein the perceptual / cognitive architecture is based on an implementation of a memory prediction framework having structures and mechanisms for abstraction and generalization as well as selective learning, the implementation of the memory prediction framework including an enhanced artificial hierarchical memory system.

9. A motor vehicle including an electronic control unit, said electronic control unit comprising the system according to any one of claims 5 to 7.

Images