improved method for generating an assessment

Abstract

A method for generating an evaluation of a situation is based on an evaluation model (MOD). The evaluation model is configured to receive an input comprising modality elements () from sensors equipping an environment and / or a user. The model is also configured to generate, depending on the input, an output evaluation element (), representative of a given aspect of a situation associated with the input modality elements (). The evaluation element makes it possible to control a triggering of a service adapted to the environment and / or to the user. The method comprises, prior to the generation of a given evaluation element (), a filtering. The filtering comprises determining whether not taking into account a given modality element () improves the accuracy of the generation, and ignoring said modality element () in the generation of said evaluation element () in the event of a positive response. Figure for the abstract: [Fig. 2]

Description

Title of the invention: improved method for generating an evaluation 1. Field of the invention

[0001] The invention relates to the field of data analysis, and more particularly the processing of data from sensors equipping an environment and / or a user to evaluate the extent to which a particular situation occurs. 2. Prior art

[0002] The processing of data from sensors is a rapidly expanding field, driven by the rise of the connected home (or "smart home"), but more generally by connected buildings (hospital services, medical residences, etc.) and complex systems incorporating a large number of sensors (connected and in particular autonomous vehicles, aircraft, etc.).

[0003] These systems have in common the implementation of a large number of sensors of different types: thermometer, LIDAR, radar, camera, hygrometer, smoke detector, biometric sensors, microphone, physiological sensors on a connected object (or "wearable" in English), and more generally connected objects (or loT, for Internet of Things). These connected objects and sensors make it possible to extract a large number of very heterogeneous measurements (speed, position, temperature, humidity, presence of smoke, electrocardiogram, respiratory rate, etc.). This extraction may require the analysis of complex data (image and / or sound processing, semantic analysis) to extract more useful information from this data (emotional state of a user, voice or gesture instructions, speed of closing a door, meaning of a traffic sign visible by a camera, etc.)) than the raw material they form (audio or video signal, text, multimedia or textual data more generally, ...).

[0004] For the remainder of this application, all of these types of data will be referred to as modalities, a modality corresponding to a type of data captured by an electronic data capture device (typically, but not exclusively, a sensor).

[0005] These data, or modalities, are very heterogeneous, and their joint analysis is a field in its own right called multimodal analysis. Multimodal analysis is a recent discipline whose objective is to extract, i.e. predict or generate, from multimodal data (i.e. data of different types from different sensors) relevant indicators relating to a given situation. Learning Multimodal allows for much better performance in analyzing a complex situation than its monomodal counterpart, hence its growing popularity, particularly in the complex systems mentioned above (connected building, autonomous vehicle, etc.).

[0006] This multimodal analysis is carried out using multimodal learning. The product of this multimodal learning is a multimodal model capable of processing a plurality of modalities from which the model can extract indicators relating to a given situation.

[0007] However, classic multimodal models have a number of drawbacks. On the one hand, some modalities may be very relevant to enable the multimodal model to evaluate a given situation (for example, the emotional state of a person, or the probability of a fire in a home) while other modalities have only a marginal influence and interfere with the evaluation. Worse still, some modalities deteriorate the prediction. We will subsequently speak of the contribution of a modality to an evaluation. This contribution may, for example, be positive (the data is crucial for evaluating a situation), weak (the data is optional, i.e. marginally benefits the evaluation), zero (taking this modality into account or not is neutral for the final evaluation), or negative (taking this modality into account harms the final evaluation).It is understood that this categorization is only an example, and that the qualification of a modality's contribution to an evaluation can be more rigid (for example binary) or on the contrary more continuous (forming a continuum ranging from crucial to harmful).

[0008] Furthermore, some modalities may contribute positively to an evaluation when they take a certain value, and deteriorate it with another value. Here again, known multimodal models are unable to discriminate modalities according to their value and their contribution to an evaluation.

[0009] More generally, these complex systems are faced with a dual problem of managing trust in data sources and the presence of uncertainty in the information. Several approaches have attempted to overcome this dual problem.

[0010] The article M. Ravi, Y. Demazeau and F. Ramparany, "Reasoning with Trust and Un-certainty Illustration in the Internet of Things,” 2015 IEEE / WIC / ACM International Conference on Web Intelligence and Intelligent Agent Technology (WLIAT), Singapore, 2015, pp. 125-128, doi: 10.1109 / WLIAT.2015.174, describes a modeling, by an ATMS (Assumption-based Truth Maintenance System) inference engine, of the trust in a source in its capacity to carry out or delegate a certain task. The source is then judged reliable or not. However, this modeling characterizes the source as such, or formulated differently, the modality associated with the source as a whole, and not the reliability of a modality according to the values ​​it can take.

[0011] The article VS Simankov, VV Buchatskaya, PY Buchatskiy and SV Te-ploukhov, "Classification of information's uncertainty in System research," 2017 XX IEEE International Conference on Soft Computing and Measurements (SCM), St. Pe-tersburg, Russia, 2017, pp. 187-189, doi: 10.1109 / SCM.2017.7970534, proposes a classification according to the degree of uncertainty on an information source, among the following types of uncertainty: total certainty, stochastic uncertainty and fuzzy. However, the type of information uncertainty described therein depends on the input, and has a significant influence on the choice of the mathematical method for modeling the process under consideration. In fact, the analysis described in this article is agnostic to the functionality targeted by the system, in other words does not depend on the nature of the evaluation of a situation.

[0012] Furthermore, conventional multimodal models only work with complete data, that is to say that all the data must be present as input for the model to be able to extract a specific indicator. However, it is common, in practice, for some data to be missing. This is typically the case for complex systems equipped with heterogeneous sensors arranged in various locations. Each sensor can fail or malfunction. A simple failure of a single sensor can then seize up the entire model. The risk of failure increases, in fact, with the complexity of the system. The problem is similar in the event of a communication failure, even temporary, between the sensor and the rest of the components of the complex system. Other events, such as changing the location of the sensor, are also likely to cause paralysis of the multimodal model during this movement of the sensor.

[0013] Thus, none of the approaches to date have been satisfactory, due to a sorting that is too coarse, or too general and poorly adapted to a concrete reality. The invention improves the situation. 3. Statement of the invention

[0014] The invention relates to a method for generating an evaluation of a situation of a new type, not having the aforementioned disadvantages. More specifically, the invention proposes a method for generating an evaluation of a situation by an evaluation model, the method being implemented by an electronic device,

[0015] the evaluation model being configured to receive a set of modality elements, called input, from sensors equipping an environment and / or a user, a modality element being associated with a modality and being representative of a state of the modality; and to generate, as a function of the input modality elements, at least one evaluation element, called output, representative of at least one given aspect of a situation associated with the input modality elements,

[0016] said evaluation element making it possible, in an operating phase, to control a triggering of a service adapted to the environment and / or to the user,

[0017] said method comprising, during the operating phase and prior to the generation of a given evaluation element, a filtering comprising the determination that the non-taking into account of a given modality element improves the precision of the generation of an evaluation element, and a non-taking into account of said modality element in the generation of said evaluation element.

[0018] Thanks to this generation method, it is thus possible to improve the precision of an evaluation model, by discarding a modality when its element (for example its value) does not contribute positively to the generation of an evaluation element (also called simply evaluation, or prediction), or in other words does not improve the quality of the prediction of the evaluation model.

[0019] The filtering is carried out a priori, that is to say before the evaluation itself. The advantage is twofold, because this filtering can be implemented upstream of any multimodal model, and allows for computational savings by carrying out, for a given situation, only a single evaluation, having filtered optimally before this.

[0020] This filtering is further optimized — it improves the quality of the model's prediction — and at fine granularity, i.e. modality by modality and element by element. Indeed, each element (for example the value) that a modality can adopt is considered individually, independently of the other elements associated with this modality. A modality can thus at times be taken into account, and at other times not, dynamically. This taking into account or not taking into account is carried out for a given evaluation, and can vary if several evaluations are carried out at the same time, which differs greatly from the prior art. Finally, this method makes it possible to automatically detect anomalies and to automatically exclude a modality when its element is clearly outside the values ​​that it should usually take.

[0021] By modality element, we mean a value, a data item, a signal, etc. that a modality can adopt. For a modality associated with a sensor (temperature, pressure, humidity, etc.), the modality element can be a measured value. For a camera embedded in a vehicle, the associated modalities can be data on an evaluation panel, a representation of the position of the road relative to the vehicle, etc. For a rotation sensor housed in a door, the associated modality can be a closing speed of the door, and so on.

[0022] According to a particular aspect, the method for generating an evaluation of a situation by an evaluation model comprises an adjustment phase during which filtering data are constructed, the filtering data comprising at least one validity information concerning at least one modality element allowing the determination during filtering that the non-consideration of this given modality element improves the precision of the generation of an evaluation element.

[0023] The adjustment phase makes it possible, prior to the operating phase of the model, to construct filtering data adapted to the model. In particular, the filtering data can be constructed and therefore optimized according to the installation location of a complex system comprising sensors and using the evaluation model to control a service according to the evaluation element. These data alone being sufficient to carry out the filtering, and being constructed upstream of the operating phase of the model, the filtering is quick to implement.

[0024] According to a particular aspect, said validity information concerning at least one modality element comprises a category to which said modality element belongs, the category being part of the set of categories comprising a so-called valid category and at least one other category.

[0025] This categorization simplifies the implementation of filtering, since it is sufficient to determine the category of a modality element in order not to take it into account.

[0026] According to a particular aspect, said set of categories comprises at least one so-called neutral category and one so-called invalid category.

[0027] This separation between a neutral category (also called non-contributory) and an invalid category makes it possible to distinguish the modality elements of the neutral category, which contribute neither positively nor negatively to the evaluation (in other words, which play the role of noise) from the elements of the invalid category which deteriorate the evaluation.

[0028] In some cases, it may indeed be more efficient to take into account elements of the neutral category, because not taking them into account would reduce the accuracy of the evaluation model. Indeed, a modality element may be classified as neutral because of an imprecision in the categorization (valid, neutral, invalid). In fact, one may want to take it into account to prevent this imprecision of categorization from harming the good prediction. Thus, at the cost of a reduced execution speed, this consideration of neutral elements makes it possible to avoid a drop in accuracy.

[0029] In other cases, ignoring neutral modality elements allows the evaluation model to be accelerated, as does ignoring invalid modality elements.

[0030] Finally, in the case of simultaneous prediction of multiple evaluation elements (each relating to an aspect of a given situation) by the evaluation model, an element can be neutral for the generation of one of these evaluation elements and valid for the generation generation of another assessment element. It is then preferable to take it into account even if this modality element is "only" neutral for the generation of the first assessment element. This allows for a joint generation of assessment elements while optimizing execution time and precision, rather than having to execute two successive generations (each with its own filtering).

[0031] According to a particular aspect, determining that ignoring a given modality element improves the accuracy of generating an evaluation element comprises determining whether said given modality element belongs to the valid category.

[0032] The filtering is thus transparent, since it is possible for a modality element to easily check whether it is taken into account or not according to its category. This makes it possible to explore the filtering data and check, for a human in particular, whether this data is consistent or not, by testing a given modality element.

[0033] According to a particular aspect, the construction of the filtering data is based on data, called test data, comprising at least one pair formed from a set of modality elements, called test data, and an evaluation element, called target data.

[0034] Thanks to this test data, it is thus not only possible to construct the filtering data, but also to test the accuracy and reliability of the model independently of whether or not modality element(s) are taken into account. Indeed, it is possible for a set of test modality elements to verify whether the model actually predicts a target evaluation element, and with what accuracy.

[0035] According to a particular aspect, the construction of the filtering data comprises, for at least one modality,

[0036] a generation, by the evaluation model and as a function of the plurality of test modality elements by excluding from said generation the modality element associated with said modality, of an evaluation element, called a test element, and

[0037] an association with said element thus excluded from validity information based on a comparison between the target evaluation element and the test evaluation element.

[0038] Thanks to this comparison by exclusion, it is thus possible to construct filtering data which make it possible to test for one or more modality elements which ones should not be taken into account.

[0039] This construction can be exhaustive (i.e. all combinations of modality element exclusion), thus covering all use cases of the complex system implementing the evaluation model.

[0040] According to a particular aspect, at least one modality defines a set formed of all or part of the modality elements associated with this modality,

[0041] the filtering data comprising for said modality information represented representative of a partition of said set associated with said modality, into subsets each associated with one of the categories of the set of categories.

[0042] Thus, for a modality whose elements belong to a domain of the modality (for example a temperature range that a temperature sensor can deliver), it is possible, thanks to this partition, to determine for any modality element, including a modality element that was not used during the filtering data construction phase, whether this modality element should be disregarded. To do this, it is determined to which subset of the domain this modality element belongs, and what the category of the subdomain is.

[0043] An exhaustive partition of the domain of a modality can thus be carried out, allowing for a small amount of information to have exhaustive information on the modality.

[0044] According to a particular aspect, the at least one piece of validity information concerning at least one modality element concerns at least one second modality element relating to another modality.

[0045] Thanks to this correlation between two elements of two modalities, it is possible to obtain a finer categorization, and therefore more optimal for a given situation.

[0046] According to one aspect of the invention, the evaluation model is a robust model in the event of missing input data.

[0047] The invention also relates to an electronic device for generating an evaluation of a situation by an evaluation model, the device comprising a processor configured to:

[0048] - implementing said evaluation model, configured to receive a game of modality elements, called input, from sensors equipping an environment and / or a user, a modality element being associated with a modality and being representative of a state of the modality; and to generate, as a function of the input modality elements, at least one evaluation element, called output, representative of at least one given aspect of a situation associated with the input modality elements,

[0049] said evaluation element making it possible, in an operating phase, to control a triggering of a service adapted to the environment and / or to the user;

[0050] said processor being further configured to, during the operating phase and prior to the generation of a given evaluation element,

[0051] - determine that not taking into account a given modality element improves the accuracy of the generation of an assessment element, and

[0052] - not taking into account said modality element in the generation of said assessment element.

[0053] The invention also relates to a computer program product comprising instructions for implementing the aforementioned method, when this program is executed by a processor. 4. List of figures

[0054] The proposed technique, as well as the various advantages that it presents, will be more easily understood, in the light of the following description of illustrative and non-limiting embodiments thereof, and of the appended drawings among which:

[0055] - [Fig.l] represents an evaluation model as employed in the invention,

[0056] - [Fig.2] schematizes the general principle of the invention,

[0057] - [Fig.3] represents a generation method according to an embodiment of the invention,

[0058] - [Fig.4] represents an example of implementation of an adjustment phase of the process of [Fig.3], and

[0059] - [Fig.5] represents a device according to the invention. 5. Detailed description

[0060] The invention starts from a multimodal evaluation model.

[0061] The general principle of the invention is to filter, upstream of the execution of an evaluation model of this type, a modality according to its current value (more generally of its modality element), when such filtering allows a priori (i.e. before the evaluation by the model itself) to improve the performance of the model. The performance concerns a gain in execution speed (since the filtered modality is ignored for the current prediction) and / or in precision (because the filtered modality degrades the precision of the model if it is taken into account, for the current modality element).

[0062] It should be noted that the filter depends not on the modality as such, but on its current value (more generally of its current modality element). Thus, for a given modality, the filtering can vary between taking it into account and not taking it into account depending on the current modality element, that is to say temporally.

[0063] The advantage of a priori filtering, i.e. before the execution of the model evaluation, is to allow you to optimize your performance while not executing it only once.

[0064] Such a model is represented in Figure 1. The evaluation model referenced MOD can be of the robust type in the event of missing data. The evaluation model MOD is configured to receive a set X of modality elements, called input, Xf, a modality element being associated with a modality m with m GM — {, mN} where M is a set of incoming modalities. These modality elements are grouped in the form of a model entry noted X = LMMuvi(ies,un integer, here between 1 and N).

[0065] These modality elements come from sensors equipping an environment and / or a user. lizer. Each element of modality Xf is representative of a state of modality m, or in other words a measurement from a sensor, directly or indirectly. By indirect measurement, we mean a measurement obtained following data processing applied to a raw measurement from the sensor, for example an image analysis to extract, from a video stream produced by a camera on board a vehicle, data such as recognition of a road sign or the position of the vehicle on a road.

[0066] From this input X, the evaluation model can generate at least one evaluation element, called output and noted St. This output evaluation element St is representative of at least one given aspect of a situation associated with the input X. This evaluation element is also called “class” predicted by the evaluation model.

[0067] The generation of an evaluation is repeated, at different times. It can be implemented at regular intervals, for example every second, every minute or every hour. This generation allows supervision of the complex system by the evaluation model, and the triggering of a service when the evaluation takes certain values ​​at the output of the model.

[0068] A modality element x"1 is a representation, at a given time r, of a modality m. For this modality m, we call the ordered series of modality elements over time a temporal sequence of the modality, noted xm = (-^)-We understand that the index ' in fact identifies a given instant!. Similarly, we call the series of evaluation elements over time the temporal sequence of evaluation, noted S = (X() ■ We can generalize the operation of the evaluation model by the fact that it generates an evaluation temporal sequence as a function of the temporal sequences of modalities.

[0069] By given situation, we mean the state of a complex system (housing, building, vehicle, etc.) as a whole, this situation being understood by the evaluation model using the modalities at its disposal. By aspect of a given situation, we mean one of the characteristics to describe this situation, a situation being able to be described by several aspects. For example, for a medical residence, which therefore presents at a given moment a given situation, one aspect can be the probability of fire, another aspect can be the need to start an air conditioner or a heater, yet another aspect can be an indicator of the state of one of the residents (emotional state, physical state, etc.).

[0070] Although subsequently, the evaluation model is described as generating an evaluation element, this evaluation model is capable of simultaneously generating several evaluation elements each relating to a given aspect of the state of the system considered.

[0071] The evaluation element St makes it possible to trigger or control a service adapted to the environment and / or the user depending on the state of the aspect of the situation being assessed. For example, if a fire is detected, an audible alarm is triggered immediately, fire doors are closed, a notification is issued to a user (e.g. on a connected bracelet or any other wearable device) and an alert for the appropriate emergency services is issued.

[0072] As explained above, the model is capable of functioning even in the absence of a modality element, in particular without modification or retraining of the model. Such a model may for example be based on an encoder / decoder approach based on Transformers. Such a model may be trained in a supervised manner on a labeled learning data set comprising all the modalities {mj, ..., used as input to the evaluation model, for the recognition of a given aspect of a given situation.

[0073] Reference is now made to [Fig.2] to describe the general principle of the invention. In this figure, upstream of the evaluation model MOD, a filter F is provided. Formulated in temporal terms, filtering is implemented before the generation of an evaluation element.

[0074] Filtering includes:

[0075] - determine whether ignoring a given modality element x"'; improves the accuracy of generating an evaluation element St, and

[0076] - in case of a positive response, ignore (i.e. do not take into account) said element of modality x^' in the generation of the evaluation element S;.

[0077] In the example shown in Figure 2, it is the modality element x",; associated with the modality mi which is filtered, i.e. ignored. Several elements can be filtered at the same time. When a modality element is filtered, the effect is the same for the evaluation model as if this modality element were absent.

[0078] Such filtering by modality element improves the performance (accuracy and execution speed) of the evaluation model. Indeed, discarding a modality when its modality element (for example its value) does not contribute positively to the generation of an evaluation element not only makes the model faster to execute (unnecessary data are not used), but also more precise (data detrimental to the accuracy of the evaluation are filtered).

[0079] This filtering is all the more interesting because it is carried out a priori, that is to say before the evaluation itself. This filtering is thus compatible with any multimodal model of the type capable of functioning in the event of a missing modality. This filter saves computation time, since a single generation of evaluation is carried out for a given input, while having relative confidence in the optimality of the filtering.

[0080] This filtering is fine-grained, that is to say that it processes modality by modality and, for the same modality, modality element by modality element. Each element that a modality can adopt is considered individually, independently of the other elements associated with this modality. A modality can thus at times be taken into account, and at other times not, dynamically depending on the input flow X. This filtering depends on the evaluation considered. It can in fact vary if several evaluations are generated at the same time, which differs greatly from the prior art.

[0081] Finally, this filtering makes it possible to automatically detect anomalies and automatically exclude a modality when its element is clearly outside the values ​​that it should usually take. In such a case, it is then detected that the filtering of this modality element improves the accuracy of the model. This filtering also makes it possible to avoid paralysis of the evaluation model in the event of movement or replacement of a sensor, as well as in the event of malfunction.

[0082] Reference is made to Figure 3 to describe in more detail the main steps of the method for generating an evaluation of a situation, according to an embodiment of the invention. A method for generating an evaluation comprises a filtering step F1 of the type described above, and a step G1 for generating an evaluation element by the evaluation model MOD. These two steps F1 and G1 are executed during a phase P2 called the operating phase, from an input X comprising a plurality of modality elements (xm^, ni G M-

[0083] This method also includes a PI adjustment phase, prior to phase P2, making it possible to construct the FLTR filtering data.

[0084] These filtering data comprise at least one validity information I^xf) concerning at least one modality element I"', this validity information j allowing the determination, during the filtering F1, whether the non-taking into account of this given modality element X™ improves the precision of the generation of an evaluation element St during the step G1.

[0085] This PI adjustment phase makes it possible to construct filtering data adapted to the model. In particular, the filtering data can be optimized according to the installation location of a complex system comprising sensors and using the MOD evaluation model to control a service according to the evaluation element. In such a case, we speak of fine adjustment, or "fine tuning" in English. The fine adjustment can also be carried out independently of the installation location of the complex system, for example in a laboratory of the model supplier, and simply be carried out with the sensors which will subsequently be installed in this system.

[0086] These FLTR filtering data are here self-sufficient to implement the filtering. As they are constructed upstream of the operating phase P2, the Filtering is quick to implement. Optionally, this FLTR filtering data can be refined during model operation, for example enriched or modified based on user feedback, if such a modification further improves model performance.

[0087] The validity information concerning a modality element Xf may comprise a category to which said modality element belongs. The category is either a so-called valid category, or at least one other category. The other category may be a “neutral” category, i.e. the modality element does not significantly improve the accuracy of the generation of the evaluation, but does not degrade it either. The other category may be “invalid”, meaning that taking this modality element into account degrades the accuracy of the generation of the evaluation by the evaluation model MOD.

[0088] This validity information is not necessarily discrete, and can also take the form of a continuous numerical value. For example, the validity information can be a decimal or real number ranging from +1 to -1, the value +1 being associated with a very strong a priori validity (probable, or with a significant influence), the value -1 with a very strong invalidity and the value zero with a neutral category. The filtering can then be done by threshold, for example valid above 0.3, neutral between 0.3 and - 0.4, and invalid below - 0.4. This allows the filtering to be finely modulated. This threshold modulation can be included in the FLTR filtering data, or be carried out at the FL filtering step. In the latter case, one can choose what degree of filtering is applied to the input X, making the filtering data flexible to use.

[0089] The distinction between neutral (also called non-contributory) and invalid makes it possible to distinguish a "neutral" element that contributes neither positively nor negatively to the evaluation (in other words, plays the role of noise) from an "invalid" element that deteriorates the evaluation.

[0090] Elements of the neutral category may in some cases be taken into account, or not filtered. For example, a modality element may be identified as neutral due to an imprecision in the categorization (valid, neutral, invalid). One may therefore wish not to filter it to avoid this imprecision harming the evaluation. The speed of execution is lower, but makes it possible to avoid a drop in precision compared to the exclusion of neutral modality elements.

[0091] Furthermore, for a simultaneous generation of multiple evaluation elements (each relating to an aspect of a given situation) by the evaluation model, an element can be neutral for the generation of one of these evaluation elements and valid for the generation of another evaluation element. It is then preferable not to filter it if it is "only" neutral for one of the evaluation generations. This allows for a joint generation of evaluation elements by optimizing the execution time and accuracy, rather than having to run two successive generations (each with its own optimal filtering).

[0092] Thus, the filtering criterion, i.e. the result of determining whether ignoring a given modality element X]" improves the accuracy of generating an evaluation element St, may be the membership of said given modality element Xf" to the valid category. It may also be its non-membership to the invalid category - in other words, neutral elements are not filtered.

[0093] In addition to the simplicity and effectiveness of this filtering criterion, it is also very transparent. This makes it possible to verify the quality of the filtering and avoid the "black box" effect. This allows a human to verify the consistency of the filtering data. We can envisage a control phase where a user obtains the validity of its modality elements for a given entry.

[0094] The construction of the filtering data can be based on data, called test data and noted Y. This test data Y comprises at least one pair called test input formed from a set of modality elements, called test, (y"') and an evaluation element, called target, g'.

[0095] Reference is made to Figure 4 to describe in more detail the PI adjustment phase during which the filtering data are constructed. Thus, according to this embodiment, the PI adjustment phase comprises, for at least one modality m: - a generation E2, by the evaluation model MOD and as a function of the plurality of test modality elements ym from which the modality element y™ associated with said modality m is excluded, of an evaluation element, called a test element, 5”, and - an E3 association with said element thus excluded from validity information based on a comparison between the target evaluation element St' and the test evaluation element

[0096] The comparison can be binary, i.e. deliver a "true" or "false" value depending on the equality between 5'' and $ (i.e. the prediction is correct or not).

[0097] The comparison can deliver more detailed information, for example a distance d^SSJ between S't and Sf> or any other similar quantitative or qualitative data. This more detailed information can subsequently undergo processing to discretely categorize the prediction, and therefore the validity or otherwise of the modality element tested. The comparison can be weighted by the confidence of the model in its prediction, for example if the evaluation model delivers, in addition to the evaluation element, a confidence of the model in its prediction.

[0098] For example, the comparison may be a value representative of the similarity between the two target and test evaluation elements, and forming the final validity information. The value can be a probability (between 0 and 1), a correlation indicator (between -1 and 1, like a p-value), ... The fact of not having discrete validity information (valid, neutral, invalid) but a continuous value allows to build more flexible filtering data, because the filtering of a modality element at step Fl can be parameterized with a threshold value adjustable as needed. For example, such filtering can use a criterion of the type a probability of validity of the modality element greater than 0.7 to not ignore.

[0099] In one example, all modalities are tested, that is, all modality elements of a test input are excluded individually and successively so that the validity of each of them can be estimated. Some elements may not be tested, typically to speed up the testing phase if their validity is known a priori. For example, the presence of a road sign on a vehicle may be automatically considered invalid for the detection or not of an anomaly in the inflation of the vehicle's tires or the presence of an oil leak.

[0100] Some modalities may adopt a large or even infinite number of modality elements, so that it is not possible to directly link a value to its category during the test. Put another way, let Dm be the set of all values ​​(more generally modality elements) that a modality m can adopt, called the domain of the modality, it may not be possible to test all these elements, or for all combinations with other possible modalities. In such a case, one can test only a smaller quantity of modality elements (i.e. temperature values), and obtain a partition (in the set-theoretic sense) of the domain of the modality Dm by associating each modality element of the domain with a category. By partition of a set (such as a domain D^), we mean a function Pm which associates with a modality element y"' of this set an element (also called a label, label or category) from a set of categories.Here, the set of categories is a two-element set including at least the valid category. For example, the set of categories can be the set {valid, neutral, invalid}. In other words, the domain Dm of a modality m is partitioned into subsets, each of which is associated with a category. We can generalize the notion of partition as a function whose arrival set is the set of validity information, in the case where we do not use a category as validity information.

[0101] For example, we consider a temperature sensor that delivers a value (the modality element) of temperature (the modality), installed in a kitchen and which can range from - 50°C to 100°C in increments of 0.05°C (the domain of the modality) and the evaluation of a “probability of a fire”. The filtering implemented by the method of the invention can consider that the values ​​of this sensor are: - invalid between -50°C and 10°C, because, considering that no dwelling can be colder than 10°C, it is plausible that the sensor is malfunctioning, - neutral between 10°C and 28°C, because this range of values ​​corresponds to normal temperatures in a home, in which case it is not the most relevant indicator for detecting a fire, unlike smoke detection), - valid between 28°C and 70°C, because the probability of a fire is very correlated with the value of this sensor in this range of values, and - invalid above 70°C, because in this case, either the sensor is malfunctioning, or the fire is such that other sensors are just as indicative of the fire, so that it is not necessary to take this modality element into account.

[0102] In the PI adjustment phase, during steps E2 and E3, the values ​​are tested in 2°C increments, and a partition of the domain Dm is carried out, for example ([-50, 10], invalid) ([10, 28], neutral), ([28, 70], valid), and ([70, 100], invalid). In the P2 operating phase, any modality element even if not tested during the adjustment phase (for example 23.2°C or - 12.3°C) can therefore be categorized and therefore filtered.

[0103] In another example, a sensor (e.g. door opening) may return a value ON (door open), OFF (door closed) and ERR7 (error message). The domain of the “door opening” modality is then {ON, OFF, ERR7], and may for example be partitioned for the “fire risk” assessment element into {ON, OFF} in neutral modality and {ERR7} for the invalid category. This example is, again, for illustrative purposes.

[0104] It is thus made possible to precisely filter all possible modality elements for a modality without having to test them all.

[0105] Furthermore, the validity information (for example the category) concerning an element of modality X™ of a modality m can be correlated to at least one other element of modality x"1 associated with another modality ni'. We then speak of conditional, or correlated, validity information. We can correlate two values ​​(for example temperature and humidity level), or two categories (that of each of the two elements), or a value and a category. We can envisage correlations with more than two elements.

[0106] It is possible, in order to construct filtering data comprising correlated validity information, to test all possible combinations of modality elements, or to explore all combinations in a statistical or probabilistic manner, for example simple average methods, the use of theories extending probabilities such as the Dempster-Shafer theory, etc.

[0107] The notion of correlated validity is particularly important for the evaluation of neutral data. Indeed, two elements of modalities can be individually neutral, because their individual absence, tested successively, has no impact on the prediction of the situation, but valid when both are absent at the same time. This situation therefore means that these two data from different modalities provide the same information and that if they are not taken into account at all, this has an impact on the prediction of the situation. It is therefore important in this case that at least one is considered valid, hence the interest in correlated validity information.

[0108] To take the example above, the values ​​10°C and 28°C can be variable according to a modality relating to the season (for example a date) or to a temperature reading by a meteorological service (i.e. a modality obtained via an internet connection). Indeed, if the temperature of 28°C is a temperature beyond which the probability of fire is high in winter (few being users heating their homes beyond this), this is not the case in summer, in which case this “threshold” temperature above which the category of the modality goes from neutral to valid can be raised from 28°C (winter) to 35°C (summer) during the months of June, July and August.

[0109] Similarly and still in this example, since a fire dries the atmosphere very strongly, the validity of the “kitchen temperature” modality above 40°C can depend on the value of a modality relating to humidity, and with the adjustment phase, it is possible to consider that the temperature modality is invalid in the 40-70°C range if the humidity modality is in the 80-100% humidity range, and valid in the 0-10% range.

[0110] Other examples can be considered, for example for the evaluation "probability that a driver falls asleep at the wheel of his vehicle", which if it exceeds a certain value (for example 0.7), causes an alert signal within the vehicle to wake the driver from his torpor. Here, the category of the modality "centering of the vehicle on the road" can be conditioned by the value of the modality "heart rate" measured by a watch connected to the driver's wrist. Indeed, the heart rate drops in the event of falling asleep, but remains very high in the event of a stressful situation, for example a sudden maneuver to avoid an object on the road, a person, an animal or another vehicle with erratic behavior.Filtering here allows the evaluation model to reliably detect drowsiness while avoiding false positives which would cause undue and unpleasant alarm triggers (or even dangerous since an undue audible alarm in the middle of a stressful driving situation amplifies the stress).

[0111] The modality elements not tested in the PI adjustment phase can be automatically (i.e. by default) associated with given validity information, for example valid or neutral.

[0112] Optionally, before steps E2 and E3 of the PI adjustment phase, the method may comprise a step E1 of testing the test data. In such a test step, each target evaluation element $' is compared to the generated evaluation element 5" with the associated test input, so as to estimate the validity of the test data. Said otherwise, this step E1 allows to increase the confidence in the filtering data constructed subsequently in steps E2 and E3, by avoiding taking into account one of the test data / / y \ which would be clearly inconsistent with what the V? ' m&M / predicted model. This also allows us to identify model malfunctions upstream, if on the contrary it is the test data in which we have the most confidence.

[0113] Finally, in relation to [Fig.5], a simplified structure of an electronic evaluation generation device according to an embodiment of the invention is presented.

[0114] As illustrated in [Fig.5], the electronic device for generating an evaluation of a situation according to an embodiment of the invention comprises a memory M, a processing unit, equipped for example with a programmable computing machine or a dedicated computing machine, for example a processor P, and controlled by a computer program Pg, implementing steps of a method for generating an evaluation of a situation as described above.

[0115] At initialization, the code instructions of the computer program Pg are for example loaded into a RAM memory before being executed by the processor of the processing unit P.

[0116] The processor of the processing unit P implements steps of the method for generating an evaluation of a situation described previously, according to the instructions of the computer program Pg, to:

[0117] - implement an evaluation model MOD, configured to receive a game X of modality elements, called input, X™, from sensors equipping an environment and / or a user, a modality element being associated with a modality m and being representative of a state of the modality m; and to generate, as a function of the input modality elements, at least one evaluation element, called output St, representative of at least one given aspect of a situation associated with the input modality elements E,

[0118] said evaluation element making it possible, in an operating phase, to control a triggering of a service adapted to the environment and / or to the user;

[0119] said processor being further configured to, during the operating phase and prior to the generation of a given evaluation element St,

[0120] - determine that not taking into account a given modality element improves the accuracy of generating an evaluation element St, and

[0121] - not taking into account said modality element X"' in the generation of said St. assessment element

[0122] A few examples of modalities have been seen through the various examples above. These examples of modalities are of course not exhaustive, as they can be very varied and of various types: visual modalities (e.g.: characteristics of the face, body, movements, environment, etc.), sound modalities (e.g.: ambient sound, exchanged words, etc.), textual modalities (e.g.: transcription of speech, contextual information, etc.), physiological modalities (e.g.: electrocardiogram, respiratory rate, etc.). Some modalities may not come from sensors.

[0123] A three-element categorization has also been seen, but other categorizations can be envisaged. For example, to determine the dangerousness of a situation by a supervision system implementing the method for generating an evaluation of a situation described above, the multimodal values ​​can be judged crucial (their presence increases performance significantly), optional (their presence increases performance marginally), neutral (their presence does not influence performance) or invalid (their presence negatively influences performance).

Claims

Claims

1. Method for generating an evaluation of a situation by an evaluation model (MOD), the method being implemented by an electronic device, the evaluation model (MOD) being configured to receive a set (X) of modality elements, called input, (X^), from sensors equipping an environment and / or a user, a modality element being associated with a modality (m) and being representative of a state of the modality (m);and to generate, as a function of the input modality elements, at least one evaluation element, called output (5r), representative of at least one given aspect of a situation associated with the input modality elements (X), said evaluation element making it possible, in an operating phase, to control a triggering of a service adapted to the environment and / or to the user, said method comprising, during the operating phase and prior to the generation of a given evaluation element (S,), a filtering comprising the determination that the non-taking into account of a given modality element (X™) improves the precision of the generation of an evaluation element (5'f), and a non-taking into account of said modality element (x“) in the generation of said evaluation element (;

2. Method for generating an evaluation of a situation by an evaluation model (MOD) according to claim 1, comprising an adjustment phase during which filtering data are constructed, the filtering data comprising at least one validity information concerning at least one modality element (xf) allowing the determination during the filtering that the non-taking into account of this given modality element (X™) improves the precision of the generation of an evaluation element (¾.

3. Method for generating an evaluation by an evaluation model (MOD) according to claim 2, in which said validity information concerning at least one modality element (X^) comprises a category to which said modality element belongs, the category being part of the set of categories comprising a so-called valid category and at least one other category.

4. Method of generating an evaluation by an evaluation model (MOD) according to claim 3, wherein said set of categories comprises at least one so-called neutral category and one so-called invalid category.

5. A method of generating an evaluation by an evaluation model (MOD) according to claim 3 or 4, wherein determining that ignoring a given modality element (X'tn) improves the accuracy of generating an evaluation element (St) comprises determining whether said given modality element (Z™) belongs to the valid category.

6. Method for generating an evaluation by an evaluation model (MOD) according to one of claims 2 to 5, in which the construction of the filtering data is based on data, called test data, (K) comprising at least one pair formed from a set of modality elements, called test, (V"') and an evaluation element, called target, (5').

7. Method for generating an evaluation by an evaluation model (MOD) according to claim 6, in which the construction of the filtering data comprises, for at least one modality (mj, a generation, by the evaluation model (MOD) and as a function of the plurality of test modality elements ( / ") by excluding from said generation the modality element associated with said modality (my of an evaluation element, called test, (-^'r), and an association with said element (y™) thus excluded from validity information as a function of a comparison between the target evaluation element (St') and the test evaluation element (5p.

8. Method for generating an evaluation by an evaluation model (MOD) according to one of claims 3 to 7, in which at least one modality defines a set (Dm) formed of all or part of the modality elements associated with this modality, the filtering data comprising for said modality (m) information representative of a partition of said set (D^ associated with said modality (m), into subsets each associated with one of the categories of the set of categories.

9. Method for generating an evaluation by an evaluation model (MOD) according to one of claims 2 to 8, in which the at least one validity information concerning at least one modality element (X™) concerns at least one second modality element (xf) relating to another modality (m'y

10. Method for generating an evaluation by an evaluation model (MOD) according to one of the preceding claims, in which the evaluation model is robust in the event of missing data.

11. Electronic device for generating an evaluation of a situation by an evaluation model (MOD), the device comprising a processor configured to: - implement said evaluation model (MOD), configured to receive a set (X) of modality elements, called input, (^), from sensors equipping an environment and / or a user, a modality element being associated with a modality (m) and being representative of a state of the modality (m); and to generate, as a function of the input modality elements, at least one evaluation element, called output (Sf), representative of at least one given aspect of a situation associated with the input modality elements (X), said evaluation element making it possible, in an operating phase, to control a triggering of a service adapted to the environment and / or to the user;said processor being further configured to, during the operating phase and prior to the generation of a given evaluation element (5f), - determine that not taking into account a given modality element (X[”) improves the precision of the generation of an evaluation element (St), and - not take into account said modality element (^"') in the generation of said evaluation element (St).;

12. Computer program product comprising instructions for implementing a method according to claims 1 to 10, when this program is executed by a processor.

Images