Improved method for generating an assessment

Abstract

Disclosed is a method for generating an assessment of a situation, using an assessment model (MOD). The assessment model is configured to receive an input comprising modality elements (Formula (I)) delivered by sensors with which an environment and / or a user are / is equipped. The model is also configured to generate, depending on the input, an output assessment element (S t) representative of a given aspect of a situation associated with the input modality elements (X). The assessment element makes it possible to control triggering of a service suitable for the environment and / or user. The method comprises filtering, prior to generation of a given assessment element (S t). The filtering comprises determining whether not taking a given modality element (Formula (I)) into account improves the accuracy of the generation, and ignoring the modality element (Formula (I)) when generating the assessment element (S t) in case of a positive response.

Description

[0001] DESCRIPTION

[0002] Title: Improved method for generating an evaluation

[0003] 1. Field of the invention

[0004] The invention relates to the field of data analysis, and more particularly to the processing of data from sensors equipping an environment and / or a user to assess the extent to which a particular situation occurs.

[0005] 2. Prior art

[0006] The processing of data from sensors is a rapidly expanding field, driven by the rise of the connected home (or "smart home"), but more generally by connected buildings (hospital services, nursing homes, etc.) and complex systems incorporating a large number of sensors (connected and particularly autonomous vehicles, aircraft, etc.).

[0007] These systems have in common the implementation of a large number of sensors of different types: thermometer, LIDAR, radar, camera, hygrometer, smoke detector, biometric sensors, microphone, physiological sensors on a connected object (or "wearable" in English), and more generally connected objects (or loT, for Internet of Things). These connected objects and sensors make it possible to extract a large number of very heterogeneous measurements (speed, position, temperature, humidity, presence of smoke, electrocardiogram, respiratory rate, etc.). This extraction may require the analysis of complex data (image and / or sound processing, semantic analysis) to extract more useful information from this data (emotional state of a user, vocal or gestural instruction, speed of closing a door, meaning of a traffic sign visible by a camera, etc.).) than the raw material they form (audio or video signal, text, multimedia or textual data more generally, ...).

[0008] For the remainder of this application, all such data types will be referred to as modalities, with a modality corresponding to a type of data captured by an electronic data capture device (typically, but not exclusively, a sensor).

[0009] These data, or modalities, are very heterogeneous, and their joint analysis is a field in its own right called multimodal analysis. Multimodal analysis is a recent discipline whose objective is to extract, i.e. predict or generate, from multimodal data (i.e. data of different types from different sensors) relevant indicators relating to a given situation. Multimodal learning allows for much better performance in the analysis of a complex situation than its monomodal counterpart, hence its growing popularity, particularly in the aforementioned complex systems (connected building, autonomous vehicle, etc.).

[0010] This multimodal analysis is carried out using multimodal learning. The product of this multimodal learning is a multimodal model capable of processing a plurality of modalities from which the model can extract indicators relating to a given situation.

[0011] However, traditional multimodal models have a number of drawbacks. On the one hand, some modalities may be very relevant to the multimodal model's assessment of a given situation (e.g., a person's emotional state, or the probability of a fire in a home), while other modalities have only a marginal influence and distort the assessment. Worse still, some modalities impair the prediction. In the following, we will refer to the contribution of a modality to an assessment. This contribution may, for example, be positive (the data is crucial for assessing a situation), weak (the data is optional, i.e. marginally benefits the assessment), zero (taking this modality into account or not is neutral for the final assessment), or negative (taking this modality into account harms the final assessment).It is understood that this categorization is only an example, and that the qualification of a modality's contribution to an evaluation can be more rigid (for example binary) or on the contrary more continuous (forming a continuum ranging from crucial to harmful).

[0012] Furthermore, some modalities may contribute positively to an evaluation when they take on a certain value, and deteriorate it with another value. Here again, known multimodal models are unable to discriminate modalities according to their value and their contribution to an evaluation.

[0013] More generally, these complex systems face a dual problem of managing trust in data sources and the presence of uncertainty in the information. Several approaches have attempted to overcome this dual problem.

[0014] The article M. Ravi, Y. Demazeau and F. Ramparany, "Reasoning with Trust and Uncertainty Illustration in the Internet of Things," 2015 IEEE / WIC / ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), Singapore, 2015, pp. 125-128, doi: 10.1109 / WI-IAT.2015.174, describes a modeling, by an ATMS (Assumption-based Truth Maintenance System) inference engine, of the trust in a source in its ability to perform or delegate a certain task. The source is then judged reliable or not. However, this modeling characterizes the source as such, or formulated differently, the modality associated with the source as a whole, and not the reliability of a modality according to the values ​​it can take.

[0015] The article VS Simankov, VV Buchatskaya, PY Buchatskiy and SV Teploukhov, "Classification of information's uncertainty in system research," 2017 XX IEEE International Conference on Soft Computing and Measurements (SCM), St. Petersburg, Russia, 2017, pp. 187-189, doi: 10.1109 / SCM.2017.7970534, proposes a classification according to the degree of uncertainty in an information source, among the following types of uncertainty: total certainty, stochastic uncertainty and fuzzy uncertainty. However, the type of information uncertainty described therein depends on the input, and has a significant influence on the choice of the mathematical method for modeling the considered process. In fact, the analysis described in this article is agnostic to the functionality targeted by the system, in other words does not depend on the nature of the assessment of a situation.

[0016] Furthermore, traditional multimodal models only work with complete data, meaning that all data must be present as input for the model to extract a specific indicator. However, in practice, it is common for some data to be missing. This is typically the case for complex systems with heterogeneous sensors arranged in various locations. Each sensor can fail or malfunction. A simple failure of a single sensor can then cause the entire model to seize up. The risk of failure increases, in fact, with the complexity of the system. The problem is similar in the event of a communication failure, even temporary, between the sensor and the rest of the components of the complex system. Other events, such as changing the location of the sensor, are also likely to cause paralysis of the multimodal model during this sensor movement.

[0017] Thus, none of the approaches so far have been satisfactory, due to a sorting that is too crude, or too general and poorly adapted to a concrete reality. The invention improves the situation. 3. Presentation of the invention

[0018] The invention relates to a method for generating an evaluation of a situation of a new type, not having the aforementioned disadvantages. More specifically, the invention proposes a method for generating an evaluation of a situation by an evaluation model, the method being implemented by an electronic device, the evaluation model being configured to receive a set of modality elements, called input, from sensors equipping an environment and / or a user, a modality element being associated with a modality and being representative of a state of the modality;and to generate, as a function of the input modality elements, at least one evaluation element, called output, representative of at least one given aspect of a situation associated with the input modality elements, said evaluation element making it possible, in an operating phase, to control a triggering of a service adapted to the environment and / or to the user, said method comprising, during the operating phase and prior to the generation of a given evaluation element, a filtering comprising the determination that the non-taking into account of a given modality element improves the precision of the generation of an evaluation element, and a non-taking into account of said modality element in the generation of said evaluation element.;

[0019] Thanks to this generation process, it is thus possible to improve the precision of an evaluation model, by discarding a modality when its element (for example its value) does not contribute positively to the generation of an evaluation element (also called simply evaluation, or prediction), or in other words does not improve the quality of the prediction of the evaluation model.

[0020] Filtering is performed a priori, that is, before the actual evaluation. The advantage is twofold, because this filtering can be implemented upstream of any multimodal model, and allows for computational savings by performing, for a given situation, only a single evaluation, having filtered optimally beforehand.

[0021] This filtering is further optimized—it improves the quality of the model's prediction—and at fine granularity, i.e. modality by modality and element by element. Indeed, each element (for example the value) that a modality can adopt is considered individually, independently of the other elements associated with this modality. A modality can thus at times be taken into account, and at other times not, dynamically. This consideration or non-consideration is carried out for a given evaluation, and can vary if several evaluations are carried out at the same time, which differs greatly from the prior art. Finally, this method makes it possible to automatically detect anomalies and automatically exclude a modality when its element is clearly outside the values ​​that it should usually take.

[0022] A modality element is a value, a piece of data, a signal, etc. that a modality can adopt. For a modality associated with a sensor (temperature, pressure, humidity, etc.), the modality element can be a measured value. For a camera embedded in a vehicle, the associated modalities can be data on an evaluation panel, a representation of the position of the road relative to the vehicle, etc. For a rotation sensor housed in a door, the associated modality can be a door closing speed, and so on.

[0023] According to a particular aspect, the method for generating an assessment of a situation by an assessment model comprises an adjustment phase during which filtering data are constructed, the filtering data comprising at least one validity information concerning at least one modality element allowing the determination during the filtering that not taking into account this given modality element improves the precision of the generation of an assessment element.

[0024] The adjustment phase allows, prior to the model operation phase, to construct filtering data adapted to the model. In particular, the filtering data can be constructed and therefore optimized according to the installation location of a complex system comprising sensors and using the evaluation model to control a service according to the evaluation element. These data alone being sufficient to carry out the filtering, and being constructed upstream of the model operation phase, the filtering is quick to implement.

[0025] According to a particular aspect, said validity information concerning at least one modality element comprises a category to which said modality element belongs, the category being part of the set of categories comprising a so-called valid category and at least one other category.

[0026] This categorization simplifies the implementation of filtering, since it is sufficient to determine the category of a modality element in order not to take it into account. According to a particular aspect, said set of categories comprises at least one so-called neutral category and one so-called invalid category.

[0027] This separation between a neutral category (also called non-contributory) and an invalid category makes it possible to distinguish the modality elements of the neutral category, which contribute neither positively nor negatively to the evaluation (in other words, which play the role of noise) from the elements of the invalid category which deteriorate the evaluation.

[0028] In some cases, it may indeed be more efficient to take into account elements of the neutral category, because not taking them into account would decrease the accuracy of the evaluation model. Indeed, a modality element may be classified as neutral due to an imprecision in the categorization (valid, neutral, invalid). In fact, we may want to take it into account to prevent this imprecision of categorization from harming the correct prediction. Thus, at the cost of a reduced execution speed, this consideration of neutral elements makes it possible to avoid a decrease in accuracy.

[0029] In other cases, ignoring neutral modality elements can speed up the evaluation model, as can ignoring invalid modality elements.

[0030] Finally, in the case of simultaneous prediction of multiple assessment elements (each relating to an aspect of a given situation) by the assessment model, an element may be neutral for the generation of one of these assessment elements and valid for the generation of another assessment element. It is then preferable to take this into account even if this modality element is "only" neutral for the generation of the first assessment element. This allows for a joint generation of assessment elements by optimizing execution time and accuracy, rather than having to execute two successive generations (each with its own filtering).

[0031] In a particular aspect, determining that ignoring a given modality element improves the accuracy of generating an assessment element includes determining whether said given modality element belongs to the valid category.

[0032] The filtering is thus transparent, since it is possible for a modality element to easily check whether it is taken into account or not according to its category. This makes it possible to explore the filtering data and verify, for a human in particular, whether this data is consistent or not, by testing a given modality element. According to a particular aspect, the construction of the filtering data is based on data, called test data, comprising at least one pair formed of a set of modality elements, called test, and an evaluation element, called target.

[0033] With this test data, it is thus not only possible to construct the filtering data, but also to test the accuracy and reliability of the model independently of whether or not modality element(s) are taken into account. Indeed, it is possible for a set of test modality elements to verify whether the model indeed predicts a target evaluation element, and with what accuracy.

[0034] According to a particular aspect, the construction of the filtering data comprises, for at least one modality, a generation, by the evaluation model and as a function of the plurality of test modality elements by excluding from said generation the modality element associated with said modality, of an evaluation element, called a test element, and an association of said element thus excluded with validity information as a function of a comparison between the target evaluation element and the test evaluation element.

[0035] Thanks to this comparison by exclusion, we can thus construct filtering data which make it possible to test for one or more modality elements which should not be taken into account.

[0036] This construction can be exhaustive (i.e. all combinations of modality element exclusion), thus covering all use cases of the complex system implementing the evaluation model.

[0037] According to a particular aspect, at least one modality defines a set formed of all or part of the modality elements associated with this modality, the filtering data comprising for said modality information representative of a partition of said set associated with said modality, into subsets each associated with one of the categories of the set of categories.

[0038] Thus, for a modality whose elements belong to a domain of the modality (for example a temperature range that a temperature sensor can deliver), it is possible, thanks to this partition, to determine for any modality element, including a modality element that was not used during the filtering data construction phase, whether this modality element should be disregarded. To do this, it is determined to which subset of the domain this modality element belongs, and what the category of the subdomain is.

[0039] An exhaustive partition of the domain of a modality can thus be carried out, allowing for a small amount of information to have exhaustive information on the modality.

[0040] According to a particular aspect, the at least one validity information concerning at least one modality element concerns at least one second modality element relating to another modality.

[0041] Thanks to this correlation between two elements of two modalities, it is possible to obtain a finer categorization, and therefore more optimal for a given situation.

[0042] According to one aspect of the invention, the evaluation model is a model robust to missing input data.

[0043] The invention also relates to an electronic device for generating an evaluation of a situation by an evaluation model, the device comprising a processor configured to: implement said evaluation model, configured to receive a set of modality elements, called input, from sensors equipping an environment and / or a user, a modality element being associated with a modality and being representative of a state of the modality; and to generate, as a function of the input modality elements, at least one evaluation element, called output, representative of at least one given aspect of a situation associated with the input modality elements, said evaluation element making it possible, in an operating phase, to control a triggering of a service adapted to the environment and / or to the user;said processor being further configured to, during the operating phase and prior to the generation of a given evaluation element, determine that not taking into account a given modality element improves the precision of the generation of an evaluation element, and not take into account said modality element in the generation of said evaluation element.;

[0044] The invention also relates to a computer program product comprising instructions for implementing the above-mentioned method, when this program is executed by a processor. 4. List of figures

[0045] The proposed technique, as well as the various advantages it presents, will be more easily understood, in the light of the following description of illustrative and non-limiting embodiments thereof, and the appended drawings among which:

[0046] [Fig. 1] represents an evaluation model as employed in the invention,

[0047] [Fig. 2] schematizes the general principle of the invention,

[0048] [Fig. 3] represents a generation method according to an embodiment of the invention,

[0049] [Fig- 4] represents an example of implementation of an adjustment phase of the process of figure 3, and

[0050] [Fig. 5] represents a device according to the invention.

[0051] 5. Detailed description

[0052] The invention is based on a multimodal evaluation model.

[0053] The general principle of the invention is to filter, upstream of the execution of an evaluation model of this type, a modality according to its current value (more generally of its modality element), when such filtering allows a priori (i.e. before the evaluation by the model itself) to improve the performance of the model. The performance concerns a gain in execution speed (since the filtered modality is ignored for the current prediction) and / or in precision (because the filtered modality degrades the precision of the model if it is taken into account, for the current modality element).

[0054] It should be noted that the filter depends not on the modality as such, but on its current value (more generally on its current modality element). Thus, for a given modality, the filtering can vary between taking it into account and not taking it into account depending on the current modality element, that is to say temporally.

[0055] The advantage of a priori filtering, i.e. before running the evaluation model, is that it allows its performance to be optimized while only running it once.

[0056] Such a model is represented in Figure 1. The evaluation model referred to as MOD can be of the type robust in case of missing data. The evaluation model MOD is configured to receive a set X of modality elements, called input, x™, a modality element being associated with a modality m with m GM = ... , m N where M is a set of incoming modalities. These modality elements x™ 1 are grouped in the form of a model entry noted X = is an entity er > > c ' between 1 and N).

[0057] These modality elements come from sensors equipping an environment and / or a user. Each modality element x™ is representative of a state of the modality m, or in other words a measurement from a sensor, directly or indirectly. By indirect measurement, we mean a measurement obtained following data processing applied to a raw measurement from the sensor, for example an image analysis to extract, from a video stream produced by a camera on board a vehicle, data such as recognition of a traffic sign or the position of the vehicle on a road.

[0058] From this input X, the evaluation model can generate at least one evaluation element, called output and noted S t . This output assessment element S t is representative of at least one given aspect of a situation associated with input X. This evaluation element is also called a “class” predicted by the evaluation model.

[0059] The generation of an evaluation is repeated, at different times. It can be implemented at regular intervals, for example every second, every minute or every hour. This generation allows monitoring of the complex system by the evaluation model, and the triggering of a service when the evaluation takes certain values ​​from the model output.

[0060] A modality element x™ is a representation, at a given time t, of a modality m. For this modality m, we call the ordered series of modality elements over time a temporal sequence of the modality, denoted x m = We understand that the index t in fact identifies a given instant t. Similarly, we call the series of evaluation elements over time the temporal sequence of evaluation, noted S = (S t ) t. We can generalize the operation of the evaluation model by the fact that it generates a temporal sequence of evaluation as a function of the temporal sequences of modalities.

[0061] By given situation, we mean the state of a complex system (housing, building, vehicle, etc.) as a whole, this situation being apprehended by the evaluation model using the modalities at its disposal. By aspect of a given situation, we mean one of the characteristics to describe this situation, a situation can be described by several aspects. For example, for a nursing home, which therefore presents a given situation at a given time, one aspect can be the probability of fire, another aspect can be the need to turn on an air conditioner or heating, yet another aspect can be an indicator of the state of one of the residents (emotional state, physical state, etc.).

[0062] Although the evaluation model is subsequently described as generating one evaluation element, this evaluation model is capable of simultaneously generating several evaluation elements, each relating to a given aspect of the state of the system considered.

[0063] The S assessment element t allows you to trigger or control a service adapted to the environment and / or the user depending on the status of the aspect of the situation being assessed. For example, if a fire is detected, an audible alarm is triggered immediately, fire doors are closed, a notification is sent to a user (for example on a connected bracelet or any other wearable device) and an alert for the appropriate emergency services is issued.

[0064] As explained above, the model is able to function even in the absence of a modality element, in particular without modification or retraining of the model. Such a model can for example be based on an encoder / decoder approach based on Transformers. Such a model can be trained in a supervised manner on a labeled training dataset comprising all the modalities [m , ..., m N} used as input to the evaluation model, for the recognition of a given aspect of a given situation.

[0065] Reference is now made to Figure 2 to describe the general principle of the invention. In this figure, upstream of the evaluation model MOD, a filter F is provided. Formulated in temporal terms, filtering is implemented before the generation of an evaluation element.

[0066] Filtering includes: determining whether to disregard a modality element %” 1' given improves the accuracy of generating an evaluation element S t , and in case of a positive response, ignore (i.e. do not take into account) the said modality element %” 1 ' in the generation of the evaluation element S t .

[0067] In the example shown in Figure 2, it is the modality element %” 1 ' associated with modality m, which is filtered, i.e. ignored. Multiple elements can be filtered at the same time. When a modality element is filtered, the effect is the same for the evaluation model as if that modality element were absent.

[0068] Such filtering by modality element improves the performance (accuracy and execution speed) of the evaluation model. Indeed, discarding a modality when its modality element (e.g. its value) does not contribute positively to the generation of an evaluation element not only makes the model faster to execute (unnecessary data are not used), but also more accurate (data that harms the accuracy of the evaluation are filtered).

[0069] This filtering is all the more interesting because it is carried out a priori, that is to say before the evaluation itself. This filtering is thus compatible with any multimodal model of the type capable of functioning in the event of a missing modality. This filter saves computation time, since a single generation of evaluation is carried out for a given input, while having relative confidence in the optimality of the filtering.

[0070] This filtering is fine-grained, that is, it processes modality by modality and, for the same modality, modality element by modality element. Each element that a modality can adopt is considered individually, independently of the other elements associated with this modality. A modality can thus at times be taken into account, and at other times not, dynamically depending on the input flow X. This filtering depends on the evaluation considered. It can in fact vary if several evaluations are generated at the same time, which differs greatly from the prior art.

[0071] Finally, this filtering allows for the automatic detection of anomalies and the automatic exclusion of a modality when its element is clearly outside the values ​​it should usually take. In such a case, it is then detected that the filtering of this modality element improves the accuracy of the model. This filtering also makes it possible to avoid paralysis of the evaluation model in the event of movement or replacement of a sensor, as well as in the event of a malfunction.

[0072] Reference is made to Figure 3 to describe in more detail the main steps of the method for generating an evaluation of a situation, according to one embodiment of the invention. A method for generating an evaluation comprises a filtering step F1 of the type described above, and a step G1 for generating an evaluation element by the evaluation model MOD. These two steps F1 and G1 are executed during a phase P2 called the operating phase, from an input X comprising a plurality of modality elements (%” 1 ), m G M.

[0073] This method also includes a PI adjustment phase, prior to phase P2, allowing the construction of the FLTR filtering data. This filtering data includes at least one validity information I v (x™') concerning at least one element of modality x™, this validity information / v (x” 1) allowing the determination, during the filtering Fl, whether the non-consideration of this given modality element x™ improves the precision of the generation of an evaluation element S t during stage Gl.

[0074] This PI adjustment phase allows the construction of filter data adapted to the model. In particular, the filter data can be optimized according to the installation location of a complex system comprising sensors and using the MOD evaluation model to control a service according to the evaluation element. In such a case, we speak of fine tuning. Fine tuning can also be carried out independently of the installation location of the complex system, for example in a laboratory of the model supplier, and simply be carried out with the sensors that will subsequently be installed in this system.

[0075] These FLTR filtering data are self-sufficient to implement the filtering. As they are built upstream of the P2 operation phase, the filtering is quick to implement. Optionally, these FLTR filtering data can be refined during the model operation, for example enriched or modified based on user feedback, if such a modification further improves the model's performance.

[0076] Validity information / v (x” 1) concerning a modality element x™ may comprise a category to which said modality element belongs. The category is either a so-called valid category, or at least one other category. The other category may be a "neutral" category, i.e. the modality element does not significantly improve the accuracy of the generation of the evaluation, but does not degrade it either. The other category may be "invalid", meaning that taking into account this modality element degrades the accuracy of the generation of the evaluation by the MOD evaluation model.

[0077] This validity information / v (x” 1 ) is not P asnecessarily discrete, and can also take the form of a continuous numerical value. For example, the validity information can be a decimal or real number ranging from +1 to -1, the value +1 being associated with a very strong a priori validity (probable, or with a significant influence), the value -1 with a very strong invalidity and the value zero with a neutral category. Filtering can then be done by threshold, for example valid above 0.3, neutral between 0.3 and - 0.4, and invalid below - 0.4. This allows fine modulation of the filtering. This modulation by threshold can be included in the FLTR filtering data, or be carried out at the Fl filtering step. In the latter case, one can choose what degree of filtering is applied to the input X, making the filtering data flexible in use.

[0078] The distinction between neutral (also called non-contributory) and invalid makes it possible to distinguish a "neutral" element that contributes neither positively nor negatively to the evaluation (in other words, plays the role of noise) from an "invalid" element that deteriorates the evaluation.

[0079] Elements in the neutral category may in some cases be considered, or not filtered out. For example, a modality element may be identified as neutral due to an imprecision in the categorization (valid, neutral, invalid). We may therefore wish not to filter it to avoid this imprecision harming the evaluation. The execution speed is lower, but allows to avoid a decrease in precision compared to the exclusion of neutral modality elements.

[0080] Furthermore, for a simultaneous generation of multiple assessment elements (each relating to an aspect of a given situation) by the assessment model, an element may be neutral for the generation of one of these assessment elements and valid for the generation of another assessment element. It is then preferable not to filter it if it is "only" neutral for one of the assessment generations. This allows for a joint generation of assessment elements by optimizing execution time and accuracy, rather than having to execute two successive generations (each with its own optimal filtering).

[0081] Thus, the filtering criterion, i.e. the result of determining whether ignoring a given modality element x™ improves the accuracy of generating an evaluation element S tcan be the membership of said given modality element x™ to the valid category. It can also be its non-membership to the invalid category — in other words, neutral elements are not filtered.

[0082] In addition to the simplicity and effectiveness of this filtering criterion, it is also very transparent. This makes it possible to verify the quality of the filtering and avoid the "black box" effect. This allows a human to verify the consistency of the filtering data. We can consider a control phase where a user obtains the validity of its modality elements for a given input. The construction of the filtering data can be based on data, called test data and noted Y. This test data Y includes at least one pair called test input formed by a set of modality elements, called test, and an evaluation element, called target, S^.

[0083] Reference is made to Figure 4 to describe in more detail the PI adjustment phase during which the filtering data are constructed. Thus, according to this embodiment, the PI adjustment phase comprises, for at least one modality m, a generation E2, by the evaluation model MOD and depending on the plurality of test modality elements y m from which the modality element y™ associated with said modality m is excluded, from an evaluation element, called a test element, S", and an association E3 with said element y™ thus excluded from validity information based on a comparison between the target evaluation element S t ' and the test assessment element S".

[0084] The comparison can be binary, i.e. deliver a "true" or "false" value depending on the equality between S" and (i.e. the prediction is correct or not).

[0085] The comparison can provide more detailed information, for example a distance between S" and S t ', or any other similar quantitative or qualitative data. This more detailed information can subsequently be processed to discretely categorize the prediction, and therefore the validity or otherwise of the modality element tested. The comparison can be weighted by the model's confidence in its prediction, for example if the evaluation model delivers, in addition to the evaluation element, a model's confidence in its prediction.

[0086] For example, the comparison can be a value representing the similarity between the two target and test evaluation elements, and forming the final validity information. The value can be a probability (between 0 and 1), a correlation indicator (between -1 and 1, like a p-value), ... The fact of not having discrete validity information (valid, neutral, invalid) but a continuous value allows to build more flexible filtering data, because the filtering of a modality element at step Fl can be parameterized with a threshold value adjustable as needed. For example, such filtering can use a criterion of the type a validity probability of the modality element greater than 0.7 to not ignore.

[0087] In one example, all modalities are tested, that is, all modality elements of a test input are excluded individually and successively so that the validity of each of them can be estimated. Some elements may not be tested, typically to speed up the testing phase if their validity is known a priori. For example, the presence of a traffic sign on a vehicle may be automatically considered invalid for the detection or not of an anomaly in the inflation of the vehicle's tires or the presence of an oil leak.

[0088] Some modalities may adopt a large or even infinite number of modality elements, so that it is not possible to directly link a value to its category during the test. Put another way, let D mthe set of all values ​​(more generally modality elements) that a modality m can adopt, called the domain of the modality, it may not be possible to test all these elements, or for all combinations with other possible modalities. In such a case, one can test only a smaller quantity of modality elements (i.e. temperature values), and obtain a partition (in the set-theoretic sense) of the domain of the modality D m by associating each modality element of the domain with a category. By partitioning a set (like a domain D m ), we hear a function P m which associates with an element of modality y mof this set an element (also called a label, tag, or category) from a set of categories. Here, the set of categories is a two-element set including at least the valid category. For example, the set of categories can be the set {valid, neutral, not — valid}. Stated another way, the domain D m of a modality m is partitioned into subsets, each of which is associated with a category. We can generalize the notion of partition as a function whose arrival set is the set of validity information, in the case where we do not use a category as validity information.

[0089] For example, we consider a temperature sensor that delivers a value (the modality element) of temperature (the modality), installed in a kitchen and which can range from - 50°C to 100°C in increments of 0.05°C (the modality domain) and the evaluation of a “probability of a fire”.The filtering implemented by the method of the invention may consider that the values ​​of this sensor are: invalid between - 50°C and 10°C, because, considering that no home can be colder than 10°C, it is plausible that the sensor malfunctions, neutral between 10°C and 28°C, because this range of values ​​corresponds to normal temperatures in a home, in which case it is not the most relevant indicator for detecting a fire, unlike smoke detection), valid between 28°C and 70°C, because the probability of a fire is highly correlated with the value of this sensor in this range of values, and invalid above 70°C, because in this case, either the sensor malfunctions, or the fire is such that other sensors are just as indicative of the fire so that it is not necessary to take this modality element into account.

[0090] In the PI adjustment phase, during steps E2 and E3, the values ​​are tested in 2°C increments, and a partition of the domain D m is operated, for example ([-50, 10], invalid) ([10, 28], neutral), ([28, 70], valid), and ([70, 100], invalid). In operating phase P2, any modality element even not tested during the adjustment phase (for example 23.2°C or - 12.3°C) can therefore be categorized and therefore filtered.

[0091] In another example, a sensor (e.g. door opening) may return a value ON (door open), OFF (door closed) and ERR7 (error message). The domain of the modality "door opening" is then {ON, OFF, ERR7], and can for example be partitioned for the assessment element "fire risk" into {ON, OFF] in neutral modality and {ERR7} for the invalid category. This example is, again, for illustrative purposes.

[0092] This makes it possible to precisely filter all possible modality elements for a modality without having to test them all.

[0093] Furthermore, the validity information (e.g., the category) concerning an element of modality x™ of a modality m can be correlated with at least one other element of modality x™' associated with another modality m'. This is called conditional, or correlated, validity information. Two values ​​(e.g., temperature and humidity level) or two categories (that of each of the two elements) or one value and one category can be correlated. Correlations with more than two elements can be considered.

[0094] To construct filtering data including correlated validity information, it is possible to test all possible combinations of modality elements, or to explore the set of combinations statistically or probabilistically, for example simple average methods, the use of theories extending probabilities such as the Dempster-Shafer theory, etc.

[0095] The concept of correlated validity is particularly important for the evaluation of neutral data. Indeed, two elements of modalities can be individually neutral, because their individual absence, tested successively, has no impact on the prediction of the situation, but valid when both are absent at the same time. This situation therefore means that these two data of different modalities provide the same information and that if they are not taken into account at all, this has an impact on the prediction of the situation. It is therefore important in this case that at least one is considered valid, hence the interest of correlated validity information.

[0096] To take the example above, the values ​​10°C and 28°C can be variable according to a modality relating to the season (for example a date) or to a temperature reading by a meteorological service (i.e. a modality obtained via an internet connection). Indeed, if the temperature of 28°C is a temperature beyond which the probability of fire is high in winter (few users heat their homes beyond this), this is not the case in summer, in which case this “threshold” temperature above which the category of the modality goes from neutral to valid can be raised from 28°C (winter) to 35°C (summer) during the months of June, July and August.

[0097] Similarly and still in this example, since a fire dries the atmosphere very strongly, the validity of the "kitchen temperature" modality above 40°C can depend on the value of a modality relating to humidity, and with the adjustment phase, it is possible to consider that the temperature modality is invalid in the 40-70°C range if the humidity modality is in the 80-100% humidity range, and valid in the 0-10% range.

[0098] Other examples can be considered, for example for the evaluation "probability that a driver falls asleep at the wheel of his vehicle", which if it exceeds a certain value (for example 0.7), triggers an alert signal within the vehicle to wake the driver from his torpor. Here, the category of the modality "centering of the vehicle on the road" can be conditioned by the value of the modality "heart rate" measured by a watch connected to the driver's wrist. Indeed, the heart rate drops in the event of falling asleep, but remains very high in the event of a stressful situation, for example a sudden maneuver to avoid an object on the road, a person, an animal or another vehicle with erratic behavior.Filtering here allows the evaluation model to reliably detect drowsiness while avoiding false positives which would cause undue and unpleasant alarm triggering (or even dangerous since an undue audible alarm in the middle of a stressful driving situation amplifies the stress).

[0099] The modality elements not tested in the PI adjustment phase can be automatically (i.e. by default) associated with a given validity information, for example valid or neutral. Optionally, before steps E2 and E3 of the PI adjustment phase, the method can comprise a step E1 for testing the test data. In such a test step, each target evaluation element Sf is compared to the evaluation element S" generated with the associated test input, so as to estimate the validity of the test data. In other words, this step E1 makes it possible to increase the confidence in the filtering data constructed subsequently in steps E2 and E3, by avoiding taking into account one of the test data which would be clearly inconsistent with what the model predicts. This also makes it possible to identify model malfunctions upstream, if on the contrary it is the test data in which we have the most confidence.

[0100] Finally, in relation to Figure 5, a simplified structure of an electronic evaluation generation device according to one embodiment of the invention is presented.

[0101] As illustrated in Figure 5, the electronic device for generating an evaluation of a situation according to one embodiment of the invention comprises a memory M, a processing unit, equipped for example with a programmable computing machine or a dedicated computing machine, for example a processor P, and controlled by a computer program Pg, implementing steps of a method for generating an evaluation of a situation as described above.

[0102] At initialization, the code instructions of the computer program Pg are, for example, loaded into a RAM memory before being executed by the processor of the processing unit P.

[0103] The processor of the processing unit P implements steps of the method for generating an evaluation of a situation described previously, according to the instructions of the computer program Pg, to: implement an evaluation model MOD, configured to receive a set X of modality elements, called input, x™, from sensors equipping an environment and / or a user, a modality element being associated with a modality m and being representative of a state of the modality m; and to generate, as a function of the input modality elements, at least one evaluation element, called output S t, representative of at least one given aspect of a situation associated with the input modality elements E, said evaluation element making it possible, in an operating phase, to control a triggering of a service adapted to the environment and / or to the user; said processor being further configured to, during the operating phase and prior to the generation of an evaluation element S t given, determine that ignoring a given modality element x™ improves the accuracy of generating an evaluation element S t , and not take into account said modality element x™ in the generation of said evaluation element S t .

[0104] The various examples above have provided some examples of modalities. These examples of modalities are of course not exhaustive, as they can be very varied and of various types: visual modalities (e.g., characteristics of the face, body, movements, environment, etc.), sound modalities (e.g., ambient sound, spoken words, etc.), textual modalities (e.g., transcription of speech, contextual information, etc.), physiological modalities (e.g., electrocardiogram, respiratory rate, etc.). Some modalities may not be derived from sensors.

[0105] A three-element categorization has also been seen, but other categorizations can be considered. For example, to determine the dangerousness of a situation by a supervision system implementing the process of generating an assessment of a situation described above, the multimodal values ​​can be judged crucial (their presence increases performance significantly), optional (their presence increases performance marginally), neutral (their presence does not influence performance) or invalid (their presence negatively influences performance).

Claims

CLAIMS 1. Method for generating an evaluation of a situation by an evaluation model (MOD), the method being implemented by an electronic device, the evaluation model (MOD) being configured to receive a set (X) of modality elements, called input, (x™), from sensors equipping an environment and / or a user, a modality element being associated with a modality (m) and being representative of a state of the modality (m); and to generate, as a function of the input modality elements, at least one evaluation element, called output (S t ), representative of at least one given aspect of a situation associated with the input modality elements (X), said evaluation element making it possible, in an operating phase, to control a triggering of a service adapted to the environment and / or to the user, said method comprising, during the operating phase and prior to the generation of an evaluation element (St ) given, a filtering comprising the determination that the disregard of a given modality element (x™) improves the accuracy of the generation of an evaluation element (S t ), and a failure to take into account said modality element (x™) in the generation of said evaluation element (S t ).

2. Method for generating an evaluation of a situation by an evaluation model (MOD) according to claim 1, comprising an adjustment phase during which filtering data are constructed, the filtering data comprising at least one validity information concerning at least one modality element (x™) allowing the determination during the filtering that not taking into account this given modality element (x™) improves the precision of the generation of an evaluation element (S t ).

3. Method for generating an evaluation by an evaluation model (MOD) according to claim 2, in which said validity information concerning at least one modality element (x™) comprises a category to which said modality element belongs, the category being part of the set of categories comprising a so-called valid category and at least one other category.

4. Method for generating an evaluation by an evaluation model (MOD) according to claim 3, in which said set of categories comprises at least one so-called neutral category and one so-called invalid category.

5. Method for generating an evaluation by an evaluation model (MOD) according to claim 3 or 4, in which the determination that the non-consideration of a given modality element (x™) improves the precision of the generation of an evaluation element (S t) includes determining whether said given modality element (x™) belongs to the valid category.

6. Method for generating an evaluation by an evaluation model (MOD) according to one of claims 2 to 5, in which the construction of the filtering data is based on data, called test data, (Y) comprising at least one pair formed from a set of modality elements, called test data, (y m ) and an evaluation element, called target, (S^).

7. Method for generating an evaluation by an evaluation model (MOD) according to claim 6, in which the construction of the filtering data comprises, for at least one modality (m), a generation, by the evaluation model (MOD) and as a function of the plurality of test modality elements (y m) by excluding from said generation the modality element (y™) associated with said modality (m), of an evaluation element, called a test element, (S”), and an association with said element (y™) thus excluded from validity information based on a comparison between the target evaluation element (S t ') and the test assessment element (S").

8. Method for generating an evaluation by an evaluation model (MOD) according to one of claims 3 to 7, in which at least one modality defines a set (D m ) formed from all or part of the modality elements associated with this modality, the filtering data comprising for said modality (m) information representative of a partition of said set (D m ) associated with said modality (m), in subsets each associated with one of the categories of the set of categories.

9. Method for generating an evaluation by an evaluation model (MOD) according to one of claims 2 to 8, in which the at least one validity information concerning at least one modality element (x™) concerns at least one second modality element (x™') relating to another modality (m').

10. Method for generating an evaluation by an evaluation model (MOD) according to one of the preceding claims, in which the evaluation model is robust in the event of missing data.

11. Electronic device for generating an evaluation of a situation by an evaluation model (MOD), the device comprising a processor configured to: implement said evaluation model (MOD), configured to receive a set (X) of modality elements, called input, (x™), from sensors equipping an environment and / or a user, a modality element being associated with a modality (m) and being representative of a state of the modality (m); and to generate, as a function of the input modality elements, at least one evaluation element, called output (S t), representative of at least one given aspect of a situation associated with the input modality elements (X), said evaluation element making it possible, in an operating phase, to control a triggering of a service adapted to the environment and / or to the user; said processor being further configured to, during the operating phase and prior to the generation of an evaluation element (S t ) given, determine that ignoring a given modality element (x™) improves the accuracy of generating an evaluation element (S t ), and not take into account said modality element (x™) in the generation of said evaluation element (S t ).

12. Computer program product comprising instructions for implementing a method according to claims 1 to 10, when this program is executed by a processor.

Images