Methods for selection of training data and for training a classifier, and system for training a classier

Abstract

A method (100) for selection of a training data set for a computer-implemented training method (110) for training a machine-learnable classifier (120), a computer-implemented training method (110) for training a machine-learnable classifier, a trained machine-learnable classifier, a computer program product, and a system (500) for training a machine-learnable classifier (510) are provided. The method comprises obtaining sensor data (200) associated with a physiological behavior of a person (210) and obtaining input data (250) associated with at least one physiological condition and / or at least one psychological condition of the person. The method further comprises mapping (270) the obtained input data to the obtained sensor data selectively based on at least one criterion to generate a subset (280) of the obtained sensor data and obtaining a selected training data set (290) based on the generated subset of the obtained sensor data.

Description

[0001] METHODS FOR SELECTION OF TRAINING DATA AND FOR TRAINING A

[0002] CLASSIFIER, AND SYSTEM FOR TRAINING A CLASSIER

[0003] FIELD OF THE INVENTION

[0004] The present invention generally relates to methods for a selection of training data and for training a classifier, as well as a system for training a classifier.

[0005] BACKGROUND OF THE INVENTION

[0006] Applying machine learning for elder care sensing systems faces several challenges. The first challenge is that the pre-trained models for elderly monitoring need to be finetuned to each specific elderly. This finetuning requires to update the model' s weights based on labeled sensing data acquired from this specific elderly' s home. Secondly, even after the model has been finetuned to the specific elderly, the machine learning system will suffer from data drift (meaning that the sensor-data distribution will shift as the health state of the specific elderly changes). Hence, if not regularly updated based on fresh training data, even the best elder care model for fall detection will go stale in production if it can’t adapt to shifting data distributions. Hence, in current state of the art fall detection models, the more the distribution shifts, the more the model’s performance will worsen. As with continual learning approach the model needs to be fine-tuned locally with a much smaller amount of new data, whereby the re-training can be performed on edge devices rather than powerful GPUs in data centers.

[0007] There is an increased demand for monitoring systems for assisting elderly people and / or persons with physical and / or psychological disabilities. For example, monitoring systems of this kind may be arranged in homes or institutions for accident prevention of persons based on information of the persons and observed movements / behavior of the persons. It has been shown that incidents or accidents in the form of falls, faintings, etc., can be estimated based on a correspondence between person / patient information and (previously) observed movements / behavior of the persons / patients.

[0008] However, there are challenges related to the handling of observed movements / behavior of persons / patients. Firstly, it is not desirable to store all this movement / behavior data for reasons of limited storage capacity and / or high cost of ingesting data. Secondly, and consequently, it is not apparently conceivable how to decide what (sensor) data related to movement / behavior to save or store, and what data to discard.

[0009] Hence, it is desired to provide methods for selection of training data and for training a classifier, as well as a system for training a classifier, for limiting the amount of training data storage whilst purposefully selecting significant training data for training a classifier.

[0010] SUMMARY OF THE INVENTION

[0011] It is an object of the present invention to provide methods for selection of training data and for training a classifier, as well as a system for training a classifier, for limiting the amount of training data storage whilst selecting significant training data for training a classifier.

[0012] This and other objects are achieved by providing a method, a classifier, a computer program product and a system having the features in the independent claims. Preferred embodiments are defined in the dependent claims.

[0013] According to a first aspect of the present invention, there is provided a method for selection of a training data set for a computer-implemented training method for training a machine-learnable classifier. The method comprises obtaining sensor data associated with a physiological behavior of a person and classifying portions of the obtained sensor data as events. The method further comprises obtaining input data indicative of a change in a physiological, Ti, and / or psychological condition, T2, of the person and selecting the portions of the classified sensor data corresponding to events assosiated with said change in the physiological and / or psychological condition of the person to generate a subset of the obtained sensor data. The method further comprises obtaining a selected training data set based on the generated subset of the obtained sensor data. For example, the input data may be indicative of a change in a memory condition of the person. The method may comprise selecting portions of the classified sensor data corresponding to events associated with said change in memory condition. The training data set that is then based on the generated subset of the obtained sensor data, such that not all obtained sensor data are stored but only the sensor data assosiated with the change. The training data set may be used to re-train / update the machine learnable classifier.

[0014] The method may further comprise omitting from the training data set sensor data corresponding to events not assosiated with the change in the physiological and / or psychological condition of the person. For example, the input data may be indicative of a change in walking speed of a person. The method may delete from the training data set sensor data corresponding to events associated with a past (previous) physiological and / or psychological condition of the person, e.g., past gait speed events.

[0015] The method may further comprise determining a value of a predetermined quality parameter of the obtained sensor data, and selecting the portions of the classified sensor data whose determined value exceeds a quality threshold, Tq. For example, the method may comprise determining a signal to noise ration of the obtained sensor data and selecting the portions of the classified sensor data for which a sgnial to noise ration is above a predetermined threshold value. In another example, the quality parameter may be a resolution of an image and the method may comprise determining a resolution value of the image input sensor data and selecting the portions of the classified data for which a resolution value exceeds a predetermined threshold value.

[0016] The method may further comprise determing a level of coverage of the person in the obtained sensor data, and selecting the portions of the classified sensor data whose determined level of coverage exceeds a coverage threshold, Tc. For example, if the obtained sensor data comprises a clear and / or non-obstructed view of the person (and of other persons and / or key objects in the scene), i.e. that the level of coverage of the person in the obtained sensor data is (relatively) high, the subset may comprise this obtained sensor data. The present embodiment is advantageous in that the method may generate a subset of the obtained sensor data which has a desired level of coverage (field of view) of the person and captures the surrounding context of the event of interest (e.g. actions undertaken by an elderly preceding the fall event; slower than usual walking speed of the elderly indicative of the elderly being afraid to fall; objects on the floor before the fall happened, etc.), which consequently leads to a selected training data set of higher quality.

[0017] According to an embodiment of the present invention, there is provided a computer-implemented training method for training a machine-learnable classifier, comprising: obtaining a training data set comprising the selected training data set obtained by the method according to any one of the preceding claims, and training the machine-learnable classifier based on the trainning dataset to obtain a trained machine-learnable classifier configured to classify portions of sensor data associated with a physiological behavior of a person as events.

[0018] According to an embodiment of the present invention, there is provided a system for generating a training data set for trainning a machine-learnable classifier. The system comprises an input arranged to obtain sensor data associated with a physiological behavior of a person, and input data indicative of a change in a physiological, Ti, and / or psychological condition, T2, of the person. The system further comprises a processor communicatively coupled to the input, wherein the processor is configured to classify, using the machine-learnable classifier, portions of the obtained sensor data as events, select the portions of the classified sensor data corresponding to events assosiated with said change in the physiological and / or psychological condition of the user to generate a subset of the obtained sensor data, and generate the training data set based on the generated subset of the obtained sensor data. Said system input may comrpise at least one of: a graphical user interface, GUI, a text input, and an audio input, wherein the input is further configured to obtain input data associated with at least one of a change of at least one physiological condition, and a change of at least one psychological condition, of the person.

[0019] In an embodiment, the method may comprise obtaining sensor data associated with a physiological behavior of a person, and obtaining input data associated with at least one of at least one physiological condition, and at least one psychological condition, of the person. The method may further comprise mapping the obtained input data to the obtained sensor data selectively based on at least one criterion to generate a subset of the obtained sensor data, and obtaining a selected training data set based on the generated subset of the obtained sensor data.

[0020] According to a second aspect of the present invention, there is provided a trained machine-learnable classifier, being trained according to a computer-implemented training method via the method according to the first aspect of the present invention.

[0021] According to a third aspect of the present invention, there is provided a computer program product comprising instructions which, when the program product is executed by a computer, cause the computer to carry out a computer-implemented training method via the method according to the first aspect of the present invention.

[0022] According to a fourth aspect of the present invention, there is provided a system for training a machine-learnable classifier, comprising an input arranged to obtain sensor data (200) associated with a physiological behavior of a person and input data (250) indicative of a change in a physiological, Ti, and / or psychological condition, T2, of the person. The system further comprises a processor communicatively coupled to the input, wherein the processor is configured to classify, using the machine-learnable classifier, portions of the obtained sensor data as events. The processor is further configured to select the portions of the classified sensor data corresponding to events assosiated with said change in the physiological and / or psychological condition of the user to generate a subset of the obtained sensor data and obtain a selected training data set based on the generated subset of the obtained sensor data. The processor is further configured to train the machine-learnable classifier based on the selected training data set to obtain a trained machine-learnable classifier configured to classify portions of sensor data associated with a physiological behavior of a person as events.

[0023] According to anotheraspect of the present invention, there is provided a system for training a machine-learnable classifier. The system comprises an input arranged to obtain sensor data associated with a physiological behavior of a person, and input data associated with at least one of at least one physiological condition, and at least one psychological condition, of the person. The system further comprises a processor communicatively coupled to the input. The processor is configured to map the obtained input data to the obtained sensor data selectively based on at least one criterion to generate a subset of the obtained sensor data. The processor is further configured to obtain a selected training data set based on the generated subset of the obtained sensor data. The processor is further configured to train the machine-learnable classifier by iteratively applying a machinelearning algorithm to the selected training data set to obtain a trained machine-learnable classifier configured to generate an estimated physiological behavior of the person.

[0024] Thus, the present invention is based on the idea of mapping obtained input data of a physiological and / or psychological condition of a person to obtained sensor data of the person’s physiological behavior based on one or more criterions in order to generate a subset of the obtained sensor data, i.e. a (smaller) subset of the (originally) obtained sensor data. A selected training data set is thereafter obtained based on the subset of the obtained sensor data. The invention hence deals with the decision of which of the obtained sensor data to store and which ones to discard or delete. The process of the present invention receives user input, which is used for the decision of which data (subset) to use. The zest of the present invention is the mapping from the user input to the obtained data based on the criterion(s) or decision(s) about which sensor data to retain (for the continuous re-training of a classifier).

[0025] The present invention is advantageous in that there is an efficient management regarding which of the obtained sensor data to retain and to use as training data. It should be noted that memory and / or storage devices (especially edge devices which may be preferred for privacy reasons) have limited data storage capacity, and the present invention hereby meets the need to manage, based on decision criteria, which of the sensor data to retain as potential neural-net training samples in the memory and which of the sensor data to discard, prune or delete.

[0026] The present invention is further advantageous by the avoidance of the associated cost of storing relatively large amounts of data in the form of obtained sensor data. Hence, the present invention achieves a cost-efficient handling or management of sensor data by its selection of which sensor data to store and / or use via the one or more criterions.

[0027] The present invention is further advantageous regarding privacy concerns and / or regulations, as it may be prohibited to store and / or use certain sensor data. Hence, the present invention may select sensor data which is allowed to store and / or use from a privacy perspective.

[0028] According to the first aspect of the present invention, there is provided a method for selection of a training data set for a computer-implemented training method for training a machine-learnable classifier. Hence, the method relates to a selection or choice of a desired and / or suitable training data set for a computer-implemented training method for training a machine-learnable classifier. The method comprises obtaining sensor data associated with a physiological behavior of a person. By “sensor data”, it is here meant substantially any kind of data from one or more sensors such as cameras, time series radar, RF sensing, ToF sensing, WiFi sensing, etc. By “physiological behavior of a person”, it is here meant (physical) movement(s), motion(s), or the like, or a person. Thus, the sensor data associated with the physiological behavior of the person means the sensor data associated with, comprising and / or related to, the person’s pose, physical movement(s), motion(s), or the like. The method further comprises obtaining input data associated with at least one of at least one physiological condition, and at least one psychological condition, of the person. Hence, the input data is associated with, comprises, or is related to, the person’s physiological and / or psychological condition. By “physiological condition”, it is here meant a physiological or physical condition or state of the person. In other terms, the “physiological condition” may represent the level of physiological (physical) well-being of the person. Furthermore, by “psychological condition”, it is here meant a psychological or mental condition or state of the person. In other terms, the “psychological condition” may represent the level of psychological well-being of the person. The method further comprises mapping the obtained input data to the obtained sensor data selectively based on at least one criterion to generate a subset of the obtained sensor data. By the term “mapping”, it is here meant projecting, associating, relating, or the like. Hence, the method comprises mapping, projecting, associating and / or relating the obtained input data to the obtained sensor data. The mapping from the obtained input data to the obtained sensor data is performed selectively by the method based on (decision) criterion(s) to generate a subset of the obained sensor data. In other words, the method generates a (smaller) subset of the (original, larger) obtained sensor data via one or more criterions which the mapping from the obtained input data to the obtained sensor data is based upon. The method further comprises obtaining a selected training data set based on the generated subset of the obtained sensor data. Hence, the method obtains a selected or chosen training data set, e.g. to which a machine-learning algorithm may be applied to obtain a trained machine-learnable classifier, based on the generated subset of the obtained sensor data.

[0029] According to an embodiment of the present invention, the method may be iteratively performed for any obtained input data being different from a previously obtained input data. Hence, the method may be configured to obtain (second) input data of a person’s physiological and / or psychological condition(s) which differ from a previously obtained (first) input data of a person’s physiological and / or psychological condition(s), and perform the method steps anew, i.e. iteratively for any input data which differs from a previously obtained input data. Hence, the method may obtain a selected training data set based on the (latest) generated subset of the obtained sensor data via the mapping of the (latest) obtained input data to the obtained sensor data based on the criterion(s). The present embodiment is advantageous considering that the behavior of a person (elderly) changes over time as he / she ages, and it is therefore advantageous if the algorithm for obtaining a trained machine- learnable classifier configured to infer a physiological behavior (e.g. fall(s) to the ground or accident(s)) of the person is dynamically adapted (re-trained) to the latest activity patterns / behaviors of the persons / elderly. The invention adaptively ensures that the most relevant (sensor) data for this re-training of the model is stored, and offers the possibility to perform on-premise retraining / fine-tuning of a classifier which was earlier extensively pretrained in the cloud with generic data (i.e. involving any sensor data of the specific user and the user' s specific home).

[0030] According to an embodiment of the present invention, the method may further comprise accumulating the selected training data sets, upon the iteratively performed method, to the obtained selected training data set. Hence, for each input data of a person’s physiological and / or psychological condition(s) which differ from a previous input data, the method may accumulate and / or prune the selected training data set via the iteratively performed method in order to generate the obtained selected training data set. The present embodiment is advantageous in that the method may obtain a selected training data set with different time stamps, and that the selected training data set thereby may be more suitable for its purposes.

[0031] According to an embodiment of the present invention, at least a portion of the obtained sensor data may be associated with a time stamp, T, wherein the subset of the obtained sensor data constitutes the at least a portion of the obtained sensor data which fulfils abs (To-T) < Tt, wherein To is present time and Ttis a time threshold.

[0032] According to an embodiment of the present invention, the method may further comprise determining a level of quality of at least a portion of the obtained sensor data based on at least one quality criterion, wherein the subset of the obtained sensor data constitutes the at least a portion of the obtained sensor data whose determined level of quality exceeds a quality threshold, Tq. Hence, the method may further comprise determining a quality level of the obtained sensor data based on one or more quality criterions, and generating a subset of the obtained sensor data of obtained sensor data with a quality level above or exceeding the quality threshold, Tq. By “quality”, it is here meant substantially any quality parameter or criterion of the obtained sensor data such as sharpness of pictures (images), (low) noise, relevance of obtained sensor data, (incomplete) view of the event due to field of view limitations of the sensor, partially incomplete time-series sensor data (e.g. sensing data of other events preceding a fall event is partially missing), etc. The present embodiment is advantageous in that the method may generate a subset of the obtained sensor data which has a higher quality compared to other obtained data, which consequently leads to a selected training data set of higher quality.

[0033] According to an embodiment of the present invention, the at least one quality criterion may be associated with a level of a field of view of the person in the obtained sensor data. Hence, the one or more quality criterions may be associated with a level or amount of the view or the sensor-based monitoring of the person. For example, if the obtained sensor data comprises a clear and / or non-obstructed view of the person (and of other persons and / or key objects in the scene), i.e. that the level of the field of view is (relatively) high, the subset may comprise this obtained sensor data. The present embodiment is advantageous in that the method may generate a subset of the obtained sensor data which has a desired field of view of the person and captures the surrounding context of the event of interest (e.g. actions undertaken by an elderly preceding the fall event; slower than usual walking speed of the elderly indicative of the elderly being afraid to fall; objects on the floor before the fall happened, etc.), which consequently leads to a selected training data set of higher quality. For instance, if the elderly exhibited a slower than usual walking speed upfront of the sensed possible fall event this may be indicative the elderly had been experiencing some motoric instabilities already preceding the sensed suspected fall event. However, the elderly' s regular walking speed vs. what constitutes an abnormal “I am right now super-afraid to fall”-gait will substantially change over the years (e.g. it is well known that the average walking speeds decrease with deteriorating health as the elderly is ageing).

[0034] According to an embodiment of the present invention, the at least one criterion may be associated with at least one of a change as a function of time of at least one physiological condition of the obtained input data exceeding a first threshold, Tl, and a change as a function of time of at least one psychological condition of the obtained input data exceeding a second threshold, T2. Hence, the one or more criterions, upon which the mapping of the obtained input data to the obtained sensor data is (are) based, may be associated with a relatively large or significant timeseries change of the physiological and / or psychological condition of the person. The present embodiment is advantageous in that the input data depends on (a) relatively quick change(s) of physical and / or mental state(s) of the person, leading to a more relevant obtained user data representative of the current (physiological / psychological) state of the person as well as the state of his environment (e.g. rearrangement of furniture leading to more fall risk in a certain area of the house), e.g. for retraining the algorithm. This will enable more accurate real-time inferences of the person’s (elderly' s) risk of incidences (e.g. falls) based on the currently observed sensor data, e.g. in the form of gait, body balance, body posture, walking speed, etc.

[0035] According to an embodiment of the present invention, the method may further comprise obtaining an assessment of the obtained sensor data from the person said assessment may be a label of an event assosiated with portions of the obtained sensor data. Hence, the method may obtain or receive an assessment or evaluation of the obtained sensor data from the person, and generate a subset of the obtained sensor data based on or associated with the assessment. For example, the person may assess the obtained sensor data concerning sensor data quality, field of view, relevance, etc., and the method may use this assessment via the criterion(s) upon which the mapping of the obtained input data to the obtained sensor data is (are) based. The present embodiment is advantageous in that this consequently leads to a selected training data set of higher relevance and / or quality.

[0036] According to an embodiment of the present invention, the method may further comprise associating at least one physiological behavior of a person with at least one event of the person from the obtained sensor data, and determining at least one action of the person preceding the at least one physiological behavior of the person and influencing the at least one physiological behavior of the person, wherein the at least one criterion is associated with the determined at least one action of the person. By the term “influencing”, it may alternatively be meant “causing”. By the term “event”, it is here meant e.g. incident, accident, etc. The present embodiment is advantageous in that the possibility to determine the person’s action may lead to an even more suitable selection of the training data set.

[0037] According to an embodiment of the present invention, the method may further comprise storing the selected training data sets in a memory in case of available memory capacity of the memory. The present embodiment is advantageous in that the most relevant data is stored in the memory. Hence, as the invention focuses on the decision of which of the obtained sensor data to store in the (sometimes scarce) memory, an improved data handling and storage process is achieved.

[0038] According to an embodiment of the present invention, there is provided a computer-implemented training method for training a machine-learnable classifier. The computer-implemented training method comprises obtaining a training data set comprising the selected training data set obtained by the method according to any one of the preceding embodiments, and obtaining the input data obtained by the method according to any one of the preceding embodiments. The computer-implemented training method further comprises training the machine-learnable classifier by iteratively applying a machine-learning algorithm to the training data set to obtain a trained machine-learnable classifier configured to generate an estimated physiological behavior of the person. The present embodiment is advantageous in that the classifier is able to estimate a person’s physiological behavior, such as accidents, risky behavior (e.g. carrying a hot kitchen pan into a dining room), incidents and / or falls, based on the input data of the persons physical and / or mental state. For instance, if the person inputs that he / she has (recently) started to suffer from a physiological deteriorating condition (e.g. dizziness), the method may give more weight to selecting / storing dizziness-related sensor events of the obtained sensor data. Upon training the machine-learnable classifier by iteratively applying the machine-learning algorithm to the training data set comprising this input data, the estimation of the physiological behavior of the person may be used to warn the person in real time that given his / her recent activities he / she may soon become dizzy and fall. Similarly, a person may be warned who is at risk of dizziness to not carry around hot cooking pans on his / her own. In other words, by virtue of the selected training data set, the Al algorithm can conveniently and efficiently predict the person’s (user’s) risk of dizziness based on the preceding activities. According to an example of the present invention, there is provided a method for estimating physiological behavior of a person. The method comprises obtaining input data associated with at least one of at least one physiological condition, and at least one psychological condition of the person, and estimating physiological behavior of the person by applying the obtained input data to the trained machine-learnable classifier according to the previously described embodiment.

[0039] According to an example of the present invention, the method may further comprise associating at least one physiological behavior of a person with at least one accident of the person, estimating at least one accident of the person based on a level of correspondence between the estimated physiological behavior of the person and the at least one physiological behavior of a person, and generating an alarm upon determining an occurrence of at least one accident based on the estimated at least one accident.

[0040] Further objectives of, features of, and advantages with, the present invention will become apparent when studying the following detailed disclosure, the drawings and the appended claims. Those skilled in the art will realize that different features of the present invention can be combined to create embodiments other than those described in the following.

[0041] BRIEF DESCRIPTION OF THE DRAWINGS

[0042] This and other aspects of the present invention will now be described in more detail, with reference to the appended drawings showing embodiment(s) of the invention.

[0043] Fig. 1 schematically shows a method for selection of a training data set for a computer-implemented training method for training a machine-learnable classifier according to an exemplifying embodiment of the present invention,

[0044] Fig. 2 schematically shows a portion of a method for selection of a training data set for a computer-implemented training method for training a machine-learnable classifier according to an exemplifying embodiment of the present invention,

[0045] Figs. 3a-3e schematically show examples of mappings of the obtained input data to the obtained sensor data according to the portion of the method according to Fig. 2,

[0046] Fig. 4 schematically shows a computer-implemented training method for training a machine-learnable classifier according to an exemplifying embodiment of the present invention, and

[0047] Fig. 5 schematically shows a system for training a machine-learnable classifier according to an exemplifying embodiment of the present invention. DETAILED DESCRIPTION

[0048] Fig. 1 schematically shows a method 100 for selection of a training data set for a computer-implemented training method for training a machine-learnable classifier according to an exemplifying embodiment of the present invention. The method 100 comprises obtaining sensor data 200 associated with a physiological behavior of a person 210, such as (physical) movement(s), motion(s), or the like, of a person 210. The sensor data 200 may encompass or comprise substantially any kind of data of and / or related to the person 210, such as pictures, videos, audio data, RF sensing data, Doppler radar data, ToF data, etc. from one or more sensors such as cameras, time series radar, WiFi sensing, etc. Thus, the sensor data 200 associated with the physiological behavior of the person 210 means the sensor data 200 being associated with, comprising and / or related to, the person’s 210 physical movement(s), motion(s), or the like. The sensor data 200 may in particular comprise movement(s) and / or motion(s) related to accidents and / or incidents such as fall(s), stumbling(s), fainting(s), of the person 210 as well as activities preceding the falls (such as walking, bending over) and after the fall event (walking with a limp).

[0049] The method 100 further comprises classifying portions of the obtained sensor data as events. Time-series portions of the obtained sensor data are processed by the machine-learning classifier to classify them as events (patterns, incidents). For example, the machine-learning classifier may classify time-series RF sensing data as a fall event. In another example, the machine-learning classifier may classify time-series heart-rate data as a heart-attack event. In another example, portions of audio and / or video data may be classified as a person walking or running, etc. An event may be any specific occurance, action or behavior of the user that can be detected by the sensor data (e.g., fall event, person walking, person being lost - wandering around, etc).

[0050] The method 100 further comprises obtaining input data (250) indicative of a change in a physiological, Ti, and / or psychological condition, T2, of the person, 210. Hence, the input data 250 is associated with, comprises, or is related to, the person’s 210 chane in the physiological and / or psychological condition or state, representing the level of physiological (physical) and / or psychological (mental) state, condition and / or well-being of the person 210 and / or fear of falling (e.g. manifested by a change in the walking speed).

[0051] The method 100 further comprises selecting the portions of the classified sensor data corresponding to events assosiated with said change in the physiological and / or psychological condition of the person to generate a subset of the obtained sensor data (280). For example, the method may select only portions of the classified sensor data corresponding to dizziness related events if the input data are indicative of an increased dizziness of the user (person reporting feeling more dizzy than before). In another example, the method may select only portions of the classified sensor data corresponding to heart-rate related events (e.g., elevated heart-rate, dizziness, excess sweating) if the input data are indicative of a change in the cardiovascular condition of the user. The subset 280 is schematically indicated in Fig. 1 by the dashed rectangle within the obtained sensor data 200. Hence, the method 100 generates a (smaller) subset 280 of the (original, larger) obtained sensor data 200 via the smart selection from the obtained sensor data 200 only sensor data that are assosiated with the change in the condition of the user. The method 100 further comprises the step of obtaining a selected training data set 290 based on the generated subset 280 of the obtained sensor data 200. The method 100 hereby obtains a selected or chosen training data set 290, e.g. to which a machine-learning algorithm may be trained to obtain a trained machine- learnable classifier, based on the generated subset 280 of the obtained sensor data 200. It should be noted that an (edge-AI) algorithm may have to run on a relatively small Al accelerator chip, limiting the maximum size of the accident / incident (fall) detection model. Unlike relatively large neural networks, which may require cloud servers or powerful GPUs, the models related to this invention may be significantly smaller. It has been recognized that adding training data in one place could impair the model’s performance elsewhere. For example, adding data on several types of accidents / falls (e.g. chair falls) might make it difficult for an algorithm to recognize other difficult to recognize fall types if the model architecture doesn’t have enough capacity to recognize both types equally well. Hence, when deciding which training data to keep, prune or discard for the selected training data set 290, the data management process of the method 100 may assess whether it is possible to add data in a way that improves performance on one slice without impairing the model’s inference performance on others.

[0052] The method 100 may comprise omitting from the selected training data set (290) the portions of the sensor data corresponding to events not assosiated with said change in the physiological and / or psychological condition of the user. For example, if the input data are indicative of a deterioration of the walking speed of the user, the method may omit sensor data indicative of walking or running pattern of the user as those portions of sensor data are not longer indicative of the current physiological and / or psychological condition of the user.

[0053] The method 100 may further comprise determining a value of a predetermined quality parameter of the obtained sensor data, and selecting the portions of the classified sensor data whose determined value exceeds a quality threshold, Tq. For example, the predetermined quality parameter may be any metric that measures the reliability or accuracy of the obtained sensor data, e.g., a signal to noise ratio, signal strength, channel utilization, an image and / or video resolution, frame rate, etc. For example, the method 100 may comprise selecting portions of the obtained sensor data for which a signal to noise ratio (SNR) value is above a predetermined SNR threshold, or for which a frame rate value is above a predetermined rate threshold, etc.

[0054] The method 100 may further comprise determing a level of coverage of the person in the obtained sensor data, and selecting the portions of the classified sensor data whose determined level of coverage exceeds a coverage threshold, Tc. For example, in the case of video data, the level of coverage could be the percentage of the person's body that is visible in the video frame. The method may select the portions of the classified sensor data for which the percentage of the person's body that is visible in the video frame is above a predetermined threshold. In another example, in the case of RF or WiFi signals, the level of coverage could be the percentage of the person's movements or behavior that are within the range of the sensors.

[0055] The method 100 further comprises obtaining input data 250 associated with at least one of one or more physiological conditions, Pi, and one or more psychological conditions, P2, of the person 210. Hence, the input data 250 is associated with, comprises, or is related to, the person’s 210 physiological and / or psychological condition or state, representing the level of physiological (physical) and / or psychological (mental) state, condition and / or well-being of the person 210 and / or fear of falling (e.g. manifested by a very slow walking speed). The method 100 further comprises mapping 270 (i.e. projecting, associating, relating, or the like) the obtained input data 250 to the obtained sensor data 200 selectively based on at least one criterion, Ci, to generate a subset 280 of the obtained sensor data 200. The mapping 270 from the obtained input data 250 to the obtained sensor data 200 is performed selectively by the method 100 based on the (decision) criterion(s), Ci, to generate a subset 280 of the obtained sensor data 200. The subset 280 is schematically indicated in Fig. 1 by the dashed rectangle within the obtained sensor data 200. Hence, the method 100 generates a (smaller) subset 280 of the (original, larger) obtained sensor data 200 via the criterion(s), Ci, which the mapping 270 from the obtained input data 250 to the obtained sensor data 200 is based upon. The method 100 further comprises the step of obtaining a selected training data set 290 based on the generated subset 280 of the obtained sensor data 200. The method 100 hereby obtains a selected or chosen training data set 290, e.g. to which a machine-learning algorithm may be applied to obtain a trained machine- learnable classifier, based on the generated subset 280 of the obtained sensor data 200. It should be noted that an (edge-AI) algorithm may have to run on a relatively small Al accelerator chip, limiting the maximum size of the accident / incident (fall) detection model. Unlike relatively large neural networks, which may require cloud servers or powerful GPUs, the models related to this invention may be significantly smaller. It has been recognized that adding training data in one place could impair the model’s performance elsewhere. For example, adding data on several types of accidents / falls (e.g. chair falls) might make it difficult for an algorithm to recognize other difficult to recognize fall types if the model architecture doesn’t have enough capacity to recognize both types equally well. Hence, when deciding which training data to keep, prune or discard for the selected training data set 290, the data management process of the method 100 may assess whether it is possible to add data in a way that improves performance on one slice without impairing the model’s inference performance on others.

[0056] Fig. 2 schematically shows a portion of a method 100 for selection of a training data set for a computer-implemented training method for training a machine- learnable classifier according to an exemplifying embodiment of the present invention. Fig. 2 is associated with the method 100 according to Fig. 1 and the text thereof, and it is also referred to this figure and text for an increased understanding. Fig. 2 emphasizes the mapping 270 of the obtained input data 250 to the obtained sensor data 200 selectively based on at least one criterion, Ci, to generate a subset 280 of the obtained sensor data 250, and Figs. 3a-e describe examples thereof.

[0057] Fig. 3a schematically shows an example of a mapping 270 of the obtained input data to the obtained sensor data based on (a) (decision) criterion(s), Ci. At least a portion of the obtained sensor data may be associated with a time stamp, T, wherein the subset of the obtained sensor data constitutes the at least a portion of the obtained sensor data which fulfils abs (To-T) < Tt, wherein To is present time and Ttis a time threshold. The mapping 270 of the obtained input data to the obtained sensor data is hereby described as being selectively based on criterion, Ci, as a function of the time stamp, T, to generate the subset of the obtained sensor data. Hence, according to this example, the mapping 270 from the obtained input data to the obtained sensor data is performed selectively by the method 100 based on the criterion, Ci, of selecting relatively new (recent) sensor data.

[0058] Fig. 3b schematically shows an example of a mapping 270 of the obtained input data to the obtained sensor data based on (a) (decision) criterion(s), C2. Here, the method may further comprise determining a level of quality of at least a portion of the obtained sensor data based on at least one quality criterion, C2, wherein the subset of the obtained sensor data constitutes the at least a portion of the obtained sensor data whose determined level of quality exceeds a quality threshold, Tq. Hence, the method comprises determining a quality level of the obtained sensor data based on quality criterion(s), C2, as a function of the quality threshold, Tq. Hence, the method generates a subset of the obtained sensor data with a quality level above or exceeding the quality threshold, Tq, concerning substantially any quality parameter or criterion of the obtained sensor data such as sharpness of pictures (images), (low) noise, relevance of obtained sensor data, etc.

[0059] Fig. 3c schematically shows an example of a mapping 270 of the obtained input data to the obtained sensor data based on (a) (decision) criterion(s), C3. Here, the (quality) criterion, C3, is associated with a level of a field of view, FoV, of the person in the obtained sensor data. Hence, the quality criterion(s), C3, may be associated with (is a function of) a level or amount of the view or the monitoring of the person. For example, the obtained sensor data (e.g. a fall event) may have been recorded in which the person was (visually) obstructed, in an environment with a relatively high level of (wireless) noise, resulting in this sensor event having poor signal-to-noise (S / N) ratio, etc. Responsive to detection of such a condition, the quality criterion(s), C3, may determine that the data of this sensing event is weakly classifiable due to the condition(s), and the raw data of this (e.g. fall) event may be discarded, i.e. left out of the subset of the obtained sensor data. Another example of this embodiment comprises augmenting the real-world training data with synthetic data to complete missing data from the sensed event (and generate a very complete training data sample). The data management process of the method may determine that the obtained sensor data from a certain fall event (or non-fall event) was taken from a viewpoint that does not capture the person’s (elderly’s) activity preceding the fall event. However, the method may have obtained from the elderly (or his / her caretaker) reliable labeling information about the activity preceding the fall event (e.g. that via a question and answer interaction, the method learned that the person was bending down to touch cables on the floor before falling sidewards when trying to return to upright body position). Similarly, the method may have obtained reliable labeling information from the elderly, e.g. that the person has dropped the phone and then has tried to pick it up and lacked to the strength to return to upright position. The method may hence augment the physically sensed events with synthetically generated sensor data representing in high resolution the person’s (elderly’s) activity preceding the fall. Fig. 3d schematically shows an example of a mapping 270 of the obtained input data to the obtained sensor data based on (a) (decision) criterion(s), C4. Here, the criterion, C4, is associated with (a function of) a change as a function of time of at least one physiological condition of the obtained input data exceeding a first threshold, Ti, and / or a change as a function of time of at least one psychological condition of the obtained input data exceeding a second threshold, T2. Hence, the criterion, C4, upon which the mapping 270 of the obtained input data to the obtained sensor data is (are) based, may be associated with a relatively large or significant change of the physiological and / or psychological condition(s) of the person. For instance, if the person (or assisting staff) inputs that the person has recently started to suffer from a deteriorated conditions (e.g. dizziness), more weight to storing dizziness-related sensor events of the obtained sensor data may be given. Consequently, an improved training data set is achieved, e.g. for generating an estimated physiological behavior of the person (e.g. for estimating and / or preventing accidents such as falls). This custom training data set may e.g. allow the retrained model to be used for a fall prevention application which is capable to upfront warn the person (elderly) in real time that given his / her recent activities he / she may soon become dizzy and fall. A fall prevention application of this kind may even ask the person that after the person has gotten up from the chair to first step for 30 seconds on the same spot and only afterwards start walking across the room. In other words, thanks to the training data, the Al algorithm can predict the user’s risk of dizziness based on the preceding activities.

[0060] Fig. 3e schematically shows an example of a mapping 270 of the obtained input data to the obtained sensor data based on (a) (decision) criterion(s), C5. The method may further comprise obtaining an assessment (labelling) of the obtained sensor data from the person, wherein the criterion(s), C5, is associated with the assessment. Hence, the method may obtain or receive an assessment or evaluation of the obtained sensor data from the person, and the criterion(s), C5, may be based on or associated with the assessment. For example, the person may assess the obtained sensor data concerning sensor data quality, field of view, relevance, etc., and the method may use this assessment via the criterion(s), C5, upon which the mapping 270 of the obtained input data to the obtained sensor data is (are) based. It is expected that many incident / accident detection systems deployed in the future in residential settings will not use cameras. For radar-based sensing systems and WiFi-based sensing system, a video of the fall event cannot be replayed. Hence, an assessment of the obtained sensor data (e.g. a fall event) may be based on a “recollection dialogue” with the person about the incident / accident, and the criterion(s), C5, may hereby be associated with the person’s recollection of the incident / accident. For example, a case may be envisaged of first sensor data associated with a first fall event on 1stOctober 2022 and a second sensor data associated with a second fall event on 1stJune 2023, wherein the assessment of the first sensing data has been received 5 minutes after the fall event and the assessment of the second fall event has been received from the person / caretaker after three days. As the memory of the person / caretaker about the context of the fall may already be clouded when he / she finally provides the assessment, it is proposed that the data management process of the present method is more inclined to discard the sensor data associated with the second fall event despite of this sensing data being younger. It should be noted that on average, 3.4 % of examples in 10 commonly used datasets in machine learning are mislabeled. In a further developed embodiment, the method may comprise obtaining assessments (labels) of the obtained sensor data (e.g. fall events) from a multitude of persons being unqualified or nonexperts (e.g. the person (elderly) himself or herself, a nurse, a visitor, etc.). Based on the multitude of assessments / labels, the method may perform a consistency analysis on the assessments inputted by this person(s) prior to populating the memory with additional training data (raw sensor data (e.g. of the fall event) and assessments / labels). For a given sensed sensor data (e.g. fall event), multiple tags may be applied by a plurality of persons (e.g., a first user such as the elderly-care-home nurse may have tagged the fall event as including both a chair fall and a fall-to-the-side while a second user and a third user such as the elderly himself and his / her room mate may have tagged it as a chair fall-only). The percentage consistency calculated may be logged as a consistency score for the fall event labeling. This consistency may be utilized by the method via the criterion(s), Cs, when deciding which recorded raw sensor data of fall-(prevention)-related sensor events to delete first from the memory. In a further developed embodiment, before the management process of the method decides to discard a recorded sensor data (event) based on deficiencies of the assessments / labels, it may attempt in a (final) effort to ask the person / user to improve the assessments / labels. According to another example, the method may encompass agreementbased labeling on the (fall detection) obtained sensor data set by first measuring where different labelers (e.g. elderly, first caretaker, second caretaker) agree vs. disagree in the labeled sensor data set and subsequently focus on improving the slice of data where most inconsistencies are. The method may hereby comprise an identification of what slice (subset) of the already labeled sensor data (fall detection data) to further improve and may ask the person / user for specific improvement actions for the labels. In this further developed embodiment, the method subsequently checks if the labeling of this data slice has improved sufficiently by the data-engineering actions taken by the end user(s). If so, the management process of the method instructs the memory to keep raw data of the accident / incident (fall event) of the obtained sensor data for future re-training of the model. If not, the raw sensing data of the fall event is discarded by the method. In other words, upon detecting inconstancies, the method allows the user a possibility to amend assessments / labels. For example, if the person / user does not perform this amendment(s) of the assessments / labels, the method may discard the raw sensor data associated therewith. Alternatively, instead of an agreement-based assessment / labeling described in the previous example, the method may alternatively identify mislabeling by following a methodology wherein a mislabeling by the person (elderly) is considered if it meets two conditions: a) the model’s predicted classification didn't match the label, and b) the model’s confidence in its classification was greater than its average confidence in its predictions of the labeled class over all examples bearing that label. If a high chance of mis-labeling is suspected, the data management process of the method is more inclined to discard the raw sensor data of this event. According to yet another example of Fig. 3e, the method may generate the subset of the obtained sensor data based on the (decision) criterion(s), Cs, only if this accident / incident (fall event) has at least a threshold consistency amount (e.g., 80%, 90%, or 100%) among different human labelers. The threshold may be a default amount (e.g., 100%), or may be assigned by the person / user. Rather than including this “add” option in a UI for all fall events, this “add” option may be populated by the method exclusively where the human labelers were consistent in their application of a label. The management process of the method may, where consistency is below the threshold consistency amount, prompt further persons / inspectors to tag the fall event. After processing by the further human labelers, the method may determine once again whether the tags given by the inspectors are consistent. Responsive to determining that the fall event still does not have a threshold consistency, the management process of the method may discard the fall event from its memory or may as a (final) effort prompt an expert to classify the fall event. Moreover, rather than prompting further inspectors, the method may determine whether the consistency is below a low consistency threshold (e.g., below 30%), in which case the sensor data associated with the fall event may be discarded.

[0061] Fig. 4 schematically shows a computer-implemented training method 110 for training a machine-learnable classifier according to an exemplifying embodiment of the present invention. The computer-implemented training method 110 comprises obtaining a training data set 300. The training data set 300 comprises the selected training data set 290 obtained by the method according to any one of the preceding embodiments, and the input data 250 obtained by the method according to any one of the preceding embodiments. The computer-implemented training method 110 further comprises training the machine-learnable classifier by iteratively applying 320 a machine-learning algorithm 330 to the training data 300 set to obtain a trained machine-learnable classifier 400 configured to generate an estimated physiological behavior of the person.

[0062] Fig. 5 schematically shows a system 500 for training a machine-learnable classifier 400 according to an exemplifying embodiment of the present invention. The system 500 comprises an input 520 arranged to obtain sensor data 200 and input data 250. The input 520 may be substantially any kind of input, such as e.g. a graphical user interface, GUI, a text input, and / or an audio input. The sensor data 200 is associated with a physiological behavior of a person, and the input data 250 is associated with one or more physiological condition(s), Pi, and one or more psychological condition(s), P2, of a person 210. According to an example, the input 520 may further, or alternatively, be configured to obtain input data 250 associated with a change of the physiological condition(s), Pi, and / or a change of the psychological condition(s), P2, of the person 210. One example of such user input 520 is a slider (G)UI allowing the user to define how much the person’s (elderly' s) physiological and / or psychological state or condition has (recently) changed / deteri orated compared to the previous input of the user (e.g. 6 months ago). The system 500 further comprises a processor 530 which is communicatively coupled to the input 520. The processor 530 is configured to map 270 the obtained input data 250 to the obtained sensor data 200 selectively based on at least one criterion, Ci, to generate a subset 280 of the obtained sensor data 200. The processor 530 is further configured to obtain a selected training data set 290 based on the generated subset 280 of the obtained sensor data 200. The processor 530 is further configured to train the machine-learnable classifier 400 by iteratively applying a machine-learning algorithm 330 to the selected training data set 290 to obtain a trained machine-learnable classifier 400 which is configured to generate an estimated physiological behavior of the person.

[0063] The person skilled in the art realizes that the present invention by no means is limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.

Claims

CLAIMS:

1. A method (100) for selection of a training data set for a computer- implemented training method (110) for training a machine-learnable classifier (120), the method comprising: obtaining sensor data (200) associated with a physiological behavior of a person (210), classifying portions of the obtained sensor data as events, obtaining input data (250) indicative of a change in a physiological, Ti, and / or psychological condition, T2, of the person, selecting the portions of the classified sensor data corresponding to events associated with said change in the physiological and / or psychological condition of the person to generate a subset of the obtained sensor data (280), and obtaining a selected training data set (290) based on the generated subset of the obtained sensor data.

2. The method according to claim 1, wherein the method is iteratively performed for any obtained input data being different from a previously obtained input data.

3. The method according to claim 2, further comprising accumulating the selected training data sets, upon the iteratively performed method, to the obtained selected training data set.

4. The method according to any preceding claim further comprising: omitting from the selected training data set the portions of the sensor data corresponding to events not assosiated with said change in the physiological and / or psychological condition of the user.

5. The method according to any preceding claim, further comprisingdetermining a value of a predetermined quality parameter of the obtained sensor data, and selecting the portions of the classified sensor data whose determined value exceeds a quality threshold, Tq.

6. The method according to any preceding claim, further comprising determing a level of coverage of the person in the obtained sensor data, and selecting the portions of the classified sensor data whose determined level of coverage exceeds a coverage threshold, Tc.

7. The method according to any one of the preceding claims, further comprising storing the selected training data sets in a memory in case of available memory capacity of the memory.

8. A computer-implemented training method (110) for training a machine- learnable classifier, comprising: obtaining a training data set (300) comprising the selected training data set obtained by the method according to any one of the preceding claims, and training (310) the machine-learnable classifier based on the trainning dataset to obtain a trained machine-learnable classifier (400) configured to classify portions of sensor data associated with a physiological behavior of a person as events.

9. A trained machine-learnable classifier, being trained according to the computer-implemented training method of claim 8.

10. A computer program product comprising instructions which, when the program product is executed by a computer, cause the computer to carry out the computer- implemented training method of claim 8.

11. A system (500) for generating a training data set for trainning a machine- learnable classifier (510), comprising an input (520) arranged to obtain sensor data (200) associated with a physiological behavior of a person (210),input data (250) indicative of a change in a physiological, Ti, and / or psychological condition, T2, of the person, a processor (530) communicatively coupled to the input, wherein the processor is configured to classify, using the machine-learnable classifier, portions of the obtained sensor data as events, select the portions of the classified sensor data corresponding to events associated with said change in the physiological and / or psychological condition of the user to generate a subset of the obtained sensor data, generate the training data set (290) based on the generated subset of the obtained sensor data.

12. A system (500) for training a machine-learnable classifier (510), comprising an input (520) arranged to obtain sensor data (200) associated with a physiological behavior of a person (210), input data (250) indicative of a change in a physiological, Ti, and / or psychological condition, T2, of the person, a processor (530) communicatively coupled to the input, wherein the processor is configured to classify, using the machine-learnable classifier, portions of the obtained sensor data as events, select the portions of the classified sensor data corresponding to events associated with said change in the physiological and / or psychological condition of the user to generate a subset of the obtained sensor data, obtain a selected training data set (290) based on the generated subset of the obtained sensor data, train the machine-learnable classifier based on the selected training data set to obtain a trained machine-learnable classifier (400) configured to classify portions of sensor data associated with a physiological behavior of a person as events.

13. The system according to claim 12, wherein the input comprises at least one of a graphical user interface, GUI, a text input, andan audio input, wherein the input is further configured to obtain input data associated with at least one of a change of at least one physiological condition, and a change of at least one psychological condition, of the person.

Images