Method for estimating the physical frailty of a person

Abstract

The present application relates to a method for estimating the physical frailty of a person, comprising an estimation phase (E2), during at least one estimation period (Pe), of the physical frailty of the person, implementing an analysis of images of the bodily movement of the person and taking into account at least one database representative of bodily movements for the person during at least one learning period (Pa), referred to as a personalised database (Pa_DB_User), the estimation phase (E2) delivering at least one data item (C_User) representative of an estimated physical frailty of the person during the estimation period (Pe) and a service recommendation (S_User) adapted to the estimated physical frailty.

Description

Method for estimating a person's physical frailty.

[0001] 1. Technical field

[0002] The present invention relates to the field of monitoring frail or elderly people at home, via e-health services for example, or isolated people in their workplace.

[0003] 2. Prior art

[0004] Within the framework of e-health services, several well-known applications allow for remote monitoring of a person, for example, while they are at home. This type of application is particularly useful for enabling elderly people to remain in their own homes.

[0005] Thus, for example, there are actigraphy systems for the implementation of these e-Health applications based on gait analysis, in order to prevent injuries or accidents (e.g. falls), to confirm the results of treatments (e.g. pre / post surgery) or to optimize sports performance.

[0006] Traditionally, gait analysis can be performed using kinetic (force) and kinematic (spatial / temporal) information and focuses on the seven-step gait cycle: heel strike, flat foot, mid-stance, push-off, acceleration, mid-stride, and deceleration. Traditional signal processing techniques rely on spatial gait parameters such as step length and duration, stride length and speed, step width, foot angle, etc. The analysis is performed using heuristics and expert rules without considering the entire human body.

[0007] However, in the context of home-based care for the elderly, it is important to have a comprehensive view of a person's health status. To do this, it is necessary to define the concepts of aging and frailty. The WHO defines human aging as follows: "From a biological perspective, it is the product of the accumulation of a wide range of molecular and cellular damage over time. This leads to a progressive decline in physical and mental capacities, an increased risk of disease, and ultimately, death." This definition also allows us to characterize a person's frailty, as the aging process is not linear: depending on their health and psychological context, their environment, and their social activity, an elderly individual can be classified into one of three identified profiles:

[0008] • “Robust” individuals: they are independent and do not have chronic illnesses. 55% to 60% of individuals are classified in this category.

[0009] • Individuals considered "frail" and "pre-frail" show signs of impairment in certain functional capacities, but these criteria for frailty can be addressed. Between 20% and 30% of individuals exhibit frailty, which, when detected and managed, can improve and lead to a return to robustness.

[0010] • People who are "dependent" require extensive and complex care. 5% to 10% of people are dependent.

[0011] The classification of a person's frailty therefore makes it possible to prevent a deterioration of their condition with the aim of maintaining them in the category of robust and autonomous people.

[0012] In the document Fried LP, Tangen CM, Walston J, Newman AB, Hirsch C, Gottdiener J, Seeman T, Tracy R, Kop WJ, Burke G, McBurnie MA; Cardiovascular Health Study Collaborative Research Group. Frailty in older adults: evidence for a phenotype. J Gerontol A Biol Sci Med Sci. 2001 Mar;56(3):M146-56. doi: 10.1093 / gerona / 56.3.m146. PMID: 11253156, the authors show that frailty can be defined as a clinical syndrome in which at least three of the following criteria were present: weight loss, exhaustion, weak grip strength, slow walking speed, and low physical activity. Therefore, frailty analysis using this technique requires numerous sensors and home visits from physicians to subjectively assess the individual being monitored, which proves to be very restrictive and intrusive.

[0013] In the document L. Wang, Y. Sun, Q. Li, T. Liu, and J. Yi, "IMU-Based Gait Normalcy Index Calculation for Clinical Evaluation of Impaired Gait," in IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 1, pp. 3-12, Jan. 2021, doi: 10.1109 / JBHI.2020.2982978, the authors estimate spatiotemporal gait parameters such as gait cycle duration and stride length using IMU (Inertial Motion Unit) sensors. This information allows for the quantification of gait abnormalities and an understanding of the effectiveness of therapy during the rehabilitation process. However, this technique lacks precision, and it is also difficult to correlate gait variables with a specific pathology. Thus, inference to a frailty category remains complex. Furthermore, equipping the person with multiple inertial sensors is too intrusive to detect vulnerabilities.

[0014] Therefore, there is a need for a technique that allows for the automatic estimation of a person's physical frailty, accurately and without constraints for the person being monitored.

[0015] 3. Description of the invention

[0016] This application relates to a method for estimating the physical frailty of a person not exhibiting the aforementioned disadvantages, comprising an estimation phase (E2), for at least one estimation period (Pe), of the person's physical frailty, implementing an analysis of images of the person's body movement and taking into account at least one database representative of body movements for the person during at least one learning period (Pa), referred to as the personalized database (Pa_DB_User), the estimation phase (E2) delivering at least one data point (C_User) representative of an estimated physical frailty of the person during the estimation period (Pe) and a service recommendation (S_User) adapted to the estimated physical frailty.

[0017] Thus, the proposed solution makes it possible to estimate the physical fragility of a person, in the medium and long term, through the use of an analysis of images of the movements of that person during an estimation period.

[0018] Furthermore, the use of a reference database associated with the person being monitored, obtained over a learning period, to obtain the estimate of frailty allows the detection of an evolution, or even a possible degradation, of the physical capacities of this person, thanks to a detailed knowledge of their behavior.

[0019] According to one particular aspect, images of the person's body movement are captured by at least one camera during the person's walking cycles.

[0020] Thus, the proposed technique is neither restrictive nor intrusive because it is based solely on images captured by one or more cameras (e.g., visible monovision, or stereovision, infrared or LiDAR), without requiring sensors worn by the person being tracked, unlike many prior art techniques.

[0021] According to a particular characteristic, the estimation phase (E2) includes an implementation of an artificial intelligence module (MOD-User) adapted to the person from said at least one personalized database (Pa_DB_User), and associating, to at least one representation of a plurality of body movements captured during a time window (T), said at least one data point (C_User) representative of an estimated physical frailty of the person.

[0022] Thus, the proposed technique relies on the implementation of an artificial intelligence module, more commonly known as a neural network, adapted to the individual being monitored through a learning phase. This module provides at least one piece of data representative of the individual's physical frailty, estimated from observations of their body movements. Because the neural network is adapted to the individual during a learning phase in which the individual was considered robust, it can very accurately detect any potential decline in their physical health and thus, if necessary, alert the system to the onset of physical frailty.

[0023] In particular, the process includes a prior learning phase (E1) implementing, during said at least one learning period (Pa), an artificial intelligence module (MOD) configured from at least one reference database (DB_Ref), the learning phase (E1) delivering said at least one personalized database (Pa_DB_User) associated with the person and said artificial intelligence module (MOD-User) adapted to the person.

[0024] Thus, the proposed technique proves to be highly reliable thanks to a series of examples allowing the learning of the neural network specific to the person being monitored, to deliver a personalized database which is then also used for estimating the frailty of that person.

[0025] For example, the learning phase (E1) includes:

[0026] - a first step (E1_1) of obtaining representative data of body movements for the person delivering said at least one personalized database (Pa_DB_User) comprising at least one graph (G(t)) representing a plurality of skeletal joints of the person, at a given time t, and at least one series (X(t)) of characteristics relating to at least one of the skeletal joints, called a multivariate time series,

[0027] - a second step (E1_2) of implementation of the artificial intelligence module (MOD) taking as input said at least one graph (G(t)) and said at least one multivariate time series (X(t)) and delivering at least one confidence measure Mi associated with at least one class among at least three classes R, F and D, respectively representing a status "Robust", "Fragile" and "Dependent" of said at least one person, the learning phase (E1) also delivering, at the end of the learning period (Pa), the artificial intelligence module (MOD-User) adapted to the person.

[0028] Thus, the specific neural network directly associates the movements of skeletal joints with a classification of the person as "robust", "fragile" or "dependent", notably through a representation in the form of graphs (spatial data) and multivariate temporal characteristics (data composed of several time-dependent variables) of the person's skeletal movements, allowing spatial and temporal data to be taken into consideration simultaneously.

[0029] Depending on a particular characteristic, the first step (E1_1) includes the following sub-steps:

[0030] - capture of images via said at least one camera;

[0031] - extraction, from the captured images, for each instant t, of said at least plurality of skeletal joints of the person in the form of at least one graph (G(t)) and of the characteristics relating to at least one of the skeletal joints in the form of at least one multivariate time series (X(t))

[0032] - application of at least one treatment on said at least one graph (G(t)) delivering at least one modified graph (G'(t)) and / or on said at least one multivariate time series (X(t)) delivering at least one multivariate time series (X'(t)).

[0033] For example, the second step (E1_2) implements the artificial intelligence module (MOD), for at least one time window T comprising three successive instants (t-1, t, t+1), including:

[0034] - a multi-scale learning (B) of motion features, from at least two pairs of multivariate time series {X(t-1), X(t)} and {X(t), X(t+1)}, delivering at least three levels of motion features;

[0035] - an adaptive learning (AL) taking into account the three levels of motion characteristics and at least the graphs G(t-1), G(t) and G(t+1) and delivering contextualized motion characteristics;

[0036] - multi-scale temporal spatial learning (MSTL) from contextualized motion features, coupled with an attention module delivering discriminating motion features;

[0037] - a classification (CL) of discriminating movement characteristics delivering said at least one confidence measure Mi associated with at least one class among at least three classes R, F and D, respectively representative of a "Robust", "Fragile" and "Dependent" status of said at least one person.

[0038] Thus, these different steps allow the neural network to adapt to the individual being monitored by classifying each temporal sequence of data collected during period Pa as "Robust." The process, according to this technique, is implemented during a period when the individual is not physically vulnerable. This learning phase therefore provides a robustness benchmark for the individual, enabling the detection, during the estimation phase, of any degradation relative to this "Robust" status.

[0039] According to a particular aspect, the estimation phase (E2) implements steps (E2_1) and (E2_2), corresponding to steps (E1_1) and (E1_2), by the artificial intelligence module (MOD-User) adapted to the person.

[0040] Thus, the estimation of the physical frailty of the person being monitored is based on the same steps as those implemented to adapt the neural network, but with movement data collected during an estimation phase, i.e. during which the person being monitored may show a deterioration in their condition, which will be detected by the adapted neural network.

[0041] For example, the data representing an estimated frailty (C_User) takes into account at least one class to which is associated a confidence measure Mimaximale during the estimation period Pe.

[0042] According to one particular aspect, the periodPa includes the periodPe.

[0043] Thus, according to this characteristic, the Pa period is dynamic, and the artificial intelligence module adapts more precisely to the individual being monitored each day, as each new day of monitoring enriches the training database associated with that person. This strengthens the accuracy of the frailty assessment.

[0044] For example, the service recommendation (S_User) adapted to the estimated physical frailty belongs to the group including at least:

[0045] - the sending, to a predetermined entity, of a notification including at least one piece of information representative of said estimated physical frailty; in this case, upon receipt of this notification, a service recommendation may be made, depending on the degree of estimated frailty and other parameters specific to the person being monitored (for example, the configuration of their accommodation, their degree of isolation, etc.);

[0046] - a decision to maintain an ongoing service for the person.

[0047] Thus, assessing the physical frailty of the individual being monitored allows for recommendations regarding support, such as implementing appropriate services like a fall detection alarm or regular home visits from a healthcare professional, as soon as any decline in the individual's strength is detected. This enables a more rapid response than prior art techniques, which do not rely on real-time analysis of a person's body movements, and also allows for a learning period to adapt the process to the individual being monitored.

[0048] This application also relates to a device for estimating the physical frailty of a person, configured to implement an estimation phase (E2), for at least one estimation period (Pe), of the person's physical frailty, implementing an analysis of images of the person's body movement and taking into account at least one database representative of body movements for the person during at least one learning period (Pa), referred to as a personalized database (Pa_DB_User), said estimation phase (E2) delivering at least one data point (C_User) representative of an estimated physical frailty of the person during the estimation period (Pe) and a service recommendation (S_User) adapted to the estimated physical frailty.

[0049] This application also relates to a computer program comprising instructions for implementing the various embodiments of the method described above, when the computer program is executed by a processor, and a recording medium readable by an electronic device and on which the computer program is recorded.

[0050] The program mentioned above may use any programming language, and be in the form of source code, object code, or code intermediate between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0051] The recording (or information) media referred to in this application may be any entity or device capable of storing the program. For example, a medium may include a storage means, such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording means.

[0052] Such a storage device could be, for example, a hard drive, a flash memory, etc.

[0053] On the other hand, an information medium can be a transmissible medium such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means. A program according to the invention can, in particular, be downloaded from a network such as the Internet.

[0054] Alternatively, an information carrier may be an integrated circuit in which a program is incorporated; in the present application, the circuit is adapted to execute or to be used in the execution of any of the embodiments of the method which is the subject of this patent application.

[0055] 4. List of figures

[0056] Other features and advantages of the invention will become more apparent upon reading the following description of a particular embodiment, given by way of illustrative and non-limiting example, and the accompanying drawings, among which:

[0057] This document presents an overview of the process for assessing a person's physical frailty, as outlined in this application, in some of its implementations.

[0058] This presents an overview of the artificial intelligence model implemented in some of the embodiments of this application,

[0059] This presents a simplified view of a device adapted to implement at least some embodiments of the process for estimating the physical frailty of a person in this application.

[0060] 5. Description of an embodiment of the invention

[0061] The general principle of this technique is based on the analysis of the body movements of a person, or a user, captured via one or more cameras while walking, in order to assess their physical frailty and, if necessary, to recommend the implementation of an appropriate service.

[0062] More specifically, this technique seeks to obtain an accurate and reliable estimate of the frailty of a person being monitored, based on knowledge of their robustness characteristics, in order to detect any deterioration in their physical health, in a simple, automatic and non-intrusive way (no need for sensors worn by the person being monitored, for example).

[0063] Thus, when a person moves freely within their home, the process using this technique tracks their movements over time and assesses, as they go, whether they are in a state of "frailty," "robustness," or "dependence." These point-by-point assessments inform medium- and long-term monitoring to analyze changes in the person's physical frailty and ultimately, if necessary, trigger a recommendation (for example, via a notification, a phone call, an addition to the medical file, etc.) for dependency support services at the appropriate time.

[0064] To achieve this, and in contrast to prior art techniques, the present solution is based on learning, through a series of examples, a specific neural network, also called an artificial intelligence model, inspired by so-called “ST-GNN” networks (for “Spatial Temporal Graph Neural Networks” in English, as described for example in the document ZA Sahili, M Awad, “Spatio-temporal graph neural networks: A survey”, arXiv preprint arXiv:2301.10569, 2023) which directly associates the body movements of the person being tracked with a classification according to the criteria “Robust”, “Fragile” or “Dependent” (described in the prior art part of this application).

[0065] Thus, the present technique is based on two phases, as illustrated for example in, according to any one of the embodiments: a learning phase E1, including in particular obtaining the body movement characteristics of a person followed during a learning period Pa and allowing the adaptation, to the person followed, of an artificial intelligence model used subsequently for the estimation of physical frailty, an estimation phase E2 of the physical frailty of the person followed implementing this adapted artificial intelligence model, by analyzing the body movements of the person followed on a daily basis.

[0066] As illustrated on the diagram, the learning and estimation phases each implement two major steps, respectively E1_1 / E1_2 and then E2_1 / E2_2, described below and allowing, during the estimation phase, to associate at least one representation of a plurality of body movements of the person being monitored, captured during a time window T, with at least one data C_User representative of an estimated physical frailty of the person.

[0067] The learning phase, illustrated in more detail on the [link / page], corresponds to a period, denoted as the learning period Pa, during which it is established that the monitored individual is robust and does not exhibit physical frailty. This learning phase will therefore not only allow the acquisition of baseline data for the monitored individual in a robust state but also allow the adaptation of the artificial intelligence model to the monitored individual, for its use in estimating, in the medium and long term, the monitored individual's physical frailty.

[0068] First, the analysis of the monitored person's body movements over time involves capturing images of the person in motion using one or more cameras installed in different locations within the person's home (and possibly also outside, within a perimeter around their home, for example). These images are then used to extract characteristics from the images at predefined times. To do this, time is discretized into instants, for example, every 10 ms. For the implementation of the MOD artificial intelligence model, at least one analysis window, for example, 30 ms, is then considered, comprising three instants and denoted T = {t-1, t, t+1}. This time discretization, and especially the use of analysis windows encompassing multiple instants, allows for an optimal estimation of the person's physical frailty, thanks to the consideration of temporal correlations between the studied characteristics.

[0069] In a first step E1_1, for each instant t, representative data of the joints of the skeleton of the person being monitored are extracted from the captured images and represented in the form of at least one graph G(t). Movement characteristics (e.g., velocity, acceleration, rotation, etc.) are also obtained, for a plurality of joints, in the form of at least one multivariate time series X(t).

[0070] To achieve this, a computer vision module, known but not described here, can be implemented. Other discretization values ​​can of course be chosen, as well as other values ​​for the temporal analysis window T, for example, depending on the characteristics of the vision module used.

[0071] According to the present technique, a multivariate time series consists of several variables, or dimensions, that are interdependent and dependent on time.

[0072] In one particular implementation, each analyzed joint is represented by a multivariate time series. This translates, for example, to a variation in the velocity of an elbow, a variation in the acceleration of a foot, a variation in the displacement of a hand, and so on. In this implementation, step E1_1 is considered to produce at least one graph G(t) and one set X(t) grouping several multivariate time series. For example, for a time window T = {t-1, t, t+1}, step E1_1 produces the graphs G(t-1), G(t), and G(t+1) and three sets X(t-1), X(t), and X(t+1). We will see later the importance of working with several graphs and several sets of multivariate time series, which is what makes this technique unique.

[0073] Depending on the implementation variants, one or more preprocessing steps are implemented on the graph(s) G(t) and / or the multivariate time series(s) X(t).

[0074] For example, one approach might be to simplify the graphs by retaining only certain joints considered more relevant for estimating physical frailty. Thus, it may be advantageous not to analyze all points in the upper body relative to points in the lower body, while still maintaining an overall distribution of the analyzed joints across the entire body. This segmentation, known as spatial segmentation, therefore corresponds to an implementation choice based on the desired accuracy for estimating physical frailty.

[0075] Another example of treatment may involve giving more or less importance to certain connections between joints, for example the different connections between the joints of the legs.

[0076] The idea is therefore to prioritize certain parts of the body over others, in order to simplify the model used while ensuring the robustness of this model.

[0077] With regard to multivariate time series X(t), denoising, filtering and / or normalization can be applied, these preprocessing steps again allowing the model to be optimized by providing it with the most suitable data.

[0078] Depending on the implementation methods used, the graph(s) G(t)(with or without preprocessing) and the multivariate time series(s) X(t)(with or without preprocessing) are stored in a database customized for the person being monitored, denoted (Pa_DB_User).

[0079] Thus, at this stage (i.e. at the end of the first step E1_1 of the learning phase), we have analysis windows where the person is known as "robust", which will serve as a reference for the E2 phase of continuous estimation of physical frailty, described below.

[0080] In a second step E1_2 of the learning phase, an artificial intelligence module, denoted MOD, is implemented to deliver at least one confidence measure Mi associated with at least one class from among at least three classes R, F, and D, respectively representing a "Robust," "Fragile," and "Dependent" status for the monitored individual, as described previously. These confidence measures Mi will then be used to obtain data representative of the monitored individual's fragility over a plurality of time windows T, as defined above.

[0081] This artificial intelligence module MOD takes as input the graphs G(t) and the multivariate time series X(t) described previously and also delivers, at the end of the learning period Pa, an artificial intelligence module adapted to the person being monitored, denoted MOD-User.

[0082] We now describe in detail the artificial intelligence module MOD, created by the inventor of this application, based on a known "STGCN" type neural network (for "Spatial Temporal Graph Convolutional Network" in English) but modified according to the present technique and whose originality lies both in a so-called Siamese learning module and a multi-scale attention.

[0083] Thus, the neural network implemented according to the present technique first works, for each time window T= {t-1,t,t+1}, with two different input vectors, in this case two pairs of multivariate time series {X(t-1), X(t)} and {X(t), X(t+1)} and on three scales of characteristics: low level, intermediate level and high level.

[0084] The MODa model is assumed to have already been learned on examples of frail, robust and dependent people, notably from a reference database (DB_Ref).

[0085] The purpose of the learning phase E1, and more specifically of the stage E1_2, is to adapt it to situations of the person being monitored, based on the data stored in the database personalized for the person being monitored (Pa_DB_User), at the end of the first stage E1_1 described above, by implementing the different modules described below.

[0086] For each window T, two pairs of multivariate time series {X(t-1), X(t)} and {X(t), X(t+1)} are fed into a first module B whose objective is to jointly extract, in the two multivariate time series of a pair, the important movement features. To do this, a module F (for "Feedforward Convolutional Network") is first implemented to reduce, in a known manner, the dimensionality of the input vectors, then a selection of three feature scales (low level, high level, and intermediate level) is implemented by a module M.

[0087] The originality of the approach here is to carry out joint learning with two multivariate time series to extract more robust and discriminating characteristics in three different levels of analysis (i.e. the scales), always with a view to a final classification of the person's movement as "robust", "fragile" or "dependent".

[0088] Thus, the MOD model according to the present technique makes it possible to take into account spatial correlations, i.e. between several joints, over time.

[0089] As an illustrative example, according to one embodiment of this technique, during the learning process of two multivariate time series concerning the two wrist joints of the tracked individual (spatial characteristics), information on speed, acceleration, or movement (direction, for example) is extracted jointly because these are correlated in the action of walking. Indeed, we naturally move our arms while walking to maintain balance, and the movement of the right arm is intrinsically linked to the left arm, which constitutes important data to extract within the framework of model learning.

[0090] Furthermore, learning about movement characteristics on different scale levels also makes the final classification more robust, by analyzing so-called low-level characteristics (e.g., a movement of an arm), intermediate-level characteristics (e.g., a correlation of the speeds of the right and left foot) and high-level characteristics (e.g., a correlation of the movement of the head at time -1 with the speed of the hand at time t and the acceleration of the knee at time +1).

[0091] It should be noted that this approach also makes it possible to detect a person's physical fragility regardless of body movements, and in particular the process described here can detect arm tremors, changes in general posture, ..., and not only body movements related to walking.

[0092] Next, for each analysis window T, the outputs of module B (i.e., the learned movement characteristics according to three scales) are introduced into an adaptive learning module, denoted L. The role of this module L is to introduce the information from the graphs G(t-1), G(t) and G(t+1), coupling them with the spatial, temporal and three-level scale characteristics from the multivariate series X(t), thus allowing us to obtain characteristics of the series X(t) in the hierarchical context of the graphs, called contextualized movement characteristics.

[0093] Indeed, one can, for example, consider in a graph representing the human body, a point on the head as the root and the joints as branches and leaves of a tree.

[0094] Thanks to this module L, connectivity links between the values ​​of the multivariate series X(t) are learned according to the three scales described previously, by three modules AL (for "Adaptive Learning") which take into account that the characteristics of each joint have more or less importance depending on their position in the graph. For example, characteristics of joints close to each other in the graph (e.g., hand and knee) are more correlated, i.e., more closely related when observing the movements of a walking person, than characteristics of joints far apart in the graph (e.g., head and foot). It is therefore advisable to promote the joint learning of highly correlated characteristics, i.e., to give more importance to their learning.

[0095] At this stage, still for each windowT, the contextualized movement characteristics (outputs of moduleL) are transmitted to a module notedMST (for "Multiscale Spatial Temporal" in English), performing spatial (at the level of each graph) and temporal learning based on a neural network of type "STGCN" to which the notion of the three scales has been added.

[0096] A spatiotemporal graph is known to be defined as a graph that captures temporal and event information of tracked objects or individuals, with the nodes of the graph representing the positions and timestamps of the objects / individuals, and the edges representing the relationships between positions, timestamps, and objects / individuals based on periodic similarities.

[0097] This MST module includes an ST module coupled with an attention module designed to focus attention on different values ​​of scale levels and time during the learning process.

[0098] The attention module corresponds to a learning strategy that can be described as follows: instead of learning all features in the same way all the time, the learning strategy varies over the course of the learning period to "focus its attention" on certain features and then others in order to better model the individual's behavior. For example, this attention module will learn, at time -1, low-level features of two nearby joints in the graph, then, at time t, high-level features of a single joint, and then, at time +1, intermediate-level features of three distant joints in the graph.

[0099] By definition, it is not possible to know, a priori, which characteristics will be chosen for each moment and for each person. It is the set of modules of the implemented artificial intelligence model that learns, for a monitored individual, what information is important in order to determine whether that person is frail, dependent, or robust.

[0100] Depending on the implementation variants, this MST module can be implemented several times to gain in final accuracy.

[0101] The originality of the approach is therefore to keep the notion of scale at the level of the learning of the neural network "STGCN" and therefore of the MST module, thanks to the attention module A which focuses its attention on the different levels of scales and allows to mix the important characteristics of low, medium and high levels and to deliver discriminating movement characteristics for the final classification.

[0102] The implementation of all these modules described above, during the learning phase, therefore makes it possible to adapt, in a unique and specific way, the artificial intelligence model MOD to the person being monitored, to deliver an adapted model MOD_USER, which will be implemented during the estimation phase described below.

[0103] Furthermore, based on the discriminating motion characteristics delivered by the MST module, and corresponding to the results of an in-depth processing in time, space and "semantic" scale level of the original values ​​of G(t) and X(t), always for each window T, a classification module CL is implemented to deliver the final classification via a confidence measure Mi associated with each of the three classes that interest us, namely "Robust", "Fragile" and "Dependent".

[0104] In one embodiment, this classification module (CL) is composed of layers called FCS (for "Flatten" and "Convolution") and a decision criterion (SA) (for "Softmax Argmax"). These are the final layers of the artificial intelligence model (MOD) that will enable the final classification into "Robust," "Fragile," and "Dependent," based on the useful characteristics extracted by all the preceding modules.

[0105] In this technical implementation choice, the CL classification module uses a flattening layer to concatenate the discriminating features, thereby converting multidimensional neural layers into a single-dimensional neural layer using a known method. A convolutional layer then extracts values ​​from this concatenation, which are then provided to a final layer of three neurons (for the three categories "Robust," "Fragile," and "Dependent"). As is known, a convolutional neural layer learns features through the optimization of a filter / kernel.

[0106] The decision criterion SA stipulates that the most activated final neuron (among the three) determines the frailty class from among the classes "Robust," "Fragile," and "Dependent" for the given time window T. For example, the first neuron is considered to correspond to the "Fragile" class, the second to the "Dependent" class, and the third to the "Robust" class. If the first neuron is more activated (by the final characteristics presented to it) than the second and third, this means that the characteristics extracted by all the preceding modules all contribute to concluding that the monitored individual is in a situation of frailty.

[0107] This is made possible by assigning an activation value to each neuron, for example, a value between 0 and 1. This activation value of a neuron can therefore be considered a confidence measure (Mi), assigned to the class corresponding to the neuron in question. In this example, where the first neuron is the most activated, the confidence measure (Mi) assigned to the "Fragile" class (corresponding to the activation of the first neuron) is therefore greater than the confidence measures associated with the other classes.

[0108] Furthermore, the greater the difference in value of the confidence measures assigned to the three classes, the more relevant the decision to classify the person being monitored into one or the other of the statuses.

[0109] In the example, if the first neuron (corresponding to a "Fragile" individual) is activated with a value of 0.9, then the estimation of the "Fragile" status of the person being monitored is 90% relevant.

[0110] However, in another example, if the activation value of the first neuron is 0.34, then the confidence measure only represents a one in three chance and is not relevant, on its own, for the final classification. In this case, the model does not decide between the three classes "Robust," "Fragile," and "Dependent" to avoid a false alarm, and an "unknown" status is preferred.

[0111] According to the present technique, this final classification is possible because the representative data of the estimated frailty C_User of the person being monitored, at the end of the time window T, takes into account these confidence measures, as described in more detail below, in relation to the estimation phase E2.

[0112] At the end of this learning phase E1, the modelMODa has therefore finished learning the examples of the person followed during the learning period Pa, and is adapted to the person followed and can be used, in the form of an artificial intelligence model MOD_USER, for the estimation phase, described below, during an estimation period Pe.

[0113] This estimation phase E2 includes steps equivalent to those of the learning phase, i.e. a step E2_1 of collecting movement data of the person being tracked, as it happens during the estimation period Pe, and a step E2_2 of implementing the artificial intelligence model MOD_USER from the collected movement data.

[0114] Thus, step E2_1 consists, like step E1_1, of capturing motion images of the person being tracked, extracting features from them in the form of both graphs and multivariate series, for each predefined time window (for example, 30 ms windows including features for three 10 ms instants as described previously).

[0115] For example, for each 30ms time window, we have three graphs and three multivariate series of associated motion characteristics which will be used by the MOD_USER model, implemented in an E2_2 step, to determine a class associated with each window, from among the three predefined classes "Robust", "Dependent" and "Fragile".

[0116] Step E2_2 is not described in detail again, as the modules implemented are the same as for step E1_2.

[0117] At the end of this step E2_2, a confidence measure Mi is associated with each class for each analysis time window, and a data point C_User representing the estimated frailty of the monitored individual is determined from these confidence measures Mi. To do this, as already described above, the confidence measures Mi are analyzed and compared in order to determine the status assigned to the given time window.

[0118] Thus, in this example, each 30ms window is labeled as "Robust", "Fragile" or "Dependent", i.e. a C_Userest data assigned to each window.

[0119] The estimation phase then consists of taking into account this C_User data, for a series of windows over a day, a week or a month, in order to estimate the evolution of the frailty of the person being monitored and deliver a service recommendation S_User, adapted to the estimated physical frailty.

[0120] To achieve this, several approaches are considered for assessing frailty over the period Pe, for example, by estimating the physical frailty of the monitored individual based on the majority of C_User data assigned to each window. Another approach involves setting a threshold on the C_User data assigned to each window: if the number of windows classified as "Frail" or "Dependent" exceeds a predetermined threshold, a shift towards the "Frail" status is detected for the monitored individual.

[0121] Next, the process determines a service recommendation (S_User) tailored to the previously estimated physical frailty. This service recommendation depends not only on the estimated physical frailty but also on how that estimated physical frailty changes over time. For example, if the monitored individual's status changes from "Robust" to "Dependent" or "Frail" from one day to the next, or over a longer period (e.g., a week with some days in "Robust" status and some days in "Dependent" or "Frail" status), then the service recommendation (S_User) is to issue an alert notification, for example, to the monitored individual themselves, or to their attending physician, or to a previously designated third party.

[0122] On the other hand, if no deterioration in the physical frailty of the person being monitored is detected, i.e., they remain in a "Robust" or even "Dependent" status, then the service recommendation may consist of disseminating information about the status quo, or updating the health record of the person being monitored…

[0123] Learning by a MOD model such as described above, the temporal and spatial links between movement characteristics allow for more precise and robust estimation of a person's frailty over each time window of the estimation phase.

[0124] Moreover, and advantageously, each new day of monitoring feeds into the history of the person being monitored and the learning period therefore evolves over time, making the neural network and thus the process according to the present technique even more precise and robust.

[0125] Finally, in relation to this, we present a simplified structure of a device for estimating the physical frailty of a person according to an embodiment of the invention.

[0126] As illustrated in, the device for estimating the physical frailty of a person according to an embodiment of the invention comprises a memory M, a processing unit, equipped for example with a programmable computing machine or a dedicated computing machine, for example a processor P, and controlled by a computer program Pg, implementing steps of a method for estimating the physical frailty of a person as described above, according to at least one of the different embodiments.

[0127] At initialization, the code instructions of the computer program Pg are, for example, loaded into RAM memory before being executed by the processor of the processing unit P.

[0128] The processor of the processing unit P implements steps of the process for estimating a person's physical frailty described above, according to the instructions of the computer program Pg, to implement an estimation phase, for at least one estimation period, of the person's physical frailty, implementing an analysis of images of the person's body movement and taking into account at least one database representative of body movements for the person during at least one training period, called the personalized database, the estimation phase delivering at least one data point representative of an estimated physical frailty of the person during the estimation period and a service recommendation adapted to the estimated physical frailty.

Claims

A method for remotely monitoring a person's physical frailty, characterized in that it comprises the following steps: - estimation (E2), for at least one estimation period (Pe), of said physical frailty of said person, implementing an analysis of body movement images of said person and taking into account at least one database representing body movements for said person built during at least one learning period (Pa), referred to as the personalized database (Pa_DB_User); - delivery of at least one data point (C_User) representative of said estimated physical frailty of said person during said estimation period (Pe) in the form of a class among at least the three classes R, F and D, respectively representing a status of "Robust", "Frail" and "Dependent" of said person; and - triggering, of a service recommendation (S_User) adapted to said estimated physical frailty. Method according to claim 1, characterized in that said images of the body movement of said person are captured by at least one camera during the walking cycles of said person. A method according to any one of claims 1 to 2, characterized in that said estimation (E2) comprises an implementation of an artificial intelligence module (MOD-User) adapted to said person from said at least one personalized database (Pa_DB_User), and associating, to at least one representation of a plurality of body movements captured during a time window (T), said at least one data point (C_User) representative of an estimated physical frailty of said person. A method according to claim 3, characterized in that it comprises a prior learning phase (E1) implementing, during said at least one learning period (Pa), an artificial intelligence module (MOD) configured from at least one reference database (DB_Ref), said learning phase (E1) delivering said at least one personalized database (Pa_DB_User) associated with said person and said artificial intelligence module (MOD-User) adapted to said person. A method according to claim 4, characterized in that said learning phase (E1) comprises: - a first step (E1_1) of obtaining said representative body movement data for said person, delivering said at least one personalized database (Pa_DB_User) comprising at least one graph (G(t)) representing a plurality of skeletal joints of said person at a given time t, and at least one series (X(t)) of features relating to at least one of said skeletal joints, said multivariate time series; - a second step (E1_2) of implementing said artificial intelligence module (MOD) taking as input said at least one graph (G(t)) and said at least one multivariate time series (X(t)) and delivering at least one confidence measure Mi associated with at least one class among at least the three classes R, F, and D, respectively representing a "Robust", " Fragile and Dependent on at least one person,said learning phase (E1) also delivering, at the end of said learning period (Pa), said artificial intelligence module (MOD-User) adapted to said person. A method according to claim 5, characterized in that said second step (E1_2) implements said artificial intelligence module (MOD), for at least one time window T comprising three successive instants (t-1, t, t+1), comprising: - a multi-scale learning (B) of motion features, from at least two pairs of multivariate time series {X(t-1), X(t)} and {X(t), X(t+1)}, delivering at least three levels of motion features; - an adaptive learning (AL) taking into account said three levels of motion features and at least the graphs G(t-1), G(t) and G(t+1) and delivering contextualized motion features; - a multi-scale spatial-temporal learning (MSTL) from said contextualized motion features, coupled with an attention module delivering discriminating motion features;- a classification (CL) of said discriminating movement characteristics delivering said at least one confidence measure Mi associated with at least one class among at least three classes R, F and D, respectively representative of a "Robust", "Fragile" and "Dependent" status of said at least one person.; Method according to claim 6, characterized in that said estimation phase (E2) implements steps (E2_1) and (E2_2), corresponding to said steps (E1_1) and (E1_2), by said artificial intelligence module (MOD-User) adapted to said person. Method according to claim 7, characterized in that said data representing an estimated fragility (C_User) takes into account at least one class to which is associated a confidence measure Mimaximale during said estimation period Pe. A method according to any one of claims 1 to 8, characterized in that said periodPa includes said periodPe. A method according to any one of claims 1 to 9, characterized in that said service recommendation (S_User) adapted to said estimated physical frailty belongs to the group comprising at least: - an issuance, to a predetermined entity, of a notification including at least one piece of information representative of said estimated physical frailty; - a decision to maintain an ongoing service for said person. A remote monitoring device for a person's physical frailty, characterized in that said device is configured to implement - an estimation (E2), for at least one estimation period (Pe), of the physical frailty of said person, said estimation (E2) implementing an analysis of body movement images of said person and taking into account at least one database representative of body movements for said person built during at least one training period (Pa), said personalized database (Pa_DB_User); the delivery of at least one data point (C_User) representative of said estimated physical frailty of said person during said estimation period (Pe) in the form of a class among at least the three classes R, F and D, respectively representative of a "Robust", "Frail" and "Dependent" status of said person;and- a triggering of a service recommendation (S_User) adapted to said estimated physical fragility. Computer program comprising instructions for carrying out the method according to any one of claims 1 to 10, when the computer program is executed by a processor. Recording medium readable by an electronic device and on which the computer program according to claim 12 is recorded.

Images