CONTINUOUS AI SCREENING OF MULTI-PATHOLOGIES WITH QUALITATIVE / QUANTITATIVE / KINETIC MEASUREMENTS OF THE ARRIVAL OF ANY PATHOLOGY WITH RESPECT FOR PRIVACY.

Abstract

This is a monitoring approach that respects privacy (no routine biometric recognition) to continuously extract multimodal markers from individuals / patients. The system can use various modules, including cameras, microphones, connected objects, wearables, sensors, etc . . .. This allows for the early detection of ANY pathology whatsoever that can be detected in the continuous extraction of multimodal markers from individuals / patients and the rapid transmission of alerts to emergency medical services. The multimodal monitoring supported by AI algorithms will not only detect the QUALIFICATION of the onset of a pathology, but especially the QUANTIFICATION of this onset of pathology, and even more so to provide a KINETIC MEASURE OF THIS QUANTIFICATION over a significant period of time.ONLY in cases where a detected pathology requires urgent and absolute medical intervention, and if the precision levels of the methods employed have been validated by medical professionals, the approval of the protocol's performance by the medical authority can trigger the use of biometric recognition. The use of biometric recognition allows for access to the patient's medical history, which can strengthen the diagnosis to send an alarm AND / OR accelerate patient care by already having knowledge of their medical files upon arrival in the emergency room. THIS IS A CASE OF FORCE MAJEURE THAT REQUIRES URGENT TREATMENT, GOVERNED BY MEDICAL ETHICS, AND THEREFORE EXEMPT FROM ANY PRIOR AUTHORISATION.The use of biometric recognition may also be employed without prior approval from an authority ( MEDICAL OR OTHERWISE ), if and only if the state reliably detected by multimodal monitoring falls within the framework of the "OBLIGATION TO ASSIST A PERSON IN DANGER" according to the LAW, biometric recognition can then be used without prior authorization if it provides assistance in the diagnosis. Outside of the two cases mentioned above, "FORCE MAJEURE" AND / OR "OBLIGATION TO ASSIST A PERSON IN DANGER", the "Life Tracking Medicine", (LTM) which gives citizens the ability to create their own data heritage, allows for the CONSIDERATION OF THE PERSON’S PRIOR CONSENT at the time of the device's activation, ONLY IN CASE OF SUSPICION OF THE ARRIVAL / DETECTION OF ANY PATHLOGY, and without prior approval from an authority (MEDICAL OR OTHERWISE), for the use of biometric recognition.This case also includes persons who, being minors or lacking the capacity to make decisions, are represented by a Parent or legal guardian. In this context, the initial consent of the legal guardian will be taken into account, only in case of suspicion of the arrival / detection of any pathology.

Description

Title of Invention: LIFE TRACKING MEDICINE - EARLY DETECTION OF BEHAVIORAL PATHOLOGIES BY SMART ALGORITHM-GUIDED MONITORING WITH PRIVACY PRESERVATION Summary

[0001] Many pathologies that are on the rise today, such as Parkinson's disease, Alzheimer's and other dementias, stroke, heart failure, heart attack and other coronary diseases, epilepsy, sepsis, Asthma attack, etc., could be detected early in order to take préventive action that could save lives. With the expansion of artificial intelligence, intelligent monitoring of people's behavioral tracking can detect the occurrence of disorders in gestures or language, allowing to predict a suspicion of the onset of a targeted pathology in the future. This intelligent monitoring must remain discreet and must respect the privacy of individuals. Existing artificial intelligence algorithms can be used to monitor individuals, but in order to be accurate enough, these algorithms must identify the individuals, typically through biométries. This allows for fine-grained détection of changes in the gestures or language of the targeted individual.

[0002] Each individual has their own unique gestures and facial / vocal expressions. Without integrating these personal characteristics into monitoring algorithms for early détection of a targeted pathology, this lack of information will resuit in a higher number of false positives.

[0003] Let’s take the example of the onset of a stroke. A significant proportion of people who expérience a stroke are characterized by paralysis on one side of the face, which can be detected by measuring facial asymmetry. However, it is scientifically recognized that 95% of people who are considered normal hâve a slight facial asymmetry, which can be more or less pronounced. Multi-neuronal classifiers in artificial intelligence, through deep leaming between two populations, one without stroke and the other with stroke, will be able to predict if a person has a stroke.

[0004] However, this approach has at least two flaws: 1. We are not sure that a person who is detected as having a stroke with this method is not actually a person without a stroke who has a natural facial asymmetry. Therefore, there will be an inévitable number of false positives. 2. In the deep leaming of the multi-neuronal classifier, between the two populations, one said to be normal, and the other with stroke, it is a matter of photos or videos of people POST-Stroke, because it is impossible to hâve a large population of people AT THE TIME WHERE THE STROKE HAPPENS. And in early détection, facial asymmetry can evolve.

[0005] The approach of identifying people by biométrie récognition is relevant for per-sonalizing the évolution of measurements over time for the targeted person, but goes against the manv rules of government institutions that regulate the protection of people's privaev.

[0006] Our procedure, which we claim to be innovative, has a different approach: 1. Multimodal Monitoring, which can be done with different modules, caméras, microphones, connected objects, wearables, sensors, etc., does not identify people for comparison with their previous behavior but is based on deep leaming between two populations, as indicated above, OR on mathematical équations measuring gestural or vocal expressions to obtain values to dissociate the two populations (one with the targeted pathology and the other without). 2. In the context where, as described above, this monitoring is based on mathematical and / or logical équations (and, or, xor, etc.) and / or other QUALITATIVE / QUANTITATIVE processes, if there is no détection of anomaly over a significant period of time, the values obtained on the monitored person during this period by the equations / processes targeting a pathology will be taken into account. These customized values will allow for a more individualized monitoring of the person for a targeted pathology, in order to hâve a more accurate and efficient détection. In this case, there is no biométrie détection to identify the individual. 3. In the context where, as described above, multimodal monitoring supported by AI algorithms will not only detect the QUALIFICATION of the onset of a pathology, but especially the QUANTIFICATION of this onset of pathology (to provide a scale in the measurement), and even more so to provide a KINETIC MEASURE OF THIS QUANTIFICATION (évolution of the pathology measurement, in the first minutes or over a significant period of time). The three types of measures we claim apply to ANY pathology, regardless of its nature, détectable through the continuous extraction of multimodal markers from individuals / patients. 4. In the context where, as described above, the multimodal monitoring supported by AI algorithms to detect the qualification / quantification / kinetic measurement of the onset of a pathology has précision levels validated by medical professionals, the approval of the protocol's performance by the medical authority can trigger the use of biométrie récognition. The use of biométrie récognition allows for access to the patient's medical history, which can strengthen the diagnosis to send an alarm AND / OR accelerate patient care by already having knowledge of their medical files upon arrivai in the emergency room. 5. The use of biométrie récognition may also be employed without prior approval from an authority (MEDICAL OR OTHERWISE). if and only if the methods used hâve already proven effective in detecting a person's agonizing State, or a prolonged fainting spell without regaining consciousness, an epileptic State unresponsive to any extemal stimuli, or a serious and imminent danger that threatens the person's bodily integrity or moral well-being (distress). 6. The use of biométrie récognition may also be employed in considering the person's prior consent at the time of the device's activation, ONLY IN CASE OF SUSPICION OF THE ARRIVAL / DETECTION OF ANY PATHLOGY, and without prior approval from any authority (MEDICAL OR OTHERWISE). 7. Biométrie récognition will never be used in routine Multimodal Monitoring. Biométrie récognition will only be used in the event of reliable détection of the arrivai of a targeted pathology according to procedures 4) 5) and 6) described above.

[0007] This particular procedure allows for greater compliance with current régulations in order to protect people's privacy. Short description of the drawings

[0008] Exemplar embodiments of the invention are disclosed in the description and il-lustrated by the drawings in which: 1. [Fig.l] illustrâtes the movements of both ankle joints of a person walking normally. 2. [Fig.2] illustrâtes the movements of both ankle joints on a post-stroke in- dividual. 3. [Fig.3] shows the landmarks of the Mouth on a Face without Facial Déviation. 4. [Fig.4] shows an example of recalculating the positions of the landmarks in [Fig.3] to simulate an asymmetry of the mouth. 5. [Fig.5] shows a face without characterized asymmetry making it possible to use this template in order to diagnose with personal parameters the arrivai of a stroke. 6. [Fig.6] shows a morphological transformation of the face of the [Fig.5] making it possible to simulate the arrivai of a stroke, to quantify it and evaluate the kinetic évolution of the pathology. Examples of embodiments of the présent invention

[0009] A computerized method for estimating a likelihood of a stroke condition of a user is performed by a computerized monitoring System comprising at least one processor and at least one monitoring device including an image capture device, having a field-of-view (FOV), and an audio capture device.

[0010] To illustrate an example of facial characteristics of a user, the facial characteristics comprise a plurality of landmark points LP (coordinates) corresponding to characteristics of the user's face. The feature F can be calculated from the landmark points LP in order to assess facial asymmetry of at least a portion of the user's face.

[0011] In one particular example, identifying at least 486 landmark points LP and cal-culating at between 15 and 36 features F.

[0012] For example, a linear équation can be performed on the features F. In that case, each feature F can be multiplied by a weighting factor. The weighting factor allows for each feature F to hâve the substantially same représentation value. In particular, assuming a linear équation as follows: W1*F1 + W2*F2 + W3*F3 + ... WN-1*FN-1 + WN*FN Eq.l where Wi is the weighting factor multiplying the feature Fi, in the case of i=N features. The features Fl to FN may take different values. For example, 20 if the feature Fl corresponds to the ratio of between the ends of the mouth and the pupils, then Fl can be equal to 0.05. If the feature F2 corresponds to the ratio of the distance between the middle of the mouth and lip ends, then F2 can be equal to 0.23, etc. In this example, the feature F2 is about four times greater than the feature Fl and will dominate in équation 1. 25 Equation 1 would then be unbalance yielding a misleading resuit.The weighting factor Wi multiplying the feature Fi thus allows the features to hâve about the same weight in the linear équation. For example, if Fl is equal to 0.05 and F12 is equal to 0.23, W1 can take the value of 4.6 and W1 can take the value of 1. .

[0013] It should be noted that, the features F calculated with the positive stroke dataset 20 and the négative stroke dataset 21 can détermine the threshold value Tv.

[0014] If, over a significant period of time, the features F calculated on the monitored person are always below the threshold value Tv, this value may be lowered in order to hâve a more précisé and efficient personalized détection.

[0015] The features F calculated on the monitored person will also allow for the mea-surement a QUANTIFICATION of the onset of a stroke (a scale in the measurement of facial asymmetry), and a KINETIC MEASURE of this QUANTIFICATION (example: évolution of facial asymmetry, in the first minutes or over a significant period of time).

[0016] For example, in our case of a positive and négative stroke dataset comprising 104 videos, an accuracy higher than 92% was obtained.

[0017] However, this QUANTITATIVE / QUALITATIVE / KINETIC measurement approach can be applied to ail types of pathologies that can be monitored in real time.

[0018] Another example: detecting gait balance and its détérioration. This détection can be done using various means: Caméra with motion analysis of the skeleton (MediaPipe Holistic to extract human pose landmarks), Wearables to detect lower limb movements, Sensors positioned at key movement points (ankles, knees, etc.).....

[0019] The [Fig. 1] illustrâtes the movements of both ankle joints of a person walking normally, as captured using the Mediapipe Holistic library and a caméra. This graph represents 2 plots, one corresponding to the kinetic readings of the right ankle and the other of the left ankle. The graph clearly demonstrates the synchronized movement between the left ankle and the right ankle. The [Fig.2] now depicts the same type of measurement of the [Fig.l] but on a post-stroke individual. The graph clearly shows a marked asymmetry between the left and the right ankle joints.

[0020] To quantify this asymmetry, we can calculate the standard déviation of the intersection of the two curves representing the movement kinetic s on the left and right sides. The experiment demonstrated that this standard déviation has a constant and low level in normal walking, while it is significantly higher in unbalanced walking.

[0021] This assessment therefore provides a means of quantifying gait quality on a scale. In addition to assessing gait quality at a single point in time, it allows us to monitor changes in gait quality over time (by measuring the kinetic évolution of the pathology). This can be used, for example, to intervene before a person falls. This monitoring ca-pability has the potential to improve patient outcomes by enabling timely interventions.

[0022] As these measures demonstrate a satisfactory level of accuracy in the early détection of stroke, which is an absolute medical emergency, the approval of the protocoPs performance bv the medical authoritv can trigger the use of biométrie récognition.

[0023] The use of biométrie récognition allows for access to the patient's medical history, which can strengthen the diagnosis AND / OR accelerate patient care by already having knowledge of their medical files upon arrivai in the emergency room.

[0024] THIS IS A CASE OF FORCE MAJEURE THAT REQUIRES URGENT TREATMENT, GOVERNED BY MEDICAE ETHICS. AND THEREFORE EXEMPT FROM ANY PRIOR AUTHORISATION.

[0025] Innovative method for obtaining a dataset of people with simulated the onset of a pathology

[0026] The methods for leaming a multi-neuronal classifier or obtaining a threshold value for distinguishing a population with or without a pathology will be more effective if the volume of these two populations is high. This type of population is even more difficult to obtain because datasets are generally collected post-pathology, rather than with the progressive onset of a targeted pathology. However, the symptoms that identify a targeted pathology are known, such as in the occurrence of a stroke, a progressive asymmetry of the mouth, a disproportion between the left and right eye, and an imbalance in walking.

[0027] Thanks to the évolution of algorithms related to artificial intelligence, we are able to detect facial keypoints as well as the full body skeleton, which allows us to monitor people's gestures, with examples of libraries such as MEDIAPIPE, DLIB, etc. Based on these 3D coordinates of points, we hâve, knowing the targeted pathological traits, been able to recalculate these coordinates to simulate the onset of a pathology in the video of a moving person.

[0028] This innovative motion décomposition is jointly implemented by morphing algorithms to achieve a realistic rendering.

[0029] This modification on each frame of a video can be decomposed into 8 steps: 1. Find Landmark Points of the Image in each frame, from example with DLIB or MEDIAPIPE libraries. 2. Recalculate the targeted points on the pathology concerned (e.g. stroke: mouth déviation, eye orbit disproportion,...). For example, the [Fig.3] shows the landmarks of the Mouth on a Face without Facial Déviation. The [Fig.4] shows an example of recalculating the positions of the landmarks in [Fig.3] to simulate an asymmetry of the mouth. 3. Triangulation Source Image (Delaunay Triangulation) 4. Triangulation of the Destination Image with the recalculated points 5. Extract And Warp Triangles 6. Link The Warped Triangles Together 7. Replace The Face On The Destination Image 8. Seamless Cloning

[0030] This is an example that we hâve successfully tested. Other morphing méthodologies can be used, but the key is step 2, the recalculation of the targeted points on a pathology concerned, which is an innovation.

[0031] Resuit example: The [Fig.5] shows a face without characterized asymmetry making it possible to use this template in order to diagnose with personal parameters the arrivai of a stroke. [Fig.6] shows a morphological transformation of the face of the [Fig.5] making it possible to simulate the arrivai of a stroke, to quantify it and evaluate the kinetic évolution of the pathology.

[0032] Conversely, it is possible to hâve videos of people with a stroke recalculate the keypoints in order to hâve a transition with a symmetrical face.

Claims

Claims

1. A computerized method for estimating a likelihood of a pathology disease of a user, such as Parkinson's disease, Alzheimer's and other dementias, stroke, heart failure, heart attack, and other coronary diseases, epilepsy, sepsis, Asthma attack, etc., the computerized method being performed by a computerized monitoring System comprising at least one processor and at least one monitoring device, such as caméras, microphones, connected objects, wearables, sensors, etc. The list of pathologies mentioned above is not exhaustive: ANY pathology whatsoever that can be detected in the continuous extraction of multimodal markers from individuals / patients falls within the scope of the claims cited in this publication.

2. The computerized method according to claim 1, will be able to detect behavioral abnormalities associated with an early suspicion of a targeted pathology, whether neuronal or motor, based on multi-neuronal classifiers in artificial intelligence, through deep learning between two populations, OR / AND on mathematical / logical équations measuring gestural or vocal expressions to obtain values to dissociate the two populations (one with the targeted pathology and the other without).

3. The computerized method according to claim 2, in order to improve the early détection of a nascent pathology, in knowledge of the gestural weaknesses that constitute this pathology, innovative algorithms hâve been created to reconstruct this occurrence of pathology by morphing simulation.

4. The computerized method according to claim 2 and 3, the monitoring based on mathematical and / or logical équations (and, or, xor, etc.) and / or other QUALITATIVE / QUANTITATIVE processes, if there is no détection of anomaly over a significant period of time, the values obtained on the monitored person during this period by the équations / processes targeting a pathology will be taken into account. These customized values will allow for a more individualized monitoring of the person for a targeted pathology, in order to hâve a more accurate and efficient détection. In this case, there is no biométrie détection to identify the individual.

5. The computerized method according to claim 2, 3 and 4, multimodal monitoring supported by AI algorithms will not only detect the QUAL-. IFICATION of the onset of a pathology, but especially the QUANTIFICATION of this onset of pathology (to provide a scale in the measurement), and even more so to provide a KINETIC MEASURE OF THIS QUANTIFICATION (évolution of the pathology measurement, in the first minutes or over a significant period of time). The three types of measures we claim apply to ANY pathology, regardless of its nature, détectable through the continuous extraction of multimodal markers from individuals / patients.

6. The computerized method according to any one claim 1 to 5, if the multimodal monitoring supported by AI algorithms to detect the qualification of the onset of a pathology has précision levels validated by medical professionals, the approval of the protocol's performance by the medical authority can trigger the use of biométrie récognition.The use of biométrie récognition allows for access to the patient's medical history, which can strengthen the diagnosis to send an alarm AND / OR accelerate patient care by already having knowledge of their medical files upon arrivai in the emergency room.

7. The computerized method according to any one daims 1 to 5, the use of biométrie récognition may also be employed without prior approval from an authority (MEDICAE OR OTHERWISE). if and only if the methods used hâve already proven effective in detecting a person's agonizing State, or a prolonged fainting spell without regaining consciousness, an epileptic State unresponsive to any external stimuli, or a serious and imminent danger that threatens the person's bodily integrity or moral well-being (distress).

8. The computerized method according to any one daims 1 to 5, outside of the two cases mentioned above, claim 5 "FORCE MAJEURE" AND / OR claim 6 "OBLIGATION TO ASSIST A PERSON IN DANGER", the "Life Tracking Medicine", (LTM) which gives citizens the ability to create their own data héritage, allows for the considération of the person's prior consent at the time of the device's activation. ONLY IN CASE OF SUSPICION OF THE ARRIVAL / DETECTION OF ANY PATHLOGY, and without prior approval from an authority (MEDICAL OR OTHERWISE), for the use of biométrie récognition. This case also includes persons who, being minors or lacking the capacity to make decisions, are represented by a Parent or legal guardian. In this context, the initial consent of the legal guardian will be taken into account, only in case of suspicion of the.

9. arrival / detection of any pathology. The computerized method according to claim 6, 7 and 8, Biométrie récognition will never be used in routine Multimodal Monitoring. Biométrie récognition will only be used in the event of reliable détection of the arrivai of a targeted pathology according to claim 6, 7 and 8 described above

Images