Support system to provide diagnostic information

Abstract

Diagnostic Information Support System The present invention relates to a diagnostic information support system, particularly for the detection of a disease in a person, especially a contagious disease such as COVID-19, this detection system comprising: a device for acquiring examination data on the person, this acquisition device comprising at least two sensors for measuring the same physiological measurement such as radar and a thermal camera for acquiring this examination data, these sensors being arranged in particular to operate without contact with the person, a data processing device arranged to receive this data obtained by the sensors of the acquisition device,This processing device is further arranged to perform a fusion of data from at least the two sensors of the acquisition device, particularly with a view to increasing the robustness of the examination data acquired on the person. Figure for the abbreviation: Fig. 5,

Description

Title of the invention: Assistance system for providing diagnostic information

[0001] The present invention relates to an assistance system for providing diagnostic information, in particular for the detection of a disease in a person, in particular a contagious disease such as COVID 19.

[0002] Detection of potential COVID-19 patients is currently carried out by taking temperatures, additional clinical examinations, and then by a COVID-19 test, which still has a relatively low level of confidence. A lung scan can confirm the severity of the disease with a good level of accuracy.

[0003] There is a significant need to be able to rapidly detect, in particular using mobile means, potential patients with contagious diseases, for example the disease linked to COVID 19.

[0004] The invention thus relates to a system for detecting a disease in a person, in particular a contagious disease such as COVID 19, this detection system comprising:

[0005] - a device for acquiring examination data on the person, this device acquisition system comprising in particular at least one physiological measurement sensor such as a radar, and a thermal camera to acquire this examination data,

[0006] - a data processing device arranged to receive these examination data obtained by the acquisition device,

[0007] - a display device arranged to display diagnostic information of the disease based on an analysis of said examination data, this diagnostic information being able to be representative of a level of probability that the person has the disease.

[0008] According to one aspect of the invention, the data processing device comprises an artificial intelligence unit arranged to process the examination data obtained by the acquisition device and provide said diagnostic information.

[0009] According to one aspect of the invention, the data processing device and the display device are part of the same device, for example a computer, in particular a laptop computer.

[0010] In this case, the display device is a screen of the laptop computer and the data processing device comprises a microprocessor of this computer.

[0011] In a variant, the display device is arranged to be visible to the person being examined, and in particular the display device, which notably comprises a screen, is remote from the data processing device, these display devices and data processing systems being, for example, connected to each other by a wireless connection, for example by a 3G, 4G or 5G communication protocol, or by the Internet network, or Wifi for example.

[0012] According to one aspect of the invention, the display device, and possibly also the data processing device, are arranged to be mounted on a motor vehicle.

[0013] Alternatively, the display device, and possibly also the data processing device, are arranged to be placed in a fixed manner, in a building or an outdoor courtyard for example.

[0014] According to one aspect of the invention, the processing device is arranged to make the diagnosis automatically, without human intervention.

[0015] Due to automatic processing, the invention allows rapid diagnosis and / or mass screening allowing a return to work or faster deconfinement.

[0016] According to one aspect of the invention, the examination data acquisition device is arranged to allow contactless and safe distance measurements of vital signals and thermal and visible imaging on the person to be examined.

[0017] The system according to the invention thus advantageously uses the fusion of contactless and safe distance measurements, vital signals and thermal and visible imaging.

[0018] According to one aspect of the invention, the artificial intelligence unit is arranged to use a diagnostic model based on artificial intelligence and fed by a reasonable number of clinical measurements.

[0019] The invention, due to its relatively light equipment, allows in particular easy deployment of field hospitals to support populations.

[0020] The invention allows for diagnostics to be carried out in a mobile manner. Its implementation, for example using a thermal camera, a physiological measurement sensor such as a radar and a portable personal computer, is easy. The invention allows for its rapid implementation throughout an entire territory.

[0021] According to one aspect of the invention, the acquisition device comprises a radar for acquiring data relating to the vital signs of the person, a thermal camera for temperature measurements providing temperature data, and a camera operating in the visible spectrum for characterizing the person tested providing characterization data of the person.

[0022] According to one aspect of the invention, the data processing device is arranged to run a diagnostic algorithm based on a fusion of vital data, in particular a respiratory rate, a breathing amplitude, an inspiration and expiration time, a cardiac rhythm and arrhythmia.

[0023] According to one aspect of the invention, the algorithm uses temperatures measured on remarkable areas, located by image processing, or the rate oxygen linked to the person being examined.

[0024] Preferably, these notable temperature measurement zones are located inside the mouth, on the tip of the nose, on the cheekbones and the palm of the hand.

[0025] According to one aspect of the invention, the diagnostic algorithm uses a characterization of the person, such as data on age, gender, clothing, height, body mass index, also called BMI.

[0026] According to one aspect of the invention, the system is arranged to carry out the acquisition of examination data until the diagnostic information is made available within a period of time of in particular between 30 and 120 seconds.

[0027] According to one aspect of the invention, the system is arranged to allow acquisition of examination data by taking measurements at a distance of 60 cm to 2 m between the acquisition device and the person. This avoids the person having to be in contact with the acquisition device.

[0028] The invention thus allows for rapid diagnosis without additional routing delay to a doctor, for example. The diagnostic information can, if necessary, be sent automatically to a doctor and can be stored on a Cloud-type data storage system.

[0029] According to one aspect of the invention, the data processing device uses an algorithm for analyzing the acquired examination data and, where appropriate, sorting people with the aim of detecting cases of sick people, based on all the data collected and artificial intelligence, the first level of learning of which is carried out on a sample in a hospital environment.

[0030] This learning by artificial intelligence can be carried out by means of a set of measurements collected by the system but also by means of medical monitoring of patients. This allows the improvement of the model over time.

[0031] According to one aspect of the invention, the measurements carried out by the acquisition device can be used to subsequently refine the diagnosis made by the artificial intelligence.

[0032] Thanks to the invention, a diagnosis can be made without contact with the person, which limits the risks of contamination, which is particularly advantageous, for example, in the case of a pandemic such as that linked to COVID 19.

[0033] According to one aspect of the invention, the measured examination data comprises at least one of the following data: temperatures measured at different points of the body of the person to be examined, a respiratory or cardiac characteristic.

[0034] According to one aspect of the invention, the acquisition device is arranged to acquire examination data comprising an external temperature, a temperature measured on a cheekbone of the person, a temperature measured on the tip of the nose of the person, and also, where appropriate, a maximum temperature of the face and a temperature of a reference temperature-controlled garment or surface.

[0035] According to one aspect of the invention, the remarkable measurement points are located by an artificial intelligence using an object identification flowchart.

[0036] According to one aspect of the invention, the temperature relative to a remarkable point is obtained by temporal average and by average of the temperatures of a surface defined by pixels from an infrared camera image near the remarkable point identified on the visible image by means of an object identification algorithm.

[0037] According to one aspect of the invention, the identification of personal characterizations is done by means of Red Green Blue cameras (or RGB camera) or Far Infrared (FIR camera) in addition to a reading of the person's identity, thanks to a classification system whose learning can be carried out on RGB (Red Green Blue) or Infrared images. The use of a greater number of parameters, in particular age, gender, size, body mass index, phenotype for example, serves to improve the diagnostic models.

[0038] According to one aspect of the invention, the diagnostic model, or diagnostic algorithm, which is fed with more data such as remarkable body temperatures, an ambient temperature, a class of personal characteristics, a time of day, can be arranged to additionally use data on the path of the person examined to check whether they have crossed paths with a sick person or passed through a risk region.

[0039] According to one aspect of the invention, the system is arranged to operate in the absence of a radar and using the RGB cameras to estimate the cardiac and respiratory parameters.

[0040] According to one aspect of the invention, the temperature relative to a remarkable point can be obtained by temporal average and by average of the temperatures of a surface defined by pixels from a camera image near the remarkable point. The remarkable point is for example defined geometrically by means of an image zone called Building box, which surrounds it, for example by means of the geometric average of the sides of the image zone. This image zone is a surface delimited by a series of points which is constructed by an object identification algorithm.

[0041] According to one aspect of the invention, the system is devoid of an RGB camera and / or does not use the surface whose temperature is controlled. In this case, the system uses the external temperature and a heat transfer model on the dressed areas or only the temperature differences between the notable points.

[0042] According to one aspect of the invention, the system is arranged to use a fusion of non-contact measurements, in particular of vital signals and thermal and visible imaging.

[0043] The diagnostic information comprises a class chosen from three predetermined classes which are “Healthy person”, “Person with a suspicion of disease”, “Person with a high probability of disease”.

[0044] The diagnostic information may also include an assessment of the severity of the disease.

[0045] The invention also relates to a method for providing diagnostic information for the detection of a disease on a person, in particular a contagious disease such as COVID 19, this method comprising the following steps: - acquiring examination data on the person, using an acquisition device comprising in particular at least one physiological measurement sensor such as a radar, and a thermal camera for acquiring this examination data, - receive these examination data obtained by the acquisition device, - process these examination data to obtain diagnostic information of the disease based on an analysis of said examination data, - display diagnostic information of the disease based on an analysis of said examination data, this diagnostic information being able to be representative of a level of probability that the person is affected by the disease.

[0046] The present invention can allow controls in public spaces in general, in particular on traffic routes for people, at entrances and exits of buildings, at airport gates, in schools.

[0047] This also allows for medical monitoring of people, for example for sick people who are kept at home.

[0048] The invention also relates, independently or in combination with the above, to an aid system for providing diagnostic information, in particular with a view to detecting a disease in a person, in particular a contagious disease such as COVID 19, this detection system comprising: - a device for acquiring examination data on the person, this acquisition device comprising in particular at least one physiological measurement sensor such as a radar, and a thermal camera for acquiring this examination data, this acquisition device comprising a plurality of sensors arranged to operate without contact with the person, at least one of the contactless sensors being arranged to acquire a response data item provided by the person, - a data processing device arranged to receive this data obtained by several sensors of the acquisition device, - a display device arranged to display diagnostic information of the disease based on an analysis of said data by the data processing device, this diagnostic information being able in particular to be representative of a level of probability that the person is affected by the disease, this level of probability being able in particular to be a score.

[0049] The invention makes it possible, thanks to the plurality of sensors which operate without contact with the person to be examined, to have sufficient data to establish a more reliable diagnosis.

[0050] According to one aspect of the invention, the data processing device comprises an artificial intelligence unit arranged to process the examination data obtained by the acquisition device and provide said diagnostic information.

[0051] According to one aspect of the invention, the data processing device and the display device are part of the same device, for example a computer, in particular a laptop computer.

[0052] According to one aspect of the invention, the display device, and possibly also the data processing device, are arranged to be mounted on a motor vehicle.

[0053] According to one aspect of the invention, the contactless sensor arranged to acquire a response data item provided by the person comprises a microphone arranged to receive responses provided orally by the person.

[0054] According to one aspect of the invention, the contactless sensor may, as a variant of the microphone which operates with voice recognition, include means for analyzing the person's face to, for example, recognize a YES or a NO, for example by nodding the head.

[0055] As a further variant, the contactless sensor may include means allowing the person to point a response on a screen with a finger, without contact with the screen.

[0056] According to one aspect of the invention, the system is arranged to present a questionnaire, for example by displaying the questions of the questionnaire on a display screen or for example using a sound device which pronounces the questions of the questionnaire, to the person to be examined, and the acquisition device is arranged to receive the responses provided by the person to this questionnaire, for example the microphone being used to record the responses of the person.

[0057] According to one aspect of the invention, the system comprises a data storage unit, for example forming part of a computer, arranged to store the questions of the questionnaire.

[0058] According to one aspect of the invention, the data processing device is arranged to receive data from the acquisition device relating to the answers provided by the person in response to the questionnaire, and this data processing device is arranged to process this data in combination with data coming from other sensors of the acquisition device, with a view to providing the diagnostic information for the disease.

[0059] According to one aspect of the invention, in order to obtain the diagnostic information, the data processing device is arranged to run a diagnostic algorithm based on a fusion of the person's vital data, in particular a respiratory rate, a breathing amplitude, an inspiration and expiration time, a cardiac rhythm and arrhythmia, with also questionnaire response data.

[0060] According to one aspect of the invention, the system is arranged so as to select the questions to be asked depending on the person to be examined, in particular one or more data available on the person.

[0061] According to one aspect of the invention, the questions to be asked are of a medical nature, useful for improving the diagnosis in combination with other data.

[0062] According to one aspect of the invention, the questions are chosen from the following list: Do you have chest pain? Do you have a dry cough? Do you have a sore throat? Do you have a runny nose? Do you have trouble breathing?

[0063] According to one aspect of the invention, the system is arranged to carry out, using one of the contactless sensors, a measurement whose result data are compared to a response provided by the person in response to a question in the questionnaire.

[0064] This makes it possible to make the measurement using the sensor more reliable, or to assign a level of reliability.

[0065] According to one aspect of the invention, the system is arranged to acquire data relating to a context, for example an ambient temperature.

[0066] According to one aspect of the invention, the system is arranged to perform a sorting based on a static model to give a score associated with the diagnosis.

[0067] According to one aspect of the invention, the system is arranged to operate the following steps: receiving consent data from the person to be subjected to the examination, acquiring characterization data of the person such as gender, age, height, and data representative of the context, acquiring data in response to questions, in particular using a microphone, and vital sign measurements, processing data to enable an evaluation of the level of reliability of the diagnostic information, possibly asking additional questions if necessary to refine the diagnostic information, performing a sorting based on a static model to give a score associated with the diagnosis.

[0068] According to one aspect of the invention, the contactless sensors are arranged to allow measurement acquisition at a distance from the person to be examined, between 50 cm and 2 m.

[0069] The invention also has as an object a terminal which includes the assistance system as mentioned above.

[0070] This terminal may for example comprise a processing unit arranged to process the data coming from the sensors and deliver the diagnostic information.

[0071] This processing unit can be a computer, with or without a screen on the terminal.

[0072] This terminal includes in particular the various sensors.

[0073] This terminal can, for example, be installed at the entrance to a building.

[0074] The invention also relates to an aid method for providing diagnostic information, in particular for the purpose of detecting a disease in a person, in particular a contagious disease such as COVID 19, this method comprising the following steps: acquire examination data on the person, this acquisition device comprising in particular at least one physiological measurement sensor such as a radar, and a thermal camera to acquire this examination data, this acquisition device comprising a plurality of sensors arranged to operate without contact with the person, at least one of the contactless sensors being arranged to acquire a response data provided by the person, this sensor being in particular a microphone, receive this data obtained by several sensors of the acquisition device, display diagnostic information of the disease based on an analysis of said data by the data processing device, this diagnostic information being able in particular to be representative of a level of probability that the person is affected by the disease, this level of probability being able in particular to be a score.

[0075] The invention also relates to an aid system for providing diagnostic information, in particular with a view to detecting a disease in a person, in particular a contagious disease such as COVID 19, this detection system comprising: a device for acquiring examination data on the person, this acquisition device comprising at least a first sensor, in particular a physiological measurement sensor such as a radar or a thermal camera for acquiring this examination data, in particular this acquisition device comprising a plurality of sensors arranged to operate without contact with the person, in particular at least one of the contactless sensors being arranged to acquire a response data provided by the person, - a data processing device arranged to receive these data obtained by the acquisition device, - in particular a display device arranged to display diagnostic information of the disease based on an analysis of said data by the data processing device, this diagnostic information being able in particular to be representative of a level of probability that the person is affected by the disease, this level of probability being able in particular to be a score - the acquisition device further comprising a correction sensor different from said first sensor and arranged to acquire at least one piece of data on the person, this data being a correction piece of data different from data acquired by the first sensor, - the data processing device being arranged to use this correction data to correct the examination data provided by the first sensor.

[0076] According to one aspect of the invention, the correction sensor is arranged to acquire data representative of a movement of the person.

[0077] According to one aspect of the invention, this correction sensor comprises a camera, in particular of the 3D type, for acquiring data representative of a movement of the person.

[0078] This camera is in particular a camera operating on the principle of time of flight (in English: Time of Flight, TOF) which allows a scene to be measured in real time in 3 dimensions (3D). This type of camera is known.

[0079] Alternatively, this camera is an RGB camera operating with, for example, a body segmentation tool or a platform equipped with a strain gauge on which the person is installed, which person can be standing, lying down or sitting.

[0080] According to one aspect of the invention, the correction sensor is arranged to acquire data representative of a person's voice.

[0081] According to one aspect of the invention, this correction sensor arranged to acquire data representative of a person's voice comprises a microphone (33), in particular a directional microphone or a camera.

[0082] According to one aspect of the invention, the acquisition device comprises both a correction sensor arranged to acquire data representative of a movement of the person and a correction sensor arranged to acquire data representative of a voice of the person.

[0083]

[0084]

[0085]

[0086]

[0087]

[0088]

[0089]

[0090]

[0091]

[0092] According to one aspect of the invention, the first sensor, in particular being a physiological measurement sensor, is a radar arranged in particular to acquire data representative of a person's respiratory rate, in particular by detecting the movement of the person's skin. According to one aspect of the invention, the radar is arranged to be placed in front of the person or behind the person. According to one aspect of the invention, the data processing device is arranged to carry out the correction of the examination data according to the correction data using artificial intelligence. According to one aspect of the invention, the data processing device is arranged to obtain a correction signal based on the correction data, this correction signal being variable over time. According to one aspect of the invention, the data processing device is arranged to provide the examination data, for example related to the respiratory rate, after correction using the correction signal. The invention makes it possible to take into account noise linked to the person's movements and / or their voice, to provide reliable and precise examination data. Indeed, the person's movements and words are likely to create interference in the acquisition of data, particularly physiological data such as respiratory rate. The invention also relates to a method of assistance for providing diagnostic information, in particular with a view to detecting a disease in a person, in particular a contagious disease such as COVID 19, this method comprising the following steps: - acquiring examination data on the person, using a device for acquiring examination data on the person, this acquisition device comprising at least a first sensor, in particular a physiological measurement sensor such as a radar or a thermal camera for acquiring this examination data, in particular this acquisition device comprising a plurality of sensors arranged to operate without contact with the person, in particular at least one of the contactless sensors being arranged to acquire a response data item provided by the person, - correct this examination data using correction data, in particular acquired using a correction sensor. The invention has the advantage of being able to cross-reference data in order to obtain a more reliable and / or more robust result. In the case of heart rate assessment using radar, this assessment being disturbed by the person's voice or the person's movement,

[0093]

[0094]

[0095]

[0096]

[0097] The invention makes it possible to evaluate this disturbance using a microphone or a camera which allows lip reading or even by means of a 4D camera or an inertial balance. The validity of the assessment of this heart rate thus depends on external noise, the person's posture or the light intensity for example. According to one aspect of the invention, the method according to the invention comprises the step of taking into account the weight of the evaluations of the acquired parameters to obtain the result. Thus, when disturbances are of greater amplitude, the invention recommends giving more weight to the least disturbed parameters. According to one aspect of the invention, in the case of evaluating the heart rate by means of radar, this evaluation being disturbed by the person's voice or the person's movement, the method has the step of evaluating this disturbance using a microphone or a camera which allows lip reading or even by means of a 4D camera or an inertial balance, to correct the examination data. The invention also relates to an aid system for providing diagnostic information, in particular for the purpose of detecting a disease in a person, in particular a contagious disease such as COVID 19, this detection system comprising: - a device for acquiring examination data on the person, this acquisition device comprising at least two sensors for measuring the same physiological measurement such as a radar and a thermal camera for acquiring this examination data, these sensors being in particular arranged to operate without contact with the person, - a data processing device (3) arranged to receive these data obtained by the sensors of the acquisition device, this processing device being further arranged to carry out a fusion of data coming from at least the two sensors of the acquisition device in particular with a view to increasing the robustness of the examination data acquired on the person, - in particular a display device (30) arranged to display diagnostic information of the disease based on an analysis of said data by the data processing device, this diagnostic information being able in particular to be representative of a level of probability that the person is affected by the disease, this level of probability being able in particular to be a score. According to one aspect of the invention, the two sensors are arranged at em- different investments.

[0099] The two sensors can be arranged in the same housing or in two different housings.

[0100] According to one aspect of the invention, the two sensors are arranged to capture signals of different natures, for example signals of different wavelengths.

[0101] According to one aspect of the invention, the two sensors comprising a radar and a thermal image camera are arranged to measure the respiratory rate or the heart rate of a person.

[0102] When "to measure the respiratory rate" is indicated above, this may involve measurements of respiratory characteristics such as rate, amplitude, inspiration time, expiration time, apnea time, etc.

[0103] The invention makes it possible, thanks to the fusion of measurement data coming from two different sensors or more sensors, to have a more robust result on the diagnostic assistance information.

[0104] According to one aspect of the invention, one or more sensors are provided, arranged to evaluate the noise of a signal and to attenuate this noise, in particular in addition to the evaluation of a reliability level.

[0105] According to one of the aspects of the invention, the processing unit is arranged to assign, for at least one of the sensors, a reliability level, for example which may be 0 or 1, in one or more measurement windows, this reliability level and this window being used to carry out the data fusion.

[0106] According to one aspect of the invention, the acquisition device comprises one or more complementary sensors arranged to determine the level of reliability of the two physiological measurement sensors.

[0107] For example, in the case where the sensors are a radar and an infrared camera which make it possible to evaluate the person's breathing rate, additional sensors such as a microphone or a device for measuring the person's body movements make it possible to evaluate the reliability of the measurement made by the camera or the radar. Indeed, when the person is moving or is speaking, the measurement made by the radar is either impossible or of poor quality.

[0108] According to one aspect of the invention, this correction sensor comprises a camera, in particular of the 3D type, for acquiring data representative of a movement of the person.

[0109] For example, the infrared camera can be associated with an RGB camera to check whether measurements by the infrared camera are possible or, when the person is moving or has their head turned for example, to conclude that measurements by the infrared camera are impossible.

[0110] According to one aspect of the invention, the processing unit is arranged to retain the measurement(s) of one or both sensors as a function of the reliability conditions of the measurement of the sensor(s).

[0111] Thus the processing unit does not use the measurements made by the sensor whose reliability is not recognized as sufficient.

[0112] For example, the processing unit does not use the measurements made by the radar sensor when the person is moving or turns their head. In this situation, the processing unit uses the other sensor, which is an infrared imaging camera.

[0113] According to one aspect of the invention, the processing unit is arranged to reconstruct a diagnostic information value on the basis of measurements obtained by the sensors and retained according to their quality and reliability.

[0114] For example, the curve of the signal representative of the diagnostic information can be formed by a signal measured by one of the sensors over a first time window and by a signal obtained by the other of the sensors over a second time window.

[0115] This fusion of data based on the reliability of sensor measurements and taking into account time windows makes it possible to make the assistance in obtaining diagnostic information robust.

[0116] According to one aspect of the invention, the processing unit comprises an artificial intelligence arranged to carry out data fusion.

[0117] In particular, artificial intelligence is arranged to assign a level of reliability for measurements made by the sensor(s).

[0118] According to one aspect of the invention, the vital signals are evaluated by artificial intelligence, and the uncertainty in the measurement evaluated by the artificial intelligence can be statistically integrated during the fusion of the data.

[0119] In particular, artificial intelligence is arranged to retain or reject a measurement made by one or other of the sensors depending on the reliability of this measurement.

[0120] The invention also relates to an assistance method for providing diagnostic information, in particular with a view to detecting a disease in a person, in particular a contagious disease such as COVID 19, this detection system comprising: - provide a device for acquiring examination data on the person, this acquisition device comprising at least two sensors for measuring the same physiological measurement such as a radar and a thermal camera for acquiring this examination data, these sensors being in particular arranged to operate without contact with the person, - receive this data obtained by the sensors of the acquisition device, this processing device being further arranged to carry out a fusion of

[0121]

[0122]

[0123]

[0124]

[0125]

[0126]

[0127]

[0128]

[0129] data coming from at least two sensors of the acquisition device, in particular with a view to increasing the robustness of the examination data acquired on the person The invention and its various applications will be better understood by reading the following description and examining the accompanying figures: [Fig.l] schematically illustrates a system according to a non-limiting embodiment of the invention. [Fig.2] illustrates a block diagram illustrating the steps implemented in the system of [Fig.l], [Fig.3] illustrates a block diagram illustrating the steps implemented in a system according to another example of implementation of the invention, [Fig.4] shows curves illustrating the effect of the corrections obtained by the invention, [Fig.5] shows curves illustrating the effect of a fusion of data obtained using the invention. Figures 1 and 2 show an aid system 1 for providing diagnostic information, in particular for the detection of a disease in a person, in particular a contagious disease such as COVID 19, according to the invention. This system 1 includes: - a device for acquiring 7 examination data on the person, - a data processing device 3 arranged to receive this data examination data obtained by the acquisition device 7, - a display device 30 arranged to display diagnostic information of the disease based on an analysis of said examination data, this diagnostic information being able to be representative of a level of probability that the person is suffering from the disease. This system 1 includes in particular: - a sensor of the cardiac activity of at least one passenger, here the heart rate, this sensor being a camera operating in the near infrared, - a sensor of the respiratory activity, in particular in respiratory frequency and / or amplitude, of at least one passenger, this sensor being a camera operating in the far infrared, or thermal camera, - a radar designed to measure the person's vital signs, - a sensor of the passenger's profile characteristics, in particular their gender, weight, height and age, this sensor being here a Red Green Blue camera, also called RGB camera in English, - a card reader to read a person's identity card and obtain the person's personal data.

[0130] These sensors and cameras, which are part of the acquisition device 7, are represented by the reference 2 in [Fig.l]. Certain sensors 2 are for example arranged on a ceiling of the vehicle. One of the other cameras 2 is arranged in a side pillar 6 of the vehicle V.

[0131] The heart rate and respiration sensor can be in the back of the seat or in the central console at the level of the passenger's thigh, this being non-limiting.

[0132] These sensors 2 are connected for information exchange with a data processing device 3 placed on the vehicle V.

[0133] The data processing device 3 comprises an artificial intelligence unit arranged to process the examination data obtained by the acquisition device 7 and provide said diagnostic information.

[0134] The data processing device 3 and the display device 30 are part of the same device, for example a computer, in particular a laptop.

[0135] In this case, the display device 30 is a screen of the portable computer and the data processing device comprises a microprocessor of this computer.

[0136] The display device 30 is arranged to be visible to the person being examined, and in particular the display device, which notably comprises a screen, is remote from the data processing device, these display and data processing devices being for example connected to each other by a wireless link, for example by a 3G, 4G or 5G communication protocol, or by the Internet network, or Wifi for example.

[0137] The display device 30, and also the data processing device 3, are here arranged to be mounted on a motor vehicle.

[0138] Alternatively, the display device 30, and possibly also the data processing device 3, are arranged to be placed in a fixed manner, in a building or an outdoor courtyard for example.

[0139] The processing device 3 is arranged to make the diagnosis automatically, without human intervention.

[0140] The examination data acquisition device 7 is arranged to allow contactless and safe distance measurements of vital signals and thermal and visible imaging on the person to be examined.

[0141] The system according to the invention thus advantageously uses the fusion of contactless and safe distance measurements, vital signals and thermal and visible imaging.

[0142] The artificial intelligence unit is arranged to use an artificial intelligence-based diagnostic model fed by a reasonable number of clinical measurements.

[0143] The invention allows for diagnostics to be carried out in a mobile manner. Its implementation, for example using a thermal camera, a physiological measurement sensor such as a radar and a portable personal computer, is easy.

[0144] The acquisition device 7 comprises a radar for acquiring data relating to the vital signs of the person, a thermal camera for temperature measurements providing temperature data, and a camera operating in the visible spectrum for characterizing the person tested providing characterization data of the person.

[0145] The data processing device 3 is arranged to run a diagnostic algorithm based on a fusion of vital data, in particular a respiratory rate, a breathing amplitude, an inspiration and expiration time, a cardiac rhythm and arrhythmia, an oximetry.

[0146] The algorithm uses temperatures measured on remarkable areas, located by image processing.

[0147] Preferably, these notable temperature measurement zones are located inside the mouth, on the tip of the nose, on the cheekbones and the palm of the hand.

[0148] The diagnostic algorithm uses a characterization of the person, such as data on age, gender, clothing, height, body mass index.

[0149] The system is arranged to carry out the acquisition of examination data until the diagnostic information is made available within a period of time of in particular between 30 and 120 seconds.

[0150] The invention thus allows a rapid diagnosis without additional routing delay to a doctor for example. The diagnostic information can, if necessary, be sent automatically to a doctor and can be stored on a Cloud-type data storage system 40.

[0151] The data processing device 3 uses an algorithm for analyzing the acquired examination data and, where appropriate, sorting people in order to detect cases of sick people, based on all the data collected and artificial intelligence, the first level of learning of which is carried out on a sample in a hospital environment.

[0152] The measurements made by the acquisition device can be used to subsequently refine the diagnosis made by artificial intelligence.

[0153] Thanks to the invention, a diagnosis can be made without contact with the person, which limits the risks of contamination, which is particularly advantageous, for example, in the case of a pandemic such as that linked to COVID 19.

[0154] According to one aspect of the invention, the measured examination data comprises at least one of the following data: temperatures measured at different points of the body of the person to be examined, a respiratory or cardiac characteristic.

[0155] The acquisition device 7 is arranged to acquire examination data comprising an outside temperature, a temperature measured on a cheekbone of the person, a temperature measured on the tip of the nose of the person, and also where appropriate, a maximum temperature of the face and a temperature of a reference temperature-controlled garment or surface.

[0156] The notable measurement points are located by artificial intelligence using an object identification flowchart.

[0157] The identification of personal characterizations is done using Red Green Blue cameras (or RGB camera) or Far Infrared (FIR camera) in addition to a reading of the person's identity, thanks to a classification system whose learning can be carried out on RGB (Red Green Blue) or Infrared images. The use of a greater number of parameters, including age, gender, size, body mass index, phenotype for example, serves to improve diagnostic models.

[0158] The diagnostic model, or diagnostic algorithm, which is fed with more data such as remarkable body temperatures, ambient temperature, a class of personal characteristics, a time of day, can be arranged to additionally use data on the path of the person examined to check whether he or she has crossed paths with a sick person or passed through a risk region.

[0159] The temperature relative to a remarkable point can be obtained by time average and by average of the temperatures of a surface defined by pixels from a camera image near the remarkable point. The remarkable point is for example defined geometrically by means of an image zone called Building box, which surrounds it. This image zone is a surface delimited by a series of points which is constructed by an object identification algorithm.

[0160] The diagnostic information comprises a class chosen from three predetermined classes which are “Healthy person”, “Person with a suspicion of disease”, “Person with a high probability of disease”.

[0161] The invention thus implements the following steps: - acquiring examination data on the person using the acquisition device 7, which are steps 20 to 25 of [Fig.2], - receive these examination data obtained by the acquisition device, - process these examination data to obtain diagnostic information of the disease based on an analysis of said examination data, using the data processing device 3 (step 28), - display diagnostic information of the disease based on an analysis of said examination data, this diagnostic information being able to be representative of a level of probability that the person is affected by the disease (stage 29).

[0162] Steps 20 to 25 are as follows: - the identification of personal characteristics is done using Red Green Blue cameras (or RGB cameras), this is step 20, - acquisition of the person's temperature at step 21 with an FIR type thermal camera, the details of this temperature acquisition having already been described, - acquisition of respiratory rhythm step 22 using the FIR camera, - acquisition of the heart rate in step 23 using the camera in the near infrared, or NIR camera, - acquisition of vital signs using radar, at step 24, - acquisition of personal data using the card reader in step 25, - potentially oximetry by near infrared (NIR) camera.

[0163] This also allows for medical monitoring of individuals.

[0164] The examination data may, where appropriate, include the size of the pupils and their position.

[0165] Diagnostic information is automatically sent to a remote cloud-based data storage system.

[0166] Similarly, information useful to the diagnostic model, or diagnostic algorithm, can be received from the remote Cloud-type system.

[0167] Potentially two cameras, NIR and FIR, are used. The FIR camera for temperatures and respiration characteristics and the NIR camera for oximetry and heart rate.

[0168] The acquisition device is in particular arranged to acquire examination data comprising an outside temperature, a temperature measured on a cheekbone of the person, a temperature measured on the tip of the nose of the person, and also where appropriate, a maximum temperature of the face and a temperature of a garment or a reference temperature-controlled surface, and where appropriate a respiratory volume, tremors, the oxygen level in the blood.

[0169] Another example of implementation of the invention is shown in [Fig.3].

[0170] In this example, the help system 1 comprises, like the example described above previously: - a device 7 for acquiring examination data on the person, this acquisition device comprising in particular at least one physiological measurement sensor such as a radar, and a thermal camera for acquiring this examination data, this acquisition device comprising a plurality of sensors 2 arranged to operate without contact with the person, at least one of the contactless sensors being arranged to acquire data response provided by the person, - a data processing device 3 arranged to receive this data obtained by several sensors of the acquisition device, - a display device 30 arranged to display diagnostic information of the disease based on an analysis of said data by the data processing device, this diagnostic information being able in particular to be representative of a level of probability that the person is affected by the disease, this level of probability being able in particular to be a score.

[0171] One of the contactless sensors 2 arranged to acquire a response data provided by the person comprises a microphone arranged to receive responses provided orally by the person.

[0172] The system 1 is arranged to present a questionnaire, for example by displaying the questions of the questionnaire on a display screen or for example using a sound device which pronounces the questions of the questionnaire, to the person to be examined, and the acquisition device is arranged to receive the responses provided by the person to this questionnaire, here the microphone 33 being used to record the responses of the person.

[0173] The system 1 comprises a data storage unit, for example forming part of a computer, arranged to store the questions of the questionnaire 34.

[0174] The data processing device 3 is arranged to receive data from the acquisition device relating to the answers provided by the person in response to the questionnaire, and this data processing device is arranged to process this data in combination with data coming from other sensors of the acquisition device, with a view to providing the diagnostic information for the disease.

[0175] In order to obtain the diagnostic information, the data processing device 3 is arranged to run a diagnostic algorithm based on a fusion of the person's vital data, in particular a respiratory rate, a breathing amplitude, an inspiration and expiration time, a cardiac rhythm and arrhythmia, with also questionnaire response data.

[0176] The system is arranged to select the questions to be asked based on the person to be examined, in particular one or more data available on the person.

[0177] The questions to be asked are of a medical nature, useful for improving the diagnosis in combination with other data.

[0178] The system 1 is arranged to carry out, using one of the contactless sensors 2, a measurement whose result data are compared with a response provided by the person in response to a question in the questionnaire.

[0179] We will now describe with reference to [Fig.3] different stages of implementation of the invention.

[0180] Step 50 corresponds to the start of the process of detecting a disease in a person.

[0181] This is followed by a step 51 of data entry by the person to be examined.

[0182] In this step 51, the person enters, for example, data concerning him / herself, for example example his identity, then possibly gives his agreement or consent to continue the diagnostic process.

[0183] This step 51 can be performed by the person via a touch screen 35 coupled to a disinfection device 36 using UV. A microphone 33 can be used to record the person's responses to questions. A voice recognition device 38 is cut to the microphone 33 to enter data taken by the microphone 33 into the data processing unit 3.

[0184] Context data 37, for example an ambient temperature, can be acquired by a sensor 2.

[0185] In step 52, characteristics linked to the person are determined using one or more sensors 2 via an object image recognition system 39, as described in the previous example.

[0186] Additional questions are asked at the end of step 52 to clarify the clinical condition of the person.

[0187] For this step 53 of questions / answers, the questions are chosen for example from the following list: Do you have chest pain? Do you have a dry cough? Do you have a sore throat? Do you have a runny nose? Do you have difficulty breathing?

[0188] Responses can be provided by the person via microphone 33.

[0189] In the following step 54, the sensors 2 proceed to acquire data relating to the vital signs of the person, as described in the preceding example.

[0190] Then at step 55, the artificial intelligence provides a preliminary score representative of the diagnosis obtained on the basis of the data entered until then.

[0191] In step 56, additional questions are asked of the person, and the answers obtained are used to refine the preliminary score obtained in step 55.

[0192] A final score is given in step 57 by the artificial intelligence, which is recorded on the data storage space 40.

[0193] In step 58, the final score is displayed for information to the person.

[0194] Another embodiment of the invention will now be presented with reference to [Fig.4],

[0195] In this example, the system 1 for providing diagnostic information, slightly different from that described above, comprises: - a device 7 for acquiring examination data on the person, this

[0196]

[0197]

[0198]

[0199]

[0200]

[0201]

[0202]

[0203] acquisition device comprising at least a first sensor, here a physiological measurement sensor which is a radar for acquiring this examination data, - a data processing device 3 arranged to receive these data obtained by the acquisition device, - a display device 30 arranged to display diagnostic information of the disease based on an analysis of said data by the data processing device, this diagnostic information being able in particular to be representative of a level of probability that the person is affected by the disease, this level of probability being able in particular to be a score - the acquisition device 7 further comprising a correction sensor 33 different from said first sensor, namely the radar, and arranged to acquire at least one piece of data on the person, this data being a correction piece of data different from data acquired by the first sensor, - the data processing device being arranged to use this correction data to correct the examination data provided by the first sensor. This correction sensor 33 is arranged to acquire data representative of the person's voice. This correction sensor 33 arranged to acquire data representative of a person's voice comprises a microphone 33, in particular a directional microphone or a camera. It can be provided that the acquisition device 7 comprises both a correction sensor arranged to acquire data representative of a movement of the person and a correction sensor arranged to acquire data representative of a voice of the person. This correction sensor for acquiring data representative of a person's movement includes a camera, in particular of the 3D type, for acquiring data representative of a person's movement. This camera is notably a camera operating on the principle of time of flight (in English: Time of Flight, TOF) which allows a scene to be measured in real time in 3 dimensions (3D). This type of camera is known The radar is designed in particular to acquire data representative of the person's breathing rate, in particular by detecting the movement of the person's skin. The radar is arranged to be placed in front of the person or behind the person. Of course, sensors other than radar can be used, for example a Near Infrared camera.

[0204] The data processing device 3 is arranged to carry out the correction of the examination data according to the correction data using artificial intelligence.

[0205] The data processing device 3 is arranged to obtain a correction signal based on the correction data, this correction signal being variable over time.

[0206] The data processing device 3 is arranged to provide the examination data, for example linked to the respiratory rate, after correction using the correction signal.

[0207] The invention makes it possible to take into account the noise linked to the movements of the person and / or their voice, to provide reliable and precise examination data.

[0208] Indeed, the person's movements and words are likely to create interference in the acquisition of data, particularly physiological data such as respiratory rate.

[0209] This can be seen for example in the curves of Figure 4.

[0210] Curve 101 illustrates the magnitude of the error, over time, in a measurement of the person's breathing rate, this error being induced by movements of the person during the measurement and by the fact that they speak during the measurement.

[0211] This error amplitude on curve 101 is relatively large.

[0212] Curve 102 shows the magnitude of the error, over time, on a measurement of the person's breathing rate, when a correction is taken into account taking into account the person's movements detected by the motion camera.

[0213] The amplitude on curve 102 is reduced compared to curve 101.

[0214] Finally, curve 103 illustrates the amplitude of the error, over time, on a measurement of the person's breathing rate, when corrections linked to both the person's movements and speech are taken into account.

[0215] This curve 103 is close to zero thanks to the corrections made.

[0216] The system of the invention thus makes it possible to implement the method comprising the following steps: - acquiring examination data on the person, using a device for acquiring examination data on the person, this acquisition device comprising at least a first sensor, in particular a physiological measurement sensor such as a radar or a thermal camera for acquiring this examination data, in particular this acquisition device comprising a plurality of sensors arranged to operate without contact with the person, in particular at least one of the contactless sensors being arranged to acquire a response data item provided by the person,

[0217]

[0218]

[0219]

[0220]

[0221]

[0222]

[0223]

[0224] - correct this examination data using correction data, in particular acquired using a correction sensor. In connection with [Fig. 5], the system 1 according to an example of the invention may comprise - the device 7 for acquiring examination data on the person, this acquisition device comprising at least two sensors for measuring the same physiological measurement such as a radar and a thermal camera for acquiring this examination data, these sensors being in particular arranged to operate without contact with the person, these sensors already being described above, - the data processing device 3 arranged to receive these data obtained by the sensors of the acquisition device, this processing device being further arranged to carry out a fusion of data coming from at least the two sensors of the acquisition device in particular with a view to increasing the robustness of the examination data acquired on the person, - the display device 30 arranged to display diagnostic information of the disease based on an analysis of said data by the data processing device, this diagnostic information being able in particular to be representative of a level of probability that the person is affected by the disease, this level of probability being able in particular to be a score. The two sensors are located in different locations. In the described example, the two sensors comprising a radar and a thermal imaging camera are arranged to measure the breathing rate of a person. The processing unit 3 is arranged to assign, for at least one of the sensors, a reliability level F, for example which may be 0 or 1, in several measurement windows T1, T2, T3 and T4 for example, this reliability level and these windows being used to carry out the data fusion. These windows are visible on the graph in [Fig.5]. The acquisition device 7 comprises one or more complementary sensors arranged to determine the level of reliability of the two physiological measurement sensors. For example, in the case where the sensors are a radar and an infrared camera that allow the person's breathing rate to be assessed, additional sensors such as a microphone or a device for measuring the person's body movements can be used to assess the reliability of the measurement made by the camera or radar. Indeed, when the person is moving or speaking, the measurement made by the radar is either impossible or of poor quality. This correction sensor includes a camera, notably of the 3D type, to acquire data representative of a person's movement.

[0225] For example, the infrared camera can be associated with an RGB camera to check whether measurements by the infrared camera are possible or, when the person is moving or has their head turned for example, to conclude that measurements by the infrared camera are impossible.

[0226] The processing unit 3 is arranged to retain the measurement(s) of one or both sensors depending on the reliability conditions of the measurement of the sensor(s).

[0227] Thus the processing unit does not use the measurements made by the sensor whose reliability is not recognized as sufficient.

[0228] For example, the processing unit does not use the measurements made by the radar sensor when the person is moving or turning their head. In this situation, the processing unit uses the other sensor, which is an infrared imaging camera.

[0229] The processing unit 3 is arranged to reconstruct a diagnostic information value on the basis of measurements obtained by the sensors and retained according to their quality and reliability.

[0230] For example, the curve of the signal representative of the diagnostic information can be formed by a signal measured by one of the sensors over a first time window T1 and by a signal obtained by the other of the sensors over a second time window T3, as in the example of [Fig.5].

[0231] In windows T2 and T4, the measurements made by the two sensors are of comparable satisfactory reliability and are merged.

[0232] On the graph of [Fig.5], curve 110 represents the theoretical value of the information, for example the respiratory rate, which we wish to obtain.

[0233] Curve 111 represents a signal measured by the camera which is one of the sensors used to measure the breathing rate on the person.

[0234] Curve 112 represents a signal measured by the radar which is the other of the sensors used to measure the breathing rate on the person.

[0235] Curve 113 represents the merging of the data represented by curves 111 and 112.

[0236] As can be seen in [Fig.5], in window Tl, the reliability of the camera measurement is not sufficient and the measurement is not taken into account in the data fusion.

[0237] In this window T1, only the measurements made by the radar, measurements represented by curve 112, are taken into account.

[0238] Conversely, in window T3, only the measurements made by the camera are taken into account to obtain the result.

[0239] In windows T2 and T4 for example, the measurements of the camera and the radar, re presented by curves 111 and 112, are sufficiently reliable to be all taken into account to obtain the resulting curve 113.

[0240] This fusion of data based on the reliability of sensor measurements and taking into account time windows makes it possible to make the assistance in obtaining diagnostic information robust.

[0241] The processing unit 3 comprises an artificial intelligence arranged to carry out data fusion.

[0242] In particular, artificial intelligence is arranged to assign a level of reliability for measurements made by the sensor(s).

[0243] In particular, artificial intelligence is arranged to retain or reject a measurement made by one or other of the sensors depending on the reliability of this measurement.

Claims

Claims

1. System (1) for assisting in providing diagnostic information, in particular for the detection of a disease in a person, in particular a contagious disease such as COVID 19, this detection system comprising: - an acquisition device (7) for examination data on the person, this acquisition device comprising at least two sensors for measuring the same physiological measurement, these two sensors comprising a radar and a thermal camera for acquiring these examination data, these sensors being in particular arranged to operate without contact with the person, - a data processing device (3) arranged to receive these data obtained by the sensors of the acquisition device, this processing device being further arranged to carry out a fusion of data coming from at least the two sensors of the acquisition device in order to increase the robustness of the examination data acquired on the person,- in particular a display device (30) arranged to display diagnostic information of the disease based on an analysis of said data by the data processing device, this diagnostic information being able in particular to be representative of a level of probability that the person is affected by the disease, this level of probability being able in particular to be a score.,

2. System according to the preceding claim, according to which the two sensors are arranged in the same housing.

3. System according to one of the preceding claims, according to which the two sensors are arranged to capture signals of different natures.

4. System according to the preceding claim, according to which the two sensors comprising a radar and a thermal image camera are arranged to measure the respiratory rate or the heart rate of a person.

5. System according to one of the preceding claims, according to which the unit processing is arranged to assign, for at least one of the sensors, a reliability level, for example which may be 0 or 1, in one or more measurement windows (T1, T2, T3, T4), this reliability level and this window being used to carry out the data fusion.

6. System according to one of the preceding claims, according to which the acquisition device (7) comprises one or more complementary sensors arranged to determine the level of reliability of the two physiological measurement sensors.

7. System according to the preceding claim, in which the two sensors are a radar and an infrared camera making it possible to evaluate the person's breathing rate, the complementary sensors being chosen from a microphone or a device for measuring the movement of the person's body.

8. System according to one of the preceding claims, according to which the processing unit is arranged to retain the measurement(s) of one or both sensors depending on the reliability conditions of the measurement of the sensor(s).

9. System according to one of the preceding claims, according to which the processing unit is arranged to reconstruct a diagnostic information value on the basis of measurements obtained by the sensors and retained according to their quality and reliability.

10. System according to one of the preceding claims, according to which the processing unit comprises an artificial intelligence arranged to carry out the data fusion.

11. Method of assistance for providing diagnostic information, in particular for the detection of a disease in a person, in particular a contagious disease such as COVID 19, this method comprising the steps: - providing an acquisition device (7) for examination data on the person, this acquisition device comprising at least two sensors for measuring the same physiological measurement such as a radar and a thermal camera for acquiring this examination data, these sensors being in particular arranged to operate without contact with the person, - receiving this data obtained by the sensors of the acquisition device, this processing device being further arranged to carry out a fusion of data coming from at least the two sensors of the acquisition device in order to increase the robustness of the examination data acquired on the person.

Images