System capable of assessing blood pressure, oximetry and heart rate

Abstract

The present invention relates to a system encompassing the acquisition, reading and mathematical processing of video images with the aim of obtaining clinical data, vital signs and other information about a patient. The system is based on the collection of video images using standard non-professional cameras with the aim of technically enhancing the video using processes of Eulerian magnification, image wave acquisition, transcription of said waves into metadata and mathematical calculation via indirect inference of data and vital signs, clinical health aspects and other medical data, using a computational tool that is balanced between local devices, an internet browser and remote cloud servers.

Description

A system capable of measuring blood pressure, oxygen saturation, and heart rate. CAPO OF THE INVENTION

[0001] The present invention relates to the field of remote physiological measurement, and to a system encompassing the acquisition, reading, and mathematical processing of video images in order to obtain clinical data, vital signs, and other information from a patient. More specifically, the present invention relates to a remote photoplethysmography (rPPG) system capable of adapting to various challenges, such as subject movement, lighting conditions, and variations in skin tone. BACKGROUND OF THE INVENTION

[0002] In public health, it is desirable that no procedure performed should have restrictions on its use, as technological aspects such as internet connectivity, hardware, and the like are limiting factors.

[0003] Remote photoplethysmography (rPPG) is a contactless method for measuring physiological parameters such as heart rate, heart rate variability, respiratory rate, and blood oxygen saturation using a video camera. While rPPG has shown promise in various applications such as telemedicine, remote patient monitoring, and fitness tracking, existing rPPG implementations often struggle to adapt to real-world conditions due to their reliance on controlled laboratory environments.

[0004] Document US10387720 refers to a video monitoring system that captures images of a patient in various positions. The captured images are analyzed by the system for changes in a patient's position or movement; images in which the system detects one or both movements and repositioning of the patient are retained. The system analyzes an area of ​​interest. Within each image, there is an area in the camera's field of view showing the patient. Sequential image frames are compared for motion; only still frames, where the patient is stationary, are analyzed.

[0005] Document US9865176, in turn, discloses systems and methods that provide automated messages to a patient on behalf of a healthcare professional, configuring one or more computer-implemented agents, each specialized in a disease state, with rules to respond to a client's condition; during runtime, receiving a communication from the client and in response selecting one or more computer-implemented agents to respond to the communication; and automatically generating a response to be processed on a client's mobile device to encourage healthy behavior.

[0006] Document EP3294106 follows a different approach, teaching systems, tools, and methods for acquiring diagnostic information, including medical information, for a user, transmitting the information to a remote location, evaluating the information, and transmitting resulting diagnostic and treatment information to the user and / or a third party for further action. The document provides consumer- and user-friendly telemedicine systems and procedures that enable health and / or diagnostic services to be delivered remotely.

[0007] It is clear that there is a severely pent-up demand for systems that promote balanced service usage across devices, lower bandwidth and data consumption, and usability on any simple device. SUMMARY OF THE INVENTION

[0008] The present invention provides a system based on the collection of video images from non-professional, home cameras, with a view to technically processing the video, going through Eulerian magnification processes, image wave acquisition, transcription of these waves into metadata and mathematical calculation by indirect inference of data and vital signs, clinical health aspects and other medical data, using a balanced computational tool between local devices, internet browser and remote servers in the cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 shows a schematic representation of the system according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] The present invention addresses the limitations of existing rPPG systems by providing a more robust and adaptable rPPG system that can handle various challenges, such as subject movement, varying lighting conditions, and variations in skin tone. The system employs an adaptive approach that dynamically selects the most appropriate combination of techniques based on skin tone, situation, and challenges present in the current scenario. This adaptive approach allows the system to maintain an accurate and reliable physiological measurement estimate for a wide range of users and conditions.

[0011] The acquisition of video images by non-professional devices, such as mobile device cameras and webcams, is common and increasingly present as a tool for promoting health and well-being.

[0012] Image magnification using Eulerian techniques, and other related techniques, has high local computational consumption on a device such as a mobile device, and demands considerable hardware specifications, including GPU, CPU, and RAM.

[0013] Sending complex, high-definition images to remote servers is also common practice, but the solutions available in the state of the art demand high consumption of local processing power, internet bandwidth, and data transmission volume, making their routine use and widespread population access, as desired for health and wellness applications, unfeasible.

[0014] Decoding images into measurable waveforms and transforming them into metadata is commonly used for many different applications of imaging, but not specifically for clinical aspects and vital data.

[0015] The decoding of images with inference of clinical data through mathematical transformation of metadata, as well as the decomposition of images into different modules, with a view to lower local processing consumption and lower internet bandwidth and data transmission consumption, benefits from processes created in this patent application, increasing the possibility of broad population use of such data. It is worth highlighting that the system according to this application uses health data inference in order to provide greater resolution and accuracy of such inferences.

[0016] The dynamic algorithm (machine learning) for interpreting clinical data through the reading of magnified image waveforms is an additional aspect of this application, as well as its progressively increasing accuracy and greater clinical use cases.

[0017] The use of clinical data obtained through wave analysis from magnified images is supported by the state of the art. This application offers the unique advantage of being applicable to blood pressure, oximetry, and heart rate, always with local processing implemented using local software, which has a high demand for local hardware.

[0018] The use of clinical data for other clinical aspects, such as blood flow in sepsis, general well-being, infectious risk index, and the like, is also contemplated by the system in accordance with this application.

[0019] Through an inventive chain of technologies and algorithms, along with specific and specialized neural networks for each of the challenges not yet solved in the state of the art, such as motion suppression, illumination, pixel locking and tracking, information processing and recovery in the process of obtaining diffuse reflection data and other adverse conditions, the system according to the present application becomes a versatile option for promoting health and well-being.

[0020] The main distinguishing features of the present invention include: 1. Adaptation to the real world: the system was designed to function effectively in real-world conditions, where the object may be moving, lighting conditions may vary, and skin reflectivity may be influenced by factors such as dirt, oil, or sweat. 2. Adaptive algorithm selection: Based on skin tone estimation and challenge assessment, the system dynamically selects the most suitable combination of techniques for ROI selection, color space transformation, signal enhancement, and other processing steps, ensuring optimal performance under various conditions. 3. Combined approach: The system uses a combination of computer vision, signal processing, and machine learning techniques to address specific challenges and adapt to a wide range of scenarios. 4. Modular design: The system features a modular design that separates the various challenges and processing steps into distinct modules, allowing for easy... Customization, updates, and improvements to individual components without affecting overall system performance. 5. System performance monitoring and calibration: Continuously monitor the performance of the selected algorithms and refine the adaptive algorithm selection process to improve system performance over time.

[0021] By incorporating these innovations and improvements, the present invention offers a more reliable and versatile rPPG system that can be used in a wide range of real-world applications.

[0022] As shown in Figure 1, the system according to the present application comprises means for capturing video images using non-professional devices; means for sending complex, high-definition images to remote servers; a device capable of magnifying images using Eulerian or similar techniques; a device capable of decoding images into measurable waves and transforming them into metadata; a device supporting a dynamic algorithm (machine learning) for interpreting clinical data through the reading of magnified image waves; and means for sending the interpreted data for verification by a qualified professional.

[0023] Figure 1 also illustrates the system according to the present invention by means of ROI selection, by means of an example of motion suppression, by means of an example of lighting compensation, by means of a neural network diagram and by means of a signal processing flowchart.

[0024] The present invention comprises several modules, including: a. Face detection module: which identifies the subject's face in the video stream; b. ROI Selection Module: selects the ideal region of interest (ROI) for rPPG signal extraction based on skin tone and other factors; c. Preprocessing Module: applies color space transformation and other preprocessing techniques to the video data; d. Motion Suppression Module: reduces the impact of object movement on the rPPG signal using dedicated neural networks or other motion suppression techniques; e. rPPG Extraction Module: extracts the rPPG signal from the video data using signal processing or machine learning techniques; f. Physiological Measurement Estimation Module: estimates physiological parameters such as heart rate, heart rate variability, and respiratory rate from the rPPG signal; g.Skin tone estimation module: which develops a method to estimate the subject's skin tone in the video, using color analysis, machine learning techniques, or even user input; h. Challenge assessment module: which assesses the challenges present in the current situation, such as lighting conditions, subject movement, and skin reflectivity, by analyzing video data or incorporating additional sensors, such as an ambient light sensor or accelerometer; i. Adaptive algorithm selection module: based on skin tone estimation and challenge assessment, dynamically selects the most suitable combination of techniques for ROI selection, color space transformation, signal enhancement, and other processing steps. This may involve using predefined rules or machine learning models to choose the best combination of methods for a given situation. j. Performance monitoring module: which continuously monitors the performance of selected algorithms, tracking the signal-to-noise ratio, accuracy of physiological measurement, or user feedback. This information can be used to further refine the adaptive algorithm selection process and improve system performance over time; k. System calibration module: which periodically calibrates the system to ensure it remains accurate and reliable under varying conditions. This may involve updating predefined rules or machine learning models based on new data, user feedback, or advances in rPPG research.

[0025] In a preferred method, the means of capturing video images using non-professional devices are mobile device cameras or webcams.

[0026] In a preferred approach, the means for sending complex, high-definition images to remote servers is via wireless internet transmission.

[0027] In a preferred embodiment, the device capable of magnifying images using Eulerian or similar techniques, the device capable of decoding images into measurable waves and transforming them into metadata, and the device supporting a dynamic (machine learning) algorithm for interpreting clinical data through the reading of magnified image waves are a single device with compatible hardware, GPU, CPU, and RAM specifications.

[0028] In a preferred approach, a cloud computing model is used, which stores data on the Internet through a cloud computing provider that manages and operates the physical data storage.

[0029] In a preferred approach, the means of sending the interpreted data for verification by a qualified professional is via wireless internet transmission.

[0030] As also shown in Figure 1, the system according to the present application operates as follows: a) video image capture module; b) preliminary local processing and treatment module for refining valid data; c) transmission module of valid data to the cloud; d) intelligent Eulerian and Biomathematical processing module with the support of proprietary algorithms capable of interpreting vital data for the purpose of this invention; e) appropriate publication module of these results for the user and patient.

[0031] The present invention presents a novel remote photoplethysmography (rPPG) system that addresses the limitations of existing rPPG implementations, offering an adaptable, accurate, and reliable solution for non-contact physiological measurement. By leveraging an adaptive approach and a combination of computer vision, signal processing, and machine learning techniques, the system is able to dynamically adjust to various challenges, such as subject movement, lighting conditions, and skin tone variations, ensuring optimal performance under diverse real-world scenarios.

[0032] The system's modular design facilitates customization, upgrades, and improvements to individual components without compromising overall performance. Continuous performance monitoring and system calibration ensure that the accuracy and reliability of the rPPG system are maintained over time.

[0033] By incorporating these features, the present invention offers a versatile and robust rPPG system that can be employed in a wide range of applications, such as telemedicine, remote patient monitoring, and tracking of... Fitness and much more. This advanced rPPG system significantly advances the field of remote physiological measurement and has the potential to improve quality of life and health outcomes for users worldwide.

[0034] Although the present application has been described with respect to a preferred embodiment, it should be understood that many variations, modifications, equivalent structural elements, combinations, sub-combinations and other applications of the system according to the present invention may be made and do not depart from the scope of protection of the present application.

Claims

CLAIMS 1. System for acquiring, reading and processing video images to obtain clinical data, vital signs and other information from a patient, characterized by the fact that it comprises: means for capturing video images by non-professional devices; means for sending complex, high-definition images to remote servers; a device capable of magnifying images using Eulerian techniques; a device capable of decoding images into measurable waves and transforming them into metadata; a device supporting a dynamic algorithm for interpreting clinical data through reading waves of magnified images; and means for sending the interpreted data for verification by a trained professional.

2. The acquisition system of claim 1, comprising 11 modules, including: a. A face detection module that identifies the subject's face in the video stream; b. A ROI selection module that selects the optimal region of interest (ROI) for rPPG signal extraction based on skin tone and other factors; c. A preprocessing module that applies color space transformation and other preprocessing techniques to the video data; d. A motion suppression module that reduces the impact of object motion on the rPPG signal using dedicated neural networks or other motion suppression techniques; e. A rPPG extraction module that extracts the rPPG signal from the video data using signal processing or machine learning techniques; f. Physiological Measurement Estimation Module: Estimates physiological parameters such as heart rate, heart rate variability, and respiratory rate from the rPPG signal; g. Skin Tone Estimation Module: Develops a method to estimate the subject's skin tone in the video, using color analysis, machine learning techniques, or even user input; h. Challenge Assessment Module: Assesses the challenges present in the current situation, such as lighting conditions, subject movement, and skin reflectivity, by analyzing the video data or incorporating additional sensors such as an ambient light sensor or accelerometer; i. Adaptive Algorithm Selection Module: Based on skin tone estimation and challenge assessment, dynamically selects the most appropriate combination of techniques for ROI selection, color space transformation, signal enhancement, and other processing steps.This may involve using predefined rules or machine learning models to choose the best combination of methods for a given situation; j. Performance monitoring module: which continuously monitors the performance of selected algorithms, tracking signal-to-noise ratio, physiological measurement accuracy, or user feedback. This information can be used to further refine the adaptive algorithm selection process and improve system performance over time; k. System calibration module: which periodically calibrates the system to ensure it remains accurate and reliable under changing conditions. This may involve updating predefined rules or machine learning models based on new data, user feedback, or advances in rPPG research.

3. Method for acquiring, reading and processing video images to obtain clinical data, vital signs and other information about a patient, characterized by the fact that it comprises the steps of: capturing video images by non-professional devices; sending complex, high-definition images to remote servers; magnifying images using Eulerian techniques; decoding images into measurable waves and transforming them into metadata; processing through a dynamic algorithm for interpreting clinical data by reading waves of magnified images; and sending the interpreted data for verification by a trained professional.

Images