Space-time based detection and correction of motion artifacts for measuring arterial pressure waveforms

Abstract

Methods and systems for spatiotemporal management of motion artifacts / blood pressure drifts in arterial pressure waveforms. The system includes an elastomeric sensor array in contact with a surface patch of skin over a superficial artery of a subject, an actuator mounted over the elastomeric sensor array, and a camera mounted on the actuator to capture image data processed by a controller. The elastomeric sensor array is digitally or mechanically divided into a pulsating region and a non-pulsating region. A controller measures an arterial pressure waveform having motion artifacts caused by deformation of the elastomeric sensor array on the skin above the artery in a pulsating region and above the skin near the artery in a non-pulsating region. Further, the controller measures spatio-temporal information of motion artifacts in the pulsation region and the non-pulsation region, and corrects the arterial pressure waveform by removing the motion artifacts based on the spatio-temporal information to measure the physiological parameter.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 525,519, filed July 7, 2023, which is incorporated herein by reference in its entirety.

[0003] Technical Field of the Invention

[0004] This invention relates to non-invasive periodic or continuous measurement of physiological parameters. More specifically, this invention relates to spatiotemporal detection and correction of motion artifacts using skin surface displacement and force, so as to accurately capture and assess arterial pressure (or other physiological phenomena) waveforms even in motion. Background Technology

[0005] Real-time capture and evaluation of arterial pressure waveforms to measure physiological parameters are essential for monitoring and assessing health both within and outside healthcare facilities. Physiological parameters are, but are not limited to, heart rate, blood pressure, respiratory rate, cardiac output, and other advanced hemodynamic parameters. Physiological parameters typically provide immediate information for making medical decisions during illness and surgical procedures, as well as long-term information that helps in the prevention and management of chronic diseases.

[0006] WO2019 / 195120A1, entitled "Tactile Blood Pressure Imager" (incorporated herein by reference in its entirety), describes a tactile blood pressure imager (TBPI) that includes an array of optomechanical force sensors to measure skin deformation or displacement above a surface artery, such as the radial artery at a patient's wrist. The TBPI provides periodic and continuous measurements of arterial pressure waveforms for measuring blood pressure, heart rate, expiratory rate, and higher hemodynamic parameters. WO / 2022 / 035841, entitled "Optomechanical Method to Measure Arterial Pulse and Assess Cardiopulmonary Hemodynamics" (incorporated herein by reference in its entirety), discloses and discusses an optomechanical sensor system with a structure and process capable of measuring surface displacement due to arterial pressure.

[0007] Unintentional motion can cause mechanical forces that can result in artifact measurements or inaccurate measurements of the arterial pressure waveform during measurement of the arterial pressure waveform. For example, routine operation of the device, motion of the wrist, hand, arm, body, or vibrational motion from a vehicle or conveyance can produce artifact forces that can corrupt the arterial pressure waveform, making it challenging to measure physiological parameters. In addition, vibrational motion during patient transport can have a similar frequency as the pulse. Movement of the hand, fingers, or other body parts can cause displacement of soft tissue and skin above the artery, which can corrupt the arterial pressure measurement. These motion artifacts cause superimposed deformation and displacement of soft tissue and skin, resulting in artifact measurements of the arterial pressure waveform. A particular challenge posed by trying to separate motion artifacts from physiological signals, such as physiological signals representing arterial pressure, is that the frequency of the motion artifact and the frequency of the signal of interest can overlap, making it difficult to simply subtract the artifact signal to reveal the signal of interest. Separating the artifact signal from the signal of interest is not simple due to the overlapping information in the spectral content of the signal of interest and the artifact signal. Complicating the separation of the signal of interest further is that over time, for example, tissue can relax, which can cause a slow drift of the signal of interest, making it difficult to track over time while removing the artifact noise from the signal of interest. Accordingly, aspects disclosed herein can not only address motion artifacts, but also blood pressure drift due to tissue relaxation.

[0008] The previously cited patent applications do not provide a method to mitigate these motion artifacts and blood pressure drift. Accordingly, there is a need to detect and correct for motion artifacts / blood pressure drift to accurately measure the arterial pressure waveform.

[0009] SUMMARY

[0010] Without limiting the scope and details of the present disclosure described herein, an important aspect of the motion artifact / blood pressure drift correction techniques herein is that it takes into account the effects of the region just outside or near the pulsatile region (referred to herein as the "non-pulsatile" region) to eliminate or suppress these effects to provide a true and accurate representation of the physiological parameter (e.g., arterial pressure). In other words, the present disclosure does not focus on the pulsatile region alone; it also examines the nearby region to help isolate the artifact motion that corrupts or reduces the signal of interest (e.g., blood pressure). A single sensor (e.g., an imaging device, such as a camera) can be aimed at the entire region including the pulsatile region and the non-pulsatile region, or multiple different sensors (e.g., two) can each be aimed at a respective pulsatile region and non-pulsatile region, with their outputs combined into a signal processor to determine the contribution or effect of the drift or motion artifact on the signal of interest. The signal processing techniques disclosed herein are then used to characterize the motion artifact / drift and correct or compensate for it (e.g., by eliminating or suppressing its effect on the physiological waveform determination). When the physiological parameter being detected is blood pressure, for example, motion of the subject or motion imparted to the subject (e.g., by external vibrations) can corrupt the blood pressure measurement, making its reading unreliable or untrustworthy. In a cuff blood pressure system, the subject must remain still and the arm should rest against a stationary object that does not receive any external motion forces (e.g., vibrations). Aspects of the present disclosure allow for accurate readings of physiological parameters even when the region being measured is moving or is subject to external motion forces. Machine learning techniques improve the algorithms and signal processing to get better at detecting and removing motion artifact / drift effects over time. According to another aspect herein, the sensing device is a simple imaging camera that takes pictures or real-time images or videos of the skin's pulsatile motion, for example, as blood passes through an artery. These images are processed by focusing on both the pulsatile region and the non-pulsatile region to isolate the motion artifact / drift effects and remove them from the signal.

[0011] In an aspect, the embodiments herein provide a method for spatiotemporal management of motion artifacts in an arterial pressure waveform. The method includes receiving image data from an elastomeric sensor array made of an elastic material mounted above a bottom surface of an actuator. The actuator is placed above the elastomeric sensor array in contact with a subject's skin. The method includes measuring an arterial pressure waveform based on the image data. The arterial pressure waveform includes motion artifacts caused by a deformation of the elastomeric sensor array over the skin in a pulsatile region above an artery, and a deformation of the elastomeric sensor array over the skin adjacent to the artery in a non-pulsatile region includes motion artifacts but does not include the arterial pressure waveform. Further, the method includes determining spatiotemporal information of the motion artifacts in the pulsatile region and the non-pulsatile region, correcting the arterial pressure waveform by removing the motion artifacts based on the spatiotemporal information, and determining a physiological parameter of the subject based on the corrected arterial pressure waveform.

[0012] In embodiments, determining the spatiotemporal information of motion artifacts in the pulsatile region and the non-pulsatile region comprises detecting motion artifacts in the pulsatile region, detecting motion artifacts in the non-pulsatile region, determining spatiotemporal information of motion artifacts in the pulsatile region and motion artifacts in the non-pulsatile region.

[0013] In embodiments, the pulsatile region and the non-pulsatile region are digitally separated by estimating a displacement of the image data at each time point, determining a variance of the signal over time based on the displacement, performing a temporal Fourier transform of the displacement of the image data at all locations based on the variance of the signal over time, determining regions having a transformed displacement satisfying a predefined threshold, and segmenting the regions having the transformed displacement satisfying the predefined threshold into the pulsatile region and segmenting the regions having the transformed displacement not satisfying the predefined threshold into the non-pulsatile region.

[0014] In embodiments, detecting motion artifacts in the non-pulsatile region comprises determining a plurality of parameters associated with the non-pulsatile region, determining a reference region in the image data based on the plurality of parameters, filtering a remaining pulsatile signal from the reference region, and detecting motion artifacts in the non-pulsatile region based on the filtered remaining signal from the reference region.

[0015] In embodiments, the remaining pulsatile signal from the reference region is filtered by applying one of a median average filter, a moving median average filter, and a Fourier transform low pass filter.

[0016] In embodiments, the plurality of parameters determines a reference region that surrounds the pulsatile region, is at a sufficient distance from the pulsatile region to remove the influence of low amplitude of the arterial pressure waveform, and has a maximum possible area to maximize spatial pattern recognition for attributing changes within the pulsatile region caused by artifact motion.

[0017] In embodiments, detecting motion artifacts in the non-pulsatile region based on the filtered remaining signal from the reference region comprises determining a time derivative using a frequency filter, and detecting motion artifacts in the non-pulsatile region based on the time derivative.

[0018] In embodiments, detecting motion artifacts in the pulsatile region comprises matching a template to detect changes in pulse waveform morphology that trigger a high frequency artifact flag, and detecting motion artifacts in the pulsatile region based on the template.

[0019] In embodiments, correcting the arterial pressure waveform by removing motion artifacts based on spatiotemporal information includes detecting a type of motion artifact in the pulsatile region and the non-pulsatile region, detecting whether an artifact flag is raised, and applying a correction technique to remove the motion artifact from the arterial pressure waveform based on the spatiotemporal information, the raised artifact flag, and the type of motion artifact in the pulsatile region and the non-pulsatile region.

[0020] In another aspect, embodiments herein provide a system for spatiotemporal management of motion artifacts. The system includes an elastomeric sensor array made of an elastic material in patch contact with a surface of a subject's skin, an actuator mounted on a top surface of the elastomeric sensor array, and a controller communicatively connected to the light source, the camera, and the actuator. The actuator has an actuated state in which a controlled amount of pressure separates a spatiotemporal signal from the subject's artery. The camera captures image data of the elastomeric sensor array. Based on the image data, the controller is configured to measure an arterial pressure waveform. The arterial pressure waveform includes motion artifacts caused by deformation of the elastomeric sensor array on the skin above the artery in a pulsatile region and deformation of the elastomeric sensor array on the skin adjacent to the artery in a non-pulsatile region. The controller is configured to determine spatiotemporal information of the motion artifacts in the pulsatile region and the non-pulsatile region, correct the arterial pressure waveform by removing the motion artifacts based on the spatiotemporal data, and determine a physiological parameter of the subject based on the corrected arterial pressure waveform.

[0021] In embodiments, the elastomeric sensor array is mechanically separated by splitting the elastomeric sensor array into a first sensor array and a second sensor array in contact with a surface of a subject's skin. The first sensor array and the second sensor array are separated by a predefined distance. The actuator is mounted above the first sensor array and the second sensor array. The first sensor array is mounted on the skin above the artery representing the pulsatile region and the second sensor array is mounted on the skin adjacent to the artery representing the non-pulsatile region.

[0022] In embodiments, the elastomeric sensor array and the actuator are mechanically separated by splitting the elastomeric sensor array into a first sensor array and a second sensor array in contact with a surface of a subject's skin. The first sensor array and the second sensor array are separated by a predefined distance. The actuator is split into a first actuator mounted above the first sensor array representing the pulsatile region and a second actuator mounted above the second sensor array representing the non-pulsatile region.

[0023] These and other aspects of the embodiments herein will be better appreciated and understood when considered in connection with the following description and accompanying drawings. It is to be expressly understood, however, that while the description and drawings depict preferred embodiments and various specific details thereof, it is stowed by way of illustration and not by way of limitation. Various changes and modifications can be made within the scope of the embodiments herein without departing from the scope thereof and the embodiments herein include all such modifications. BRIEF DESCRIPTION OF DRAWINGS

[0025] The proposed spatio-temporal based detection and correction of motion artifacts is shown in the attached figures, in which the same reference letters indicate corresponding parts in the various figures. The embodiments herein will be better understood and appreciated from the following description taken with the drawings in which: Figure 1 An optical mechanical system for spatio-temporal based detection and correction of motion artifacts to accurately measure arterial pressure waveforms is shown in accordance with the embodiments disclosed herein; Figure 2 A scenario of digital separation of elastomeric sensor array is shown in accordance with the embodiments disclosed herein; Figure 3 A scenario of mechanically separating elastomeric sensor array is shown in accordance with the embodiments disclosed herein; Figure 4 A scenario of mechanically separating elastomeric sensor array and actuator is shown in accordance with the embodiments disclosed herein; Figure 5 A force sensor array system for spatio-temporal based detection and correction of motion artifacts to accurately measure arterial pressure is shown in accordance with the embodiments disclosed herein; a scenario is shown in which digital separation of force sensing array is performed and an actuator is used to exert force / pressure on the artery; Figure 6 A force sensor array system for spatio-temporal based detection and correction of motion artifacts to accurately measure arterial pressure is shown in accordance with the embodiments disclosed herein; a scenario is shown in which digital separation of force sensing array is performed and a belt system is used to exert force / pressure on the artery instead of an actuator; and Figure 7 A flow chart showing a method for spatio-temporal based detection and correction of motion artifacts to measure arterial pressure waveforms in accordance with the embodiments disclosed herein.

[0026] To the extent possible, reference numerals have been used consistently throughout the drawings to refer to the same or like components. Moreover, those skilled in the art will appreciate that the elements in the figures are shown for the purpose of illustrating preferred aspects of the application only and can not be drawn to scale. For example, the dimensions of some of the elements in the figures can be exaggerated relative to other elements to help improve the understanding of the principles of the application. Further, conventional symbols are used in the drawings to indicate certain components. Finally, the figures can show only those specific details that are necessary to understand the embodiments of the application, to the exclusion of other details that are believed to be of little consequence to an appreciation of the application by those skilled in the art. DETAILED DESCRIPTION

[0028] The embodiments herein, and various features and advantageous details thereof, are explained more fully below with reference to the non-limiting embodiments illustrated in the accompanying drawings and described in the following description. It should be understood, however, that the following description is by way of example only and is not intended to limit the scope of the embodiments herein. As used herein, the example is intended to be illustrative only and are not in any way limiting of the embodiments herein as described herein. Rather, it is submitted that the scope of the embodiments herein would be covered by any technology that is considered to be a descendant of the technology described herein. Many changes and modifications can be suggested to the embodiments herein, and it is understood that each of the embodiments herein includes all such changes and modifications of the embodiments herein. It is therefore intended that the embodiments herein be taken only by way of the example and to be limited by the scope of the appended claims and their equivalents.

[0029] Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. Moreover, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments can be combined with other embodiments to form new embodiments.

[0030] Reference is now made to the drawings, and more specifically to Figures 1 to 7 wherein like reference numerals refer to corresponding features throughout the drawings.

[0031] Figure 1 An optical-mechanical system 100 for spatiotemporal-based detection and correction of motion artifacts to accurately measure arterial pressure waveforms is shown in accordance with the embodiments disclosed herein. The optical-mechanical sensor 100 includes an elastomeric sensor array 101, an optional actuator 102, a light source 103a, a video camera 103b, and a controller 104. Although in this example the sensor array is an elastomeric sensor array, the present disclosure contemplates other types of sensor arrays, including sensors that include piezoresistive materials or substrates or other materials or devices that are capable of deforming in response to motion of in-vivo skin.

[0032] The elastomeric sensor array 101 is in surface patch contact or direct contact with the subject's skin 106 and an actuator is mounted above the elastomeric sensor array 101. The elastomeric sensor array 101 forms the interface between the actuator 102 and the skin 106. Below the skin is soft tissue 107 and below the soft tissue is bone 108. In other aspects, the sensor array has a material or substrate that can deform in response to motion of the skin and the substrate or material is sensitive enough to move flexibly and conformally with the motion of the skin. The idea here is to exaggerate or transfer the skin motion to the substrate / material so that those motions can be acquired by the video camera 103b, for example, as spatiotemporal information.

[0033] The actuator 102 is mounted above the elastomeric sensor array 101. In embodiments, the actuator 102 is a controlled balloon. The actuator 102 is inflated to achieve an actuated state in which a controlled amount of pressure separates spatiotemporal signals from the subject's artery.

[0034] In embodiments, an optical mechanical sensor system 100 is described herein.

[0035] The elastomeric sensor array 101 is mounted below the surface of the actuator 102. Deformation or displacement of the underlying skin 106 causes deformation or displacement of the elastomeric sensor array 101. The video camera 103b captures the spatiotemporal elastomeric sensor array deformation as continuous video (image data) with sub-millimeter resolution.

[0036] The controller 104 is communicatively connected to the optical-mechanical sensor system 100. The controller 104 is configured to measure an arterial pressure waveform based on the image data. The arterial pressure waveform includes motion artifacts caused by the deformation of the elastomeric sensor array over the skin above the artery in the pulsatile region and the deformation of the elastomeric sensor array over the skin adjacent to the artery in the non-pulsatile region. The non-pulsatile region can be a region immediately adjacent or proximate to a region of skin under which an artery exists. The non-pulsatile region does not overlap with the pulsatile region such that no portion of the non-pulsatile region is directly over the artery of interest. No appreciable arterial signal or skin deformation caused by the pulsatile artery is detected in the non-pulsatile region. In other words, the arterial pulse is too weak or non-existent in the non-pulsatile region for the sensor array to reliably detect or measure physiological parameters therein. Depending on the weight and height of the subject, the non-pulsatile region can be located a predetermined distance from the pulsatile region. For example, the non-pulsatile region can be located 1 mm, 2 mm, 3 mm, 4 mm, 5 mm, 6 mm, 7 mm, 8 mm, 9 mm, 10 mm, or 11-15 mm, or 15-25 mm from the artery. Those skilled in human physiology will appreciate that the pulsatile region is a region in which physiological parameters can be reliably detected. The non-pulsatile region is a region in which such parameters cannot be detected at all, or cannot be detected in any reliable manner (e.g., there can be a weak pulse, but it is too weak to reliably measure). The controller 104 is configured to determine the spatiotemporal information of the motion artifacts in the pulsatile region and in the non-pulsatile region, correct the arterial pressure waveform by removing or suppressing the motion artifacts based on the spatiotemporal information, particularly including the motion artifacts in the non-pulsatile region, and determine physiological parameters of the subject based on the corrected arterial pressure waveform.

[0037] During routine operation, motion of the device, motion of the wrist, hand, arm, or body, or external vibrational motion from a vehicle or conveyance (e.g., within an ambulance or emergency vehicle or airplane) can produce unwanted artifact forces within the pulsatile region that can corrupt or degrade the arterial pressure waveform, making it challenging to measure physiological parameters, including blood pressure. In addition, the motion artifact forces within the pulsatile region can have similar temporal characteristics to the arterial pressure waveform of interest, making it difficult to distinguish actual changes in physiological parameters from those changes caused by artifact-induced pseudo-changes. Vibrational motion during patient transport can have similar frequencies to the arterial pressure waveform (signal of interest), making them difficult to separate from the signal of interest. Also, motion of the hand or fingers can cause displacement of the soft tissue 107 and skin 106 over the artery 109 at the wrist within seconds, which is on the same time scale as the arterial pulse waveform. These motion artifacts cause superimposed deformations and displacements of the elastomeric sensor array, and if not corrected, can result in artifact measurements of the arterial pulse waveform (and thus inaccurate measurements). Thus, the controller 104 measures the deformations or displacements of the elastomeric sensor array not only over the skin above the artery 109 in the presence of arterial pulsations (pulsatile region), but also over the skin adjacent to the artery 109 in the absence of arterial pulsations (non-pulsatile region). In embodiments, the elastomeric sensor array 101 is digitally or mechanically separated into the pulsatile region and the non-pulsatile region. Reference is made to Figure 2 Digital separation of the elastomeric sensor array 101 is described, while reference is made to Figures 3-4 Mechanical separation is described.

[0038] In embodiments, a force sensing array system 200 is described herein.

[0039] The force sensor array 201 can be made of a piezoresistive material and mounted under the surface of the actuator 202. Deformation of the underlying skin 106 causes deformation or displacement of the displaceable elements in the force sensor array 201. The controller 203 has a high resolution analog-to-digital converter data acquisition system that captures data from all array elements and converts the captured data into a digital image, as shown in Figure 5

[0040] In embodiments, a force sensing array system 200 is described herein.

[0041] ​The force sensor array 201 made of piezoresistive material can be mounted directly on top of the skin 106 (e.g., in direct contact with the skin 106). The sensor system 200 can be held together using a belt 202a or equivalent system (used to provide a holding force and increase the reaction force on the artery by tightening the belt 202a). The deformation of the underlying skin 106 causes deformation or displacement of the elements in the force sensor array 201. The force sensing array 201 described herein can alternatively be one of the following: a capacitive sensor array, a force sensing resistor array, or a load cell array. The controller 203 has a data acquisition system that captures data from all array elements, a transducer that converts changes in resistance or capacitance from each element to an analog voltage, and a high-resolution analog-to-digital converter that converts the analog signal to a digital signal. The digital signals form a digital image when organized into an array, as shown in Figures 5-6 The same process described above can be used to process the data.

[0042] The main basis of the proposed disclosure is the recognition that the forces that cause artifact deformation and displacement affect both the pulsatile region 110 and the non-pulsatile region 111 of the elastomeric sensor array 101, while the physiological changes that cause arterial waveform changes are primarily limited to the pulsatile region 110. Therefore, using spatial information from the non-pulsatile region 111, artifact deformation that is not related to physiological changes can be captured, quantified, and used to correct the arterial pulse waveform.

[0043] Figure 2 A scenario of digital separation of the elastomeric sensor array 101 according to the embodiments disclosed herein is shown. As Figure 1 described, the optical mechanical system 100 is in contact with the skin 106. The layers of soft tissue 107 under the skin 106 are in contact with the bone 108. The layers of soft tissue 107 include the artery 109. The optical mechanical system 100 measures the deformation or displacement on the skin above the artery 109 with arterial pulsation (referred to as the pulsatile region 110) and also measures the deformation or displacement on the skin above the artery with no arterial pulsation (referred to as the non-pulsatile region 111).

[0044] The elastomer sensor array deformation on the skin 106 above the artery 109 in the pulsatile region 110 is caused by the physical forces generated from the pulsation of the artery 109. It is transmitted through the artery 109 to the soft tissue 107 and the skin 106. This elastomer sensor array deformation or displacement is captured continuously as image data by the camera 103b. In one embodiment, a computer vision engine residing in the controller 104 transforms the image data into a linear time signal by first estimating the motion caused by the underlying pulse, then localizing the pulse by integrating the motion vectors, then segmenting it, as shown in Figure 2

[0045] The pulsatile region 110 and the non-pulsatile region 111 are first outlined. The goal is to digitally separate the elastomer sensor array 101 into the pulsatile region or area 110 and the non-pulsatile region 111. To detect motion artifacts in the pulsatile region 110 and the non-pulsatile region 111, the digital separation of the pulsatile region 110 and the non-pulsatile region 111 can be performed by estimating the displacement of the image data at each time point, determining the variance of the signal over time based on the displacement, performing a time Fourier transform of the displacement of the image data at all locations based on the variance of the signal over time, determining regions having transformed displacements satisfying a predefined threshold, and segmenting the regions having transformed displacements satisfying the predefined threshold into the pulsatile region 110 and segmenting the regions having transformed displacements not satisfying the predefined threshold into the non-pulsatile region 111.

[0046] In embodiments, detecting motion artifacts in the non-pulsatile region includes determining a plurality of parameters associated with the non-pulsatile region, determining a reference region 112 in the image data based on the plurality of parameters, filtering a residual pulsatile signal from the reference region 112, and detecting motion artifacts in the non-pulsatile region 111 based on the filtered residual signal from the reference region 112. The residual pulsatile signal from the reference region 112 is filtered by applying a median average filter, a moving median average filter, and a Fourier transform low pass filter.

[0047] In examples, in the case of a single elastomer sensor array 101, the pulsatile region 110 is first identified by (A) estimating the displacement of the image (Ax, Ay) at each time point, (B) second, determining the variance of the signal over time V(x, y, t) = [<Ax(t) + <Ay(t) >]2+ [<Ay(t) - <Ay(t) >]2]1 / 2, (C) third, performing a time Fourier transform of the displacement of the image at all locations based on the variance of the signal over time, (D) fourth, determining regions having transformed displacements satisfying a predefined threshold, and (E) fifth, segmenting the regions having transformed displacements satisfying the predefined threshold into the pulsatile region 110. t 2 2 1 / 2 ​​​​and the time Fourier transform of the displacement of all the positions images F(x, y, w) = F[Δx(t), Ay(t)]. (C) Finally, thresholding the region with sufficient power at the heart rate (about 40 BPM - 200 BPM) allows to segment the pulsatile region 110 where the physiological signal is present.

[0048] The non-pulsatile region 111 is generally defined as the region of the elastomer sensor array outside the pulsatile region 110. The non-pulsatile region 111 can also be refined into reference regions 112, which consist of specific regions of the non-pulsatile region 111 used for digital signal subtraction. In particular, these reference regions 112 are selected to have specific additional characteristics of motion detection that improve motion correction. The selection of the reference regions 112 includes a number of parameters. One of the reference regions 112a is based on parameters around the pulsatile region 110, including proximity to the pulsatile region 110 to capture deformations in the elastomer sensor array as close as possible to the pulsatile region 110, while being at a sufficient distance from the pulsatile region 110 to remove low amplitude effects that can be caused by actual physiological signals, and as large an area as possible to maximize spatial pattern recognition for imputing changes within the pulsatile region 110 caused by artifact motion (also known as motion artifacts). Despite the use of these parameters, there can be some residual arterial pulsations in the reference region 112 that cause deformations. As a result, it is necessary to filter out the residual pulsatile signals. In one embodiment, a moving median average filter is applied directly to the images before estimating the deformations of the elastomer sensor array. Alternatively, a median average filter can be applied directly to the linear physiological signals output by the device, with a window length determined by the heart rate of the subject under investigation. Additional methods include time Fourier transform of the signals and low pass filtering below the heart rate.

[0049] Figure 3 A scenario of mechanically separating the elastomer sensor array 101 according to the embodiments disclosed herein is shown. In an embodiment, the elastomer sensor array 101 is mechanically separated into a first elastomer sensor array 101a and a second elastomer sensor array 101b. A single actuator 102 is mounted above the first elastomer sensor array 101a and the second elastomer sensor array 101b and is optically accessible by a single camera 103b.

[0050] In this embodiment, one elastomer sensor array 101a is placed over the artery 109, separating the mechanical forces caused by arterial pulsation to the first elastomer sensor array 101a, representing the pulsatile region 110. Then, the physically separated second elastomer sensor array 101b is designed to be proximal (but not in contact) to the artery 109, and is defined as the non-pulsatile region 111 without arterial pulsation. The pulsatile and non-pulsatile regions 110 and 111 will have deformations reflecting global artifact motion from the subject. The elastomer sensor array 101 is split into arbitrary proportions (e.g., 75% of the camera field of view with the pulsatile elastomer sensor array and 25% of the visual field of view with the non-pulsatile elastomer sensor array). Alternatively, a second camera (not shown, but in the region of the light source 103a) can be arranged so that the camera 103b is aligned to the pulsatile region, and the second camera is aligned to the non-pulsatile region. While their fields of view can overlap, each camera focuses on the region within the respective pulsatile and non-pulsatile regions. The non-pulsatile region represents a region of skin motion that is not affected by the blood passing through the artery 109. The skin in the non-pulsatile region can move due to other reasons (e.g., bending of the wrist), but these motions are not relevant to the arterial waveform of interest.

[0051] Figure 4 Scenarios of mechanically separating (e.g., different hardware device structures) the elastomer sensor array 101 and the actuator 102 are shown in accordance with the embodiments disclosed herein. In embodiments, the elastomer sensor array 101 is split into a first elastomer sensor array 101a and a second elastomer sensor array 101b. The first elastomer sensor array 101a and the second elastomer sensor array 101b are separated. The actuator 102 is split into a first actuator 102a mounted over the first elastomer sensor array 101a representing the pulsatile region 110 and a second actuator 102b mounted over the first elastomer sensor array 101b representing the non-pulsatile region 111.

[0052] In this example, two separate actuators 102a and 102b are used to mechanically separate and pneumatically actuate the elastomeric sensor array 101, and can be optically accessed by a single camera 103b (or alternatively through a second camera as discussed above). One elastomeric sensor array 101a is placed over the artery 109, and the entire elastomeric sensor array 101a is defined as the pulsatile region 110. Then, a second physically separate elastomeric sensor array 101b is designed to be proximal to the artery 109 and represents the non-pulsatile region 111. The non-pulsatile region is a region proximal to the artery where the deformation of the skin is not caused by the blood passing through the artery, but can be caused by other artificial or external events, such as bending of the wrist or waving of the hand or external vibrations transmitted to the non-pulsatile region of the skin. The elastomeric sensor array 101 is split into arbitrary proportions (e.g., a pulsatile elastomeric sensor array with 75% of the camera’s field of view and a non-pulsatile elastomeric sensor array with 25% of the visual field of view). Similarly, the actuators 102 are split into arbitrary proportions.

[0053] Figure 5 A method of spatiotemporal-based detection and correction of motion artifacts to measure an arterial pressure waveform is shown in accordance with the embodiments disclosed herein. At step SI, the method includes receiving image data from an elastomeric sensor array (sensor array) 103 mounted on the bottom surface of an actuator 102. The elastomeric sensor array 101 is in contact with the skin of a subject. The actuator has an actuated state in which a controlled amount of pressure separates a spatiotemporal signal or other signal from an artery of the subject. The elastomeric sensor array 101 deformation is captured by a camera 103b as image data.

[0054] At step S2, the method includes measuring an arterial pressure waveform based on the image data. The arterial pressure waveform includes motion artifacts caused by the deformation of the elastomeric sensor array over the skin above the artery in the pulsatile region and the deformation of the elastomeric sensor array over the skin proximal to the artery in the non-pulsatile region. In embodiments, generating the arterial pressure waveform based on the image data includes transforming the image data into a linear time signal and generating the arterial pressure waveform based on the linear time signal.

[0055] At step S3, the method includes digitally or mechanically separating the pulsatile region 110 and the non-pulsatile region 111 in the image data. Regarding Figure 2 Digital separation of the pulsatile region 110 and the non-pulsatile region 111 is described. Regarding Figures 3-4 Mechanical separation of the pulsatile region 110 and the non-pulsatile region 111 is described.

[0056] At step S4, the method includes detecting motion artifacts in the pulsatile region 110. In embodiments, the motion artifacts in the pulsatile region 110 are matched to a template to detect changes in pulse waveform morphology that trigger a high-frequency artifact flag.

[0057] At step S5, the method comprises detecting motion artifacts in the non-pulsatile region 111 and, in embodiments, detecting reference regions 112 in the image data.

[0058] The non-pulsatile region 111 can also be refined into reference regions, which consist of specific regions of the non-pulsatile region 111 used for digital signal subtraction. In particular, these reference regions 112 are selected to have specific additional characteristics that improve motion detection and motion correction. The selection of reference regions includes a number of parameters, such as proximity to the pulsatile region to capture elastomeric sensor array deformation, distance from the pulsatile region to remove low amplitude, and length of the pulsatile region.

[0059] At step S6, the method comprises detecting motion artifacts in the non-pulsatile region 111. In embodiments, the residual pulsatile signal from the reference regions 112 is filtered by applying one of a median average filter, a moving median average filter, and a Fourier transform low pass filter. Motion artifacts in the non-pulsatile region 111 are then detected based on the filtered residual signal from the reference regions 112.

[0060] At step S7, the method comprises determining the spatiotemporal information of motion artifacts in the pulsatile region and the non-pulsatile region. The motion artifacts described herein fall into two categories: (1) high frequency artifacts and (2) slow drifts in blood pressure caused by, for example, tissue relaxation. High frequency artifacts are detected using a frequency filter and by calculation of the time derivative.

[0061] In embodiments, a spatial method is used for motion artifact detection in the non-pulsatile region 111. Once a reference region 112 is identified in the non-pulsatile region 111, the reference region 112 is analyzed to detect motion artifacts in the non-pulsatile region 111.

[0062] In embodiments, a temporal method is used for motion artifact detection in the pulsatile region 110. This is a temporal method that detects high frequency artifacts within the pulsatile region 110. High frequency artifact detection includes template matching to detect changes in pulse waveform morphology that trigger a high frequency artifact flag. Frequency and derivative cutoff values and beat templates are derived from a large set of artifact data that has been acquired. These cutoff values are updated as additional artifact data is added to the database.

[0063] The net displacement of the reference region 112 is input into a voting model that accumulates votes that increase or decrease over time. The votes are exponentially weighted based on how old they are. Once a predefined threshold has been met, a drift flag is set. In addition, a dictionary of spatial artifact patterns is generated offline using a large dataset of artifact data. The spatial displacement pattern in the entire elastomer sensor array is compared to the motion artifact dictionary. If the spatial pattern matches beyond a predefined threshold, an artifact flag is also set.

[0064] In step S7, the method includes correcting the arterial pressure waveform by removing motion artifacts based on the spatio-temporal information. Correcting the arterial pressure waveform includes detecting a type of motion artifact in the pulsatile region and the non-pulsatile region, detecting whether the artifact flag is set, and applying a correction technique to remove the motion artifact from the arterial pressure waveform based on the spatio-temporal information, the set artifact flag, and the type of motion artifact in the pulsatile region and the non-pulsatile region.

[0065] In embodiments, a spatial method for motion artifact correction is used. Once the artifact flag has been set, a correction technique is applied to the arterial pressure waveform. The selection and use of a particular correction technique depends on the particular type of artifact flag and the spatial pattern of the motion artifact.

[0066] In the case of high frequency artifacts, the motion artifact detector keeps the flag in the open state until the motion artifact has been eliminated. It is assumed that the motion artifact length is less than a predefined threshold. In this case, the new physiological signal is calibrated to the most recent high-fidelity signal before the motion artifact.

[0067] In the case of blood pressure drift caused by tissue relaxation, which usually occurs slowly, the signals from the reference region 112 and the pulsatile region 110 are compared. An optimization routine is used to find the coefficients A and B that multiply the displacement in the reference region, i.e., A Δx ref , B Δy ref that minimizes the mean squared error between the pulsatile signal and the scaled reference signal over a predefined time period. This initial correction technique uses the reference region with the largest spatial separation from the pulsatile region 110. If the mean squared error cannot be achieved, the spatial pattern in the reference region 112 is matched to a dictionary of artifact deformations. A larger reference region is used to maximize the spatial features of the motion artifact. The spatial signal is projected onto the dictionary components g(x, y) to determine the coefficients µ such that µ g(x ref , y ref ) = (Δx ref , Δy ref ). The dictionary components are then used to correct the arterial pressure waveform by (Δx, Δy) = (A + µ Δx, B + µ Δy). The corrected arterial pressure waveform is then output. corr corr ​) = (Ax puls , Ay pulse ) = (Ax pulse , Ay pulse ) - h(x corr , y corr ) to compute the corrected signal. If the motion artifact is not sufficiently similar to the predefined dictionary element, then the reference region is interpolated into the pulsatile region using a polynomial interpolation h(x, y). The corrected signal is then estimated by (Ax puls , Ay pulse ) = (Ax pulse , Ay pulse ) - h(x, y).

[0068] The end result of the motion artifact processing pipeline is to determine a correction in the pulsatile region based on the spatiotemporal information of the non-pulsatile region. The main difference between the technology presented here and previously described technologies is due to the spatial manifestation of the artifact on the elastomer sensor array. For example, previously described digital video stabilization techniques utilize a reference region to correct for camera shake. These techniques generally rely on motion that affects the individual camera, resulting in an artifact that manifests as a relatively simple affine transformation in the foreground or background of the image. These simple motions can be directly subtracted from the reference region. Because the motion artifact affects not only the camera but also the elastomer sensor array, the motion artifact is more complex in space. As a result, existing camera shake techniques are no longer applicable and do not allow for off-the-shelf applications for motion correction, making the development of the technology described above necessary. In particular, the motion correction technique uses spatial and temporal information to determine a reference region and uses complex spatial reconstruction techniques to error correct the pulsatile region.

[0069] At step S9, the method includes determining a physiological parameter of the subject based on the corrected arterial pressure waveform. In embodiments, once the error correction has been performed, the physiological parameter can be estimated as described in PCT Patent Application Publication No. WO 2022 / 035841 Al, entitled “Optomechanical Method to Measure Arterial Pulse and Assess Cardiopulmonary Hemodynamics,” filed August 10, 2021. The motion artifact correction process produces additional changes in the device output.

[0070] Machine learning techniques can be used to improve motion artifact removal with high accuracy. A learning model is created from videos showing only pulsations and other videos without pulsations but with motion artifacts (e.g., hand or wrist motion). The model is trained to isolate only the pulse signal of interest and adaptively learn to distinguish motion artifacts from the signal of interest and select only the pulsation signal. Artifact detection and drift correction are also enhanced by the model. For example, in the case of blood pressure drift caused by, for example, tissue relaxation around the pulsation area, the coefficients for the optimization routine discussed above can be adjusted by the model and, additionally or alternatively, a dictionary of artifact morphologies can be updated or modified based on a drift model in communication with the learning model to determine the accuracy with which the drift has been corrected, separating the pulse signal of interest from motion artifacts.

[0071] The foregoing description of a specific implementation will so fully reveal the general nature of the methods herein that others can modify and / or adapt various applications thereof without departing from the general concept, and therefore, such modifications and adaptations should and are intended to be comprehended within the meaning and range of equivalents of the disclosed implementations. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, although the methods herein have been described in terms of preferred implementations, those skilled in the art will recognize that the methods herein can be practiced with modifications within the spirit and scope of the implementations as described herein.

Claims

1. A method of spatio-temporal management of motion artifacts in a physiological waveform, comprising: receiving image data from a sensor array in contact with a subject's skin; measuring a physiological waveform based on the image data, wherein the physiological waveform includes a deformation or motion on the skin above an artery in a pulsatile region by the sensor array in response to an externally applied positive pressure force to it and a motion artifact caused by the sensor array on the skin above the artery in a non-pulsatile region; determining information related to the motion artifact in the pulsatile region and in the non-pulsatile region; correcting the physiological waveform by removing or suppressing the motion artifact based on at least the information related to the motion artifact in the non-pulsatile region; and determining a physiological parameter of the subject based on the corrected physiological waveform.

2. The method of claim 1, wherein the physiological waveform represents an arterial pressure, and wherein determining the information comprises: detecting a feature of the motion artifact in the pulsatile region; detecting the feature or another feature of the motion artifact in the non-pulsatile region; determining information of the motion artifact in the pulsatile region and information of the motion artifact in the non-pulsatile region.

3. The method of claim 2, in response to the pulsatile region and the non-pulsatile region not being separated, a controller digitally separates the pulsatile region and the non-pulsatile region in the image data by: estimating a displacement of the image data at each time point; determining a variance of a signal over time based on the displacement; performing a time Fourier transform of the displacement of the image data at all locations based on the variance of the signal over time; determining regions having a transformed displacement satisfying a predefined threshold; and segmenting the regions having the transformed displacement satisfying the predefined threshold into the pulsatile region and segmenting the regions having the transformed displacement not satisfying the predefined threshold into the non-pulsatile region.

4. The method of claim 2, wherein detecting the feature of the motion artifact in the non-pulsatile region comprises: determining a plurality of parameters associated with the non-pulsatile region; determining at least one reference region in the image data based on the plurality of parameters; filtering a residual pulsatile signal from the at least one reference region; and detecting the feature of the motion artifact in the non-pulsatile region based on the filtered residual pulsatile signal from the at least one reference region.

5. The method of claim 4, wherein the residual pulsatile signal from the at least one reference region is filtered by applying one of a median average filter, a moving median average filter, and a Fourier transform low pass filter.

6. The method of claim 4, wherein the plurality of parameters determine the reference region, the reference region surrounds the pulsatile region, is at a sufficient distance from the pulsatile region to remove low amplitude effects of the arterial pressure waveform, and has a maximum possible area to maximize spatial pattern recognition for attributing changes within the pulsatile region to the artifact motion.

7. The method of claim 4, detecting the feature or another feature of motion artifact in the non-pulsatile region based on the filtered residual signal from at least one reference region comprises: determining a time derivative using a frequency filter; and detecting the feature or another feature of motion artifact in the non-pulsatile region based on the time derivative.

8. The method of claim 2, wherein detecting a feature of motion artifact in the pulsatile region comprises: matching a template to detect a change in pulse waveform morphology that triggers a high frequency artifact flag; and detecting a feature of motion artifact in the pulsatile region based on the template.

9. The method of claim 1, wherein correcting the physiological waveform comprises: detecting a type of motion artifact in the pulsatile region and in the non-pulsatile region; detecting whether an artifact flag is set in response to the detected type of motion; and applying a correction technique to remove the motion artifact from the physiological waveform based at least on the information, the set artifact flag, and the type of motion artifact in the pulsatile region and in the non-pulsatile region.

10. The method of claim 1, wherein generating the physiological waveform based on the image data comprises: transforming the image data into a linear time signal; and generating the physiological waveform based on the linear time signal.

11. A system for managing motion artifact, comprising: a sensor array in patch contact with a surface of a subject's skin, wherein the elastomeric sensor array includes a camera that captures image data of the elastomeric sensor array; and a controller communicatively connected to the camera, the controller configured to: measure an arterial pressure waveform based on the image data, wherein the arterial pressure waveform includes motion artifacts caused by respective deformations or motions of the sensor array over the skin above an artery in a pulsatile region in response to positive pressure applied thereto and the sensor array over the skin adjacent to the artery in a non-pulsatile region, determine information of motion artifacts in the pulsatile region and in the non-pulsatile region, correct the arterial pressure waveform by removing the motion artifacts based on the determined information, and determine a physiological parameter of the subject based on the corrected arterial pressure waveform.

12. The system of claim 11, wherein the sensor array is split into a first sensor array mounted on the skin over the artery and a second sensor array mounted on the skin adjacent to the artery, wherein the first sensor array and the second sensor array are separated from each other by a predefined distance, and wherein an actuator is mounted over the first sensor array representing the pulsatile region and a second actuator is mounted over the first sensor array representing the non-pulsatile region.

13. The system of claim 11, wherein the sensor array is split into a first sensor array mounted on the skin over the artery and a second sensor array mounted on the skin adjacent to the artery, wherein the first sensor array and the second elastomeric sensor array are separated from each other by a predefined distance, wherein an actuator is mounted over the first sensor array representing the pulsatile region and a second actuator is mounted over the first sensor array representing the non-pulsatile region. The system further comprises an actuator having an actuation state in which a controlled amount of pressure is detached from the spatiotemporal signal from the subject's artery, and wherein the actuator is split into a first actuator mounted above a first sensor array representing the pulsatile region and a second actuator mounted above a first sensor array representing the non-pulsatile region.

14. The system of claim 11, wherein determining information of motion artifacts in the pulsatile region and the non-pulsatile region comprises: detecting motion artifacts in the pulsatile region; detecting motion artifacts in the non-pulsatile region; determining spatiotemporal information of motion artifacts in the pulsatile region and in the non-pulsatile region.

15. The system of claim 14, wherein the sensor array has only one camera observing the pulsatile region and the non-pulsatile region such that the pulsatile region and the non-pulsatile region are not sensed separately, the controller digitally separating the pulsatile region and the non-pulsatile region in the image data by: estimating a displacement of the image data at each time point; determining a variance of a signal over time based on the displacement; performing a temporal Fourier transform of the displacement of the image data at all locations based on the variance of the signal over time; determining regions having a transformed displacement satisfying a predefined threshold; and segmenting the determined regions having a transformed displacement satisfying the predefined threshold into the pulsatile region and segmenting regions having a transformed displacement not satisfying the predefined threshold into the non-pulsatile region.

16. The system of claim 14, wherein detecting motion artifacts in the non-pulsatile region comprises: determining a plurality of parameters associated with the pulsatile region; determining a reference region in the image data based on the plurality of parameters; filtering a residual pulsatile signal from the reference region; and detecting motion artifacts in the non-pulsatile region based on the filtered residual signal from the reference region.

17. The system of claim 16, wherein filtering a residual pulsatile signal from the reference region is performed by applying one of a median average filter, a moving median average filter, and a Fourier transform low pass filter.

18. The system of claim 16, wherein the plurality of parameters comprises one or more of: proximity to a pulsatile region to capture the sensor array deformation, distance from the pulsatile region to remove low amplitudes, and length of the pulsatile region.

19. The system of claim 16, wherein detecting motion artifacts in the non-pulsatile region based on the filtered residual signal from the reference region comprises: determining a temporal derivative using a frequency filter; and detecting motion artifacts in the non-pulsatile region based on the temporal derivative.

20. The system of claim 14, wherein detecting motion artifacts in the pulsatile region comprises: matching a template to detect changes in pulse waveform morphology that trigger high frequency artifact flags; and detecting motion artifacts in the pulsatile region based on the template.

21. The system of claim 11, wherein correcting the arterial pressure waveform comprises: detecting a type of motion artifact in the pulsatile region and the non-pulsatile region; detecting whether an artifact flag is set; and applying a correction technique to remove the motion artifact from the arterial pressure waveform based at least on the information, the set artifact flag, and the type of motion artifact in the pulsatile region and the non-pulsatile region.

22. The system of claim 11, wherein generating the arterial pressure waveform based on the image data comprises: transforming the image data into a linear time signal; and generating the arterial pressure waveform based on the linear time signal.

23. The system of claim 11, wherein deformation or motion of a sensor array on skin over an artery in the pulsatile region is caused at least in part by physical forces generated due to pulsation of the artery and transmitted to soft tissue and skin over the artery.

24. The method of claim 1, wherein the physiological waveform is an arterial pressure waveform, and wherein the information is spatiotemporal information.

25. The method of claim 1, wherein the physiological waveform is or represents heart rate, blood pressure or arterial pressure, respiratory rate, cardiac output, or a hemodynamic parameter.

26. The system of claim 11, wherein the physiological waveform is an arterial pressure waveform, and wherein the information is spatiotemporal information.

27. The method of claim 1, wherein the sensor array is an elastomeric sensor array mounted on an inner surface of an actuator, wherein the actuator is over the elastomeric sensor array.

28. The system of claim 11, wherein the sensor array is an elastomeric sensor array, the system further comprising an actuator having an actuated state in which a controlled amount of pressure is decoupled from a spatiotemporal signal in an artery of the subject.

29. The system of claim 11, wherein the sensor array comprises a piezoresistive material.

Images