Methods, systems, terminal devices, and media for assessing upper limb arterial occlusion using portable multi-camera devices.

Abstract

This invention discloses a method, system, terminal device, and medium for assessing upper limb arterial occlusion based on a portable multi-camera device, relating to the field of arterial occlusion assessment technology. The method includes: acquiring a reference signal, a left-hand pulse wave signal, and a right-hand pulse wave signal using a portable multi-camera device, with the reference signal, left-hand pulse wave signal, and right-hand pulse wave signal synchronized at the acquisition time; extracting features from the reference signal, left-hand pulse wave signal, and right-hand pulse wave signal to obtain signal transmission time difference features and signal waveform similarity features; inputting the signal transmission time difference features and signal waveform similarity features into a pre-trained assessment model to obtain the upper limb arterial occlusion assessment result. This invention uses a portable multi-camera device to simultaneously acquire physiological signals from three locations on the human body, utilizing signal transmission time difference features and signal waveform similarity features to easily, conveniently, and accurately assess the occlusion status of upper limb arteries.

Description

Technical Field

[0001] This invention relates to the field of arterial occlusion assessment technology, and in particular to an upper limb arterial occlusion assessment method, system, terminal device and medium based on a portable multi-camera device. Background Technology

[0002] Upper limb arterial occlusion, caused by factors such as atherosclerosis and thrombosis, leads to narrowing or blockage of the lumen, resulting in upper limb pain, numbness, and ischemic complications, severely impacting patients' quality of life. Clinical assessment relies on imaging techniques such as ultrasound, CT (Computed Tomography) angiography, and MRI (Magnetic Resonance Imaging). While these provide anatomical and blood flow information, they have significant limitations. First, existing assessment methods are highly dependent on equipment, requiring specialized medical equipment and facilities, making them difficult to implement in homes or primary healthcare settings and unable to meet routine screening needs. Second, existing methods are invasive or costly. CT and MRI involve radiation exposure or high examination fees, while ultrasound, although non-invasive, relies on operator experience and has poor repeatability. Third, existing assessment methods lack portability; traditional equipment is bulky and cannot be used for mobile testing, especially for elderly or mobility-impaired patients, posing physical obstacles to regular monitoring. Furthermore, existing methods often intervene when symptoms are already evident, lacking convenient early warning mechanisms, causing some patients to miss the optimal intervention window.

[0003] Screening for unilateral upper limb arterial occlusion presents greater challenges. Studies have shown that prolonged pulse wave conduction time and waveform distortion in the distal fingertips of the affected side can serve as early warning signals of occlusion. However, traditional pulse wave signal monitoring requires specialized contact equipment and can only collect signals from a single location, failing to enable bilateral comparison and dynamic assessment. With the development of mobile health technology, the widespread availability of portable devices has provided a new pathway for non-invasive physiological signal acquisition. However, existing health monitoring functions based on mobile terminals such as smartphones are mostly limited to single-camera heart rate detection and have not yet addressed the simultaneous acquisition of multi-channel pulse wave signals and arterial occlusion assessment.

[0004] Therefore, there is an urgent need for a portable, non-invasive, and low-cost method for assessing upper limb arterial occlusion, to overcome the limitations of traditional medical devices and achieve early screening and long-term monitoring in a home environment. Currently, how to combine the multi-camera system of portable devices with physiological signal analysis to achieve reliable arterial occlusion assessment remains a pressing technical challenge. Summary of the Invention

[0005] The technical problem this invention aims to solve is that in the field of upper limb arterial occlusion assessment, existing technologies mainly rely on imaging methods such as ultrasound, CT, and MRI. These methods suffer from drawbacks such as bulky equipment, invasiveness, high testing costs, and insufficient portability, making it difficult to meet the needs of early screening and dynamic monitoring in home or primary healthcare settings. For unilateral upper limb arterial occlusion, traditional single-site pulse wave signal acquisition methods cannot achieve bilateral comparative analysis, resulting in a lag in the assessment of pulse wave transit time (PTT) and waveform characteristics, affecting early risk identification. Although mobile health technologies have developed, existing portable devices can only complete single-site heart rate monitoring with a single camera, lacking the ability to simultaneously acquire multi-channel pulse wave signals and assess arterial occlusion. Therefore, how to utilize widely available portable devices to achieve non-invasive, low-cost, and convenient multi-site signal acquisition and analysis has become a current technical bottleneck, urgently requiring an effective solution.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention provides a method for assessing upper limb arterial occlusion based on a portable multi-camera device, the method comprising:

[0008] The reference signal, the left hand pulse wave signal, and the right hand pulse wave signal are collected using a portable multi-camera device. The reference signal, the left hand pulse wave signal, and the right hand pulse wave signal are synchronized with the collection time as the reference time.

[0009] Feature extraction is performed on the reference signal, the left hand pulse wave signal, and the right hand pulse wave signal to obtain signal propagation time difference features and signal waveform similarity features;

[0010] The signal transmission time difference feature and the signal waveform similarity feature are input into the pre-trained evaluation model to obtain the upper limb arterial occlusion evaluation result.

[0011] In one implementation, the reference signal is a facial pulse wave signal, and the acquisition of the reference signal, left hand pulse wave signal, and right hand pulse wave signal based on a portable multi-camera device includes:

[0012] The first camera of the portable multi-camera device is used to capture a face to obtain a facial video;

[0013] Face detection is performed on the facial video to obtain a mask of the facial skin area;

[0014] Based on the mask of the facial skin region, all pixels of the facial skin region in the facial video are averaged to obtain a sequence of pixel values ​​for the face.

[0015] The pixel value sequence of the face is preprocessed to obtain the facial pulse wave signal.

[0016] In one implementation, the reference signal is a cardiac vibration signal or a heart sound signal, and the acquisition of the reference signal, left-hand pulse wave signal, and right-hand pulse wave signal based on a portable multi-camera device includes:

[0017] Using the accelerometer of the portable multi-camera device, cardiac vibrations transmitted through the chest wall are collected and preprocessed to obtain cardiac vibration signals;

[0018] Alternatively, the microphone of the portable multi-camera device can be used to collect sounds near the heart in the chest, and preprocessed to obtain heart sound signals.

[0019] In one implementation, the acquisition of reference signals, left-hand pulse wave signals, and right-hand pulse wave signals based on a portable multi-camera device includes:

[0020] Using the second and third cameras of the portable multi-camera device, the same fingertip of either the left or right hand is captured, with the fingertip in contact with the corresponding camera and completely covering the lens during the capture, to obtain a video of the left fingertip and a video of the right fingertip.

[0021] The left fingertip video and the right fingertip video are averaged to obtain the pixel value sequence of the left fingertip and the pixel value sequence of the right fingertip.

[0022] The pixel value sequences of the left and right fingertips are preprocessed to obtain the left and right hand pulse wave signals.

[0023] In one implementation, the evaluation features include pulse wave propagation time difference and pulse wave waveform similarity features. The step of extracting features from the reference signal, the left-hand pulse wave signal, and the right-hand pulse wave signal to obtain signal propagation time difference features and signal waveform similarity features includes:

[0024] The positioning algorithm was used to locate all the reference signal feature points, left hand pulse wave signal feature points and right hand pulse wave signal feature points. Each feature point corresponds one-to-one with the heartbeat cycle.

[0025] Based on the reference signal feature points, the left hand pulse wave signal feature points, and the right hand pulse wave signal feature points, the conduction time difference of the left hand pulse wave and the conduction time difference of the right hand pulse wave are calculated.

[0026] Based on the reference signal, the left hand pulse wave signal, and the right hand pulse wave signal, the waveform similarity features of the left hand pulse wave and the waveform similarity features of the right hand pulse wave are calculated.

[0027] In one implementation, calculating the left-hand pulse wave conduction time difference and the right-hand pulse wave conduction time difference based on the reference signal feature points, the left-hand pulse wave signal feature points, and the right-hand pulse wave signal feature points includes:

[0028] For each heartbeat cycle, the timestamp of the left hand pulse wave signal feature point is subtracted from the timestamp of the reference signal feature point to obtain the left hand pulse wave conduction time difference.

[0029] For each heartbeat cycle, the timestamp of the right pulse wave signal feature point is subtracted from the timestamp of the reference signal feature point to obtain the right pulse wave conduction time difference.

[0030] In one implementation, the step of calculating the left-hand pulse wave waveform similarity features and the right-hand pulse wave waveform similarity features based on the reference signal, the left-hand pulse wave signal, and the right-hand pulse wave signal includes:

[0031] For each heartbeat cycle, the similarity between the left-hand pulse wave signal and the reference signal is calculated to obtain the left-hand pulse wave waveform similarity features;

[0032] For each heartbeat cycle, the similarity between the right-hand pulse wave signal and the reference signal is calculated to obtain the right-hand pulse wave waveform similarity features.

[0033] Secondly, embodiments of the present invention also provide an upper limb arterial occlusion assessment system based on a portable multi-camera device, the system comprising:

[0034] The signal acquisition module is used to acquire a reference signal, a left hand pulse wave signal, and a right hand pulse wave signal based on a portable multi-camera device. The reference signal, the left hand pulse wave signal, and the right hand pulse wave signal are synchronized with the acquisition time as the reference time.

[0035] The feature acquisition module is used to extract features from the reference signal, the left hand pulse wave signal, and the right hand pulse wave signal to obtain signal transmission time difference features and signal waveform similarity features;

[0036] The upper limb arterial occlusion assessment module is used to input the signal transmission time difference feature and the signal waveform similarity feature into a pre-trained assessment model to obtain the upper limb arterial occlusion assessment result.

[0037] Thirdly, embodiments of the present invention also provide a terminal device, the terminal device including a memory, a processor, and an upper limb arterial occlusion assessment program based on a portable multi-camera device stored in the memory and executable on the processor. When the processor executes the upper limb arterial occlusion assessment program based on a portable multi-camera device, it implements the steps of the upper limb arterial occlusion assessment method based on a portable multi-camera device as described in any of the above schemes.

[0038] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing an upper limb arterial occlusion assessment program based on a portable multi-camera device. When the upper limb arterial occlusion assessment program based on a portable multi-camera device is executed by a processor, it implements the steps of the upper limb arterial occlusion assessment method based on a portable multi-camera device as described in any of the above schemes.

[0039] Beneficial Effects: This invention discloses a method, system, terminal device, and medium for assessing upper limb arterial occlusion based on a portable multi-camera device. The method involves acquiring a reference signal, a left-hand pulse wave signal, and a right-hand pulse wave signal using the portable multi-camera device. These signals are synchronized with the acquisition time as the reference time. Feature extraction is performed on the reference signal, left-hand pulse wave signal, and right-hand pulse wave signal to obtain signal transmission time difference features and signal waveform similarity features. These features are then input into a pre-trained assessment model to obtain the assessment result of upper limb arterial occlusion. This invention addresses the characteristic that upper limb arterial occlusion often manifests in localized areas of the left and right upper limbs, reflected in changes in pulse wave transmission time and waveform. It selects a reference signal that is almost unaffected by occlusion in the left and right upper limbs, and synchronously acquires the human body reference signal, left-hand pulse wave signal, and right-hand pulse wave signal using a portable multi-camera device. A simple assessment algorithm is designed by combining the signal transmission time difference features and waveform similarity features. This invention is based on the collaborative analysis of three location signals, which overcomes the interference of local occlusion on a single signal and achieves accurate assessment of upper limb arterial occlusion with a simple and convenient algorithm. Attached Figure Description

[0040] Figure 1 A flowchart illustrating a specific implementation method for an upper limb arterial occlusion assessment method based on a portable multi-camera device, as provided in this embodiment of the invention.

[0041] Figure 2 This is a schematic diagram of the upper limb arterial occlusion assessment process based on a portable multi-camera device provided in an embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram of the upper limb arterial occlusion assessment device based on a portable multi-camera device provided in an embodiment of the present invention.

[0043] Figure 4 This is a block diagram illustrating the internal structure of the terminal device provided in an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0045] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content, operations, or steps, nor does it require execution in the described order. For example, some operations or steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0046] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0047] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. For example, "first control information" and "second control information" are only used to distinguish different control information and do not limit their order.

[0048] Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or the order of execution, and that the words "first" and "second" do not necessarily imply that they are different.

[0049] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0050] In the field of upper limb arterial occlusion assessment, existing technologies mainly rely on imaging methods such as ultrasound, CT, and MRI. These methods suffer from drawbacks such as bulky equipment, invasiveness, high testing costs, and insufficient portability, making it difficult to meet the needs of early screening and dynamic monitoring in home or primary healthcare settings. For unilateral upper limb arterial occlusion, traditional single-site pulse wave signal acquisition methods cannot achieve bilateral comparative analysis, leading to a lag in the assessment of pulse wave propagation time difference and waveform characteristics, thus affecting early risk identification. Although mobile health technologies have developed, existing portable devices can only perform single-site heart rate monitoring using a single camera, lacking the ability to simultaneously acquire multi-channel pulse wave signals and assess arterial occlusion. Therefore, how to utilize widely available portable devices to achieve non-invasive, low-cost, and convenient multi-site signal acquisition and analysis has become a current technological bottleneck, urgently requiring an effective solution.

[0051] In the early stages of stenosis or occlusion of the upper limb arteries, the PPG signal (Photoplethysmography Signal, used here to represent pulse wave signal) exhibits characteristic changes, including significant amplitude attenuation, waveform distortion, and phase delay. These changes can be used to assess the degree of upper limb arterial obstruction by analyzing PPG signals from multiple finger locations. Based on this, Figure 2 A schematic diagram illustrating the overall workflow of an upper limb arterial occlusion assessment method based on a portable multi-camera device is presented. Figure 2 This demonstration showcases an algorithmic process for data acquisition, computation, and evaluation using a smartphone with one front-facing camera and two rear-facing cameras as a multi-camera system. Besides smartphones, other portable devices with multiple cameras, such as smart tablets, smart bracelets, smartwatches, action cameras, robotic multi-camera systems, and VR glasses, can also be used. The system acquires simultaneous PPG signals from different peripheral blood vessels in the human body using a mobile device multi-camera system, including facial PPG and fingertip PPG. Facial PPG signals are used as a baseline signal to calculate the PTT (post-traumatic stress test) of both hands, and the ability to identify arterial occlusion is enhanced by analyzing PPG waveform characteristics. Furthermore, based on the PTT and waveform characteristics of both hands, machine learning or mathematical models can be used to assess upper limb arterial occlusion. Figure 2 The demonstration showed that in addition to using smartphones to collect facial PPG signals as a reference signal, portable multi-camera devices can also use accelerometers to collect heart vibration signals or microphone sensors to collect heart sound signals. The pulse wave signals from both hands were compared with the reference signals to determine their respective pulse wave conduction time, waveform characteristics, etc., for assessing upper limb obstruction.

[0052] This embodiment provides a method for assessing upper limb arterial occlusion based on a portable multi-camera device, such as... Figure 1 As shown, the specific steps include the following:

[0053] Step S100: Acquire a reference signal, a left hand pulse wave signal, and a right hand pulse wave signal based on a portable multi-camera device. The reference signal, the left hand pulse wave signal, and the right hand pulse wave signal are synchronized with the acquisition time as the reference time.

[0054] In this embodiment, preferably, the portable multi-camera device is a mobile device containing three cameras. The first camera is used to acquire facial pulse wave signals and needs to have a high image acquisition frame rate and low light sensitivity to ensure that it can capture subtle light intensity fluctuations caused by changes in blood flow in facial skin under natural light or indoor environments, providing the raw data basis for subsequent signal processing. During signal acquisition, the camera must be kept stable to avoid data distortion caused by motion blur. It is also important to ensure that the arms are not compressed by overly tight clothing to prevent external force from affecting blood flow in the arms; loose, comfortable clothing is recommended to facilitate normal blood circulation. The second and third cameras are used to acquire pulse wave signals from the left and right hands and need to have macro shooting capabilities to clearly capture the microvascular pulsation of the fingertips. During shooting, the fingertips must be in physical contact with the lens and completely block the lens aperture to form a closed light path, avoiding ambient light interference and ensuring that the acquired light signal comes only from changes in light absorption of the fingertip tissue. Furthermore, to lay the foundation for feature extraction from the three subsequent pulse wave signals, it is necessary to ensure that the three pulse wave signals are synchronized with the acquisition time as the reference time, that is, the timestamps of the three pulse wave signals are synchronized in parallel.

[0055] In one implementation, the reference signal is a facial pulse wave signal, and the acquisition of the reference signal, left hand pulse wave signal, and right hand pulse wave signal based on a portable multi-camera device specifically includes the following steps:

[0056] Step S110: Use the first camera of the portable multi-camera device to capture a face to obtain a face video;

[0057] Step S120: Perform face detection on the face video to obtain a mask for the facial skin area;

[0058] Step S130: Based on the mask of the facial skin region, average all pixels of the facial skin region in the facial video to obtain the pixel value sequence of the face;

[0059] Step S140: Perform signal preprocessing on the pixel value sequence of the face to obtain the facial pulse wave signal.

[0060] In this embodiment, a smartphone is preferably selected as the multi-camera device. Its first camera, preferably a front-facing camera, must have optical parameters that meet near-infrared light sensing requirements to enhance the light absorption sensitivity of hemoglobin and improve the signal-to-noise ratio of the face's PPG signal. For example, a smartphone with high dynamic range shooting capabilities can be used to ensure stable acquisition of subtle color changes in facial skin even under complex lighting conditions such as backlighting and indoor lighting.

[0061] After capturing facial video, a mask for the facial skin region is obtained using face detection methods. Specifically, face detection can employ open-source libraries or deep learning models, such as convolutional neural networks, to automatically identify facial regions. The generated mask must accurately cover major skin areas such as the forehead and cheeks, excluding interfering areas such as eyebrows and nose shadows. Then, after applying the mask to each frame of the facial video, a smooth sequence of facial pixel values ​​is obtained by calculating the average grayscale value or the average RGB (Red, Green, Blue) three-channel value of the pixels within the mask. When averaging the skin region pixels, dynamic interference frames such as blinking and facial expression changes must be simultaneously removed to ensure that the pixel value sequence accurately reflects arterial blood volume fluctuations. Finally, in the signal preprocessing step, filters can be used to remove baseline drift or to extract the pulse wave frequency band. The bandpass filter parameters can be adaptively adjusted according to the heart rate range to retain effective pulse wave frequency bands. Ultimately, through the above processing, the facial pulse wave signal is obtained.

[0062] In one implementation, the reference signal is a cardiac vibration signal or a heart sound signal, and the acquisition of the reference signal, left-hand pulse wave signal, and right-hand pulse wave signal based on a portable multi-camera device specifically includes the following steps:

[0063] S150. Using the accelerometer of the portable multi-camera device, collect the cardiac vibration transmitted through the chest wall and perform preprocessing to obtain the cardiac vibration signal;

[0064] S160, or, using the microphone of the portable multi-camera device, to collect sounds near the heart in the chest and perform preprocessing to obtain heart sound signals.

[0065] In this embodiment, cardiac vibration signal or heart sound signal can be used as the reference signal. When acquiring the cardiac vibration signal, the accelerometer of the portable device is first placed firmly against the left chest wall of the body using an elastic strap or by hand, ensuring close contact between the sensor and the chest wall surface to reduce motion displacement interference. The mechanical vibration signal caused by cardiac contraction and expansion is acquired through the accelerometer of the portable device. Next, a bandpass filter is used to remove respiratory motion and high-frequency electromagnetic interference, and polynomial fitting or wavelet transform is used to eliminate low-frequency drift. Then, a peak detection algorithm is used to locate the starting point of cardiac contraction, such as the point of maximum amplitude, generating a time series of cardiac vibration signal feature points, completing feature point extraction, and finally constructing the cardiac vibration signal.

[0066] During heart sound signal acquisition, the microphone of the device is first placed close to the left chest to collect sound, ensuring an unobstructed acoustic path and controlling ambient noise. After acquisition, a bandpass filter is applied to extract the characteristic frequency bands of the heart sounds, suppressing low-frequency ambient noise and high-frequency circuit noise. The time-domain signal is converted into a spectrum using short-time Fourier transform or wavelet transform to identify the peak positions of the first and second heart sounds. Using the peak value of the first heart sound as a reference point, a complete heartbeat cycle is captured, eliminating signal offsets caused by respiration and movement.

[0067] Finally, a unified device clock timestamp needs to be added to the cardiac vibration signal and heart sound signal to ensure that the reference signal is synchronized with the time of the left and right hand pulse wave signals.

[0068] In one implementation, the acquisition of reference signals, left-hand pulse wave signals, and right-hand pulse wave signals based on a portable multi-camera device specifically includes the following steps:

[0069] Step S170: Using the second and third cameras of the portable multi-camera device, take pictures of any identical fingertip of the left and right hands respectively. During the shooting, the fingertip keeps in contact with the corresponding camera and completely covers the lens to obtain the left fingertip video and the right fingertip video.

[0070] Step S180: Average all pixels of the left fingertip video and the right fingertip video to obtain the pixel value sequence of the left fingertip and the pixel value sequence of the right fingertip;

[0071] Step S190: Perform signal preprocessing on the pixel value sequence of the left fingertip and the pixel value sequence of the right fingertip to obtain the left hand pulse wave signal and the right hand pulse wave signal.

[0072] In this embodiment, the second and third cameras are preferably the two rear cameras of a smartphone. The same fingertip being photographed is preferably the index or middle finger of both hands, as the fingertip has a dense distribution of capillaries, resulting in high signal stability. Simultaneously, both arms should be kept at the same horizontal height during shooting, which can be achieved using a table for support to avoid errors in pulse wave propagation time measurement due to gravity. Furthermore, in the pixel averaging process, for the pure fingertip area without background interference, the grayscale or RGB mean value of all pixels in the video frame needs to be directly calculated to generate a fingertip pixel value sequence. Subsequent signal preprocessing can also employ detrending, filtering, and smoothing processes, synchronized with the facial signal preprocessing flow, to ensure that the time axis alignment accuracy of the three signals meets the PTT calculation requirements.

[0073] Step S200: Extract features from the reference signal, the left hand pulse wave signal, and the right hand pulse wave signal to obtain signal transmission time difference features and signal waveform similarity features.

[0074] In this embodiment, the arteries of the human upper limbs may become narrowed or blocked due to ruptured atherosclerotic plaques, thromboembolism, or vascular inflammatory damage. Prolonged narrowing or blockage of the upper limb arteries will cause pain, numbness, and weakness in the upper limbs, especially during physical activity. Furthermore, the affected limbs may exhibit paleness, coldness, or decreased temperature, and in severe cases, even necrosis of the fingertips. The heart constantly pumps blood throughout the body; arterial blood flows from the heart through the upper limb arteries to the peripheral blood vessels, such as the fingers. The subtle color changes in the skin detected by PPG signals are essentially fluctuations in light absorption caused by changes in hemoglobin concentration and oxygenation status. In the early stages of narrowing or blockage in the upper limbs, the PPG signal exhibits characteristic changes, including significant amplitude attenuation, waveform distortion, and phase delay. The significant amplitude attenuation is due to reduced distal blood perfusion, leading to weakened fluctuations in arterial blood volume. Waveform distortion is due to a flattened and prolonged ascending limb, reflecting increased blood resistance; a wider, rounded, and less sharp systolic peak; and the disappearance or blurring of the dicrotic wave, indicating decreased vascular elasticity and the influence of retrograde blood flow. Phase delay stems from obstructed pulse wave transmission. Therefore, analyzing PPG signals from multiple finger locations can help assess the approximate extent of upper limb arterial occlusion.

[0075] In one implementation, the evaluation features include pulse wave conduction time difference and pulse wave waveform similarity features. The step of extracting features from the reference signal, the left-hand pulse wave signal, and the right-hand pulse wave signal to obtain signal conduction time difference features and signal waveform similarity features specifically includes the following steps:

[0076] Step S210: Using a positioning algorithm, locate all the reference signal feature points, left hand pulse wave signal feature points, and right hand pulse wave signal feature points, where each feature point corresponds one-to-one in the heartbeat cycle.

[0077] Step S220: Based on the reference signal feature points, the left hand pulse wave signal feature points, and the right hand pulse wave signal feature points, calculate the left hand pulse wave conduction time difference and the right hand pulse wave conduction time difference;

[0078] Step S230: Based on the reference signal, the left hand pulse wave signal and the right hand pulse wave signal, calculate the similarity features of the left hand pulse wave waveform and the similarity features of the right hand pulse wave waveform.

[0079] In this embodiment, an internal unified clock or external synchronization signal is used to ensure that the time deviation of the three signal acquisitions is less than a preset threshold, ensuring that the pulse wave signals obtained from the three locations of the human body are time-synchronized. Based on this, since the facial skin is located in the central position of the human anatomy, the PPG signal acquired from the face can be used as the reference signal when calculating the pulse wave conduction time difference for each hand. Alternatively, a cardiac vibration signal or heart sound signal can be used as the reference signal to calculate the pulse wave conduction time difference for each hand. Before calculating the PTT for both hands, methods such as instantaneous signal-to-noise ratio are needed to eliminate signal segments severely interfered with by hand noise. For each heartbeat cycle, peak localization and other methods are used to determine the characteristic points of the three synchronized PPG signals, including typical PPG waveform characteristic points such as the contraction peak and the maximum slope point. The contraction peak represents the highest point of the pulse wave waveform, reflecting the peak blood flow during cardiac systole. The maximum slope point can be the maximum slope point of the rising limb, which represents the point where the slope of the rising edge of the waveform is at its maximum, characterizing the left ventricular ejection rate. Furthermore, the dicrotic notch can be used as a feature point to represent the waveform inflection point formed by arterial elastic recoil, reflecting vascular compliance. The peak of the systolic peak is located using a peak detection algorithm, and the dicrotic notch is identified using the second derivative zero-crossing method, ensuring that the feature point types of the three signals are consistent within each heartbeat cycle and that the time alignment error is less than a preset threshold. In other words, by setting a threshold for the systolic peak amplitude or a threshold for the rising edge slope, the feature points of each heartbeat cycle can be located, ensuring that the feature points of the face, left hand, and right hand signals are strictly aligned on the time axis for subsequent PTT calculations.

[0080] In one implementation, the step of calculating the conduction time difference of the left-hand pulse wave and the conduction time difference of the right-hand pulse wave based on the reference signal feature points, the left-hand pulse wave signal feature points, and the right-hand pulse wave signal feature points specifically includes the following steps:

[0081] Step S221: For each heartbeat cycle, subtract the timestamp of the reference signal feature point from the timestamp of the left hand pulse wave signal feature point to obtain the left hand pulse wave conduction time difference.

[0082] Step S222: For each heartbeat cycle, subtract the timestamp of the reference signal feature point from the timestamp of the right pulse wave signal feature point to obtain the right pulse wave conduction time difference.

[0083] In this embodiment, preferably, the peak of each heartbeat cycle of the pulse wave signal is used as a feature point. Figure 2 Taking the facial PPG signal as a reference signal in the illustrated process, after determining the feature points of each heartbeat cycle, the timestamps of the facial PPG signal feature points are subtracted from the timestamps of the left and right hand PPG signal feature points to obtain the PTT of both upper limbs. Figure 2 As shown, PPG-1 is the left hand pulse wave signal, PPG-2 is the facial pulse wave signal, and PPG-3 is the right hand pulse wave signal. The left hand pulse wave conduction time difference is obtained by subtracting the peak characteristic point timestamp of PPG-2 from the peak characteristic point timestamp of PPG-1 within the same heartbeat cycle. Similarly, the right hand pulse wave conduction time difference is obtained by subtracting the peak characteristic point timestamp of PPG-3 from the peak characteristic point timestamp of PPG-2 within the same heartbeat cycle. Likewise, if a cardiac vibration signal or heart sound signal is used as the reference signal to replace the facial PPG, bilateral upper limb PTT calculation can also be achieved. Specifically, the PTT difference is calculated by locating cardiac characteristic points, such as the systolic initiation point, and comparing them with the systolic peak characteristic point timestamps of the PPG signals from both hands.

[0084] In one implementation, the step of calculating the left-hand pulse wave waveform similarity features and the right-hand pulse wave waveform similarity features based on the reference signal, the left-hand pulse wave signal, and the right-hand pulse wave signal specifically includes the following steps:

[0085] Step S231: For each heartbeat cycle, calculate the similarity between the left hand pulse wave signal and the reference signal to obtain the left hand pulse wave waveform similarity features;

[0086] Step S232: For each heartbeat cycle, calculate the similarity between the right hand pulse wave signal and the reference signal to obtain the right hand pulse wave waveform similarity features.

[0087] In this embodiment, with Figure 2 Taking the facial PPG signal as a reference signal in the illustrated process, the cosine similarity algorithm is used to calculate waveform similarity. For example... Figure 2As shown, after normalizing the waveform curves of the left hand pulse wave signal PPG-1 and the face signal PPG-2 within each heartbeat cycle, the cosine value of their vectors is calculated. The closer the value is to 1, the more similar the waveform morphology. Similarly, the same calculation process is performed on the right hand pulse wave signal PPG-3 and the face signal to obtain the waveform similarity features of the right hand. This method can quantitatively assess the morphological differences between the bilateral upper limb pulse waves and the central reference signal, effectively identifying waveform distortions caused by arterial obstruction, such as amplitude attenuation and smoothing of the ascending limb, providing multi-dimensional feature support for judging the degree of obstruction. In addition to using the cosine similarity algorithm to calculate waveform similarity, Euclidean distance or correlation coefficient methods can also be used to calculate the similarity features of their respective PPG waveforms. These features can further enhance the model's ability to identify arterial obstruction. Similarly, after filtering and feature point alignment preprocessing of the cardiac vibration signal or heart sound signal, its waveform can also be morphologically compared with the PPGs of both hands. For example, the peak value of the first heart sound of the cardiac vibration signal is used as a reference point and time-aligned with the contraction peak of the PPG in both hands, and then the similarity index of the waveform curves is calculated.

[0088] Step S300: Input the signal transmission time difference feature and the signal waveform similarity feature into the pre-trained evaluation model to obtain the upper limb arterial occlusion evaluation result.

[0089] In this embodiment, the pre-trained evaluation model evaluates pulse wave signals based on evaluation features. These evaluation features include the aforementioned bilateral PTT (post-traumatic stress test) features and waveform similarity features. For a set of synchronized pulse wave signals from three locations on the human body, the evaluation features are calculated and input into the pre-trained evaluation model. The pre-trained evaluation model then outputs the upper limb arterial occlusion evaluation result corresponding to this set of data.

[0090] In one implementation, the method further includes:

[0091] Step S400: Using the evaluation features of several normal pulse wave signal samples and the evaluation features of abnormal pulse wave signal samples as training samples, the evaluation model is trained to obtain a pre-trained evaluation model.

[0092] In this embodiment, before training the evaluation model, a large amount of sample data needs to be collected, including normal and abnormal pulse wave signals and their corresponding PTT values ​​and waveform similarity features, to ensure the diversity and representativeness of the dataset and improve the model's generalization ability. In addition to PTT and waveform similarity features, other physiological features, such as heart rate variability, pulse wave amplitude, and waveform symmetry, are considered as model inputs. These features can provide more comprehensive hemodynamic information and enhance the model's predictive ability. The evaluation model can be tested using various machine learning algorithms, such as support vector machines, random forests, and neural networks. The optimal model is selected through cross-validation and hyperparameter tuning to ensure the best classification performance. The model is trained using the training set and its performance is evaluated on the validation set. The model's accuracy, recall, and F1 score are monitored to ensure its effectiveness and reliability in arterial occlusion detection. The F1 score is the harmonic mean of accuracy and recall. Finally, a pre-trained evaluation model is obtained through training and used to evaluate samples.

[0093] In summary, under the technical solution of the above embodiments, considering that upper limb arterial occlusion symptoms are mostly manifested in local areas of the left and right upper limbs and reflected in pulse wave propagation time and waveform changes, a reference signal that is almost unaffected by occlusion in the left and right upper limbs is selected. A portable multi-camera device is used to simultaneously acquire the human body reference signal, left hand pulse wave signal, and right hand pulse wave signal. A simple evaluation algorithm is designed by combining signal propagation time difference characteristics and waveform similarity characteristics. Based on the collaborative analysis of the three location signals, the interference of local occlusion on a single signal is overcome, and a simple and convenient algorithm is used to achieve accurate assessment of upper limb arterial occlusion.

[0094] like Figure 3 As shown in the figure, this embodiment of the invention provides an upper limb arterial occlusion assessment system based on a portable multi-camera device. The system includes: a signal acquisition module 10, a feature acquisition module 20, and an upper limb arterial occlusion assessment module 30.

[0095] Specifically, the pulse wave signal acquisition module 10 is used to acquire a reference signal, a left-hand pulse wave signal, and a right-hand pulse wave signal based on a portable multi-camera device. The reference signal, the left-hand pulse wave signal, and the right-hand pulse wave signal are synchronized with the acquisition time as the reference time. The evaluation feature acquisition module 20 is used to extract features from the reference signal, the left-hand pulse wave signal, and the right-hand pulse wave signal to obtain signal transmission time difference features and signal waveform similarity features. The upper limb arterial occlusion assessment module 30 is used to input the signal transmission time difference features and the signal waveform similarity features into a pre-trained assessment model to obtain the upper limb arterial occlusion assessment result.

[0096] In one implementation, the reference signal is a facial pulse wave signal, and the signal acquisition module includes:

[0097] A facial video capture unit is used to capture a face using the first camera of the portable multi-camera device to obtain a facial video.

[0098] A facial skin mask acquisition unit is used to perform face detection on the facial video to obtain a mask of the facial skin area;

[0099] The face pixel value sequence acquisition unit is used to perform average processing on all pixels of the face skin region in the face video based on the mask of the face skin region to obtain the face pixel value sequence;

[0100] A facial pixel value sequence preprocessing unit is used to perform signal preprocessing on the pixel value sequence of the face to obtain a facial pulse wave signal.

[0101] In one implementation, the reference signal is a cardiac vibration signal or a heart sound signal, and the signal acquisition module includes:

[0102] The cardiac vibration signal acquisition unit is used to acquire cardiac vibrations transmitted through the chest wall using the accelerometer of the portable multi-camera device, and to preprocess the data to obtain cardiac vibration signals.

[0103] The heart sound signal acquisition unit is used to collect sounds near the heart in the chest using the microphone of the portable multi-camera device, and to preprocess them to obtain heart sound signals.

[0104] In one implementation, the signal acquisition module includes:

[0105] The dual-finger tip video acquisition unit is used to capture images of any identical fingertip of the left and right hands using the second and third cameras of the portable multi-camera device, respectively. During the capture, the fingertip keeps in contact with the corresponding camera and completely covers the lens, thus obtaining a video of the left fingertip and a video of the right fingertip.

[0106] The dual-finger tip pixel value sequence acquisition unit is used to perform average processing on all pixels of the left finger tip video and the right finger tip video to obtain the pixel value sequence of the left finger tip and the pixel value sequence of the right finger tip.

[0107] The dual-finger tip pixel value sequence preprocessing unit is used to perform signal preprocessing on the pixel value sequence of the left fingertip and the pixel value sequence of the right fingertip to obtain the left hand pulse wave signal and the right hand pulse wave signal.

[0108] In one implementation, the feature acquisition module includes:

[0109] The feature point acquisition unit is used to locate all the reference signal feature points, left hand pulse wave signal feature points and right hand pulse wave signal feature points through a positioning algorithm. Each feature point corresponds one-to-one in the heartbeat cycle.

[0110] The signal propagation time difference acquisition unit is used to calculate the left hand pulse wave propagation time difference and the right hand pulse wave propagation time difference based on the reference signal feature points, the left hand pulse wave signal feature points, and the right hand pulse wave signal feature points;

[0111] The signal waveform similarity feature acquisition unit is used to calculate the left hand pulse wave waveform similarity feature and the right hand pulse wave waveform similarity feature based on the reference signal, the left hand pulse wave signal and the right hand pulse wave signal.

[0112] In one implementation, the signal propagation time difference acquisition unit includes:

[0113] The left-hand pulse wave conduction time difference acquisition subunit is used to subtract the timestamp of the reference signal feature point from the timestamp of the left-hand pulse wave signal feature point for each heartbeat cycle to obtain the left-hand pulse wave conduction time difference.

[0114] The right-hand pulse wave conduction time difference acquisition subunit is used to subtract the timestamp of the reference signal feature point from the timestamp of the right-hand pulse wave signal feature point for each heartbeat cycle to obtain the right-hand pulse wave conduction time difference.

[0115] In one implementation, the signal waveform similarity feature acquisition unit includes:

[0116] The left-hand pulse wave waveform similarity feature acquisition subunit is used to calculate the similarity between the left-hand pulse wave signal and the reference signal for each heartbeat cycle, and obtain the left-hand pulse wave waveform similarity feature.

[0117] The right-hand pulse wave waveform similarity feature acquisition subunit is used to calculate the similarity between the right-hand pulse wave signal and the reference signal for each heartbeat cycle, and obtain the right-hand pulse wave waveform similarity feature.

[0118] In one implementation, the system further includes:

[0119] The evaluation model training module is used to train the evaluation model using evaluation features of several normal pulse wave signal samples and evaluation features of abnormal pulse wave signal samples as training samples, so as to obtain a pre-trained evaluation model.

[0120] Based on the above embodiments, the present invention also provides a terminal device, the principle block diagram of which can be as follows: Figure 4As shown, the terminal device includes a processor, memory, network interface, display screen, and temperature sensor connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a method for assessing upper limb arterial occlusion based on a portable multi-camera device. The display screen can be an LCD screen or an e-ink screen. The temperature sensor is pre-installed inside the terminal device to detect the operating temperature of the internal components.

[0121] Those skilled in the art will understand that Figure 4 The schematic diagram shown is only a partial structural diagram related to the present invention and does not constitute a limitation on the terminal device to which the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0122] In one embodiment, a terminal device is provided, including a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations:

[0123] The reference signal, the left hand pulse wave signal, and the right hand pulse wave signal are collected using a portable multi-camera device. The reference signal, the left hand pulse wave signal, and the right hand pulse wave signal are synchronized with the collection time as the reference time.

[0124] Feature extraction is performed on the reference signal, the left hand pulse wave signal, and the right hand pulse wave signal to obtain signal propagation time difference features and signal waveform similarity features;

[0125] The signal transmission time difference feature and the signal waveform similarity feature are input into the pre-trained evaluation model to obtain the upper limb arterial occlusion evaluation result.

[0126] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0127] In summary, this invention discloses a method, system, terminal device, and medium for assessing upper limb arterial occlusion based on a portable multi-camera device. The method involves acquiring a reference signal, a left-hand pulse wave signal, and a right-hand pulse wave signal using the portable multi-camera device. These signals are synchronized with the acquisition time as the reference time. Feature extraction is performed on the reference signal, the left-hand pulse wave signal, and the right-hand pulse wave signal to obtain signal transmission time difference features and signal waveform similarity features. These features are then input into a pre-trained assessment model to obtain the assessment result of upper limb arterial occlusion. This invention addresses the characteristic that upper limb arterial occlusion often manifests in localized areas of the left and right upper limbs, reflected in changes in pulse wave transmission time and waveform. It selects a reference signal that is almost unaffected by occlusion in either upper limb, and synchronously acquires the human body reference signal, left-hand pulse wave signal, and right-hand pulse wave signal using a portable multi-camera device. A simple assessment algorithm is designed by combining the signal transmission time difference features and waveform similarity features. This invention is based on the collaborative analysis of three location signals, which overcomes the interference of local occlusion on a single signal and achieves accurate assessment of upper limb arterial occlusion with a simple and convenient algorithm.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for assessing upper limb arterial occlusion based on a portable multi-camera device, characterized in that, The method includes: Based on the portable multi-camera device, a reference signal, a left hand pulse wave signal, and a right hand pulse wave signal are collected. The reference signal, the left hand pulse wave signal, and the right hand pulse wave signal are synchronized with the collection time as the reference time. The reference signal is the facial pulse wave signal. Feature extraction is performed on the reference signal, the left hand pulse wave signal, and the right hand pulse wave signal to obtain signal propagation time difference features and signal waveform similarity features; The signal transmission time difference feature and the signal waveform similarity feature are input into the pre-trained evaluation model to obtain the upper limb arterial occlusion evaluation result; The step of extracting features from the reference signal, the left-hand pulse wave signal, and the right-hand pulse wave signal to obtain signal propagation time difference features and signal waveform similarity features includes: The positioning algorithm was used to locate all the reference signal feature points, left hand pulse wave signal feature points and right hand pulse wave signal feature points. Each feature point corresponds one-to-one with the heartbeat cycle. Based on the reference signal feature points, the left hand pulse wave signal feature points, and the right hand pulse wave signal feature points, the conduction time difference of the left hand pulse wave and the conduction time difference of the right hand pulse wave are calculated. Based on the reference signal, the left hand pulse wave signal, and the right hand pulse wave signal, the waveform similarity features of the left hand pulse wave and the waveform similarity features of the right hand pulse wave are calculated. The calculation of the left-hand pulse wave conduction time difference and the right-hand pulse wave conduction time difference based on the reference signal feature points, the left-hand pulse wave signal feature points, and the right-hand pulse wave signal feature points includes: For each heartbeat cycle, the timestamp of the left hand pulse wave signal feature point is subtracted from the timestamp of the reference signal feature point to obtain the left hand pulse wave conduction time difference. For each heartbeat cycle, the timestamp of the right pulse wave signal feature point is subtracted from the timestamp of the reference signal feature point to obtain the right pulse wave conduction time difference. The calculation of left-hand pulse wave waveform similarity features and right-hand pulse wave waveform similarity features based on the reference signal, the left-hand pulse wave signal, and the right-hand pulse wave signal includes: For each heartbeat cycle, the similarity between the left-hand pulse wave signal and the reference signal is calculated to obtain the left-hand pulse wave waveform similarity features; For each heartbeat cycle, the similarity between the right-hand pulse wave signal and the reference signal is calculated to obtain the right-hand pulse wave waveform similarity features.

2. The method for assessing upper limb arterial occlusion based on a portable multi-camera device according to claim 1, characterized in that, The method for acquiring reference signals, left-hand pulse wave signals, and right-hand pulse wave signals based on a portable multi-camera device includes: The first camera of the portable multi-camera device is used to capture a face to obtain a facial video; Face detection is performed on the facial video to obtain a mask of the facial skin area; Based on the mask of the facial skin region, all pixels of the facial skin region in the facial video are averaged to obtain a sequence of pixel values ​​for the face. The pixel value sequence of the face is preprocessed to obtain the facial pulse wave signal.

3. The method for assessing upper limb arterial occlusion based on a portable multi-camera device according to claim 1, characterized in that, The method for acquiring reference signals, left-hand pulse wave signals, and right-hand pulse wave signals based on a portable multi-camera device includes: Using the second and third cameras of the portable multi-camera device, the same fingertip of either the left or right hand is captured, with the fingertip in contact with the corresponding camera and completely covering the lens during the capture, to obtain a video of the left fingertip and a video of the right fingertip. The left fingertip video and the right fingertip video are averaged to obtain the pixel value sequence of the left fingertip and the pixel value sequence of the right fingertip. The pixel value sequences of the left and right fingertips are preprocessed to obtain the left and right hand pulse wave signals.

4. A system for assessing upper limb arterial occlusion based on a portable multi-camera device, characterized in that, The system, used in implementing the upper limb arterial occlusion assessment method based on a portable multi-camera device as described in any one of claims 1-3, comprises: The signal acquisition module is used to acquire a reference signal, a left hand pulse wave signal, and a right hand pulse wave signal based on a portable multi-camera device. The reference signal, the left hand pulse wave signal, and the right hand pulse wave signal are synchronized with the acquisition time as the reference time. The feature acquisition module is used to extract features from the reference signal, the left hand pulse wave signal, and the right hand pulse wave signal to obtain signal transmission time difference features and signal waveform similarity features; The upper limb arterial occlusion assessment module is used to input the signal transmission time difference feature and the signal waveform similarity feature into a pre-trained assessment model to obtain the upper limb arterial occlusion assessment result.

5. A terminal device, characterized in that, The terminal device includes a memory, a processor, and an upper limb arterial occlusion assessment program based on a portable multi-camera device stored in the memory and executable on the processor. When the processor executes the upper limb arterial occlusion assessment program based on a portable multi-camera device, it implements the steps of the upper limb arterial occlusion assessment method based on a portable multi-camera device as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an upper limb arterial occlusion assessment program based on a portable multi-camera device. When the upper limb arterial occlusion assessment program based on a portable multi-camera device is executed by a processor, it implements the steps of the upper limb arterial occlusion assessment method based on a portable multi-camera device as described in any one of claims 1-3.

Images