A method and system for monitoring respiratory motion based on Kinect

Abstract

This invention relates to the interdisciplinary field of computer image detection and biomedical engineering, and discloses a method and system for respiratory motion monitoring based on Kinect. The method includes: simultaneously acquiring Kinect 3D point cloud sequences and bioimpedance time-series data; extracting first and second respiratory signals respectively; evaluating Kinect signal quality in real time, and generating continuous respiratory waveforms using a main signal output or dynamic weighted fusion strategy under non-interference, interference, and transitional states respectively; and calculating respiratory rate, relative rate of change of tidal volume, and rhythm variation coefficient. The system includes a Kinect sensing unit, a bioimpedance measurement unit, a signal processing and fusion unit, and a respiratory parameter calculation unit. This invention significantly improves the robustness and reliability of respiratory monitoring in complex scenarios through dual-modal redundant sensing and adaptive fusion mechanisms.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of computer image detection and biomedical engineering, specifically relating to a respiratory motion monitoring method and system based on Kinect. Background Technology

[0002] With the widespread application of non-contact physiological signal monitoring technology in scenarios such as telemedicine, sleep analysis, and intensive care, respiratory motion monitoring methods based on depth cameras have attracted widespread attention due to their advantages such as no need to attach sensors and high user compliance. Depth imaging devices, represented by Kinect, can invert respiratory waveforms by capturing minute undulations on the surface of the chest and abdomen. Their core principle relies on real-time tracking of dynamic deformations of the body surface using high-precision point cloud data.

[0003] In practical applications, the human body inevitably undergoes actions such as turning over, sitting up, or large limb movements, which can cause the target area to be obscured or the depth image to be locally distorted, resulting in interruption or severe distortion of the respiratory signal. In addition, complex factors such as changes in ambient lighting and multiple people sharing a room can further reduce the signal-to-noise ratio of the depth image, significantly reducing the robustness of single-modal monitoring systems in real-life scenarios.

[0004] Bioimpedance-based respiratory monitoring technology can stably acquire continuous respiratory signals by measuring the micro-impedance fluctuations caused by volume changes in the thoracic cavity during inspiration and expiration. This method is highly sensitive to micro-movements on the body surface and is not affected by visual obstruction. However, it is susceptible to factors such as poor electrode contact, changes in skin perspiration, and external electromagnetic interference. When used alone, it is difficult to balance long-term wearing comfort with signal stability.

[0005] Existing technologies mostly employ single-modal approaches for respiratory monitoring or simply superimpose multi-source signals without a deep fusion mechanism. While depth images provide spatial morphological information, they are prone to losing effective signals under dynamic interference; bioimpedance, while possessing good temporal continuity, has weak spatial localization capabilities and is susceptible to physiological noise contamination. The complementary potential of these two technologies in complex application scenarios remains untapped, resulting in difficulties maintaining high-precision and robust respiratory waveform reconstruction capabilities when faced with challenges such as significant body movement, partial occlusion, or sensor displacement. Therefore, a multimodal collaborative monitoring method fusing depth images and bioimpedance signals is urgently needed to overcome the inherent limitations of single-modal approaches and improve the reliability and adaptability of respiratory motion monitoring in real-world environments. Summary of the Invention

[0006] This invention provides a Kinect-based respiratory motion monitoring method and system, aiming to solve the technical problem in existing technologies where significant body movement or clothing obstruction leads to interruption of Kinect depth image signals and loss of respiratory waveforms, resulting in respiratory parameter monitoring failure. This invention constructs a dual-channel redundant sensing architecture by fusing two heterogeneous sensing modalities: Kinect depth visual sensing and bioimpedance measurement. When the Kinect signal is interfered with, it seamlessly switches to the bioimpedance channel to acquire respiratory information. Furthermore, a multi-source signal dynamic weighted fusion mechanism generates continuous and stable respiratory motion representations, significantly improving the robustness and reliability of monitoring in complex dynamic scenarios.

[0007] This invention provides a Kinect-based method for monitoring respiratory motion, comprising: The Kinect depth camera is used to acquire real-time three-dimensional point cloud sequences of the chest and abdominal regions of the monitored object. A constant current excitation was applied simultaneously through electrodes attached to both sides of the chest of the monitored object, and voltage response signals were collected to obtain bioimpedance time series data; The three-dimensional point cloud sequence is segmented into chest and abdominal surface regions and respiratory displacement features are extracted to generate a first respiratory signal; The bioimpedance time series data are subjected to baseline drift correction and bandpass filtering to generate a second respiratory signal; The signal-to-noise ratio and continuity index of the first respiratory signal are evaluated in real time. When the index is lower than a preset threshold, the Kinect channel is determined to be in an interference state. In the non-interference state, the first respiratory signal is used as the main output respiratory waveform; in the interference state, the second respiratory signal is used as the main output respiratory waveform. During the transition state, the first respiratory signal and the second respiratory signal are dynamically weighted and fused based on the quantized values ​​of the signal-to-noise ratio and the continuity index to generate a fused respiratory signal. Based on the output respiratory waveform or fused respiratory signal, calculate the respiratory rate, relative rate of change of tidal volume, and coefficient of variation of respiratory rhythm.

[0008] The present invention specifically includes the real-time acquisition of a three-dimensional point cloud sequence of the chest and abdominal region of a monitored object using a Kinect depth camera: Fix the Kinect depth camera 1 to 2 meters in front of the subject being monitored, so that its field of view covers the entire front of the torso. Depth images are acquired at a sampling frequency of 30 frames per second, and the depth images are converted into 3D point clouds using a voxelization method; Using pre-calibrated coordinates of human skeletal joints, the suprasternal notch and xiphoid process are located, thereby defining the boundary of the region of interest in the thoracoabdominal region. Within the region of interest, the time sequence of the mean Z-axis coordinates of all points is extracted as the original respiratory displacement signal.

[0009] The present invention describes the process of segmenting the thoracic and abdominal surface regions and extracting respiratory displacement features from the three-dimensional point cloud sequence to generate a first respiratory signal, specifically including: The original respiratory displacement signal was segmented using a sliding window with a length of 5 seconds and a step size of 0.1 seconds. Within each window, calculate the standard deviation and peak-to-valley difference of the signal. If the standard deviation is less than 0.5 mm and the peak-to-valley difference is less than 2 mm, then mark the window as an invalid window. The signal within the effective window is subjected to a fifth-order Butterworth low-pass filter with a cutoff frequency of 0.5 Hz; The filtered signal is normalized to zero mean to obtain the first respiratory signal.

[0010] The present invention describes the synchronous application of constant current excitation to electrodes attached to both sides of the chest of the monitored object and the acquisition of voltage response signals to obtain bioimpedance time-series data, specifically including: A pair of excitation electrodes and a pair of detection electrodes were attached to the fifth intercostal space on the left side and the fourth intercostal space on the right side of the midclavicular line of the monitored object, respectively. A sinusoidal alternating current with an amplitude of 0.5 mA and a frequency of 50 kHz is injected into the excitation electrode through a constant current source. The voltage difference signal between the detection electrodes is acquired by a high input impedance differential amplifier; The voltage difference signal was converted from analog to digital at a sampling rate of 200 Hz to obtain the original bioimpedance voltage sequence. According to Ohm's law, the voltage difference sequence is divided by the excitation current amplitude to obtain the bioimpedance magnitude time series data.

[0011] The present invention describes baseline drift correction and bandpass filtering of the bioimpedance time-series data to generate a second respiratory signal, specifically including: Wavelet transform soft thresholding denoising method is used to suppress noise in bioimpedance magnitude time series data; A high-pass filter with a cutoff frequency of 0.12 Hz was used to eliminate baseline drift caused by slow movement of body fluids; A low-pass filter with a cutoff frequency of 0.5 Hz was used to filter out cardiac artifacts and electromyographic interference. The filtered signal is normalized in amplitude so that its peak value falls within the range of -1 to +1, thus obtaining the second respiratory signal.

[0012] The real-time evaluation of the signal-to-noise ratio and continuity index of the first respiratory signal described in this invention specifically includes: Calculate the ratio of the main peak of the power spectral density of the first respiratory signal within the current window to the noise power of the adjacent frequency band, and use it as the signal-to-noise ratio; The percentage of valid windows across 10 consecutive windows is used as a continuity indicator. The signal-to-noise ratio threshold is set at 6 dB, and the continuity index threshold is set at 80%. When the signal-to-noise ratio is below 6 dB or the continuity index is below 80%, the Kinect channel is determined to be in an interference state.

[0013] The present invention describes a method for dynamically weighting and fusing the first and second respiratory signals in a transitional state based on the quantized values ​​of the signal-to-noise ratio and continuity index to generate a fused respiratory signal. Specifically, this includes: ; in, This formula maps the signal-to-noise ratio, expressed in decibels, to the [0,1] interval—default. The effective range is 6–16 dB (when When =6, this term is 0; when When the signal-to-noise ratio is 16, this item scores 1, reflecting that "the higher the signal-to-noise ratio, the higher the score of this item"; For continuity (values ​​between 0 and 1), this formula maps it to the interval [0,1]—the default valid range for continuity is 0.8–1.0 (when…). When the value is 0.8, this term is 0; When =1, this item is 1), reflecting that "the better the continuity, the higher the score of this item"; The function restricts the calculation result to the range of 0 to 1.

[0014] ; in, The final fusion breathing signal at that moment; The first respiratory signal This is the second respiratory signal. Dynamic weighting factor. The larger the first respiratory signal The higher the proportion in the fusion result; The smaller the value, the stronger the second respiratory signal. The higher the proportion of the signal, the more "higher quality signal dominates the fusion result." Phase alignment compensation is performed on the fused respiratory signal, and the amount of compensation is determined by the position of the maximum value of the cross-correlation function of the two signals.

[0015] The calculation of respiratory rate, relative rate of change of tidal volume, and coefficient of variation of respiratory rhythm based on the output respiratory waveform or fused respiratory signal, as described in this invention, specifically includes: The instantaneous phase of the respiratory signal is extracted using Hilbert transform, and the instantaneous respiratory rate is obtained by differentiating the phase and dividing by 2π. A stable respiratory rate is obtained by applying a moving average filter to the instantaneous respiratory rate with a window length of 10 seconds. Calculate the ratio of the integral area of ​​the waveforms in adjacent respiratory cycles as the relative rate of change of tidal volume; The ratio of the standard deviation to the mean of the cycle length over 30 consecutive respiratory cycles is used as the coefficient of variation of respiratory rhythm.

[0016] According to another aspect of the present invention, a Kinect-based respiratory motion monitoring system is provided, comprising: The Kinect depth sensing unit is used to acquire three-dimensional point cloud sequences of the chest and abdominal regions of the monitored object in real time. The bioimpedance measurement unit is used to synchronously acquire the bioimpedance time-series data of the monitored object; The Kinect signal processing unit is used to segment the chest and abdominal surface regions and extract respiratory displacement features from the three-dimensional point cloud sequence to generate a first respiratory signal. A bioimpedance signal processing unit is used to perform baseline drift correction and bandpass filtering on the bioimpedance time series data to generate a second respiratory signal. The interference state discrimination unit is used to evaluate the signal-to-noise ratio and continuity index of the first respiratory signal in real time, and to determine whether the Kinect channel is in an interference state. The signal selection and fusion unit is used to output a first breathing signal in a non-interference state, output a second breathing signal in an interference state, and dynamically weight and fuse the two signals to generate a fused breathing signal in a transition state. The respiratory parameter calculation unit is used to calculate the respiratory rate, the relative rate of change of tidal volume, and the coefficient of variation of respiratory rhythm based on the output respiratory waveform or fused respiratory signal.

[0017] The Kinect depth sensing unit of the present invention includes a depth camera module, a point cloud generation module, and a region of interest localization module; the depth camera module outputs depth images at a rate of 30 frames per second; the point cloud generation module maps the depth images into a set of points in three-dimensional space; and the region of interest localization module automatically delineates the thoracic and abdominal joint region based on a human skeleton model.

[0018] The bioimpedance measurement unit of this invention includes a constant current excitation source, a four-electrode patch, a differential amplifier circuit, and a high-speed analog-to-digital converter; the constant current excitation source outputs a sinusoidal current with a frequency of 50 kHz and an amplitude of 0.5 mA; the four-electrode patch is made of silver chloride and has a diameter of 20 mm; the common-mode rejection ratio of the differential amplifier circuit is not less than 120 dB; the high-speed analog-to-digital converter has a sampling rate of 200 Hz and a resolution of 16 bits.

[0019] The interference state discrimination unit of the present invention has a built-in signal-to-noise ratio calculation submodule and a continuity evaluation submodule; the signal-to-noise ratio calculation submodule obtains the power spectral density through fast Fourier transform and calculates the main peak signal-to-noise ratio; the continuity evaluation submodule counts the proportion of effective signal window per unit time.

[0020] The signal selection and fusion unit of the present invention includes a weight calculation submodule, a weighted fusion submodule, and a phase alignment submodule; the weight calculation submodule generates dynamic weights based on the signal-to-noise ratio and continuity index; the weighted fusion submodule performs linear weighting operations; and the phase alignment submodule compensates for the transmission delay difference between the two signals through cross-correlation analysis.

[0021] The respiratory parameter calculation unit of the present invention is configured with an instantaneous frequency extraction module, a tidal volume estimation module, and a rhythm variation analysis module; the instantaneous frequency extraction module uses the analytical signal method to obtain the respiratory phase derivative; the tidal volume estimation module reflects the relative ventilation change through the area ratio of the envelope integral of the respiratory waveform; and the rhythm variation analysis module calculates the coefficient of variation of the respiratory cycle sequence to evaluate the autonomic nervous system regulation function.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention effectively overcomes the inherent limitation of single Kinect solutions in scenarios involving significant body movement or clothing obstruction by constructing a dual-modal sensing system combining Kinect depth vision and bioimpedance measurement. The bioimpedance channel, as an independent redundant sensing path, can seamlessly take over the respiratory monitoring task when Kinect fails, ensuring signal continuity.

[0023] Furthermore, the dynamic weighted fusion mechanism proposed in this invention can adjust the contribution weights of the two signals in real time according to the Kinect signal quality, avoiding waveform jumps caused by hard switching and ensuring a smooth transition and physiological authenticity of the respiratory waveform. In addition, this invention implements targeted preprocessing procedures for the two signals, including precise segmentation of the chest and abdominal regions based on a skeleton model, bioimpedance signal purification using a combination of wavelet denoising and bandpass filtering, and interference criteria based on power spectrum and window effectiveness, significantly improving the signal-to-noise ratio and feature extraction accuracy of each channel. The final output respiratory parameters not only include the basic respiratory rate but also cover higher-order physiological indicators such as the relative rate of change of tidal volume and the coefficient of variation of respiratory rhythm, providing more comprehensive and robust technical support for clinical respiratory function assessment and sleep disorder screening. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the overall technical solution architecture of the present invention; Figure 2 This is a schematic diagram illustrating the core principle framework of the dual-modal perception and dynamic weighted fusion mechanism in this invention; Figure 3 This is a flowchart illustrating the logical flow of Kinect depth signal acquisition and respiratory displacement feature extraction in this invention. Figure 4 This is a flowchart illustrating the logical flow of bioimpedance time-series data acquisition and respiratory signal generation in this invention. Figure 5 This is a flowchart illustrating the logical process of Kinect signal quality assessment and interference state discrimination in this invention. Figure 6 This is a schematic diagram of the multi-level interaction relationship and data flow between the terminal device and the signal processing unit in this invention. Detailed Implementation

[0025] Please refer to the attached document. Figures 1 to 6 This invention provides a Kinect-based respiratory motion monitoring method and system, aiming to solve the technical problem in the prior art where the Kinect depth image signal is interrupted and the respiratory waveform is lost due to large body movements or clothing obstruction, resulting in the failure of respiratory parameter monitoring. This embodiment describes the specific execution flow of the method in detail and discloses the necessary structural composition of the supporting system.

[0026] The method includes the following steps: S1, uses the Kinect depth camera to collect real-time three-dimensional point cloud sequences of the chest and abdominal regions of the monitored object; S2, synchronously apply constant current excitation through electrodes attached to both sides of the chest of the monitored object and collect voltage response signals to obtain bioimpedance time series data; S3, perform chest and abdominal surface region segmentation and respiratory displacement feature extraction on the three-dimensional point cloud sequence to generate a first respiratory signal; S4, perform baseline drift correction and bandpass filtering on the bioimpedance time series data to generate a second respiratory signal; S5. Real-time evaluation of the signal-to-noise ratio and continuity index of the first respiratory signal. When the index is lower than a preset threshold, it is determined that the Kinect channel is in an interference state. S6, In the non-interference state, the first respiratory signal is the main output respiratory waveform; in the interference state, the second respiratory signal is the main output respiratory waveform; in the transition state, the first respiratory signal and the second respiratory signal are dynamically weighted and fused according to the quantized values ​​of the signal-to-noise ratio and continuity index to generate a fused respiratory signal. S7. Based on the output respiratory waveform or fused respiratory signal, calculate the respiratory rate, the relative rate of change of tidal volume, and the coefficient of variation of respiratory rhythm.

[0027] In step S1, a three-dimensional point cloud sequence of the chest and abdominal region of the monitored object is acquired in real time using the Kinect depth camera. Specifically, the Kinect depth camera is fixed 1 to 2 meters directly in front of the monitored object, ensuring its field of view covers the entire front of the torso. The Kinect depth camera continuously outputs depth images at a sampling frequency of 30 frames per second. The pixel value in each frame of the depth image represents the vertical distance from the corresponding spatial point to the camera plane. After receiving the depth images, the point cloud generation module, in conjunction with the camera intrinsic parameter matrix, maps each pixel coordinate and its depth value to a three-dimensional spatial point in the world coordinate system, forming a point cloud set containing X, Y, and Z coordinates. To accurately locate respiratory-related areas, the system calls the Kinect's built-in human skeleton tracking algorithm to acquire the coordinates of key torso points, including the suprasternal notch and xiphoid process joints, in real time.

[0028] The suprasternal notch joint is located at the top of the sternum below the neck, and the xiphoid joint is located at the lowest point of the sternum. Using the midpoint of the horizontal projection of these two joints as a reference, an area is extended upwards to the suprasternal notch, downwards to the xiphoid process, and laterally to the left and right anterior axillary lines, forming a rectangular region as the boundary of the region of interest (ROI) within the thoracoabdominal region. Within this ROI, the Z-axis coordinates (i.e., depth coordinates) of all 3D points are extracted, and their arithmetic mean is calculated to obtain the average depth value of the thoracoabdominal surface for that frame. The average depth values ​​of consecutive frames are arranged chronologically to form the original respiratory displacement signal. The periodic fluctuations of this signal directly reflect the anterior-posterior displacement of the thoracoabdominal region caused by respiratory movements.

[0029] In step S3, the 3D point cloud sequence is segmented into chest and abdominal surface regions and respiratory displacement features are extracted to generate a first respiratory signal. The original respiratory displacement signal needs to undergo validity screening and filtering. A sliding window mechanism is used to segment the signal, with a window length of 5 seconds and a step size of 0.1 seconds. For each signal segment within the sliding window, its standard deviation and peak-to-valley difference are calculated. The standard deviation reflects the dispersion of signal fluctuations, and the peak-to-valley difference is the difference between the maximum and minimum values ​​within the window, representing the respiratory amplitude. If the standard deviation of a window is less than 0.5 mm and the peak-to-valley difference is less than 2 mm, it is determined that there is no effective respiratory movement within that window and it is marked as an invalid window. Only signal segments within the valid windows are retained.

[0030] A fifth-order Butterworth low-pass filter is applied to all signals within the valid window, with a cutoff frequency set to 0.5 Hz, to filter out high-frequency noise above the normal respiratory rate range, such as interference introduced by body tremors or equipment vibration. The filtered signal is then subjected to zero-mean normalization, i.e., the mean of the signal within the window is subtracted, and then divided by its standard deviation, so that the output signal has a mean of 0 and a variance of 1. The signal after this processing is the first respiratory signal, which has a smooth waveform, stable amplitude, and good respiratory cycle characteristics.

[0031] In step S2, constant current excitation is applied synchronously through electrodes attached to both sides of the chest of the monitored subject, and voltage response signals are acquired to obtain bioimpedance time-series data. Four silver chloride electrode pads are attached to specific locations on the body surface of the monitored subject: one pair of excitation electrodes are placed in the fifth intercostal space on the left and the fourth intercostal space on the right midclavicular line, and another pair of detection electrodes are placed adjacent to the excitation electrodes on the inner side, forming a four-electrode measurement configuration. A constant current excitation source injects a sinusoidal alternating current with an amplitude of 0.5 mA and a frequency of 50 kHz into the excitation electrodes.

[0032] This frequency is much higher than the physiological frequencies of heartbeat and respiration, effectively avoiding reverse interference from physiological signals to the excitation source. A high input impedance differential amplifier is connected across the detection electrodes to acquire the voltage difference signal between them. The common-mode rejection ratio of the differential amplifier is no less than 120 dB, ensuring effective suppression of environmental electromagnetic interference. A high-speed analog-to-digital converter digitizes the amplified voltage difference analog signal at a sampling rate of 200 Hz to obtain the original bioimpedance voltage sequence. According to Ohm's law, the bioimpedance magnitude is equal to the voltage difference divided by the excitation current amplitude. Therefore, dividing the value of each sampling point in the original voltage sequence by 0.5 mA yields the bioimpedance magnitude time-series data. This data fluctuates periodically with changes in thoracic cavity volume, and its trend is synchronized with respiratory movements.

[0033] In step S4, baseline drift correction and bandpass filtering are performed on the bioimpedance time series data to generate a second respiratory signal. The original bioimpedance modulus time series data contains various noise components. First, a wavelet transform soft-thresholding denoising method is used for preliminary noise reduction. The db4 wavelet basis function is selected, and the decomposition is performed in five layers. A general threshold is applied to the detail coefficients of each layer for soft-thresholding processing to reconstruct the signal and remove random white noise.

[0034] Subsequently, a high-pass filter with a cutoff frequency of 0.12 Hz was used to eliminate low-frequency baseline drift caused by factors such as slow fluid movement and electrode contact impedance drift. Next, a low-pass filter with a cutoff frequency of 0.5 Hz was used to filter out high-frequency components such as cardiac artifacts (typically above 1.5 Hz) and electromyographic interference. After the above bandpass filtering (0.12 Hz to 0.5 Hz), the signal mainly retains the effective information of the respiratory frequency band. Finally, the amplitude of the filtered signal was normalized, scaling its maximum value to +1 and its minimum value to -1 to obtain the second respiratory signal. This signal waveform is clear and highly consistent with the actual respiratory cycle.

[0035] In step S5, the signal-to-noise ratio (SNR) and continuity index of the first respiratory signal are evaluated in real time. When the index is lower than a preset threshold, the Kinect channel is determined to be in an interference state. The SNR is calculated based on power spectral density analysis. A fast Fourier transform is performed on the first respiratory signal within the current sliding window to obtain its spectrum. The maximum value of the power spectral density is found in the respiratory frequency band from 0.12 Hz to 0.5 Hz and recorded as the main peak power. The average noise power is calculated in the two sidelobe frequency bands adjacent to the main peak (e.g., the main peak frequency plus or minus 0.05 Hz). The SNR is defined as the ratio of the main peak power to the average noise power, expressed in decibels (dB). The continuity index is obtained by statistically analyzing the effective window ratio.

[0036] The system maintains a 10-fold FIFO queue, recording the validity status of the 10 most recent sliding windows (1 for valid, 0 for invalid). The continuity metric is calculated by dividing the number of valid windows in the queue by 10. The preset signal-to-noise ratio (SNR) threshold is 6 dB, and the continuity metric threshold is 80%. When the real-time calculated SNR is below 6 dB, or the continuity metric is below 80%, the system determines that the Kinect channel is in an interference state. If both metrics are above their respective thresholds, it is in a non-interference state. If one metric is below its threshold while the other is above, it enters a transition state.

[0037] In step S6, under non-interference conditions, the first respiratory signal is used as the primary output respiratory waveform; under interference conditions, the second respiratory signal is used as the primary output respiratory waveform; under transition conditions, the first and second respiratory signals are dynamically weighted and fused based on the quantized values ​​of the signal-to-noise ratio and continuity index to generate a fused respiratory signal. The formula for calculating the weighting factor W is as follows: ; in, This formula maps the signal-to-noise ratio, expressed in decibels, to the [0,1] interval—default. The effective range is 6–16 dB (when When =6, this term is 0; when When the signal-to-noise ratio is 16, this item scores 1, reflecting that "the higher the signal-to-noise ratio, the higher the score of this item"; For continuity (values ​​between 0 and 1), this formula maps it to the interval [0,1]—the default valid range for continuity is 0.8–1.0 (when…). When the value is 0.8, this term is 0; When =1, this item is 1), reflecting that "the better the continuity, the higher the score of this item"; The function restricts the calculation results to the interval between 0 and 1. The formula for calculating the fused respiratory signal is: ; in, The final fusion breathing signal at that moment; The first respiratory signal This is the second respiratory signal. Dynamic weighting factor. The larger the first respiratory signal The higher the proportion in the fusion result; The smaller the value, the stronger the second respiratory signal. The higher the proportion of the signal, the more "higher quality signal dominates the fusion result." Due to inherent delay differences in the acquisition and processing links of the two signals, direct weighting may lead to phase mismatch. Therefore, phase alignment compensation is required before weighting. The phase alignment submodule calculates the cross-correlation function of the two signals within the most recent 5-second window and finds the time offset when the cross-correlation function reaches its maximum value. The second respiratory signal is delayed overall. Seconds (or ahead of the first respiratory signal) (seconds) to align the two on the time axis, and then perform a weighted fusion operation. The final output breathing waveform comes entirely from the Kinect channel in the non-interference state, completely switches to the bioimpedance channel in the interference state, and is a smoothly transitioned fused signal in the transition state, avoiding waveform discontinuity or jumps caused by hard switching.

[0038] In step S7, based on the output respiratory waveform or fused respiratory signal, the respiratory rate, relative rate of change of tidal volume, and coefficient of variation of respiratory rhythm are calculated. The respiratory rate is calculated using the Hilbert transform method. The output respiratory signal is subjected to a Hilbert transform to obtain its analytic signal. The phase angle of the analytic signal is the instantaneous phase. The instantaneous respiratory rate is obtained by differentiating the instantaneous phase with respect to time and dividing by 2π. To eliminate glitches in the instantaneous frequency, a moving average filter is applied with a window length of 10 seconds to obtain a stable respiratory rate. The relative rate of change of tidal volume reflects the relative change in ventilation between adjacent respiratory cycles. The system first identifies the inspiratory peak and expiratory trough of the respiratory signal using a peak detection algorithm to determine the start and end points of each complete respiratory cycle. The absolute value of the respiratory signal within each cycle is integrated to obtain the waveform integral area for that cycle.

[0039] The relative rate of change of tidal volume is defined as the ratio of the integral area of ​​the current cycle to the integral area of ​​the previous cycle. The coefficient of variation (COP) is used to assess the stability of the respiratory rhythm. The system records the cycle length (i.e., the time interval between adjacent inspiratory peaks) of 30 consecutive respiratory cycles. The COP is calculated as the ratio of the standard deviation to the mean of these 30 cycle lengths. A larger COP indicates a more irregular respiratory rhythm, which may be related to a pathological condition.

[0040] The Kinect-based respiratory motion monitoring system includes a Kinect depth sensing unit, a bioimpedance measurement unit, a Kinect signal processing unit, a bioimpedance signal processing unit, an interference state discrimination unit, a signal selection and fusion unit, and a respiratory parameter calculation unit.

[0041] The Kinect depth sensing unit comprises a depth camera module, a point cloud generation module, and a region of interest (ROI) localization module. The depth camera module, based on the Microsoft Kinect for Windows v2 device, outputs 512×424 resolution depth images at 30 frames per second. The point cloud generation module, running on an embedded processor, converts the depth images into 3D point clouds in real time using camera calibration parameters. The ROI localization module utilizes the Kinect SDK's human skeleton tracking interface to obtain the coordinates of the suprasternal notch and xiphoid process joints, and automatically delineates the thoracoabdominal region accordingly.

[0042] The bioimpedance measurement unit includes a constant current excitation source, a four-electrode patch, a differential amplifier circuit, and a high-speed analog-to-digital converter (ADC). The constant current excitation source consists of a function generator and a voltage-to-current conversion circuit, outputting a sinusoidal current with a frequency of 50 kHz and an amplitude of 0.5 mA. The four-electrode patch uses 20 mm diameter medical-grade silver chloride electrodes to ensure low contact impedance and high signal stability. The differential amplifier circuit uses an instrumentation amplifier chip with a measured common-mode rejection ratio of 125 dB. The high-speed ADC is a 16-bit Σ-Δ ADC with a sampling rate of 200 Hz, integrated into the signal acquisition board.

[0043] The Kinect signal processing unit performs all the functions of step S3, including sliding window segmentation, validity determination, Butterworth filtering, and normalization. The bioimpedance signal processing unit performs all the functions of step S4, including wavelet denoising, high-pass and low-pass filtering, and amplitude normalization.

[0044] The interference state discrimination unit incorporates a signal-to-noise ratio (SNR) calculation submodule and a continuity assessment submodule. The SNR calculation submodule performs a Fast Fourier Transform on the first respiratory signal and calculates the main peak SNR according to a preset frequency band. The continuity assessment submodule maintains an effective window counting queue and updates the continuity index in real time.

[0045] The signal selection and fusion unit comprises a weight calculation submodule, a weighted fusion submodule, and a phase alignment submodule. The weight calculation submodule calculates the weight factor W in real time based on the discrimination result and quantization index from step S5. The weighted fusion submodule performs linear weighting operations. The phase alignment submodule determines the optimal alignment delay between the two signals through cross-correlation analysis and implements compensation.

[0046] The respiratory parameter calculation unit is equipped with an instantaneous frequency extraction module, a tidal volume estimation module, and a rhythm variability analysis module. The instantaneous frequency extraction module performs Hilbert transform and phase differentiation. The tidal volume estimation module performs period segmentation and integral area calculation. The rhythm variability analysis module calculates the period length and the coefficient of variation.

[0047] This embodiment achieves continuous, stable, and highly robust monitoring of respiratory movements under complex scenarios such as significant body movement or clothing obstruction, through the aforementioned method and system. The dual-modal sensing architecture ensures signal source redundancy, the dynamic weighted fusion mechanism guarantees the physiological authenticity of the output waveform, and the output of multi-dimensional respiratory parameters provides comprehensive support for clinical assessment.

Claims

1. A Kinect-based respiratory motion monitoring method, characterized in that, include: The Kinect depth camera is used to acquire real-time three-dimensional point cloud sequences of the chest and abdominal regions of the monitored object. A constant current excitation was applied simultaneously through electrodes attached to both sides of the chest of the monitored object, and voltage response signals were collected to obtain bioimpedance time series data; The three-dimensional point cloud sequence is segmented into chest and abdominal surface regions and respiratory displacement features are extracted to generate a first respiratory signal; The bioimpedance time series data are subjected to baseline drift correction and bandpass filtering to generate a second respiratory signal; The signal-to-noise ratio and continuity index of the first respiratory signal are evaluated in real time. When the index is lower than a preset threshold, the Kinect channel is determined to be in an interference state. In a non-interference state, the first respiratory signal is used as the primary output respiratory waveform. Under interference conditions, the second respiratory signal is used as the primary output respiratory waveform. During the transition state, the first respiratory signal and the second respiratory signal are dynamically weighted and fused based on the quantized values ​​of the signal-to-noise ratio and the continuity index to generate a fused respiratory signal. Based on the output respiratory waveform or fused respiratory signal, calculate the respiratory rate, relative rate of change of tidal volume, and coefficient of variation of respiratory rhythm.

2. The Kinect-based respiratory motion monitoring method according to claim 1, characterized in that, The Kinect depth camera acquires real-time 3D point cloud sequences of the chest and abdominal regions of the monitored object, including: Fix the Kinect depth camera 1 to 2 meters in front of the subject being monitored, so that its field of view covers the entire front of the torso. Depth images are acquired at a sampling frequency of 30 frames per second, and the depth images are converted into 3D point clouds using a voxelization method; Using pre-calibrated coordinates of human skeletal joints, the suprasternal notch and xiphoid process are located, thereby defining the boundary of the region of interest in the thoracoabdominal region. Within this region of interest, the time sequence of the mean Z-axis coordinates of all points is extracted as the original respiratory displacement signal.

3. The Kinect-based respiratory motion monitoring method according to claim 2, characterized in that, The three-dimensional point cloud sequence is segmented into chest and abdominal surface regions and respiratory displacement features are extracted to generate a first respiratory signal, including: The original respiratory displacement signal was segmented using a sliding window with a length of 5 seconds and a step size of 0.1 seconds. Within each window, calculate the standard deviation and peak-to-valley difference of the signal. If the standard deviation is less than 0.5 mm and the peak-to-valley difference is less than 2 mm, then mark the window as an invalid window. The signal within the effective window is subjected to a fifth-order Butterworth low-pass filter with a cutoff frequency of 0.5 Hz; The filtered signal is normalized to zero mean to obtain the first respiratory signal.

4. The Kinect-based respiratory motion monitoring method according to claim 3, characterized in that, A constant current excitation was applied simultaneously through electrodes attached to both sides of the chest of the monitored subject, and voltage response signals were acquired to obtain bioimpedance time-series data, including: A pair of excitation electrodes and a pair of detection electrodes were attached to the fifth intercostal space on the left side and the fourth intercostal space on the right side of the midclavicular line of the monitored object, respectively. A sinusoidal alternating current with an amplitude of 0.5 mA and a frequency of 50 kHz is injected into the excitation electrode through a constant current source. The voltage difference signal between the detection electrodes is acquired by a high input impedance differential amplifier; The voltage difference signal was converted from analog to digital at a sampling rate of 200 Hz to obtain the original bioimpedance voltage sequence. According to Ohm's law, the voltage difference sequence is divided by the excitation current amplitude to obtain the bioimpedance magnitude time series data.

5. The Kinect-based respiratory motion monitoring method according to claim 4, characterized in that, The bioimpedance time series data are subjected to baseline drift correction and bandpass filtering to generate a second respiratory signal, including: Wavelet transform soft thresholding denoising method is used to suppress noise in bioimpedance magnitude time series data; A high-pass filter with a cutoff frequency of 0.12 Hz was used to eliminate baseline drift caused by slow movement of body fluids; A low-pass filter with a cutoff frequency of 0.5 Hz was used to filter out cardiac artifacts and electromyographic interference. The filtered signal is normalized in amplitude so that its peak value falls within the range of -1 to +1, thus obtaining the second respiratory signal.

6. The Kinect-based respiratory motion monitoring method according to claim 5, characterized in that, Real-time evaluation of the signal-to-noise ratio and continuity indices of the first respiratory signal, including: Calculate the ratio of the main peak of the power spectral density of the first respiratory signal within the current window to the noise power of the adjacent frequency band, and use it as the signal-to-noise ratio; The percentage of valid windows across 10 consecutive windows is used as a continuity indicator. The signal-to-noise ratio threshold is set at 6 dB, and the continuity index threshold is set at 80%. When the signal-to-noise ratio is below 6 dB or the continuity index is below 80%, the Kinect channel is determined to be in an interference state.

7. The Kinect-based respiratory motion monitoring method according to claim 6, characterized in that, In the transition state, the first respiratory signal and the second respiratory signal are dynamically weighted and fused based on the quantized values ​​of the signal-to-noise ratio and the continuity index to generate a fused respiratory signal, including: Define a weighting factor, which, after amplitude limiting, takes a value between 0 and 1; Phase alignment compensation is performed on the fused respiratory signal, and the compensation amount is determined by the position of the maximum value of the cross-correlation function of the two signals.

8. The Kinect-based respiratory motion monitoring method according to claim 7, characterized in that, Based on the output respiratory waveform or fused respiratory signal, calculate the respiratory rate, relative rate of change of tidal volume, and coefficient of variation of respiratory rhythm, including: The instantaneous phase of the respiratory signal is extracted using Hilbert transform, and the instantaneous respiratory rate is obtained by differentiating the phase and dividing by 2π. A stable respiratory rate is obtained by applying a moving average filter to the instantaneous respiratory rate with a window length of 10 seconds. Calculate the ratio of the integral area of ​​the waveforms in adjacent respiratory cycles as the relative rate of change of tidal volume; The ratio of the standard deviation to the mean of the cycle length over 30 consecutive respiratory cycles is used as the coefficient of variation of respiratory rhythm.

9. A Kinect-based respiratory motion monitoring system, characterized in that, include: The Kinect depth sensing unit is used to acquire three-dimensional point cloud sequences of the chest and abdominal regions of the monitored object in real time. The bioimpedance measurement unit is used to synchronously acquire the bioimpedance time-series data of the monitored object; The Kinect signal processing unit is used to segment the chest and abdominal surface regions and extract respiratory displacement features from the three-dimensional point cloud sequence to generate a first respiratory signal. A bioimpedance signal processing unit is used to perform baseline drift correction and bandpass filtering on the bioimpedance time series data to generate a second respiratory signal. The interference state discrimination unit is used to evaluate the signal-to-noise ratio and continuity index of the first respiratory signal in real time, and to determine whether the Kinect channel is in an interference state. The signal selection and fusion unit is used to output a first breathing signal in a non-interference state, output a second breathing signal in an interference state, and dynamically weight and fuse the two signals to generate a fused breathing signal in a transition state. The respiratory parameter calculation unit is used to calculate the respiratory rate, the relative rate of change of tidal volume, and the coefficient of variation of respiratory rhythm based on the output respiratory waveform or fused respiratory signal.

10. A method for monitoring respiratory motion based on Kinect, characterized in that, The Kinect-based respiratory motion monitoring system described in any one of claims 1-9.

Images