A non-contact electrocardio signal monitoring method and device based on dot matrix structured light irradiation

Abstract

The application provides a non-contact electrocardio signal monitoring method and device based on dot matrix structured light irradiation. The method comprises the following steps: projecting a dot matrix structured light spot to a target area in front of the chest of a living body, and collecting a structured light spot image sequence generated by displacement change of the dot matrix structured light spot due to surface vibration of the target area; tracking the dot matrix light spot in the structured light spot image sequence to obtain displacement trajectories of each light spot, and constructing a mechanical sequence signal related to a cardiac cycle according to the displacement trajectories; processing the mechanical sequence signal to obtain a heart sound signal and an envelope signal corresponding to the heart sound signal, and inputting the heart sound signal and the envelope signal into a pre-trained electrocardio signal fitting model to generate a fitted electrocardio signal. The technical scheme of the application can stably obtain mechanical information with high signal-to-noise ratio which is strongly related to the cardiac cycle without electrode attachment, and realize fitting of the electrocardio signal, thereby improving the comfort, reliability and application range of ECG continuous monitoring.

Description

Technical Field

[0001] This invention relates to the field of biomedical signal detection technology, specifically to a non-contact electrocardiogram (ECG) signal monitoring method and device based on dot matrix structured light irradiation. Background Technology

[0002] An electrocardiogram (ECG) is an important physiological signal reflecting the electrical activity of the heart and is widely used for the diagnosis, monitoring, and long-term follow-up of cardiovascular diseases. Traditional ECG monitoring generally uses surface electrodes to obtain the electrocardiogram waveform by detecting changes in surface potential caused by myocardial depolarization and repolarization.

[0003] However, the ECG monitoring method based on electrode attachment has some shortcomings in practical use:

[0004] 1) Poor comfort and compliance: The patch electrodes need to be directly attached to the skin, which may cause discomfort, skin irritation or allergies, affecting the user's willingness to wear them for a long time;

[0005] 2) Limited reliability of continuous monitoring: Electrodes are prone to falling off or loosening due to sweating, limb movement or poor adhesion, which leads to a decrease in signal quality or even interruption, affecting the stability of long-term continuous monitoring.

[0006] 3) Limited application scenarios: For burn patients, people at risk of skin infection, premature infants, etc., attaching electrodes to the body surface may be inconvenient or pose no safety risks.

[0007] To reduce reliance on electrodes, various non-contact or low-contact cardiac monitoring technologies have emerged, such as remote photoplethysmography (PPG) based on optical video, radar-based chest wall micromotion detection, and acoustic or acceleration-based cardiac oscillation (SCG) signal detection. Most of these technologies indirectly reflect cardiac status by detecting cardiac mechanical activity, but their output is usually limited to heart rate or some mechanical event parameters, making it difficult to directly reconstruct standard ECG waveforms suitable for clinical diagnosis.

[0008] Existing research has attempted to reconstruct ECG waveforms using cardiac mechanical signals. The main idea is to leverage the coupling relationship between cardiac electrical and mechanical activities to establish a mapping model from the mechanical domain to the electrical domain, thereby achieving ECG waveform estimation without an electrical motor. For example, some studies have used millimeter-wave radar to measure chest wall micromotions and combined this with deep learning algorithms to reconstruct ECG waveforms, verifying the technical feasibility of non-contact ECG monitoring.

[0009] Nevertheless, the relevant contactless ECG technology still faces the following major problems:

[0010] First, signal acquisition is susceptible to interference: the mechanical movements of the body surface caused by the heart are usually in the sub-millimeter range, with an amplitude of about 0.2–0.5 mm. Therefore, it is difficult to extract the signal stably under stronger interference such as breathing and body movement.

[0011] Second, it is sensitive to posture and individual differences: In reflective measurement schemes such as radar, echo reflection is highly sensitive to human posture. Small body twists may cause significant changes in reflection characteristics. There is a lack of consistent characterization of reflection characteristics among different subjects. Superimposed with sub-millimeter-level cardiac micromotion, it leads to difficulties in robust capture.

[0012] Third, the feasibility of the solution is not high: the solution relies on high-precision equipment or strict measurement conditions, and has high requirements for ambient lighting, equipment alignment, and algorithm computing resources, which is not conducive to popularization and promotion in home or routine clinical environments. Summary of the Invention

[0013] In view of this, this application provides a non-contact ECG signal monitoring method and device based on dot matrix structured light illumination, which can stably acquire high signal-to-noise mechanical information strongly correlated with the cardiac cycle without the need for electrode attachment, and achieve ECG signal fitting, thereby improving the comfort, reliability and applicability of continuous ECG monitoring.

[0014] Specifically, this application is implemented through the following technical solution:

[0015] According to a first aspect of the embodiments of this specification, a non-contact electrocardiogram (ECG) signal monitoring method based on dot matrix structured light illumination is provided, comprising the following steps:

[0016] Step S1: Project a dot matrix structured light spot onto the target area on the chest of the organism, and collect a sequence of structured light spot images showing displacement changes as the surface of the target area vibrates.

[0017] Step S2: Track the dot matrix spots in the structured light spot image sequence to obtain the displacement trajectory of each spot, and construct a mechanical sequence signal related to the cardiac cycle based on the displacement trajectory;

[0018] Step S3: Process the mechanical sequence signal to obtain the heart sound signal and its corresponding envelope signal;

[0019] Step S4: Input the heart sound signal and its corresponding envelope signal into a pre-trained ECG signal fitting model to generate a fitted ECG signal; wherein the ECG signal fitting model is configured to process the input using a first feature extraction branch corresponding to the heart sound signal and a second feature extraction branch corresponding to the envelope signal, and then adaptively fuse the outputs of the two branches to output the fitted ECG signal.

[0020] According to a second aspect of the embodiments of this specification, a non-contact electrocardiogram (ECG) signal monitoring device based on dot matrix structured light illumination is provided, the device comprising:

[0021] An image acquisition unit is used to project a dot matrix structured light spot onto a target area on the chest of an organism, and to acquire a sequence of structured light spot images showing displacement changes as the surface of the target area vibrates.

[0022] The signal construction unit is used to track the dot matrix spots in the structured light spot image sequence, obtain the displacement trajectory of each spot, and construct a mechanical sequence signal related to the cardiac cycle based on the displacement trajectory.

[0023] A signal processing unit is used to process the mechanical sequence signal to obtain the heart sound signal and its corresponding envelope signal;

[0024] The signal fitting unit is used to input the heart sound signal and its corresponding envelope signal into a pre-trained electrocardiogram (ECG) signal fitting model to generate a fitted ECG signal; wherein the ECG signal fitting model is configured to process the input using a first feature extraction branch corresponding to the heart sound signal and a second feature extraction branch corresponding to the envelope signal, and then adaptively fuse the outputs of the two branches to output the fitted ECG signal.

[0025] According to a third aspect of the embodiments of this specification, an electronic device is provided, including a processor; and a computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method described in the first aspect.

[0026] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executed by a processor of the method described in the first aspect.

[0027] The embodiments of this application have achieved at least the following technical effects:

[0028] (1) It can output ECG-like waveforms without the need for surface electrodes, which significantly improves monitoring comfort and lowers the barrier to use;

[0029] (2) Dot matrix light source projection has the characteristics of stable pattern, clear spatial sampling and stronger adaptability to low texture skin areas, which can improve the reliability of chest wall micro-vibration extraction;

[0030] (3) By using a unified link of lattice displacement, mechanical vibration sequence and electrocardiogram fitting, a closed-loop framework for non-contact mechanical observation of electrocardiogram waveform output is realized, which facilitates quality assessment and clinical interpretation.

[0031] (4) The system has a clear structure and is easy to integrate and deploy, making it suitable for long-term continuous non-contact ECG monitoring in scenarios such as clinical monitoring, rehabilitation follow-up and family health management. Attached Figure Description

[0032] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0033] Figure 1 This is a schematic diagram illustrating an electrocardiogram (ECG) signal fitting process according to an exemplary embodiment of this application;

[0034] Figure 2 This is a flowchart illustrating an exemplary embodiment of the present application of a non-contact electrocardiogram signal monitoring method based on dot matrix structured light illumination;

[0035] Figure 3 This is a schematic diagram of the light spot encoding in a structured light spot image shown in an exemplary embodiment of this application;

[0036] Figure 4 This is a schematic diagram of the mechanical sequence signal of the main vibration signal shown in an exemplary embodiment of this application;

[0037] Figure 5 This is a schematic diagram of heart sound signals illustrated in an exemplary embodiment of this application;

[0038] Figure 6 This is a schematic diagram of the envelope curve of a heart sound signal illustrated in an exemplary embodiment of this application;

[0039] Figure 7 This is a schematic diagram of the architecture of an electrocardiogram signal fitting model illustrated in an exemplary embodiment of this application;

[0040] Figure 8 This is a schematic diagram of the skeleton key points shown in an exemplary embodiment of this application;

[0041] Figure 9 This is a block diagram illustrating an electronic device according to an exemplary embodiment of this application;

[0042] Figure 10 This is a block diagram of a non-contact electrocardiogram signal monitoring device based on dot matrix structured light illumination, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0043] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0044] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0045] In view of the shortcomings of existing electrode-attached ECG monitoring in terms of comfort, long-term continuity and adaptation to special populations, as well as the technical problems of existing non-contact ECG reconstruction schemes such as small cardiac micro-motion amplitude, susceptibility to physiological motion interference, sensitivity to posture changes, and insufficient signal stability and engineering feasibility, this application proposes a non-contact ECG signal monitoring scheme based on dot matrix structured light illumination.

[0046] like Figure 1 As shown, the basic concept of this application embodiment is as follows: a dot matrix laser projection device is used to project a dot matrix pattern onto a target area on the chest of a human body, and an imaging device is used to collect a sequence of structured light spot images in which the dot matrix pattern changes displacement due to micro-vibration of the chest wall; dot matrix feature extraction and motion estimation are performed on the structured light spot image sequence to obtain multi-point displacement trajectories; by estimating the main motion direction of the multi-point displacement trajectories, a one-dimensional mechanical sequence signal related to the cardiac cycle is constructed; on this basis, causal filtering, drift suppression, short window normalization and anti-interference enhancement are used to improve the heart sound signal and its corresponding envelope signal of cardiac-related components; finally, a fitted electrocardiogram signal is output through a pre-trained electrocardiogram signal fitting model.

[0047] In addition, to improve the usability and stability of the system under natural body position changes, this application embodiment also introduces a posture estimation and detection quality assessment mechanism: the current body position of the subject is judged, and when a side-lying, severely tilted, occluded, or other body position that is not conducive to the characterization of chest wall micro-vibration is detected, an alert signal is output to prompt the user to adjust the posture. At the same time, the confidence of the fitted output is automatically reduced, the output is paused, the ROI is updated, or the processing strategy is switched according to the posture occlusion situation, thereby avoiding the generation of misfit results under low-quality input conditions.

[0048] The embodiments described in this specification will now be described in detail.

[0049] This application provides a non-contact electrocardiogram signal monitoring method based on dot matrix structured light illumination. Figure 2 This is a flowchart illustrating an exemplary embodiment of a non-contact electrocardiogram signal monitoring method based on lattice structured light illumination, as shown in this application. Figure 2 As shown, the electrocardiogram signal monitoring method of this embodiment includes at least the following steps:

[0050] Step S1: Project a dot matrix structured light spot onto the target area on the chest of the organism, and collect a sequence of structured light spot images showing displacement changes as the surface of the target area vibrates.

[0051] The target area includes the skin surface area located in the anterior part of the thoracic cavity of an organism, which can effectively transmit and reflect the micro-vibrations on the body surface caused by the mechanical activity of the heart.

[0052] The dot matrix structured light spot refers to a pattern formed on the target surface by a set of discrete light spots with clear positioning, created through a specific optical projection system. For example, this pattern is usually composed of multiple independent light spots, and their spatial distribution can be a regular matrix arrangement or an optimized pseudo-random distribution, in order to provide rich and stable texture features for subsequent image tracking.

[0053] In practical applications, a dot-matrix laser projection device can project a dot-matrix structured light spot onto the target area on the chest of a living organism. A near-infrared vertical-cavity surface-emitting laser with a wavelength of approximately 850 nm is preferred. This wavelength is safe for the human eye and is less likely to cause attention or discomfort to the subject compared to visible light; furthermore, this wavelength has a certain degree of tissue penetration and can be well received by ordinary silicon-based image sensors.

[0054] After the dot pattern is projected onto the chest wall, it will undergo synchronous displacement due to the minute vibrations on the body surface caused by the heartbeat. In order to record this dynamic process, the system is equipped with an imaging device to acquire the above process and obtain a sequence of structured light spot images.

[0055] Step S2: Track the dot matrix spots in the structured light spot image sequence to obtain the displacement trajectory of each spot, and construct a mechanical sequence signal related to the cardiac cycle based on the displacement trajectory.

[0056] Step S3: Process the mechanical sequence signal to obtain the heart sound signal and its corresponding envelope signal.

[0057] The heart sound signal is used to characterize local vibration information related to transient events such as valve opening and closing and blood flow impact during cardiac activity. It can provide richer short-range detail features, such as local peak morphology, instantaneous energy changes and high-frequency textures, thereby improving the morphological fidelity and detail expression of the reconstructed electrocardiogram waveform.

[0058] The envelope signal is obtained by extracting and smoothing the amplitude envelope of the heart sound signal. It is used to reflect the overall trend and rhythm changes of energy fluctuations within the cardiac cycle. It can provide stable long-range structural features and periodic constraint information, including cardiac cycle rhythm, low-frequency trend and cross-cycle consistency, thereby enhancing the model's temporal stability and anti-interference ability over long time scales.

[0059] Step S4: Input the heart sound signal and its corresponding envelope signal into a pre-trained ECG signal fitting model to generate a fitted ECG signal; wherein the ECG signal fitting model is configured to process the input using a first feature extraction branch corresponding to the heart sound signal and a second feature extraction branch corresponding to the envelope signal, and then adaptively fuse the outputs of the two branches to output the fitted ECG signal.

[0060] like Figure 2 As shown in the ECG signal monitoring method, this embodiment projects an invisible structural light spot onto the body surface and analyzes its micro-movements, acquiring information on cardiac mechanical activity without the need for electrode attachment. This effectively avoids skin irritation, allergies, and discomfort associated with traditional electrodes, improving user compliance with long-term monitoring and making it suitable for special populations such as burn victims and newborns. By analyzing the displacement trajectory of the structural light spot, not only are heart sound signals reflecting rapid cardiac vibration events extracted, but also envelope signals characterizing the overall motion trend are obtained. This allows for a more comprehensive and layered capture of the detailed information and macroscopic morphology of cardiac mechanical activity, providing a richer feature base for subsequent high-precision waveform reconstruction. Correspondingly, the pre-trained ECG signal fitting model used in this embodiment, by setting dual feature extraction branches to process heart sound signals and envelope signals respectively, can perform targeted feature learning for the unique patterns of different signal types. By integrating the two information streams through an adaptive fusion strategy, the reconstructed fitted ECG signal can more reliably restore the waveform morphology and rhythm characteristics of the real ECG, reducing sensitivity to noise or interference from a single signal source. This embodiment is based on the principle of optical imaging, has a relatively simple structure, is easy to integrate into devices such as monitors and cameras, and its non-contact operation mode restricts users' daily activities less, making it suitable for long-term, continuous ECG monitoring applications in home, rehabilitation center and clinical environments.

[0061] In some embodiments, step S2 includes: performing dot matrix spot encoding on each frame of the structured spot image sequence, wherein the dot matrix spot encoding is as follows: Figure 3 As shown, cross-frame tracking is performed on each encoded light spot to obtain the displacement trajectory of each light spot; the signal-to-noise ratio of the displacement trajectory of each light spot is evaluated, and the displacement trajectory of the light spot with the best signal-to-noise ratio is determined as the mechanical sequence signal.

[0062] In some embodiments, the first feature extraction branch and the second feature extraction branch are configured to encode the input through a one-dimensional residual block, input the encoded feature map into a dilated temporal convolutional network for multi-scale temporal modeling, and output a branch feature map.

[0063] In some embodiments, step S3 includes: performing low-pass filtering and band-pass filtering on the mechanical sequence signal in sequence, and performing wavelet transform on the band-pass filtered mechanical sequence signal to obtain the heart sound signal; extracting the amplitude envelope of the heart sound signal, and smoothing the extracted amplitude envelope to obtain the envelope signal.

[0064] In some embodiments, the electrocardiogram signal monitoring method of this embodiment further includes assessing the body posture of the organism based on the structured light spot image sequence; and performing an adaptive response when the assessed body posture is unfavorable. The unfavorable body posture refers to a body posture that is not conducive to the characterization of chest wall microvibrations, such as lying on one's side, severe trunk tilting, or obstruction of the target area.

[0065] In one example, evaluating the body posture of the organism based on the structured light spot image sequence includes: inputting the structured light spot image sequence into a keypoint detection network to obtain the location information of the skeleton keypoints of the organism; and obtaining the spatial distribution features of all skeleton keypoints based on the location information of the skeleton keypoints, so as to determine the body posture of the organism based on the spatial distribution features.

[0066] In one example, the adaptability response includes at least one of outputting posture adjustment cues and adjusting the ECG signal fitting strategy. The posture adjustment cues are at least one of visual, auditory, or tactile cues; the adjusting ECG signal fitting strategy includes at least one of the following strategies:

[0067] Reduce the output confidence of the fitted ECG signal;

[0068] Pause the output of the fitted ECG signal;

[0069] Update the target area used to track the dot matrix light spot;

[0070] The tracking process for the lattice structure light spot is reinitialized.

[0071] The following section uses a non-contact ECG signal fitting system based on dot-matrix laser irradiation as an example to describe the non-contact ECG signal monitoring method in detail. The system can indirectly reconstruct and fit the ECG waveform through chest wall micro-vibration information without the need for surface electrodes. It can also output posture reminders to the user when the detection conditions are unfavorable, thereby improving the availability and reliability of long-term continuous monitoring.

[0072] This system includes a workflow for dot-matrix optical measurement and acquisition, motion inversion and signal construction, electrocardiogram fitting model, posture assessment and human-computer interaction. The following details each processing step.

[0073] First, the dot matrix optical measurement signal processing procedure is executed.

[0074] This step is used to obtain information on minute displacements or deformations of the chest wall surface caused by cardiac mechanical activity under non-contact conditions.

[0075] For example, this embodiment uses a near-infrared structured light dot matrix projection device to project a static dot matrix structured light pattern onto a target area on the chest of a human body. This dot matrix structured light pattern presents as a pseudo-random dense dot matrix, forming multiple discrete light spots with high brightness and high contrast on the surface of the area to be measured, thereby constructing a set of feature points that can be stably tracked. The near-infrared structured light dot matrix projection device can use a vertical cavity surface-emitting laser (VCSEL) light source combined with a diffractive optical element (DOE) to form a high-contrast dot matrix texture. Its operating wavelength can be near-infrared (850nm), and it can meet eye safety requirements.

[0076] An industrial camera is used to acquire image sequences of structured light spots on the chest wall surface that change over time. The imaging acquisition device may include a lens, a narrowband filter, an exposure control and gain control unit, and the acquisition frame rate may be set to, for example, 200 fps.

[0077] In practical applications, to improve the deployability of the project, the imaging acquisition device includes, for example, an industrial camera, a dot matrix laser port, a camera mount, and a light-shielding structure, so that the dot matrix covers the chest ROI and the camera's field of view is stable.

[0078] Next, motion inversion and signal construction are performed.

[0079] Image processing algorithms are applied to locate the dot matrix light spots in each frame of the image, resulting in... Figure 3 The encoding results are shown. Based on the template matching motion estimation algorithm and optical flow method, trajectory estimation is performed on the encoding results of each dot matrix spot to obtain the two-dimensional coordinate trajectory of each dot matrix spot on the time axis. The signal-to-noise ratio (SNR) of the displacement trajectory of each spot is evaluated, and the displacement trajectory of the spot with the optimal SNR is determined as the mechanical sequence signal. For example, [the following is a separate sentence fragment:] Figure 3 The dot matrix light spot with a 32-bit code is used as the main vibration light spot, and its displacement trajectory is determined as a mechanical sequence signal that is strongly correlated with the cardiac cycle.

[0080] For example Figure 4 The mechanical sequence signal shown in this embodiment uses a multi-stage signal enhancement and cleaning method to generate the heart sound signal and its envelope signal for subsequent modeling.

[0081] For example, a fourth-order Butterworth low-pass filter is used to process the mechanical sequence signal, with the cutoff frequency set to 25Hz, to extract and estimate the low-frequency trend component in the signal. The low-pass filter is used to extract slow-changing trends in the signal, such as overall fluctuations caused by respiration, slight body movement, or changes in measurement conditions. Subsequently, the low-frequency component obtained from the low-pass filter is subtracted from the original signal to achieve detrending processing, effectively suppressing low-frequency drift and improving the stability of the signal on the time axis. To highlight the effective frequency band components related to heart sounds and suppress non-target frequency band noise, a second-order Butterworth bandpass filter is applied to the detrended signal, with the passband range set to 10–49.5Hz, enhancing the transient vibration characteristics of the heartbeat while reducing the influence of low-frequency residue and high-frequency random noise. Finally, Symlet4 wavelets are used to perform multi-scale decomposition and reconstruction of the bandpass signal to enhance the signal's time-frequency local expression capability and further improve the signal-to-noise ratio, thereby obtaining a more stable heart sound signal more suitable for ECG reconstruction while preserving the key transient structure. The waveform of the heart sound signal is as follows: Figure 5 As shown.

[0082] For example, to obtain a long-range structural characterization for describing the overall energy fluctuations and rhythm changes of the cardiac cycle, this embodiment constructs an envelope signal based on the heart sound signal. For instance, it obtains instantaneous amplitude information by performing absolute value operations on the heart sound signal and smooths the amplitude sequence using a sliding window moving average method, thereby forming a signal reflecting the change of heart sound energy over time. Figure 6 The envelope curve is shown. The sliding window length is set according to the sampling rate to preserve the main rhythmic structure of the cardiac cycle while suppressing high-frequency spikes and random noise.

[0083] In this embodiment, the heart sound signal is used to characterize the local vibration information related to transient events such as valve opening and closing and blood flow impact during cardiac activity. It can provide richer short-range detail features, such as local peak morphology, instantaneous energy changes and high-frequency textures, thereby improving the morphological fidelity and detail expression of the reconstructed electrocardiogram waveform.

[0084] The envelope signal is obtained by extracting and smoothing the amplitude envelope of the heart sound signal. It is used to reflect the overall trend and rhythm changes of energy fluctuations within the cardiac cycle. It can provide stable long-range structural features and periodic constraint information, including cardiac cycle rhythm, low-frequency trend and cross-cycle consistency, thereby enhancing the model's temporal stability and anti-interference ability over long time scales.

[0085] Then perform signal fitting.

[0086] This step uses a pre-trained ECG signal fitting model to fit the ECG signal. The ECG signal fitting model adopts a dual-input structure, with input signals including heart sound signals and their corresponding envelope signals. By inputting heart sound signals and envelope signals as complementary information in parallel, the model can simultaneously utilize short-range local details and long-range rhythm structure to achieve continuous reconstruction of the target ECG signal, and maintain higher robustness and consistency even in the presence of noise perturbations, amplitude fluctuations, or individual differences.

[0087] In this embodiment, the ECG signal fitting model uses an end-to-end structure to achieve ECG signal fitting. For example... Figure 7 As shown, the ECG signal fitting model first inputs the heart sound signal and its envelope signal into the corresponding feature extraction branches. In each branch, the original time series is initially encoded by a one-dimensional residual block. The one-dimensional residual block consists of multiple one-dimensional convolutions, batch normalization, and nonlinear activation, and residual connections are introduced to enhance gradient propagation capability. This allows for the extraction of more stable local morphological features while suppressing noise interference, thereby improving the model training stability and robustness to differences in individual signals.

[0088] Subsequently, the branch features are further input into a dilated temporal convolutional network for multi-scale temporal modeling. The dilated temporal convolutional network expands the temporal receptive field by setting convolutional kernels with different dilation rates, capturing contextual dependencies across multiple cardiac cycles without significantly increasing computational cost. The central tone branch focuses on extracting short-range details and local transient changes, while the envelope branch focuses on characterizing long-range rhythm structure and periodic trends, thus forming complementary short-range and long-range representations.

[0089] Next, the two features are fused in the fusion stage by gating fusion through element-wise multiplication. That is, the long-range structural information of the envelope branch is used as dynamic weights to adaptively modulate the detailed features of the heart sound branch, so as to highlight the effective components related to heartbeat and suppress non-heartbeat noise segments, thereby improving the temporal consistency and anti-interference ability in the ECG reconstruction process.

[0090] After obtaining the fusion features, the model introduces a multi-head attention mechanism to model the global temporal correlation. The multi-head attention achieves cross-time segment information interaction and key segment focus by allocating attention to different subspaces. It can enhance the cycle alignment capability and reduce the risk of waveform drift and misalignment under the conditions of heart rate changes or rhythm fluctuations.

[0091] To enhance the continuity and physiological rationality of the output waveform, the fused features can be sequentially input into a causal dilated convolutional network for temporal smoothing and structural enhancement under causal constraints, ensuring that the model relies solely on historical information to complete sequence inference, thus enabling real-time inference. At the same time, a state-space model module is introduced to efficiently model the dynamic evolution of long sequences, thereby improving the ability to express long-range dependence, slow-varying rhythms, and cross-cycle consistency, making the reconstructed ECG signal more stable and continuous over long time scales.

[0092] Finally, the model maps high-dimensional temporal features to the target output space through a fully connected layer, outputting a continuous fitted ECG signal, thus achieving end-to-end reconstruction and continuous monitoring of the ECG signal.

[0093] In addition, this system also performs posture assessment and human-computer interaction.

[0094] Furthermore, to improve the system's usability and stability under natural body position changes, this embodiment introduces a human pose estimation and detection quality evaluation mechanism based on OpenPose.

[0095] This system uses an imaging acquisition device to acquire video frames of the chest region and inputs the images into the OpenPose human keypoint detection network. It outputs the coordinates and confidence scores of key points in the human skeleton, such as those related to the shoulders, hips, and thorax. By analyzing the spatial distribution characteristics of these key points, the system can determine the subject's torso orientation and tilt, and thereby assess whether the current posture is suitable for chest wall micro-vibration measurement and electrocardiogram fitting.

[0096] For example, such as Figure 8 As shown, the system calculates the trunk's principal axis direction based on the relative positions of the left and right shoulder keypoints and the left and right hip keypoints. It further classifies body postures such as supine, lateral, and side-lying based on keypoint height differences, left-right symmetry, and keypoint confidence changes. When side-lying, severe tilting, occlusion, or persistently low keypoint confidence are detected, the system outputs a warning signal to prompt the user to adjust their posture. It can also trigger strategies such as reducing the confidence of the fitted output, pausing output, updating the ROI, or reinitializing dot matrix tracking, thereby avoiding misfitting results under low-quality input conditions.

[0097] Based on the above embodiments of this application, the technical solution of this application has achieved at least the following technical effects:

[0098] (1) By actively projecting textures using dot matrix lasers, the measurement robustness across groups and in complex environments is improved.

[0099] This application embodiment does not rely on traditional contact-based physiological electrical signal acquisition methods, but actively projects a dot matrix laser pattern to form a stable, high-contrast multi-point feature on the chest, enabling stable extraction of chest wall micromovements under different populations, lighting conditions, and background conditions.

[0100] (2) Real-time causal processing and enhanced anti-interference capabilities.

[0101] The embodiments of this application employ real-time links such as causal filtering, drift suppression, short-window normalization, and smoothing enhancement to make the mechanical sequence more stable and continuously output. It does not rely on the offline method of viewing the entire signal before recalculation, making it suitable for long-term monitoring and engineering implementation.

[0102] (3) Fitting output from mechanical sequence to ECG-like sequence.

[0103] The embodiments of this application map mechanical sequences into ECG-like waveforms (such as P-QRS-T morphological trends and R-peak rhythms) to form interpretable waveform outputs, which facilitates quality assessment, storage, and subsequent analysis.

[0104] (4) Non-contact subject posture estimation and adverse posture reminder strategies.

[0105] Figure 9 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 9 At the hardware level, the device includes a processor 902, an internal bus 904, a network interface 906, memory 908, a hardware acceleration device 910, and non-volatile memory 912, and may also include other hardware required for its functions. One or more embodiments of this application can be implemented in software, for example, the processor 902 reads the corresponding computer program from the non-volatile memory 912 into memory 908 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0106] Figure 10 This is a block diagram illustrating an exemplary embodiment of a non-contact electrocardiogram (ECG) signal monitoring device based on lattice structured light illumination. The non-contact ECG signal monitoring device can be applied to, for example... Figure 9 The electronic device shown implements the technical solution of this application. The non-contact electrocardiogram signal monitoring device includes: an image acquisition unit 1001, a signal construction unit 1002, a signal processing unit 1003, and a signal fitting unit 1004, wherein:

[0107] Image acquisition unit 1001 is used to project a dot matrix structured light spot onto a target area in front of a living organism, and to acquire a sequence of structured light spot images showing displacement changes as the surface of the target area vibrates.

[0108] The signal construction unit 1002 is used to track the dot matrix spots in the structured light spot image sequence, obtain the displacement trajectory of each spot, and construct a mechanical sequence signal related to the cardiac cycle based on the displacement trajectory.

[0109] The signal processing unit 1003 is used to process the mechanical sequence signal to obtain the heart sound signal and its corresponding envelope signal;

[0110] The signal fitting unit 1004 is used to input the heart sound signal and its corresponding envelope signal into a pre-trained electrocardiogram (ECG) signal fitting model to generate a fitted ECG signal; wherein the ECG signal fitting model is configured to process the input using a first feature extraction branch corresponding to the heart sound signal and a second feature extraction branch corresponding to the envelope signal, and then adaptively fuse the outputs of the two branches to output the fitted ECG signal.

[0111] In some embodiments, the first feature extraction branch and the second feature extraction branch are configured to encode the input through a one-dimensional residual block, input the encoded feature map into a dilated temporal convolutional network for multi-scale temporal modeling, and output a branch feature map.

[0112] In some embodiments, the signal processing unit 1003 is configured to perform low-pass filtering and band-pass filtering on the mechanical sequence signal in sequence, and perform wavelet transform on the band-pass filtered mechanical sequence signal to obtain the heart sound signal; extract the amplitude envelope of the heart sound signal, and smooth the extracted amplitude envelope to obtain the envelope signal.

[0113] In some embodiments, the non-contact electrocardiogram signal monitoring device further includes a posture assessment unit for assessing the body posture of the organism based on the structured light spot image sequence; and performing an adaptive response when the assessed body posture is unfavorable.

[0114] In some embodiments, the posture evaluation unit is used to input the structured light spot image sequence into a key point detection network to obtain the position information of the skeleton key points of the organism; based on the position information of the skeleton key points, it obtains the spatial distribution features of all skeleton key points, so as to determine the body posture of the organism based on the spatial distribution features.

[0115] In some embodiments, the signal construction unit 1002 is used to encode the dot matrix light spots in each frame of the structured light spot image sequence, and to perform cross-frame tracking on each encoded light spot to obtain the displacement trajectory of each light spot; to evaluate the signal-to-noise ratio of the displacement trajectory of each light spot, and to determine the displacement trajectory of the light spot with the best signal-to-noise ratio as the mechanical sequence signal.

[0116] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0117] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0118] Accordingly, embodiments of this application also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0119] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0120] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0121] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0122] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0123] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0124] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0125] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0126] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0127] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A non-contact electrocardiogram (ECG) signal monitoring method based on dot matrix structured light illumination, characterized in that, Includes the following steps: Step S1: Project a dot matrix structured light spot onto the target area on the chest of the organism, and collect a sequence of structured light spot images showing displacement changes as the surface of the target area vibrates. Step S2: Track the dot matrix spots in the structured light spot image sequence to obtain the displacement trajectory of each spot, and construct a mechanical sequence signal related to the cardiac cycle based on the displacement trajectory; Step S3: Process the mechanical sequence signal to obtain the heart sound signal and its corresponding envelope signal. The envelope signal is obtained by amplitude envelope extraction and smoothing of the heart sound signal and is used to reflect the overall trend and rhythm changes of energy fluctuations within the cardiac cycle. Step S4: Input the heart sound signal and its corresponding envelope signal into a pre-trained ECG signal fitting model to generate a fitted ECG signal; wherein the ECG signal fitting model is configured to process the input using a first feature extraction branch corresponding to the heart sound signal and a second feature extraction branch corresponding to the envelope signal, and then adaptively fuse the outputs of the two branches to output the fitted ECG signal.

2. The method according to claim 1, characterized in that, The first feature extraction branch and the second feature extraction branch are configured to encode the input through one-dimensional residual blocks, and input the encoded feature map into a dilated temporal convolutional network for multi-scale temporal modeling to output branch feature maps.

3. The method according to claim 1, characterized in that, Step S3 includes: The mechanical sequence signal is subjected to low-pass filtering and band-pass filtering in sequence, and wavelet transform is performed on the band-pass filtered mechanical sequence signal to obtain the heart sound signal; The amplitude envelope of the heart sound signal is extracted and smoothed to obtain the envelope signal.

4. The method according to claim 1, characterized in that, It also includes the following steps: The body shape of the organism was assessed based on the structured light spot image sequence; When the body posture is assessed as unfavorable, an adaptive response is performed.

5. The method according to claim 4, characterized in that, The assessment of the organism's morphology based on the structured light spot image sequence includes: The structured light spot image sequence is input into a key point detection network to obtain the location information of the key points of the organism's skeleton; Based on the location information of the key points of the skeleton, the spatial distribution characteristics of all key points of the skeleton are obtained, and the body shape of the organism is determined based on the spatial distribution characteristics.

6. The method according to claim 4, characterized in that, The adaptive response includes at least one of outputting posture adjustment prompts and adjusting ECG signal fitting strategies.

7. The method according to claim 1, characterized in that, Step S2 includes: Each frame of the structured spot image sequence is encoded with a dot matrix pattern, and the encoded spots are tracked across frames to obtain the displacement trajectory of each spot. The signal-to-noise ratio of the displacement trajectory of each light spot is evaluated, and the displacement trajectory of the light spot with the optimal signal-to-noise ratio is determined as the mechanical sequence signal.

8. A non-contact electrocardiogram signal monitoring device based on dot matrix structured light illumination, characterized in that, The device includes: An image acquisition unit is used to project a dot matrix structured light spot onto a target area on the chest of an organism, and to acquire a sequence of structured light spot images showing displacement changes as the surface of the target area vibrates. The signal construction unit is used to track the dot matrix spots in the structured light spot image sequence, obtain the displacement trajectory of each spot, and construct a mechanical sequence signal related to the cardiac cycle based on the displacement trajectory. The signal processing unit is used to process the mechanical sequence signal to obtain the heart sound signal and its corresponding envelope signal. The envelope signal is obtained by amplitude envelope extraction and smoothing of the heart sound signal and is used to reflect the overall trend and rhythm changes of energy fluctuations within the cardiac cycle. The signal fitting unit is used to input the heart sound signal and its corresponding envelope signal into a pre-trained electrocardiogram (ECG) signal fitting model to generate a fitted ECG signal; wherein the ECG signal fitting model is configured to process the input using a first feature extraction branch corresponding to the heart sound signal and a second feature extraction branch corresponding to the envelope signal, and then adaptively fuse the outputs of the two branches to output the fitted ECG signal.

9. An electronic device, characterized in that, include: processor; as well as A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a processor according to any one of claims 1 to 7.

Images