Thoracic cavity heart sound signal non-contact extraction system and method based on structured light

Abstract

The invention discloses a thoracic cavity heart sound signal non-contact extraction system and method based on structured light. The system comprises a structured light projection module, an image acquisition module, an image processing and displacement extraction module and a signal processing and trigger point identification module, structured light is projected on the surface of the chest of a human body to form a regular pattern, and a high-speed image acquisition means is utilized to record micro displacement change generated by light spots under cardiac drive. Thoracic cavity vibration information is extracted through an image processing technology, periodic feature points corresponding to a cardiac rhythm are recognized in combination with a time sequence analysis strategy, a thoracic cavity heart sound signal is extracted, rhythm analysis is conducted, and a cardiac trigger point is output to serve as a trigger signal for synchronous control of medical equipment. The technical problems of signal distortion, equipment interference, potential safety hazards, inconvenience in use and the like in a strong electromagnetic environment in a traditional contact mode are solved, and a high-precision synchronous trigger signal is provided for dynamic imaging.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of signal detection, and relates to a biological signal detection and medical imaging system, in particular to a non-contact extraction system and method for chest sound signals based on structured light. BACKGROUND

[0002] In the field of medical imaging, especially in cardiac magnetic resonance imaging (Cardiac MRI), the synchronous trigger signal of the cardiac cycle has a decisive significance for the temporal resolution and spatial stability of the image. The traditional synchronization method is to attach electrodes to the surface of the human body to collect electrocardiogram signals, and to realize trigger control by detecting R waves. However, the electrocardiogram signal is easily affected by external factors such as strong magnetic field, radio frequency pulse and current induction in the MRI environment, resulting in signal distortion, trigger failure or increased artifacts, which seriously reduces the image quality and acquisition efficiency. Some methods try to obtain the heart beat trigger signal through contact phonocardiogram (PCG) or acceleration cardiogram (SCG), although it partially avoids electromagnetic interference, but still depends on the sensor attached to the skin, and the signal quality is greatly affected by the attachment position, body type, skin condition, clothing coverage and other factors. Overall, this kind of contact device has obvious shortcomings in comfort, adaptability and operation convenience.

[0003] Some studies try to use non-contact optical methods such as photoplethysmography (PPG) to detect blood flow fluctuations at the fingertips, earlobes and other peripheral parts of the human body, thereby indirectly inferring the heart beat information. However, the signal quality of the optical method is highly sensitive to environmental light, skin condition and motion interference, and it is difficult to apply to the chest cavity, and cannot be used as a source of synchronous trigger signal in the MRI environment.

[0004] In summary, the existing heart beat synchronization technology generally has low comfort, poor electromagnetic compatibility, lack of robustness and other shortcomings, and there is an urgent need for a solution that is suitable for MRI scenarios, has high synchronization accuracy and can stably output heart beat trigger signals. SUMMARY

[0005] In view of the shortcomings of the prior art, the present application provides a non-contact extraction system and method for chest sound signals based on structured light, which records the changes of the ordered projection pattern formed by structured light on the surface of the human chest, extracts the chest vibration signal trajectory, obtains the heart sound-like signal, and accurately extracts the heart beat cycle trigger point, realizes a non-contact, anti-interference, highly adaptable and high-precision heart beat synchronization scheme, and solves the problems of signal interference, safety hazards and complex operation of traditional ECG and contact heart beat detection methods in the MRI environment.

[0006] A kind of chest thoracic sound signal non-contact extraction system based on structured light, including structured light projection module, image acquisition module, image processing and displacement extraction module and signal processing and trigger point identification module.

[0007] The structured light projection module is used to project the pattern with spatial coding information composed of coding units to the chest surface of the measured person.

[0008] The image acquisition module is used to acquire the image of the chest surface of the measured person, and the image sequence is formed in the order of acquisition time.

[0009] The image processing and displacement extraction module acquires the moving track of each coding unit from the image sequence, selects the analysis target by signal-to-noise ratio evaluation, extracts the displacement change sequence of the selected coding unit in the time axis from the moving track by optical flow method, generates the vibration channel signal, and takes the channel with the best signal-to-noise ratio as the main vibration signal after signal-to-noise ratio evaluation again.

[0010] The signal processing and trigger point identification module performs multi-stage signal enhancement and cleaning processing on the main vibration signal to obtain the reconstructed signal with PCG signal cycle shape, and then detects the extreme value point of the reconstructed signal to extract the heart beat trigger point.

[0011] A kind of chest thoracic sound signal non-contact extraction method based on structured light, the specific steps are as follows: Step 1, project the pattern with spatial coding information composed of coding units to the chest surface of the measured person.

[0012] Step 2, use the image acquisition device to capture the image sequence of the chest region of the measured person.

[0013] Step 3, use image recognition algorithm to locate the position of each coding unit in the image sequence, generate continuous two-dimensional coordinate track, and represent the spatial displacement process of the coding unit in the time axis. According to the average energy and fluctuation characteristics, the optimal coding unit is selected as the analysis target, the displacement change sequence of the coding unit in the two coordinate axes of the two-dimensional plane coordinate system is extracted, and the sequence with the best signal-to-noise ratio is extracted from the two sequences as the main vibration signal.

[0014] Step 4, filter and frequency domain enhance the main vibration signal, then absolute value, adjust all amplitudes to be non-negative, and finally construct the envelope to obtain the reconstructed signal. The extreme value point of the reconstructed signal is detected as the heart beat trigger point.

[0015] The application of a kind of chest thoracic sound signal non-contact extraction system based on structured light takes the heart beat trigger point output by the system as the gating signal of the imaging system.

[0016] The present application has the following beneficial effects: 1. Based on structured light spot array projection and image tracking technology, independent of electrode attachment, realizing non-contact, cable-free, and electric signal coupling-free chest cavity micro-vibration sensing, with the characteristics of complete non-contact, non-invasive, and strong anti-electromagnetic interference, providing a new scheme for heart motion sensing in strong electromagnetic environment such as MRI.

[0017] 2. Two-stage signal-to-noise ratio analysis mechanism is designed for image-level element positioning and directional displacement channel extraction, realizing dynamic selection of the optimal chest vibration signal, improving the anti-interference ability and signal quality of the system, and still having good stability under different postures of the measured object or changes in illumination, which is a key robustness guarantee means.

[0018] 3. Trend removal, frequency band filtering, wavelet decomposition, and rhythm detection methods are fused, multi-stage signal reconstruction and time sequence feature extraction are proposed, and a complete vibration signal processing path is constructed, which can effectively extract clear and consistent rhythm heart sound signal sequences, so as to stably identify the heart motion cycle trigger point, and is suitable for high-precision synchronization requirements of medical equipment.

[0019] 4. The extracted heart motion trigger point can be converted into a standard synchronization signal, which can be directly used for the trigger control interface of MRI or CT equipment, has the characteristics of high time sequence accuracy and standard signal format, can be widely compatible with mainstream medical imaging equipment, has practical landing ability and promotion value. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a schematic diagram of a chest heart sound signal non-contact extraction system based on structured light; Figure 2 is a schematic diagram of a structured light projection module; Figure 3 is a schematic diagram of the dot array spot pattern projected in embodiment 1; Figure 4 is the positioning and extraction result of the image recognition algorithm in embodiment 2; Figure 5 is the main vibration signal reconstruction result in embodiment 2; Figure 6 is the heart motion trigger point extraction result in embodiment 2. DETAILED DESCRIPTION

[0021] The present application will be further explained and described below in conjunction with the accompanying drawings; Embodiment 1 The present embodiment provides a chest heart sound signal non-contact extraction system based on structured light, as shown in Figure 1 , which includes a structured light projection module, an image acquisition module, an image processing and displacement extraction module, and a signal processing and trigger point identification module.

[0022] As shown in Figure 2 The structured light projection module is used to project a point array spot pattern with spatial coding as shown in Figure 5 When the heartbeat causes the chest surface to vibrate slightly, the pattern will shift slightly over time.

[0023] The image acquisition module is used to acquire images of the chest surface of the subject, and the images are arranged in sequence according to the acquisition time, so as to obtain the change of the spot pattern over time in real time as the basis for subsequent signal analysis.

[0024] The image processing and displacement extraction module extracts the movement trajectory of each spot from the image sequence, selects the analysis target through signal-to-noise ratio evaluation, and extracts the displacement change sequence of the selected spot on the time axis from the movement trajectory through the optical flow method, to generate a vibration channel signal. After signal-to-noise ratio evaluation again, the channel with the best signal-to-noise ratio is taken as the main vibration signal to ensure the stability of the signal and the expression effect of the heartbeat characteristics.

[0025] The signal processing and trigger point identification module performs multi-stage signal enhancement and cleaning processing on the main vibration signal, including trend removal, frequency band extraction, feature enhancement and noise suppression, to obtain a reconstructed signal with clear morphology and PCG-like signal periodic morphology. Then, the reconstructed signal is analyzed through a sliding window, and the extreme value points are detected to extract the heartbeat trigger points.

[0026] Embodiment 2 This embodiment cooperates with the extraction system described in Embodiment 1 to provide a non-contact extraction method of chest heart sound signals based on structured light, and the specific steps are as follows: Step 1, project a point array spot pattern with spatial coding to the chest surface of the subject, as shown in Figure 3 The regular reference spot distribution is formed on the chest surface of the subject. When the heartbeat drives the chest to produce slight fluctuations, the spots will move slightly with the vertical and horizontal displacement of the skin.

[0027] As an optional embodiment, a stripe, grid, random speckle or circular ring array image with spatial coding is projected to the chest surface of the subject.

[0028] Step 2, use a high-speed industrial camera to vertically align the chest region of the subject, and acquire an image sequence of the chest region of the subject at a frame rate of 200 fps to capture high-frequency detail motion for subsequent vibration channel signal extraction.

[0029] As an optional embodiment, a near-infrared camera, ToF depth camera or laser three-dimensional camera is used to acquire images of the chest surface of the subject.

[0030] Step 3, collect the moving track of each spot from the image sequence, extract the main vibration signal, the specific steps are as follows: s3.1, using the image recognition algorithm based on circular template matching (Circular Template Matching), the dot spot of each frame image in the sequence is positioned, extracted and numbered, as shown in Figure 4 . Extract the continuous two-dimensional coordinate track of each spot in the sequence, representing the spatial displacement process of the spot on the time axis. Through gray distribution fitting and edge consistency analysis, sub-pixel level spot positioning can be realized to ensure the accuracy of subsequent motion extraction.

[0031] s3.2, in order to improve the analysis efficiency and signal stability, the first layer signal-to-noise ratio of the spot extracted in s3.1 is evaluated, the average energy and fluctuation characteristics of each spot on its motion track are calculated, and the spot with the highest signal-to-noise ratio is selected as the analysis target to avoid interference from local shielding or reflection distortion.

[0032] s3.3, for the spot selected in s3.2, apply dense optical flow method in two directions of two-dimensional coordinate system respectively, extract its displacement change sequence on the time axis from the motion track, generate two original vibration channel signals, respectively corresponding to the mechanical motion response of the chest in two directions.

[0033] As an optional embodiment, block matching method, phase correlation method, template matching or deep learning optical flow network is used to extract the displacement change sequence of the spot on the time axis from the motion track.

[0034] s3.4, the original vibration channel signal obtained in s3.3 is subjected to second layer signal-to-noise ratio evaluation, the periodicity, stability and heart feature retention degree of the two original vibration channel signals are compared respectively, and the channel with the best signal-to-noise ratio is selected as the main vibration signal, as shown in Figure 5 . Through double layer signal-to-noise ratio evaluation, the reliability and anti-interference ability of the extracted heart trigger point can be significantly enhanced, especially in the scene where the shooting condition is not ideal or the posture of the measured person is not standard, the advantage is more obvious.

[0035] As an optional embodiment, energy spectrum, mutual information or statistical feature index is used to select the spot and the original vibration channel signal.

[0036] Step 4, reconstruct the main vibration signal and extract the heart trigger point from it, the specific steps are as follows: s4.1, the main vibration signal is filtered by a 4th order Butterworth low-pass filter, the cut-off frequency is set to 5 Hz, and the baseline drift and low-frequency respiratory signal are removed. Then a 2nd order Butterworth band-pass filter is used, the passband range is set to 10-50 Hz, the main frequency band content of the heart is extracted, and the posture, muscle or device noise is effectively suppressed.

[0037] s4.2, the filtered signal is decomposed by a Symlet-4 (sym4) wavelet function, and the main scale coefficient is selected for signal reconstruction, so as to enhance the edge definition and mutation characteristics of the heart signal, and realize noise reduction, and obtain a stable heart sound-like signal.

[0038] As an optional embodiment, the filtered signal is subjected to frequency domain enhancement by empirical mode decomposition (EMD) or short-time Fourier transform (STFT).

[0039] s4.3, the amplitude is adjusted to be non-negative by further signal absolute valueization, and the ability change curve is emphasized. Then a sliding average window is used to further suppress the small wave of the reconstructed signal, and the reconstructed signal as shown in Figure 5 has a periodic pattern similar to PCG, and S1 and S2 heart sounds correspond to obvious fluctuation, which is convenient for subsequent peak and valley detection and timing analysis.

[0040] s4.4, in a sliding window with a length of 5 seconds, a first derivative method is applied to detect the extreme points of the reconstructed signal, and the heart beat trigger points are obtained, as shown in Figure 6 .

[0041] As an optional embodiment, further judgment is made on the detected extreme points: ①Judge the position of the extreme point, and delete the extreme point outside the delay preset interval after the heart sound wave valley mark point.

[0042] ②Judge the interval of the extreme point, set the minimum trigger interval threshold to 0.5 seconds, delete the point less than the trigger interval threshold from the last extreme point; and the inter-beat interval (IBI) of the continuous candidate trigger point needs to meet the tolerance standard that the cycle variance is less than 10%.

[0043] Through time delay constraint, heart cycle consistency judgment and minimum trigger interval control, false peak interference is excluded, and the output heart beat trigger point has high physiological correlation, rhythm regularity and signal stability.

[0044] Embodiment 3 The chest heart sound signal non-contact extraction system based on structured light described in Embodiment 1 is integrated with an MRI system, the heart beat trigger point output by the chest heart sound signal non-contact extraction system based on structured light is converted into a digital output signal, which is used as a synchronous trigger signal of cardiac magnetic resonance imaging, and synchronous acquisition control of the MRI system is realized.

[0045] The signal stability and accuracy output by the chest heart sound signal non-contact extraction system based on structured light can reach millisecond level precision, and precise coupling with an MRI scanning sequence can be realized, thereby supporting advanced functions such as cardiac gating acquisition and real-time dynamic image acquisition.

[0046] Embodiment 4 The chest heart sound signal non-contact extraction system based on structured light described in Embodiment 1 is integrated with a CT, ultrasound, electroencephalogram or other cardiac gating acquisition system, the heart beat trigger point output by the chest heart sound signal non-contact extraction system based on structured light is converted into a digital output signal, which is used as a gating signal of the imaging system. Or used for remote vital sign monitoring, non-invasive physiological parameter analysis, cardiopulmonary rehabilitation evaluation, sports medicine evaluation, sleep state detection, infant health monitoring and the like.

Claims

1. A non-contact chest heart sound signal extraction system based on structured light, characterized by: It includes structured light projection module, image acquisition module, image processing and displacement extraction module and signal processing and trigger point recognition module; The structured light projection module is used to project a pattern composed of coding units and having spatial coding information onto the chest surface of the subject; The image acquisition module is used to acquire images of the chest surface of the subject and form an image sequence in the order of acquisition time; The image processing and displacement extraction module collects the movement trajectory of each coding unit from the image sequence, selects the analysis target through signal-to-noise ratio evaluation, and then extracts the displacement change sequence of the selected coding unit on the time axis from the movement trajectory using the optical flow method to generate a vibration channel signal. After further signal-to-noise ratio evaluation, the channel with the best signal-to-noise ratio is selected as the main vibration signal. The signal processing and trigger point identification module performs multi-stage signal enhancement and cleaning processing on the main vibration signal to obtain a reconstructed signal with a periodic morphology similar to a PCG signal, and then performs extreme point detection on the reconstructed signal to extract cardiac trigger points.

2. The application of the structured light-based non-contact chest heart sound signal extraction system as claimed in claim 1, characterized in that: The non-contact extraction system is integrated with the MRI system, and the cardiac trigger points output by the image processing and displacement extraction module are converted into digital output signals as synchronous trigger signals for cardiac magnetic resonance imaging, thereby realizing synchronous acquisition control of the MRI system.

3. The application of the structured light-based non-contact chest heart sound signal extraction system as claimed in claim 1, characterized in that: The non-contact extraction system is integrated with CT, ultrasound, EEG or other cardiac gating acquisition systems, and the cardiac trigger points output by the image processing and displacement extraction module are converted into digital output signals as gating signals for the imaging system; or used for remote vital sign monitoring, non-invasive physiological parameter analysis, cardiopulmonary rehabilitation assessment, sports medicine assessment, sleep state detection, and infant health monitoring.

4. A non-contact method for extracting chest heart sound signals based on structured light, characterized by: The specific steps are as follows: Step 1: Projecting a pattern composed of coding units and containing spatial coding information onto the chest surface of the subject; Step 2: using an image acquisition device to capture an image sequence of the chest area of ​​the subject; Step 3: Use an image recognition algorithm to locate the position of each coding unit in the image sequence and generate a continuous two-dimensional coordinate trajectory. Based on the average energy and fluctuation characteristics, select the optimal coding unit as the analysis target, extract the displacement change sequence of the analysis target in the two coordinate axis directions of the two-dimensional plane coordinate system, and then extract the sequence with the best signal-to-noise ratio from the two sequences as the main vibration signal. Step 4: Filter and enhance the frequency domain of the main vibration signal, then perform absolute value conversion, adjust all amplitudes to be non-negative, and finally construct the envelope to obtain the reconstructed signal; detect the extreme points of the reconstructed signal as the cardiac trigger point.

5. The method for non-contact extraction of chest heart sound signals based on structured light according to claim 4, characterized in that: Projecting spots, stripes, grids, random speckles or circular array images with spatially coded information onto the chest surface of the subject.

6. The method for non-contact extraction of chest heart sound signals based on structured light according to claim 4, characterized in that: The specific method for extracting the main vibration signal in step 3 is: s3.

1. Use an image recognition algorithm to locate, extract, and number each coding unit in each frame of the image sequence, and extract the continuous two-dimensional coordinate trajectory of each coding unit in the sequence; S3.

2. Perform a first-level signal-to-noise ratio evaluation on the code units extracted in S3.

1. Calculate the average energy and fluctuation characteristics of each code unit along its motion trajectory, and select the code unit with the highest signal-to-noise ratio as the analysis target. S3.

3. For the analysis target selected in S3.2, extract its displacement change sequence on the time axis in two directions of the two-dimensional coordinate system to generate two original vibration channel signals, corresponding to the mechanical motion response of the chest cavity in two directions respectively; s3.

4. Perform a second-level signal-to-noise ratio evaluation on the original vibration channel signal obtained in s3.

3. Compare the periodicity, stability, and cardiac feature retention of the two original vibration channel signals, and select the channel with the best signal-to-noise ratio as the main vibration signal.

7. The method for non-contact extraction of chest heart sound signals based on structured light according to claim 6, characterized in that: Energy spectrum, mutual information or statistical characteristic indicators are used to select analysis targets and main vibration signals.

8. The method for non-contact extraction of chest heart sound signals based on structured light according to claim 4, characterized in that: The specific method to obtain the reconstructed signal is: s4.

1. Filter the main vibration signal using a low-pass filter with a cutoff frequency of 5 Hz to remove baseline drift and low-frequency respiratory signals. Then, use a band-pass filter with a passband of 10–50 Hz to extract the main cardiac frequency band. s4.

2. Use the wavelet function to perform multi-scale wavelet decomposition on the filtered signal and select the main scale coefficient to reconstruct the signal; s4.

3. Absolute value conversion of the signal adjusts the signal amplitude to be non-negative, and then uses a sliding average window to further suppress small fluctuations to obtain the reconstructed signal.

9. The method for non-contact extraction of chest heart sound signals based on structured light according to claim 8, characterized in that: The frequency domain enhancement is achieved by using empirical mode decomposition or short-time Fourier transform on the filtered signal.

10. The method for non-contact extraction of chest heart sound signals based on structured light according to claim 4 or 8, characterized in that: Through time delay constraints, cardiac cycle consistency judgment and minimum trigger interval control, the detected extreme points are further screened: ① Determine the location of the extreme points and delete the extreme points outside the delay preset interval after the heart sound wave valley mark point; ② Determine the interval between extreme points, set the minimum trigger interval threshold to 0.5 seconds, and delete points whose distance from the previous extreme point is less than the trigger interval threshold; In addition, the heartbeat intervals of consecutive candidate trigger points must meet the tolerance standard of cycle variance less than 10%; The retained extreme value points are used as cardiac trigger points.

Images