A non-contact heart sound monitoring method, system, terminal and medium

Abstract

The present invention provides a non-contact heart sound monitoring method, system, terminal and medium, which specifically relate to the field of biomedical engineering technology. The scheme includes: using lasers of different bands to perform multi-point synchronous monitoring of the heart to obtain a defocused speckle image; performing signal extraction on the defocused speckle image to obtain a heart motion signal corresponding to the multi-band; and filtering the heart motion signal corresponding to the multi-band to obtain a heart sound signal. The scheme utilizes the characteristics of multi-band laser defocused independent interference imaging, and uses lasers of different bands to perform multi-point synchronous monitoring of the tiny movements of the heart position in the chest cavity, which can realize non-contact monitoring of heart sounds, which can not only reduce the requirements for the measurement environment noise while ensuring the detection accuracy, but also help to improve the comfort of the target living body.

Description

Technical Field

[0001] The present invention relates to the field of biomedical engineering technology, and in particular to a non-contact heart sound monitoring method, system, terminal and medium. Background Art

[0002] Heart sounds are the sounds produced by mechanical wave phenomena caused by myocardial contraction, heart valve closure, and blood hitting the ventricular wall and aorta wall. Phonocardiogram (PCG) is a graph that converts the vibration of heart sounds into a time series of vibration wave records. Since phonocardiogram can provide important information about heart valve function and cardiac hemodynamics, cardiovascular diseases and defects usually lead to changes in heart sounds or abnormal murmurs, so the use of phonocardiogram has great potential in assisting the detection of various heart diseases.

[0003] In the prior art, the detection of phonocardiogram is mainly based on the contact method, that is, by placing a sensor or a stethoscope on a specific part of the patient's chest to obtain the heart sound signal. The collection positions of heart sound usually include the apex area, the left edge of the sternum, and the right edge of the sternum. If the sensor or stethoscope is not placed accurately, the collected heart sound signal may be distorted, and other body sounds and external environmental noise may also interfere with the heart sound signal, which may easily lead to insufficient monitoring accuracy of the phonocardiogram. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, an object of the present invention is to provide a non-contact heart sound monitoring method, system, terminal and medium, aiming to solve the problem of low heart sound monitoring accuracy caused by noise interference in the contact heart sound detection method in the prior art.

[0005] In order to achieve the above object, the first aspect of the present invention provides a non-contact heart sound monitoring method, comprising:

[0006] Use lasers of different wavelengths to monitor the heart at multiple points synchronously and obtain defocused speckle images;

[0007] Performing signal extraction on the defocused speckle image to obtain cardiac motion signals corresponding to multiple bands;

[0008] A heart sound signal is obtained based on the heart motion signals corresponding to the multiple bands.

[0009] Optionally, the method of using lasers of different wavelength bands to synchronously monitor the heart at multiple points to obtain a defocused speckle image includes:

[0010] Use lasers of different wavelengths to illuminate different locations of the heart and determine the target point corresponding to each wavelength.

[0011] Based on the target points corresponding to the laser in each wavelength band, the heart is monitored synchronously at multiple points to obtain a defocused speckle image.

[0012] Optionally, performing signal extraction on the defocused speckle image to obtain cardiac motion signals corresponding to multiple bands includes:

[0013] Using an optical flow algorithm to extract signals from the defocused speckle image to obtain cardiac motion features corresponding to each band;

[0014] Based on the cardiac motion features corresponding to all bands, cardiac motion signals corresponding to multiple bands are obtained.

[0015] Optionally, obtaining a heart sound signal based on the heart motion signal corresponding to the multi-bands includes:

[0016] Using high-pass filtering to filter the cardiac motion signals corresponding to the multi-bands to obtain motion signals related to the generation of heart sounds;

[0017] A heart sound signal is obtained based on the motion signal related to the generation of heart sounds.

[0018] Optionally, after obtaining the heart sound signal, including:

[0019] fusing and denoising the heart sound signal to obtain heart sound features;

[0020] Based on the heart sound features, a first heart sound signal and a second heart sound signal are extracted.

[0021] Optionally, after extracting the first heart sound signal and the second heart sound signal based on the heart sound feature, the method further includes:

[0022] Based on the first heart sound signal and the second heart sound signal, a first heart sound phase cycle and a second heart sound phase cycle are determined.

[0023] A second aspect of the present invention provides a non-contact heart sound monitoring device, the device comprising:

[0024] The defocused speckle image acquisition module is used to perform multi-point synchronous monitoring of the heart using lasers of different bands to obtain a defocused speckle image; the cardiac motion signal extraction module is used to extract signals from the defocused speckle image to obtain cardiac motion signals corresponding to multiple bands; and the cardiac sound signal generation module is used to obtain a cardiac sound signal based on the cardiac motion signals corresponding to the multiple bands.

[0025] Optionally, the defocused speckle image acquisition module includes a laser and a camera, wherein the laser is used to emit lasers of different wavelength bands and irradiate different positions of the heart respectively; and the camera is used to simultaneously acquire defocused speckle images corresponding to different positions of the heart through a sensor.

[0026] A third aspect of the present invention provides a terminal, which includes a memory, a processor, and a non-contact heart sound monitoring program stored in the memory and executable on the processor, wherein the non-contact heart sound monitoring program, when executed by the processor, implements any one of the steps of the above-mentioned non-contact heart sound monitoring method.

[0027] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a non-contact heart sound monitoring program, which, when executed by a processor, implements any one of the steps of the above-mentioned non-contact heart sound monitoring method.

[0028] Compared with the prior art, the beneficial effects of this solution are as follows:

[0029] The present invention adopts multi-band laser defocused independent interference imaging technology, uses lasers of different bands to perform multi-point synchronous monitoring of the heart, obtains defocused speckle images, realizes accurate monitoring of tiny movements of the heart position, and exchanges band redundancy for spatial redundancy of monitoring the heart position, thereby obtaining multi-dimensional blood flow dynamic information; extracts the heart motion signal corresponding to each band from the defocused speckle image, thereby obtaining a heart sound signal. This method makes full use of the characteristics of multi-band laser defocused independent interference imaging, performs multi-point synchronous monitoring of tiny movements of the heart position in the chest cavity by lasers of different bands, has the characteristics of high sensitivity and non-contact, and can realize contactless monitoring of heart sounds; and can obtain more dimensional information through laser imaging of different bands, which can not only reduce the requirements for measurement environment noise while ensuring detection accuracy, but also help improve the comfort of the target living body. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0031] Figure 1 is a flow chart of the non-contact heart sound monitoring method of the present invention;

[0032] Figure 2 A waveform diagram of a cardiac motion signal obtained by the method of the present invention;

[0033] Figure 3 It is a multi-cycle comparison waveform diagram of the cardiac motion signal and the phonocardiogram signal in the same period of the present invention;

[0034] Figure 4It is a single-cycle comparison waveform diagram of the cardiac motion signal and the phonocardiogram signal in the same period of the present invention;

[0035] Figure 5 It is a schematic diagram of the terminal structure of the present invention. DETAILED DESCRIPTION

[0036] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0037] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0038] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0039] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0040] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0041] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0043] In view of the problem of insufficient detection accuracy of phonocardiogram in the prior art, the present invention proposes a non-contact phonocardiogram monitoring method, which mainly combines defocused speckle imaging technology and signal processing technology, and utilizes the characteristics of multi-band laser defocused independent interference imaging to achieve synchronous monitoring of micro-motions of the heart surface at multiple positions in the chest cavity, extract cardiac motion signals related to the generation of phonocardiograms, and obtain motion signals related to the generation of phonocardiograms after processing the cardiac motion signals using high-pass filtering, thereby obtaining phonocardiogram signals. The method of the present invention can realize non-contact monitoring of phonocardiograms, which can not only reduce the requirements for measurement environment noise while ensuring detection accuracy, but also help improve the comfort of the target living body.

[0044] An embodiment of the present invention provides a non-contact cardiac sound monitoring device for implementing each step of the above-mentioned non-contact cardiac sound monitoring method, the device mainly includes: a defocused speckle image acquisition module, a cardiac motion signal extraction module and a cardiac sound signal generation module, wherein the defocused speckle image acquisition module is used to use lasers of different bands to perform multi-point synchronous monitoring of the heart to obtain a defocused speckle image. The defocused speckle image acquisition module includes a laser and a camera, wherein the laser is used to emit lasers of different bands and irradiate different positions of the heart respectively; the camera is provided with a number of sensors and firmware processing chips, which are used to simultaneously acquire defocused speckle images corresponding to different positions of the heart through sensors. In the process of collecting defocused speckle images, a multi-band laser is first used to emit laser beams of different bands to irradiate the heart area of ​​the target living body, and then the defocus degree of the camera is controlled by adjusting the distance between the focal plane and the lens to adjust the distance between the focal plane (far object distance) and the heart area, so that the laser beam is irradiated to different target points in the heart area in the chest cavity after defocusing to realize multi-band laser defocused independent interference imaging, so that the monitored interference imaging obtains speckles with larger particle size and speckle motion with higher sensitivity, and finally the camera is used to shoot all target points to synchronously receive the reflected laser signal from the heart position to obtain a defocused speckle image containing motion signals of different dimensions of the heart, so as to facilitate the subsequent acquisition of cardiac motion signals based on the defocused speckle image. The cardiac motion signal extraction module is used to extract signals from the defocused speckle image to obtain cardiac motion signals corresponding to multiple bands. The cardiac sound signal generation module is used to process the cardiac motion signal corresponding to each band to obtain a cardiac sound signal.

[0045] It should be stated that this embodiment does not impose any specific restrictions on the types of lasers and cameras. Any laser that can generate stable multi-band lasers with sufficient power is within the protection scope of the present invention, and any camera configuration that can clearly capture slight changes in the defocused speckle image is within the protection scope of the present invention.

[0046] Based on the above-mentioned non-contact heart sound monitoring device, the non-contact heart sound monitoring method provided by the embodiment of the present invention is applied to the heart sound monitoring scenario, and is aimed at the situation of non-contact heart sound monitoring. Specifically, Figure 1 As shown, the steps of the method in this embodiment include:

[0047] Step S100: using lasers of different wavelength bands to perform multi-point synchronous monitoring of the heart to obtain a defocused speckle image.

[0048] Specifically, considering that the speckle image in the defocused state is more sensitive to tiny changes in blood flow, when the camera is in the defocused state, the interference speckle formed in the video after the laser irradiates and reflects on the surface of the target living body presents a granular shape with alternating light and dark and strong contrast. In addition, since lasers of different bands have different penetration depths and scattering characteristics, the weak movement of the chest cavity caused by the vibration of the heart will change the reflection direction of the laser on its surface, thereby causing the spatial distribution of the speckle interference fringes in the reflected light field to change, forming a defocused speckle image. Based on this, this embodiment determines multiple target points for the positions corresponding to different layers and structures of the heart, uses lasers of different bands to perform multi-point synchronous monitoring of the heart, uses a camera in a defocused state to shoot a video stream within a preset time period formed by the reflected laser signals of lasers of different bands irradiating at different target points, and extracts a number of defocused speckle images from the video stream to obtain rich blood flow dynamic information. Among them, the preset time period refers to a certain time period when the target living body is in a static state. Collecting the video stream within this time period can improve the accuracy of the monitored heart motion signal. This embodiment adopts multi-band laser defocus independent interference imaging technology, which measures the rich blood flow dynamic information in living tissue in a non-contact manner based on the speckle pattern generated by laser irradiation on the surface of heart tissue, which is conducive to the subsequent accurate extraction of motion signals at different points of the living heart. This solution has the advantages of high resolution, high sensitivity and non-contact.

[0049] Step S200: performing signal extraction on the defocused speckle image to obtain cardiac motion signals corresponding to multiple bands.

[0050] Specifically, since the brightness and darkness changes of pixels in the defocused speckle image are closely related to the blood flow state in the heart tissue, the changes in pixel values ​​can also reflect the blood flow dynamic information at different locations in the heart region collected by lasers of different bands. Therefore, this embodiment obtains the cardiac motion signals corresponding to the heart in different bands by performing signal extraction on the defocused speckle image. The specific process is as follows: since the speckle contrast can be used to measure the fluctuation intensity of the pixel value of each pixel in the speckle image, the speckle contrast image is obtained by calculating the local contrast of each pixel, and the faster the blood flow speed, the lower the speckle contrast. Then, the speckle contrast image of each band is subjected to time series analysis to extract the speckle contrast signal that changes with time, thereby obtaining the cardiac motion signal corresponding to the multi-band, and the cardiac motion signal includes signal characteristics such as heart rate, cardiac interval, and amplitude.

[0051] Furthermore, before extracting the signal from the defocused speckle image, the defocused speckle image can be preprocessed, for example, by using Gaussian filtering, median filtering and other methods to remove noise in the image and improve the image quality; by using image processing techniques such as contrast enhancement and sharpening to improve the visibility and clarity of the speckle image; by image registration to ensure the spatial consistency between images of different bands, etc.

[0052] Step S300: obtaining a heart sound signal based on the heart motion signal corresponding to the multi-bands.

[0053] Specifically, this embodiment selects a suitable filter type and sets corresponding parameters, such as cutoff frequency, filter order, etc., according to the frequency characteristics and noise characteristics of the heart sound signal; the set filter is used to filter the heart motion signal corresponding to each band to obtain a filtered signal; and the filtered signal is extracted to obtain a heart sound signal.

[0054] In this embodiment, the multi-band laser defocus independent interference imaging technology is used to non-contactly collect the defocused speckle image formed by the reflected laser signal of the multi-band irradiation at different target points, so as to realize the accurate monitoring of the micro-movement of the heart position, and the spatial redundancy of the monitoring heart position is exchanged for the band redundancy, so as to obtain multi-dimensional blood flow dynamic information; based on the correlation between the light and dark changes of pixels in the defocused speckle image and the blood flow state in the heart tissue, the heart motion signal corresponding to each band is extracted from the defocused speckle image, and the heart sound signal is obtained by filtering. Since the heart sound signal obtained in this embodiment contains the key signal features for characterizing the heart motion in the phonocardiogram signal, the heart sound signal is regarded as a PCG-like signal. It can be seen that this embodiment makes full use of the characteristics of multi-band laser defocus independent interference imaging, and performs multi-point synchronous monitoring of the micro-movement of the heart position in the chest cavity by lasers of different bands, has the characteristics of high sensitivity and non-contact, and can obtain more dimensional information through laser imaging of different bands, can realize non-contact monitoring of heart sounds, and can reduce the requirements for the measurement environment noise while ensuring the detection accuracy, which is conducive to improving the comfort of the target living body.

[0055] In a preferred embodiment, the step S100 of performing multi-point synchronous monitoring of the heart using lasers of different wavelength bands to obtain a defocused speckle image includes:

[0056] Step S110: using lasers of different wavelengths to irradiate different positions of the heart respectively, and determining the target point corresponding to the laser of each wavelength;

[0057] Step S120: Based on the target points corresponding to the laser beams in each wavelength band, the heart is monitored synchronously at multiple points to obtain a defocused speckle image.

[0058] Specifically, since lasers of different wavelengths have different penetration abilities and scattering characteristics in cardiac tissue, this embodiment first determines the target points corresponding to lasers of each wavelength for the positions corresponding to different layers and structures of the heart, and the target points include but are not limited to key positions such as coronary arteries, ventricular walls, myocardium, etc., so as to achieve comprehensive monitoring of structures at different depths of the heart. Then, lasers of different wavelengths are used to perform multi-point synchronous monitoring of the heart, which is based on the principle of multi-band laser defocus independent interference imaging, and lasers of different wavelengths are used to irradiate different target points in the heart area for synchronous monitoring, and the reflected laser signals of the heart positions corresponding to all target points are synchronously received by the camera to obtain a defocused speckle image containing motion signals of the heart in different dimensions. During the synchronous monitoring process, laser light waves of different wavelengths will be scattered by different structures (such as blood cells) in the cardiac tissue to form a defocused speckle image with uneven brightness. The brightness changes of pixels in the defocused speckle image are closely related to the blood flow state in the cardiac tissue, and the changes in pixel values ​​can also reflect the blood flow dynamic information at different sites in the heart area collected by lasers of different wavelengths. Among them, multi-band laser defocus-independent interferometric imaging refers to the use of coherence and interference phenomena of lasers in different bands for imaging to obtain multi-dimensional motion information of the target living heart, which can improve the accuracy and sensitivity of monitoring.

[0059] In a preferred embodiment, the step S200 of performing signal extraction on the defocused speckle image to obtain cardiac motion signals corresponding to multiple bands includes:

[0060] Step S210: extracting signals from the defocused speckle image using an optical flow algorithm to obtain cardiac motion features corresponding to each band;

[0061] Step S220: based on the cardiac motion features corresponding to all the bands, obtaining cardiac motion signals corresponding to multiple bands.

[0062] Specifically, the optical flow algorithm is a method for detecting pixel motion information in an image sequence, which mainly analyzes the change of pixel intensity between adjacent frames in the image over time and calculates the motion information such as the pixel's motion direction and speed. In cardiac motion monitoring, by comprehensively considering the characteristics of the image sequence and the requirements of computing resources and computing accuracy, a suitable optical flow algorithm (such as Lucas-Kanade algorithm, Horn-Schunck algorithm, etc.) is selected, and the window size, smoothness and other parameters are set to analyze the defocused speckle image sequence, calculate the speckle image changes caused by cardiac motion, so as to capture the motion characteristics of the heart in different bands, and convert these characteristics into time series signals, that is, to obtain the cardiac motion signal corresponding to each band. Since the laser irradiation position and depth of different bands may be different, the extracted cardiac motion signals are aligned through time synchronization and spatial registration, and the aligned cardiac motion signals are fused to obtain the cardiac motion signals corresponding to the multi-bands. In this embodiment, the optical flow algorithm is used to extract the defocused speckle image, which can accurately identify the change of pixel intensity over time, thereby facilitating the improvement of the accuracy of the obtained cardiac motion signals corresponding to the multi-bands.

[0063] Furthermore, by analyzing the fused cardiac motion signal, various parameters of cardiac motion, such as heart rate, heart beat interval, ventricular contraction and relaxation speed, etc., can be extracted.

[0064] In a preferred embodiment, obtaining the heart sound signal based on the heart motion signal corresponding to the multi-bands in step S300 includes:

[0065] Step S310: filtering the cardiac motion signals corresponding to the multi-bands by high-pass filtering to obtain motion signals related to the generation of heart sounds;

[0066] Step S320: obtaining a heart sound signal based on the motion signal related to the heart sound generation.

[0067] Specifically, after a lot of research and experiments, it is known that frequencies above 50HZ in the speckle motion signal are related to the activity of the valve, and the movement of the valve is closely related to heart sounds. By high-pass filtering the speckle motion signal, a signal highly similar to the phonocardiogram can be obtained, thereby extracting the heart sound information.

[0068] Therefore, this embodiment designs a high-pass filter according to the frequency characteristics of the heart valve motion signal, uses the high-pass filter to filter out signals below the preset cutoff frequency, and retains signals above the preset cutoff frequency (such as 50HZ) to obtain a filtered signal; then extracts the motion signal related to the generation of heart sounds in the filtered signal, including physiological signal characteristics related to heart valve motion and physiological signal characteristics related to myocardial contraction. Among them, the physiological signal characteristics related to heart valve motion include the opening and closing time of the valve, as well as the signal changes and motion speed related to the heart valve motion; the physiological signal characteristics related to myocardial contraction include the intensity and speed of myocardial contraction. It is easy to understand that since the heart movement is carried out synchronously in multiple bands, it is necessary to ensure that the heart valve motion signals corresponding to each band extracted are synchronized.

[0069] In this embodiment, by processing the cardiac motion signal using high-pass filtering, high-frequency components of the cardiac motion signal corresponding to multiple bands can be retained, thereby obtaining a motion signal with waveform characteristics similar to a phonocardiogram.

[0070] In a preferred embodiment, after obtaining the heart sound signal in step S300, step S400 is included, specifically:

[0071] Step S410: fusing and denoising the heart sound signal to obtain heart sound features;

[0072] Step S420: extracting a first heart sound signal and a second heart sound signal based on the heart sound characteristics.

[0073] Specifically, in order to improve the quality of the signal, this embodiment uses a noise reduction method to perform noise reduction processing on the heart sound signal to obtain a noise-reduced heart sound signal. The noise reduction method can be based on a frequency domain filtering method such as wavelet transform and Fourier transform, or a time-frequency analysis method such as empirical mode decomposition and variational mode decomposition, or an adaptive filtering method, a Kalman filtering method, or the like.

[0074] Then, the denoised heart sound signal is analyzed in the time domain to extract time domain features such as heart sound duration, heart rate, and heart beat interval; the heart sound signal is analyzed in the frequency domain to extract frequency domain features such as energy distribution and main frequency of each frequency band, which helps to identify different components in the heart sound, such as the first heart sound and the second heart sound; the heart sound signal is subjected to envelope detection to extract the envelope features of the heart sound to reflect the amplitude changes of the heart sound, which helps to identify the starting and ending positions of the heart sound.

[0075] Based on the extracted time domain features, frequency domain features and envelope features related to heart sounds, feature matching is performed on the heart sound signal to identify the key feature point positions of the first heart sound (S1) and the second heart sound (S2) in the heart sound signal; based on the key feature point positions, clear and independent first heart sound (S1) and second heart sound (S2) signals are separated from the heart sound signal, where S1 and S2 correspond to the sound information during heart contraction and diastole, respectively.

[0076] Further, after extracting the first heart sound signal and the second heart sound signal based on the heart sound feature in step S420, the method further includes:

[0077] Based on the first heart sound signal and the second heart sound signal, a first heart sound phase cycle and a second heart sound phase cycle are determined.

[0078] Specifically, this embodiment uses Hilbert transform or support vector machine algorithm to determine the first heart sound phase cycle (ie, S1 cycle) and the second heart sound phase cycle (ie, S2 cycle) based on the first heart sound signal (S1) and the second heart sound signal (S2).

[0079] In a preferred example, the Hilbert transform is a linear transform used to transform a real signal into the imaginary part of its analytical signal. The analytical signal is a complex signal, whose real part is the original signal and whose imaginary part is the Hilbert transform of the original signal. Through the Hilbert transform, the instantaneous amplitude, instantaneous frequency and instantaneous phase of the signal can be obtained. The process of processing the heart sound signal using the Hilbert transform includes: performing Hilbert transform on the first heart sound signal (S1) and the second heart sound signal (S2) respectively to obtain the S1 analytical signal and the S2 analytical signal; using the instantaneous phase information of the S1 analytical signal and the S2 analytical signal to determine the starting point and the end point of S1 and S2 respectively; calculating the time interval between two adjacent S1 (or S2) to obtain the S1 cycle (or S2 cycle). The Hilbert transform method used in this example is simple and intuitive, and can accurately extract the phase information of the heart sound signal, and thus can accurately determine the S1 cycle and the S2 cycle.

[0080] In another preferred example, the support vector machine algorithm is a machine learning algorithm based on statistical learning theory, which is mainly used to distinguish different categories of data points by finding an optimal hyperplane, while maximizing the distance (i.e., interval) between the two categories of data points and the hyperplane. The process of processing heart sound signals using the support vector machine algorithm includes: extracting features from the S1 and S2 signals respectively to obtain time domain features, frequency domain features, and / or envelope features; using these features to train a support vector machine classifier to distinguish S1 and S2 signals, and determine the starting and ending points of S1 and S2; calculating the time interval between two adjacent S1 (or S2) to obtain the S1 cycle (or S2 cycle). The support vector machine algorithm used in this example has a strong generalization ability and can ensure the accuracy of the extracted S1 cycle and S2 cycle.

[0081] In this embodiment, multi-band non-contact phonocardiogram signals are fused and denoised to enhance the valve motion characteristics in the phonocardiogram signals, and the Hilbert transform or support vector machine algorithm is used to detect the phase periods of the first heart sound (S1) and the second heart sound (S2). This method not only realizes contactless phonocardiogram monitoring, but also can effectively resist the interference of environmental noise.

[0082] Furthermore, the heart sound signal, the first heart sound signal, the second heart sound signal, the S1 cycle and / or the S2 cycle extracted by the method of the present invention can be comprehensively analyzed to comprehensively and accurately monitor the cardiac movement of the target living body from different dimensions.

[0083] In order to verify the feasibility of the method of the present invention, it was verified in a laboratory environment. The devices used in the experimental environment mainly include cameras and lasers. The acquisition frame rate of the camera is set to 250 frames per second (fps), the wavelength of the laser is 650 nanometers (nm), and the power is 10 milliwatts (mW). During the experiment, the target living heart area is first irradiated with a laser, and then the defocus degree of the camera is controlled by adjusting the distance between the focal plane of the camera and the lens, so that the monitored interference phenomenon obtains speckles with larger particle sizes and speckle motion with higher sensitivity. Next, the motion signal is extracted from the recorded speckle video by the optical flow method. Finally, the motion signal is filtered with a high-pass filter with a cutoff frequency of 50 Hz, and after noise reduction and normalization processing combined with wavelet transform, the obtained signal is obtained. Figure 2 The PCG signal waveform shown has a sampling frequency of 250HZ.

[0084] In addition, a comparative experiment was further set up between heart sound monitoring using the method of the present invention and heart sound monitoring using a contact stethoscope in the prior art. The stethoscope of a certain brand used in the comparative experiment has a Bluetooth connection function, which can transmit the monitored heart sound data to a mobile phone, and process and analyze it through the auscultation tool on the stethoscope of this brand. In the experiment, the sampling frequency of the stethoscope was set to 11025Hz, and the sampling frequency of the video was set to 250Hz. A camera was used to shoot video to record speckle motion and a stethoscope was used to record heart sounds, respectively, to obtain cardiac motion signals and phonocardiogram signals. In order to achieve alignment of the two signals in subsequent experiments, the cardiac motion signal was first upsampled to 1000Hz, and the phonocardiogram signal was downsampled to 1000Hz, and then the sampled cardiac motion signal and phonocardiogram signal were filtered and denoised, respectively, and finally normalized to obtain Figure 3 and Figure 4 The comparison results of PCG signals in the same time period are shown. Figure 3 and Figure 4 The waveform shown in (1) is the normalized camera PCG obtained by the method of the present invention based on camera acquisition, and the waveform shown in (2) is the normalized camera PCG obtained by the prior art based on stethoscope acquisition. Figure 3 The following is a comparison of the multi-cycle waveforms of the PCG signal in the same period. Figure 4 The figure shows the single cycle comparison waveform of PCG signal in the same period. Figure 3 and Figure 4 It can be seen that the signals extracted by the two different methods show a high degree of similarity in the overall waveform and the first and second heart sounds. Therefore, the detection effect of the phonocardiogram signal obtained by the method of the present invention on the first and second heart sounds is almost consistent with that of the contact method, and the waveforms have a high correlation, which effectively verifies the feasibility and effectiveness of the method of the present invention.

[0085] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Figure 5 As shown. The terminal includes a processor, a memory, a network interface and a display screen connected via a system bus. The processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a non-contact heart sound monitoring program. The internal memory provides an environment for the operating system in the non-volatile storage medium and the operation based on the non-contact heart sound monitoring program. The network interface of the terminal is used to communicate with an external terminal via a network connection. When the non-contact heart sound monitoring program is executed by the processor, the steps of any one of the above-mentioned non-contact heart sound monitoring methods are implemented. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0086] Those skilled in the art will understand that Figure 5 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0087] In one embodiment, a terminal is provided, which includes a memory, a processor, and a non-contact heart sound monitoring program stored in the memory and executable on the processor. When the non-contact heart sound monitoring program is executed by the processor, the steps of any one of the non-contact heart sound monitoring methods provided in the embodiments of the present invention are implemented.

[0088] An embodiment of the present invention further provides a computer-readable storage medium, on which a non-contact heart sound monitoring program is stored. When the non-contact heart sound monitoring program is executed by a processor, the steps of any one of the non-contact heart sound monitoring methods provided in the embodiments of the present invention are implemented.

[0089] It should be understood that the serial numbers of the steps in the above embodiments do not imply a sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0090] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0091] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0092] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0093] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal equipment and method can be implemented in other ways. For example, the apparatus / terminal equipment embodiments described above are only illustrative, for example, the division of the above modules or units is only a logical function division, and in actual implementation, other division methods can be used, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0094] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features therein may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A non-contact heart sound monitoring method, characterized in that: The following steps are involved: Use lasers of different wavelengths to monitor the heart at multiple points synchronously, and obtain defocused speckle images corresponding to different positions of the heart; Performing signal extraction on the defocused speckle image to obtain cardiac motion signals corresponding to multiple bands; Obtaining a heart sound signal based on the heart motion signal corresponding to the multi-bands; The method of using lasers of different wavelengths to synchronously monitor the heart at multiple points to obtain a defocused speckle image includes: using lasers of different wavelengths to irradiate different positions of the heart respectively, and determining the target point corresponding to the laser of each wavelength; Based on the target point corresponding to each band of laser light, the heart is monitored synchronously at multiple points to obtain a defocused speckle image. The target point corresponding to each band of laser is determined according to the positions corresponding to different layers and structures of the heart.

2. The non-contact heart sound monitoring method according to claim 1, characterized in that: The extracting signals from the defocused speckle image to obtain cardiac motion signals corresponding to multiple bands includes: Using an optical flow algorithm to extract signals from the defocused speckle image to obtain cardiac motion features corresponding to each band; Based on the cardiac motion features corresponding to all bands, cardiac motion signals corresponding to multiple bands are obtained.

3. The non-contact heart sound monitoring method according to claim 1, characterized in that: The step of obtaining a heart sound signal based on the heart motion signal corresponding to the multi-bands includes: Using high-pass filtering to filter the cardiac motion signals corresponding to the multi-bands to obtain motion signals related to the generation of heart sounds; A heart sound signal is obtained based on the motion signal related to the generation of heart sounds.

4. The non-contact heart sound monitoring method according to any one of claims 1 to 3, characterized in that: After receiving the heart sound signal, including: fusing and denoising the heart sound signal to obtain heart sound features; Based on the heart sound features, a first heart sound signal and a second heart sound signal are extracted.

5. The non-contact heart sound monitoring method according to claim 4, characterized in that: After extracting the first heart sound signal and the second heart sound signal based on the heart sound feature, the method includes: Based on the first heart sound signal and the second heart sound signal, a first heart sound phase cycle and a second heart sound phase cycle are determined.

6. A non-contact heart sound monitoring device, characterized in that: The device comprises: A defocused speckle image acquisition module is used to perform multi-point synchronous monitoring of the heart using lasers of different wavelength bands to obtain defocused speckle images corresponding to different positions of the heart; A cardiac motion signal extraction module, used for performing signal extraction on the defocused speckle image to obtain cardiac motion signals corresponding to multiple bands; A heart sound signal generating module, used for obtaining a heart sound signal based on the heart motion signal corresponding to the multi-bands; The non-contact heart sound monitoring device is also used to use lasers of different bands to irradiate different positions of the heart respectively, and determine the target point corresponding to the laser of each band; based on the target point corresponding to the laser of each band, the heart is monitored synchronously at multiple points to obtain a defocused speckle image; and the target point corresponding to the laser of each band is determined for the positions corresponding to different layers and structures of the heart.

7. The non-contact heart sound monitoring device according to claim 6, characterized in that: The defocused speckle image acquisition module includes a laser and a camera, wherein the laser is used to emit lasers of different wavelength bands and irradiate different positions of the heart respectively; and the camera is used to simultaneously acquire defocused speckle images corresponding to different positions of the heart through a sensor.

8. A terminal, characterized in that: The terminal includes a memory, a processor, and a non-contact heart sound monitoring program stored in the memory and executable on the processor. When the non-contact heart sound monitoring program is executed by the processor, the steps of the non-contact heart sound monitoring method as described in any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a non-contact heart sound monitoring program, which, when executed by a processor, implements the steps of the non-contact heart sound monitoring method according to any one of claims 1 to 5.

Images