Hemocardiogram signal measurement method and device based on structured light, terminal equipment and storage medium

Abstract

The invention relates to the technical field of biomedical engineering. The invention discloses a seismocardiogram signal measurement method and device based on structured light, terminal equipment and a storage medium. The seismocardiogram signal measurement accuracy can be improved. The structured light-based seismocardiogram signal measurement method comprises the following steps: acquiring structured light speckle videos of the chest and abdomen of a user by adopting a target sensor; performing region extraction processing on the structured light speckle video based on a speckle positioning method and a region screening method to obtain at least one first target speckle region; and performing signal extraction processing on the at least one first target speckle region corresponding to each frame of image in the structured light speckle video to obtain a target seismocardiogram signal.

Description

Technical Field

[0001] The present application relates to the field of biomedical engineering technology. More specifically, the present application relates to a method, device, terminal device and storage medium for measuring seismogram signals based on structured light. Background Art

[0002] The traditional method of measuring seismocardiogram signals is to obtain the original seismocardiogram signal with motion artifact noise by using a seismocardiogram signal sensor; according to the rigid body motion model, the acceleration state of different points of the chest cavity is obtained by using an auxiliary sensor, and the noise data is used after corresponding processing; the noise data is subtracted from the original seismocardiogram signal to remove the motion artifact noise, and the denoised seismocardiogram signal is obtained. In this method, two different types of data are collected by two sensors respectively, and then the two data are simply subtracted to obtain the final seismocardiogram signal. If the shooting positions of the two sensors for collecting the two types of data are deviated, errors will occur in the two types of data, thereby affecting the accuracy of the final seismocardiogram signal, which is not conducive to medical staff using the final seismocardiogram signal to assist in the diagnosis of heart-related diseases. Therefore, the accuracy problem of seismocardiogram signals in the prior art needs to be solved urgently. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method, device, terminal device and storage medium for measuring seismocardiogram signals based on structured light, which can improve the accuracy of seismocardiogram signal measurement. The embodiments of the present application are mainly implemented through the following technical solutions: According to a first aspect of an embodiment of the present application, a method for measuring a seismogram signal based on structured light is provided, comprising: A target sensor is used to obtain structured light speckle video of the user's chest and abdomen; Performing region extraction processing on the structured light speckle video based on a speckle positioning method and a region screening method to obtain at least one first target speckle region; Signal extraction processing is performed on at least one first target speckle area corresponding to each frame of the structured light speckle video to obtain a target seismogram signal.

[0004] According to an embodiment of the present application, the step of performing region extraction processing on the structured light speckle video based on a speckle positioning method and a region screening method to obtain at least one first target speckle region includes: The speckle positioning method is used to screen the speckle area of ​​each frame image in the structured light speckle video to obtain a plurality of second target speckle areas; The region screening method is used to perform region screening processing on the plurality of second target speckle regions to obtain the at least one first target speckle region.

[0005] According to an embodiment of the present application, the steps of screening the speckle regions of each frame of image in the structured light speckle video by using the speckle localization method to obtain a plurality of second target speckle regions include: Preprocess the structured light speckle video to obtain a denoised video; Use the speckle localization method to screen the speckle regions of each frame of image in the denoised video to obtain the plurality of second target speckle regions.

[0006] According to an embodiment of the present application, the steps of screening the speckle regions of each frame of image in the denoised video by using the speckle localization method to obtain the plurality of second target speckle regions include: Set a circular template; Perform matching processing on the circular template and all speckle regions of each frame of image in the denoised video, and use all successfully matched speckle regions as the plurality of second target speckle regions.

[0007] According to an embodiment of the present application, the steps of performing region screening processing on the plurality of second target speckle regions by using the region screening method to obtain the at least one first target speckle region include: Use the optical flow method of the region screening method to perform signal extraction processing on each second target speckle region corresponding to each frame of image in the structured light speckle video or the denoised video to obtain a to-be-evaluated phonocardiogram signal corresponding to each second target speckle region; Use the signal evaluation method of the region screening method to perform evaluation processing on all to-be-evaluated phonocardiogram signals to obtain an evaluation result corresponding to each to-be-evaluated phonocardiogram signal; Based on a preset condition and all evaluation results, screen out the at least one first target speckle region from the plurality of second target speckle regions.

[0008] According to an embodiment of the present application, the steps of using the signal evaluation method of the region screening method to perform evaluation processing on all to-be-evaluated phonocardiogram signals to obtain an evaluation result corresponding to each to-be-evaluated phonocardiogram signal include: Calculate the signal-to-noise ratio of each to-be-evaluated phonocardiogram signal, and use the signal-to-noise ratio of each to-be-evaluated phonocardiogram signal as the evaluation result; or, Use a standard waveform template to match each to-be-evaluated phonocardiogram signal to obtain an evaluation result corresponding to each to-be-evaluated phonocardiogram signal.

[0009] According to an embodiment of the present application, the steps of performing signal extraction processing on at least one first target speckle region corresponding to each frame of image in the structured light speckle video to obtain a target phonocardiogram signal include: When there is one first target speckle region among the at least one first target speckle region, signal extraction processing is performed on the first target speckle region corresponding to each frame image in the structured light speckle video to obtain the target phonocardiogram signal; When there are multiple first target speckle regions among the at least one first target speckle region, the following steps are performed: Signal extraction processing is performed on each first target speckle region corresponding to each frame image in the structured light speckle video to obtain an initial phonocardiogram signal corresponding to each first target speckle region; Principal component analysis method is used to perform fusion processing on all the initial phonocardiogram signals to obtain the target phonocardiogram signal.

[0010] In the second aspect of the embodiments of the present application, a phonocardiogram signal measurement device based on structured light is provided, including: A video acquisition module, configured to acquire a structured light speckle video of a user's chest and abdomen by using a target sensor; A region extraction module, configured to perform region extraction processing on the structured light speckle video based on a speckle localization method and a region screening method to obtain at least one first target speckle region; A signal extraction module, configured to perform signal extraction processing on at least one first target speckle region corresponding to each frame image in the structured light speckle video to obtain a target phonocardiogram signal.

[0011] In the third aspect of the embodiments of the present application, a terminal device is provided, including: a processor and a memory, where the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the steps of the method for measuring a phonocardiogram signal based on structured light provided in the first aspect of the embodiments of the present application.

[0012] In the fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, where the computer-readable storage medium is used to store a computer program, and the computer program enables a computer to execute the steps of the method for measuring a phonocardiogram signal based on structured light provided in the first aspect of the embodiments of the present application.

[0013] The beneficial effects of the embodiments of the present application include: In the embodiment of the present application, a target sensor is used to collect a structured light speckle video, and then the structured light speckle video is subjected to speckle localization by the speckle localization method and the region screening method to obtain at least one first target speckle region, and then the target ballistocardiogram signal is extracted from each frame of image in the structured light speckle video by using the at least one first target speckle region. Compared with the prior art in which a ballistocardiogram signal sensor is used to obtain the original ballistocardiogram signal and an auxiliary sensor is used to obtain noise data, and the two types of data, i.e., the original ballistocardiogram signal and the noise data, are used to obtain the final ballistocardiogram signal, the embodiment of the present application uses only one type of sensor, i.e., the target sensor, to collect one type of video data (i.e., the structured light speckle video), which can effectively avoid the shooting position deviation problem caused by the two sensors collecting two different types of data respectively, thereby reducing data errors and improving the accuracy of ballistocardiogram signal measurement.

[0014] Moreover, the embodiment of the present application uses the speckle localization method and the region screening method to perform speckle localization on the structured light speckle video, which can improve the accuracy of the region for screening and extracting the ballistocardiogram signal, and further improve the accuracy of ballistocardiogram signal measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0016] Figure 1 It is a flowchart of the ballistocardiogram signal measurement method based on structured light in some embodiments of the present application; Figure 2 It is a reference diagram of the defocus principle of the lens of the high-speed camera in the present application; Figure 3 It is a flowchart of the ballistocardiogram signal measurement method based on structured light in some other embodiments of the present application; Figure 4 It is a flowchart block diagram of the ballistocardiogram signal measurement method based on structured light in some embodiments of the present application; Figure 5 It is a flowchart of the ballistocardiogram signal measurement method based on structured light in some other embodiments of the present application; Figure 6 It is a flowchart of the ballistocardiogram signal measurement method based on structured light in some other embodiments of the present application; Figure 7 It is a flowchart of the ballistocardiogram signal measurement method based on structured light in some other embodiments of the present application; Figure 8Flowchart of the method for measuring the heart vibration signal based on structured light in some other embodiments of the present application; Figure 9 Flowchart of the method for measuring the heart vibration signal based on structured light in some other embodiments of the present application; Figure 10 Flowchart of the method for measuring the heart vibration signal based on structured light in some other embodiments of the present application; Figure 11 Flowchart of the method for measuring the heart vibration signal based on structured light in some other embodiments of the present application; Figure 12 Flow block diagram of the method for measuring the heart vibration signal based on structured light in some other embodiments of the present application; Figure 13 Flowchart of the method for measuring the heart vibration signal based on structured light in some other embodiments of the present application; Figure 14 Flowchart of the method for measuring the heart vibration signal based on structured light in some other embodiments of the present application; Figure 15 Flowchart of the method for measuring the heart vibration signal based on structured light in some other embodiments of the present application; Figure 16 Flowchart of the method for measuring the heart vibration signal based on structured light in some other embodiments of the present application; Figure 17 Flowchart of the method for measuring the heart vibration signal based on structured light in some other embodiments of the present application; Figure 18 Principle block diagram of the device for measuring the heart vibration signal based on structured light in some embodiments of the present application; Figure 19 Principle block diagram of the terminal device in some embodiments of the present application. Detailed implementation manners

[0017] To make the above objects, features, and advantages of the present application more obvious and understandable, the following describes the detailed implementation manners of the present application in conjunction with the accompanying drawings. Many specific details are set forth in the following description to fully understand the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present application. Therefore, the present application is not limited by the specific embodiments disclosed below.

[0018] It should be noted that the terms "first" and "second" are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of this application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0019] The term "exemplary" or "for example" and the like are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or designs. Rather, the use of the terms "exemplary" or "for example" and the like is intended to present relevant concepts in a specific manner.

[0020] The term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0021] The seismocardiogram signal, that is, the SCG (Seismocardiography) signal, mainly records the chest wall vibrations caused by the heart during systole and diastole, and can provide important information related to the cardiac mechanical function (such as ventricular contraction, valve movement, and myocardial activity), thereby assisting medical staff in evaluating the cardiac health status.

[0022] Unless otherwise defined, all technical and scientific terms used in the description of this application have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used in the description of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in the description of this application includes any and all combinations of one or more of the related listed items.

[0023] The following further describes the specific embodiments of this application with reference to the accompanying drawings.

[0024] Refer to Figure 1 As shown, it is a flowchart of a method for measuring seismocardiogram signals based on structured light provided in the first aspect of the embodiments of this application. In Figure 1 it, the method for measuring seismocardiogram signals based on structured light includes: S1. Use a target sensor to acquire the structured light speckle video of the user's chest and abdomen.

[0025] When collecting the structured light speckle video, the embodiment of the present application uses structured light as a light source to irradiate the chest and abdomen of the user, and the lens of the target sensor is simultaneously aligned with the chest and abdomen of the user. The structured light is composed of multiple infrared laser beams, and the structured light complies with safety standards (Class I level, that is, the I-class level).

[0026] Compared with a single laser beam, the structured light can cover a larger area on the chest and abdomen of the user, thus effectively solving the problem of relying on manual positioning in the traditional single-point laser defocus vibration measurement method.

[0027] The structured light speckle video is a defocused speckle video collected by a high-speed camera with a defocus lens at a high frame rate of 200 fps (Frames Per Second). Each frame image in the structured light speckle video is a speckle image. In other embodiments, the frame rate of the high-speed camera can be set by those skilled in the art according to actual needs.

[0028] In the speckle image, due to the influence of cardiac pulsation and respiratory movement, the internal texture of the speckle image near the cardiac region will move periodically with the regular pulsation of the heart; while the speckle image near the abdominal region will move regularly as a whole or locally due to the influence of respiratory movement. These regular changes provide an important basis for non-invasive detection of ballistocardiogram signals and respiratory signals.

[0029] The realization of the defocus of the high-speed camera can refer to Figure 2 shown, set the distance between the focus plane of the high-speed camera (expressed as "camera" in Figure 2 ) and the lens of the high-speed camera to L2. L2 is a value, and the specific value can be set by those skilled in the art according to actual needs; set the distance between the focus plane of the high-speed camera and the chest and abdomen of the user to L1. L1 is a value, and the specific value can be set by those skilled in the art according to actual needs; where L1 is greater than L2. By setting the distances of L1 and L2, the speckle image formed by the interference of structured light can be significantly magnified.

[0030] The magnification effect of the speckle is closely related to the defocus degree of the lens. The defocus degree can be expressed by L1 / L2. The motion amplification gain of the target ballistocardiogram signal is approximately proportional to L1 / L2.

[0031] S2. Perform region extraction processing on the structured light speckle video based on the speckle localization method and the region screening method to obtain at least one first target speckle region.

[0032] Further, referring to Figure 3 shown, step S2 includes:[[]] S21. Use the speckle localization method to screen the speckle regions in each frame of the structured light speckle video, and obtain a plurality of second target speckle regions. Refer to Figure 4 the steps of "structured light speckle image", "speckle localization" and "second target speckle region" shown in

[0033] Further, refer to Figure 5 shown, the S21 step includes: S211. Preprocess the structured light speckle video to obtain a denoised video.

[0034] Further, refer to Figure 6 shown, the S211 step includes: S2111. Binarize the structured light speckle video to obtain a grayscale video.

[0035] Specifically, the embodiments of the present application can use the Bradley threshold algorithm (i.e., the Bradley threshold algorithm), the Wolf-Jolion adaptive threshold algorithm (i.e., the Wolf-Jolion adaptive threshold algorithm), the Gaussian threshold algorithm or the Sauvola adaptive threshold algorithm (i.e., the Sauvola adaptive threshold algorithm) to binarize each frame of the structured light speckle video, so as to obtain the grayscale image corresponding to each frame of the structured light speckle video; then, arrange all the grayscale images in the same image sequence as the structured light speckle video to form the grayscale video.

[0036] The above threshold algorithms can dynamically adjust the threshold according to local characteristics, so as to effectively distinguish the speckle region and the background in the image.

[0037] S2112. Perform morphological operation processing on the grayscale video to obtain the denoised video.

[0038] Specifically, first perform erosion processing on the grayscale video to obtain an eroded video; then perform dilation processing on the eroded video to obtain the denoised video.

[0039] The erosion processing can remove the small noise points in each frame of the grayscale video. The dilation processing can restore the integrity of the speckle region in each frame of the eroded video.

[0040] The S2112 step can also remove the background noise and other non-speckle regions in the grayscale video.

[0041] S212. Use the speckle localization method to screen the speckle regions in each frame of the denoised video, and obtain the plurality of second target speckle regions.

[0042] Further, refer to Figure 7As shown, step S212 includes: S2121. Set a circular template.

[0043] Since speckles usually present a circular structure, a circular template is adopted in the embodiments of the present application. In other embodiments, those skilled in the art can set the shape of the template according to actual needs.

[0044] S2122. Perform matching processing on all speckle regions of the circular template and each frame of image in the denoised video, and use all successfully matched speckle regions as the multiple second target speckle regions.

[0045] Furthermore, the embodiments of the present application use an image convolutional neural network to perform matching processing on all speckle regions of the circular template and each frame of image in the denoised video.

[0046] Furthermore, referring to Figure 8 As shown, the steps of using an image convolutional neural network to perform matching processing on all speckle regions of the circular template and each frame of image in the denoised video include: S21221. Set the convolutional kernel of the image convolutional neural network to match the circular template.

[0047] S21222. The convolutional kernel slides on each frame of image in the denoised video, and calculates the first matching degree between each speckle region corresponding to the convolutional kernel and the circular template.

[0048] S21223. When the first matching degree reaches a first preset threshold, it is regarded as a successful match, and the speckle region corresponding to the first matching degree is used as the second target speckle region.

[0049] The first preset threshold can be set by those skilled in the art according to actual needs, and the present application does not further limit this.

[0050] In other embodiments, referring to Figure 9 As shown, step S212 includes: S21201. Set a circular template.

[0051] S21202. Perform matching processing on all speckle regions of the circular template and each frame of image in the denoised video, and use all successfully matched speckle regions as the speckle regions to be screened out.

[0052] Furthermore, the embodiments of the present application use an image convolutional neural network to perform matching processing on all speckle regions of the circular template and each frame of image in the denoised video.

[0053] Furthermore, referring to Figure 10As shown in the figure, the steps of using an image convolutional neural network to perform matching processing on all speckle regions of each frame of the circular template and the noise-reduced video include: S212021. Set the convolutional kernel of the image convolutional neural network to match the circular template.

[0054] S212022. The convolutional kernel slides on each frame of the noise-reduced video, and calculates the second matching degree between each speckle region corresponding to the convolutional kernel and the circular template.

[0055] S212023. When the second matching degree reaches the third preset threshold, it is regarded as a successful match, and the speckle region corresponding to the second matching degree is used as the speckle region to be screened out.

[0056] The third preset threshold can be set by those skilled in the art according to actual needs, and the present application does not further limit this.

[0057] S21203. Calculate the coincidence degree between each speckle region to be screened out and the circular template. When the coincidence degree reaches the second preset threshold, the speckle region to be screened out corresponding to the coincidence degree is used as the second target speckle region.

[0058] The coincidence degree can be understood as the coincidence area.

[0059] The specific value of the second preset threshold can be set by those skilled in the art according to actual needs.

[0060] The implementation of steps S21201 - S21203 can eliminate mis-matched regions and ensure the accuracy of speckle positioning.

[0061] S22. Use the region screening method to perform region screening processing on the multiple second target speckle regions to obtain the at least one first target speckle region.

[0062] Further, as shown in Figure 11 the figure, step S22 includes: S221. Use the optical flow method of the region screening method to perform signal extraction processing on each second target speckle region corresponding to each frame of the structured light speckle video or the noise-reduced video, and obtain the heart vibration map signal to be evaluated corresponding to each second target speckle region. Refer to the steps of "optical flow method" and "heart vibration map signal to be evaluated" in Figure 4 .

[0063] Further, as shown in Figure 13 the figure, step S221 includes: S2211. Estimate the first target features of each pixel point in the candidate target speckle region corresponding to each frame image in the structured light speckle video or the denoised video in multiple first predetermined directions between consecutive frames by using the optical flow method, where the candidate target speckle region is any one of all the second target speckle regions.

[0064] The multiple first predetermined directions are the horizontal direction and the vertical direction. In other embodiments, those skilled in the art can set other directions according to actual needs.

[0065] The first target feature is the velocity component. In other embodiments, those skilled in the art can set it as other features according to actual needs, such as the acceleration component.

[0066] S2212. Perform cumulative sum calculation processing on all the first target features of each pixel point in the candidate target speckle region in each first predetermined direction between consecutive frames to obtain the first displacement signal corresponding to each first predetermined direction of each pixel point in the candidate target speckle region between consecutive frames.

[0067] S2213. Calculate the average value of all the first displacement signals in each first predetermined direction to obtain the first physiological motion signal corresponding to each first predetermined direction.

[0068] S2214. Perform fusion processing on the first physiological motion signals in all the first predetermined directions to obtain the electrocardiogram signal to be evaluated corresponding to the candidate target speckle region.

[0069] In other embodiments, the background subtraction method or the frame difference method can be used to replace the optical flow method, which can be specifically set by those skilled in the art according to actual needs.

[0070] S222. Evaluate all the electrocardiogram signals to be evaluated by using the signal evaluation method of the region screening method to obtain the evaluation result corresponding to each electrocardiogram signal to be evaluated.

[0071] Further, step S222 includes: S2221. Calculate the signal-to-noise ratio of each electrocardiogram signal to be evaluated and use the signal-to-noise ratio of each electrocardiogram signal to be evaluated as the evaluation result.

[0072] In other embodiments, step S222 includes: S2222. Match the standard waveform template with each electrocardiogram signal to be evaluated to obtain the evaluation result corresponding to each electrocardiogram signal to be evaluated.

[0073] S223. Screen out the at least one first target speckle region from the multiple second target speckle regions based on a preset condition and all evaluation results. Refer to Figure 12 the steps of "the heart shock map signal to be evaluated" and "the first target speckle region" in

[0074] Further, refer to Figure 14 As shown, step S223 includes: S2231. Sort all the evaluation results, and use the evaluation results that meet the preset condition after sorting as the target results.

[0075] The sorting process is sorting from large to small or from small to large. The specific sorting method can be set by those skilled in the art according to actual needs.

[0076] When the sorting process is sorting from large to small, the preset condition is that the evaluation results are ranked in the top 20% or the top 30%. Of course, in other embodiments, it can also be the top 50% of the evaluation results. The specific value can be set by those skilled in the art according to actual needs.

[0077] In other embodiments, when the sorting process is sorting from small to large, the preset condition is that the evaluation results are ranked in the bottom 20% or the bottom 30%. In other embodiments, it can also be the bottom 50% of the evaluation results. The specific value can be set by those skilled in the art according to actual needs.

[0078] S2232. Use the second target speckle regions corresponding to all the target results as the first target speckle regions.

[0079] Each of the first target speckle regions is a high-quality speckle region.

[0080] In other embodiments, before step S222, step S22 further includes: performing detrending and filtering processing on all the heart shock map signals to be evaluated to obtain updated heart shock map signals to be evaluated. Refer to Figure 4 the "detrending and filtering" step in

[0081] In the case where there is a step of "performing detrending and filtering processing on all the heart shock map signals to be evaluated to obtain updated heart shock map signals to be evaluated", step S222 is to use the signal evaluation method of the region screening method to evaluate all the updated heart shock map signals to be evaluated, and obtain evaluation results corresponding to each heart shock map signal to be evaluated.

[0082] The implementation of the above steps can eliminate low-frequency drift or background noise in all the heart shock map signals to be evaluated, and further enhance the quality of the updated heart shock map signals to be evaluated.

[0083] S3. Perform signal extraction processing on at least one first target speckle region corresponding to each frame image in the structured light speckle video to obtain a target phonocardiogram signal. Refer to Figure 12 the step of "target phonocardiogram signal" in

[0084] The target phonocardiogram signal is used to assist medical staff in diagnosing heart-related diseases.

[0085] Further, refer to Figure 15 shown, step S3 includes: S31. When the at least one first target speckle region is a single first target speckle region, perform signal extraction processing on the first target speckle region corresponding to each frame image in the structured light speckle video to obtain the target phonocardiogram signal.

[0086] Further, refer to Figure 16 shown, step S31 includes: S311. Use the optical flow method to estimate the second target features of each pixel point in the first target speckle region corresponding to each frame image in the structured light speckle video in multiple second predetermined directions between consecutive frames. Refer to Figure 12 the step of "optical flow method" in

[0087] The multiple second predetermined directions are the horizontal direction and the vertical direction. In other embodiments, those skilled in the art can set other directions according to actual needs.

[0088] The second target feature is a velocity component. In other embodiments, those skilled in the art can set it as other features according to actual needs, such as an acceleration component.

[0089] S312. Perform cumulative sum calculation processing on all the second target features of each pixel point in the first target speckle region in each second predetermined direction between consecutive frames to obtain a second displacement signal corresponding to each second predetermined direction of each pixel point in the first target speckle region between consecutive frames.

[0090] S313. Calculate the average value of all the second displacement signals in each second predetermined direction to obtain a second physiological motion signal corresponding to each second predetermined direction.

[0091] S314. Perform fusion processing on the second physiological motion signals in all the second predetermined directions to obtain a target phonocardiogram signal corresponding to the first target speckle region.

[0092] S32. When the at least one first target speckle region is multiple first target speckle regions, perform the following steps S33 and S34.

[0093] S33. Perform signal extraction processing on each first target speckle region corresponding to each frame image in the structured light speckle video to obtain an initial phonocardiogram signal corresponding to each first target speckle region.

[0094] Further, referring to Figure 17 as shown, step S33 includes: S331. Use the optical flow method to estimate the third target features of each pixel point in each first target speckle region corresponding to each frame image in the structured light speckle video in multiple third predetermined directions between consecutive frames.

[0095] The multiple third predetermined directions are the horizontal direction and the vertical direction. In other embodiments, those skilled in the art can set other directions according to actual needs.

[0096] The third target feature is a velocity component. In other embodiments, those skilled in the art can set it to other features according to actual needs, such as an acceleration component.

[0097] S332. Perform cumulative sum calculation processing on all the third target features of each pixel point in each first target speckle region in each third predetermined direction between consecutive frames to obtain a third displacement signal corresponding to each pixel point in each first target speckle region in each third predetermined direction between consecutive frames.

[0098] S333. Calculate the average value of all the third displacement signals in each third predetermined direction to obtain a third physiological motion signal corresponding to each third predetermined direction of each first target speckle region.

[0099] S334. Perform fusion processing on the third physiological motion signals in all the third predetermined directions to obtain an initial phonocardiogram signal corresponding to each first target speckle region.

[0100] S34. Use the principal component analysis method to perform fusion processing on all the initial phonocardiogram signals to obtain the target phonocardiogram signal.

[0101] In some embodiments, referring to Figure 15 as shown, step S3 also includes: S35. Perform noise reduction processing on the target phonocardiogram signal to obtain a high-quality phonocardiogram signal.

[0102] The noise reduction processing can be detrending filtering processing. In other embodiments, the noise reduction processing can also be other noise reduction operations, which can be specifically set by those skilled in the art according to actual needs.

[0103] The implementation of step S35 can further enhance the quality and stability of the target phonocardiogram signal.

[0104] Through the above embodiments, the embodiment of the present application acquires a structured light speckle video through a target sensor, then performs speckle localization on the structured light speckle video by means of a speckle localization method and a region screening method to obtain at least one first target speckle region, and then extracts a target phonocardiogram signal from each frame of image in the structured light speckle video by using the at least one first target speckle region. Compared with the prior art that uses a phonocardiogram signal sensor to acquire an original phonocardiogram signal and an auxiliary sensor to acquire noise data, and uses these two types of data, namely the original phonocardiogram signal and the noise data, to obtain a final phonocardiogram signal, the embodiment of the present application only uses one type of sensor, i.e., the target sensor, to acquire one type of video data (i.e., the structured light speckle video), which can effectively avoid the shooting position deviation problem caused by the separate acquisition of two different types of data by two sensors, thereby reducing data errors and improving the accuracy of phonocardiogram signal measurement.

[0105] Moreover, the embodiment of the present application uses a speckle localization method and a region screening method to perform speckle localization on the structured light speckle video, which can improve the accuracy of screening and extracting the region of the phonocardiogram signal, and further improve the accuracy of phonocardiogram signal measurement.

[0106] In some embodiments, the method for measuring a phonocardiogram signal based on structured light further includes: S4. Extract features from the target phonocardiogram signal or the high-quality phonocardiogram signal to obtain a heartbeat cycle and an amplitude change.

[0107] The implementation of step S4 can be used to assist medical staff in evaluating the heart health of a user.

[0108] In other embodiments, other characteristic parameters can also be extracted from the target phonocardiogram signal or the high-quality phonocardiogram signal, such as the phase difference and time difference between heart vibrations at different positions. Specifically, those skilled in the art can set them according to actual needs.

[0109] In other embodiments, the acquisition of the at least one target speckle region can also be achieved by shooting a global light spot in the early stage, determining a specific light spot position among them, and then using an auto-focus lens to lock the selected light spot position to ensure that only the selected light spot is retained in the image region, thereby obtaining the at least one target speckle region.

[0110] Reference Figure 18 As shown, it is a schematic block diagram of a device for measuring a phonocardiogram signal based on structured light provided in the second aspect of the embodiment of the present application. In Figure 18 it, the device 100 for measuring a phonocardiogram signal based on structured light includes: A video acquisition module 101, configured to acquire a structured light speckle video of a user's chest and abdomen by using a target sensor; The region extraction module 102 is configured to perform region extraction processing on the structured light speckle video based on the speckle localization method and the region screening method to obtain at least one first target speckle region; The signal extraction module 103 is configured to perform signal extraction processing on at least one first target speckle region corresponding to each frame of the structured light speckle video to obtain a target phonocardiogram signal.

[0111] In a third aspect of the embodiments of the present application, a terminal device is provided. The principle block diagram of the terminal device may be as Figure 19 shown. The terminal device includes a processor, a memory, a network interface, a display screen, and a temperature sensor connected through a system bus. Among them, the processor is configured to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is configured to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for measuring a phonocardiogram signal based on structured light is implemented. The display screen may be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0112] Those skilled in the art can understand that Figure 19 the principle block diagram shown in

[0113] merely shows the block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the terminal device to which the solution of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have a different component layout.

[0114] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0115] Without changing the basic principle of the present application, the technical features of the above embodiments can be combined. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0116] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. A method for measuring the heart vibration signal based on structured light, characterized in that Including: Obtaining a structured light speckle video of a user's chest and abdomen using a target sensor; Performing region extraction processing on the structured light speckle video based on a speckle localization method and a region screening method to obtain at least one first target speckle region; Performing signal extraction processing on at least one first target speckle region corresponding to each frame image in the structured light speckle video to obtain a target ballistocardiogram signal.

2. The method for measuring the heart vibration signal based on structured light according to claim 1, characterized in that The step of performing region extraction processing on the structured light speckle video based on a speckle localization method and a region screening method to obtain at least one first target speckle region includes: Performing screening processing on the speckle regions of each frame image in the structured light speckle video using the speckle localization method to obtain a plurality of second target speckle regions; Performing region screening processing on the plurality of second target speckle regions using the region screening method to obtain the at least one first target speckle region.

3. The method for measuring the heart vibration signal based on structured light according to claim 2, wherein The step of performing screening processing on the speckle regions of each frame image in the structured light speckle video using the speckle localization method to obtain a plurality of second target speckle regions includes: Performing preprocessing on the structured light speckle video to obtain a noise-reduced video; Performing screening processing on the speckle regions of each frame image in the noise-reduced video using the speckle localization method to obtain the plurality of second target speckle regions.

4. The method for measuring the heart vibration signal based on structured light according to claim 3, characterized in that, The step of performing screening processing on the speckle regions of each frame image in the noise-reduced video using the speckle localization method to obtain the plurality of second target speckle regions includes: Setting a circular template; Performing matching processing on the circular template and all speckle regions of each frame image in the noise-reduced video, and taking all the successfully matched speckle regions as the plurality of second target speckle regions.

5. The method for measuring the heart vibration signal based on structured light according to claim 3, wherein The step of performing region screening processing on the plurality of second target speckle regions using the region screening method to obtain the at least one first target speckle region includes: Performing signal extraction processing on each second target speckle region corresponding to each frame image in the structured light speckle video or the noise-reduced video using the optical flow method of the region screening method to obtain a to-be-evaluated ballistocardiogram signal corresponding to each second target speckle region; Performing evaluation processing on all the to-be-evaluated ballistocardiogram signals using the signal evaluation method of the region screening method to obtain an evaluation result corresponding to each to-be-evaluated ballistocardiogram signal; Screening out the at least one first target speckle region from the plurality of second target speckle regions based on a preset condition and all the evaluation results.

6. The method for measuring the heart vibration signal based on structured light according to claim 5, wherein The step of performing evaluation processing on all the to-be-evaluated ballistocardiogram signals using the signal evaluation method of the region screening method to obtain an evaluation result corresponding to each to-be-evaluated ballistocardiogram signal includes: Calculating the signal-to-noise ratio of each to-be-evaluated ballistocardiogram signal and taking the signal-to-noise ratio of each to-be-evaluated ballistocardiogram signal as the evaluation result; or, Performing matching between a standard waveform template and each to-be-evaluated ballistocardiogram signal to obtain an evaluation result corresponding to each to-be-evaluated ballistocardiogram signal.

7. The method for measuring the heart vibration signal based on structured light according to claim 1, wherein The step of performing signal extraction processing on at least one first target speckle region corresponding to each frame image in the structured light speckle video to obtain a target ballistocardiogram signal includes: When there is one first target speckle region among the at least one first target speckle region, signal extraction processing is performed on the first target speckle region corresponding to each frame of the structured light speckle video to obtain the target cardiogram signal; When there are multiple first target speckle regions among the at least one first target speckle region, the following steps are performed: Signal extraction processing is performed on each first target speckle region corresponding to each frame of the structured light speckle video to obtain an initial cardiogram signal corresponding to each first target speckle region; Principal component analysis method is used to fuse all the initial cardiogram signals to obtain the target cardiogram signal.

8. A structural-light-based cardiac vibration signal measuring device, characterized in that It includes: A video acquisition module, configured to acquire a structured light speckle video of a user's chest and abdomen by using a target sensor; A region extraction module, configured to perform region extraction processing on the structured light speckle video based on a speckle localization method and a region screening method to obtain at least one first target speckle region; A signal extraction module, configured to perform signal extraction processing on at least one first target speckle region corresponding to each frame of the structured light speckle video to obtain a target cardiogram signal.

9. A terminal device, characterized in that, It includes: A processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the steps of the structured light-based cardiogram signal measurement method according to any one of claims 1 to 7 above.

10. A computer-readable storage medium, characterized in that, For storing a computer program, the computer program causes a computer to execute the steps of the structured light-based cardiogram signal measurement method according to any one of claims 1 to 7 above.

Images