Method, device, terminal equipment and storage medium for measuring seismogram signals based on structured light

Abstract

The present application relates to the field of biomedical engineering technology. Disclosed are a structured light-based seismogram signal measurement method, apparatus, terminal device, and storage medium, capable of improving the accuracy of seismogram signal measurement. The structured light-based seismogram signal measurement method comprises acquiring a structured light speckle video of a user's chest and abdomen using a target sensor; performing region extraction processing on the structured light speckle video based on a speckle localization method and a region screening method to obtain at least one first target speckle region; and performing signal extraction processing on the at least one first target speckle region corresponding to each frame of the structured light speckle video to obtain a target seismogram signal.

Description

Technical Field

[0001] The present application relates to the field of biomedical engineering technology. More specifically, the present application relates to a method, apparatus, terminal device, and storage medium for measuring seismogram signals based on structured light. Background Art

[0002] The traditional method of measuring seismocardiogram signals is to obtain the original seismocardiogram signal with motion artifact noise by using a seismocardiogram signal sensor; according to the rigid body motion model, the acceleration state of different points in the chest cavity is obtained by using auxiliary sensors, and the noise data is processed accordingly; the noise data is subtracted from the original seismocardiogram signal to remove the motion artifact noise and obtain the denoised seismocardiogram signal. In this method, two different types of data are collected by two sensors respectively, and then the two data are simply subtracted to obtain the final seismocardiogram signal. If the shooting positions of the two sensors for collecting the two types of data deviate, errors will occur in the two types of data, thereby affecting the accuracy of the final seismocardiogram signal, which is not conducive to medical staff using the final seismocardiogram signal to assist in the diagnosis of heart-related diseases. Therefore, the accuracy problem of the seismocardiogram signal in the existing technology needs to be solved urgently. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a method, apparatus, terminal device, and storage medium for measuring seismocardiogram signals based on structured light, which can improve the accuracy of seismocardiogram signal measurement. The embodiments of the present application are mainly achieved through the following technical solutions:

[0004] A first aspect of an embodiment of the present application provides a method for measuring a seismogram signal based on structured light, comprising:

[0005] A target sensor is used to obtain structured light speckle video of the user's chest and abdomen;

[0006] Performing region extraction processing on the structured light speckle video based on a speckle positioning method and a region screening method to obtain at least one first target speckle region;

[0007] Signal extraction processing is performed on at least one first target speckle area corresponding to each frame of the structured light speckle video to obtain a target seismogram signal.

[0008] According to one embodiment of the present application, the step of performing region extraction processing on the structured light speckle video based on a speckle location method and a region screening method to obtain at least one first target speckle region includes:

[0009] Using the speckle localization method, the speckle area of ​​each frame image in the structured light speckle video is screened to obtain a plurality of second target speckle areas;

[0010] The region screening method is used to perform region screening processing on the plurality of second target speckle regions to obtain the at least one first target speckle region.

[0011] According to one embodiment of the present application, the step of using the speckle localization method to filter the speckle area of ​​each frame image in the structured light speckle video to obtain multiple second target speckle areas includes:

[0012] Preprocessing the structured light speckle video to obtain a noise-reduced video;

[0013] The speckle location method is used to screen the speckle area of ​​each frame image in the noise reduction video to obtain the multiple second target speckle areas.

[0014] According to one embodiment of the present application, the step of using the speckle localization method to filter the speckle area of ​​each frame image in the noise reduction video to obtain the plurality of second target speckle areas includes:

[0015] Set up a circular template;

[0016] Matching processing is performed on the circular template and all speckle areas of each frame image in the noise reduction video, and all speckle areas that are successfully matched are used as the multiple second target speckle areas.

[0017] According to one embodiment of the present application, the step of performing regional screening processing on the plurality of second target speckle regions using the regional screening method to obtain the at least one first target speckle region includes:

[0018] performing signal extraction processing on each second target speckle region corresponding to each frame of the structured light speckle video or the noise reduction video using the optical flow method of the region screening method to obtain a seismogram signal to be evaluated corresponding to each second target speckle region;

[0019] Using the signal evaluation method of the regional screening method to evaluate all seismocardiogram signals to be evaluated, and obtaining an evaluation result corresponding to each seismocardiogram signal to be evaluated;

[0020] The at least one first target speckle area is screened out from the plurality of second target speckle areas based on preset conditions and all evaluation results.

[0021] According to one embodiment of the present application, the signal evaluation method of the regional screening method is used to evaluate all seismocardiogram signals to be evaluated, and the step of obtaining an evaluation result corresponding to each seismocardiogram signal to be evaluated includes:

[0022] Calculating the signal-to-noise ratio of each seismocardiogram signal to be evaluated, and using the signal-to-noise ratio of each seismocardiogram signal to be evaluated as an evaluation result; or,

[0023] A standard waveform template is used to match each seismocardiogram signal to be evaluated to obtain an evaluation result corresponding to each seismocardiogram signal to be evaluated.

[0024] According to one embodiment of the present application, the step of performing signal extraction processing on at least one first target speckle area corresponding to each frame of the structured light speckle video to obtain a target seismocardiogram signal includes:

[0025] When the at least one first target speckle region is one first target speckle region, performing signal extraction processing on the first target speckle region corresponding to each frame of the structured light speckle video to obtain the target seismocardiogram signal;

[0026] When the at least one first target speckle area is a plurality of first target speckle areas, the following steps are performed:

[0027] performing signal extraction processing on each first target speckle region corresponding to each frame of the structured light speckle video to obtain an initial seismogram signal corresponding to each first target speckle region;

[0028] All initial seismocardiogram signals are fused using principal component analysis to obtain the target seismocardiogram signal.

[0029] A second aspect of an embodiment of the present application provides a structured light-based seismogram signal measurement device, comprising:

[0030] A video acquisition module, configured to acquire a structured light speckle video of the user's chest and abdomen using a target sensor;

[0031] a region extraction module, configured to perform region extraction processing on the structured light speckle video based on a speckle positioning method and a region screening method to obtain at least one first target speckle region;

[0032] The signal extraction module is configured to perform signal extraction processing on at least one first target speckle region corresponding to each frame of the structured light speckle video to obtain a target seismogram signal.

[0033] According to a third aspect of an embodiment of the present application, a terminal device is provided, comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the steps of the structured light-based seismogram signal measurement method provided in the first aspect of the embodiment of the present application.

[0034] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, and the computer program enables a computer to execute the steps of the structured light-based seismogram signal measurement method provided in the first aspect of the embodiment of the present application.

[0035] The beneficial effects of the embodiments of the present application include:

[0036] In one embodiment of the present application, a target sensor is used to capture a structured light speckle video. The speckle localization method and region screening method are then used to locate the speckle pattern in the structured light speckle video, obtaining at least one first target speckle region. This at least one first target speckle region is then used to extract a target seismocardiogram signal from each frame of the structured light speckle video. Compared to the prior art, which uses a seismocardiogram signal sensor to capture the original seismocardiogram signal and an auxiliary sensor to capture noise data, and then uses both the original seismocardiogram signal and the noise data to generate the final seismocardiogram signal, the present embodiment uses only one sensor, the target sensor, to capture one type of video data (i.e., the structured light speckle video). This effectively avoids the problem of shooting position deviation caused by two sensors collecting two different types of data, thereby reducing data errors and improving the accuracy of seismocardiogram signal measurement.

[0037] Furthermore, the embodiments of the present application use a speckle localization method and a region screening method to perform speckle localization on a structured light speckle video, which can improve the accuracy of screening and extracting regions for seismocardiogram signals, and further improve the accuracy of seismocardiogram signal measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 A flowchart of the structured light-based seismogram signal measurement method of the present application in some embodiments;

[0040] Figure 2 This is a reference diagram of the lens defocus principle of the high-speed camera in this application;

[0041] Figure 3 Flowcharts of other embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0042] Figure 4 This is a flowchart of the structured light-based seismogram signal measurement method of the present application in some embodiments;

[0043] Figure 5 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0044] Figure 6 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0045] Figure 7 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0046] Figure 8 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0047] Figure 9 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0048] Figure 10 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0049] Figure 11 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0050] Figure 12 A flowchart of another embodiment of the method for measuring seismocardiogram signals based on structured light of the present application;

[0051] Figure 13 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0052] Figure 14 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0053] Figure 15 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0054] Figure 16 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0055] Figure 17 Flowcharts of some further embodiments of the method for measuring seismocardiogram signals based on structured light of the present application;

[0056] Figure 18This is a principle block diagram of the structured light-based seismogram signal measurement device of the present application in some embodiments;

[0057] Figure 19 This is a principle block diagram of the terminal device of the present application in some embodiments. DETAILED DESCRIPTION

[0058] To make the above-mentioned objects, features, and advantages of the present application more clearly understood, the specific embodiments of the present application are described in detail below with reference to the accompanying drawings. The following description sets forth many specific details to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0059] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0060] The terms "exemplary" or "for example" are used to indicate an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0061] The terms "comprises," "comprising," or any other variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0062] Seismocardiography (SCG) signals primarily record chest wall vibrations caused by the heart during contraction and relaxation, providing important information related to the heart's mechanical function (such as ventricular contraction, valve movement, and myocardial activity), thereby assisting medical staff in assessing heart health.

[0063] Unless otherwise defined, all technical and scientific terms used in the specification of this application have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in the specification of this application includes any and all combinations of one or more of the relevant listed items.

[0064] The specific implementation of this application is further described below with reference to the accompanying drawings.

[0065] refer to Figure 1 FIG. 1 is a flow chart of a method for measuring a seismogram signal based on structured light provided in the first aspect of the embodiment of the present application. Figure 1 In the embodiment, the structured light-based seismogram signal measurement method includes:

[0066] S1. Use the target sensor to obtain the structured light speckle video of the user's chest and abdomen.

[0067] When capturing the structured light speckle video, this embodiment of the present application uses structured light as a light source to illuminate the user's chest and abdomen, with the target sensor's lens simultaneously focused on the user's chest and abdomen. The structured light consists of multiple infrared laser beams and complies with safety standards (Class I).

[0068] Compared with a single laser beam, the structured light can cover a larger area on the chest and abdomen of the user, thereby effectively solving the problem of relying on manual positioning in traditional single-point laser defocus vibrometer methods.

[0069] The structured light speckle video is a defocused speckle video captured at a high frame rate of 200 fps (Frames Per Second) using a high-speed camera with a defocus lens. Each frame in the structured light speckle video is a speckle image. In other embodiments, the frame rate of the high-speed camera can be set by those skilled in the art based on actual needs.

[0070] In speckle images, due to the influence of cardiac and respiratory motion, the internal texture of the speckle image near the heart region will produce periodic movement with the regular heartbeat; while the speckle image near the abdomen region will produce overall or local regular movement due to respiratory motion. These regular changes provide an important basis for non-invasive detection of seismogram and respiratory signals.

[0071] The realization of the defocus of the high-speed camera can refer to Figure 2 As shown, the high-speed camera ( Figure 2The distance between the focal plane of the high-speed camera (referred to as "camera" in the figure) and the lens of the high-speed camera is set to L2. L2 is a numerical value that can be set by those skilled in the art based on actual needs. The distance between the focal plane of the high-speed camera and the chest and abdomen of the user is set to L1. L1 is a numerical value that can be set by those skilled in the art based on actual needs. Wherein, L1 is greater than L2. By setting the distances of L1 and L2, the speckle image formed by structured light interference can be significantly amplified.

[0072] The amplification effect of speckle is closely related to the defocus degree of the lens, which can be represented by L1 / L2. The motion amplification gain of the target seismogram signal is approximately proportional to L1 / L2.

[0073] S2. Performing region extraction processing on the structured light speckle video based on a speckle positioning method and a region screening method to obtain at least one first target speckle region.

[0074] Further, refer to Figure 3 As shown, step S2 includes:

[0075] S21, using the speckle positioning method to filter the speckle area of ​​each frame image in the structured light speckle video to obtain multiple second target speckle areas. Figure 4 As shown in the "Structured light speckle image", "Speckle positioning" and "Second target speckle area" steps.

[0076] Further, refer to Figure 5 As shown, step S21 includes:

[0077] S211 : Preprocess the structured light speckle video to obtain a noise-reduced video.

[0078] Further, refer to Figure 6 As shown, step S211 includes:

[0079] S2111 . Binarize the structured light speckle video to obtain a grayscale video.

[0080] Specifically, in an embodiment of the present application, a Bradley threshold algorithm (also known as the Bradley threshold algorithm), a Wolf-Jolion adaptive threshold algorithm (also known as the Wolf-Jolion adaptive threshold algorithm), a Gaussian threshold algorithm, or a Sauvola adaptive threshold algorithm (also known as the Sauvola adaptive threshold algorithm) may be used to perform binarization processing on each frame of the structured light speckle video, thereby obtaining a grayscale image corresponding to each frame of the structured light speckle video; then, all the grayscale images are arranged in the same image sequence as the structured light speckle video to form the grayscale video.

[0081] The above threshold algorithm can dynamically adjust the threshold according to local characteristics, thereby effectively distinguishing the speckle area from the background in the image.

[0082] S2112: Perform morphological operations on the grayscale video to obtain the noise-reduced video.

[0083] Specifically, the grayscale video is first corroded to obtain a corroded video; and then the corroded video is expanded to obtain the noise-reduced video.

[0084] The erosion process can remove small noise points in each frame of the grayscale video, and the dilation process can restore the integrity of the speckle area in each frame of the eroded video.

[0085] Step S2112 can also remove background noise and other non-speckle areas in the grayscale video.

[0086] S212: Using the speckle location method, perform screening processing on the speckle region of each frame image in the noise reduction video to obtain the plurality of second target speckle regions.

[0087] Further, refer to Figure 7 As shown, step S212 includes:

[0088] S2121. Set a circular template.

[0089] Since speckles usually present a circular structure, a circular template is used in the embodiment of the present application. In other embodiments, those skilled in the art can set the shape of the template according to actual needs.

[0090] S2122: Perform matching processing on the circular template and all speckle areas of each frame image in the noise reduction video, and use all successfully matched speckle areas as the multiple second target speckle areas.

[0091] Furthermore, the embodiment of the present application uses an image convolutional neural network to perform matching processing on the circular template and all speckle areas of each frame image in the noise reduction video.

[0092] Further, refer to Figure 8 As shown, the steps of using an image convolutional neural network to match the circular template with all speckle areas of each frame image in the noise reduction video include:

[0093] S21221. Set the convolution kernel of the image convolutional neural network to match the circular template.

[0094] S21222: Slide the convolution kernel on each frame of the denoised video, and calculate a first matching degree between each speckle area corresponding to the convolution kernel and the circular template.

[0095] S21223: When the first matching degree reaches a first preset threshold, it is considered that the matching is successful, and the speckle area corresponding to the first matching degree is used as the second target speckle area.

[0096] The first preset threshold can be set by those skilled in the art according to actual needs, and this application does not impose any further restrictions on this.

[0097] In other embodiments, reference Figure 9 As shown, step S212 includes:

[0098] S21201. Set a circular template.

[0099] S21202: Perform matching processing on the circular template and all speckle areas of each frame image in the noise reduction video, and use all speckle areas that are successfully matched as speckle areas to be screened out.

[0100] Furthermore, the embodiment of the present application uses an image convolutional neural network to perform matching processing on the circular template and all speckle areas of each frame image in the noise reduction video.

[0101] Further, refer to Figure 10 As shown, the steps of using an image convolutional neural network to match the circular template with all speckle areas of each frame image in the noise reduction video include:

[0102] S212021. Set the convolution kernel of the image convolutional neural network to match the circular template.

[0103] S212022. The convolution kernel slides on each frame of the denoised video, and calculates a second matching degree between each speckle area corresponding to the convolution kernel and the circular template.

[0104] S212023: When the second matching degree reaches a third preset threshold, the matching is deemed successful, and the speckle area corresponding to the second matching degree is used as the speckle area to be screened out.

[0105] The third preset threshold can be set by those skilled in the art according to actual needs, and this application does not impose any further restrictions on this.

[0106] S21203: Calculate the degree of overlap between each speckle area to be removed and the circular template, and when the degree of overlap reaches a second preset threshold, use the speckle area to be removed corresponding to the degree of overlap as the second target speckle area.

[0107] The degree of overlap can be understood as the overlapping area.

[0108] The specific value of the second preset threshold can be set by those skilled in the art according to actual needs.

[0109] The implementation of steps S21201-S21203 can eliminate the incorrectly matched areas and ensure the accuracy of speckle positioning.

[0110] S22: Performing area screening processing on the plurality of second target speckle areas using the area screening method to obtain the at least one first target speckle area.

[0111] Further, refer to Figure 11 As shown, step S22 includes:

[0112] S221, using the optical flow method of the region screening method, performing signal extraction processing on each second target speckle region corresponding to each frame image in the structured light speckle video or the noise reduction video, to obtain a seismogram signal to be evaluated corresponding to each second target speckle region. Figure 4 The "Optical flow method" and "Seismogram signal to be evaluated" steps in the .

[0113] Further, refer to Figure 13 As shown, step S221 includes:

[0114] S2211. Estimate, using the optical flow method, first target features of each pixel in a candidate target speckle region corresponding to each frame image in the structured light speckle video or the denoised video in multiple first predetermined directions between consecutive frames, where the candidate target speckle region is any one of all the second target speckle regions.

[0115] The plurality of first predetermined directions are horizontal directions and vertical directions. In other embodiments, those skilled in the art can set other directions according to actual needs.

[0116] The first target feature is a velocity component. In other embodiments, those skilled in the art may set it to other features according to actual needs, such as an acceleration component.

[0117] S2212: Accumulate and calculate all first target features of each pixel point in the candidate target speckle area in each first predetermined direction between consecutive frames to obtain a first displacement signal corresponding to each pixel point in the candidate target speckle area in each first predetermined direction between consecutive frames.

[0118] S2213: Calculate an average value of all first displacement signals in each first predetermined direction to obtain a first physiological motion signal corresponding to each first predetermined direction.

[0119] S2214: Fusing all first physiological motion signals in the first predetermined direction to obtain a seismogram signal to be evaluated corresponding to the candidate target speckle region.

[0120] In other implementations, a background subtraction method or a frame difference method may be used to replace the optical flow method, and the specific configuration may be made by those skilled in the art according to actual needs.

[0121] S222: Use the signal evaluation method of the region screening method to evaluate all the seismocardiogram signals to be evaluated, and obtain an evaluation result corresponding to each seismocardiogram signal to be evaluated.

[0122] Furthermore, step S222 includes:

[0123] S2221. Calculate the signal-to-noise ratio of each seismocardiogram signal to be evaluated, and use the signal-to-noise ratio of each seismocardiogram signal to be evaluated as an evaluation result.

[0124] In other embodiments, step S222 includes:

[0125] S2222: Match each seismocardiogram signal to be evaluated using a standard waveform template to obtain an evaluation result corresponding to each seismocardiogram signal to be evaluated.

[0126] S223: Filter out the at least one first target speckle area from the plurality of second target speckle areas based on the preset conditions and all the evaluation results. Figure 12 In the "Seismogram Signal to be Evaluated" and "First Target Speckle Region" steps.

[0127] Further, refer to Figure 14 As shown, step S223 includes:

[0128] S2231. Sort all evaluation results and take the evaluation results that meet the preset conditions after sorting as the target results.

[0129] The sorting process is from large to small or from small to large. The specific sorting method can be set by those skilled in the art according to actual needs.

[0130] When the sorting process is from largest to smallest, the preset condition is that the evaluation results are ranked in the top 20% or the top 30%. Of course, in other embodiments, the evaluation results may also be ranked in the top 50%. The specific value can be set by those skilled in the art according to actual needs.

[0131] In other embodiments, when the sorting process is performed in ascending order, the preset condition is that the evaluation results are ranked in the bottom 20% or the bottom 30%. In other embodiments, the preset condition may be that the evaluation results are ranked in the bottom 50%. The specific value may be set by those skilled in the art according to actual needs.

[0132] S2232: Use the second target speckle areas corresponding to all target results as the first target speckle areas.

[0133] Each of the first target speckle regions is a high-quality speckle region.

[0134] In other embodiments, before step S222, step S22 further includes: performing detrending and filtering processing on all seismocardiogram signals to be evaluated to obtain updated seismocardiogram signals to be evaluated. Figure 4 The "Detrending Filtering" step in

[0135] In the case where there is a step of "detrending and filtering all seismocardiogram signals to be evaluated to obtain updated seismocardiogram signals to be evaluated", step S222 uses the signal evaluation method of the regional screening method to evaluate all updated seismocardiogram signals to be evaluated to obtain an evaluation result corresponding to each seismocardiogram signal to be evaluated.

[0136] The implementation of the above steps can eliminate low-frequency drift or background noise in all seismocardiogram signals to be evaluated, and further enhance the quality of the updated seismocardiogram signals to be evaluated.

[0137] S3, performing signal extraction processing on at least one first target speckle region corresponding to each frame of the structured light speckle video to obtain a target seismogram signal. Figure 12 Target Seismogram Signal step in .

[0138] The target seismocardiogram signal is used to assist medical personnel in diagnosing heart-related diseases.

[0139] Further, refer to Figure 15 As shown, the S3 step includes:

[0140] S31. When the at least one first target speckle region is one first target speckle region, perform signal extraction processing on the first target speckle region corresponding to each frame of the structured light speckle video to obtain the target seismocardiogram signal.

[0141] Further, refer to Figure 16 As shown, step S31 includes:

[0142] S311, using the optical flow method to estimate the second target features of each pixel in the first target speckle area corresponding to each frame image in the structured light speckle video in multiple second predetermined directions between consecutive frames. Figure 12 The "Optical Flow" step in

[0143] The plurality of second predetermined directions are horizontal directions and vertical directions. In other embodiments, those skilled in the art can set other directions according to actual needs.

[0144] The second target feature is a velocity component. In other embodiments, those skilled in the art can set it to other features according to actual needs, such as an acceleration component.

[0145] S312: Accumulate and calculate all second target features of each pixel point in the first target speckle area in each second predetermined direction between consecutive frames to obtain a second displacement signal corresponding to each pixel point in the first target speckle area in each second predetermined direction between consecutive frames.

[0146] S313: Calculate the average value of all second displacement signals in each second predetermined direction to obtain a second physiological motion signal corresponding to each second predetermined direction.

[0147] S314: Fusing all second physiological motion signals in the second predetermined direction to obtain a target seismogram signal corresponding to the first target speckle area.

[0148] S32. When the at least one first target speckle area is a plurality of first target speckle areas, perform the following steps S33 and S34.

[0149] S33 , performing signal extraction processing on each first target speckle region corresponding to each frame of the structured light speckle video to obtain an initial seismogram signal corresponding to each first target speckle region.

[0150] Further, refer to Figure 17 As shown, step S33 includes:

[0151] S331 , using the optical flow method to estimate third target features of each pixel point in each first target speckle area corresponding to each frame of the structured light speckle video in multiple third predetermined directions between consecutive frames.

[0152] The plurality of third predetermined directions are horizontal directions and vertical directions. In other embodiments, those skilled in the art can set other directions according to actual needs.

[0153] The third target feature is a velocity component. In other embodiments, those skilled in the art may set it to other features according to actual needs, such as an acceleration component.

[0154] S332: Accumulate and calculate all third target features of each pixel point in each first target speckle area in each third predetermined direction between consecutive frames to obtain a third displacement signal corresponding to each pixel point in each first target speckle area in each third predetermined direction between consecutive frames.

[0155] S333: Calculate an average value of all third displacement signals in each third predetermined direction to obtain a third physiological motion signal corresponding to each third predetermined direction of each first target speckle area.

[0156] S334: Fusing all third physiological motion signals in the third predetermined direction to obtain an initial seismogram signal corresponding to each first target speckle area.

[0157] S34. Using principal component analysis to fuse all initial seismocardiogram signals, to obtain the target seismocardiogram signal.

[0158] In some embodiments, reference Figure 15 As shown, step S3 also includes:

[0159] S35. Perform noise reduction processing on the target seismocardiogram signal to obtain a high-quality seismocardiogram signal.

[0160] The noise reduction process may be a detrending filtering process. In other embodiments, the noise reduction process may also be other noise reduction operations, which may be specifically configured by those skilled in the art according to actual needs.

[0161] The implementation of step S35 can further enhance the quality and stability of the target seismocardiogram signal.

[0162] Through the above-described implementation, the present embodiment uses a target sensor to capture a structured light speckle video. The speckle localization method and region screening method are then used to locate the structured light speckle video, obtaining at least one first target speckle region. This at least one first target speckle region is then used to extract a target seismocardiogram signal from each frame of the structured light speckle video. Compared to the prior art, which uses a seismocardiogram signal sensor to acquire the original seismocardiogram signal and an auxiliary sensor to acquire noise data, and then uses both the original seismocardiogram signal and the noise data to derive the final seismocardiogram signal, the present embodiment uses only one sensor, the target sensor, to capture one type of video data (i.e., the structured light speckle video). This effectively avoids the problem of shooting position deviation caused by two sensors collecting two different types of data, thereby reducing data errors and improving the accuracy of seismocardiogram signal measurement.

[0163] Furthermore, the embodiments of the present application use a speckle localization method and a region screening method to perform speckle localization on a structured light speckle video, which can improve the accuracy of screening and extracting regions for seismocardiogram signals, and further improve the accuracy of seismocardiogram signal measurement.

[0164] In some embodiments, the structured light-based seismogram signal measurement method further includes:

[0165] S4. Perform feature extraction on the target seismocardiogram signal or the high-quality seismocardiogram signal to obtain heartbeat period and amplitude changes.

[0166] The implementation of step S4 can be used to assist medical personnel in assessing the user's heart health.

[0167] In other embodiments, other characteristic parameters may be extracted from the target seismocardiogram signal or the high-quality seismocardiogram signal, such as phase difference, time difference and other parameters between cardiac vibrations at different positions, which may be specifically set by those skilled in the art according to actual needs.

[0168] In other embodiments, the at least one target speckle area may be obtained by capturing a global light spot in advance and determining a specific light spot position therein, and then using an automatic zoom lens to lock the selected light spot position to ensure that only the selected light spot is retained in the image area, thereby obtaining the at least one target speckle area.

[0169] refer to Figure 18 FIG. 1 is a block diagram showing the principle of a structured light-based seismogram signal measurement device provided in the second aspect of the present invention. Figure 18 In the embodiment, the structured light-based seismogram signal measurement device 100 includes:

[0170] The video acquisition module 101 is used to acquire a structured light speckle video of the user's chest and abdomen using a target sensor;

[0171] A region extraction module 102 is configured to perform region extraction processing on the structured light speckle video based on a speckle positioning method and a region screening method to obtain at least one first target speckle region;

[0172] The signal extraction module 103 is configured to perform signal extraction processing on at least one first target speckle region corresponding to each frame of the structured light speckle video to obtain a target seismogram signal.

[0173] The third aspect of the embodiment of the present application provides a terminal device, the principle block diagram of the terminal device can be as follows: Figure 19 As shown. The terminal device includes a processor, a memory, a network interface, a display screen and a temperature sensor connected via a system bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a structured light-based seismogram signal measurement method is implemented. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0174] Those skilled in the art will understand that Figure 19 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal device to which the solution of the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0175] In some embodiments, embodiments of the present application provide a terminal device comprising a processor and a memory, the memory being configured to store a computer program, the processor being configured to call and execute the computer program stored in the memory to perform the steps of the structured light-based seismocardiogram signal measurement method provided in the first aspect of the embodiments of the present application. A fourth aspect of embodiments of the present application provides a computer-readable storage medium, the computer-readable storage medium being configured to store a computer program that causes a computer to perform the steps of the structured light-based seismocardiogram signal measurement method provided in the first aspect of the embodiments of the present application.

[0176] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0177] The technical features of the above embodiments can be combined without changing the basic principles of this application. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0178] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of patent protection for the present application shall be determined by the appended claims.

Claims

1. A method for measuring seismogram signals based on structured light, characterized in that: include: A target sensor is used to obtain structured light speckle video of the user's chest and abdomen; Performing region extraction processing on the structured light speckle video based on a speckle positioning method and a region screening method to obtain at least one first target speckle region; performing signal extraction processing on at least one first target speckle region corresponding to each frame of the structured light speckle video to obtain a target seismocardiogram signal; The step of performing region extraction processing on the structured light speckle video based on a speckle location method and a region screening method to obtain at least one first target speckle region includes: using the speckle location method to screen the speckle region of each frame image in the structured light speckle video to obtain a plurality of second target speckle regions; and using the region screening method to screen the plurality of second target speckle regions to obtain the at least one first target speckle region. The step of using the speckle localization method to filter the speckle area of ​​each frame image in the structured light speckle video to obtain a plurality of second target speckle areas includes: preprocessing the structured light speckle video to obtain a noise reduction video; and using the speckle localization method to filter the speckle area of ​​each frame image in the noise reduction video to obtain the plurality of second target speckle areas. The step of screening the speckle areas of each frame of the denoised video using the speckle localization method to obtain the multiple second target speckle areas includes: setting a circular template; matching the circular template with all speckle areas of each frame of the denoised video, and using all successfully matched speckle areas as the multiple second target speckle areas; The step of performing regional screening processing on the multiple second target speckle regions using the regional screening method to obtain the at least one first target speckle region includes: performing signal extraction processing on each second target speckle region corresponding to each frame of the structured light speckle video or the noise reduction video using an optical flow method using the regional screening method to obtain a seismocardiogram signal to be evaluated corresponding to each second target speckle region; performing evaluation processing on all seismocardiogram signals to be evaluated using a signal evaluation method using the regional screening method to obtain an evaluation result corresponding to each seismocardiogram signal to be evaluated; and screening out the at least one first target speckle region from the multiple second target speckle regions based on preset conditions and all the evaluation results.

2. The method for measuring seismogram signals based on structured light according to claim 1, wherein: The signal evaluation method of the regional screening method is used to evaluate and process all seismocardiogram signals to be evaluated, and the steps of obtaining an evaluation result corresponding to each seismocardiogram signal to be evaluated include: Calculating the signal-to-noise ratio of each seismocardiogram signal to be evaluated, and using the signal-to-noise ratio of each seismocardiogram signal to be evaluated as an evaluation result; or, A standard waveform template is used to match each seismocardiogram signal to be evaluated to obtain an evaluation result corresponding to each seismocardiogram signal to be evaluated.

3. The method for measuring seismocardiogram signals based on structured light according to claim 1, wherein: The step of performing signal extraction processing on at least one first target speckle area corresponding to each frame of the structured light speckle video to obtain a target seismocardiogram signal includes: When the at least one first target speckle region is one first target speckle region, performing signal extraction processing on the first target speckle region corresponding to each frame of the structured light speckle video to obtain the target seismocardiogram signal; When the at least one first target speckle area is a plurality of first target speckle areas, the following steps are performed: performing signal extraction processing on each first target speckle region corresponding to each frame of the structured light speckle video to obtain an initial seismogram signal corresponding to each first target speckle region; All initial seismocardiogram signals are fused using principal component analysis to obtain the target seismocardiogram signal.

4. A device for measuring seismogram signals based on structured light, characterized in that: include: A video acquisition module, configured to acquire a structured light speckle video of the user's chest and abdomen using a target sensor; a region extraction module, configured to perform region extraction processing on the structured light speckle video based on a speckle positioning method and a region screening method to obtain at least one first target speckle region; a signal extraction module, configured to perform signal extraction processing on at least one first target speckle region corresponding to each frame of the structured light speckle video to obtain a target seismocardiogram signal; The region extraction module is further configured to use the speckle localization method to filter the speckle region of each frame image in the structured light speckle video to obtain a plurality of second target speckle regions; and use the region screening method to perform region screening on the plurality of second target speckle regions to obtain the at least one first target speckle region. The region extraction module is further configured to pre-process the structured light speckle video to obtain a noise reduction video; and to filter the speckle regions of each frame of the noise reduction video using the speckle localization method to obtain the plurality of second target speckle regions. The region extraction module is also used to set a circular template; performing matching processing on the circular template and all speckle regions of each frame image in the noise reduction video, and taking all successfully matched speckle regions as the multiple second target speckle regions; The region extraction module is further configured to perform signal extraction processing on each second target speckle region corresponding to each frame of the structured light speckle video or the noise reduction video using the optical flow method of the region screening method to obtain a seismocardiogram signal to be evaluated corresponding to each second target speckle region; perform evaluation processing on all seismocardiogram signals to be evaluated using the signal evaluation method of the region screening method to obtain an evaluation result corresponding to each seismocardiogram signal to be evaluated; and screen out the at least one first target speckle region from the multiple second target speckle regions based on a preset condition and all the evaluation results.

5. A terminal device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the steps of the structured light-based seismogram signal measurement method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that Used to store a computer program, wherein the computer program enables a computer to execute the steps of the method for measuring seismogram signals based on structured light according to any one of claims 1 to 3.

Images