Method, device, system and medium for measuring carotid pulse based on defocus speckle imaging

Abstract

The application relates to the technical field of biomedical engineering. The application discloses a jugular vein pulsation measurement method, device, system and medium based on defocus speckle imaging, which can improve the sensitivity and accuracy of measuring the jugular vein pulsation signal. The method comprises the following steps: after positioning the vein part of the neck of a target object, performing defocus speckle video acquisition processing to obtain a target defocus speckle video; performing directional motion signal extraction processing on the target defocus speckle video to obtain a target directional motion signal corresponding to each pixel point in a target image, the target image being one frame of image of the target defocus speckle video; and performing average value calculation processing on the target directional motion signals corresponding to all pixels of the target image to obtain the jugular vein pulsation signal of the target object.

Description

Technical Field

[0001] This application relates to the field of biomedical engineering technology. More specifically, this application relates to a method, apparatus, system, and medium for measuring jugular vein pulsation based on defocus speckle imaging. Background Technology

[0002] Traditional methods for measuring jugular vein pulsation involve directly photographing the neck of the subject to obtain a video image. The pulsation signal is then measured by tracking the subtle movements of the neck skin within the video. However, because the jugular vein pulsation is extremely weak and often interfered with or masked by the pulsation of the adjacent carotid artery, directly photographing the neck without additional auxiliary techniques cannot accurately capture changes in the jugular vein pulsation, thus reducing the sensitivity of the measurement. Furthermore, photographing the neck requires a high frame rate. However, under normal lighting conditions, a high frame rate compresses the camera's exposure time, significantly reducing the signal-to-noise ratio of the acquired image and affecting the accuracy of the jugular vein pulsation measurement. Therefore, existing techniques require improvement. Summary of the Invention

[0003] The purpose of this application is to provide a method, device, system, and medium for measuring jugular vein pulsation based on defocus speckle imaging, which can improve the sensitivity and accuracy of measuring jugular vein pulsation signals. This application is mainly achieved through the following technical solutions:

[0004] A first aspect of this application provides a method for measuring jugular vein pulsation based on defocus speckle imaging, comprising:

[0005] After locating the veins in the neck of the target object, defocus speckle video acquisition and processing are performed to obtain the target defocus speckle video.

[0006] The directional motion signal extraction process is performed on the target defocus speckle video to obtain the target directional motion signal corresponding to each pixel in the target image, wherein the target image is one frame of the target defocus speckle video.

[0007] The average value of the motion signals in the target direction corresponding to all pixels of the target image is calculated to obtain the jugular vein pulsation signal of the target object.

[0008] According to one embodiment of this application, the steps of locating the veins in the neck of the target object and then performing defocus speckle video acquisition processing to obtain the target defocus speckle video include:

[0009] Infrared image acquisition and processing are performed on the vein area of ​​the neck of the target object to obtain at least one first infrared image;

[0010] The vein location is localized in the at least one first infrared image to obtain the first target region of the neck.

[0011] The first target area is subjected to defocus speckle video acquisition and processing to obtain the target defocus speckle video.

[0012] According to one embodiment of this application, the steps of locating the veins in the neck of the target object and then performing defocus speckle video acquisition processing to obtain the target defocus speckle video include:

[0013] Preliminary localization of the veins in the neck of the target object is performed to obtain the second target area;

[0014] Set preset conditions;

[0015] If the preset conditions are not met, the following steps are executed repeatedly: video acquisition processing is performed on the second target area to obtain a sub-video, and the sub-video is added to the processing set; the vein is repositioned based on a preset time interval to obtain a third target area, and the third target area is used as the second target area;

[0016] After completing the above loop steps, all the sub-videos in the set to be processed are stitched together to obtain the target out-of-focus speckle video.

[0017] According to one embodiment of this application, the step of performing directional motion signal extraction processing on the target defocus speckle video to obtain the target directional motion signal corresponding to each pixel in the target image includes:

[0018] A motion estimation algorithm is used to calculate the pixel displacement of each pixel in the target defocus speckle video relative to the reference image, so as to obtain the pixel displacement vector corresponding to each pixel.

[0019] Each pixel displacement vector corresponding to a pixel point is processed by pixel component calculation to obtain the target component corresponding to each pixel point; or, each pixel displacement vector corresponding to a pixel point is projected onto the direction of the vein in the vein part to obtain the target component corresponding to each pixel point.

[0020] The target components corresponding to all pixels at the same pixel position as each pixel in the target image are accumulated to obtain the target direction motion signal corresponding to each pixel in the target image.

[0021] According to one embodiment of this application, the step of performing pixel component calculation processing on the pixel displacement vector corresponding to each pixel to obtain the target component corresponding to each pixel includes:

[0022] The first displacement vectors of the pixel displacement vectors corresponding to all pixels at the same pixel position in all images of the target defocus speckle video are accumulated and calculated to obtain the first displacement signal corresponding to each pixel position.

[0023] The second displacement vectors of the pixel displacement vectors corresponding to all pixels at the same pixel position in all images of the target defocus speckle video are accumulated and calculated to obtain the second displacement signal corresponding to each pixel position.

[0024] Pixel component calculations are performed on the pixel displacement vector corresponding to each pixel point, the first displacement signal corresponding to each pixel position, and the second displacement signal corresponding to each pixel position to generate the target component corresponding to each pixel point.

[0025] According to one embodiment of this application, the calculation formula for generating the target component corresponding to each pixel is as follows: (This formula is not provided in the original text.)

[0026] ;

[0027] in, It is the first The target components corresponding to each pixel; It is the first The first displacement vector of the pixel displacement vector corresponding to each pixel; It is the first The first displacement signal corresponding to each pixel position; It is the first The second displacement vector corresponding to the pixel displacement vector of each pixel; It is the first The second displacement signal corresponding to each pixel position.

[0028] According to one embodiment of this application, the calculation formula for the step of accumulating the target components corresponding to all pixels at the same pixel position as each pixel in the target image to obtain the target direction motion signal corresponding to each pixel in the target image is as follows:

[0029] ;

[0030] in, It is the first in the target image The target direction motion signal corresponding to each pixel. It is the total number of all pixels that are at the same pixel position as each pixel in the target image.

[0031] A second aspect of this application provides a device for measuring jugular vein pulsation based on defocus speckle imaging, comprising:

[0032] The target defocus speckle video acquisition module is used to locate the vein in the neck of the target object and then perform defocus speckle video acquisition and processing to obtain the target defocus speckle video.

[0033] The signal extraction module is used to perform directional motion signal extraction processing on the target defocus speckle video to obtain the target directional motion signal corresponding to each pixel in the target image, wherein the target image is one frame of the target defocus speckle video.

[0034] The calculation and processing module is used to calculate the average value of the motion signals in the target direction corresponding to all pixels of the target image to obtain the jugular vein pulsation signal of the target object.

[0035] A third aspect of this application provides a system for measuring jugular vein pulsation based on defocus speckle imaging, comprising: an infrared light source device, a laser, a camera, and a terminal device. The terminal device includes a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the steps of the method for measuring jugular vein pulsation based on defocus speckle imaging provided in the first aspect of this application.

[0036] The processor is communicatively connected to the infrared light source device, the laser, and the camera.

[0037] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that causes a computer to perform the steps of the method for measuring jugular vein pulsation based on defocus speckle imaging provided in the first aspect of this application.

[0038] The beneficial effects of the embodiments of this application include:

[0039] This application employs a strategy of locating the neck first and then capturing video to overcome the problems of low sensitivity and low accuracy of jugular vein pulsation signals. Specifically, this application performs location processing on the veins in the neck of the target object, followed by defocus speckle video acquisition processing to obtain a target defocus speckle video. The target defocus speckle video is then processed to extract directional motion signals, obtaining the target directional motion signal corresponding to each pixel in the target image, where the target image is one frame of the target defocus speckle video. The average value of the target directional motion signals corresponding to all pixels in the target image is then calculated to obtain the jugular vein pulsation signal of the target object. Compared with existing technologies that directly capture images of the neck without employing other auxiliary techniques, the technique of locating the neck first and then capturing video, as used in this application, can accurately capture changes in the jugular vein pulsation, thereby improving the sensitivity of measuring the jugular vein pulsation signal. Furthermore, the embodiments of this application acquire target defocus speckle video, which is formed by defocus speckle interferometry imaging, which can significantly amplify the movement of the vein and improve the signal-to-noise ratio of the image, thereby improving the accuracy of measuring the jugular vein pulsation signal. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 The flowcharts for some embodiments of the method for measuring jugular vein pulsation based on defocus speckle imaging according to this application are shown below.

[0042] Figure 2 The flowcharts for the method of measuring jugular vein pulsation based on defocus speckle imaging in this application are shown in some other embodiments.

[0043] Figure 3 The flowcharts are shown in some embodiments of the method for measuring jugular vein pulsation based on defocus speckle imaging according to this application.

[0044] Figure 4 This is a reference diagram of the jugular vein pulsation signal in this application;

[0045] Figure 5 The flowcharts for the method of measuring jugular vein pulsation based on defocus speckle imaging in this application are shown in some of the embodiments.

[0046] Figure 6This is a schematic diagram of the jugular vein pulsation measurement device based on defocus speckle imaging according to this application.

[0047] Figure 7 This is a block diagram illustrating the principle of the jugular vein pulsation measurement system based on defocus speckle imaging according to this application. Detailed Implementation

[0048] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0049] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0050] The terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0051] The terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or that are inherent to such process, method, product, or apparatus.

[0052] The term "jugular pulsation" is a key clinical diagnostic sign, primarily based on pressure changes in the right atrium. This pulsation is transmitted to the body surface via the internal jugular vein, exhibiting periodic and dynamic characteristics, and directly reflecting the hemodynamic state of the right atrium. The jugular pulsation waveform (i.e., the jugular pulsation signal) contains three characteristic waves (a, c, and v) and two descending branches (x and y). Any deviation from the morphology of the jugular pulsation waveform may be closely related to cardiac structural or functional abnormalities such as valvular defects or ventricular remodeling.

[0053] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items.

[0054] The specific embodiments of this application will be further described below with reference to the accompanying drawings.

[0055] refer to Figure 1 The diagram shown is a flowchart of a method for measuring jugular vein pulsation based on defocus speckle imaging, provided in the first aspect of an embodiment of this application. Figure 1 The method for measuring jugular vein pulsation based on defocus speckle imaging includes:

[0056] S1. After locating the veins in the neck of the target object, perform defocus speckle video acquisition and processing to obtain the target defocus speckle video.

[0057] Further, refer to Figure 2 and Figure 3 As shown, step S1 includes:

[0058] S11. Perform infrared image acquisition and processing on the vein area of ​​the neck of the target object to obtain at least one first infrared image.

[0059] More specifically, when the infrared light source device of the jugular vein pulsation measurement system based on defocus speckle imaging emits an 850-nanometer infrared light source to illuminate the vein, the camera of the jugular vein pulsation measurement system based on defocus speckle imaging is used to acquire the at least one first infrared image.

[0060] The camera is in a defocused state. This defocus setting allows the laser speckle motion caused by minute skin deformation due to jugular vein pulsation to be significantly amplified, thereby improving the accuracy of jugular vein pulsation signal extraction.

[0061] S12. Perform vein localization processing on the at least one first infrared image to obtain the first target region of the neck.

[0062] Further, step S12 includes: performing neck vein segmentation and recognition processing on each first infrared image using an image segmentation algorithm to obtain a segmentation probability map corresponding to each first infrared image; performing binarization processing on the segmentation probability map corresponding to each first infrared image to obtain a binary mask corresponding to each first infrared image; performing erosion followed by dilation (i.e., opening operation) processing on the binary mask corresponding to each first infrared image to obtain a denoised image corresponding to each first infrared image; performing dilation followed by erosion (i.e., closing operation) processing on the denoised image corresponding to each first infrared image to fill the area covered by the neck vein in the denoised image corresponding to each first infrared image to obtain a filled image corresponding to each first infrared image; and selecting the area with the largest filled area as the first target area of ​​the neck from all the filled images corresponding to the first infrared images.

[0063] The image segmentation algorithm can be the Canny edge detection algorithm or the Otsu algorithm. In other embodiments, the image segmentation algorithm can also be other algorithms, which can be set according to actual needs by those skilled in the art.

[0064] The image segmentation algorithm can automatically and accurately identify and locate the jugular vein, thereby improving the accuracy of the first target region.

[0065] S13. Perform defocus speckle video acquisition and processing on the first target area to obtain the target defocus speckle video.

[0066] When performing steps S11 to S13 in this embodiment, it can be understood that the camera, the infrared light source device, and the laser of the jugular vein pulsation measurement system based on defocus speckle imaging are in a time-differential working mode.

[0067] The core of the time-differential operating mode lies in the fact that the infrared light source device, the laser, and the camera operate in a time-separated manner to avoid crosstalk between light sources or signals. When the light source emitted by the infrared light source device and the light source emitted by the laser are the same, interference from light sources in the same band cannot be removed by the filter. Therefore, the embodiments of this application must adopt the time-differential operating mode.

[0068] In another embodiment, when the present application embodiment performs steps S11 to S13, it can also be understood that the three hardware devices, namely the camera, the infrared light source device and the laser, are in a time-synchronized working mode.

[0069] The core of the time-synchronous working mode lies in utilizing light sources with different spectra and corresponding filtering / separation techniques to enable the camera, the infrared light source device, and the laser to work in parallel, achieving true real-time positioning and measurement. This time-synchronous working mode ensures that even if the target object undergoes minute movement, the laser speckle pattern will always accurately illuminate the target vein.

[0070] S2. Perform directional motion signal extraction processing on the target defocus speckle video to obtain the target directional motion signal corresponding to each pixel in the target image, wherein the target image is one frame of the target defocus speckle video.

[0071] Further, the step of extracting directional motion signals from the target defocus speckle video to obtain the target direction motion signal corresponding to each pixel in the target image includes: calculating the pixel displacement of each pixel in the target defocus speckle video relative to the reference image using a motion estimation algorithm to obtain the pixel displacement vector corresponding to each pixel; performing pixel component calculation on the pixel displacement vector corresponding to each pixel to obtain the target component corresponding to each pixel, or projecting the pixel displacement vector corresponding to each pixel onto the direction of the vein in the vein area to obtain the target component corresponding to each pixel; and accumulating the target components corresponding to all pixels at the same pixel position as each pixel in the target image to obtain the target direction motion signal corresponding to each pixel in the target image.

[0072] The motion estimation algorithm can be an optical flow method, a background subtraction method, or a frame difference method. The optical flow method can be the Farneback algorithm (also known as the Farneback optical flow algorithm) or other dense optical flow algorithms, or it can also be a sparse optical flow algorithm. In other embodiments, the motion estimation algorithm is not limited to an optical flow method, a background subtraction method, or a frame difference method, and can be specifically set by those skilled in the art according to actual needs.

[0073] The reference image is relative to the current pixel in the target defocus speckle video. For example, if the current pixel is a pixel in the first frame of the target defocus speckle video, then the reference image is the second frame of the target defocus speckle video; if the current pixel is a pixel in the second frame of the target defocus speckle video, then the reference image is the third frame of the target defocus speckle video; if the current pixel is a pixel in the third frame of the target defocus speckle video, then the reference image is the fourth frame of the target defocus speckle video. That is, in the target defocus speckle video, the reference image is the next frame after the image containing the current pixel.

[0074] Further, the step of using a motion estimation algorithm to calculate the pixel displacement of each pixel in the target defocus speckle video relative to the reference image to obtain the pixel displacement vector corresponding to each pixel includes: using the motion estimation algorithm to estimate the first target feature in the horizontal direction between each pixel in the target defocus speckle video and the reference image to obtain the first displacement vector of the pixel displacement vector corresponding to each pixel; using the motion estimation algorithm to estimate the second target feature in the vertical direction between each pixel in the target defocus speckle video and the reference image to obtain the second displacement vector of the pixel displacement vector corresponding to each pixel.

[0075] The first target feature can be a velocity component or an acceleration component. The second target feature can be a velocity component or an acceleration component. The first target feature and the second target feature can be the same or different, and can be set by those skilled in the art according to actual needs. In other embodiments, those skilled in the art can set the first target feature and the second target feature to other features according to actual needs.

[0076] Further, the step of performing pixel component calculation processing on the pixel displacement vector corresponding to each pixel to obtain the target component corresponding to each pixel includes: performing cumulative calculation processing on the first displacement vector of the pixel displacement vector corresponding to all pixels at the same pixel position in all images of the target defocus speckle video to obtain the first displacement signal corresponding to each pixel position; performing cumulative calculation processing on the second displacement vector of the pixel displacement vector corresponding to all pixels at the same pixel position in all images of the target defocus speckle video to obtain the second displacement signal corresponding to each pixel position; and performing pixel component calculation processing on the pixel displacement vector corresponding to each pixel, the first displacement signal corresponding to each pixel position, and the second displacement signal corresponding to each pixel position to generate the target component corresponding to each pixel.

[0077] When the pixel displacement vector corresponding to each pixel point is processed by pixel component calculation, the pixel displacement vector corresponding to each pixel point is projected onto the radial direction, and the target component is the radial component.

[0078] Furthermore, the calculation formula for the step of accumulating and calculating the first displacement vector of the pixel displacement vector corresponding to all pixels at the same pixel position in all images of the target defocus speckle video to obtain the first displacement signal corresponding to each pixel position is as follows:

[0079] ;

[0080] in, It is the first The first displacement signal corresponding to the pixel position of each pixel; It is the first The first displacement vector of the pixel displacement vector corresponding to each pixel; It is the total number of all pixels at the same pixel position in all images of the target defocus speckle video.

[0081] Furthermore, the formula for calculating the second displacement vector of the pixel displacement vector corresponding to all pixels at the same pixel position in all images of the target defocus speckle video, and obtaining the second displacement signal corresponding to each pixel position, is as follows:

[0082] ;

[0083] in, It is the first The second displacement signal corresponding to the pixel position of each pixel; It is the first The second displacement vector corresponding to each pixel displacement vector.

[0084] Furthermore, the calculation formula for generating the target component corresponding to each pixel is as follows: (This involves performing pixel component calculations on the pixel displacement vector corresponding to each pixel point, the first displacement signal corresponding to each pixel position, and the second displacement signal corresponding to each pixel position.)

[0085] ;

[0086] in, It is the first The target components corresponding to each pixel; It is the first The first displacement vector of the pixel displacement vector corresponding to each pixel; It is the first The first displacement signal corresponding to each pixel position; It is the first The second displacement vector corresponding to the pixel displacement vector of each pixel; It is the first The second displacement signal corresponding to each pixel position.

[0087] Because the pulsation center of the jugular vein propagates along the blood vessel, the skin deformation it causes on the skin surface manifests as a complex rotational motion. To accurately capture this pulsation, an analysis center point needs to be set within the first, second, or third target area irradiated by the laser (for example, the geometric center of the first, second, or third target area is used as the analysis center point). Using this analysis center point as the origin, the displacement in the Cartesian coordinate system is transformed to the polar coordinate system to separate the radial displacement component, that is, the target component corresponding to each pixel.

[0088] Furthermore, the calculation formula for the step of accumulating the target components corresponding to all pixels at the same pixel position as each pixel in the target image to obtain the target direction motion signal corresponding to each pixel in the target image is as follows:

[0089] ;

[0090] in, It is the first in the target image The target direction motion signal corresponding to each pixel. It is the total number of all pixels that are at the same pixel position as each pixel in the target image.

[0091] Further, the step of projecting the pixel displacement vector corresponding to each pixel point onto the direction of the vein in the vein area to obtain the target component corresponding to each pixel point includes: performing cumulative calculation on the first displacement vector of the pixel displacement vector corresponding to all pixels at the same pixel position in all odd-numbered frames of the target defocus speckle video to obtain the third displacement signal corresponding to each pixel position in the odd-numbered frames; performing cumulative calculation on the second displacement vector of the pixel displacement vector corresponding to all pixels at the same pixel position in all even-numbered frames of the target defocus speckle video to obtain the fourth displacement signal corresponding to each pixel position in the even-numbered frames; and generating the target component corresponding to each pixel point based on the pixel displacement vector corresponding to each pixel point, the third displacement signal corresponding to each pixel position in the odd-numbered frames, and the fourth displacement signal corresponding to each pixel position in the even-numbered frames.

[0092] For example, the step of generating a target component corresponding to each pixel based on the pixel displacement vector corresponding to each pixel, the third displacement signal corresponding to each pixel position in the odd-numbered frame image, and the fourth displacement signal corresponding to each pixel position in the even-numbered frame image includes: mapping the fourth displacement signal corresponding to the target pixel position in the even-numbered frame image to the pixel position corresponding to the target pixel position in the odd-numbered frame image based on the pixel displacement vector corresponding to the target pixel position, generating a compensation signal corresponding to the target pixel position, wherein the target pixel position is any pixel in the even-numbered frame image; performing residual calculation processing on the third displacement signal corresponding to the pixel position corresponding to the target pixel position in the odd-numbered frame image and the compensation signal corresponding to the target pixel position to obtain the target component corresponding to the target pixel.

[0093] S3. Calculate the average value of the motion signals in the target direction corresponding to all pixels of the target image to obtain the jugular vein pulsation signal of the target object.

[0094] The jugular vein pulsation signal can be used by medical staff to determine the cardiovascular disease of the target patient (such as heart failure, tricuspid valve disease, or pericardial disease), thereby enabling medical staff to improve prognostic risks and reduce the risk of readmission.

[0095] The jugular vein pulsation signal can be referenced. Figure 4 As shown.

[0096] Compared to existing technologies that directly photograph the neck without employing other auxiliary techniques, the present application's embodiment uses a technique of first performing positioning processing and then video acquisition processing. This technique can accurately capture changes in the jugular vein pulsation, thereby improving the sensitivity of measuring the jugular vein pulsation signal. Furthermore, the embodiment of this application acquires target defocus speckle video, which is formed by defocus speckle interferometry imaging. This significantly amplifies the movement of the vein, improves the signal-to-noise ratio of the image, and thus enhances the accuracy of measuring the jugular vein pulsation signal.

[0097] In some implementations, reference Figure 5 As shown, step S1 includes:

[0098] S14. Perform preliminary localization processing on the veins in the neck of the target object to obtain the second target area.

[0099] Further, step S14 includes: performing infrared image acquisition processing on the vein portion of the neck of the target object to obtain at least one second infrared image; and performing vein location processing on the at least one second infrared image to obtain a second target region.

[0100] Further, the step of performing vein localization processing on the at least one second infrared image to obtain the second target region includes: using the image segmentation algorithm to perform neck vein segmentation and recognition processing on each second infrared image to obtain a segmentation probability map corresponding to each second infrared image; performing binarization processing on the segmentation probability map corresponding to each second infrared image to obtain a binary mask corresponding to each second infrared image; performing erosion followed by dilation (i.e., opening operation) processing on the binary mask corresponding to each second infrared image to obtain a denoised image corresponding to each second infrared image; performing dilation followed by erosion (i.e., closing operation) processing on the denoised image corresponding to each second infrared image to fill the area covered by the neck vein in the denoised image corresponding to each second infrared image to obtain a filled image corresponding to each second infrared image; and selecting the region with the largest filled area as the second target region among all the filled images corresponding to the second infrared images.

[0101] S15. Set preset conditions.

[0102] The preset condition is a preset total time or a preset total number of cycles. The specific value of the total time or the total number of cycles can be set by those skilled in the art according to actual needs.

[0103] S16. If the preset conditions are not met, the following steps are executed repeatedly: video acquisition processing is performed on the second target area to obtain a sub-video, and the sub-video is added to the processing set; the vein is repositioned based on a preset time interval to obtain a third target area, and the third target area is used as the second target area.

[0104] The duration of the sub-video can be set by those skilled in the art according to actual needs.

[0105] The specific value of the preset time interval can be set by those skilled in the art according to actual needs.

[0106] The step of repositioning the vein based on a preset time interval to obtain a third target region includes: acquiring infrared images of the vein in the neck of the target object based on a preset time interval to obtain at least one third infrared image; and performing vein localization processing on the at least one third infrared image to obtain a third target region.

[0107] Further, the step of performing vein localization processing on the at least one third infrared image to obtain the third target region includes: using the image segmentation algorithm to perform neck vein segmentation and recognition processing on each third infrared image to obtain a segmentation probability map corresponding to each third infrared image; performing binarization processing on the segmentation probability map corresponding to each third infrared image to obtain a binary mask corresponding to each third infrared image; performing erosion and dilation processing on the binary mask corresponding to each third infrared image to obtain a denoised image corresponding to each third infrared image; performing dilation and erosion processing on the denoised image corresponding to each third infrared image to fill the area covered by the neck vein in the denoised image corresponding to each third infrared image to obtain a filled image corresponding to each third infrared image; and selecting the region with the largest filled area among all the filled images corresponding to the third infrared images as the third target region.

[0108] S17. After completing the above loop steps, all sub-videos in the set to be processed are stitched together to obtain the target defocus speckle video. The "above loop steps" in step S17 refer to the steps in step S16: "Perform video acquisition processing on the second target area to obtain sub-videos, and add the sub-videos to the set to be processed; perform repositioning processing on the vein based on a preset time interval to obtain a third target area, and use the third target area as the second target area."

[0109] When performing steps S14 to S17 in this embodiment, it can be understood as another implementation of the three hardware devices—the camera, the infrared light source device, and the laser—being in a time-differential working mode.

[0110] In some embodiments, after step S3, the method for measuring jugular vein pulsation based on defocus speckle imaging further includes: performing bandpass filtering on the jugular vein pulsation signal to obtain a final signal. The final signal is the signal after removing respiratory and noise interference. The frequency range used for the bandpass filtering is 1.33 to 10 Hz.

[0111] refer to Figure 6 The diagram shown is a schematic block diagram of a jugular vein pulsation measurement device based on defocus speckle imaging, provided in the second aspect of an embodiment of this application. Figure 6 The jugular vein pulsation measuring device 200 based on defocus speckle imaging includes:

[0112] The target defocus speckle video acquisition module 201 is used to locate the vein in the neck of the target object and then perform defocus speckle video acquisition and processing to obtain the target defocus speckle video.

[0113] The signal extraction module 202 is used to perform directional motion signal extraction processing on the target defocus speckle video to obtain the target directional motion signal corresponding to each pixel in the target image, wherein the target image is one frame of the target defocus speckle video.

[0114] The calculation and processing module 203 is used to perform average value calculation processing on the target direction motion signals corresponding to all pixels of the target image to obtain the jugular vein pulsation signal of the target object.

[0115] refer to Figure 7 The diagram shown is a schematic block diagram of a jugular vein pulsation measurement system based on defocus speckle imaging, provided in the third aspect of an embodiment of this application. Figure 7 The jugular vein pulsation measurement system 300 based on defocus speckle imaging includes: an infrared light source device 301, a laser 302, a camera 303, and a terminal device 304. The terminal device 304 includes a processor and a memory. The memory is used to store computer programs, and the processor is used to call and run the computer programs stored in the memory to execute the steps of the jugular vein pulsation measurement method based on defocus speckle imaging provided in the first aspect of the present application. The processor is communicatively connected to the infrared light source device 301, the laser 302, and the camera 303.

[0116] The infrared light source device 301 can emit an 850-nanometer infrared light source array to clearly illuminate the jugular vein during the positioning stage in this embodiment of the application.

[0117] The laser 302 emits a low-power coherent light source. For example, the laser 302 may be a 1-mW, 520-nanometer laser diode used to generate laser speckle when the camera 303 captures images of the veins in the neck. Alternatively, the laser 302 may emit 850-nanometer structured light (laser array). In other embodiments, the light source emitted by the laser 302 can be configured by those skilled in the art according to actual needs.

[0118] The camera 303 is an industrial camera in a defocused state, equipped with an adjustable focusing lens (e.g., a 25mm focal length, F1.4 aperture, and C-mount). A key feature of the camera 303 is its defocused operating state; for example, the distance between the camera 303 and the target object can be set to 0.6 meters, while the distance between the focal plane of the camera 303 and the target object can be set to 0.15 meters. The camera 303 features a high frame rate (e.g., 650 frames per second) to fully capture the jugular vein pulsation signal.

[0119] When the coherent light source illuminates the rough surface of the skin, a speckle interference field is formed. Although the amplitude of the minute pulsation of the skin caused by the jugular vein is very small, the formation of the speckle interference field allows the pixel displacement to be calculated from the speckle pattern in this embodiment, thereby improving the accuracy of measuring the jugular vein pulsation signal. The minute rotation angle of the human skin surface... This will produce an amplified speckle shift on the out-of-focus camera's image plane. Their relationship can be quantified by the following formula: ;in, It is the distance from the vein in the neck of the target object to the focal plane of the camera 303; It is the distance from the lens of the camera 303 to the focal plane of the camera 303; It is a fixed distance from the lens of the camera 303 to the sensor of the camera 303 (that is, the position of the imaging plane of the camera 303);

[0120] This application embodiment will use a small rotation angle The speckle shift is magnified to a scale much greater than 1, making it easily detectable on the sensor of the camera 303. This greatly improves the signal-to-noise ratio and sensitivity of measuring jugular vein pulsation signals.

[0121] In practical applications, when the infrared light source device, the laser, and the camera are in a time-differential operating mode, the operating steps of the processor, the infrared light source device, the laser, and the camera can be implemented through the following steps:

[0122] The processor sends a first activation command to the infrared light source device. Upon receiving the first activation command, the infrared light source device emits an 850-nanometer infrared light source array to illuminate the veins in the neck of the target object. Next, the processor sends a second activation command to the camera. Upon receiving the second activation command, the camera begins infrared image acquisition processing of the veins in the neck of the target object, obtaining at least one first infrared image. Then, after acquiring the at least one first infrared image, the processor sends a first deactivation command to the infrared light source device and a second deactivation command to the camera. Upon receiving the first deactivation command, the infrared light source device is deactivated, and the camera is also deactivated upon receiving the second deactivation command. Next, the processor sends a third activation command to the laser. Upon receiving the third activation command, the laser emits a coherent light source to illuminate the first target area. Finally, the processor sends a fourth activation command to the camera. Upon receiving the fourth activation command, the camera begins defocus speckle video acquisition processing of the first target area to obtain the target defocus speckle video.

[0123] It should be understood that when the infrared light source device, the laser, and the camera are in the time-differential operating mode, if the infrared light source device is in the on state, then the laser is in the off state; or, if the infrared light source device is in the off state, then the laser is in the on state.

[0124] In practical applications, and when the infrared light source device, the laser, and the camera are in a time-differential operating mode, the operating steps of the processor, the infrared light source device, the laser, and the camera can also be implemented through the following steps:

[0125] The processor sends a fifth activation command to the infrared light source device. Upon receiving the fifth activation command, the infrared light source device emits an 850-nanometer infrared light source array to irradiate the veins in the neck of the target object. Next, the processor sends a sixth activation command to the camera. Upon receiving the sixth activation command, the camera begins infrared image acquisition processing of the veins in the neck of the target object, obtaining at least one second infrared image. Then, after acquiring the at least one second infrared image, the processor sends a fifth deactivation command to the infrared light source device and a sixth deactivation command to the camera. Upon receiving the fifth deactivation command, the infrared light source device is deactivated, and the camera is also deactivated upon receiving the sixth deactivation command. Then, if a preset condition is not met, the following steps are executed cyclically: the processor sends a seventh activation command to the laser. Upon receiving the seventh activation command, the laser emits a coherent light source to irradiate the second or third target area. Next, the processor sends an eighth activation command to the camera. Upon receiving the eighth activation command, the camera begins to irradiate the second or third target area. The processor performs video acquisition and processing to obtain sub-videos. When the time for the camera to acquire the sub-videos reaches a preset time interval, the processor sends a seventh shutdown command to the laser and an eighth shutdown command to the camera. Upon receiving the seventh shutdown command, the laser and the camera are both turned off. Next, the processor sends a ninth activation command to the infrared light source device. Upon receiving the ninth activation command, the infrared light source device emits an 850-nanometer infrared light source array to irradiate the veins in the neck of the target object. Next, the processor sends a tenth activation command to the camera. Upon receiving the tenth activation command, the camera begins infrared image acquisition processing of the veins in the neck of the target object, obtaining at least one third infrared image. After acquiring the at least one third infrared image, the processor sends a ninth shutdown command to the infrared light source device and a tenth shutdown command to the camera. Upon receiving the ninth shutdown command, the infrared light source device and the camera are both turned off.

[0126] In practical applications, when the infrared light source device, the laser, and the camera are in a time-synchronized operating mode, the operating steps of the processor, the infrared light source device, the laser, and the camera can be implemented through the following steps:

[0127] The processor simultaneously sends a first power-on command to the infrared light source device, the laser, and the camera. Upon receiving the first power-on command, the infrared light source device, the laser, and the camera start simultaneously. The infrared light source device emits an 850-nanometer infrared light source array to illuminate the veins in the neck of the target object. The laser emits a 520-nanometer coherent light source to illuminate the veins in the neck of the target object. The camera performs video acquisition and processing on the veins to obtain the defocus speckle video of the target. In this application, special optical elements need to be added in front of the camera lens or the Bayer filter characteristics of a special color camera need to be utilized so that the defocus speckle video of the target has two spectral signals. The two spectral signals are separated into different color channels or pixel groups. One spectral signal is used to generate a vein mask in real time, and the other spectral signal is used to extract the jugular vein pulsation signal in real time, thereby realizing real-time dynamic tracking of the measurement location (i.e., the vein).

[0128] In practical applications, when the infrared light source device, the laser, and the camera are in a time-synchronized operating mode, the operating steps of the processor, the infrared light source device, the laser, and the camera can be implemented through the following steps:

[0129] The processor simultaneously sends a first power-on command to the infrared light source device, the laser, and the camera. Upon receiving the first power-on command, the infrared light source device, the laser, and the camera start up simultaneously. The infrared light source device emits an 850-nanometer infrared light source array to illuminate the veins in the neck of the target object. The laser emits a 520-nanometer coherent light source to illuminate the veins in the neck of the target object. The camera performs video acquisition and processing on the veins to obtain a defocus speckle video of the target.

[0130] This application proposes two selectable collaborative working modes: time-differential and time-synchronous. It covers a variety of specific implementation methods, from static positioning to dynamic tracking, from the same spectral light source to different spectral light sources, and from a single camera to dual cameras (i.e., the camera and the image sensor), which greatly expands the applicability and robustness of the embodiments of this application.

[0131] In some embodiments, the jugular vein pulsation measurement system 300 based on defocus speckle imaging further includes an image sensor that is communicatively connected to the processor.

[0132] In practical applications, when the infrared light source device, the laser, the camera, and the image sensor are in a time-synchronized operating mode, the operating steps of the processor, the infrared light source device, the laser, the camera, and the image sensor can be implemented through the following steps:

[0133] The processor simultaneously sends a second power-on command to the infrared light source device, the laser, the camera, and the image sensor. Upon receiving the second power-on command, the infrared light source device, the laser, the camera, and the image sensor simultaneously start up. The infrared light source device emits an 850-nanometer infrared light source array to illuminate the veins in the neck of the target object. The laser emits a 520-nanometer coherent light source to illuminate the veins in the neck of the target object. The camera performs video acquisition processing on the veins to obtain a defocus speckle video of the target. The image sensor performs image acquisition processing on the veins to obtain at least one first infrared image. It should be noted that a bandpass or long-pass filter that only allows infrared light to pass through should be installed in front of the image sensor; a bandpass filter that only allows coherent light to pass through should be installed in front of the camera.

[0134] In other embodiments, the coordinate systems of the image sensor and the camera should be registered before they are started to eliminate differences in their physical positions.

[0135] The jugular vein pulsation measurement system 300 based on defocus speckle imaging can be integrated into a portable handheld device, or into a bedside monitoring device, or even explored into a wearable patch device for long-term dynamic jugular vein pulsation measurement.

[0136] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that causes a computer to perform the steps of the method for measuring jugular vein pulsation based on defocus speckle imaging provided in the first aspect of this application.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0138] The technical features of the above embodiments can be combined without changing the basic principles of this application. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the patent protection scope of this application should be determined by the appended claims.

Claims

1. A method of measuring carotid pulse based on defocus speckle imaging, characterized in that, The method comprises the following steps: After positioning the vein part of the neck of the target object, defocus speckle video acquisition processing is performed to obtain a target defocus speckle video; Directional motion signal extraction processing is performed on the target defocus speckle video to obtain a target directional motion signal corresponding to each pixel point in a target image, the target image being one frame of image of the target defocus speckle video; Average value calculation processing is performed on the target directional motion signals corresponding to all pixels of the target image to obtain a jugular vein pulsation signal of the target object; The step of performing directional motion signal extraction processing on the target defocus speckle video to obtain a target directional motion signal corresponding to each pixel point in a target image comprises: performing calculation processing on the pixel displacement of each pixel point in the target defocus speckle video relative to a reference image by using a motion estimation algorithm to obtain a pixel displacement vector corresponding to each pixel point; performing pixel component calculation processing on the pixel displacement vector corresponding to each pixel point to obtain a target component corresponding to each pixel point, or projecting the pixel displacement vector corresponding to each pixel point to the direction of the vein running in the vein part to obtain a target component corresponding to each pixel point; performing accumulation processing on the target components corresponding to all pixel points at the same pixel position in the target image to obtain a target directional motion signal corresponding to each pixel point in the target image; The step of performing pixel component calculation processing on the pixel displacement vector corresponding to each pixel point to obtain a target component corresponding to each pixel point comprises: performing accumulation calculation processing on a first displacement vector of the pixel displacement vectors corresponding to all pixel points at the same pixel position on all images of the target defocus speckle video to obtain a first displacement signal corresponding to each pixel position; performing accumulation calculation processing on a second displacement vector of the pixel displacement vectors corresponding to all pixel points at the same pixel position on all images of the target defocus speckle video to obtain a second displacement signal corresponding to each pixel position; performing pixel component calculation processing on the pixel displacement vector corresponding to each pixel point, the first displacement signal corresponding to each pixel position, and the second displacement signal corresponding to each pixel position to generate a target component corresponding to each pixel point.

2. The method of measuring carotid pulse based on defocus speckle imaging according to claim 1, wherein, The step of performing defocus speckle video acquisition processing on the vein part of the neck of the target object after positioning processing to obtain a target defocus speckle video comprises: Performing infrared image acquisition processing on the vein part of the neck of the target object to obtain at least one first infrared image; Performing vein part positioning processing on the at least one first infrared image to obtain a first target area of the neck; Performing defocus speckle video acquisition processing on the first target area to obtain the target defocus speckle video.

3. The method of measuring carotid pulse based on defocus speckle imaging according to claim 1, wherein, The step of performing defocus speckle video acquisition processing on the vein part of the neck of the target object after positioning processing to obtain a target defocus speckle video comprises: Performing preliminary positioning processing on the vein part of the neck of the target object to obtain a second target area; Setting a preset condition; In the case where the preset condition is not reached, the following steps are cyclically performed: video acquisition processing is performed on the second target region to obtain a sub-video, and the sub-video is added to a to-be-processed set; the vein part is repositioned based on a preset time interval to obtain a third target region, and the third target region is taken as the second target region; After the above cyclic steps are ended, all the sub-videos in the to-be-processed set are spliced to obtain the target defocus speckle video.

4. The method of measuring carotid pulse based on defocus speckle imaging according to claim 1, wherein, The calculation formula of the step of performing pixel component calculation processing on the pixel displacement vector corresponding to each pixel point, the first displacement signal corresponding to each pixel position, and the second displacement signal corresponding to each pixel position to generate the target component corresponding to each pixel point is: ； in, It is the first The target components corresponding to each pixel; It is the first The first displacement vector of the pixel displacement vector corresponding to each pixel; It is the first The first displacement signal corresponding to each pixel position; It is the first The second displacement vector corresponding to the pixel displacement vector of each pixel; It is the first The second displacement signal corresponding to each pixel position.

5. The method of measuring carotid pulse based on defocus speckle imaging according to claim 1, wherein, The calculation formula of the step of performing accumulation processing on the target components corresponding to all the pixel points at the same pixel position in each pixel point in the target image to obtain the target direction motion signal corresponding to each pixel point in the target image is: ； wherein, is a target direction motion signal corresponding to a pixel point at a th pixel position in the target image, is a total number of all pixel points at the same pixel position as each pixel point in the target image.

6. A device for measuring carotid pulse based on out-of-focus speckle imaging, characterized in that, It comprises: A target defocus speckle video obtaining module is configured to perform defocus speckle video acquisition processing on the vein part of the neck of the target object after positioning processing, and obtain a target defocus speckle video. A signal extraction module is configured to perform directional motion signal extraction processing on the target defocus speckle video, and obtain a target direction motion signal corresponding to each pixel point in a target image, which is one frame of image of the target defocus speckle video. A calculation processing module is configured to perform average value calculation processing on the target direction motion signals corresponding to all the pixels of the target image to obtain a carotid vein pulsation signal of the target object. The signal extraction module is further configured to calculate the pixel displacement of each pixel point relative to a reference image by using a motion estimation algorithm to obtain a pixel displacement vector corresponding to each pixel point; perform pixel component calculation processing on the pixel displacement vector corresponding to each pixel point to obtain a target component corresponding to each pixel point, or project the pixel displacement vector corresponding to each pixel point to the direction of the vein direction in the vein part to obtain a target component corresponding to each pixel point; and perform accumulation processing on the target components corresponding to all the pixel points at the same pixel position in each pixel point in the target image to obtain a target direction motion signal corresponding to each pixel point in the target image. The signal extraction module is further configured to perform accumulation calculation processing on the first displacement vectors of the pixel displacement vectors corresponding to all the pixel points at the same pixel position on all the images in the target defocus speckle video to obtain a first displacement signal corresponding to each pixel position; and perform accumulation calculation processing on the second displacement vectors of the pixel displacement vectors corresponding to all the pixel points at the same pixel position on all the images in the target defocus speckle video to obtain a second displacement signal corresponding to each pixel position. The pixel component calculation processing is performed on the pixel displacement vector corresponding to each pixel point, the first displacement signal corresponding to each pixel position, and the second displacement signal corresponding to each pixel position to generate the target component corresponding to each pixel point.

7. A system for measuring carotid pulse based on out-of-focus speckle imaging, characterized in that, It comprises: An infrared light source device, a laser, a camera and a terminal device, the terminal device comprising a processor and a memory, the memory for storing a computer program, the processor for invoking and running the computer program stored in the memory, executing the steps of the measurement method of the jugular vein pulse based on the defocus speckle imaging according to any one of claims 1 to 5. The processor is in communication connection with the infrared light source device, the laser and the camera.

8. A computer-readable storage medium, characterized in that, A computer program for storing, the computer program enabling a computer to execute the steps of the measurement method of the jugular vein pulse based on the defocus speckle imaging according to any one of claims 1 to 5.

Images