A Visual-Based Pulse Wave Detection Method and Device at Cunkou, Guanchang, and Chize Positions

Abstract

The present invention discloses a method and device for detecting the positions of cun, guan, and chi of pulse waves based on vision. The method includes: acquiring a wrist video based on a wrist fluctuation acquisition device, and extracting a region of interest from the wrist video; obtaining a wrist digital speckle video including only the region of interest, performing block processing on each frame of the region of interest in the wrist digital speckle video to obtain a number of adjacent and non-overlapping speckle sub-regions, and determining the pulse signals of each speckle sub-region; based on the pulse signals of each speckle sub-region, determining the signal differences between each speckle sub-region, and determining the acquisition positions of cun, guan, and chi according to the signal differences and the prior knowledge of the human physiological structure. After acquiring the wrist video, the present invention segments the region of interest, performs a series of analyses based on the region of interest, and non-contact determines the acquisition positions of cun, guan, and chi according to the differences in pulse signals, thereby solving the problem of automatic positioning in the signal acquisition process.

Description

Technical Field

[0001] The present invention relates to the technical field of pulse signal acquisition and positioning, and particularly to a method and device for detecting the positions of cun, guan, and chi of pulse waves based on vision. Background Art

[0002] In the prior art, for the automatic and intelligent recognition of the acquisition position, visual information is mainly used to directly obtain the pulse signal or judge the acquisition position. Although some research progress and achievements have been made, its deficiencies and drawbacks are also very obvious. Specifically, the vision-based method mainly utilizes the subtle deformation of the skin caused by the beating of the radial artery, detects and amplifies the deformation, and converts it into a pulse signal. However, the pulse signals detected by such methods usually have poor quality, a lot of noise, and low signal frequencies. And due to individual differences in the human body, in many wrist regions, it is impossible to detect the fluctuations solely relying on vision technology, resulting in the inability to accurately determine the positions of cun, guan, and chi. It can be seen that in the prior art, there is a lack of a set of standard analysis and judgment methods for the acquisition positions of cun, guan, and chi.

[0003] Therefore, the prior art still needs to be improved. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and device for detecting the positions of cun, guan, and chi of pulse waves based on vision in view of the above-mentioned defects of the prior art, aiming to solve the problem that it is difficult to ensure the computational load balance of each server simultaneously even after a communication link with high bandwidth requirements is allocated within the same server in the prior art.

[0005] In a first aspect, the present invention provides a method for detecting the positions of cun, guan, and chi of pulse waves based on vision, wherein the method includes:

[0006] Obtaining a wrist video based on a preset wrist fluctuation acquisition device, and extracting a region of interest from the wrist video;

[0007] Obtaining a wrist digital speckle video including only the region of interest, performing block processing on each frame of the region of interest in the wrist digital speckle video to obtain a plurality of adjacent and non-overlapping speckle sub-regions, and determining the pulse signal of each speckle sub-region according to the speckle sub-regions;

[0008] Determining the signal difference between each speckle sub-region based on the pulse signals of each speckle sub-region, and determining the acquisition positions of cun, guan, and chi according to the signal difference and the prior knowledge of the human physiological structure.

[0009] In an implementation manner, the wrist fluctuation acquisition device includes:

[0010] A shielding cover, with an opening provided on the shielding cover;

[0011] Light sources, with multiple light sources provided and respectively arranged on different inner walls of the shielding cover to provide light at different angles;

[0012] A support member, which is arranged inside the shielding cover and is connected to the bottom plate and the rear side plate of the shielding cover;

[0013] A palm posture maintaining assembly, which is arranged at the bottom of the support member and is used for limiting the wrist posture;

[0014] A digital speckle generator, which is arranged on the support member and the digital speckle generator faces the palm posture maintaining assembly and is used for irradiating a speckle image onto the wrist skin;

[0015] An imaging device, which is arranged on the support member and is used for acquiring the wrist video at a preset speed.

[0016] In one implementation, extracting the region of interest from the wrist video includes:

[0017] Reading the first-frame wrist image of the wrist video and extracting a wrist target image with a smooth and continuous left edge from the first-frame wrist image;

[0018] Performing left-edge extraction on the wrist target image and using a polynomial for fitting to determine the coordinate of the maximum curvature of the fitting curve, where the coordinate of the maximum curvature is the position of the connection between the palm and the wrist in the wrist target image;

[0019] Taking the coordinate of the maximum curvature as a reference and selecting the region of interest from the wrist target image.

[0020] In one implementation, extracting a wrist target image with a smooth and continuous left edge from the first-frame wrist image includes:

[0021] Converting the first-frame wrist image into a grayscale image and performing binarization processing;

[0022] Performing erosion and dilation operations on the image after binarization processing and deleting connected regions with an area smaller than a preset value to obtain a wrist target image with a smooth and continuous left edge.

[0023] In one implementation, performing block processing on each frame of the region of interest in the wrist digital speckle video to obtain a number of adjacent and non-overlapping speckle sub-regions includes:

[0024] Perform Euler video magnification operation on the wrist digital speckle video, and perform image filtering processing on the wrist digital speckle video after Euler video magnification operation;

[0025] Convert the wrist digital speckle video after image filtering processing from the RGB image space to the YIO image space, and extract the Y-channel image;

[0026] According to the Y-channel image, perform block processing on each frame of the region of interest to obtain a number of adjacent and non-overlapping speckle sub-regions of m*m.

[0027] In one implementation, the determining the pulse signal of each speckle sub-region according to the speckle sub-region includes:

[0028] Use the digital speckle correlation method to calculate the maximum value of the correlation coefficient of each speckle sub-region in each frame of the region of interest;

[0029] Arrange the maximum values of the correlation coefficients of each speckle sub-region in time sequence to obtain the pulse signal.

[0030] In one implementation, the determining the signal difference between each speckle sub-region based on the pulse signal of each speckle sub-region, and determining the collection positions of cun, guan, and chi according to the signal difference and the prior knowledge of the human physiological structure includes:

[0031] Based on the pulse signal of each speckle sub-region, calculate the heart rate of each speckle sub-region, and delete the speckle sub-regions with abnormal heart rates to obtain the remaining speckle sub-regions;

[0032] Perform single-period segmentation on the remaining speckle sub-regions, calculate the single-period similarity between the remaining speckle sub-regions, and according to the single-period similarity, screen out the speckle sub-regions with the single-period similarity exceeding the preset similarity to obtain a set of speckle sub-regions;

[0033] Sort the set of speckle sub-regions in descending order of amplitude intensity to obtain the first sorting;

[0034] Determine a speckle sub-region close to the center of the region of interest as the collection position of guan from the set of speckle sub-regions according to the first sorting, and the amplitude intensity of the speckle sub-region corresponding to the collection position of guan is the largest;

[0035] Delete the speckle sub-region with the collection position of guan from the set of speckle sub-regions, and obtain the second sorting;

[0036] Determine the collection positions of cun and chi from the second sorting according to the relative positions of cun and guan and the relative positions of cun and chi.

[0037] In a second aspect, an embodiment of the present invention provides a vision-based pulse wave cun-guan-chi position detection device, where the device includes:

[0038] An interested region extraction module, configured to obtain a wrist video based on a preset wrist fluctuation acquisition device, and extract an interested region from the wrist video;

[0039] A pulse signal determination module, configured to obtain a wrist digital speckle video including only the interested region, perform a block processing on each frame of the interested region in the wrist digital speckle video to obtain a plurality of adjacent and non-overlapping speckle sub-regions, and determine the pulse signal of each speckle sub-region according to the speckle sub-regions;

[0040] An acquisition position determination module, configured to determine the signal difference between each speckle sub-region based on the pulse signal of each speckle sub-region, and determine the acquisition positions of cun, guan, and chi according to the signal difference and the prior knowledge of the human physiological structure.

[0041] In a third aspect, an embodiment of the present invention further provides a terminal device, where the terminal device includes a memory, a processor, and a vision-based pulse wave cun-guan-chi position detection program stored in the memory and executable on the processor. When the processor executes the vision-based pulse wave cun-guan-chi position detection program, the steps of the vision-based pulse wave cun-guan-chi position detection method in any one of the above solutions are implemented.

[0042] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where a vision-based pulse wave cun-guan-chi position detection program is stored on the computer-readable storage medium. When the vision-based pulse wave cun-guan-chi position detection program is executed by a processor, the steps of the vision-based pulse wave cun-guan-chi position detection method in any one of the above solutions are implemented.

[0043] Beneficial effects: Compared with the prior art, the present invention provides a method for detecting the positions of cun, guan, and chi of the pulse wave based on vision. First, a wrist video is acquired based on a preset wrist fluctuation acquisition device, and a region of interest is extracted from the wrist video. Then, a wrist digital speckle video including only the region of interest is obtained, and each frame of the region of interest in the wrist digital speckle video is divided into blocks to obtain a number of adjacent and non-overlapping speckle sub-regions, and the pulse signal of each speckle sub-region is determined according to the speckle sub-region. Finally, based on the pulse signals of each speckle sub-region, the signal difference between each speckle sub-region is determined, and according to the signal difference and the prior knowledge of the human physiological structure, the acquisition positions of cun, guan, and chi are determined. After obtaining the wrist video, the present invention divides out the region of interest, and based on the region of interest, a series of analyses are performed. According to the difference of the pulse signals, the acquisition positions of cun, guan, and chi are determined without contact, solving the problem of automatic positioning in the signal acquisition process. Description of the Drawings

[0044] Figure 1 It is a flowchart of the specific implementation manner of the method for detecting the positions of cun, guan, and chi of the pulse wave based on vision provided by the embodiment of the present invention.

[0045] Figure 2 It is a schematic structural diagram of the wrist fluctuation acquisition device in the method for detecting the positions of cun, guan, and chi of the pulse wave based on vision provided by the embodiment of the present invention.

[0046] Figure 3 It is a schematic diagram of the first frame of the wrist image and the wrist target image in the method for detecting the positions of cun, guan, and chi of the pulse wave based on vision provided by the embodiment of the present invention.

[0047] Figure 4 It is the fitting curve of the left edge contour of the wrist in the method for detecting the positions of cun, guan, and chi of the pulse wave based on vision provided by the embodiment of the present invention.

[0048] Figure 5 It is a schematic diagram of the intercepting process of the region of interest in the method for detecting the positions of cun, guan, and chi of the pulse wave based on vision provided by the embodiment of the present invention.

[0049] Figure 6 It is a schematic diagram of the pulse signal extracted from one speckle sub-region in the method for detecting the positions of cun, guan, and chi of the pulse wave based on vision provided by the embodiment of the present invention.

[0050] Figure 7 It is a functional schematic diagram of the device for detecting the positions of cun, guan, and chi of the pulse wave based on vision provided by the embodiment of the present invention.

[0051] Figure 8 It is a schematic block diagram of the terminal device provided by the embodiment of the present invention. Detailed Implementation Manner

[0052] To make the objectives, technical solutions and effects of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] Since the pulse wave can cause rhythmic undulations of the skin, imaging sensors are used to directly or indirectly capture such changes and are used to construct a series of non-contact acquisition systems. In the prior art, there is a technique of using a single camera to record videos on the wrist of a human body, and then adopting the Eulerian videomagnification method to recover the pulse wave signal from the original video by analyzing the color and gray-scale components of different frames. There is also a technique of detecting changes on the skin surface by using a charge-coupled device (CCD) image sensor in combination with laser triangulation and projection moire techniques. In addition, in order to overcome the shortcoming of only measuring the vibration on the skin surface, scholars designed an optical moire pulse measurement device based on near-infrared to measure the pulse vibration under the skin. In order to achieve full-field and high-efficiency deformation analysis of the pulse wave, a pulse detection method based on binocular vision was introduced to record the overall fluctuations in certain areas. In addition, digital speckle pattern interferometry (DSPI) was also adopted to dynamically measure the micro / nano-scale displacement of the wrist skin.

[0054] To achieve the full - automatic positioning of the sensor, related research uses imaging photoplethysmography (iPPG) and optical triangulation technology to automatically determine the acquisition position. Experts learn to use iPPG to measure the blood flow changes in the radial artery and have a certain positioning ability for the pulse signal acquisition position. In addition, a convolutional neural network (CNN) is designed to combine static images and iPPG images, improving the accuracy of target positioning. Moreover, the mechanical components of the automatic positioning system in the existing technology include a linear laser and a CMOS image sensor. The positioning system analyzes the change of the optical centroid of the images collected by the CMOS image sensor and returns the specific positions of Cun, Guan, and Chi, but it is necessary to predict and align the corresponding positions. From the research status in related fields, although a large amount of basic research work has been carried out in the pulse wave acquisition method, equipment, and related data - processing methods in the existing technology, there are still certain deficiencies in the automatic and intelligent detection of the pulse wave signal acquisition position, and there is a lack of effective and practical solutions.

[0055] Therefore, this embodiment provides a vision - based method for detecting the positions of Cun, Guan, and Chi of the pulse wave. The method based on this embodiment can non - contactedly determine the acquisition positions of Cun, Guan, and Chi and solve the problem of automatic positioning during the signal acquisition process. Specifically, this embodiment first obtains a wrist video based on a preset wrist fluctuation acquisition device and extracts the region of interest from the wrist video. Then, a wrist digital speckle video including only the region of interest is obtained, and each frame of the region of interest in the wrist digital speckle video is divided into blocks to obtain a number of adjacent and non - overlapping speckle sub - regions, and the pulse signal of each speckle sub - region is determined according to the speckle sub - regions. Finally, based on the pulse signals of each speckle sub - region, the signal difference between each speckle sub - region is determined, and according to the signal difference and the prior knowledge of the human physiological structure, the acquisition positions of Cun, Guan, and Chi are determined. It can be seen that after obtaining the wrist video in this embodiment, the region of interest is segmented, and a series of analyses are carried out based on the region of interest. According to the difference of the pulse signals, the acquisition positions of Cun, Guan, and Chi are non - contactedly determined, solving the problem of automatic positioning during the signal acquisition process, so as to provide a reference acquisition position for the user, and then provide the acquisition position to the intelligent pulse acquisition system to control the contact - type pressure sensor to collect high - precision pulse signals.

[0056] Exemplary method

[0057] The pulse wave Cun, Guan, and Chi position detection method based on vision in this embodiment can be applied to a terminal device, which can be a computer. Specifically, the pulse wave Cun, Guan, and Chi position detection method based on vision in this embodiment specifically includes the following steps:

[0058] Step S100: acquiring a wrist video based on a preset wrist fluctuation collection device, and extracting a region of interest from the wrist video.

[0059] The present embodiment is based on the pulse signal acquisition principle of vision, utilizes the color / brightness change of the same pixel in time sequence caused by skin fluctuation, converts it into a one-dimensional signal, and then analyzes the one-dimensional signal, thereby determining the pulse wave Cun, Guan, and Chi positions. However, this process is affected by multiple factors. First, the illumination needs to be of appropriate intensity during acquisition. Too strong or too weak will cause the color / brightness change to be unclear. Secondly, due to large individual differences, the crowd with stronger radial artery pulsation can be detected relatively simply, while the crowd with stronger or weaker radial artery pulsation is difficult to directly obtain the pulse signal through simple video analysis. Furthermore, due to the involuntary slight shaking of the human body, the pixel will produce a positional offset, thereby introducing a noise signal. In order to solve the above three problems, the present embodiment is pre-designed as follows Figure 2 The wrist wave collection device described in the embodiment is connected to a terminal device. When the wrist video is collected by the wrist wave collection device, the wrist video is transmitted to the terminal device for subsequent analysis by the terminal device, thereby determining the collection positions of the pulse wave Cun, Guan, and Chi. The wrist wave collection device of this embodiment can solve the problems of illumination consistency, too weak beating, and fixed human body posture as much as possible.

[0060] Specifically, Figure 2As shown in the figure, the wrist fluctuation acquisition device in this embodiment includes a shielding cover 101, and an opening 102 is provided on the shielding cover 101. The shielding cover 101 in this embodiment is set in a closed cuboid shape, and the opening 102 is provided on only one side surface. Light sources 103 are provided on different inner walls of the shielding cover 101. Therefore, a plurality of light sources 103 are provided, which can provide light at different angles, facilitating the provision of continuous and stable light. And in cooperation with the shielding cover, the internal light can be protected from the external environment, thereby ensuring that the noise interference brought by the light change to the final pulse signal is eliminated. A support member 104 is provided on the bottom plate inside the shielding cover 101. The support member 104 is attached to the rear side plate of the shielding cover, which is conducive to ensuring the stability of the support member 104. A palm posture maintaining assembly 105 is provided at the bottom of the support member 104. The palm posture maintaining assembly 105 is used to limit the wrist posture. Specifically, the palm posture maintaining assembly 105 in this embodiment includes a groove and a palm grip. The groove is used to place the arm. During acquisition, the person to be acquired places the arm in the groove and holds the grip with the palm, which can effectively limit the wrist posture and reduce involuntary tremors. A digital speckle generator 106 is provided on the top of the support member 104. The digital speckle generator 106 faces the palm posture maintaining assembly 105 and is used to irradiate a speckle image onto the wrist skin. An imaging device 107 is also provided on the support member 104. The imaging device 107 acquires a wrist video at a preset speed.

[0061] In one implementation, when extracting the Region of Interest (ROI) in this embodiment, the following steps are included:

[0062] Step S101: Read the first frame of wrist image of the wrist video, and extract a wrist target image with a smooth and continuous left edge from the first frame of wrist image;

[0063] Step S102: Extract the left edge of the wrist target image, and use a polynomial for fitting to determine the coordinate of the maximum curvature of the fitting curve, where the coordinate of the maximum curvature is the position of the connection between the palm and the wrist in the wrist target image;

[0064] Step S103: Select the region of interest from the wrist target image based on the coordinate of the maximum curvature.

[0065] Specifically, after a wrist video is acquired by the wrist fluctuation acquisition device, this embodiment uses the first frame of wrist image of the wrist video to extract the region of interest. Specifically, this embodiment first reads the first frame of wrist image of the wrist video to obtain the entire wrist, as Figure 3Figure a in it. Then, convert the first-frame wrist image into a grayscale image and perform binarization processing to better extract the wrist target image. Next, in this embodiment, erosion and dilation operations are performed on the binarized image, and connected regions with an area smaller than a preset value are deleted to obtain a wrist target image with a smooth and continuous left edge, as shown in Figure 3 Figure b in it. Next, in this embodiment, the left edge of the wrist target image is extracted, and polynomial fitting is used to obtain a fitting curve of the left edge contour of the wrist, as shown in Figure 4 shown in. Then, according to the fitting curve, the curvature change of the fitting curve is determined, and the coordinate of the maximum curvature value of the fitting curve is determined, where the coordinate of the maximum curvature value is the position of the connection between the palm and the wrist in the wrist target image. According to traditional Chinese medicine theory and the physiological characteristics of the wrist, the ROI (region of interest) suitable for the cun, guan, and chi distributions of most people should be in the left half of the wrist and in a region where the palm faces downward. Therefore, in this embodiment, the upper left starting point of the rectangle is at the position of the coordinate of the maximum curvature value of the fitting curve, the width is 0.6 times the width of the wrist, and the edge of the wrist target image frame is used as the intercept length to intercept a rectangular region, as shown in Figure 5 Figure a in it, so as to obtain the Figure 5 region of interest in Figure b. In this embodiment, the size of the region of interest can be adjusted according to actual needs. The core is the detection of the connection between the wrist and the palm, and the rectangular region is selected based on this. By extracting the region of interest in this embodiment, the determined region of interest is used for the analysis of subsequent steps, which is beneficial to reducing the noise and interference brought by other regions of the image.

[0066] Step S200: Obtain a wrist digital speckle video only including the region of interest, perform block processing on each frame of the region of interest in the wrist digital speckle video to obtain a number of adjacent and non-overlapping speckle sub-regions, and determine the pulse signal of each speckle sub-region according to the speckle sub-region.

[0067] After selecting the ROI (region of interest) in this embodiment, the algorithm-level magnification is continued in the corresponding region. The embodiment uses the Euler video motion magnification technology to process each frame of the ROI region, improve the pixel value difference of the same pixel between frames, and then determine the pulse signal based on the pixel value difference.

[0068] In one implementation manner, when determining the pulse signal in this embodiment, the following steps are included:

[0069] Step S201: Perform Euler video magnification operation on the wrist ROI digital speckle video, and perform image filtering processing on the wrist digital speckle video after the Euler video magnification operation;

[0070] Step S202: Convert the wrist digital speckle video after image filtering processing from the RGB image space to the YIO image space, and extract the Y-channel image;

[0071] Step S203: According to the Y-channel image, perform block processing on each frame of the region of interest to obtain a number of adjacent and non-overlapping speckle sub-regions of m*m;

[0072] Step S204: Use the digital speckle correlation method to calculate the maximum value of the correlation coefficient of each speckle sub-region in each frame of the region of interest;

[0073] Step S205: Arrange the maximum values of the correlation coefficients of each speckle sub-region in time sequence to obtain the pulse signal.

[0074] Specifically, in this embodiment, first input the wrist digital speckle video, and each frame image of the wrist digital speckle video only includes the region of interest part. Then, perform Euler video magnification operation on the wrist digital speckle video, and perform image filtering processing on the wrist digital speckle video after Euler video magnification operation. Since the Euler video magnification process will synchronize the noise in the wrist digital speckle video, it is necessary to reduce the noise interference through image filtering operation. Then, convert the wrist digital speckle video after image filtering processing from the RGB image space to the YIO image space, extract the Y-channel image, and then continue the next analysis on the Y-channel image. Then, perform block processing on each frame of the region of interest to obtain a number of adjacent and non-overlapping speckle sub-regions of m*m, and use the digital speckle correlation method to calculate the maximum value of the correlation coefficient of each speckle sub-region in each frame of the region of interest. Finally, arrange the maximum values of the correlation coefficients of each speckle sub-region in time sequence to obtain the corresponding pulse signal, as Figure 6 shown in Figure 6 a schematic diagram of the pulse signal extracted from one speckle sub-region.

[0075] Step S300: Based on the pulse signals of each speckle sub-region, determine the signal difference between each speckle sub-region, and determine the collection positions of cun, guan, and chi according to the signal difference and the prior knowledge of the human physiological structure.

[0076] In this embodiment, after generating the pulse signals for each speckle sub-region, by comparing the signal differences between the pulse signals, estimate the approximate positions of cun, guan, and chi to provide positioning guidance for signal collection.

[0077] In one implementation, step S300 in this embodiment specifically includes the following steps:

[0078] Step S301: Based on the pulse signals of each speckle sub-region, calculate the heart rate of each speckle sub-region, and delete the speckle sub-regions with abnormal heart rates to obtain the remaining speckle sub-regions;

[0079] Step S302: Perform single-cycle segmentation on the remaining speckle sub-regions, calculate the single-cycle similarity between the remaining speckle sub-regions, and based on the single-cycle similarity, screen out the speckle sub-regions with a single-cycle similarity exceeding a preset similarity to obtain a set of speckle sub-regions;

[0080] Step S303: Sort the set of speckle sub-regions in descending order of amplitude intensity to obtain a first sorting;

[0081] Step S304: Determine a speckle sub-region located at the center of the region of interest from the set of speckle sub-regions according to the first sorting as the relevant acquisition position, and the speckle sub-region corresponding to the relevant acquisition position has the maximum amplitude intensity;

[0082] Step S305: Delete the speckle sub-region with the relevant acquisition position from the set of speckle sub-regions and obtain a second sorting;

[0083] Step S306: Determine the acquisition positions of Cun and Chi from the second sorting according to the relative positions of Cun and Guan and the relative positions of Cun and Chi.

[0084] Specifically, in this embodiment, after extracting the pulse signal from each speckle sub-region, the heart rate of each speckle sub-region is calculated, and the speckle sub-regions with abnormal heart rates are deleted to obtain the remaining speckle sub-regions. Since there is no pulse signal in these speckle sub-regions with abnormal heart rates and they do not belong to the candidate regions of cun, guan, and chi, the subsequent analysis workload can be reduced. Then, single-cycle segmentation is performed on the remaining speckle sub-regions, and the single-cycle similarity between the remaining speckle sub-regions is calculated. Since the pulsation at the position of the radial artery is strong and the influence of noise on the corresponding region is relatively small, the speckle sub-regions located at the radial artery should have a relatively high single-cycle similarity. Therefore, in this embodiment, the speckle sub-regions with a single-cycle similarity exceeding the preset similarity can be screened out according to the single-cycle similarity to obtain a set of speckle sub-regions, that is, all the speckle sub-regions in this set of speckle sub-regions have a relatively high single-cycle similarity. Next, this embodiment sorts the set of speckle sub-regions in descending order of amplitude intensity to obtain the first sorting. Based on the prior knowledge of the human physiological structure, it can be known that the amplitude of the guan part should be the largest and should be in the relatively central region of the ROI. Therefore, according to this idea, this embodiment determines a speckle sub-region close to the center of the region of interest as the acquisition position of guan from the set of speckle sub-regions according to the first sorting, and moreover, the amplitude intensity of the speckle sub-region corresponding to the acquisition position of guan is the largest. Then, the speckle sub-region with the acquisition position of guan is deleted from the set of speckle sub-regions, and the second sorting is obtained. When determining the acquisition positions of cun and chi subsequently, analysis is performed based on the second sorting, which can effectively prevent the acquisition positions of cun and chi from overlapping with the acquisition position of guan in the next step of selection. Based on the prior knowledge of the human physiological structure, it can be known that the acquisition positions of cun and chi should be at the distal and proximal ends of guan respectively. Since they are distributed along the radial artery, the acquisition positions of cun and chi should be basically in a straight line, without too much deviation, and should not exceed the region of interest. Based on this, this embodiment determines the acquisition positions of cun and chi from the second sorting according to the relative positions of cun and guan and the relative positions of cun and chi.

[0085] It can be seen that in this embodiment, after obtaining a wrist video, the region of interest is segmented from the wrist video; Euler video motion magnification is performed on the region of interest, and temporal signals are extracted from different speckle sub-regions; after analyzing the signal differences, combined with the prior knowledge of wrist morphology, human physiological structure, etc., the acquisition positions of cun, guan, and chi are determined.

[0086] Exemplary device

[0087] Based on the above embodiments, the present invention further provides a vision-based pulse wave cun-guan-chi position detection device, as Figure 7As shown in the figure, the vision-based pulse wave cun-guan-chi position detection device is connected to the wrist fluctuation acquisition device. After the wrist video is collected by the wrist fluctuation acquisition device, the wrist video will be transmitted to the vision-based pulse wave cun-guan-chi position detection device. The vision-based pulse wave cun-guan-chi position detection device includes: an interested region extraction module 10, a pulse signal determination module 20, and a collection position determination module 30. Specifically, the interested region extraction module 10 is configured to obtain a wrist video based on a preset wrist fluctuation acquisition device and extract an interested region from the wrist video. The pulse signal determination module 20 is configured to obtain a wrist digital speckle video including only the interested region, perform block processing on each frame of the interested region in the wrist digital speckle video to obtain a plurality of adjacent and non-overlapping speckle sub-regions, and determine the pulse signal of each speckle sub-region according to the speckle sub-regions. The collection position determination module 30 is configured to determine the signal difference between each speckle sub-region based on the pulse signal of each speckle sub-region, and determine the collection positions of cun, guan, and chi according to the signal difference and the prior knowledge of the human physiological structure.

[0088] In one implementation, the interested region extraction module 10 includes:

[0089] A wrist target image extraction unit, configured to read the first frame of the wrist image of the wrist video and extract a wrist target image with a smooth and continuous left edge from the first frame of the wrist image;

[0090] A left edge extraction and fitting unit, configured to extract the left edge of the wrist target image and perform fitting using a polynomial to determine the coordinate of the maximum curvature of the fitting curve, where the coordinate of the maximum curvature is the position of the connection between the palm and the wrist in the wrist target image;

[0091] An interested region selection unit, configured to select the interested region from the wrist target image based on the coordinate of the maximum curvature.

[0092] In one implementation, the wrist target image extraction unit includes:

[0093] A binarization processing sub-unit, configured to convert the first frame of the wrist image into a grayscale image and perform binarization processing;

[0094] A target image extraction sub-unit, configured to perform erosion and dilation operations on the binarized image and delete connected regions with an area smaller than a preset value to obtain a wrist target image with a smooth and continuous left edge.

[0095] In one implementation, the pulse signal determination module 20 includes:

[0096] An amplification and filtering processing unit for performing Euler video magnification operation on the wrist digital speckle video and performing image filtering processing on the wrist digital speckle video after the Euler video magnification operation;

[0097] An image conversion unit for converting the wrist digital speckle video after image filtering processing from the RGB image space to the YIO image space and extracting the Y-channel image;

[0098] A speckle sub-region extraction unit for performing block processing on each frame of the region of interest according to the Y-channel image to obtain a plurality of adjacent and non-overlapping m*m speckle sub-regions;

[0099] A speckle sub-region processing unit for calculating the maximum value of the correlation coefficient of each speckle sub-region in each frame of the region of interest by using the digital speckle correlation method;

[0100] A pulse signal generation unit for arranging the maximum values of the correlation coefficients of each speckle sub-region in time sequence to obtain the pulse signal.

[0101] In one implementation manner, the acquisition position determination module 30 includes:

[0102] A heart rate calculation unit for calculating the heart rate of each speckle sub-region based on the pulse signal of each speckle sub-region, deleting the speckle sub-regions with abnormal heart rates, and obtaining the remaining speckle sub-regions;

[0103] A similarity calculation unit for performing single-cycle segmentation on the remaining speckle sub-regions, calculating the single-cycle similarity between the remaining speckle sub-regions, and screening out the speckle sub-regions with the single-cycle similarity exceeding a preset similarity according to the single-cycle similarity to obtain a speckle sub-region set;

[0104] An amplitude sorting unit for sorting the speckle sub-region set in descending order of amplitude intensity to obtain a first sorting;

[0105] A Guan position determination unit for determining a speckle sub-region close to the center of the region of interest as the acquisition position of Guan from the speckle sub-region set according to the first sorting, and the amplitude intensity of the speckle sub-region corresponding to the acquisition position of Guan is the largest;

[0106] A set update unit for deleting the speckle sub-region with the acquisition position of Guan from the speckle sub-region set and obtaining a second sorting;

[0107] A Cun and Chi position determination unit for determining the acquisition position of Cun and the acquisition position of Chi from the second sorting according to the relative position between Cun and Guan and the relative position between Cun and Chi.

[0108] In the device for detecting the positions of cun, guan, and chi of the pulse wave based on vision according to this embodiment, the working principles of the various modules are the same as those of the various steps in the above method embodiment, and will not be elaborated here.

[0109] Based on the above embodiments, the present invention further provides a terminal device, and the principle block diagram of the terminal device can be as Figure 8 shown. The terminal device may include one or more processors 100 ( Figure 8 only one is shown in the figure), a memory 101, and a computer program 102 stored in the memory 101 and executable on one or more processors 100. For example, a program for detecting the positions of cun, guan, and chi of the pulse wave based on vision. When one or more processors 100 execute the computer program 102, the various steps in the method embodiment for detecting the positions of cun, guan, and chi of the pulse wave based on vision can be implemented. Alternatively, when one or more processors 100 execute the computer program 102, the functions of the various modules / units in the device embodiment for detecting the positions of cun, guan, and chi of the pulse wave based on vision can be implemented, which is not limited here.

[0110] In one embodiment, the so-called processor 100 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0111] In one embodiment, the memory 101 may be an internal storage unit of the electronic device, such as the hard disk or memory of the electronic device. The memory 101 may also be an external storage device of the electronic device, such as a plug-in hard disk equipped on the electronic device, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 101 may also include both the internal storage unit and the external storage device of the electronic device. The memory 101 is used to store the computer program and other programs and data required by the terminal device. The memory 101 may also be used to temporarily store the data that has been output or will be output.

[0112] Those skilled in the art can understand, Figure 8The principle block diagram shown only shows the block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the terminal device to which the solution of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0113] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, operation database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0114] In summary, the present invention discloses a method and device for detecting the cun, guan, and chi positions of the pulse wave based on vision. The method includes: acquiring a wrist video based on a wrist fluctuation acquisition device, and extracting a region of interest from the wrist video; obtaining a wrist digital speckle video only including the region of interest, performing block processing on each frame of the region of interest in the wrist digital speckle video to obtain a number of adjacent and non-overlapping speckle sub-regions, and determining the pulse signal of each speckle sub-region; based on the pulse signals of each speckle sub-region, determining the signal difference between each speckle sub-region, and determining the acquisition positions of cun, guan, and chi according to the signal difference and the prior knowledge of the human physiological structure. After acquiring the wrist video, the present invention segments the region of interest, and performs a series of analyses based on the region of interest. According to the difference of the pulse signals, the acquisition positions of cun, guan, and chi are determined without contact, solving the problem of automatic positioning in the signal acquisition process.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting the positions of cun, guan, and chi of pulse waves based on vision, characterized in that, the method includes: obtaining a wrist video based on a preset wrist fluctuation acquisition device, and extracting a region of interest from the wrist video; obtaining a wrist digital speckle video only including the region of interest, performing block processing on each frame of the region of interest in the wrist digital speckle video to obtain a number of adjacent and non-overlapping speckle sub-regions, and determining the pulse signal of each speckle sub-region according to the speckle sub-regions; determining the signal difference between each speckle sub-region based on the pulse signals of each speckle sub-region, and determining the acquisition positions of cun, guan, and chi according to the signal difference and the prior knowledge of human physiological structure; the determining the signal difference between each speckle sub-region based on the pulse signals of each speckle sub-region, and determining the acquisition positions of cun, guan, and chi according to the signal difference and the prior knowledge of human physiological structure includes: calculating the heart rate of each speckle sub-region based on the pulse signals of each speckle sub-region, and deleting the speckle sub-regions with abnormal heart rates to obtain the remaining speckle sub-regions; performing single-cycle segmentation on the remaining speckle sub-regions, calculating the single-cycle similarity between the remaining speckle sub-regions, and screening out the speckle sub-regions with a single-cycle similarity exceeding a preset similarity according to the single-cycle similarity to obtain a set of speckle sub-regions; sorting the set of speckle sub-regions in descending order of amplitude intensity to obtain a first sorting; determining a speckle sub-region close to the center of the region of interest as the acquisition position of guan from the set of speckle sub-regions according to the first sorting, and the amplitude intensity of the speckle sub-region corresponding to the acquisition position of guan is the largest.

2. The method for detecting the positions of cun, guan, and chi of pulse waves based on vision according to claim 1, characterized in that, the wrist fluctuation acquisition device includes: a shielding cover provided with an opening; a light source, with multiple light sources respectively arranged on different inner walls of the shielding cover to provide illumination at different angles; a support member arranged inside the shielding cover and connected to the bottom plate and the rear side plate of the shielding cover; a palm posture maintaining component arranged at the bottom of the support member for limiting the wrist posture; a digital speckle generator arranged on the support member and facing the palm posture maintaining component for irradiating a speckle image onto the wrist skin; an imaging device arranged on the support member for acquiring the wrist video at a preset speed.

3. The method for detecting the positions of cun, guan, and chi of pulse waves based on vision according to claim 1, characterized in that, the extracting the region of interest from the wrist video includes: reading the first frame of the wrist image of the wrist video, and extracting a wrist target image with a smooth and continuous left edge from the first frame of the wrist image; Extract the left edge of the wrist target image and fit it with a polynomial to determine the coordinates of the maximum curvature of the fitted curve, where the coordinates of the maximum curvature are the position of the connection between the palm and the wrist in the wrist target image; Taking the coordinates of the maximum curvature as a reference, select the region of interest from the wrist target image.

4. The method for detecting the cun-gu-chi positions of the pulse wave based on vision according to claim 3, characterized in that, The extracting the wrist target image with a smooth and continuous left edge from the first-frame wrist image includes: Convert the first-frame wrist image into a grayscale image and perform binarization processing; And perform erosion and dilation operations on the binarized image, and delete the connected regions with an area smaller than a preset value to obtain a wrist target image with a smooth and continuous left edge.

5. The method for detecting the cun-gu-chi positions of the pulse wave based on vision according to claim 4, characterized in that, The dividing the region of interest in each frame of the wrist digital speckle video into blocks to obtain a number of adjacent and non-overlapping speckle sub-regions includes: Perform Euler video magnification operation on the wrist digital speckle video, and perform image filtering processing on the wrist digital speckle video after the Euler video magnification operation; Convert the wrist digital speckle video after image filtering processing from the RGB image space to the YIO image space, and extract the Y-channel image; According to the Y-channel image, divide the region of interest in each frame into blocks to obtain a number of adjacent and non-overlapping m*m speckle sub-regions.

6. The method for detecting the cun-gu-chi positions of the pulse wave based on vision according to claim 4, characterized in that, The determining the pulse signal of each speckle sub-region according to the speckle sub-region includes: Use the digital speckle correlation method to calculate the maximum value of the correlation coefficient of each speckle sub-region in the region of interest in each frame; Arrange the maximum values of the correlation coefficients of each speckle sub-region in time sequence to obtain the pulse signal.

7. The method for detecting the cun-gu-chi positions of the pulse wave based on vision according to claim 6, characterized in that, It further includes: Delete the speckle sub-region with the acquisition position of the "guan" from the set of speckle sub-regions, and obtain a second sorting; According to the relative positions of "cun" and "guan" and the relative positions of "cun" and "chi", determine the acquisition positions of "cun" and "chi" from the second sorting.

8. A device for detecting the cun-gu-chi positions of the pulse wave based on vision, characterized in that, The device includes: A region of interest extraction module, configured to obtain a wrist video based on a preset wrist fluctuation acquisition device, and extract a region of interest from the wrist video; A pulse signal determination module, configured to obtain a wrist digital speckle video including only the region of interest, divide the region of interest in each frame of the wrist digital speckle video into blocks to obtain a number of adjacent and non-overlapping speckle sub-regions, and determine the pulse signal of each speckle sub-region according to the speckle sub-region; A collection position determination module, configured to determine the signal difference between each speckle sub-region based on the pulse signals of each speckle sub-region, and determine the collection positions of cun, guan, and chi according to the signal difference and the prior knowledge of the human physiological structure; The collection position determination module includes: A heart rate calculation unit, configured to calculate the heart rate of each speckle sub-region based on the pulse signals of each speckle sub-region, and delete the speckle sub-regions with abnormal heart rates to obtain the remaining speckle sub-regions; A similarity calculation unit, configured to perform single-cycle segmentation on the remaining speckle sub-regions, calculate the single-cycle similarity between the remaining speckle sub-regions, and screen out the speckle sub-regions with a single-cycle similarity exceeding a preset similarity according to the single-cycle similarity to obtain a set of speckle sub-regions; An amplitude sorting unit, configured to sort the set of speckle sub-regions in descending order of amplitude intensity to obtain a first sorting; A guan position determination unit, configured to determine a speckle sub-region close to the center of the region of interest from the set of speckle sub-regions according to the first sorting as the collection position of guan, and the amplitude intensity of the speckle sub-region corresponding to the collection position of guan is the largest.

9. A terminal device Characterized in that The terminal device includes a memory, a processor, and a vision-based pulse wave cun-guan-chi position detection program stored in the memory and executable on the processor. When the processor executes the vision-based pulse wave cun-guan-chi position detection program, the steps of the vision-based pulse wave cun-guan-chi position detection method according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium Characterized in that A vision-based pulse wave cun-guan-chi position detection program is stored on the computer-readable storage medium. When the vision-based pulse wave cun-guan-chi position detection program is executed by the processor, the steps of the vision-based pulse wave cun-guan-chi position detection method according to any one of claims 1-7 are implemented.

Images