Foot sole video evaluation method and device based on time difference of arrival of pulse waves of two feet, terminal equipment and storage medium

Abstract

The invention relates to the technical field of biomedical engineering. The invention discloses a foot sole video evaluation method and device based on the time difference of arrival of pulse waves of two feet, terminal equipment and a storage medium. The accuracy of an evaluation result can be improved. The method comprises the following steps: acquiring sole videos of both feet of a target user; extracting a first pixel time domain change signal of each pixel point in the foot sole region of each frame of image according to the foot sole video to obtain a pulse wave signal corresponding to each foot sole region; determining a plurality of regions of interest in the sole video based on the pulse wave signals corresponding to the sole regions to obtain a plurality of target regions of interest; extracting a target rPPG signal corresponding to each foot based on the plurality of target regions of interest, and constructing a biped time difference wave used for reflecting the time difference of arrival change of the target rPPG signals corresponding to the biped; and performing evaluation processing based on the biped time difference wave to obtain an evaluation result, and using the evaluation result to assist in diagnosing the peripheral artery disease and the severity of the peripheral artery disease.

Description

Technical Field

[0001] This application relates to the field of biomedical engineering technology. More specifically, this application relates to a plantar video evaluation method, device, terminal device, and storage medium based on the time difference of bipedal pulse wave arrival times. Background Art

[0002] Traditional methods for evaluating peripheral artery disease involve collecting pulse wave data; extracting the main wave amplitude and dicrotic notch amplitude from the collected pulse wave data; calculating the vascular resistance coefficient based on the main wave amplitude, dicrotic notch amplitude, and preset rules; and evaluating the vascular resistance coefficient according to preset rules to obtain an evaluation result, enabling medical staff to assist in diagnosing peripheral artery disease based on this evaluation result. Among them, the step of evaluating the vascular resistance coefficient according to preset rules to obtain an evaluation result includes analyzing the linear relationship between the vascular resistance coefficient and the vascular resistance condition based on historical data, setting N thresholds for the vascular resistance coefficient according to the vascular resistance condition in historical data, and dividing into grades to obtain N + 1 progressive grades; comparing the vascular resistance coefficient with the N thresholds to determine the grade where the vascular resistance coefficient is located and outputting the evaluation result of the located grade. That is to say, the evaluation result of the traditional method is limited by historical data. If there are outliers in the historical data, it will affect the accuracy of the evaluation result, and further cause medical staff to be unable to accurately diagnose peripheral artery disease based on the evaluation result. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a plantar video evaluation method, device, terminal device, and storage medium based on the time difference of bipedal pulse wave arrival times, which can improve the accuracy of the evaluation result. The embodiments of this application are mainly implemented through the following technical solutions: In the first aspect of the embodiments of this application, a plantar video evaluation method based on the time difference of bipedal pulse wave arrival times is provided, including: Obtaining a plantar video of the two feet of a target user; Extracting the first pixel time-domain change signal of each pixel point in the plantar region of both feet in each frame image according to the plantar video to obtain the pulse wave signal corresponding to each plantar region; Based on the pulse wave signals corresponding to each plantar region, determining multiple regions of interest in the plantar video to obtain multiple target regions of interest; Extracting the target rPPG signal corresponding to each foot based on the multiple target regions of interest and constructing a bipedal time difference wave, where the bipedal time difference wave is used to reflect the change in the arrival time difference of the target rPPG signals corresponding to the two feet; Performing evaluation processing based on the bipedal time difference wave to obtain the evaluation result of the plantar video.

[0004] According to an embodiment of the present application, the steps of extracting the first pixel time-domain change signal of each pixel point in the biped plantar region of each frame image from the plantar video to obtain the pulse wave signal corresponding to each plantar region include: Generating the first pixel time-domain change signal of each plantar region based on the time sequence of each frame image in the plantar video and the pixel value corresponding to each pixel point in each plantar region of each frame image; Performing downsampling processing on the first pixel time-domain change signal of each plantar region according to a preset first sliding time window and a first step length to obtain the pulse wave signal corresponding to each plantar region.

[0005] According to an embodiment of the present application, the steps of performing downsampling processing on the first pixel time-domain change signal of each plantar region according to a preset first sliding time window and a first step length to obtain the pulse wave signal corresponding to each plantar region include: Performing division processing on each first pixel time-domain change signal based on the first sliding time window and the first step length to obtain a plurality of first sub-signals corresponding to each first pixel time-domain change signal; Filtering all the first sub-signals by using a filter to obtain the pulse wave signal corresponding to each plantar region.

[0006] According to an embodiment of the present application, the steps of determining a plurality of regions of interest in the plantar video based on the pulse wave signal corresponding to each plantar region to obtain a plurality of target regions of interest include: Converting the pulse wave signal corresponding to each plantar region into the frequency domain and calculating the signal-to-noise ratio of each foot region; Sorting according to the signal-to-noise ratio of each foot region and selecting the plantar regions that meet the preset conditions as candidate regions of interest; Performing optimization processing on each candidate region of interest to obtain the target region of interest corresponding to each candidate region of interest.

[0007] According to an embodiment of the present application, the steps of extracting the target rPPG signal corresponding to each foot based on the plurality of target regions of interest and constructing a biped time difference wave, where the biped time difference wave is used to reflect the change in the arrival time difference of the target rPPG signals corresponding to the two feet include: Extracting the second pixel time-domain change signal of each pixel point in the plurality of target regions of interest in the plantar video to obtain the initial rPPG signal corresponding to each foot; Calculating the average value of the initial rPPG signal corresponding to each foot to obtain the target rPPG signal corresponding to each foot; Using a target algorithm to calculate the time difference between the first feature point and the second feature point among all the target rPPG signals to determine a plurality of biped pulse wave arrival time differences; Generate the bipedal time difference wave according to the multiple bipedal pulse wave arrival time differences.

[0008] According to an embodiment of the present application, when the target algorithm is a peak detection algorithm, calculating the time difference between the first feature point and the second feature point among all target rPPG signals by using the target algorithm, the steps of determining the multiple bipedal pulse wave arrival time differences include: Perform partitioning processing on the target rPPG signal corresponding to the left foot based on a preset third sliding time window and a third step length to obtain multiple third left sub-signals; Perform partitioning processing on the target rPPG signal corresponding to the right foot based on the third sliding time window and the third step length to obtain multiple third right sub-signals; Detect the first peak of the target third left sub-signal and the second peak of the target third right sub-signal by using the peak detection algorithm, where the target third left sub-signal is any one of the multiple third left sub-signals, and the target third right sub-signal is the signal in the multiple third right sub-signals that is at the same time point as the target third left sub-signal; Calculate the time difference corresponding to the target third left sub-signal based on the first peak and the second peak.

[0009] According to an embodiment of the present application, the steps of performing evaluation processing based on the bipedal time difference wave to obtain the evaluation result of the plantar video include: Perform feature extraction processing on the bipedal time difference wave to obtain at least one target feature; Input the at least one target feature into a classifier for evaluation processing to obtain the evaluation result of the plantar video.

[0010] In a second aspect of the embodiments of the present application, there is provided a plantar video evaluation device based on bipedal pulse wave arrival time differences, including: a terminal device, an optical imaging device, a fixing bracket, and a transparent plate, where the optical imaging device includes a camera and an LED fill light; The terminal device includes a processor, and the processor is used to execute the steps of the plantar video evaluation method provided in the first aspect of the embodiments of the present application; The transparent plate is placed on the upper part of the fixing bracket. During the process of the camera recording the plantar video of the target user's two feet, the target user's two feet are placed on the transparent plate, and the LED fill light irradiates the transparent plate; The optical imaging device is placed inside the fixing bracket; After recording the plantar video, the camera sends the plantar video to the processor.

[0011] In a third aspect of the embodiments of the present application, a terminal device is provided, including: a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the steps of the method for evaluating a plantar video based on the time difference of bipedal pulse wave arrival time provided in the first aspect of the embodiments of the present application.

[0012] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium is used to store a computer program, and the computer program enables a computer to execute the steps of the method for evaluating a plantar video based on the time difference of bipedal pulse wave arrival time provided in the first aspect of the embodiments of the present application.

[0013] The beneficial effects of the embodiments of the present application include: The core of the embodiments of the present application is to use the bipedal time difference wave of the target rPPG signals corresponding to the two feet of the target user to obtain an evaluation result, so that medical staff can assist in diagnosing peripheral artery disease and the severity of peripheral artery disease based on the evaluation result. Specifically, the embodiments of the present application obtain the plantar videos of the two feet of the target user; extract the first pixel time-domain change signals of each pixel point in the plantar regions of both feet in each frame image according to the plantar videos to obtain the pulse wave signals corresponding to each plantar region; determine multiple regions of interest in the plantar videos based on the pulse wave signals corresponding to each plantar region to obtain multiple target regions of interest; extract the target rPPG signals corresponding to each foot based on the multiple target regions of interest, and construct a bipedal time difference wave, where the bipedal time difference wave is used to reflect the change in the arrival time difference of the target rPPG signals corresponding to the two feet; perform evaluation processing based on the bipedal time difference wave to obtain the evaluation result of the plantar video. Compared with the prior art, the embodiments of the present application do not use historical data as an evaluation factor for evaluation, but use the arrival time difference of the target rPPG signals corresponding to the two feet for evaluation, thereby avoiding the situation that the evaluation result is affected by abnormal values in historical data, and thus improving the accuracy of the evaluation result, enabling medical staff to accurately evaluate the lower extremity atherosclerosis situation. Description of the Drawings

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0015] Figure 1 It is a flowchart of the method for evaluating a plantar video based on the time difference of bipedal pulse wave arrival time of the present application in some embodiments; Figure 2 Flow chart of the plantar video evaluation method based on the time difference of bipedal pulse wave arrival time in some embodiments of the present application; Figure 3 Reference diagrams of the target left plantar rPPG signal and the target right plantar rPPG signal in the present application; Figure 4 Principle block diagram of the plantar video evaluation device based on the time difference of bipedal pulse wave arrival time in some embodiments of the present application; Figure 5 Principle block diagram of the optical imaging device in some embodiments of the present application; Figure 6 Principle block diagram of the terminal device in some embodiments of the present application. Detailed implementation manners

[0016] To make the above objects, features, and advantages of the present application more obvious and understandable, the following will describe the detailed implementation manners of the present application in conjunction with the accompanying drawings. Many specific details are set forth in the following description to fully understand the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present application. Therefore, the present application is not limited by the specific embodiments disclosed below.

[0017] It should be noted that the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present application, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0018] The term "exemplary" or "for example" and the like are used to indicate examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of the term "exemplary" or "for example" and the like is intended to present the relevant concepts in a specific manner.

[0019] The term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0020] The term "Peripheral Artery Disease (PAD)" is a common cardiovascular disease, and its main pathological feature is atherosclerosis of peripheral arteries. This lesion mainly occurs in the intima layer of medium and large arteries, showing chronic and progressive characteristics. The main pathological features of atherosclerosis include lipid deposition and fibrous tissue hyperplasia in the arterial intima, ultimately leading to stenosis or complete occlusion of the blood vessel lumen.

[0021] Unless otherwise defined, all technical and scientific terms used in the specification of this application have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in the specification of this application includes any and all combinations of one or more of the related listed items.

[0022] The following further describes the specific embodiments of this application with reference to the accompanying drawings.

[0023] Reference Figure 1 As shown, it is a flowchart of a plantar video evaluation method based on the time difference of bipedal pulse wave arrival time provided in the first aspect of the embodiment of this application. In Figure 1 it, the plantar video evaluation method based on the time difference of bipedal pulse wave arrival time includes: S1. Obtain the plantar videos of both feet of the target user.

[0024] The plantar videos are obtained by recording with the camera of the plantar video evaluation device based on the time difference of bipedal pulse wave arrival time.

[0025] The plantar videos are RGB videos. The RGB video refers to video data encoded and represented using the RGB (red, green, blue) color space.

[0026] In the embodiment of this application, the size of each frame image in the plantar videos is 480px×300px, where px is the pixel unit. In other embodiments, the size of each frame image in the plantar videos can be set by those skilled in the art according to actual needs.

[0027] Step S1 can also refer to Figure 2 the "Obtain plantar videos" step in

[0028] S2. Extract the first pixel time-domain change signals of each pixel point in the plantar regions of both feet in each frame image of the plantar videos to obtain the pulse wave signals corresponding to each plantar region.

[0029] Further, step S2 includes: S21. Generate a first pixel time-domain variation signal for each plantar region based on the time sequence of each frame image in the plantar video and the pixel values corresponding to each pixel point in each plantar region of each frame image.

[0030] Exemplarily, generate a first pixel time-domain variation signal for the left plantar region based on the time sequence of each frame image in the plantar video and the pixel values corresponding to each pixel point in the left plantar region; generate a first pixel time-domain variation signal for the right plantar region based on the time sequence of each frame image in the plantar video and the pixel values corresponding to each pixel point in the right plantar region.

[0031] It should be understood that each pixel point in a left plantar region corresponds to one of the first pixel time-domain variation signals of the left plantar region, and the pixel values of the pixel points at the same position in multiple left plantar regions are connected to form one first pixel time-domain variation signal of the left plantar region. Similarly, each pixel point in a right plantar region corresponds to one of the first pixel time-domain variation signals of the right plantar region, and the pixel values of the pixel points at the same position in multiple right plantar regions are connected to form one first pixel time-domain variation signal of the right plantar region.

[0032] Further, step S21 includes: S211. After performing downsampling processing on each frame image in the plantar video, extract the double-plantar regions in each frame image and determine the pixel values corresponding to each pixel point in each plantar region.

[0033] Specifically, the step of performing downsampling processing on each frame image in the plantar video is to perform average pooling processing on each frame image. In the embodiments of the present application, average pooling processing is performed with a pooling window of 50px×50px to achieve downsampling. In other embodiments, the specific size of the pooling window can be set by those skilled in the art according to actual needs.

[0034] It should be understood that the purpose of performing the downsampling operation on the plantar video is to reduce the amount of calculation, reduce quantization error, weaken the influence of noise on a single pixel point, and enhance the robustness of pulse wave signal extraction.

[0035] In other embodiments, max pooling, min pooling, superpixel image segmentation method, pixel fusion method or linear interpolation method can be used to replace the average pooling for downsampling to achieve regional pixel fusion. Further, the step of extracting the bipedal plantar regions in each frame of image can be implemented by using an object detection algorithm. The object detection algorithm can be R-CNN (Region-based Convolutional Network), Fast R-CNN (Fast Region-based Convolutional Network) or Faster R-CNN (Faster Region-based Convolutional Neural Networks).

[0036] The extraction of the bipedal plantar regions in each frame of image can refer to Figure 2 the step of "selecting the region of interest on the sole" in

[0037] Further, the pixel values corresponding to each pixel point in each plantar region are RGB three-channel pixel values or single-channel pixel values. Since the signal-to-noise ratio, contrast and sensitivity of the G (Green) channel are all better than those of the R (Red) channel and the B (Blue) channel, therefore, the G-channel pixel values in each plantar region are selected in the embodiments of the present application.

[0038] When the pixel values corresponding to each pixel point in each plantar region are RGB three-channel pixel values, the information of multiple channels can be fused, the quality of extracting the first pixel time-domain change signal can be improved, and at the same time, the stability of the first pixel time-domain change signal can be enhanced.

[0039] When the pixel values corresponding to each pixel point in each plantar region are single-channel pixel values, the speed of extracting the first pixel time-domain change signal can be increased.

[0040] It should be understood that RGB described herein refers to the three colors of red, green and blue.

[0041] S212. Generate the first pixel time-domain change signal of each plantar region based on the time sequence of each frame of image and the pixel values corresponding to each pixel point in each plantar region.

[0042] S22. Perform downsampling processing on the first pixel time-domain change signal of each plantar region according to a preset first sliding time window and a first step length to obtain the pulse wave signal corresponding to each plantar region.

[0043] In the embodiment of the present application, the length of the first sliding time window is 30 s, and the first step length is 5 s. In other embodiments, the specific values of the first sliding time window and the first step length can be set by those skilled in the art according to actual requirements.

[0044] Further, step S22 includes: S221. Perform a partitioning process on each first pixel time-domain change signal based on the first sliding time window and the first step length to obtain a plurality of first sub-signals corresponding to each first pixel time-domain change signal.

[0045] The plurality of first sub-signals are selected from a plurality of first left sub-signals or a plurality of first right sub-signals.

[0046] Specifically, perform a partitioning process on the first target pixel time-domain change signal in the left sole area based on the first sliding time window and the first step length to obtain a plurality of first left sub-signals corresponding to the first target pixel time-domain change signal in the left sole area; perform a partitioning process on the first target pixel time-domain change signal in the right sole area based on the first sliding time window and the first step length to obtain a plurality of first right sub-signals corresponding to the first target pixel time-domain change signal in the right sole area; the first target pixel time-domain change signal in the left sole area is one of the first pixel time-domain change signals of all the first pixel time-domain change signals in the left sole area; the first target pixel time-domain change signal in the right sole area is one of the first pixel time-domain change signals of all the first pixel time-domain change signals in the right sole area.

[0047] The first sub-signal is an RGB signal.

[0048] S222. Filter all the first sub-signals using a filter to obtain the pulse wave signals corresponding to each sole area.

[0049] In the embodiment of the present application, the filter is a Butterworth filter, and the filtering frequency of the Butterworth filter is set to [0.6, 4] Hz to remove the noise interference in each first sub-signal. In other embodiments, other filters can also be used to replace the Butterworth filter, and specifically, it can be set by those skilled in the art according to actual requirements.

[0050] It should be understood that the left plantar region contains multiple pulse wave signals. Each pulse wave signal is obtained by dividing a first pixel time-domain change signal in the left plantar region to obtain multiple first left sub-signals, and then filtering the multiple first left sub-signals. Similarly, the right plantar region also contains multiple pulse wave signals. Each pulse wave signal in the right plantar region is obtained by dividing a first pixel time-domain change signal in the right plantar region to obtain multiple first right sub-signals, and then filtering the multiple first right sub-signals.

[0051] When the pixel values corresponding to the pixel points in each plantar region are RGB three-channel pixel values, the pulse wave signal is a three-channel pulse wave signal; when the pixel values corresponding to the pixel points in each plantar region are single-channel pixel values, the pulse wave signal is a single-channel pulse wave signal.

[0052] In the embodiment of the present application, the G (Green) channel signal is selected as the signal to be used.

[0053] S3. Determine multiple regions of interest in the plantar video based on the pulse wave signals corresponding to each plantar region, and obtain multiple target regions of interest. The S3 step can be understood as Figure 2 the step of "extracting multiple target regions of interest" in

[0054] Further, the S3 step includes: S31. Convert the pulse wave signals corresponding to each plantar region into the frequency domain, and calculate the signal-to-noise ratio of each plantar region.

[0055] Specifically, the S31 step includes: S311. Perform frequency domain conversion processing on each pulse wave signal corresponding to the left plantar region by using Fourier transform to obtain a left frequency domain signal corresponding to each pulse wave signal in the left plantar region.

[0056] S312. Calculate the first signal-to-noise ratio of each left frequency domain signal.

[0057] Further, the formula for calculating the first signal-to-noise ratio of the left frequency domain signal is: ; where is the first signal-to-noise ratio of the th left frequency domain signal; is the signal power of the th left frequency domain signal; is the noise power of the th left frequency domain signal; is power; is signal; is noise.

[0058] S313. Perform frequency-domain conversion processing on each pulse wave signal corresponding to the right plantar region using Fourier transform to obtain the right frequency-domain signal corresponding to each pulse wave signal in the right plantar region.

[0059] S314. Calculate the second signal-to-noise ratio of each right frequency-domain signal.

[0060] Further, the calculation formula for the second signal-to-noise ratio of the th right frequency-domain signal is ; where is the second signal-to-noise ratio of the th right frequency-domain signal; is the signal power of the th right frequency-domain signal; is the noise power of the

[0061] S315. Combine all the first signal-to-noise ratios and all the second signal-to-noise ratios to form the signal-to-noise ratios of each foot region.

[0062] S32. Sort according to the signal-to-noise ratios of each foot region, and select the plantar regions that meet the preset conditions as candidate regions of interest.

[0063] In the embodiments of the present application, the preset conditions are used to screen candidate regions of interest from all the plantar regions in the plantar video, and it can be a preset ratio threshold in the signal-to-noise ratio sorting. Taking the sorting from large to small as an example, this preset ratio threshold can be set to the top 20% or other values; taking the sorting from small to large as an example, this preset ratio threshold can be set to the bottom 20% or other values. The embodiments of the present application do not make specific limitations. For example, in the sorting process from large to small, select the plantar regions with the top 20% signal-to-noise ratio in the sorting as candidate regions of interest.

[0064] Specifically, perform sorting processing on all the first signal-to-noise ratios, and select the plantar regions corresponding to the first signal-to-noise ratios in the top 20% of all the first signal-to-noise ratios as candidate regions of interest; perform sorting processing on all the second signal-to-noise ratios, and select the plantar regions corresponding to the second signal-to-noise ratios in the top 20% of all the second signal-to-noise ratios as candidate regions of interest as well.

[0065] The sorting processing is either sorting from large to small or sorting from small to large. The specific sorting method of the sorting processing can be set by those skilled in the art according to actual needs.

[0066] S33. Perform optimization processing on each candidate region of interest to obtain the target region of interest corresponding to each candidate region of interest.

[0067] Specifically, step S33 includes: S331. Upsample each candidate region of interest to obtain a first region to be processed corresponding to each candidate region of interest.

[0068] The upsampling process is inverse average pooling. The inverse average pooling is used to upsample to the size of the original video stream image, that is, the size of the original image of the plantar video. In other embodiments, the upsampling process can also be other processes, which can be specifically set by those skilled in the art according to actual needs.

[0069] Further, step S331 includes: S3311. Upsample each candidate region of interest to obtain a third region to be processed corresponding to each candidate region of interest.

[0070] S3312. Perform boundary processing on each third region to be processed to obtain a first region to be processed corresponding to each candidate region of interest.

[0071] The boundary processing can be zero-padding or mirror extension. In other embodiments, the boundary processing can also be other processing methods, which can be specifically set by those skilled in the art according to actual needs.

[0072] S332. Perform closing operation on each first region to be processed to obtain a second region to be processed corresponding to each first region to be processed.

[0073] The closing operation refers to performing dilation first and then erosion on each first region to be processed, so as to obtain a second region to be processed corresponding to each first region to be processed.

[0074] The closing operation is used to fill small holes or slits in each first region to be processed, connect adjacent objects, and smooth the boundaries of objects.

[0075] Further, step S332 includes: S3321. Determine a first structuring element for morphological operations and the size of the first structuring element.

[0076] The first structuring element can be a rectangle, a circle, or an ellipse. In other embodiments, the first structuring element can also be a triangle or other shapes, which can be specifically set by those skilled in the art according to actual needs. The size of the first structuring element can also be set by those skilled in the art according to actual needs.

[0077] S3322. Perform dilation operation on each first region to be processed using the first structuring element and the size of the first structuring element to obtain a fourth region to be processed corresponding to each first region to be processed.

[0078] S3323. Perform an erosion operation on each fourth area to be processed using the first structural element and the size of the first structural element, and obtain a second area to be processed corresponding to each first area to be processed.

[0079] S333. Perform an opening operation on each second area to be processed, and obtain a target region of interest corresponding to each second area to be processed.

[0080] The opening operation refers to performing an erosion operation first and then a dilation operation on each second area to be processed, so as to obtain a target region of interest corresponding to each second area to be processed.

[0081] The opening operation is used to remove small objects in each second area to be processed, break small connections between objects, and smooth the boundaries of larger objects.

[0082] Further, step S333 includes: S3331. Determine a second structural element for the morphological operation and the size of the second structural element.

[0083] The second structural element has the same shape as the first structural element. In other embodiments, the shape of the second structural element may also be different from the shape of the first structural element, and specifically may be set by those skilled in the art according to actual needs. The size of the second structural element may also be the same as the size of the first structural element. In other embodiments, the size of the second structural element may also be different from the size of the first structural element, and is specifically set by those skilled in the art according to actual needs.

[0084] S3332. Perform an erosion operation on each second area to be processed using the second structural element and the size of the second structural element, and obtain a fifth area to be processed corresponding to each second area to be processed.

[0085] S3333. Perform a dilation operation on each fifth area to be processed using the second structural element and the size of the second structural element, and obtain a target region of interest corresponding to each second area to be processed.

[0086] S4. Extract the target rPPG signals corresponding to each foot based on the multiple target regions of interest, and construct a bipedal time difference wave, which is used to reflect the change in the arrival time difference of the target rPPG signals corresponding to the two feet.

[0087] Further, step S4 includes: S41. Extract the second pixel time-domain change signals of each pixel point in the multiple target regions of interest in the plantar video, and obtain the initial rPPG signals corresponding to each foot.

[0088] Further, step S41 includes: S411. Generate a second pixel time-domain variation signal for each plantar region based on the chronological order of the multiple target regions of interest and the pixel values corresponding to each pixel point in the multiple target regions of interest in the plantar video.

[0089] It should be understood that in the multiple target regions of interest in the plantar video, each pixel point in a left plantar region corresponds to one of the second pixel time-domain variation signals of the left plantar region, and the pixel values of the pixel points at the same position in the multiple left plantar regions are connected to form one second pixel time-domain variation signal of the left plantar region. Similarly, each pixel point in a right plantar region corresponds to one of the second pixel time-domain variation signals of the right plantar region, and the pixel values of the pixel points at the same position in the multiple right plantar regions are connected to form one second pixel time-domain variation signal of the right plantar region.

[0090] The pixel values corresponding to each pixel point in the multiple target regions of interest in the plantar video are RGB three-channel pixel values or single-channel pixel values.

[0091] S412. Perform downsampling processing on the second pixel time-domain variation signal of each plantar region according to a preset second sliding time window and a second step length to obtain an initial rPPG signal corresponding to each foot.

[0092] It should be understood that the rPPG (Remote Photo Plethysmo Graphy) described in this article refers to remote photoplethysmography.

[0093] In the embodiments of the present application, the length of the second sliding time window may be the same as the length of the first sliding time window, and the specific value of the second step length may be the same as the specific value of the first step length. In other embodiments, the specific values of the second sliding time window and the second step length may also be set by those skilled in the art according to actual needs.

[0094] Further, step S412 includes: S4121. Perform partitioning processing on each second pixel time-domain variation signal based on the second sliding time window and the second step length to obtain a plurality of second sub-signals corresponding to each second pixel time-domain variation signal.

[0095] The plurality of second sub-signals are selected from a plurality of second left sub-signals or a plurality of second right sub-signals.

[0096] Specifically, based on the second sliding time window and the second step length, the time-domain change signal of the second target pixel in the left sole area is divided to obtain a plurality of second left sub-signals corresponding to the time-domain change signal of the second target pixel in the left sole area; based on the second sliding time window and the second step length, the time-domain change signal of the second target pixel in the right sole area is divided to obtain a plurality of second right sub-signals corresponding to the time-domain change signal of the second target pixel in the right sole area; the time-domain change signal of the second target pixel in the left sole area is one of the second pixel time-domain change signals of all the second pixel time-domain change signals in the left sole area; the time-domain change signal of the second target pixel in the right sole area is one of the second pixel time-domain change signals of all the second pixel time-domain change signals in the right sole area.

[0097] The second sub-signal is an RGB signal.

[0098] S4122. Filter all the second sub-signals using a filter to obtain the initial rPPG signals corresponding to each foot.

[0099] The filter is a Butterworth filter, and the filtering frequency of the Butterworth filter is set to [0.6, 4] Hz to remove the noise interference in each second sub-signal. In other embodiments, other filters can be used to replace the Butterworth filter, and the specific filter can be set by those skilled in the art according to actual needs.

[0100] The initial rPPG signal and the pulse wave signal are of the same type of signal.

[0101] It should be understood that the left sole area contains multiple initial rPPG signals. Each initial rPPG signal is obtained by dividing a second pixel time-domain change signal in the left sole area to obtain a plurality of second left sub-signals, and then filtering the plurality of second left sub-signals. Similarly, the right sole area also contains multiple initial rPPG signals. Each initial rPPG signal in the right sole area is obtained by dividing a second pixel time-domain change signal in the right sole area to obtain a plurality of second right sub-signals, and then filtering the plurality of second right sub-signals.

[0102] S42. Calculate the average value of the initial rPPG signals corresponding to each foot to obtain the target rPPG signals corresponding to each foot.

[0103] The average value includes a first average value and a second average value. Among them, the first average value is used to indicate the average value of all the initial rPPG signals corresponding to the left sole area at each time point; the second average value is used to indicate the average value of all the initial rPPG signals corresponding to the right sole area at each time point.

[0104] Further, step S42 includes: S421. Calculate the first average value of all initial rPPG signals corresponding to the left sole area at each time point.

[0105] S422. Form the target rPPG signal corresponding to the left foot by using all the first average values.

[0106] The target rPPG signal corresponding to the left foot can refer to Figure 3 the signal pointed to by label L in the figure.

[0107] In the embodiment of the present application, the left foot corresponds to only one target rPPG signal.

[0108] Step S422 can refer to Figure 2 the step of "extracting the target rPPG signal corresponding to the left foot" in

[0109] S423. Calculate the second average value of all initial rPPG signals corresponding to the right sole area at each time point.

[0110] S424. Form the target rPPG signal corresponding to the right foot by using all the second average values.

[0111] The target rPPG signal corresponding to the right foot can refer to Figure 3 the signal pointed to by label R in the figure.

[0112] In the embodiment of the present application, the right foot corresponds to only one target rPPG signal.

[0113] Step S424 can refer to Figure 2 the step of "extracting the target rPPG signal corresponding to the right foot" in

[0114] S43. Use the target algorithm to calculate the time difference between the first feature point and the second feature point among all the target rPPG signals, and determine a plurality of bipedal pulse wave arrival time differences.

[0115] The target algorithm can be a peak detection algorithm, a signal cross-correlation algorithm, a valley detection algorithm, etc., and the present embodiment does not make specific limitations.

[0116] When the target algorithm is a peak detection algorithm, the first feature point is the first wave peak, and the second feature point is the second wave peak.

[0117] Taking the target algorithm being a peak detection algorithm as an example, the embodiment of the present application elaborates on the specific implementation steps of S43, including: S431. Perform partitioning processing on the target rPPG signal corresponding to the left foot based on a preset third sliding time window and a third step length to obtain a plurality of third left sub-signals.

[0118] The specific values of the third sliding time window and the third step length can be set by those skilled in the art according to actual needs.

[0119] Each of the third left sub-signals has only one first peak.

[0120] S432. Perform a partitioning process on the target rPPG signal corresponding to the right foot based on the third sliding time window and the third step length to obtain a plurality of third right sub-signals.

[0121] Each of the third right sub-signals has only one second peak.

[0122] S433. Detect the first peak of the target third left sub-signal and the second peak of the target third right sub-signal by using a peak detection algorithm, where the target third left sub-signal is any one of the plurality of third left sub-signals, and the target third right sub-signal is the signal in the plurality of third right sub-signals that is at the same time point as the target third left sub-signal.

[0123] S434. Calculate the time difference corresponding to the target third left sub-signal based on the first peak and the second peak.

[0124] The time difference corresponding to the target third left sub-signal is a bipedal pulse wave arrival time difference.

[0125] Step S434 can refer to Figure 2 the "calculate time difference" step in

[0126] S435. Form the plurality of bipedal pulse wave arrival time differences from all the time differences.

[0127] S44. Generate the bipedal time difference wave based on the plurality of bipedal pulse wave arrival time differences.

[0128] Specifically, it can be to connect all the bipedal pulse wave arrival time differences to form the bipedal time difference wave. The bipedal time difference wave is a curve. In other embodiments, the bipedal time difference wave can also be a straight line, a stepped line segment, a serrated line segment, or other irregularly combined line segments, which can be specifically set by those skilled in the art according to actual needs.

[0129] S5. Perform an evaluation process based on the bipedal time difference wave to obtain the evaluation result of the plantar video.

[0130] Step S5 can refer to Figure 2 the "evaluate PAD" step in

[0131] Further, step S5 includes: S51. Extract feature from the biped time difference wave to obtain at least one target feature.

[0132] Further, the features extracted in step S51 are the mean value, standard deviation, maximum value, and / or minimum value of the biped time difference wave. That is, the at least one target feature is the mean value, standard deviation, maximum value, and / or minimum value.

[0133] Exemplarily, in this embodiment, the mean value, standard deviation, maximum value, and minimum value can also be input into the classifier for evaluation to improve the accuracy of the evaluation result.

[0134] In other embodiments, those skilled in the art can set the type of features to be extracted according to actual needs.

[0135] S52. Input the at least one target feature into a classifier for evaluation to obtain the evaluation result of the plantar video.

[0136] Medical staff can assist in diagnosing peripheral artery disease and the severity of peripheral artery disease based on the evaluation result.

[0137] The classifier is a classifier for supervised learning. Exemplarily, the classifier can be a support vector machine, random forest, or multilayer perceptron (MLP).

[0138] In other embodiments, the classifier can be replaced with other machine learning models.

[0139] Further, the classifier includes an input layer, a first hidden layer, a second hidden layer, and an output layer.

[0140] The specific steps for the classifier to evaluate the at least one target feature include: S521. After the input layer receives the at least one target feature, process the at least one target feature into a column vector to obtain a first feature vector. The first feature vector is a column vector of m×1, where m is the number of the at least one target feature. Exemplarily, when the at least one target feature is the mean value, standard deviation, maximum value, and minimum value of the biped time difference wave, the mean value is used as the first row data of the first feature vector, the standard deviation is used as the second row data of the first feature vector, the maximum value is used as the third row data of the first feature vector, and the minimum value is used as the fourth row data of the first feature vector.

[0141] S522. Input the first feature vector into the first hidden layer for feature extraction and non-linear transformation to obtain a second feature vector.

[0142] Further, step S522 can be implemented by the following formula: ; ; wherein, is the result of feature extraction by the first hidden layer; is the first weight matrix, representing the connection weight; is the first feature vector; is the first bias vector, used to adjust the activation threshold of the neuron; is the rectified linear unit function; is the second feature vector.

[0143] S523. Input the second feature vector into the second hidden layer for feature extraction and non-linear transformation processing to obtain a third feature vector.

[0144] Further, step S523 can be implemented by the following formula: ; ; wherein, is the result of feature extraction by the second hidden layer; is the second weight matrix; is the second bias vector; is the third feature vector; is the hyperbolic tangent function.

[0145] S524. Input the third feature vector into the output layer for transformation processing to obtain the evaluation result.

[0146] Further, step S524 can be implemented by the following formula: ; ; wherein, is the result of feature extraction by the output layer; is the third weight matrix; is the third bias vector; is the evaluation result; is the sigmoid growth curve, which is an activation function.

[0147] The core of the embodiment of the present application lies in using the bipedal time difference wave of the target rPPG signal corresponding to the biped of the target user to obtain an evaluation result, so that medical staff can assist in diagnosing peripheral artery disease and the severity of peripheral artery disease according to the evaluation result. Compared with the prior art, the embodiment of the present application does not use historical data as an evaluation factor for evaluation, but uses the arrival time difference of the target rPPG signal corresponding to the biped to conduct the evaluation, thereby avoiding the situation that the evaluation result is affected by abnormal values in the historical data. Therefore, the accuracy of the evaluation result can be improved, enabling medical staff to accurately evaluate the lower extremity atherosclerosis situation.

[0148] In some embodiments, the plantar video evaluation method based on the bipedal pulse wave arrival time difference further includes a training step of the classifier, and the training step of the classifier includes: S6. Obtain a training data set and a true label set, and each training data in the training data set has a one-to-one correspondence with one of the true labels in the true label set.

[0149] Each training data in the training data set is a plantar training video of the biped of the target user recorded by the camera.

[0150] Each true label in the true label set is a true ultrasonic waveform diagram.

[0151] The ultrasonic waveform diagram is obtained by a medical ultrasonic instrument.

[0152] S7. Perform feature extraction processing on the target training data to obtain training features, where the target training data is any training data in the training data set.

[0153] Specifically, the S7 step includes: S71. Extract the first pixel time-domain change training signal of each pixel point in the bipedal plantar region of each frame of the image according to the target training data to obtain the pulse wave training signal corresponding to each plantar region.

[0154] The implementation of the S71 step is the same as that of the S2 step and will not be elaborated here.

[0155] S72. Determine multiple regions of interest in the target training data based on the pulse wave training signal corresponding to each plantar region to obtain multiple target training regions.

[0156] The implementation of the S72 step is the same as that of the S3 step and will not be elaborated here.

[0157] S73. Extract the target rPPG training signals corresponding to each foot based on the multiple target training regions, and construct a bipedal time difference training wave, where the bipedal time difference training wave is used to reflect the change in the arrival time difference of the target rPPG training signals corresponding to the two feet.

[0158] The implementation of step S73 is the same as that of step S4, and will not be elaborated here.

[0159] S74. Perform feature extraction processing on the bipedal time difference training wave to obtain at least one target training feature.

[0160] The at least one target training feature is the average value, standard deviation, maximum value, and / or minimum value of the bipedal time difference training wave.

[0161] S75. Input the at least one target training feature into the original classifier for evaluation processing to obtain the prediction result of the target training data.

[0162] The original classifier is an original classifier for supervised learning. Exemplarily, the original classifier can be a support vector machine, a random forest, or a multilayer perceptron (MLP).

[0163] S76. Calculate the loss function based on the prediction result and the true label corresponding to the target training data.

[0164] Further, the calculation formula for step S76 is: ; where is the loss function; is the length of the training data set, that is, the total number of all training data; is the true label corresponding to the target training data, that is, the true label corresponding to the th training data in the training data set; is the prediction result, that is, the prediction result corresponding to the target training data.

[0165] In other embodiments, the specific calculation formula of the loss function can be set by those skilled in the art according to actual needs.

[0166] S77. Adjust the parameters of the original classifier based on the loss function to obtain the classifier.

[0167] In some embodiments, the embodiments of the present application use the time difference of bipedal pulse waves reaching the soles of the feet to evaluate PAD. Similar methods can further explore multiple indices to improve the diagnostic accuracy, such as adding human physiological parameters such as blood oxygen and blood pressure. In terms of ROI (Region of Interest) optimization, parameters such as peak deviation can be introduced to further optimize the ROI and obtain a more suitable rPPG signal for calculating PATD.

[0168] Reference Figure 4 As shown, it is a schematic block diagram of a plantar video evaluation device based on the time difference of bipedal pulse waves reaching the soles of the feet provided by the second aspect of the embodiments of the present application. In Figure 4 it, the plantar video evaluation device 100 based on the time difference of bipedal pulse waves reaching the soles of the feet includes: a terminal device 101, an optical imaging device 102, a fixing bracket 103, and a transparent plate 104. The optical imaging device 102 includes a camera 1021 and an LED fill light 1022. Reference Figure 5 As shown; The terminal device 101 includes a processor, and the processor is used to execute the steps of the plantar video evaluation method based on the time difference of bipedal pulse waves reaching the soles of the feet provided by the first aspect of the embodiments of the present application; The transparent plate 104 is placed on the upper part of the fixing bracket 103. During the process of the camera 1021 recording the plantar video of the bipedal feet of the target user, the bipedal feet of the target user are placed on the transparent plate 104, and the LED fill light 1022 irradiates the transparent plate 104; The optical imaging device 102 is placed inside the fixing bracket 103; After the camera 1021 finishes recording the plantar video, it sends the plantar video to the processor.

[0169] The camera 1021 is an RGB camera sensor. Specifically, the camera 1021 can be an RGB camera sensor of the IDS UI-3860-CP model. In other embodiments, the camera 1021 can be any RGB camera sensor including the RGB band of 400 - 700 nm. The camera 1021 is used to obtain a video of continuous image frames including the human skin.

[0170] The LED fill light 1022 is an LED lamp with a continuous broadband spectrum of 400 - 700 nm.

[0171] The optical imaging device 102, the fixing bracket 103, and the transparent plate 104 form a plantar imager.

[0172] The fixed bracket 103 is made of steel beams. In other embodiments, other stable and torsion-resistant materials can be used as the manufacturing materials of the fixed bracket 103.

[0173] The transparent plate 104 can be a glass plate or a fully transparent acrylic plate. In other embodiments, other transparent material plates can be placed above the fixed bracket 103 for placing the two feet.

[0174] The plantar video evaluation device based on the time difference of arrival of bipedal pulse waves provided by the embodiments of the present application does not require the use of additional auxiliary tools such as straps or clips to restrain the target user to obtain the evaluation results for medical staff to assist in the diagnosis of peripheral artery disease. Therefore, the embodiments of the present application can improve the comfort of the target user during the evaluation process. At the same time, since no auxiliary tools such as straps or clips are used, the embodiments of the present application will not occur the situation that the auxiliary tools fall off the target user and affect the evaluation results. Therefore, the embodiments of the present application can also improve the accuracy of the evaluation results, so that medical staff can accurately evaluate the lower limb atherosclerosis situation.

[0175] The embodiments of the present application use the method of plantar pressurization to enhance the rPPG signal. Therefore, the target user needs to place the soles of the feet on the transparent plate 104. This method greatly reduces the noise interference of the plantar mechanical movement on the signal, makes the measurement more accurate, and ensures the comfort of the target user during the diagnosis process.

[0176] The embodiments of the present application make the PAD detection convenient and daily, and realize the use scenario of PAD detection from the hospital to life. The system of the embodiments of the present application is simple to operate and has low professional requirements, making the application scenario of the system more extensive. During the measurement process, the target user only needs to place the two feet on the transparent plate 104 for 2 minutes, and the evaluation situation of PAD can be obtained according to the algorithm, making the home detection of PAD popularized.

[0177] The third aspect of the embodiments of the present application provides a terminal device, and the principle block diagram of the terminal device can be as Figure 6As shown in the figure. The terminal device includes a processor, a memory, a network interface, a display screen, and a temperature sensor connected through a system bus. Among them, the processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a plantar video evaluation method based on the time difference of arrival of bipedal pulse waves is implemented. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0178] Those skilled in the art can understand that Figure 6 the block diagram of the principle shown in the figure is only a block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the terminal device to which the solution of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0179] In some embodiments, the present application provides a terminal device. The terminal device includes a processor and a memory. The memory is used to store a computer program. The processor is used to call and run the computer program stored in the memory and execute the steps of the plantar video evaluation method based on the time difference of arrival of bipedal pulse waves provided in the first aspect of the embodiments of the present application. In the fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium is used to store a computer program. The computer program causes a computer to execute the steps of the plantar video evaluation method based on the time difference of arrival of bipedal pulse waves provided in the first aspect of the embodiments of the present application.

[0180] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0181] Without changing the basic principle of the present application, the technical features of the above embodiments can be combined. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0182] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it cannot be understood as a limitation on the scope of the patent application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. A foot video evaluation method based on the arrival time difference of the pulse waves of both feet, characterized in that: include: Obtaining videos of the soles of the feet of the target user; Extracting the first pixel time domain change signal of each pixel point in the plantar area of ​​both feet in each frame image according to the plantar video to obtain the pulse wave signal corresponding to each plantar area; Determine multiple regions of interest in the sole video based on the pulse wave signals corresponding to each sole region, and obtain multiple target regions of interest; Extracting target rPPG signals corresponding to each foot based on the multiple target regions of interest, and constructing a two-foot time difference wave, wherein the two-foot time difference wave is used to reflect the change in arrival time difference of the target rPPG signals corresponding to the two feet; An evaluation process is performed based on the double-foot time difference wave to obtain an evaluation result of the sole video.

2. The foot video evaluation method based on the arrival time difference of the pulse waves of both feet according to claim 1 is characterized in that: The step of extracting the first pixel time domain change signal of each pixel point in the sole area of ​​both feet of each frame image according to the sole video to obtain the pulse wave signal corresponding to each sole area includes: Generate a first pixel time domain change signal for each sole area based on the time sequence of each frame image in the sole video and the pixel value corresponding to each pixel point in each sole area of ​​each frame image; The first pixel time domain variation signal of each sole area is downsampled according to the preset first sliding time window and the first step length to obtain the pulse wave signal corresponding to each sole area.

3. The foot video evaluation method based on the arrival time difference of the double-foot pulse waves according to claim 2 is characterized in that: The step of downsampling the first pixel time domain change signal of each sole area according to the preset first sliding time window and the first step length to obtain the pulse wave signal corresponding to each sole area includes: Based on the first sliding time window and the first step length, each first pixel time-domain variation signal is divided and processed to obtain a plurality of first sub-signals corresponding to each first pixel time-domain variation signal; A filter is used to filter all the first sub-signals to obtain the pulse wave signal corresponding to each sole area.

4. The foot video evaluation method based on the arrival time difference of the double-foot pulse waves according to claim 1 is characterized in that: The step of determining a plurality of regions of interest in the sole video based on the pulse wave signals corresponding to each sole region, and obtaining a plurality of target regions of interest comprises: The pulse wave signal corresponding to each foot area is converted into the frequency domain, and the signal-to-noise ratio of each foot area is calculated; Sort the foot regions according to their signal-to-noise ratios, and select the plantar regions that meet the preset conditions as candidate regions of interest; Each candidate region of interest is optimized to obtain a target region of interest corresponding to each candidate region of interest.

5. The foot video evaluation method based on the arrival time difference of the double-foot pulse waves according to claim 1 is characterized in that: The steps of extracting target rPPG signals corresponding to each foot based on the multiple target regions of interest, and constructing a two-foot time difference wave, wherein the two-foot time difference wave is used to reflect the change in the arrival time difference of the target rPPG signals corresponding to the two feet include: Extracting a second pixel time domain change signal of each pixel point in multiple target regions of interest in the sole video to obtain an initial rPPG signal corresponding to each foot; Calculate the average value of the initial rPPG signal corresponding to each foot to obtain the target rPPG signal corresponding to each foot; The target algorithm is used to calculate the time difference between the first characteristic point and the second characteristic point between all target rPPG signals, and the arrival time difference of multiple bipedal pulse waves is determined; The bipedal time difference wave is generated according to the plurality of bipedal pulse wave arrival time differences.

6. The foot video evaluation method based on the arrival time difference of the pulse waves of both feet according to claim 5 is characterized in that: In the case where the target algorithm is a peak detection algorithm, the step of using the target algorithm to calculate the time difference between the first feature point and the second feature point between all target rPPG signals and determining the arrival time differences of multiple bipedal pulse waves includes: Based on a preset third sliding time window and a third step length, the target rPPG signal corresponding to the left foot is divided and processed to obtain a plurality of third left sub-signals; Based on the third sliding time window and the third step length, the target rPPG signal corresponding to the right foot is divided and processed to obtain a plurality of third right sub-signals; A peak detection algorithm is used to detect a first peak of a target third left sub-signal and a second peak of a target third right sub-signal, wherein the target third left sub-signal is any one of the plurality of third left sub-signals, and the target third right sub-signal is a signal at the same time point as the target third left sub-signal in the plurality of third right sub-signals; A time difference corresponding to the target third left sub-signal is calculated based on the first wave peak and the second wave peak.

7. The foot video evaluation method based on the arrival time difference of the double-foot pulse waves according to claim 1 is characterized in that: The step of performing evaluation processing based on the two-foot time difference wave to obtain the evaluation result of the sole video includes: Performing feature extraction processing on the bipedal time difference wave to obtain at least one target feature; The at least one target feature is input into a classifier for evaluation processing to obtain an evaluation result of the sole video.

8. A foot video evaluation device based on the arrival time difference of the pulse waves of both feet, characterized in that: include: Terminal equipment, optical imaging equipment, a fixing bracket and a transparent plate, wherein the optical imaging equipment includes a camera and an LED fill light; The terminal device comprises a processor, and the processor is used to execute the steps of the foot video evaluation method based on the arrival time difference of the double foot pulse waves as described in any one of claims 1 to 7 above; The transparent plate is placed on the upper part of the fixed bracket, and when the camera is recording the sole video of the feet of the target user, the feet of the target user are placed on the transparent plate, and the LED fill light is irradiated onto the transparent plate; The optical imaging device is placed inside the fixed bracket; After recording the sole video, the camera sends the sole video to the processor.

9. A terminal device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the steps of the plantar video assessment method based on the arrival time difference of the pulse waves of both feet as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the steps of the plantar video evaluation method based on the arrival time difference of the pulse waves of both feet as described in any one of claims 1 to 7.

Images