Remote pulse rate measurement system based on video images

Abstract

The invention discloses a remote pulse rate measurement system based on video images, and relates to the field of image communication technology. The remote pulse rate measurement system based on video images includes a video acquisition module, an image stabilization processing module, a motion correction algorithm module, and a real-time feedback and correction module. The acquisition module is used to acquire original image contrast, original sharpness, and original noise level parameters, capture continuous high-resolution video frames, and record the number of original video frames; the stabilized image processing module is used to analyze high-resolution video frames through a convolutional neural network model to predict pixel-level motion vectors, obtain predicted motion vectors, and obtain motion evaluation image stabilization evaluation values ​​through comprehensive analysis. Then, corresponding optimization measures are taken to solve the problem that motion artifacts caused by the head movement of the detected object or the movement of the camera affect the accuracy of pulse rate measurement.

Description

Technical Field

[0001] The invention relates to the technical field of image communication, in particular to a remote pulse rate measurement system based on video images. Background Art

[0002] In recent years, with the rapid development of information technology, video-based remote pulse rate measurement technology has been widely used in the field of remote medical monitoring and health management. This technology allows an individual's heart rate to be estimated by analyzing video images in a non-invasive way, providing a convenient and effective way to monitor health.

[0003] However, despite the progress made in many aspects, existing video-based pulse rate measurement systems still face some important challenges in practical applications. First, these systems often lack sufficient real-time processing capabilities when processing high-resolution video data and cannot output measurement results in a timely manner, which directly affects the user experience and practicality of the system. Second, the system has difficulty in accurately measuring pulse rate in dynamic environments, especially when there is slight movement of the user or the camera, and the accuracy is affected. In addition, when the system faces complex environmental changes, such as lighting conditions and background interference, it often fails to maintain stable and accurate output. These environmental factors greatly limit the reliability and accuracy of the system in a wider range of application scenarios.

[0004] In the prior art, due to insufficient real-time processing capability and sensitivity to dynamic environments and environmental changes, there is a problem of being unable to accurately measure pulse rate in real time and adapt to complex environmental changes.

[0005] Therefore, in view of the above problems, a remote pulse rate measurement system based on video images is urgently needed. Summary of the invention

[0006] Technical issues solved

[0007] In view of the deficiencies of the prior art, the present invention provides a remote pulse rate measurement system based on video images, which solves the problem that the accuracy of pulse rate measurement is affected by motion artifacts caused by the head movement of the detected object or the movement of the camera.

[0008] Technical Solution

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions: a remote pulse rate measurement system based on video images, including a video acquisition module, an image stabilization processing module, a motion correction algorithm module and a real-time feedback and correction module, wherein the acquisition module is used to acquire original image contrast, original sharpness and original noise level parameters, capture continuous high-resolution video frames, and record the number of original video frames; the stabilized image processing module is used to analyze high-resolution video frames through a convolutional neural network model to predict pixel-level motion vectors, obtain predicted motion vectors, obtain actual motion vectors through a high-precision sensor, and obtain a motion evaluation image stabilization evaluation value based on a comprehensive analysis of the number of original video frames, the predicted motion vectors and the actual motion vectors; the image stabilization evaluation value is compared with a preset first threshold, and the motion between the original video frames is predicted and compensated, thereby outputting a stabilized video frame and a stabilized motion vector, and correcting the original image contrast, original sharpness and original noise level parameters. The sharpness and original noise level parameters are used to obtain the corrected image contrast, corrected sharpness and corrected noise level parameters; the dynamic correction algorithm module is used to accept the stabilized video frame and the stabilized motion vector in the self-image stabilization processing module; the accelerated robust feature algorithm is used to evaluate the relative displacement and transformation between frames to obtain the corrected motion vector; the image quality improvement index is obtained by comprehensive calculation of the original image contrast, original sharpness, original noise level parameters, corrected image contrast, corrected sharpness and corrected noise level parameters; the motion correction evaluation value is calculated according to the corrected motion vector, the stabilized motion vector, the stabilized video frame and the image quality improvement index; the real-time feedback and correction module is used to compare the motion correction evaluation value corresponding to the real-time monitoring data with the preset second threshold value to obtain a motion correction analysis report; according to the motion correction analysis report, corresponding dynamic adjustment and optimization are made to offset motion artifacts and output clear video data.

[0010] Furthermore, the specific process of analyzing the high-resolution video frames through the convolutional neural network model to predict the pixel-level motion vector and obtain the predicted motion vector is as follows: constructing a neural network model through historical actual motion vectors and continuous high-resolution video frames, obtaining the actual motion vectors and continuous high-resolution video frames from the acquisition module, and using the streamer method to annotate the actual motion vectors and continuous high-resolution video frames to generate accurate motion vectors as training data; configuring and using a graphics processing algorithm to train the model, accelerating the training process and applying cross-validation technology to optimize the model and prevent overfitting; using real-time received video frames to output the predicted motion vector through model training.

[0011] Furthermore, the specific calculation process of the motion evaluation image stabilization evaluation value is as follows:

[0012]

[0013] Where MA represents the image stabilization evaluation value, V p (i) represents the predicted motion vector of the i-th frame, V t (i) represents the actual motion vector of the i-th frame, and N represents the number of original video frames.

[0014] Furthermore, the specific calculation process of the image quality improvement index is:

[0015] QI(i)=w C ·(C a (i)-C b (i))+w S ·(S a (i)-S b (i))+w N ·(N b (i)-N a (i))

[0016] Where QI(i) represents the image quality improvement index of the i-th frame, C a (i) represents the contrast of the i-th frame after correction, C b (i) represents the original contrast of the i-th frame, w C represents the contrast improvement weight factor, S a (i) represents the sharpness of the i-th frame after correction, S b (i) represents the original sharpness of the i-th frame, w S represents the sharpness improvement weight factor, N a (i) represents the noise level parameter after correction of the i-th frame, N b (i) represents the original noise level parameter of the i-th frame, w N Represents the noise level improvement weight factor.

[0017] Furthermore, the image stabilization evaluation value is compared with a preset first threshold, and the motion between the original video frames is predicted and compensated, so as to output a stabilized video frame and a stabilized motion vector. The specific process is as follows: the image stabilization evaluation value is compared with the first threshold in real time; when the image stabilization evaluation value is higher than the first threshold, the image of this frame is not processed; when the image stabilization evaluation value is lower than the first threshold, the image processing technology is used to adjust the position and direction of the video frame that does not meet the standard to offset the motion artifact; the processed video frame will be re-encoded into a stabilized video frame, and the actual motion vector of the corresponding frame number will be updated at the same time, and the processed motion vector will be re-encoded into a stable motion vector.

[0018] Furthermore, the motion correction evaluation value acquisition process is: obtaining the correction motion vector, stable motion vector, stable video frame and image quality improvement index data, and obtaining the motion correction evaluation value based on comprehensive analysis of the correction motion vector, stable motion vector, stable video frame and image quality improvement index.

[0019] Furthermore, the specific calculation process of the motion correction evaluation value is:

[0020]

[0021] Where MCV represents the motion correction evaluation value, V c (i) represents the corrected motion vector of the i-th frame, V s (i) represents the stabilized motion vector of the i-th frame, N′ represents the stabilized video frame, and QI(i) represents the image quality improvement index of the i-th frame.

[0022] Furthermore, the motion correction evaluation value corresponding to the real-time monitoring data is compared with a preset second threshold to obtain a motion correction analysis report, and corresponding dynamic adjustments and optimizations are made according to the motion correction analysis report to offset motion artifacts and output clear video data. The specific process is: receiving the motion correction evaluation value calculated in the real-time feedback and correction module, comparing the motion correction evaluation value with the preset second threshold in real time, and when the motion correction evaluation value is always higher than the second threshold, no adjustment is made; when the motion correction evaluation value is lower than the second threshold, it indicates that there are obvious motion artifacts or image quality problems in the video, and the frequency of comparing the motion correction evaluation value with the second threshold is increased, and the image stabilization processing module is repeatedly executed until the motion correction evaluation value is increased to above the second threshold.

[0023] Furthermore, the specific process of using the accelerated robust feature algorithm to evaluate the relative displacement and transformation between frames to obtain the corrected motion vector is as follows: identifying key points in consecutive high-resolution video frames, and generating descriptions for the key points that include detailed features of the surrounding areas; feature matching: determining matching feature points by comparing feature descriptions between consecutive high-resolution video frames, and screening out high-quality matching pairs; motion evaluation: calculating the geometric transformation between the two frames based on the matched feature points, the geometric transformation including position change values, rotation change values, and scale change values; generating the corrected motion vector: generating the corrected motion vector based on the geometric transformation.

[0024] Furthermore, the specific process of generating the correction motion vector according to the geometric transformation is: merging the position change value, rotation change value and scale change value into a matrix, using the matrix to adjust the pixel position in all high-resolution video frames, and marking the corresponding motion vector after adjustment as the correction motion vector.

[0025] Beneficial Effects

[0026] The present invention has the following beneficial effects:

[0027] (1) This video image-based remote pulse rate measurement system, by combining advanced image stabilization processing and motion correction algorithms, can effectively offset motion artifacts caused by user or camera movement, thereby improving the accuracy of pulse rate measurement. This achieves the effect of improving measurement accuracy and effectively solves the problem of inaccurate data caused by motion artifacts in the prior art.

[0028] (2) This video image-based remote pulse rate measurement system significantly improves the speed of processing high-resolution video frames by optimizing model training and real-time video data processing using graphics processing algorithms. This enhances the effect of real-time processing and feedback, and effectively solves the problem of insufficient real-time performance in existing technologies.

[0029] (3) This remote pulse rate measurement system based on video images optimizes the video quality by correcting the contrast, sharpness and noise level of the original image, providing users with clearer and more stable video output. This improves the image quality and effectively solves the problem in the prior art that the image quality is affected by environmental changes.

[0030] (4) The remote pulse rate measurement system based on video images dynamically adjusts and optimizes the image processing process through real-time monitoring and analysis of the motion correction analysis report, thereby achieving a highly adaptive dynamic adjustment effect, effectively solving the problem that the existing technology cannot adapt to changes in complex monitoring environments.

[0031] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a structural diagram of the remote pulse rate measurement system based on video images of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1The embodiment of the present invention provides a technical solution: a remote pulse rate measurement system based on video images, including a video acquisition module, an image stabilization processing module, a motion correction algorithm module and a real-time feedback and correction module, wherein the acquisition module is used to acquire original image contrast, original sharpness and original noise level parameters, capture continuous high-resolution video frames, and record the number of original video frames; the stabilized image processing module is used to analyze the high-resolution video frames through a convolutional neural network model to predict pixel-level motion vectors, obtain predicted motion vectors, obtain actual motion vectors through a high-precision sensor, and obtain a motion evaluation image stabilization evaluation value based on a comprehensive analysis of the number of original video frames, the predicted motion vectors and the actual motion vectors; the image stabilization evaluation value is compared with a preset first threshold, and the motion between the number of original video frames is predicted and compensated, thereby outputting a stabilized video frame and a stabilized motion vector, and correcting the original image contrast, original sharpness and original noise level parameters. The initial noise level parameters are used to obtain the corrected image contrast, corrected sharpness and corrected noise level parameters; the dynamic correction algorithm module is used to accept the stabilized video frame and the stabilized motion vector in the self-image stabilization processing module; the accelerated robust feature algorithm is used to evaluate the relative displacement and transformation between frames to obtain the corrected motion vector; the image quality improvement index is obtained by comprehensive calculation of the original image contrast, original sharpness, original noise level parameters, corrected image contrast, corrected sharpness and corrected noise level parameters; the motion correction evaluation value is calculated according to the corrected motion vector, the stabilized motion vector, the stabilized video frame and the image quality improvement index; the real-time feedback and correction module is used to compare the motion correction evaluation value corresponding to the real-time monitoring data with the preset second threshold value to obtain a motion correction analysis report; according to the motion correction analysis report, corresponding dynamic adjustment and optimization are made to offset motion artifacts and output clear video data.

[0035] Specifically, the convolutional neural network model is used to analyze high-resolution video frames to predict pixel-level motion vectors, and the specific process of obtaining the predicted motion vectors is as follows:

[0036] First, the system builds a neural network model through historical actual motion vectors and continuous high-resolution video frames. This involves obtaining actual motion vectors and video frames from the video acquisition module, and then annotating these data using the streamer method to generate accurate motion vectors as data for model training. This step ensures the quality and relevance of the training data, laying a solid foundation for subsequent model training.

[0037] Next, we configured and used efficient graphics processing algorithms to train the neural network model. At this stage, we significantly increased the processing speed by using graphics processing units to accelerate the training process. In addition, we used cross-validation techniques to not only optimize the model, but also effectively prevent overfitting, ensuring the model’s strong generalization capabilities.

[0038] Finally, the system uses the real-time received video frames to output the predicted motion vectors through the trained model. This step is the key to achieving dynamic and real-time pulse rate measurement, enabling the system to quickly respond and adjust in practical applications to provide accurate pulse rate data. The implementation of this method not only enhances the accuracy of the measurement, but also improves the application capability and stability of the system in complex dynamic environments. Through the integration of these technical steps, the system can effectively provide high-quality and real-time health monitoring services to meet the needs of modern telemedicine.

[0039] In this implementation, in a remote pulse rate measurement system based on video images, a convolutional neural network model is used to analyze high-resolution video frames and predict pixel-level motion vectors, a step that greatly enhances the core functionality of the system. By training the neural network with historical and real-time motion data, the system is able to generate accurate motion vectors, which is essential for stabilizing video frames and improving the accuracy of pulse rate measurement. Model training uses efficient graphics processing algorithms and cross-validation techniques, which not only speeds up the processing process but also improves the accuracy and generalization of the model. In this way, the system can receive and process video frames in real time in actual operation and output predicted motion vectors, thereby ensuring accurate measurement of pulse rate in dynamic environments. The implementation of the entire process significantly improves the performance of the system, making it a reliable and effective tool suitable for the needs of modern telemedicine and health monitoring.

[0040] Specifically, the specific calculation process of the motion evaluation image stabilization evaluation value is as follows:

[0041]

[0042] Where M A represents the image stabilization evaluation value, V p (i) represents the predicted motion vector of the i-th frame, V t (i) represents the actual motion vector of the i-th frame, and N represents the number of original video frames.

[0043] This calculation method provides a quantitative stabilization index for each frame by quantifying the difference in the Euclidean distance between the predicted vector and the actual vector. This index not only helps to evaluate the stability of a single frame, but also provides a comprehensive evaluation of the stabilization effect of the entire video sequence by analyzing the average of all frames. In this way, the system can identify and adjust unstable video frames caused by camera shake or fast target movement, thereby improving the accuracy and reliability of pulse rate measurement.

[0044] In this embodiment, the step of calculating the image stabilization evaluation value is crucial. The image stabilization evaluation value effectively measures the stability of each frame of video by comparing the deviation between the predicted motion vector and the actual motion vector. This calculation not only helps to identify unstable frames caused by camera shaking or rapid target movement, but also enables the system to perform specific image processing on these frames to reduce errors and improve overall video quality. The whole system is more accurate and reliable in measuring pulse rate in dynamic environments.

[0045] Specifically, the specific calculation process of the image quality improvement index is:

[0046] QI(i)=w C ·(C a (i)-C b (i))+w S ·(S a (i)-S b (i))+w N ·(N b (i)-N a (i))

[0047] Where QI(i) represents the image quality improvement index of the i-th frame, C a (i) represents the contrast of the i-th frame after correction, C b (i) represents the original contrast of the i-th frame, w C represents the contrast improvement weight factor, S a (i) represents the sharpness of the i-th frame after correction, S b (i) represents the original sharpness of the i-th frame, w S represents the sharpness improvement weight factor, N a (i) represents the noise level parameter after correction of the i-th frame, N b (i) represents the original noise level parameter of the i-th frame, w N Denotes the noise level improvement weight factor. Weight factor w C 、、w S and w S They are the weights of contrast, sharpness and noise level improvement respectively, and their values ​​are derived based on experience or through optimization algorithms to ensure that different aspects of image quality improvement are properly considered.

[0048] The acquisition of weight factors usually depends on the design requirements of the system and the expected user experience. In some applications, the improvement of contrast may be more important, while in other applications, the optimization of sharpness or noise level may be more critical. The weight factors can be adjusted through experiments or learned from a large amount of image data using machine learning methods to adapt to specific application scenarios and improve the overall performance of the system.

[0049] In this embodiment, in the remote pulse rate measurement system based on video images, calculating the image quality improvement index is an important step to improve the overall video processing quality. This index quantifies the effect of image processing by comparing the contrast, sharpness and noise level before and after correction. Using this index, the system can specifically evaluate the degree of improvement of each frame of video after processing to ensure that the visual quality of the image is effectively improved. By setting a suitable weight factor w C 、、w S and w S The system can flexibly adjust the degree of optimization of different image quality parameters to better adapt to different application requirements and environmental changes. This not only enhances the clarity and viewing experience of the image, but also improves the accuracy of pulse rate detection, which is an indispensable part of remote health monitoring.

[0050] Specifically, by comparing the image stabilization evaluation value with the preset first threshold, and predicting and compensating the motion between the original video frames, the stable video frame and the stable motion vector are output. The specific process is as follows: the system first compares the image stabilization evaluation value of each frame with the preset first threshold in real time. When the image stabilization evaluation value is higher than the first threshold, the image of this frame is not processed, because the image is stable enough at this time and no further adjustment is required. When the image stabilization evaluation value is lower than the first threshold, the system will use image processing technology to adjust the video frame that does not meet the standard, including adjusting the position and direction of the frame to offset the motion artifact. The processed video frame will be re-encoded into a stable video frame to ensure that the output video is more stable and clear. At the same time, the system will also update the actual motion vectors corresponding to these frames, and re-encode the processed motion vectors into stable motion vectors. In this way, the system can effectively compensate for the image instability caused by camera shaking or target movement, thereby providing high-quality stable video output and improving the accuracy and reliability of pulse rate measurement.

[0051] In this embodiment, in the remote pulse rate measurement system based on video images, by comparing the image stabilization evaluation value with the preset first threshold in real time and adjusting the position and direction of the video frames below the threshold, the system can effectively offset the motion artifacts. The processed video frames are re-encoded into stable video frames, and the corresponding motion vectors are updated to ensure that the output video is more stable and clear. This process significantly improves the stability and quality of the video, thereby enhancing the accuracy and reliability of the pulse rate measurement.

[0052] Specifically, the process of obtaining the motion correction evaluation value is as follows: first, the system obtains the correction motion vector, stable motion vector, stable video frame and image quality improvement index data. These data come from different types of motion and quality parameters generated by the system when processing and stabilizing video frames. Then, the system performs a comprehensive analysis based on these correction motion vectors, stable motion vectors, stable video frames and image quality improvement index data to calculate the motion correction evaluation value. This evaluation value reflects the overall stability and correction effect of the video frame, ensuring that the system can continuously provide high-quality video output and improve the accuracy and reliability of pulse rate measurement. Through this process, the system can effectively monitor and optimize the video processing and stabilization process to ensure optimal performance in various usage environments.

[0053] In this implementation, by obtaining the corrected motion vector, stabilized motion vector, stabilized video frame, and image quality improvement index data, and performing a comprehensive analysis to calculate the motion correction evaluation value, this step significantly improves the overall stability and correction effect of the video frame. The motion correction evaluation value provides a quantitative indicator that can accurately reflect the stability of the video frame and ensure that the system continuously outputs high-quality video. This not only improves the accuracy and reliability of pulse rate measurement, but also enables the system to maintain optimal performance in various environments.

[0054] Specifically, the specific calculation process of the motion correction evaluation value is:

[0055]

[0056] Where MCV represents the motion correction evaluation value, V c (i) represents the corrected motion vector of the i-th frame, V s (i) represents the stable motion vector of the i-th frame, N′ represents the stable video frame, and QI(i) represents the image quality improvement index of the i-th frame. α in the formula corresponds to the weight factor of the motion vector error, and β corresponds to the weight factor of the image quality improvement, which is used to balance the relative importance of the motion vector error and the image quality improvement. Through this calculation method, the system can comprehensively evaluate the correction effect and image quality of each frame of video, thereby ensuring that the output video data is optimal in terms of stability and clarity.

[0057] In this implementation, by calculating the motion correction evaluation value, the system can comprehensively evaluate the correction effect and image quality of each frame of video. The specific calculation process combines the error between the correction motion vector and the stable motion vector, as well as the image quality improvement index, and balances the relative importance of these two factors through weight factors α and β. Through this process, the system significantly improves the overall quality and stability of the video, thereby improving the accuracy and reliability of pulse rate measurement, and effectively solves the measurement errors caused by motion artifacts and image quality problems.

[0058] Specifically, the motion correction evaluation value corresponding to the real-time monitoring data is compared with the preset second threshold to obtain a motion correction analysis report, and the corresponding dynamic adjustment and optimization are made according to the motion correction analysis report to offset the motion artifacts and output clear video data. The specific process is: the system first receives the motion correction evaluation value calculated in the real-time feedback and correction module, and compares it with the preset second threshold in real time. When the motion correction evaluation value is always higher than the second threshold, the system believes that the current video quality is good and no adjustment is required. However, when the motion correction evaluation value is lower than the second threshold, this indicates that there are obvious motion artifacts or image quality problems in the video. At this time, the system will increase the comparison frequency of the motion correction evaluation value and the second threshold, and repeatedly execute the image stabilization processing module to adjust the position and direction of the frame until the motion correction evaluation value is raised above the second threshold. Through this process, the system can dynamically detect and correct problems in the video, ensure that the final output video data is clear and stable, and effectively improve the accuracy and reliability of pulse rate measurement.

[0059] In this embodiment, by monitoring the motion correction evaluation value in real time and comparing it with a preset second threshold, the system can dynamically detect and correct motion artifacts in the video. When the motion correction evaluation value is lower than the threshold, the system increases the comparison frequency and repeats the image stabilization process until the evaluation value reaches the standard. This process ensures the clarity and stability of the video data, thereby significantly improving the accuracy and reliability of pulse rate measurement.

[0060] Specifically, the accelerated robust feature algorithm is used to evaluate the relative displacement and transformation between frames to obtain the corrected motion vector. The specific process is as follows: First, key points are identified in consecutive high-resolution video frames, and descriptions containing detailed features of the surrounding area are generated for these key points. Next, through feature matching, the feature descriptions between consecutive high-resolution video frames are compared, the matching feature points are determined, and high-quality matching pairs are screened out. Then, motion evaluation is performed, and the geometric transformation between the two frames is calculated based on these matched feature points. These geometric transformations include position change values, rotation change values, and scale change values. Finally, the corrected motion vector is generated based on the calculated geometric transformation. Through this process, the system can accurately evaluate and correct the relative motion between video frames, thereby improving the stability of the video and the accuracy of the measurement.

[0061] In this implementation, by using an accelerated robust feature algorithm to evaluate the relative displacement and transformation between frames and generate correction motion vectors, the system can significantly improve the stability of the video and the accuracy of the measurement. Specifically, the process identifies and matches key points in the video frames, calculates the position, rotation, and scale changes between frames, and then generates correction motion vectors, effectively correcting the unstable factors in the video. Ultimately, this step ensures the continuity and clarity of the video data, and enhances the reliability and accuracy of the pulse rate measurement.

[0062] Specifically, the specific process of generating a correction motion vector based on geometric transformation is as follows: First, by identifying and matching key points in consecutive high-resolution video frames, the system calculates the position change values, rotation change values, and scale change values ​​between frames. Next, these change values ​​are combined into a comprehensive geometric transformation matrix. Using this matrix, the system adjusts the pixel positions in all high-resolution video frames to eliminate instability caused by camera movement or object motion. Finally, the corresponding motion vector in the adjusted video frame is marked as a correction motion vector. Through this process, the system can ensure the continuity and clarity of the video data, significantly improving the stability of the video and the accuracy of the pulse rate measurement.

[0063] In this embodiment, by combining the position change value, the rotation change value and the scale change value into a comprehensive geometric transformation matrix and using the matrix to adjust the pixel positions in all high-resolution video frames, the system can effectively correct the unstable factors in the video. The adjusted motion vector is marked as a correction motion vector. This process ensures the continuity and clarity of the video data and significantly improves the stability of the video and the accuracy of the pulse rate measurement.

[0064] It should be noted that, in this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0065] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A remote pulse rate measurement system based on video images, comprising a video acquisition module, an image stabilization processing module, a motion correction algorithm module and a real-time feedback and correction module, characterized in that: The video acquisition module is used to acquire original image contrast, original sharpness and original noise level parameters, capture continuous high-resolution video frames, and record the number of original video frames; The image stabilization processing module is used to analyze high-resolution video frames through a convolutional neural network model to predict pixel-level motion vectors, obtain predicted motion vectors, obtain actual motion vectors through a high-precision sensor, and obtain a motion evaluation image stabilization evaluation value based on a comprehensive analysis of the original video frame number, predicted motion vectors and actual motion vectors; By comparing the image stabilization evaluation value with a preset first threshold, and predicting and compensating the motion between the original video frames, a stabilized video frame and a stabilized motion vector are output, and the original image contrast, original sharpness and original noise level parameters are corrected to obtain corrected image contrast, corrected sharpness and corrected noise level parameters; The motion correction algorithm module is used to receive the stabilized video frame and the stabilized motion vector from the image stabilization processing module; The accelerated robust feature algorithm is used to evaluate the relative displacement and transformation between frames to obtain the corrected motion vector; The image quality improvement index is obtained by comprehensively calculating the original image contrast, original sharpness, original noise level parameters, corrected image contrast, corrected sharpness and corrected noise level parameters; A motion correction evaluation value is calculated based on the corrected motion vector, the stabilized motion vector, the stabilized video frame and the image quality improvement index; The real-time feedback and correction module is used to compare the motion correction evaluation value corresponding to the real-time monitoring data with a preset second threshold value to obtain a motion correction analysis report; According to the motion correction analysis report, corresponding dynamic adjustments and optimizations are made to offset motion artifacts and output clear video data.

2. The video image-based remote pulse rate measurement system according to claim 1, characterized in that: The specific process of analyzing the high-resolution video frame by the convolutional neural network model to predict the pixel-level motion vector and obtain the predicted motion vector is as follows: A neural network model is constructed through historical actual motion vectors and continuous high-resolution video frames, actual motion vectors and continuous high-resolution video frames are obtained from the video acquisition module, and the actual motion vectors and continuous high-resolution video frames are annotated using the streamer method to generate accurate motion vectors as training data; Configure and use graph processing algorithms for model training, accelerate the training process and apply cross-validation techniques to optimize the model and prevent overfitting; The real-time received video frames are used to output the predicted motion vectors through model training.

3. The video image-based remote pulse rate measurement system according to claim 1, characterized in that: The specific calculation process of the motion evaluation image stabilization evaluation value is as follows: In the formula, represents the image stabilization evaluation value, represents the predicted motion vector of the i-th frame, represents the actual motion vector of the i-th frame, and N represents the number of original video frames.

4. The video image-based remote pulse rate measurement system according to claim 1, characterized in that: The specific calculation process of the image quality improvement index is as follows: In the formula, represents the image quality improvement index of the i-th frame, represents the contrast of the i-th frame after correction, represents the original contrast of the i-th frame, represents the contrast improvement weight factor, represents the sharpness of the i-th frame after correction, represents the original sharpness of the i-th frame, represents the sharpness improvement weight factor, represents the noise level parameter after correction of the i-th frame, represents the original noise level parameter of the i-th frame, Represents the noise level improvement weight factor.

5. The video image-based remote pulse rate measurement system according to claim 1, characterized in that: The specific process of comparing the image stabilization evaluation value with a preset first threshold value, predicting and compensating the motion between the original video frames, and outputting a stabilized video frame and a stabilized motion vector is as follows: Compare the image stabilization evaluation value with the first threshold in real time. When the image stabilization evaluation value is higher than the first threshold, do not process the image of this frame. When the image stabilization evaluation value is lower than the first threshold, use image processing technology to adjust the position and direction of the video frame that does not meet the standard to offset motion artifacts. The processed video frames will be re-encoded into stable video frames, and the actual motion vectors corresponding to the frame numbers will be updated at the same time, and the processed motion vectors will be re-encoded into stable motion vectors.

6. The video image-based remote pulse rate measurement system according to claim 1, characterized in that: The process of obtaining the motion correction evaluation value is as follows: The corrected motion vector, the stabilized motion vector, the stabilized video frame and the image quality improvement index data are obtained, and the motion correction evaluation value is obtained based on the comprehensive analysis of the corrected motion vector, the stabilized motion vector, the stabilized video frame and the image quality improvement index.

7. The video image-based remote pulse rate measurement system according to claim 1, characterized in that: The specific calculation process of the motion correction evaluation value is: In the formula, represents the motion correction evaluation value, represents the corrected motion vector of the i-th frame, represents the stable motion vector of the i-th frame, Indicates the frame number of the stable video frame, represents the image quality improvement index of the i-th frame, α corresponds to the weight factor of the motion vector error, and β corresponds to the weight factor of the image quality improvement.

8. The video image-based remote pulse rate measurement system according to claim 1, characterized in that: The motion correction evaluation value corresponding to the real-time monitoring data is compared with the preset second threshold value to obtain a motion correction analysis report, and corresponding dynamic adjustment and optimization are made according to the motion correction analysis report to offset motion artifacts and output clear video data. The specific process is: Receiving the motion correction evaluation value calculated in the real-time feedback and correction module, comparing the motion correction evaluation value with a preset second threshold in real time, and when the motion correction evaluation value is always higher than the second threshold, no adjustment is made; When the motion correction evaluation value is lower than the second threshold, it indicates that there are obvious motion artifacts in the video. The frequency of comparing the motion correction evaluation value with the second threshold is increased, and the image stabilization processing module is repeatedly executed until the motion correction evaluation value is increased to above the second threshold.

9. The video image-based remote pulse rate measurement system according to claim 1, characterized in that: The specific process of using the accelerated robust feature algorithm to evaluate the relative displacement and transformation between frames and obtain the corrected motion vector is as follows: Identify key points in consecutive high-resolution video frames and generate descriptions for the key points that include detailed features of the surrounding area; Feature matching: By comparing the feature descriptions between consecutive high-resolution video frames, matching feature points are determined and high-quality matching pairs are selected; Motion evaluation: Calculate the geometric transformation between two frames based on the matched feature points, the geometric transformation includes position change value, rotation change value and scale change value; Generate Correction Motion Vectors: Generates correction motion vectors based on geometric transformations.

10. The video image-based remote pulse rate measurement system according to claim 9, characterized in that: The specific process of generating the correction motion vector according to the geometric transformation is as follows: The position change values, rotation change values ​​and scale change values ​​are combined into a matrix, the matrix is ​​used to adjust the pixel positions in all high-resolution video frames, and the corresponding motion vectors after adjustment are marked as corrected motion vectors.

Images