Dual-spectrum super-resolution enhanced non-contact multi-physiological parameter synchronous detection method

Abstract

The application provides a kind of dual-spectrum super-resolution enhanced non-contact multi-physiological parameter synchronous detection method, it is related to physiological parameter detection field, the method comprises: visible light image and thermal infrared image are collected and image processing is carried out to obtain visible light image sequence and thermal infrared image sequence;Each frame of image in visible light image sequence and thermal infrared image sequence is carried out enhancement processing, and visible light target sequence and thermal infrared image target sequence are obtained;rPPG signal and respiration signal are extracted;Signal processing is carried out to generate aligned first discrete time signal sequence and second discrete time signal sequence.The application can carry out non-contact synchronous detection to heart rate, respiratory rate and other multi-physiological parameters under strong interference conditions of illumination and motion, by image enhancement, signal extraction and signal data alignment processing, generate two groups of signal sequences with constant sampling rate and accurate alignment in time dimension, so as to accurately evaluate the physiological state of individual based on multi-physiological parameter.

Description

Technical Field

[0001] This application relates to the field of physiological parameter detection technology, specifically to a non-contact method for simultaneous detection of multiple physiological parameters using dual-spectral super-resolution enhancement. Background Technology

[0002] Breathing is an important physiological activity, and heart rate reflects a person's cardiac function. The speed, variability, and regularity of heart rate can be used to assess an individual's cardiac health. Both breathing and heart rate are important indicators for physiological monitoring. Simultaneous measurement of multiple physiological parameters can comprehensively reflect an individual's physiological state, facilitating accurate assessment.

[0003] In related technologies, physiological detection often employs wearable devices or medical instruments to quantitatively assess physiological and psychological responses by capturing autonomic nervous system symptoms triggered by emotions. However, these methods generally suffer from poor convenience and universality, as well as interference from consciousness. The key to solving the challenge of simultaneous detection under strong interference lies in achieving non-contact measurement of multiple physiological parameters applicable to various scenarios while minimizing the impact of complex lighting, motion, posture, and video quality conditions. Currently, research on the mixed representation of facial morphology, spectral, and thermal distribution characteristics during cardiac cycles and respiration remains immature, making it difficult to perform non-contact simultaneous detection of multiple physiological parameters such as heart rate and respiratory rate under conditions of strong light and motion interference. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this application provides a non-contact method for simultaneous detection of multiple physiological parameters using dual-spectral super-resolution enhancement, which solves the current problem of difficulty in performing non-contact simultaneous detection of multiple physiological parameters such as heart rate and respiratory rate under conditions of strong light and motion interference.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] In a first aspect, embodiments of this application provide a non-contact method for simultaneous detection of multiple physiological parameters using dual-spectral super-resolution enhancement. The method includes: acquiring visible light and thermal infrared images using a constructed dual-spectral integrated acquisition module, and performing image processing to obtain visible light image sequences and thermal infrared image sequences; processing each frame of the visible light and thermal infrared image sequences based on a preset image super-resolution enhancement module to obtain enhanced visible light target sequences and thermal infrared image target sequences; extracting rPPG signals and respiratory signals from the visible light target sequences and thermal infrared image target sequences, respectively; processing the extracted rPPG signals and respiratory signals to generate a first discrete-time signal sequence and a second discrete-time signal sequence with a constant sampling rate and precise alignment in the time dimension; and synchronously outputting the first discrete-time signal sequence and the second discrete-time signal sequence, which correspond to the rPPG signal and the respiratory signal, respectively.

[0007] According to a first aspect of the embodiments of this application, the dual-spectrum integrated acquisition module includes a visible light camera and a thermal infrared camera, which are fixed on the same base to ensure that their optical centers are on the same horizontal reference plane. The dual-spectrum integrated acquisition module also includes an adjustment mechanism, which is used to control the thermal infrared camera to rotate horizontally around its own vertical axis of optical center, so as to set the deflection angle of the thermal infrared camera according to a preset working distance and optimize the overlapping field of view of the thermal infrared camera and the visible light camera. The working distance is the distance between the target face and the dual-spectrum integrated acquisition module.

[0008] According to a first aspect of the embodiments of this application, the aforementioned acquisition of visible light images and thermal infrared images by the constructed dual-spectrum integrated acquisition module, and the processing of the images to obtain visible light image sequences and thermal infrared image sequences, specifically includes the following steps: after fixing the relative positions of the visible light camera and the thermal infrared camera, a temperature difference calibration plate is used as a calibration reference; the temperature difference calibration plate is assembled based on a preset working distance, so that the temperature difference calibration plate is completely imaged within the overlapping field of view of the thermal infrared camera and the visible light camera; through a synchronous triggering mechanism, visible light images and thermal infrared images are acquired simultaneously, and at least 10 sets of image pairs at different positions and angles are obtained by adjusting the pose of the temperature difference calibration plate; for all acquired visible light images, a standard camera calibration algorithm is applied to automatically detect and extract the coordinates of the first corner point of the temperature difference calibration plate; and the first intrinsic parameter matrix K of the visible light camera is calculated. rgb With the first distortion coefficient D rgb For all acquired thermal infrared images, image processing techniques adapted to thermal imaging characteristics are used to extract the coordinates of the second corner points in the same group corresponding to the first corner point coordinates, and the second intrinsic parameter matrix K of the thermal infrared camera is calculated. thermal With the second distortion coefficient D thermal .

[0009] According to a first aspect of the embodiments of this application, the coordinates of the first corner point and the coordinates of the second corner point constitute a corner point coordinate set; the first intrinsic parameter matrix K rgb Second intrinsic parameter matrix K thermal The intrinsic parameter matrix constituting the two cameras; the first distortion coefficient D rgb With the second distortion coefficient D thermal The distortion coefficients that make up the two cameras.

[0010] According to a first aspect of the embodiments of this application, the aforementioned acquisition of visible light images and thermal infrared images by the constructed dual-spectrum integrated acquisition module, and the processing of images to obtain visible light image sequences and thermal infrared image sequences, may further include the following steps: performing a stereo calibration algorithm based on the corresponding corner coordinate sets within each image pair, the intrinsic parameter matrices of the two cameras, and the distortion coefficients of the two cameras; iteratively optimizing the stereo calibration algorithm to minimize the reprojection error of all corner pairs on the imaging planes of the two cameras, and solving for the rotation matrix R and translation vector T describing the spatial relationship between the thermal infrared camera coordinate system and the visible light camera coordinate system; determining the extrinsic parameter matrix (R, T) based on the rotation matrix R and translation vector T to form a rigid transformation connecting the thermal infrared camera coordinate system and the visible light camera coordinate system; and utilizing the calibrated first distortion coefficient D... rgb Second distortion coefficient D thermal Distortion correction is performed on both the visible light image and the thermal infrared image. The multiple visible light images after distortion correction constitute a visible light image sequence. Based on the obtained intrinsic and extrinsic parameter matrices (R,T) of the two cameras, a perspective transformation is applied to the distortion-corrected thermal infrared image, reprojecting the pixels of the thermal infrared image onto the imaging plane of the visible light camera to generate a new thermal infrared image. The new thermal infrared image is perfectly aligned with the corresponding visible light image at a preset working distance by eliminating the parallax caused by the physical separation of the cameras. The multiple new thermal infrared images constitute a thermal infrared image sequence.

[0011] According to a first aspect of the embodiments of this application, the image super-resolution enhancement module includes a Fast Super-Resolution Convolutional Neural Network (FSRCNN); the Fast Super-Resolution Convolutional Neural Network (FSRCNN) includes: a feature extraction layer, a shrinking layer, a nonlinear mapping layer, and an expanding layer.

[0012] According to a first aspect of the embodiments of this application, the aforementioned image super-resolution enhancement module based on a preset process processes each frame of the visible light image sequence and the thermal infrared image sequence to obtain an enhanced visible light target sequence and a thermal infrared image target sequence. Specifically, the process may include the following steps: inputting each frame of the visible light image sequence and the thermal infrared image sequence into a 5x5 two-dimensional convolutional feature extraction layer to extract shallow features of the input low-resolution image; compressing the feature dimension through a 1x1 two-dimensional convolutional shrinking layer to reduce subsequent computational complexity; learning the complex mapping relationship from low-resolution to high-resolution features in the dimensionality-reduced feature space using a nonlinear mapping layer composed of multiple 3x3 two-dimensional convolutions stacked together; restoring the feature dimension through a 1x1 two-dimensional convolutional expansion layer; and finally reconstructing a high-resolution image by performing an upsampling operation through a transposed convolutional layer; wherein, multiple high-resolution images corresponding to the visible light image sequence constitute the visible light target sequence, and multiple high-resolution images corresponding to the thermal infrared image sequence constitute the thermal infrared image target sequence.

[0013] According to a first aspect of the embodiments of this application, the aforementioned extraction of rPPG signals and respiratory signals from visible light target sequences and thermal infrared image target sequences respectively may specifically include the following steps: selecting local images of corresponding facial regions from the visible light target sequence and performing scaling and normalization processing to obtain a standard visible light sequence; inputting the standard visible light sequence into a pre-trained rPPG signal extraction network to extract rPPG signals; processing the thermal infrared image target sequence frame by frame using a thermal infrared nose ROI detection method to obtain a mixed respiratory signal sequence containing valid respiratory signals and missing regions; wherein, the thermal infrared nose ROI detection method is based on single-frame missing region determination; and performing signal completion and reconstruction on the mixed respiratory signal sequence to obtain a reconstructed respiratory signal sequence for extracting respiratory signals.

[0014] According to a first aspect of the embodiments of this application, each rPPG signal and each respiratory signal are composed of timestamp-amplitude data pairs, and multiple rPPG signals and multiple respiratory signals are signal segments to be processed.

[0015] The aforementioned processing of the extracted rPPG signal and respiratory signal to generate a first discrete-time signal sequence and a second discrete-time signal sequence with a constant sampling rate and precise alignment in the time dimension may specifically include the following steps: Presetting a target synchronization frequency F targetBased on the start and end timestamps of the signal segments to be processed, a standard, uniformly distributed target time vector is generated. This target time vector constitutes a unified time reference for the synchronization of all signals, and adjacent elements in the vector have a fixed time interval. A continuous interpolation function f(t) is constructed using the timestamp sequence of each rPPG signal and each respiratory signal as the independent variable and the amplitude sequence as the dependent variable. The type of interpolation function is selected according to the signal characteristics and processing accuracy requirements, and it can estimate the signal amplitude at any time t based on discrete original data points. The interpolation function f(t) is evaluated at each time point of the target time vector to calculate the resampled signal amplitudes of the rPPG signal and respiratory signal. All calculated signal amplitudes are arranged in order to obtain a new, resampled first discrete-time signal sequence and a second discrete-time signal sequence.

[0016] Secondly, embodiments of this application provide a dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection system, which includes: an image acquisition module, an image enhancement module, a signal extraction module, a signal processing module, and a signal output module.

[0017] Specifically, the image acquisition module is used to acquire visible light images and thermal infrared images through a constructed dual-spectrum integrated acquisition module, and perform image processing to obtain visible light image sequences and thermal infrared image sequences; the image enhancement module is used to process each frame of the visible light image sequence and thermal infrared image sequence based on a preset image super-resolution enhancement module to obtain enhanced visible light target sequences and thermal infrared image target sequences; the signal extraction module is used to extract rPPG signals and respiratory signals from the visible light target sequence and thermal infrared image target sequence, respectively; the signal processing module is used to process the extracted rPPG signals and respiratory signals to generate a first discrete-time signal sequence and a second discrete-time signal sequence with a constant sampling rate and precise alignment in the time dimension; the signal output module is used to synchronously output the first discrete-time signal sequence and the second discrete-time signal sequence, which correspond to the rPPG signal and the respiratory signal, respectively.

[0018] Thirdly, embodiments of this application provide an electronic device, which includes: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection method described in the first aspect.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program or instructions that, when executed by a processor, implement the dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection method described in the first aspect.

[0020] This application provides a non-contact method for simultaneous detection of multiple physiological parameters using dual-spectral super-resolution enhancement. Compared with existing technologies, it has the following advantages:

[0021] This application constructs a dual-spectrum integrated acquisition module to simultaneously acquire visible light and thermal infrared images. After processing to obtain two sets of image sequences, due to the low resolution of images acquired under conditions of strong light and motion interference, an image super-resolution enhancement module is used to process each frame of the image sequence to improve the image resolution, resulting in visible light target sequences and thermal infrared image target sequences. rPPG signals and respiratory signals are then extracted. The rPPG signal is a remote pulse wave signal that can characterize heart rate data. Based on the rPPG signal and respiratory signal, this application also generates two sets of signal sequences with a constant sampling rate and precise alignment in the time dimension, thereby accurately assessing an individual's physiological state based on multiple physiological parameters. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a dual-spectral super-resolution enhanced non-contact method for simultaneous detection of multiple physiological parameters provided in an embodiment of this application.

[0024] Figure 2 yes Figure 1 An exemplary process diagram of S120;

[0025] Figure 3 This is an exemplary structural diagram of a Fast Super-Resolution Convolutional Neural Network (FSRCNN) provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of the structure of a dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection system provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0030] This application provides a dual-spectral super-resolution enhanced non-contact method for simultaneous detection of multiple physiological parameters, solving the current problem of difficulty in performing non-contact simultaneous detection of multiple physiological parameters such as heart rate and respiratory rate under conditions of strong light and motion interference.

[0031] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0032] The following section first introduces a dual-spectral super-resolution enhanced non-contact method for simultaneous detection of multiple physiological parameters provided in the embodiments of this application.

[0033] This application provides a flowchart illustrating a dual-spectral super-resolution enhanced non-contact method for simultaneous detection of multiple physiological parameters, as shown in the embodiments below. Figure 1 As shown, the dual-spectral super-resolution enhanced non-contact simultaneous detection method for multiple physiological parameters may include the following steps S110-S150.

[0034] S110. Visible light images and thermal infrared images are acquired through the constructed dual-spectrum integrated acquisition module, and image processing is performed to obtain visible light image sequences and thermal infrared image sequences.

[0035] S120. Based on the preset image super-resolution enhancement module, each frame of the visible light image sequence and the thermal infrared image sequence is processed to obtain the enhanced visible light target sequence and the thermal infrared image target sequence.

[0036] S130. rPPG signal and respiratory signal are extracted from visible light target sequence and thermal infrared image target sequence, respectively.

[0037] S140. The extracted rPPG signal and respiratory signal are processed to generate a first discrete-time signal sequence and a second discrete-time signal sequence with a constant sampling rate and precise alignment in the time dimension.

[0038] S150, synchronously outputs a first discrete-time signal sequence and a second discrete-time signal sequence, the first discrete-time signal sequence and the second discrete-time signal sequence corresponding to the rPPG signal and the respiratory signal, respectively.

[0039] The above describes the specific implementation of the dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection method provided in this application. It is understood that this application constructs a dual-spectral integrated acquisition module to simultaneously acquire visible light and thermal infrared images. After processing to obtain two sets of image sequences, due to the low resolution of images acquired under conditions of strong light and motion interference, a set image super-resolution enhancement module processes each frame of the image sequence to improve image resolution, obtaining visible light target sequences and thermal infrared image target sequences, and extracting rPPG signals and respiratory signals. The rPPG signal is a remote pulse wave signal and can characterize heart rate data. Based on the rPPG signal and respiratory signal, this application also generates two sets of signal sequences with a constant sampling rate and precise alignment in the time dimension, thereby accurately assessing an individual's physiological state based on multiple physiological parameters. This application can perform non-contact synchronous detection of multiple physiological parameters such as heart rate and respiratory rate under conditions of strong light and motion interference. Through image enhancement, signal extraction, and signal data alignment processing, it can accurately reflect an individual's physiological state.

[0040] In some embodiments, the dual-spectrum integrated acquisition module includes a visible light camera and a thermal infrared camera, which are fixed to the same base to ensure that their optical centers are on the same horizontal reference plane.

[0041] The dual-spectrum integrated acquisition module also includes an adjustment mechanism, which controls the thermal infrared camera to rotate horizontally around its own optical center vertical axis to set the deflection angle of the thermal infrared camera according to the preset working distance, thereby optimizing the overlapping field of view of the thermal infrared camera and the visible light camera; wherein, the working distance is the distance between the target face and the dual-spectrum integrated acquisition module.

[0042] In the embodiments of this application, it is understood that the dual-spectrum integrated acquisition module, through the setting of an adjustment mechanism, can precisely set the deflection angle of the thermal infrared camera according to a preset working distance, aiming to achieve the optimal overlap area of ​​the fields of view of the two cameras at that working distance. This mechanical structure design provides a stable and quantifiable physical constraint for subsequent high-precision pixel-level registration through algorithms, thereby transforming independent heterogeneous sensor data sources into a homogeneous multimodal data stream with a unified perspective.

[0043] In some embodiments, the aforementioned dual-spectrum integrated acquisition module acquires visible light images and thermal infrared images, and performs image processing to obtain visible light image sequences and thermal infrared image sequences. Specifically, S110 may include the following steps:

[0044] S210. After fixing the relative positions of the visible light camera and the thermal infrared camera, use the temperature difference calibration plate as the calibration reference.

[0045] S220: Assemble the temperature difference calibration plate based on the preset working distance, so that the temperature difference calibration plate is completely imaged in the overlapping field of view of the thermal infrared camera and the visible light camera.

[0046] S230: Through a synchronous triggering mechanism, visible light images and thermal infrared images are acquired simultaneously. By adjusting the pose of the temperature difference calibration plate, at least 10 sets of image pairs at different positions and angles are obtained.

[0047] S240. For all acquired visible light images, apply the standard camera calibration algorithm to automatically detect and extract the coordinates of the first corner point of the temperature difference calibration plate; and calculate the first intrinsic parameter matrix K of the visible light camera. rgb With the first distortion coefficient D rgb .

[0048] S250. For all acquired thermal infrared images, image processing techniques adapted to thermal imaging characteristics are used to extract the coordinates of the second corner points in the same group corresponding to the first corner point coordinates, and the second intrinsic parameter matrix K of the thermal infrared camera is calculated. thermal With the second distortion coefficient D thermal .

[0049] In the embodiments of this application, it is understood that the temperature difference calibration plate has the characteristic of exhibiting high contrast features (such as checkerboard corners) in both visible light images and thermal infrared images. The standard camera calibration algorithm can be the findChessboardCorners and calibrateCamera functions based on the OpenCV library; the aforementioned image processing techniques adapted to thermal imaging characteristics can be threshold segmentation, contour analysis, and corner fitting.

[0050] In some embodiments, the coordinates of the first corner point and the coordinates of the second corner point constitute a corner point coordinate set; the first intrinsic parameter matrix K rgb Second intrinsic parameter matrix K thermal The intrinsic parameter matrix constituting the two cameras; the first distortion coefficient D rgb With the second distortion coefficient D thermal The distortion coefficients that make up the two cameras.

[0051] The aforementioned dual-spectrum integrated acquisition module acquires visible light and thermal infrared images, and performs image processing to obtain visible light image sequences and thermal infrared image sequences. Specifically, S110 may further include the following steps:

[0052] S310. Based on the corresponding corner coordinate set of each image pair, the intrinsic parameter matrices of the two cameras, and the distortion coefficients of the two cameras, execute the stereo calibration algorithm.

[0053] S320. Through iterative optimization using a stereo calibration algorithm, minimize the reprojection error of all corner point pairs on the imaging planes of the two cameras, and solve for the rotation matrix R and translation vector T that describe the spatial relationship between the thermal infrared camera coordinate system and the visible light camera coordinate system.

[0054] S330. Determine the extrinsic parameter matrix (R,T) based on the rotation matrix R and the translation vector T to form a rigid transformation connecting the thermal infrared camera coordinate system and the visible light camera coordinate system.

[0055] S340, using the calibrated first distortion coefficient D rgb Second distortion coefficient D thermal Distortion correction is performed on visible light images and thermal infrared images respectively; the multiple visible light images after distortion correction constitute a visible light image sequence.

[0056] S350: Based on the obtained intrinsic and extrinsic parameter matrices (R,T) of the two cameras, a perspective transformation is applied to the distortion-free thermal infrared image, and the pixels of the thermal infrared image are reprojected onto the imaging plane of the visible light camera to generate a new thermal infrared image.

[0057] Among them, the new thermal infrared image is perfectly aligned with the corresponding visible light image at a preset working distance by eliminating the parallax caused by the physical separation of the camera; multiple new thermal infrared images constitute a thermal infrared image sequence.

[0058] In the embodiments of this application, it is understood that the stereo calibration algorithm can be the stereoCalibrate function based on the OpenCV library. By performing perspective transformation processing on the distortion-reduced thermal infrared image to eliminate parallax, this application completes the dynamic spatial registration of two modal data.

[0059] In some embodiments, the image super-resolution enhancement module includes a Fast Super-Resolution Convolutional Neural Network (FSRCNN); the Fast Super-Resolution Convolutional Neural Network (FSRCNN) includes: a feature extraction layer, a shrinking layer, a nonlinear mapping layer, and an expansion layer.

[0060] Please refer to the above as well. Figure 2 and Figure 3 The aforementioned image super-resolution enhancement module, based on a preset method, processes each frame of the visible light image sequence and the thermal infrared image sequence to obtain the enhanced visible light target sequence and the thermal infrared image target sequence. Specifically, the aforementioned S120 may include the following steps:

[0061] S410. Input each frame of the visible light image sequence and the thermal infrared image sequence into a 5x5 two-dimensional convolutional feature extraction layer, and extract shallow features of the input low-resolution image through the feature extraction layer.

[0062] S420: The feature dimension is compressed by a shrinking layer of 1x1 two-dimensional convolution to reduce the computational complexity of subsequent operations.

[0063] S430. In the reduced feature space, a nonlinear mapping layer consisting of multiple stacked 3x3 two-dimensional convolutions is used to learn the complex mapping relationship from low-resolution to high-resolution features.

[0064] S440 recovers the feature dimension through a 1x1 two-dimensional convolutional extension layer, and finally reconstructs a high-resolution image by performing an upsampling operation through a transposed convolutional layer.

[0065] Among them, multiple high-resolution images corresponding to the visible light image sequence constitute the visible light target sequence, and multiple high-resolution images corresponding to the thermal infrared image sequence constitute the thermal infrared image target sequence.

[0066] In the embodiments of this application, it is understood that subtle color changes in skin in visible light image sequences and changes in the nasal alar region in thermal infrared image sequences are highly susceptible to the influence of their inherent quality parameters. Especially at low frame rates and low resolutions, the amplitude of color changes is severely suppressed, and the inflection point of color changes is significantly shifted, making it difficult to detect periodic color change patterns and achieve accurate physiological information perception. To improve the accuracy and robustness of physiological signal extraction, especially under long-distance or low-resolution acquisition conditions, this method further introduces an image super-resolution enhancement module.

[0067] Through this enhancement process, this application can convert low-resolution visible light image sequences and thermal infrared image sequences into video frames with significantly improved clarity. This provides a high-quality data foundation for the stable division of the infrared nasal ROI region and the effective extraction of color changes from rPPG, thereby enhancing the performance and applicability of the entire non-contact physiological parameter detection method.

[0068] In some embodiments, the rPPG signal and respiratory signal are extracted from the visible light target sequence and the thermal infrared image target sequence, respectively. That is, the aforementioned S130 may specifically include the following steps:

[0069] S510. Select local images of the corresponding facial region from the visible light target sequence, and perform scaling and normalization processing to obtain a standard visible light sequence.

[0070] S520. Input the standard visible light sequence into the pre-trained rPPG signal extraction network to extract the rPPG signal.

[0071] In this embodiment, it should be noted that during the extraction of rPPG signals, a self-supervised rPPG signal extraction method based on contrastive learning can be combined with face reconstruction and illumination control techniques based on Neural Radiance Fields (NeRF) for positive sample enhancement, thereby improving the model's ability to extract rPPG signals under unlabeled conditions, including head movement and changes in facial illumination. Specifically, firstly, facial regions are extracted from an input video. The face sequence is reconstructed using face reconstruction and illumination control techniques based on Neural Radiance Fields, and synthetic image sequences of this sequence under different head poses and illumination conditions are generated, constructing positive sample pairs with similar rPPG signals. Then, these sample pairs are input into a contrastive learning framework. The contrastive loss function is used to narrow the representation distance between positive samples and distinguish them from negative samples in other video segments. The model is trained to learn rPPG embedding feature representations robust to illumination and head movement. After training the rPPG signal extraction network, signal extraction is performed on standard visible light sequences.

[0072] For example, the specific implementation process of extracting the rPPG signal may include the following steps:

[0073] This method utilizes facial landmark localization and face detection algorithms to detect faces and locate regions of interest in video sequences. It employs a facial keypoint neural network to perform face detection on each frame of the image, detecting face bounding boxes and outputting the precise coordinates of facial key points (such as eyes, nose tip, and corners of the mouth).

[0074] By integrating multi-view and multi-illumination positive sample generation modules, high-quality extraction of rPPG signals under label-free conditions is achieved. Specifically, a 3D face reconstruction method based on neural radiation fields is used to control illumination and perturb pose in the input image to construct multiple sets of positive sample pairs with semantic consistency. Contrastive learning is introduced for self-supervised training to enhance the network's robustness to changes in facial illumination and head movement.

[0075] After generating data on the same sample with different head poses and lighting conditions, a contrastive learning-based cross-entropy loss was used to achieve self-supervised training. Multiple variant images of the same face were generated under different lighting conditions and head poses, serving as positive sample pairs. Simultaneously, samples were extracted from videos of different individuals or time periods as negative sample pairs to ensure the differences in physiological feature expression between samples. Then, a contrastive learning framework was trained based on these samples to obtain an rPPG signal extraction network for rPPG signal extraction.

[0076] S530. The thermal infrared image target sequence is processed frame by frame using the thermal infrared nasal ROI detection method to obtain a mixed respiratory signal sequence containing valid respiratory signals and missing regions; wherein, the thermal infrared nasal ROI detection method is based on single-frame missing determination.

[0077] S540. Perform signal completion and reconstruction on the mixed respiratory signal sequence to obtain the reconstructed respiratory signal sequence for extracting respiratory signals.

[0078] In this embodiment, it should also be noted that during the specific implementation of respiratory signal extraction, the input thermal infrared image target sequence is first processed frame by frame, and the target detection algorithm based on the YOLOv8 deep neural network model is used to accurately locate the nasal Region of Interest (ROI). For image frames where an ROI is detected, the average pixel intensity within that region is calculated and used as the respiratory signal value corresponding to the current frame; while for image frames where no ROI is detected, it is recorded and marked to explicitly indicate its unusable state. By traversing the entire image sequence, a mixed respiratory signal sequence containing valid respiratory signals and missing segments is finally constructed.

[0079] For this incomplete mixed respiratory signal sequence, a two-stage signal denoising and completion reconstruction algorithm can be designed to recover continuous, high-quality respiratory waveforms. The first stage designs a signal completion method based on the dominant frequency reconstruction, specifically filling and completing signals with different missing components to achieve initial signal reconstruction. The second stage designs a Breath-GAN deep denoising and reconstruction model, further denoising the signal while simultaneously completing more accurate filling and reconstruction of missing parts under time-frequency domain fusion. Finally, a high-quality reconstructed respiratory signal curve with denoising is output, and respiratory signals, including key respiratory characteristic parameters such as respiratory rate and respiratory interval, are extracted from this curve, providing accurate and reliable data support for health analysis and physiological monitoring tasks.

[0080] Understandably, this work employs a deep learning-based object detection framework to address the challenge of detecting small targets in the nasal region of thermal infrared images. YOLOv8, by introducing an anchor-free mechanism and a multi-scale feature fusion strategy, effectively handles the detection of small targets like those in the nasal region. Furthermore, its independent detection mechanism based on a single frame avoids the dependence on temporal correlation found in traditional multi-frame tracking algorithms, thus significantly reducing the risk of detection failure due to the accumulation of errors between frames when the subject experiences sudden posture changes (such as head turning or shaking) or complex environmental interference. This high robustness and low latency characteristic enables it to provide reliable support for nasal region detection in thermal infrared videos, thereby laying a data foundation for subsequent respiratory signal extraction. Therefore, this work selects this method as the nasal ROI detection method.

[0081] For the obtained mixed respiratory signal sequence, the first stage of preliminary completion and reconstruction is performed. This method aims to solve the problem of local missing respiratory signals caused by target occlusion, loss of ROI in thermal infrared images, or sensor abnormalities. This method can dynamically select different completion strategies based on the time length of the missing interval, thereby achieving reasonable restoration of the signal structure and smooth boundary connection, exhibiting strong adaptability and real-time performance. The core idea of ​​the technical solution is to determine the length of the missing segment based on a preset time threshold, and then use interpolation or frequency domain analysis methods to perform differentiated completion of the missing interval. First, the acquired mixed respiratory signal sequence is detrended and normalized. Then, by identifying missing values, the location of the missing signal is determined. The entire mixed respiratory signal sequence is traversed, and all missing intervals are identified by marking the start and end points of consecutive missing points. Different completion strategies are applied based on the set time threshold. The first-stage signal completion method can restore the spatiotemporal continuity of the respiratory signal. However, due to the susceptibility of thermal infrared imaging to environmental thermal radiation interference, motion artifacts, and the limitations of the preliminary reconstruction algorithm, the completed signal still suffers from residual noise complexity and distortion of physiological characteristics. Therefore, this application performs noise reduction on the complete respiratory signal in the second stage while reconstructing the signal to a certain extent to achieve higher signal quality. Breath-GAN performs physiological consistency correction while reducing noise through generative adversarial learning, which can simultaneously repair signal problems such as amplitude drift, phase misalignment and waveform distortion, and finally output a high-fidelity thermal infrared respiratory signal.

[0082] In designing the Breath-GAN deep denoising and reconstruction model, the generator can use a bidirectional LSTM to capture the long-term temporal dependence of the respiratory signal, and combine deconvolutional layers and skip connections to preserve waveform details; the discriminator can mine local morphological features based on one-dimensional convolution. A multi-constraint joint loss function is introduced during model training, and through related designs, the signal is effectively denoised and further denoised and reconstructed from the completed segments.

[0083] In some embodiments, each rPPG signal and each respiratory signal consists of timestamp-amplitude data pairs, and multiple rPPG signals and multiple respiratory signals are signal segments to be processed.

[0084] The aforementioned processing of the extracted rPPG signal and respiratory signal generates a first discrete-time signal sequence and a second discrete-time signal sequence with a constant sampling rate and precise alignment in the time dimension. Specifically, S140 may include the following steps:

[0085] S610, Preset a target synchronization frequency F targetBased on the start and end timestamps of the signal segment to be processed, a standard, uniformly distributed target time vector is generated. The target time vector constitutes a unified time reference on which all signals are synchronized, and there is a fixed time interval between adjacent elements in the vector.

[0086] S620. Using the timestamp sequence of each rPPG signal and each respiratory signal as the independent variable and the amplitude sequence as the dependent variable, construct a continuous interpolation function f(t); the type of interpolation function is selected according to the signal characteristics and processing accuracy requirements, and it can estimate the signal amplitude at any time t based on discrete raw data points.

[0087] S630. Calculate the resampled signal amplitude of the rPPG signal and the respiratory signal by evaluating the interpolation function f(t) at each time point of the target time vector.

[0088] S640. Arrange all the calculated signal amplitudes in order to obtain a new, resampled first discrete-time signal sequence and a second discrete-time signal sequence.

[0089] In this embodiment, the target synchronization frequency Ftarget can be 30Hz, and the type of interpolation function can be linear interpolation or cubic spline interpolation. By arranging all the calculated amplitudes in order, a new, resampled discrete-time signal sequence can be obtained, ensuring the synchronization and consistency of the output first discrete-time signal sequence and the second discrete-time signal sequence in the time dimension, so as to synchronously characterize the heart rate characteristics and respiratory characteristics.

[0090] In some embodiments, this application provides a dual-spectral super-resolution enhanced non-contact multi-physiological parameter simultaneous detection system 700, such as... Figure 4 As shown, the dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection system 700 may include the following modules:

[0091] The image acquisition module 710 is used to acquire visible light images and thermal infrared images through the constructed dual-spectrum integrated acquisition module, and to perform image processing to obtain visible light image sequences and thermal infrared image sequences.

[0092] The image enhancement module 720 is used to process each frame of the visible light image sequence and the thermal infrared image sequence based on a preset image super-resolution enhancement module to obtain the enhanced visible light target sequence and the thermal infrared image target sequence.

[0093] The signal extraction module 730 is used to extract rPPG signals and respiratory signals from visible light target sequences and thermal infrared image target sequences, respectively.

[0094] The signal processing module 740 is used to process the extracted rPPG signal and respiratory signal to generate a first discrete-time signal sequence and a second discrete-time signal sequence with a constant sampling rate and precise alignment in the time dimension.

[0095] The signal output module 750 is used to synchronously output a first discrete-time signal sequence and a second discrete-time signal sequence, which correspond to the rPPG signal and the respiratory signal, respectively.

[0096] According to embodiments of this application, any multiple modules among the image acquisition module 710, image enhancement module 720, signal extraction module 730, signal processing module 740, and signal output module 750 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module.

[0097] Figure 4 Each module in the system shown has the function of realizing each step in the aforementioned dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection method, and can achieve its corresponding technical effect. For the sake of brevity, it will not be elaborated here.

[0098] In some embodiments, this application provides an electronic device, the structural schematic of which is shown below. Figure 5 As shown.

[0099] The electronic device may include a processor 810 and a memory 820 storing computer program instructions.

[0100] Specifically, the processor 810 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0101] Memory 820 may include mass storage for data or instructions. For example, and not limitingly, memory 820 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 820 may include removable or non-removable (or fixed) media. Where appropriate, memory 820 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 820 is non-volatile solid-state memory.

[0102] The memory 820 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, the memory 820 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any of the dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection methods in the above embodiments.

[0103] The processor 810 reads and executes computer program instructions stored in the memory 820 to implement any of the dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection methods in the above embodiments.

[0104] In one example, the electronic device may also include a communication interface 830 and a bus 800. For example, Figure 5 As shown, the processor 810, memory 820, and communication interface 830 are connected via bus 800 and communicate with each other.

[0105] The communication interface 830 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0106] Bus 800 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 800 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0107] Furthermore, in conjunction with the dual-spectral super-resolution enhanced non-contact simultaneous detection method for multiple physiological parameters described in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any one of the dual-spectral super-resolution enhanced non-contact simultaneous detection methods for multiple physiological parameters described in the above embodiments.

[0108] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0109] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0110] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0111] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0112] In summary, compared with the prior art, this application has the following beneficial effects:

[0113] 1. This application constructs a dual-spectrum integrated acquisition module to simultaneously acquire visible light images and thermal infrared images. After processing to obtain two sets of image sequences, the parallax is eliminated by performing perspective transformation on the distortion-free thermal infrared images. This application completes the dynamic spatial registration of the two modal data.

[0114] 2. Due to the low resolution of images acquired under conditions of strong light and motion interference, this application utilizes an image super-resolution enhancement module to convert low-resolution visible light and thermal infrared image sequences into video frames with significantly improved clarity. This provides a high-quality data foundation for the stable division of the infrared nasal ROI region and the effective extraction of rPPG color changes, thereby enhancing the performance and applicability of the entire non-contact physiological parameter detection method.

[0115] 3. This application calculates the signal amplitude of the extracted rPPG signal and respiratory signal through an interpolation function, and arranges all the calculated amplitudes in order to obtain a resampled discrete-time signal sequence, ensuring the synchronization and consistency of the output first discrete-time signal sequence and the second discrete-time signal sequence in the time dimension, so as to synchronously represent the heart rate characteristics and respiratory characteristics.

[0116] 4. This application enables non-contact synchronous detection of multiple physiological parameters such as heart rate and respiratory rate under conditions of strong light and motion interference. Through image enhancement, signal extraction, and signal data alignment processing, two sets of signal sequences with constant sampling rate and precise alignment in the time dimension are generated, thereby accurately assessing an individual's physiological state based on multiple physiological parameters. This solves the current problem of difficulty in non-contact synchronous detection of multiple physiological parameters such as heart rate and respiratory rate under conditions of strong light and motion interference.

[0117] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A non-contact method for simultaneous detection of multiple physiological parameters using dual-spectral super-resolution enhancement, characterized in that, include: Visible light images and thermal infrared images are acquired by a constructed dual-spectrum integrated acquisition module, and image processing is performed to obtain visible light image sequences and thermal infrared image sequences. Based on a preset image super-resolution enhancement module, each frame of the visible light image sequence and the thermal infrared image sequence is processed to obtain the enhanced visible light target sequence and the thermal infrared image target sequence. rPPG signal and respiratory signal were extracted from the visible light target sequence and the thermal infrared image target sequence, respectively. The extracted rPPG signal and respiratory signal are processed to generate a first discrete-time signal sequence and a second discrete-time signal sequence with a constant sampling rate and precise alignment in the time dimension. The first discrete-time signal sequence and the second discrete-time signal sequence are output synchronously, and the first discrete-time signal sequence and the second discrete-time signal sequence correspond to the rPPG signal and the respiratory signal, respectively. The image super-resolution enhancement module includes a Fast Super-Resolution Convolutional Neural Network (FSRCNN); The Fast Super-Resolution Convolutional Neural Network (FSRCNN) includes: a feature extraction layer, a shrinking layer, a nonlinear mapping layer, and an expansion layer; The preset image super-resolution enhancement module processes each frame of the visible light image sequence and the thermal infrared image sequence to obtain enhanced visible light target sequences and thermal infrared target sequences, including: Each frame of the visible light image sequence and the thermal infrared image sequence is input into the feature extraction layer of a 5x5 two-dimensional convolution, and shallow features of the input low-resolution image are extracted through the feature extraction layer. The shrinking layer, consisting of 1x1 two-dimensional convolutions, compresses the feature dimension to reduce subsequent computational complexity. In the reduced feature space, the nonlinear mapping layer, which consists of multiple stacked 3x3 two-dimensional convolutions, is used to learn the complex mapping relationship from low-resolution to high-resolution features. The feature dimensions are recovered by the extended layer of 1x1 two-dimensional convolution, and finally an upsampling operation is performed by a transposed convolutional layer to reconstruct a high-resolution image. Among them, multiple high-resolution images corresponding to the visible light image sequence constitute a visible light target sequence, and multiple high-resolution images corresponding to the thermal infrared image sequence constitute a thermal infrared image target sequence.

2. The non-contact simultaneous detection method for multiple physiological parameters with dual-spectral super-resolution enhancement as described in claim 1, characterized in that, The dual-spectrum integrated acquisition module includes a visible light camera and a thermal infrared camera. The visible light camera and the thermal infrared camera are fixed on the same base to ensure that the optical centers of the two are on the same horizontal reference plane. The dual-spectrum integrated acquisition module also includes an adjustment mechanism, which controls the thermal infrared camera to rotate horizontally around its own optical center vertical axis to set the deflection angle of the thermal infrared camera according to a preset working distance, thereby optimizing the overlapping field of view of the thermal infrared camera and the visible light camera; wherein, the working distance is the distance between the target face and the dual-spectrum integrated acquisition module.

3. The non-contact simultaneous detection method for multiple physiological parameters with dual-spectral super-resolution enhancement as described in claim 2, characterized in that, The method involves acquiring visible light and thermal infrared images using a constructed dual-spectrum integrated acquisition module, and performing image processing to obtain visible light image sequences and thermal infrared image sequences, including: After fixing the relative positions of the visible light camera and the thermal infrared camera, a temperature difference calibration plate was used as a calibration reference. The temperature difference calibration plate is assembled based on a preset working distance, so that the temperature difference calibration plate is completely imaged within the overlapping field of view of the thermal infrared camera and the visible light camera; By using a synchronous triggering mechanism, visible light images and thermal infrared images are acquired simultaneously. By adjusting the pose of the temperature difference calibration plate, at least 10 sets of image pairs at different positions and angles are obtained. For all acquired visible light images, a standard camera calibration algorithm is applied to automatically detect and extract the coordinates of the first corner point of the temperature difference calibration plate; and the first intrinsic parameter matrix K of the visible light camera is calculated. rgb With the first distortion coefficient D rgb ; For all acquired thermal infrared images, image processing techniques adapted to thermal imaging characteristics are used to extract the coordinates of the second corner points in the same group corresponding to the coordinates of the first corner point, and the second intrinsic parameter matrix K of the thermal infrared camera is calculated. thermal With the second distortion coefficient D thermal .

4. The non-contact simultaneous detection method for multiple physiological parameters with dual-spectral super-resolution enhancement as described in claim 3, characterized in that, The coordinates of the first corner point and the coordinates of the second corner point constitute the corner point coordinate set; the first intrinsic parameter matrix K rgb and the second intrinsic parameter matrix K thermal The intrinsic parameter matrix constituting the two cameras; the first distortion coefficient D rgb With the second distortion coefficient D thermal The distortion coefficients that make up the two cameras; The method of acquiring visible light and thermal infrared images through a constructed dual-spectrum integrated acquisition module, and performing image processing to obtain visible light image sequences and thermal infrared image sequences, further includes: Based on the set of corresponding corner coordinates in each image pair, the intrinsic parameter matrices of the two cameras, and the distortion coefficients of the two cameras, a stereo calibration algorithm is executed. The stereo calibration algorithm is iteratively optimized to minimize the reprojection error of all corner point pairs on the imaging planes of the two cameras, and the rotation matrix R and translation vector T describing the spatial relationship between the thermal infrared camera coordinate system and the visible light camera coordinate system are solved. The extrinsic parameter matrix (R, T) is determined based on the rotation matrix R and the translation vector T to form a rigid transformation connecting the thermal infrared camera coordinate system and the visible light camera coordinate system; Using the calibrated first distortion coefficient D rgb Second distortion coefficient D thermal Distortion correction is performed on visible light images and thermal infrared images respectively; the multiple visible light images after distortion correction constitute a visible light image sequence. Based on the obtained intrinsic parameter matrices of the two cameras and the extrinsic parameter matrix (R, T), a perspective transformation is applied to the distortion-free thermal infrared image to reproject the pixels of the thermal infrared image onto the imaging plane of the visible light camera, generating a new thermal infrared image. Among them, the new thermal infrared image is perfectly aligned with the corresponding visible light image at a preset working distance by eliminating the parallax caused by the physical separation of the camera; multiple new thermal infrared images constitute a thermal infrared image sequence.

5. The non-contact simultaneous detection method for multiple physiological parameters with dual-spectral super-resolution enhancement as described in any one of claims 1-4, characterized in that, The extraction of rPPG signals and respiratory signals from the visible light target sequence and the thermal infrared image target sequence, respectively, includes: Local images of the corresponding facial regions are selected from the visible light target sequence and scaled and normalized to obtain a standard visible light sequence; The standard visible light sequence is input into a pre-trained rPPG signal extraction network to extract the rPPG signal; The thermal infrared image target sequence is processed frame by frame using a thermal infrared ROI detection method to obtain a mixed respiratory signal sequence containing valid respiratory signals and missing regions; wherein, the thermal infrared ROI detection method is based on single-frame missing region determination. The mixed respiratory signal sequence is completed and reconstructed to obtain a reconstructed respiratory signal sequence for extracting respiratory signals.

6. The non-contact simultaneous detection method for multiple physiological parameters with dual-spectral super-resolution enhancement as described in any one of claims 1-4, characterized in that, Each of the rPPG signals and each of the respiratory signals consists of timestamp-amplitude data pairs, and multiple rPPG signals and multiple respiratory signals are signal segments to be processed; The process of processing the extracted rPPG signal and respiratory signal to generate a first discrete-time signal sequence and a second discrete-time signal sequence with a constant sampling rate and precise alignment in the time dimension includes: Preset a target synchronization frequency F target Based on the start and end timestamps of the signal segment to be processed, a standard, uniformly distributed target time vector is generated; the target time vector constitutes a unified time reference on which all signals are synchronized, and there is a fixed time interval between adjacent elements in the vector. Using the timestamp sequence of each rPPG signal and each respiratory signal as the independent variable and the amplitude sequence as the dependent variable, a continuous interpolation function f(t) is constructed; the type of the interpolation function is selected according to the signal characteristics and processing accuracy requirements, and it can estimate the signal amplitude at any time t based on discrete raw data points; The interpolation function f(t) is used to evaluate the target time vector at each time point to calculate the resampled signal amplitude of the rPPG signal and the respiratory signal. Arrange all the calculated signal amplitudes in order to obtain a new, resampled first discrete-time signal sequence and a second discrete-time signal sequence.

7. A dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection system, characterized in that, include: The image acquisition module is used to acquire visible light images and thermal infrared images through the constructed dual-spectrum integrated acquisition module, and to perform image processing to obtain visible light image sequences and thermal infrared image sequences; An image enhancement module is used to process each frame of the visible light image sequence and the thermal infrared image sequence based on a preset image super-resolution enhancement module to obtain an enhanced visible light target sequence and a thermal infrared image target sequence. The signal extraction module is used to extract rPPG signals and respiratory signals from the visible light target sequence and the thermal infrared image target sequence, respectively. The signal processing module is used to process the extracted rPPG signal and respiratory signal to generate a first discrete-time signal sequence and a second discrete-time signal sequence with a constant sampling rate and precise alignment in the time dimension. The signal output module is used to synchronously output the first discrete-time signal sequence and the second discrete-time signal sequence, wherein the first discrete-time signal sequence and the second discrete-time signal sequence correspond to the rPPG signal and the respiratory signal, respectively. The image super-resolution enhancement module includes a Fast Super-Resolution Convolutional Neural Network (FSRCNN); The Fast Super-Resolution Convolutional Neural Network (FSRCNN) includes: a feature extraction layer, a shrinking layer, a nonlinear mapping layer, and an expansion layer; The preset image super-resolution enhancement module processes each frame of the visible light image sequence and the thermal infrared image sequence to obtain enhanced visible light target sequences and thermal infrared target sequences, including: Each frame of the visible light image sequence and the thermal infrared image sequence is input into the feature extraction layer of a 5x5 two-dimensional convolution, and shallow features of the input low-resolution image are extracted through the feature extraction layer. The shrinking layer, consisting of 1x1 two-dimensional convolutions, compresses the feature dimension to reduce subsequent computational complexity. In the reduced feature space, the nonlinear mapping layer, which consists of multiple stacked 3x3 two-dimensional convolutions, is used to learn the complex mapping relationship from low-resolution to high-resolution features. The feature dimensions are recovered by the extended layer of 1x1 two-dimensional convolution, and finally an upsampling operation is performed by a transposed convolutional layer to reconstruct a high-resolution image. Among them, multiple high-resolution images corresponding to the visible light image sequence constitute a visible light target sequence, and multiple high-resolution images corresponding to the thermal infrared image sequence constitute a thermal infrared image target sequence.

8. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the dual-spectral super-resolution enhanced non-contact simultaneous detection method for multiple physiological parameters as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that, when executed by a processor, implement the dual-spectral super-resolution enhanced non-contact multi-physiological parameter synchronous detection method as described in any one of claims 1 to 6.

Images