Multi-camera cooperative long-term stable remote physiological signal detection method and system

Abstract

The application discloses a long-term stable remote physiological signal detection method and system based on multi-camera cooperation, which comprises the following steps: collecting video data of a target object from different shooting angles by a plurality of cameras; extracting candidate physiological signals and local features; regarding each camera as a graph node, establishing a connection edge between nodes with shooting angle correlation, assigning a corresponding weight to the connection edge according to the shooting angle matching degree index between nodes, and forming a connection topology graph; each graph node combines its own node features and adjacent node features to interact and compensate cross-angle information, and the fusion features corresponding to each graph node are obtained through multiple iterations; the fusion features are aggregated and the central stable signal is dynamically calculated; and the central stable signal is mapped to a physiological signal to solve the physiological feature parameters. The application realizes the fusion and information interaction compensation of multi-angle physiological signal estimation, and significantly improves the robustness, stability and environmental adaptability of remote physiological signal detection.

Description

Technical Field

[0001] This application relates to the field of remote physiological signal detection technology, and in particular to a long-term stable remote physiological signal detection method and system using multiple cameras in collaboration. Background Technology

[0002] Remote physiological signal detection is a technology that uses cameras to acquire human vital signs without contact. Its core principle is to analyze subtle changes in the intensity of reflected light from skin areas in a video sequence, extracting photovolume pulse signals related to heartbeats, and then estimating key physiological parameters such as heart rate and respiratory rate. Due to its non-invasiveness, low cost, and ease of deployment, this technology has gained widespread attention in recent years in areas such as health monitoring, sleep quality assessment, hospital care, and driving safety.

[0003] Existing methods for long-term stable remote physiological signal detection using multi-camera collaboration are mainly based on single-camera video input. Early studies often employed signal processing algorithms, such as color channel mapping, independent component analysis, and principal component analysis, to separate periodic components related to physiological activities. With the development of deep learning technology, researchers have further proposed using deep models such as convolutional neural networks, temporal convolutional networks, and long short-term memory networks to directly regress physiological signals or corresponding parameters from video frames.

[0004] However, due to limitations imposed by viewing angle and lighting conditions, single-camera systems experience rapid degradation of the acquired skin reflection signals when the subject changes posture, turns its head, experiences partial occlusion, or suffers from uneven lighting. This leads to distortion or even interruption of physiological signals. Because these systems lack mechanisms to acquire compensation information from other angles, they cannot achieve signal redundancy or alignment at the spatial level, resulting in poor continuity of the output signal. While introducing multiple cameras can theoretically improve system robustness by acquiring skin signals simultaneously from different angles, differences in lighting direction, reflection angle, and regional brightness across different viewing angles lead to significantly different physiological signal intensity distributions and noise characteristics in the output signal. When a signal from a particular viewpoint is affected by local overexposure, shadows, or motion blur, its contribution is difficult to automatically adjust, making direct comparability of multi-view signals difficult and hindering effective fusion through simple averaging or weighting strategies. Therefore, existing remote physiological signal detection methods still face challenges in stability and generalization in practical applications. Summary of the Invention

[0005] Therefore, it is necessary to provide a method and system for long-term stable remote physiological signal detection using multi-camera collaboration to address the above-mentioned technical problems and solve at least one of the problems existing in the prior art.

[0006] Firstly, a long-term stable remote physiological signal detection method using multi-camera collaboration is provided, including: Video data of the target object is collected from different shooting angles using several cameras; Candidate physiological signals and local features are extracted from various video data. Each camera is treated as a graph node. Connection edges are established between nodes that are related by shooting angle. The connection edges are assigned corresponding weights based on the angle matching index between nodes to form a connection topology graph. Based on the connection topology graph, each graph node performs cross-view information interaction compensation by combining its own node features and the features of adjacent nodes. After multiple rounds of iteration, the fusion features corresponding to each graph node are obtained. The node features include the candidate physiological signals, local features, and camera imaging attribute information. Feature aggregation is performed on each fused feature, and the central stable signal is dynamically calculated based on the aggregated features; The central stabilization signal is mapped to a physiological signal so that physiological characteristic parameters can be calculated from the physiological signal.

[0007] In one possible implementation, the extraction of candidate physiological signals and local features from each video data includes: Extract human skin regions from various video data; The human skin region is divided into several sub-regions; The color values ​​of corresponding pixels in the sub-regions are extracted sequentially according to a preset order to obtain a temporal color signal; The time-series color signal is subjected to illumination normalization processing; The signal after illumination normalization is subjected to temporal detrending and standardization to obtain the candidate physiological signal and local features.

[0008] In one possible implementation, each graph node combines its own node features with those of adjacent nodes to perform cross-view information interaction compensation, and obtains the fused features corresponding to each graph node through multiple iterations, including: In each iteration, the features of all neighboring nodes of each graph node are weighted and aggregated to obtain the neighborhood aggregated features; The neighborhood aggregation features are fused with the individual graph node features to obtain new graph node features; The next iteration is performed using the new graph node features until the preset convergence condition is met, thus obtaining the fused features corresponding to each graph node.

[0009] In one possible implementation, the weighted aggregation of features from all neighboring nodes of each graph node to obtain neighborhood aggregation features includes: The quality of each adjacent node is evaluated to obtain the node quality factor; Based on the node quality factor and edge weight, the contribution weight corresponding to each adjacent node is calculated. Based on the contribution weights, the features of the adjacent nodes are weighted and aggregated to obtain the neighborhood aggregated features.

[0010] In one possible implementation, the step of aggregating the fused features and dynamically calculating the center stability signal based on the aggregated features includes: Calculate the node weight based on the node quality factor and consistency score corresponding to each graph node; Based on the node weights, the fusion features are weighted and aggregated to obtain the center candidate signal; The center candidate signal is updated by sliding according to the preset update coefficient and retention coefficient to obtain the center stable signal.

[0011] In one possible implementation, the step of sliding-updating the candidate center signal according to preset update coefficients and retention coefficients to obtain the stable center signal includes: The quality of the central candidate signal is evaluated; Based on the quality assessment results, the update coefficient and retention coefficient are determined; Within the current time window, based on the update coefficient and the retention coefficient, the center candidate signal and the corresponding center stable signal in the previous time window are weighted and summed to obtain the center stable signal corresponding to the current time window.

[0012] In one possible implementation, mapping the central stabilization signal to a physiological signal includes: A linear transformation is performed on the centrally stable signal using a preset weight matrix and bias term, and the complete physiological signal waveform is obtained by splicing the signals according to the sampling time sequence; or After performing nonlinear fitting and dimension mapping on the central stable signal, the physiological signal amplitude is output point by point and then combined in time sequence to form a complete physiological waveform. After amplitude limiting and smoothing filtering of the complete physiological waveform generated by any of the above methods, a standardized physiological signal is finally obtained.

[0013] Secondly, a long-term stable remote physiological signal detection system with multi-camera collaboration is provided, including: A multi-view video data acquisition unit is used to acquire video data of the target object from different shooting angles using several cameras. The feature extraction unit is used to extract candidate physiological signals and local features from each video data. The graph construction unit is used to treat each camera as a graph node, establish connection edges between nodes with shooting perspective correlation, assign corresponding weights to the connection edges according to the perspective matching degree index between nodes, and form a connection topology graph. A cross-view information interaction compensation unit is used to perform cross-view information interaction compensation based on the connection topology graph, whereby each graph node combines its own node features and the features of adjacent nodes, and obtains the fusion features corresponding to each graph node through multiple iterations. The node features include the candidate physiological signals, local features, and camera imaging attribute information. The center stabilization signal generation unit is used to perform feature aggregation on each fused feature and dynamically calculate the center stabilization signal based on the aggregated features. A physiological feature parameter generation unit is used to map the central stable signal into a physiological signal, so as to calculate the physiological feature parameters through the physiological signal.

[0014] Thirdly, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and running on the processor, wherein the processor, when executing the computer-readable instructions, implements the long-term stable remote physiological signal detection method with multi-camera collaboration as described above.

[0015] Fourthly, a readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a processor, implement the long-term stable remote physiological signal detection method of multi-camera collaboration as described above.

[0016] The aforementioned method and system for long-term stable remote physiological signal detection using multi-camera collaboration includes the following steps: acquiring video data corresponding to a target object from different shooting angles using several cameras; extracting candidate physiological signals and local features from each video data; establishing connections between nodes with shooting angle correlations using each camera as a graph node, assigning corresponding weights to the connections based on the inter-node perspective matching index, and forming a connection topology graph; based on the connection topology graph, each graph node performs cross-perspective information interaction compensation by combining its own node features and the features of adjacent nodes, obtaining fusion features corresponding to each graph node through multiple iterations, wherein the node features include the candidate physiological signals, local features, and camera imaging attribute information; aggregating the fusion features and dynamically calculating the central stable signal based on the aggregated features; mapping the central stable signal to a physiological signal to calculate physiological feature parameters. In this embodiment, video data is acquired from different perspectives using multiple cameras, and a graph structure is constructed based on the cameras. Then, a multi-camera feature fusion method based on the graph structure is used to establish dynamic correlations between different perspectives, realizing structured fusion and adaptive compensation for multi-perspective remote physiological signal estimation. When the acquired signal from a single viewpoint becomes abnormal due to changes in lighting, changes in the posture of the object being measured, or occlusion interference, the system can automatically recover the signal by relying on effective features provided by other viewpoints. The maintenance mechanism for a centrally stable signal ensures the smoothness and continuity of the output signal over time, enabling the detection results to maintain high consistency under long-term operating conditions while keeping the signal drift level low. Compared with traditional single-camera detection methods, this scheme significantly improves the robustness, long-term stability, and environmental adaptability of remote physiological signal detection, and can be widely applied in fields such as health monitoring, medical care, and behavior recognition. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of an application environment for a long-term stable remote physiological signal detection method using multiple cameras in collaboration, as described in one embodiment of this application. Figure 2 This is a flowchart illustrating a method for long-term stable remote physiological signal detection using multi-camera collaboration in one embodiment of this application; Figure 3 This is a schematic diagram of a long-term stable remote physiological signal detection system with multi-camera collaboration in one embodiment of this application; Figure 4 This is a schematic diagram of a computer device according to one embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] The multi-camera collaborative long-term stable remote physiological signal detection method provided in this embodiment can be applied to, for example... Figure 1 In the application environment, it can specifically include a multi-camera video acquisition device, a data processing unit, and a storage unit. The multi-camera video acquisition device can consist of several cameras of the same model, such as camera number one, camera number two, ..., camera N, used to acquire video sequences of the object under test from different directions. The data processing unit is used to perform computational tasks such as graph structure modeling, feature fusion, signal compensation, and result output. The storage unit is used to save the acquired video data, feature information, center stabilization signals, and output physiological characteristic parameters.

[0021] Specifically, firstly, video data of the target object is acquired from different perspectives by each camera of a multi-camera video acquisition device. Then, the human body regions in each video data are detected and preprocessed, and corresponding candidate physiological signals and local features are extracted. Simultaneously, each camera is treated as a graph node, and edges are established based on the correlation between nodes and assigned corresponding weights, thereby constructing a multi-camera connection topology graph. Next, multi-view feature fusion and interactive information compensation are performed based on this topology graph. After obtaining the fused features, the center candidate signal is dynamically updated through a sliding time window to obtain a stable center signal. This signal is then mapped to a physiological signal, filtered, and frequency analyzed to finally obtain physiological parameters such as heart rate. This achieves structured fusion and adaptive compensation for multi-view remote physiological signal estimation, significantly improving the robustness, long-term stability, and environmental adaptability of remote physiological signal detection. In one embodiment, such as Figure 2 As shown, a method for long-term stable remote physiological signal detection using multi-camera collaboration is provided, comprising the following steps: In step S110, video data corresponding to the target object is collected from different shooting angles using several cameras; Optionally, several cameras are arranged around the perimeter of the area to be tested, with the optical axis of each camera pointing towards the center of the area. A preset angular difference can be formed between adjacent cameras to achieve synchronous image acquisition from different orientations and perspectives of the target. When the target object is located within the target area, a synchronous trigger signal can be automatically output to control each camera to expose and image simultaneously, continuously acquiring raw video sequences from different perspectives such as the front, left, back, and right sides of the target. A time-division trigger mode can also be used to control each camera to acquire images sequentially, reducing the instantaneous bandwidth pressure on data transmission. The video data may include the target object's face, hands, neck, or other areas of human skin.

[0022] It should be noted that the number of cameras and the acquisition angle can be automatically adjusted according to the current external environment of the target area, the shape of the target object (such as height, number of people, etc.), and actual needs, so as to improve the integrity of multi-view image acquisition. In step S120, candidate physiological signals and local features are extracted from each video data. Optionally, preprocessing is performed frame-by-frame on each video data segment captured by each camera. This includes target region detection (such as face, hand, neck, etc.), alignment and cropping of the detected regions to obtain the target regions. Then, color signals are extracted from the target regions, and the extracted color signals are normalized. For example, pixel intensity normalization is performed on each frame channel, and detrending and standardization are performed in the time dimension to suppress slowly varying illumination and overall brightness drift. This provides a stable and consistent basis for candidate physiological signals and local features to be output for subsequent graph modeling and multi-view fusion.

[0023] Among them, the candidate physiological signal can be a normalized time-series color waveform; the local features can be a multi-dimensional feature vector that characterizes the skin mesh imaging quality and spatial matching relationship, specifically including information such as the effective skin pixel ratio, exposure level, motion blur, and color distribution.

[0024] In step S130, each camera is used as a graph node, and connection edges are established between nodes that are related to the shooting viewpoint. The connection edges are assigned corresponding weights according to the viewpoint matching index between nodes to form a connection topology graph. Optionally, each camera can be treated as an independent graph node in the graph structure. For example, if the cameras are deployed in eight directions around the target area, eight graph nodes can be generated for each camera, labeled C1, C2, C3...C8. Each node stores attribute features related to the imaging conditions of each camera, such as ambient light intensity, changes in light direction, exposure or saturation statistics, and camera viewpoint or imaging stability indicators. These attribute features, along with the aforementioned candidate signals and local features, are used as node features. Then, spatial connection edges can be constructed based on the spatial correlation of each camera's shooting viewpoint, illumination differences, and signal similarity, such as C1 and C2, C1 and C4, C2 and C3, to describe the correlation between different shooting viewpoints. Each spatial connection edge carries an edge weight, which can be calculated based on the geometric relationship between viewpoints, illumination differences, and signal similarity. The resulting graph structure reflects the interdependence of each camera in terms of spatial layout and signal characteristics.

[0025] It should be noted that if a new camera needs to be added to enhance the image acquisition coverage, only the corresponding camera node needs to be added, and the spatial adjacent edges and edge weights can be updated based on the installation location to expand and update the overall graph structure. In addition, a weight filtering threshold can be set to delete low-weight weakly related edges, simplifying the graph structure and reducing computational overhead.

[0026] In step S140, based on the connection topology graph, each graph node performs cross-view information interaction compensation by combining its own node features and the features of adjacent nodes. After multiple rounds of iteration, the fusion features corresponding to each graph node are obtained. The node features include the candidate physiological signals, local features, and camera imaging attribute information. Optionally, based on the connected topology graph, multi-layer progressive graph feature calculation is performed. By aggregating node neighborhood information, cross-view correlation features, and spatial topology information at different levels, deep fusion and global feature update of multi-view image features are achieved. The specific implementation process is as follows: During each layer iteration, each camera node is used as a calculation unit. Based on the spatial adjacency edges in the graph structure, the neighboring node features of all adjacent nodes of the camera are first read. Then, the neighboring node features are aggregated to obtain neighborhood aggregated features. Next, the original node features of the camera node are read, such as inherent attribute features, candidate physiological signals, and local features. Then, the original node features of each camera node and its corresponding neighborhood aggregated nodes are fused to obtain a new node feature representation. The next iteration is then performed until all layers are iterated, finally obtaining the fused features corresponding to each node. After the above multi-layer graph calculation, each node not only considers its own signal but also refers to the feature information of its neighboring nodes. The node features can simultaneously contain single-view image information, neighborhood spatial correlation information, and global multi-view collaborative information, enabling views with lower signal quality to be enhanced from high-quality views. This mechanism is equivalent to establishing a dynamic communication channel between multi-view signals, which can adaptively suppress noise and artifacts from a single viewpoint, thereby improving the reliability of the overall signal.

[0027] In step S150, feature aggregation is performed on each fused feature, and the center stable signal is dynamically calculated based on the aggregated features; Optionally, based on the multi-view fusion features, weights are dynamically calculated according to the quality, similarity, and temporal continuity of the signals from each node. These weights are then used to aggregate the multi-view fusion features, resulting in a central stable signal that represents the overall physiological state of the current target object. This central stable signal can maintain continuity through a sliding update over time, smoothing out instantaneous fluctuations and mitigating interference from abnormal nodes. When the quality of some viewpoint signals deteriorates due to occlusion, uneven illumination, or motion interference, the central stable signal can still maintain a stable output by relying on reliable features from other viewpoints, thus achieving adaptive signal compensation.

[0028] In step S160, the central stabilization signal is mapped to a physiological signal so as to calculate physiological characteristic parameters through the physiological signal.

[0029] Optionally, after obtaining the central stable signal, the system maps the central stable signal into a physiological signal using a simplified prediction head. This prediction head can employ a lightweight linear mapping or a small-scale neural network structure. Then, the physiological signal undergoes preprocessing, filtering and noise reduction, and quantitative analysis to obtain physiological characteristic parameters such as heart rate and respiratory rate. Since physiological signals are easily affected by noise such as human movement, environmental electromagnetic interference, and baseline drift, high-frequency noise and low-frequency baseline drift components are first filtered out using bandpass filtering, detrending processing, and smoothing filtering to retain effective physiological fluctuation signals, resulting in a pure physiological waveform signal with improved signal-to-noise ratio.

[0030] Taking heart rate estimation as an example, a sliding time window can be used to segment the pure physiological signal. A fixed-duration time window is set to truncate and sample the complete physiological signal waveform, ensuring the real-time performance and stability of the physiological characteristic parameter output. For the effective physiological signal within each time window, the time-domain physiological waveform is converted into a frequency-domain spectral distribution using a fast Fourier transform, obtaining the amplitude energy characteristics of the signal at different frequencies.

[0031] By pre-defining an effective heart rate frequency range based on basic human physiology, invalid frequency components outside the normal human heart rate range are eliminated to avoid false detections caused by environmental interference and harmonics. Within the constrained effective heart rate frequency range, the frequency component with the largest spectral amplitude is retrieved as the dominant heart rate frequency for the current window. This dominant frequency corresponds to the actual fluctuation frequency of the human heartbeat. Finally, the dominant frequency value is converted into heart rate per minute using a standard physiological characteristic parameter conversion formula to obtain an accurate heart rate estimate for the current time window.

[0032] Similarly, by setting a specific effective frequency band corresponding to the respiratory rate and repeating the above frequency domain analysis and main frequency extraction process, the real-time respiratory rate parameters can be stably calculated.

[0033] It should be noted that this application is applicable to non-contact physiological monitoring tasks with high requirements for continuity and stability, such as remote vital sign monitoring in hospital wards, long-term health monitoring in home-based elderly care environments, and state perception in cockpits or office environments. In these scenarios, the subjects being measured often experience changes in posture, partial occlusion, or uneven lighting, making single-camera solutions prone to signal interruption or distortion. This application, however, achieves information complementarity between different perspectives through multi-camera collaborative acquisition and graph structure feature fusion, combined with a dynamic maintenance mechanism for a central stable signal, effectively ensuring continuous signal output in complex environments. Therefore, it can significantly improve the temporal continuity and anti-interference capability of remote physiological signals, reduce measurement fluctuations caused by occlusion or movement, achieve long-term stable monitoring, and provide more reliable vital sign data support for practical systems.

[0034] This application provides a method for long-term stable remote physiological signal detection using multiple cameras, comprising: acquiring video data corresponding to a target object from different shooting angles using several cameras; extracting candidate physiological signals and local features from each video data; establishing connection edges between nodes with shooting angle correlation, and assigning corresponding weights to the connection edges according to the inter-node perspective matching index to form a connection topology graph; based on the connection topology graph, each graph node performs cross-perspective information interaction compensation by combining its own node features and the features of adjacent nodes, and obtaining fusion features corresponding to each graph node through multiple iterations, wherein the node features include the candidate physiological signals, local features, and camera imaging attribute information; performing feature aggregation on each fusion feature, and dynamically calculating a central stable signal based on the aggregated features; mapping the central stable signal to a physiological signal, and calculating physiological feature parameters through the physiological signal. In this application embodiment, video data is acquired from different perspectives using multiple cameras, and a graph structure is constructed based on the cameras. Then, a multi-camera feature fusion method based on the graph structure is used to establish dynamic correlations between different perspectives, realizing structured fusion and adaptive compensation for multi-perspective remote physiological signal estimation. When the acquired signal from a single viewpoint becomes abnormal due to changes in lighting, changes in the posture of the object being measured, or occlusion interference, the system can automatically recover the signal by relying on effective features provided by other viewpoints. The maintenance mechanism for a centrally stable signal ensures the smoothness and continuity of the output signal over time, enabling the detection results to maintain high consistency under long-term operating conditions while keeping the signal drift level low. Compared with traditional single-camera detection methods, this scheme significantly improves the robustness, long-term stability, and environmental adaptability of remote physiological signal detection, and can be widely applied in fields such as health monitoring, medical care, and behavior recognition.

[0035] In one embodiment of this application, the step of extracting candidate physiological signals and local features from various video data includes: Extract human skin regions from various video data; The human skin region is divided into several sub-regions; The color values ​​of corresponding pixels in the sub-regions are extracted sequentially according to a preset order to obtain a temporal color signal; The time-series color signal is subjected to illumination normalization processing; The signal after illumination normalization is subjected to temporal detrending and standardization to obtain the candidate physiological signal and local features.

[0036] Optionally, the video data acquired by each camera is processed frame by frame to obtain a continuous video frame sequence. Then, preprocessing operations are performed frame by frame. The specific process is as follows: First, a target detection model (such as YOLO) is used to detect target regions in each video, such as human body regions like faces, hands, and necks, to obtain the current region bounding boxes and confidence scores. If the confidence score is greater than a preset confidence score, key feature points within the target region can be obtained through a key point localization algorithm. For example, taking the face region as an example, key point localization can be used to obtain feature points such as the eyes and nose tip to estimate the face pose. Based on the bounding boxes and key points, the face is geometrically aligned and cropped according to a preset ratio to obtain a ROI of consistent size. At the same time, skin segmentation / masking can be used to remove non-skin pixels such as eyes, eyebrows, lips, and hair to obtain the human skin region.

[0037] Then, the obtained human skin region is divided into several sub-regions. For example, a fixed grid size (e.g., 8×8) can be set, and the entire human skin region is divided according to an equal segmentation strategy to obtain several grid sub-regions. For each independent grid sub-region, the original RGB color values ​​of all pixels within the grid are read row by row, and the RGB mean and color distribution characteristics of pixels within a single grid are statistically analyzed. This statistical value is used as the local color representation of the current grid at the current moment. Furthermore, color features of the skin grid at the same spatial location in consecutive video frames are continuously collected, and the color data of multiple frames are concatenated in chronological order to form a continuous temporal color signal corresponding to each grid. Abnormal exposure and saturation pixels are removed or downweighted. This temporal color signal can accurately reflect the subtle color fluctuations of small areas of the skin over time, providing a reliable raw signal basis for subsequent low-frequency fluctuation analysis of physiological characteristic parameters such as heart rate and respiration.

[0038] Finally, the temporal color signal undergoes illumination normalization to eliminate signal distortion caused by changes in ambient lighting, overall brightness drift, and slow-changing light and shadow interference. Specifically, firstly, the pixel intensity of each color channel is normalized and corrected in the single-frame spatial dimension to eliminate overall brightness differences and local illumination unevenness in a single frame image; secondly, detrending and temporal normalization operations are performed on the temporal color signal of consecutive frames in the temporal dimension to effectively suppress slowly changing illumination shifts, device brightness drift, and low-frequency environmental interference components. This provides a stable and consistent foundation of candidate physiological signals and local features for subsequent graph modeling and multi-view fusion.

[0039] In one embodiment of this application, each graph node combines its own node features and the features of adjacent nodes to perform cross-view information interaction compensation, and obtains the fused features corresponding to each graph node through multiple rounds of iteration, including: In each iteration, the features of all neighboring nodes of each graph node are weighted and aggregated to obtain the neighborhood aggregated features; The neighborhood aggregation features are fused with the individual graph node features to obtain new graph node features; The next iteration is performed using the new graph node features until the preset convergence condition is met, thus obtaining the fused features corresponding to each graph node.

[0040] Optionally, by using cameras as graph nodes and constructing connecting edges based on the geometric relationships between nodes, illumination differences, and signal similarity, and assigning corresponding weights, a multi-camera connection topology graph is constructed. Then, a graph-structured neural network is used to perform neighborhood feature aggregation, cross-node information transmission, and global feature updates layer by layer through multiple graphs, realizing information interaction and compensation between different shooting perspectives. Specifically, in each layer update, for each graph node, the node features of the surrounding neighboring nodes are first weighted and aggregated to ensure that neighbors that are more similar to the current node and of higher quality contribute more. For example, if the neighboring nodes of the current node i are nodes j, k, and g, then the node features of nodes j, k, and g are first aggregated to obtain the neighborhood aggregated features of node A.

[0041] Among them, neighborhood aggregation features Specifically, this can be expressed by the following formula: ; in, The characteristics of node j in layer l, The weight is determined by both the edge weight and the node quality factor.

[0042] Then, the obtained neighborhood aggregation features are aggregated with the original node features of the current node, such as by concatenation or weighted fusion, and updated through a linear layer or a small neural network to obtain new graph node features. For example, the neighborhood aggregation feature = the sum of the features of each neighboring node × their corresponding contribution weights, then the new node feature = activation function (self-feature transformation + neighborhood feature transformation + bias).

[0043] In addition, residual update can also be used, which retains a portion of its original features in the fusion result to avoid the node features being completely covered by other perspectives, thereby taking into account both its own perspective information and compensation information from other high-quality perspectives.

[0044] Subsequently, the next layer of iteration is executed again using the new graph node features until the preset convergence condition is met. After completing the preset number of iterations, the fused features corresponding to each graph node are obtained. Through multi-layer iterative propagation, the stable features of high-quality perspectives can be transmitted along the graph structure and compensated for to perspectives with lower quality due to occlusion, uneven illumination, or motion interference. At the same time, the noise features of low-quality perspectives are suppressed due to their lower weights, thereby realizing cross-perspective information interaction and adaptive compensation, and improving the globally consistent multi-perspective physiological signal representation.

[0045] In one embodiment of this application, the step of weighted aggregation of the features of all neighboring nodes of each graph node to obtain neighborhood aggregation features includes: The quality of each adjacent node is evaluated to obtain the node quality factor; Based on the node quality factor and edge weight, the contribution weight corresponding to each adjacent node is calculated. Based on the contribution weights, the features of the adjacent nodes are weighted and aggregated to obtain the neighborhood aggregated features.

[0046] Optionally, the weights of each adjacent node are determined by both the node quality factor and the edge weights. The edge weights represent the correlation between the views of two cameras, while the node quality factor represents the reliability of the current signal from a particular camera. During aggregation, the edge weights are calculated first, then multiplied by the quality factors of neighboring nodes, and finally normalized to obtain the final contribution weights.

[0047] For example, for the current node i, its neighboring nodes j and k have edge weights of 0.8 and 0.6 respectively, and node quality factors of 0.9 and 0.4 respectively, then: The unnormalized contribution of node j is 0.8 × 0.9 = 0.72; The unnormalized contribution of node k is 0.6 × 0.4 = 0.24; After normalization, the contribution weight of node j is 0.72 / (0.72+0.24)=0.75, and the contribution weight of node k is 0.24 / (0.72+0.24)=0.25.

[0048] Therefore, perspectives with higher similarity and better signal quality contribute more to feature fusion.

[0049] Then, based on the calculated contribution weights, the features of neighboring nodes are weighted and aggregated to obtain the neighborhood aggregated features. For example, the node features of the current node's neighboring node j are... The node characteristics of adjacent node k are: Then the neighborhood aggregation feature = 0.75 × +0.25× .

[0050] In one embodiment of this application, the step of aggregating the features of each fused feature and dynamically calculating the center stability signal based on the aggregated features includes: Calculate the node weight based on the node quality factor and consistency score corresponding to each graph node; Based on the node weights, the fusion features are weighted and aggregated to obtain the center candidate signal; The center candidate signal is updated by sliding according to the preset update coefficient and retention coefficient to obtain the center stable signal.

[0051] Optionally, the node quality factor and consistency score of each camera node are obtained. The node weight is obtained by weighted summation and normalization of these factors. Then, based on these weights, the fusion features corresponding to each node are weighted and aggregated to strengthen the contribution of high-reliability node features and weaken interference from low-quality signals with large spatiotemporal deviations. Finally, a unified central candidate physiological signal at the current moment is output. A sliding window mechanism is used to update the central candidate signal to suppress instantaneous fluctuations and maintain long-term continuity, thus forming a stable central signal. This stable central signal can still be mainly contributed by other high-quality viewpoints even when some viewpoints are obstructed, there is uneven illumination, or motion interference. The output remains stable through a sliding time update mechanism and is used for subsequent filtering, frequency analysis, and vital sign parameter calculation.

[0052] The node quality is calculated by combining real-time video data from the corresponding camera with multi-dimensional signal evaluation indicators. These multi-dimensional evaluation indicators quantify the effectiveness of single-view images and physiological temporal signals, and may include parameters such as the confidence level of human skin target region detection, the proportion of effective skin pixels in the image, the overall exposure level of the image, the image motion blur level, and the signal-to-noise ratio of the normalized temporal color signal in the physiological feature frequency band. A weighted sum of these normalized indicators yields a camera node quality factor between 0 and 1.

[0053] Consistency scoring includes two types of quantitative indicators: spatial consistency and temporal consistency. Spatial consistency characterizes the degree of matching between the current node's signal, local features, and features of other highly reliable nodes, and can be quantified using similarity algorithms such as cosine similarity and Pearson correlation coefficient. Temporal consistency measures the deviation of the node's current features from the baseline stable physiological signal at the previous moment. For example, it can be quantified using mean square error, mean absolute error, Euclidean distance, or correlation difference coefficient. The smaller the feature difference, the better the continuity and stability of the node's temporal fluctuations.

[0054] In one embodiment of this application, the step of performing a sliding update on the candidate center signal according to preset update coefficients and retention coefficients to obtain the stable center signal includes: The quality of the central candidate signal is evaluated; Based on the quality assessment results, the update coefficient and retention coefficient are determined; Within the current time window, based on the update coefficient and the retention coefficient, the center candidate signal and the corresponding center stable signal in the previous time window are weighted and summed to obtain the center stable signal corresponding to the current time window.

[0055] Optionally, after acquiring the central candidate physiological signal, the quality of the central candidate physiological signal can be evaluated. The overall quality score of the current central candidate signal is quantitatively output by comprehensively considering indicators such as the signal-to-noise ratio, temporal fluctuation stability, and effective energy ratio of the physiological frequency band within the window. Based on the quality score, the signal weight coefficients adapted to the subsequent physiological feature parameter solution are obtained. Then, within the current time window, the central candidate signal and the corresponding central stable signal in the previous time window are weighted and summed using the weight coefficients to obtain the central stable signal corresponding to the current time window.

[0056] The central stability signal can be specifically represented by the following formula: ; in, This represents the central candidate signal at time t. This represents the center-stabilized signal at the previous moment. This represents the update coefficient.

[0057] For example, the current center stability signal can be calculated using an exponential moving average algorithm, as shown below: Current center stable signal = retention coefficient × previous time-stable center signal + update coefficient × current center candidate signal.

[0058] Wherein, retention coefficient = 1 - update coefficient.

[0059] If the current candidate signal is of high quality and continuous with historical signals, the update coefficient is increased to allow the system to follow real changes more quickly. If the current candidate signal fluctuates greatly, is of low quality, or differs too much from historical signals, the retention coefficient is increased to make the output more dependent on historical stable signals. Simultaneously, amplitude or frequency change thresholds can be set to limit abnormal jumps, thereby achieving smooth processing and continuity constraints.

[0060] In one embodiment of this application, mapping the central stabilization signal to a physiological signal includes: A linear transformation is performed on the centrally stable signal using a preset weight matrix and bias term, and the complete physiological signal waveform is obtained by splicing the signals according to the sampling time sequence; or After performing nonlinear fitting and dimension mapping on the central stable signal, the physiological signal amplitude is output point by point and then combined in time sequence to form a complete physiological waveform. After amplitude limiting and smoothing filtering of the complete physiological waveform generated by any of the above methods, a standardized physiological signal is finally obtained.

[0061] Optionally, after acquiring the central stable signal, a prediction head can be used to map it into a physiological signal. This prediction head can employ a lightweight linear mapping or a small-scale neural network structure. If a lightweight linear mapping is used, a single fully connected layer can directly perform a linear dimension transformation on the central stable features at each time step using a preset weight matrix and bias term, outputting the physiological signal value at a single time step, such as the corresponding remote PPG waveform sampling point. If a small-scale neural network is used, a two-layer fully connected network or a one-dimensional temporal convolutional network can be used to perform nonlinear fitting and depth dimension mapping on the central stable signal. After linear transformation and nonlinear activation operations, the physiological signal amplitude is output point by point, and the complete initial physiological waveform is reconstructed temporally. Regardless of whether the initial physiological waveform obtained using a linear mapping structure or a small-scale neural network structure is used, amplitude limiting and smoothing filtering are uniformly performed post-processing to remove abnormal amplitude abrupt changes in the waveform and suppress temporal high-frequency noise interference, ultimately outputting a smooth, stable final physiological signal that conforms to the true physiological change pattern.

[0062] In this embodiment, video data is acquired from different perspectives using multiple cameras, and a graph structure is constructed based on the cameras. Then, a multi-camera feature fusion method based on the graph structure is used to establish dynamic correlations between different perspectives, achieving structured fusion and adaptive compensation for multi-view remote physiological signal estimation. When the acquired signal from a single perspective is abnormal due to changes in illumination, changes in the posture of the measured object, or occlusion interference, the system can automatically recover the signal by relying on effective features provided by other perspectives. The maintenance mechanism of the central stable signal ensures the smoothness and continuity of the output signal in the time dimension, enabling the detection results to maintain high consistency under long-term operating conditions, while maintaining a low level of signal drift. Compared with traditional single-camera detection methods, this scheme significantly improves the robustness, long-term stability, and environmental adaptability of remote physiological signal detection, and can be widely used in fields such as health monitoring, medical care, and behavior recognition.

[0063] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0064] In one embodiment, a long-term stable remote physiological signal detection system with multi-camera collaboration is provided. This system corresponds one-to-one with the long-term stable remote physiological signal detection method with multi-camera collaboration described in the above embodiments. Figure 3As shown, this multi-camera collaborative long-term stable remote physiological signal detection system includes a multi-view video data acquisition unit 10, a feature extraction unit 20, a graph construction unit 30, a cross-view information interaction compensation unit 40, a central stable signal generation unit 50, and a physiological feature parameter generation unit 60. Detailed descriptions of each functional module are as follows: The multi-view video data acquisition unit 10 uses several cameras to acquire video data corresponding to the target object from different shooting angles. The feature extraction unit 20 is used to extract candidate physiological signals and local features from each video data. Graph construction unit 30 is used to take each camera as a graph node, establish connection edges between nodes with shooting view relevance, assign corresponding weights to the connection edges according to the view matching degree index between nodes, and form a connection topology graph. The cross-view information interaction compensation unit 40 is used to perform cross-view information interaction compensation based on the connection topology graph, where each graph node combines its own node features and the features of adjacent nodes, and obtains the fusion features corresponding to each graph node through multiple iterations. The node features include the candidate physiological signals, local features, and camera imaging attribute information. The center stabilization signal generation unit 50 is used to perform feature aggregation on each fused feature and dynamically calculate the center stabilization signal based on the aggregated features. The physiological characteristic parameter generation unit 60 is used to map the central stable signal into a physiological signal so as to calculate the physiological characteristic parameters through the physiological signal.

[0065] In one embodiment of this application, the feature extraction unit 20 is further configured to: Extract human skin regions from various video data; The human skin region is divided into several sub-regions; The color values ​​of corresponding pixels in the sub-regions are extracted sequentially according to a preset order to obtain a temporal color signal; The time-series color signal is subjected to illumination normalization processing; The signal after illumination normalization is subjected to temporal detrending and standardization to obtain the candidate physiological signal and local features.

[0066] In one embodiment of this application, the cross-view information interaction compensation unit 40 is further configured to: In each iteration, the features of all neighboring nodes of each graph node are weighted and aggregated to obtain the neighborhood aggregated features; The neighborhood aggregation features are fused with the individual graph node features to obtain new graph node features; The next iteration is performed using the new graph node features until the preset convergence condition is met, thus obtaining the fused features corresponding to each graph node.

[0067] In one embodiment of this application, the cross-view information interaction compensation unit 40 is further configured to: The quality of each adjacent node is evaluated to obtain the node quality factor; Based on the node quality factor and edge weight, the contribution weight corresponding to each adjacent node is calculated. Based on the contribution weights, the features of the adjacent nodes are weighted and aggregated to obtain the neighborhood aggregated features.

[0068] In one embodiment of this application, the center stabilization signal generation unit 50 is further configured to: Calculate the node weight based on the node quality factor and consistency score corresponding to each graph node; Based on the node weights, the fusion features are weighted and aggregated to obtain the center candidate signal; The center candidate signal is updated by sliding according to the preset update coefficient and retention coefficient to obtain the center stable signal.

[0069] In one embodiment of this application, the center stabilization signal generation unit 50 is further configured to: The quality of the central candidate signal is evaluated; Based on the quality assessment results, the update coefficient and retention coefficient are determined; Within the current time window, based on the update coefficient and the retention coefficient, the center candidate signal and the corresponding center stable signal in the previous time window are weighted and summed to obtain the center stable signal corresponding to the current time window.

[0070] In one embodiment of this application, the physiological characteristic parameter generation unit 60 is further configured to: A linear transformation is performed on the centrally stable signal using a preset weight matrix and bias term, and the complete physiological signal waveform is obtained by splicing the signals according to the sampling time sequence; or After performing nonlinear fitting and dimension mapping on the central stable signal, the physiological signal amplitude is output point by point and then combined in time sequence to form a complete physiological waveform. After amplitude limiting and smoothing filtering of the complete physiological waveform generated by any of the above methods, a standardized physiological signal is finally obtained.

[0071] In this embodiment, video data is acquired from different perspectives using multiple cameras, and a graph structure is constructed based on the cameras. Then, a multi-camera feature fusion method based on the graph structure is used to establish dynamic correlations between different perspectives, achieving structured fusion and adaptive compensation for multi-view remote physiological signal estimation. When the acquired signal from a single perspective is abnormal due to changes in illumination, changes in the posture of the measured object, or occlusion interference, the system can automatically recover the signal by relying on effective features provided by other perspectives. The maintenance mechanism of the central stable signal ensures the smoothness and continuity of the output signal in the time dimension, enabling the detection results to maintain high consistency under long-term operating conditions, while maintaining a low level of signal drift. Compared with traditional single-camera detection methods, this scheme significantly improves the robustness, long-term stability, and environmental adaptability of remote physiological signal detection, and can be widely used in fields such as health monitoring, medical care, and behavior recognition.

[0072] Specific limitations regarding the long-term stable remote physiological signal detection system using multi-camera collaboration can be found in the limitations of the method for long-term stable remote physiological signal detection using multi-camera collaboration described above, and will not be repeated here. Each module in the aforementioned long-term stable remote physiological signal detection system using multi-camera collaboration can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0073] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a readable storage medium storing computer-readable instructions. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer-readable instructions implement a long-term, stable, remote physiological signal detection method using multiple cameras. The readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.

[0074] In this application embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it implements the steps of the long-term stable remote physiological signal detection method with multi-camera collaboration as described above.

[0075] In this application embodiment, a readable storage medium is provided, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, they implement the steps of the multi-camera collaborative long-term stable remote physiological signal detection method described above.

[0076] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0077] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0078] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for long-term stable remote physiological signal detection using multi-camera collaboration, characterized in that, The method includes: Video data of the target object is collected from different shooting angles using several cameras; Candidate physiological signals and local features are extracted from various video data. Each camera is treated as a graph node. Connection edges are established between nodes that are related by shooting angle. The connection edges are assigned corresponding weights based on the angle matching index between nodes to form a connection topology graph. Based on the connection topology graph, each graph node performs cross-view information interaction compensation by combining its own node features and the features of adjacent nodes. After multiple rounds of iteration, the fusion features corresponding to each graph node are obtained. The node features include the candidate physiological signals, local features, and camera imaging attribute information. Feature aggregation is performed on each fused feature, and the central stable signal is dynamically calculated based on the aggregated features; The central stabilization signal is mapped to a physiological signal so that physiological characteristic parameters can be calculated from the physiological signal.

2. The method for long-term stable remote physiological signal detection using multi-camera collaboration as described in claim 1, characterized in that, The extraction of candidate physiological signals and local features from each video data includes: Extract human skin regions from various video data; The human skin region is divided into several sub-regions; The color values ​​of corresponding pixels in the sub-regions are extracted sequentially according to a preset order to obtain a temporal color signal; The time-series color signal is subjected to illumination normalization processing; The signal after illumination normalization is subjected to temporal detrending and standardization to obtain the candidate physiological signal and local features.

3. The method for long-term stable remote physiological signal detection using multi-camera collaboration as described in claim 1, characterized in that, Each graph node combines its own node features with those of its neighboring nodes to perform cross-view information interaction compensation. Through multiple iterations, the fused features corresponding to each graph node are obtained, including: In each iteration, the features of all neighboring nodes of each graph node are weighted and aggregated to obtain the neighborhood aggregated features; The neighborhood aggregation features are fused with the individual graph node features to obtain new graph node features; The next iteration is performed using the new graph node features until the preset convergence condition is met, thus obtaining the fused features corresponding to each graph node.

4. The method for long-term stable remote physiological signal detection using multi-camera collaboration as described in claim 3, characterized in that, The step of weighted aggregation of the features of all neighboring nodes of each graph node to obtain neighborhood aggregation features includes: The quality of each adjacent node is evaluated to obtain the node quality factor; Based on the node quality factor and edge weight, the contribution weight corresponding to each adjacent node is calculated. Based on the contribution weights, the features of the adjacent nodes are weighted and aggregated to obtain the neighborhood aggregated features.

5. The method for long-term stable remote physiological signal detection using multi-camera collaboration as described in claim 1, characterized in that, The process of aggregating the fused features and dynamically calculating the center-stabilized signal based on the aggregated features includes: Calculate the node weight based on the node quality factor and consistency score corresponding to each graph node; Based on the node weights, the fusion features are weighted and aggregated to obtain the center candidate signal; The center candidate signal is updated by sliding according to the preset update coefficient and retention coefficient to obtain the center stable signal.

6. The method for long-term stable remote physiological signal detection using multi-camera collaboration as described in claim 5, characterized in that, The step of performing a sliding update on the candidate center signal according to preset update coefficients and retention coefficients to obtain the stable center signal includes: The quality of the central candidate signal is evaluated; Based on the quality assessment results, the update coefficient and retention coefficient are determined; Within the current time window, based on the update coefficient and the retention coefficient, the center candidate signal and the corresponding center stable signal in the previous time window are weighted and summed to obtain the center stable signal corresponding to the current time window.

7. The method for long-term stable remote physiological signal detection using multi-camera collaboration as described in any one of claims 1-6, characterized in that, The process of mapping the central stabilization signal to a physiological signal includes: A linear transformation is performed on the centrally stable signal using a preset weight matrix and bias term, and the complete physiological signal waveform is obtained by splicing the signals according to the sampling time sequence; or After performing nonlinear fitting and dimension mapping on the central stable signal, the physiological signal amplitude is output point by point and then combined in time sequence to form a complete physiological waveform. After amplitude limiting and smoothing filtering of the complete physiological waveform generated by any of the above methods, a standardized physiological signal is finally obtained.

8. A long-term stable remote physiological signal detection system with multi-camera collaboration, characterized in that, The system includes: A multi-view video data acquisition unit is used to acquire video data of the target object from different shooting angles using several cameras. The feature extraction unit is used to extract candidate physiological signals and local features from each video data. The graph construction unit is used to treat each camera as a graph node, establish connection edges between nodes with shooting perspective correlation, assign corresponding weights to the connection edges according to the perspective matching degree index between nodes, and form a connection topology graph. A cross-view information interaction compensation unit is used to perform cross-view information interaction compensation based on the connection topology graph, whereby each graph node combines its own node features and the features of adjacent nodes, and obtains the fusion features corresponding to each graph node through multiple iterations. The node features include the candidate physiological signals, local features, and camera imaging attribute information. The center stabilization signal generation unit is used to perform feature aggregation on each fused feature and dynamically calculate the center stabilization signal based on the aggregated features. A physiological feature parameter generation unit is used to map the central stable signal into a physiological signal, so as to calculate the physiological feature parameters through the physiological signal.

9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and running on the processor, characterized in that, When the processor executes the computer-readable instructions, it implements the long-term stable remote physiological signal detection method with multi-camera collaboration as described in any one of claims 1 to 7.

10. A readable storage medium having computer-readable instructions stored thereon, characterized in that, When the computer-readable instructions are executed by the processor, they implement the long-term stable remote physiological signal detection method with multi-camera collaboration as described in any one of claims 1 to 7.

Images