Multi-modal physiological parameter fusion continuous monitoring method and system

Abstract

The present application relates to the technical field of artificial intelligence monitoring, and more particularly to a multi-modal physiological parameter fusion continuous monitoring method and system, the method comprising: continuously collecting physiological electrical signals of a target individual under multiple behavior states; periodically dividing the multi-lead electrical signals according to the heartbeat rhythm, and standardizing them into feature segments with the same time length and channel number; performing two-stage weighted processing on each feature segment to obtain high-weight feature representation; performing feature alignment on the high-weight feature representation according to the corresponding behavior state, and performing fusion operation; inputting the fusion feature representation into a convolutional neural network to recognize the individual physiological state, and periodically determining whether the target individual currently has a physiological abnormality risk. Through the present application, the signal distortion problem caused by ignoring the multi-behavior state switching in the prior art is effectively solved, the feature adaptability and cross-state recognition robustness in the dynamic scene are significantly improved, and continuous and accurate physiological abnormality risk tracking is realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence monitoring technology, and in particular to a continuous monitoring method and system for multimodal physiological parameter fusion. Background Technology

[0002] With the rapid development of wearable devices, edge computing and artificial intelligence, intelligent monitoring of physiological signals has been widely used in health management, chronic disease early warning and sports rehabilitation. As one of the core physiological indicators, electrocardiogram (ECG) can reflect changes in the electrical activity of the heart, and it will show significant differences in rhythm, morphology and noise interference under different behavioral states.

[0003] In existing technologies, common methods in continuous monitoring scenarios typically extract and model features based on a single physiological signal segment in a static scene. This approach cannot adapt to the non-stationary characteristics of signals caused by dynamic behavior switching such as walking and running. It ignores the fact that people in real-world scenarios experience multiple behavioral state switching, such as prolonged sitting, walking, running, and climbing stairs. The signal features change significantly under different behavioral states, and there are many interfering factors, which pose challenges to model recognition. In addition, some solutions use static statistical features, such as heart rate variability and QRS interval, or use frequency domain features as input, but it is difficult to achieve the model's ability to adapt to signals under behavioral state changes, resulting in a decrease in cross-state recognition accuracy. Summary of the Invention

[0004] This invention provides a method and system for continuous monitoring of multimodal physiological parameters, thereby effectively solving the problems pointed out in the background art.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] Continuous monitoring methods that integrate multimodal physiological parameters include:

[0007] The physiological electrical signals of the target individual are continuously collected under multiple behavioral states, and the physiological electrical signals are multi-lead electrical signals;

[0008] The multi-lead electrical signal is periodically segmented according to the heart rhythm and standardized into characteristic segments with the same time length and number of channels;

[0009] Each feature segment is subjected to a two-level weighting process to obtain a high-weight feature representation;

[0010] The high-weight feature representations are aligned with the corresponding behavioral states, and a fusion operation is performed to obtain a fused feature representation containing multiple behavioral state information.

[0011] The fused feature representation is input into a convolutional neural network composed of channel-wise convolution and cross-channel convolution to identify individual physiological states.

[0012] Based on the output of the convolutional neural network, the system periodically determines whether the target individual is at risk of physiological abnormalities and continuously updates the output results.

[0013] Further, the high-weighted feature representations are aligned according to the corresponding behavioral states, and a fusion operation is performed to obtain a fused feature representation containing multiple behavioral state information, including:

[0014] The high-weight features are divided into several feature groups according to the corresponding behavioral states, and within each feature group, the high-weight features are synchronously aligned on the periodic time axis.

[0015] The global statistical information and local temporal variation information of the fused feature representation in the periodic time dimension and feature dimension are extracted respectively;

[0016] The weights of the fusion stage are calculated based on the global statistical information and the local temporal change information to obtain the fusion feature representation containing multi-behavioral state information.

[0017] Furthermore, the weights for the fusion stage are calculated based on the global statistical information and local temporal change information, including:

[0018] The feature mean vector and feature standard deviation vector on the global statistical information are concatenated with the vectorized form of the feature correlation matrix on the feature dimension in the first level to obtain the first level concatenation result.

[0019] The time-series change sequence and local variance sequence in the local time-series change information are concatenated in a second-level manner to obtain the second-level concatenation result;

[0020] The first-level concatenation result and the second-level concatenation result are combined to generate a joint feature vector;

[0021] Perform a nonlinear transformation on the joint feature vector to output the initial weight values;

[0022] The initial weight values ​​are normalized to generate the fusion weight coefficients corresponding to each feature group.

[0023] Furthermore, the first weighting of the two-level weighting includes:

[0024] Average pooling and max pooling operations are performed on the lead channel characteristic responses of the multi-lead electrical signals under each behavioral state to extract statistical features used to characterize the lead response intensity.

[0025] Based on the statistical characteristics, calculate the first weighting coefficient for each lead channel;

[0026] Multiply the first weighting coefficient by the feature corresponding to each lead channel to obtain the lead feature representation after channel weighting.

[0027] Furthermore, the second weighting of the two-level weighting includes:

[0028] The channel-weighted features are subjected to global pooling on the original time-series dimension to extract statistical descriptions reflecting the trend of response changes in different time-series segments under behavioral states;

[0029] A second weighting coefficient related to time and location is generated based on the statistical description;

[0030] The second weighting coefficient is multiplied point by point with the feature at each time position to obtain the time-weighted representation of the time series features.

[0031] Furthermore, the convolutional neural network includes:

[0032] Depthwise one-dimensional convolutional layers are used to extract local temporal features within each lead channel of the fused feature representation;

[0033] A one-dimensional channel mapping convolutional layer is used to establish feature association information between different lead channels based on the output of the one-dimensional depthwise convolutional layer.

[0034] The output layer is used to generate the recognition results of an individual's physiological state.

[0035] Furthermore, the multi-lead electrical signal is periodically segmented according to the heart rhythm, including:

[0036] The raw data of the multi-lead electrical signal were acquired over a continuous time period, and the rhythmic variation characteristics under different behavioral states were analyzed.

[0037] Based on the rhythmic variation characteristics, the periodic boundaries in the continuous signal are determined, and the periodic boundaries are divided into multiple independent periodic segments.

[0038] Furthermore, the system periodically determines whether the target individual currently faces any physiological abnormality risk and continuously updates the output results, including:

[0039] Based on the state probabilities output by the convolutional neural network, the cumulative probability intensity of abnormal states within the continuous monitoring window is obtained.

[0040] An adaptive baseline is established based on the historical state data of the target individual. When the cumulative probability intensity deviates from the baseline beyond the dynamic tolerance range, an anomaly judgment is triggered.

[0041] Each judgment result is associated with a timestamp to construct a continuously updated risk state sequence, which is refreshed each time a new time window analysis is completed.

[0042] A continuous monitoring system for multimodal physiological parameter fusion, the system comprising:

[0043] The signal acquisition module continuously acquires the physiological electrical signals of the target individual under multiple behavioral states, and the physiological electrical signals are multi-lead electrical signals;

[0044] The period segmentation module performs periodic segmentation on the multi-lead electrical signal according to the heart rhythm and standardizes it into characteristic segments with the same time length and number of channels;

[0045] The feature weighting module performs two-level weighting on each feature segment to obtain a high-weight feature representation.

[0046] The feature fusion module aligns the high-weight feature representations according to the corresponding behavioral states and performs a fusion operation to obtain a fused feature representation containing multiple behavioral state information.

[0047] The state recognition module inputs the fused feature representation into a convolutional neural network composed of channel-wise convolution and cross-channel convolution to perform individual physiological state recognition;

[0048] The output module periodically determines whether the target individual is at risk of physiological abnormalities based on the output of the convolutional neural network, and continuously updates the output results.

[0049] Furthermore, the feature fusion module includes:

[0050] The feature alignment unit divides the high-weight feature representation into several feature groups according to the corresponding behavioral state, and performs synchronous alignment of the high-weight features on the periodic time axis within each feature group.

[0051] The information extraction unit extracts global statistical information and local temporal change information of the fused feature representation in the periodic time dimension and feature dimension, respectively;

[0052] The weight calculation unit calculates the weights for the fusion stage based on the global statistical information and local temporal change information, and obtains the fusion feature representation containing multi-behavioral state information.

[0053] The technical solution of this invention can achieve the following technical effects:

[0054] It effectively solves the signal distortion problem caused by ignoring multiple behavioral state transitions in existing technologies, significantly improves feature adaptability and cross-state recognition robustness in dynamic scenarios, and achieves continuous and accurate tracking of physiological abnormality risks. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating a continuous monitoring method for multimodal physiological parameter fusion;

[0057] Figure 2 A flowchart illustrating the process of obtaining a fused feature representation containing multi-behavioral state information;

[0058] Figure 3 A flowchart illustrating the process of calculating the weights in the fusion phase;

[0059] Figure 4 The flowchart for the first weighted average;

[0060] Figure 5 This is a flowchart illustrating the second weighting process;

[0061] Figure 6 This is a flowchart illustrating the process of periodically segmenting a multi-lead electrical signal.

[0062] Figure 7 A flowchart illustrating the process of periodically assessing whether a target individual is at risk of physiological abnormalities and continuously updating the output results. Detailed Implementation

[0063] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0065] Example 1

[0066] like Figure 1 As shown, this invention provides a method for continuous monitoring of multimodal physiological parameters, the method comprising:

[0067] S1: Continuously collect physiological electrical signals of the target individual under multiple behavioral states. The physiological electrical signals are multi-lead electrical signals.

[0068] Specifically, the acquisition method continuously acquires the electrocardiogram (ECG) signals of the target individual using a multi-lead wearable ECG acquisition system to meet the needs of long-term monitoring under different behavioral states. The system arranges multiple electrodes on the target individual's chest and limbs according to standard ECG lead specifications, forming a twelve-lead acquisition layout. This allows for comprehensive capture of cardiac electrical activity from multiple directions and locations. The electrode material is preferably a low-impedance, flexible, and skin-friendly conductive material to reduce contact impedance changes and artifact interference during movement. Each lead electrode is connected to the portable acquisition unit via flexible connectors to ensure stable signal contact quality under different exercise states such as sitting, walking, jogging, and climbing stairs. During acquisition, the system continuously records ECG signals from each lead at a high sampling rate and simultaneously records the target individual's behavioral state information, providing data annotation for subsequent cross-state analysis. Preferably, the acquisition system has the capability for long-term continuous operation, still able to obtain ECG waveforms with high signal-to-noise ratio and no significant drift in dynamic environments, achieving accurate capture of cardiac electrical activity.

[0069] S2: The multi-lead electrical signal is periodically segmented according to the heart rhythm and standardized into characteristic segments with the same time length and number of channels;

[0070] Specifically, to overcome the problems of rhythm instability, morphological fluctuations, and noise interference caused by changes in behavioral state in ECG signals under dynamic monitoring scenarios, this invention introduces a period segmentation strategy based on heartbeat rhythm in the processing stage of multi-lead ECG signals. This strategy divides continuous ECG waveforms into mutually independent periodic segments with complete physiological significance by using the heartbeat as the period boundary, so that different segments have consistency in rhythm structure. Compared with direct segmentation by fixed time window, this method can better adapt to the differences in period length caused by heart rate changes under different behavioral states, and avoid feature loss or aliasing caused by misalignment between the window and the cardiac cycle. In addition, the multi-lead data after period segmentation is adjusted to the same time length and number of channels through unified standardization processing, thereby ensuring the comparability and fusion of cross-cycle and cross-lead data in subsequent analysis.

[0071] S3: Perform two-level weighting on each feature segment to obtain a high-weight feature representation;

[0072] Specifically, this invention, based on the periodic segmentation and standardized feature segments, introduces a two-level weighting mechanism in the order of channel weighting followed by time weighting to obtain high-weight feature representations. The first level of weighting addresses the differences in importance of different lead channels by analyzing the response intensity of each channel under specific behavioral states, assigning higher weights to leads that contribute more to the recognition task, while reducing the influence of channels with more noise or poor stability, thus completing feature selection in the spatial dimension. The second level of weighting is performed based on the features after the first level of weighting, addressing differences in the temporal dimension by evaluating the response change trends at different time positions within a cardiac cycle, highlighting key time periods closely related to physiological states, and reducing interference from irrelevant time segments. This step-by-step weighting order from channel to time ensures that the final high-weight feature representation possesses both spatial selectivity and temporal attention, preserving the global structure of multi-lead signals while significantly improving the signal-to-noise ratio of features relevant to the target task.

[0073] S4: Align the high-weight feature representations according to their corresponding behavioral states and perform a fusion operation to obtain a fused feature representation containing multiple behavioral state information;

[0074] Specifically, the core of this step is to map the high-weight features obtained under different behavioral states onto a unified reference frame, making them comparable across states, and retaining the key information and differences contained in each state in the fusion result. Since ECG signals show significant changes in rhythm, amplitude and waveform morphology under states such as sitting, walking, running, and climbing stairs, directly mixing these features can easily lead to problems such as feature misalignment and dilution of important information.

[0075] S5: Input the fused feature representation into a convolutional neural network composed of channel-wise convolution and cross-channel convolution to identify individual physiological states;

[0076] Specifically, firstly, local feature patterns are extracted within the time series range of each lead using channel-wise convolution, fully preserving the independence and detailed information of each lead's features. Then, cross-channel convolution is used to establish correlations between different leads, integrating spatial correlations and complementary information into a unified feature representation. Finally, the integrated features are classified by the output layer to obtain the individual's current physiological state recognition result. This phased processing approach of "channel-wise first, then cross-channel" reduces redundant computation caused by traditional fully connected convolution and avoids excessive mixing of features between channels. It can accurately capture temporal local changes and cross-channel correlations in multi-state fusion features, thereby significantly improving computational efficiency and the system's real-time processing capabilities while ensuring recognition accuracy. In this embodiment, a lightweight CNN model is preferably used. This choice is not only to reduce computational complexity and power consumption, but also based on the special requirements of this invention in terms of channel connection method and convolution strategy. Unlike traditional convolution which directly performs fully connected computation across all channels, lightweight CNN achieves targeted establishment of correlation between leads while preserving the independent feature details of each lead by separating channel-wise convolution and cross-channel convolution. This separated channel connection method reduces unnecessary cross-channel computation overhead and avoids excessive mixing of channel features by fully connected convolution, thus being more conducive to preserving the spatial structural features of multi-lead ECG signals. In terms of convolution strategy, lightweight CNN adopts a staged processing method of first extracting local patterns in the time dimension through channel-wise convolution, and then integrating inter-lead information through cross-channel convolution. This is different from traditional one-time convolution, which can more accurately capture local changes and cross-channel correlations of ECG signals in multi-state fusion features. This structure not only improves recognition accuracy, but also ensures the real-time operation capability of the model in resource-constrained environments such as wearable devices, enabling it to achieve stable and efficient physiological state recognition in dynamic, multi-state ECG monitoring.

[0077] S6: Based on the output of the convolutional neural network, periodically determine whether the target individual is at risk of physiological abnormalities and continuously update the output results.

[0078] Specifically, by continuously monitoring the physiological state of the target individual, the convolutional neural network outputs the probability value of the individual's physiological state in real time to reflect the normality or abnormality of the current state. In each monitoring cycle, the presence of abnormal risks is assessed based on these probability values, and potential physiological abnormalities are identified when they exceed a preset threshold. At the same time, as new monitoring data is continuously input, the output results are continuously updated, thereby achieving dynamic risk assessment and adjustment.

[0079] This invention effectively solves the signal distortion problem caused by ignoring multiple behavioral state transitions in existing technologies, significantly improves feature adaptability and cross-state recognition robustness in dynamic scenarios, and achieves continuous and accurate tracking of physiological abnormality risks.

[0080] As a preferred embodiment of the above, such as Figure 2 As shown, in step S4, the high-weight feature representations are aligned according to their corresponding behavioral states, and a fusion operation is performed to obtain a fused feature representation containing multiple behavioral state information, including:

[0081] S41: Divide the high-weight feature representation into several feature groups according to the corresponding behavioral state, and synchronize the high-weight features on the periodic time axis within each feature group.

[0082] S42: Extract the global statistical information and local temporal change information of the fused feature representation in the periodic time dimension and the feature dimension, respectively;

[0083] S43: Calculate the weights of the fusion stage based on global statistical information and local temporal change information to obtain a fusion feature representation containing multi-behavioral state information.

[0084] Specifically, the cycle time dimension and the cycle time axis are two key concepts used to describe the periodic fluctuations in electrocardiogram (ECG) signals. They are closely related and together constitute an effective analytical framework for the temporal characteristics of ECG signals. The cycle time dimension refers to the time range of an ECG signal within a complete cardiac cycle, from the start of one cycle to the start of the next. Within this dimension, each cycle of the ECG signal has a fixed duration and reflects all the important changes in cardiac electrical activity within that cycle. The cycle time axis, on the other hand, is the specific representation of the cycle time dimension, indicating all time points within each cardiac cycle, particularly the time distribution of different waveform phases within the cycle. The cycle time axis is a specific timeline representing the state of the signal at each instant between one R-wave and the next. It provides a unified time framework for each cycle of the ECG signal, enabling the signal characteristics of each cycle to be compared and analyzed on the same time scale.

[0085] Behavioral states can include multiple states such as sitting, walking, running, and climbing stairs. The electrocardiogram signal characteristics may change significantly in each state. Therefore, the features of different states are divided into different feature groups to ensure that the physiological information of different states can be correctly distinguished during feature processing. For the high-weight features in each feature group, they are further synchronized according to their cycle time axis to ensure that the time position and signal waveform in each cycle can be correctly aligned, thereby ensuring that the signal features between different behavioral states can be accurately fused in subsequent processing. Secondly, the aligned features are further processed to extract the global statistical information and local temporal change information of the fused features in the cycle time dimension and feature dimension, respectively. Global statistical information reflects the overall trend of the entire ECG signal cycle, including the mean and standard deviation, and is mainly used to describe the global characteristics of physiological signals under different behavioral states. Local temporal variation information focuses on describing physiological fluctuations at certain key moments within the cycle, usually reflecting changes in an individual's physiological state during a specific period. This information, extracted by a specific algorithm, provides the necessary basis for subsequent weighting and fusion. Finally, based on the global statistical information and local temporal variation information, the weights of the fusion stage are calculated to obtain a fused feature representation containing information from multiple behavioral states. The purpose of this process is to reasonably fuse features from different behavioral states through weighting, thereby obtaining a more comprehensive and representative feature representation. The weighting process combines the characteristics of each behavioral state, assigning different weights according to the importance of physiological information in each state. In this way, the final fused feature representation can better adapt to the differences in signal features under multiple states, improving the accuracy and robustness of subsequent individual physiological state identification and anomaly detection.

[0086] As a preferred embodiment of the above, such as Figure 3 As shown, the weights for the fusion stage are calculated based on global statistical information and local temporal change information, including:

[0087] A10: Perform a first-level concatenation of the feature mean vector and feature standard deviation vector from the global statistical information with the vectorized form of the feature correlation matrix from the feature dimension to obtain the first-level concatenation result;

[0088] A20: Perform a second-level concatenation of the time-series change sequence and the local variance sequence in the local time-series change information to obtain the second-level concatenation result;

[0089] A30: Combine the first-level concatenation result with the second-level concatenation result to generate a joint feature vector;

[0090] A40: Performs a nonlinear transformation on the joint feature vector and outputs the initial weight values;

[0091] A50: Normalize the initial weight values ​​to generate the fusion weight coefficients for each feature group.

[0092] Specifically, firstly, feature mean vectors and feature standard deviation vectors are extracted from global statistical information. The feature mean vector represents the average trend of the ECG signal throughout the entire cycle, revealing the global variation characteristics of the ECG signal. The feature standard deviation vector describes the degree of signal fluctuation, reflecting the stability and instability of the signal in the time dimension. Based on this, a feature correlation matrix is ​​calculated, which represents the interdependence between different features in the signal. By vectorizing the mean vector, standard deviation vector, and feature correlation matrix and concatenating them, the first-level concatenation result is obtained. This step combines global statistical information, such as mean and standard deviation, with the correlation between features, providing a global information foundation for subsequent weighted fusion. Next, local temporal variation information is extracted, including a temporal variation sequence and a local variance sequence. The temporal variation sequence reflects the fluctuation amplitude of the signal within a specific time period, showing the dynamic volatility of the signal. The local variance sequence is used to describe the fluctuation range of the signal within a local time period, revealing the local stability of the signal. The second-level concatenation result is obtained by concatenating the temporal variation sequence and the local variance sequence. This process fully reflects the fluctuation information of local temporal data, providing more detailed temporal features for subsequent fusion and weighting. The first-level and second-level concatenation results are merged through a concatenation operation to generate a joint feature vector. This joint feature vector integrates the global trend and local fluctuations of the ECG signal, providing a more comprehensive feature representation. This vector allows the integration of global and local variation features in the signal, providing rich information for subsequent weighting and recognition. Subsequently, a nonlinear transformation is performed on the joint feature vector. The nonlinear transformation introduces an activation function, such as Re... Nonlinear transformations such as LU and Sigmoid allow feature vectors to be mapped to a nonlinear space, capturing more complex relationships between features. This process improves the model's adaptability to complex signal patterns. After nonlinear transformation, the output is the initial weight value, which is the initial weight value calculated based on the joint feature vector. Finally, the initial weight value is normalized so that the weights of all feature groups can be compared under the same standard. The purpose of normalization is to eliminate the influence that may be caused by different dimensions between different features, ensuring that the weights of each feature group are comparable. After normalization, the final fusion weight coefficient will determine the contribution of each feature group to the final feature representation.

[0093] As a preferred embodiment of the above, such as Figure 4 As shown, the first weight in a two-level weighted average includes:

[0094] B10: Perform average pooling and max pooling operations on the lead channel characteristic responses of multi-lead electrical signals under various behavioral states to extract statistical features used to characterize the lead response intensity;

[0095] B20: Calculate the first weighting coefficient for each lead channel based on statistical characteristics;

[0096] B30: Multiply the first weighting coefficient by the feature corresponding to each lead channel to obtain the lead feature representation after channel weighting.

[0097] Specifically, lead channel features refer to the relevant information extracted from each lead signal in a multi-lead ECG signal, including the signal's temporal variation, amplitude, and morphology. These features reflect the manifestation of cardiac electrical activity in each lead. To weight important features in the signal and ensure that key information is highlighted, a two-stage weighting first step is adopted. First, lead channel features of each lead are extracted from the multi-lead ECG signal. These features include temporal characteristics such as signal amplitude, waveform morphology, and frequency. To extract global information and the strongest response, average pooling and max pooling operations are performed on the signal of each lead channel. Average pooling calculates the average value within each time period, reflecting the overall trend of the lead channel signal; while max pooling selects the maximum value within each time period, emphasizing the strongest response part of the signal. Through these pooling operations, the extracted statistical features can characterize the response intensity of the lead under different behavioral states, providing a basic description of the signal for subsequent weighting. Next, based on the extracted statistical features, i.e., the average response intensity and maximum response intensity of each lead channel, the first weighting coefficient of each lead channel is calculated. This weighting coefficient reflects the importance of each lead channel in a specific behavioral state. When the signal response of a lead is strong, its weighting coefficient is large, indicating that the lead contributes more to the change in physiological state; conversely, a lower weighting coefficient is assigned. This weighting process ensures that the strongest signal response in a behavioral state receives greater attention, thereby enhancing the influence of key information. The first weighting coefficient calculated earlier is multiplied by the original feature of each lead channel. Through this weighting process, the feature of each lead is adjusted according to its response intensity in a specific behavioral state, so that important lead channel features can receive more attention in subsequent analysis. This process ensures that the most relevant lead signals occupy a more important position in the final feature expression.

[0098] As a preferred embodiment of the above, such as Figure 5 As shown, the second weighting in the two-level weighting includes:

[0099] C10: Perform global pooling on the original time-series dimension on the channel-weighted features to extract statistical descriptions that reflect the trend of response changes in different time segments under behavioral conditions;

[0100] C20: Generates a second weighting coefficient related to time location based on statistical description;

[0101] C30: Multiply the second weighting coefficient by the feature at each time position point by point to obtain the time-weighted representation of the time series features.

[0102] Specifically, the second weighting step of the two-level weighting is used to further weight the lead features obtained after the first weighting process, ensuring that the temporal dimension information of the signal is accurately reflected in the final feature representation. The second weighting mainly focuses on the changes of the signal on the time axis. By weighting the time dimension, the physiological characteristics of key time segments under different behavioral states are highlighted.

[0103] The original time-series timeline refers to the complete sequence of all time points from the start to the end of the entire signal acquisition process. Unlike the periodic timeline, the original time-series timeline spans multiple ECG cycles and can include changes in multiple behavioral states. First, on the original time-series timeline, a global pooling operation is performed on each lead signal after the first weighting. Global pooling calculates characteristic statistics within the entire original time-series timeline, such as average, maximum, and minimum values, thereby extracting the global trend and fluctuations of the signal under different behavioral states. This global pooling operation captures the signal response change trends across multiple cycles and behavioral states, providing complete temporal features for subsequent weighting processing. Subsequently, based on the extracted statistical descriptions, second weighting coefficients are calculated according to the change characteristics at each time position on the original time-series timeline. These weighting coefficients reflect the signal at certain moments or time periods on the original time-series timeline under different behavioral states. The relative importance of the signal is determined by the weighting coefficients. If the signal changes significantly during a certain period, the weighting coefficient for that period will be higher, indicating that the signal during that period contributes more to the overall state change. These weighting coefficients provide a basis for subsequent time-weighted processing. Finally, the calculated second weighting coefficients are multiplied point-by-point with the feature values ​​at each time point to form a weighted temporal feature representation. This operation adjusts the feature values ​​at each time point according to their corresponding weighting coefficients, thereby highlighting the periods with significant physiological meaning in the original time-series timeline. Through this weighting process, the important information of the signal in the time dimension can be enhanced, providing more accurate input for subsequent feature fusion and physiological state recognition.

[0104] As a preferred embodiment of the above, the convolutional neural network includes:

[0105] Depthwise one-dimensional convolutional layers are used to extract local temporal features within each lead channel of the fused feature representation;

[0106] Channel mapping one-dimensional convolutional layers are used to establish feature association information between different lead channels based on the output of the depthwise one-dimensional convolutional layer.

[0107] The output layer is used to generate the recognition results of an individual's physiological state.

[0108] Specifically, firstly, the fused feature representation is processed through a Depthwise one-dimensional convolutional layer. Unlike traditional fully convolutional methods, Depthwise convolution performs convolution operations independently within each lead channel, without cross-computing features between different channels. This convolutional method is specifically designed to extract local temporal features within each lead channel, i.e., local changes in each lead over time. These local temporal features reflect changes in an individual's physiological state over a specific time period. Through Depthwise convolution, each lead channel retains its unique temporal features, reducing computational load and avoiding unnecessary interference between channels, focusing on signal feature extraction for each lead. Next, a channel mapping one-dimensional convolutional layer processes the output of the Depthwise one-dimensional convolutional layer. The aim is to establish feature association information between multiple lead channels and to improve the signal characteristics between different leads. The signals exhibit a certain spatial correlation. Channel mapping convolution extracts the cross-channel correlation by performing convolution calculations on the features of different lead channels. The role of this layer is to fuse the local temporal features of each lead channel, enabling the model to consider the signal interrelationships across lead channels and global physiological state changes. In this way, channel mapping convolution provides multi-lead signal correlation features for subsequent physiological state identification. Finally, after processing by the Depthwise one-dimensional convolutional layer and the channel mapping one-dimensional convolutional layer, the fused temporal features enter the output layer. The output layer is responsible for generating the physiological state identification result of the target individual based on the features extracted by the convolutional network. The output layer usually uses a fully connected layer or other structures suitable for classification tasks to perform the final classification. Based on the output of the convolutional neural network, it identifies whether the individual is currently in a normal physiological state or has an abnormal risk, and provides a specific physiological state classification result.

[0109] As a preferred embodiment of the above, such as Figure 6 As shown, the multi-lead electrical signal is periodically segmented according to the heart rhythm, including:

[0110] D10: Acquire raw data of multi-lead electrical signals over a continuous time period and analyze the rhythmic variation characteristics under different behavioral states;

[0111] D20: Based on the rhythmic variation characteristics, determine the periodic boundaries in the continuous signal and divide the periodic boundaries into multiple independent periodic segments.

[0112] Specifically, in this step, signals are first extracted from the raw data of multi-lead electrocardiograms. These signals typically originate from electrophysiological data collected from multiple leads over a continuous time period. During signal acquisition, different behavioral states can cause changes in the rhythmicity of the electrocardiogram (ECG) signals. For example, in a sitting state, the heart rate is slower and the ECG waveform is more stable, while during running, the heart rate increases, and the frequency and amplitude of the ECG waveform change. By analyzing these temporal changes in the signals, rhythmic variation characteristics can be extracted. Rhythmic variation characteristics refer to the periodic fluctuations and change patterns exhibited by the ECG signals under different behavioral states, usually described by temporal features such as heart rate, waveform morphology, and frequency. Rhythmic variation characteristics help identify periodic changes in the signal, thereby... Precisely define periodic boundaries; based on the extracted rhythmic variation features, analyze the periodic fluctuations in the signal, and determine the periodic boundaries in the signal through feature detection algorithms. The periodic boundaries refer to the start and end points of each cardiac cycle in the electrocardiogram (ECG) signal, which is the basis for period segmentation. By identifying these periodic boundaries, continuous signals can be divided into multiple independent periodic segments, each segment representing a complete cardiac cycle. In this way, the signal can be segmented according to the rhythm of the heartbeat, thereby providing accurate periodic data for subsequent feature extraction, weighted processing, and identification analysis. Each independent periodic segment corresponds to a complete ECG signal cycle in time, and because the characteristics of rhythmic variation are fully considered, period segmentation can adapt to changes in ECG signals under different behavioral states.

[0113] As a preferred embodiment of the above, such as Figure 7 As shown, the system periodically assesses whether the target individual currently faces any physiological abnormality risk and continuously updates the output, including:

[0114] E10: Based on the state probability output by the convolutional neural network, obtain the cumulative probability intensity of abnormal states within a continuous monitoring window;

[0115] E20: Establish an adaptive baseline based on the historical state data of the target individual, and trigger anomaly judgment when the cumulative probability intensity deviates from the baseline beyond the dynamic tolerance range;

[0116] E30: Correspond each judgment result with a timestamp to construct a continuously updated risk status sequence, and refresh the risk status sequence every time a new time window analysis is completed.

[0117] Specifically, firstly, based on the state probabilities output by the convolutional neural network, the current physiological state of the target individual is assessed. These state probabilities represent the probability that the target individual is in a normal or abnormal physiological state at a certain moment, quantifying the duration of the abnormal state within a specific time period. Within a continuous monitoring window, the cumulative probability intensity of the abnormal state is calculated. This intensity represents the probability value that the target individual continues to be in an abnormal state within the time window, quantifying the duration of the abnormal state within a specific time period. By calculating this cumulative probability intensity, the changing trend of the target individual's physiological state can be tracked, and potential abnormal patterns in the signal can be identified, providing a basis for subsequent abnormal risk assessment. Next, based on the target individual's historical state data, an adaptive benchmark is established. This benchmark describes the fluctuation range of the target individual in a normal physiological state and is dynamically updated over time and with changes in physiological state. When new data is input, the current cumulative probability intensity is calculated and compared with the historical benchmark. If the cumulative probability intensity deviates from the benchmark and exceeds the set dynamic tolerance range, it is determined that there is a risk of physiological abnormality. The dynamic tolerance range can adaptively adjust according to the long-term physiological changes of the target individual, ensuring the individualization and accuracy of anomaly judgment. Each time an anomaly judgment is made, the judgment result is associated with the corresponding timestamp, the time point of each judgment result is recorded, and it is stored in the risk status sequence. This sequence reflects the physiological risk status of the target individual in different time periods. When each new monitoring window is analyzed, the risk status sequence is refreshed to ensure the timeliness and accuracy of the sequence. In this way, the physiological status of the target individual can be continuously tracked and the risk status can be updated in real time.

[0118] Example 2

[0119] Based on the same inventive concept as the continuous monitoring method for multimodal physiological parameter fusion in the foregoing embodiments, the present invention also provides a continuous monitoring system for multimodal physiological parameter fusion, the system comprising:

[0120] The signal acquisition module continuously acquires the physiological electrical signals of the target individual under multiple behavioral states. The physiological electrical signals are multi-lead electrical signals.

[0121] The period segmentation module performs periodic segmentation of the multi-lead electrical signal according to the heart rhythm and standardizes it into characteristic segments with the same time length and number of channels;

[0122] The feature weighting module performs two-level weighting on each feature segment to obtain a high-weight feature representation.

[0123] The feature fusion module aligns the high-weight feature representations according to their corresponding behavioral states and performs fusion operations to obtain a fused feature representation containing multiple behavioral state information.

[0124] The state recognition module inputs the fused feature representation into a convolutional neural network composed of channel-wise convolution and cross-channel convolution to identify individual physiological states.

[0125] The output module, based on the output of the convolutional neural network, periodically determines whether the target individual is at risk of physiological abnormalities and continuously updates the output results.

[0126] The monitoring system described above in this invention can effectively realize the continuous monitoring method of multimodal physiological parameter fusion, and the technical effects it can achieve are as described in the above embodiments, and will not be repeated here.

[0127] As a preferred embodiment of the above, the feature fusion module includes:

[0128] The feature alignment unit divides the high-weight feature representation into several feature groups according to the corresponding behavioral state, and performs synchronous alignment of the high-weight features on the periodic time axis within each feature group.

[0129] The information extraction unit extracts global statistical information and local temporal change information of the fused feature representation in the periodic time dimension and feature dimension, respectively.

[0130] The weight calculation unit calculates the weights for the fusion stage based on global statistical information and local temporal change information, and obtains a fusion feature representation that includes multi-behavioral state information.

[0131] Similarly, the above-mentioned optimization schemes for the system can also achieve the optimization effects corresponding to the methods in Embodiment 1, which will not be repeated here.

[0132] Although this application has been described in conjunction with specific features and embodiments, it will be apparent that various modifications and combinations can be made therein without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of the application as defined herein, and are to be considered as covering any and all modifications, variations, combinations, or equivalents within the scope of this application.

[0133] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. A continuous monitoring method for multimodal physiological parameter fusion, characterized in that, include: The physiological electrical signals of the target individual are continuously collected under multiple behavioral states, and the physiological electrical signals are multi-lead electrical signals; The multi-lead electrical signal is periodically segmented according to the heart rhythm and standardized into characteristic segments with the same time length and number of channels; Each of the aforementioned feature segments undergoes a two-level weighting process to obtain a high-weight feature representation. The first weighting of the two-level weighting includes: Average pooling and max pooling operations are performed on the lead channel characteristic responses of the multi-lead electrical signals under each behavioral state to extract statistical features used to characterize the lead response intensity. Based on the statistical characteristics, calculate the first weighting coefficient for each lead channel; Multiply the first weighting coefficient by the feature corresponding to each lead channel to obtain the lead feature representation after channel weighting; The second weighting of the two-level weighting includes: The channel-weighted features are subjected to global pooling on the original time-series dimension to extract statistical descriptions reflecting the trend of response changes in different time-series segments under behavioral states; A second weighting coefficient related to time and location is generated based on the statistical description; The second weighting coefficient is multiplied point by point with the feature at each time position to obtain the time-weighted representation of the time series features. The high-weight feature representations are aligned with the corresponding behavioral states, and a fusion operation is performed to obtain a fused feature representation containing multiple behavioral state information. The fused feature representation is input into a convolutional neural network composed of channel-wise convolution and cross-channel convolution to identify individual physiological states. Based on the output of the convolutional neural network, the system periodically determines whether the target individual is at risk of physiological abnormalities and continuously updates the output results.

2. The continuous monitoring method for multimodal physiological parameter fusion according to claim 1, characterized in that, The high-weighted feature representations are aligned with the corresponding behavioral states, and a fusion operation is performed to obtain a fused feature representation containing multiple behavioral state information, including: The high-weight features are divided into several feature groups according to the corresponding behavioral states, and within each feature group, the high-weight features are synchronously aligned on the periodic time axis. The global statistical information and local temporal variation information of the fused feature representation in the periodic time dimension and feature dimension are extracted respectively; The weights of the fusion stage are calculated based on the global statistical information and the local temporal change information to obtain the fusion feature representation containing multi-behavioral state information.

3. The continuous monitoring method for multimodal physiological parameter fusion according to claim 2, characterized in that, The weights for the fusion stage are calculated based on the global statistical information and local temporal change information, including: The feature mean vector and feature standard deviation vector on the global statistical information are concatenated with the vectorized form of the feature correlation matrix on the feature dimension in the first level to obtain the first level concatenation result. The time-series change sequence and local variance sequence in the local time-series change information are concatenated in a second-level manner to obtain the second-level concatenation result; The first-level concatenation result and the second-level concatenation result are combined to generate a joint feature vector; Perform a nonlinear transformation on the joint feature vector to output the initial weight values; The initial weight values ​​are normalized to generate the fusion weight coefficients corresponding to each feature group.

4. The continuous monitoring method for multimodal physiological parameter fusion according to claim 1, characterized in that, The convolutional neural network includes: Depthwise one-dimensional convolutional layers are used to extract local temporal features within each lead channel of the fused feature representation; A one-dimensional channel mapping convolutional layer is used to establish feature association information between different lead channels based on the output of the one-dimensional depthwise convolutional layer. The output layer is used to generate the recognition results of an individual's physiological state.

5. The continuous monitoring method for multimodal physiological parameter fusion according to claim 1, characterized in that, The multi-lead electrical signal is periodically segmented according to the heart rhythm, including: The raw data of the multi-lead electrical signal were acquired over a continuous time period, and the rhythmic variation characteristics under different behavioral states were analyzed. Based on the rhythmic variation characteristics, the periodic boundaries in the continuous signal are determined, and the periodic boundaries are divided into multiple independent periodic segments.

6. The continuous monitoring method for multimodal physiological parameter fusion according to claim 1, characterized in that, Periodically determine whether the target individual currently has a risk of physiological abnormalities, and continuously update the output results, including: Based on the state probabilities output by the convolutional neural network, the cumulative probability intensity of abnormal states within the continuous monitoring window is obtained. An adaptive baseline is established based on the historical state data of the target individual. When the cumulative probability intensity deviates from the baseline beyond the dynamic tolerance range, an anomaly judgment is triggered. Each judgment result is associated with a timestamp to construct a continuously updated risk state sequence, which is refreshed each time a new time window analysis is completed.

7. A continuous monitoring system for multimodal physiological parameter fusion, characterized in that, The system includes: The signal acquisition module continuously acquires the physiological electrical signals of the target individual under multiple behavioral states, and the physiological electrical signals are multi-lead electrical signals; The period segmentation module performs periodic segmentation on the multi-lead electrical signal according to the heart rhythm and standardizes it into characteristic segments with the same time length and number of channels; The feature weighting module performs two-level weighting on each feature segment to obtain a high-weight feature representation. The first weighting of the two-level weighting includes: Average pooling and max pooling operations are performed on the lead channel characteristic responses of the multi-lead electrical signals under each behavioral state to extract statistical features used to characterize the lead response intensity. Based on the statistical characteristics, calculate the first weighting coefficient for each lead channel; Multiply the first weighting coefficient by the feature corresponding to each lead channel to obtain the lead feature representation after channel weighting; The second weighting of the two-level weighting includes: The channel-weighted features are subjected to global pooling on the original time-series dimension to extract statistical descriptions reflecting the trend of response changes in different time-series segments under behavioral states; A second weighting coefficient related to time and location is generated based on the statistical description; The second weighting coefficient is multiplied point by point with the feature at each time position to obtain the time-weighted representation of the time series features. The feature fusion module aligns the high-weight feature representations according to the corresponding behavioral states and performs a fusion operation to obtain a fused feature representation containing multiple behavioral state information. The state recognition module inputs the fused feature representation into a convolutional neural network composed of channel-wise convolution and cross-channel convolution to perform individual physiological state recognition; The output module periodically determines whether the target individual is at risk of physiological abnormalities based on the output of the convolutional neural network, and continuously updates the output results.

8. The continuous monitoring system for multimodal physiological parameter fusion according to claim 7, characterized in that, The feature fusion module includes: The feature alignment unit divides the high-weight feature representation into several feature groups according to the corresponding behavioral state, and performs synchronous alignment of the high-weight features on the periodic time axis within each feature group. The information extraction unit extracts global statistical information and local temporal change information of the fused feature representation in the periodic time dimension and feature dimension, respectively; The weight calculation unit calculates the weights for the fusion stage based on the global statistical information and local temporal change information, and obtains the fusion feature representation containing multi-behavioral state information.

Images