Multi-modal physiological signal fusion method and system based on wearable device

Abstract

The application provides a multi-modal physiological signal fusion method and system based on a wearable device. The method comprises: collecting, by the wearable device, multi-source physiological signals representing different physiological dimensions of the human body and auxiliary signals representing the motion state of the human body; determining interference processing parameters based on the signal attributes of the multi-source physiological signals and the motion state within the fusion time window, generating effective physiological signals and signal quality representations; extracting low-dimensional physiological features from each effective physiological signal, and forming a set of to-be-fused features according to the time correspondence; determining a target fusion strategy from the preset fusion strategy; performing associated combination on the set of to-be-fused features according to the target fusion strategy, and generating a multi-modal physiological fusion representation; matching the multi-modal physiological fusion representation with the state judgment condition corresponding to the preset monitoring task, and outputting the human state evaluation result. The application can enhance the correlation of multi-source physiological signals, improve the anti-interference ability, and reduce the fusion calculation overhead.

Description

Technical Field

[0001] This application relates to the fields of intelligent wearable devices and physiological signal processing technology, and in particular to a method and system for multimodal physiological signal fusion based on wearable devices. Background Technology

[0002] With the development of smart sensors, low-power chips, and mobile health technologies, wearable devices are now able to continuously collect various signals reflecting the physiological and activity states of the human body, and are applied to scenarios such as health monitoring, motion analysis, fatigue recognition, and sleep assessment. Existing wearable devices typically use different sensing units to collect corresponding physiological signals, and independently perform threshold judgment, trend statistics, or abnormal alerts for each type of signal. Some solutions also upload the collected data to external platforms for analysis.

[0003] However, different physiological signals exhibit inconsistencies in sampling frequency, data scale, and noise characteristics. Existing solutions lack correlation processing under a unified time reference, making it difficult to leverage the complementary relationships between different physiological dimensions. Furthermore, human movement, environmental changes, wear position misalignment, and poor sensor contact can easily cause signal distortion. Independent analysis of a single signal may amplify local errors, leading to biased or inaccurate health status assessments. While some multimodal fusion methods can comprehensively process multi-source data, they typically rely on complex models and high computational resources, making them unsuitable for the limited computing power and battery life of wearable devices. Therefore, how to synchronously correlate, suppress interference, and achieve lightweight fusion of multiple physiological signals under low-power constraints to improve the accuracy and completeness of human condition assessment remains a technical challenge that needs to be addressed. Summary of the Invention

[0004] In view of this, embodiments of this application provide a multimodal physiological signal fusion method and system based on wearable devices to solve the problems of insufficient correlation of multi-source physiological signals, susceptibility to interference, and complex fusion calculation in the prior art.

[0005] A first aspect of this application provides a method for multimodal physiological signal fusion based on a wearable device, comprising: collecting multi-source physiological signals representing different physiological dimensions of the human body and auxiliary signals representing the human body's motion state through a wearable device; dividing each signal into a corresponding fusion time window according to a unified time reference; determining corresponding interference processing parameters based on the signal attributes of the multi-source physiological signals and the motion state within the fusion time window; performing noise reduction, anomaly identification, and missing signal repair on the multi-source physiological signals to generate effective physiological signals and corresponding signal quality representations; extracting low-dimensional physiological features associated with a preset monitoring task from each effective physiological signal and forming a set of features to be fused according to the time correspondence; determining a target fusion strategy from a preset fusion strategy based on the signal quality representation and the operating resource status of the wearable device, wherein the target fusion strategy is used to determine the participation status and contribution weight of the features to be fused; performing association combination on the set of features to be fused according to the target fusion strategy to generate a multimodal physiological fusion representation; matching the multimodal physiological fusion representation with the state judgment conditions corresponding to the preset monitoring task, outputting a human body state assessment result, and storing or transmitting the assessment process data and the human body state assessment result.

[0006] A second aspect of this application provides a multimodal physiological signal fusion system based on a wearable device, comprising: a data acquisition module, configured to acquire multi-source physiological signals representing different physiological dimensions of the human body and auxiliary signals representing the human body's motion state through the wearable device, and divide each signal into a corresponding fusion time window according to a unified time reference; a generation module, configured to determine corresponding interference processing parameters based on the signal attributes of the multi-source physiological signals and the motion state within the fusion time window, perform noise reduction, anomaly identification, and missing signal repair on the multi-source physiological signals, and generate effective physiological signals and corresponding signal quality representations; and an extraction module, configured to extract from each effective physiological signal a signal quality representation and a preset signal quality representation. The system monitors low-dimensional physiological features associated with monitoring tasks and forms a set of features to be fused based on time correspondence. A determination module determines a target fusion strategy from preset fusion strategies based on signal quality characterization and the operational resource status of the wearable device. The target fusion strategy determines the participation status and contribution weight of the features to be fused. A combination module performs associated combination of the feature set to be fused according to the target fusion strategy to generate a multimodal physiological fusion representation. An output module matches the multimodal physiological fusion representation with the state judgment conditions corresponding to the preset monitoring tasks, outputs the human state assessment results, and stores or transmits the assessment process data and the human state assessment results.

[0007] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: This invention collects multi-source physiological signals representing different physiological dimensions of the human body and auxiliary signals representing human motion state using wearable devices. Each signal is divided into a corresponding fusion time window according to a unified time reference. Based on the signal attributes of the multi-source physiological signals and the motion state within the fusion time window, corresponding interference processing parameters are determined. Noise reduction, anomaly identification, and missing signal repair are performed on the multi-source physiological signals to generate effective physiological signals and corresponding signal quality representations. Low-dimensional physiological features associated with a preset monitoring task are extracted from each effective physiological signal, and a set of features to be fused is formed according to the time correspondence. Based on the signal quality representation and the operating resource status of the wearable device, a target fusion strategy is determined from the preset fusion strategies. The target fusion strategy is used to determine the participation status and contribution weight of the features to be fused. The set of features to be fused is associated and combined according to the target fusion strategy to generate a multimodal physiological fusion representation. The multimodal physiological fusion representation is matched with the state judgment conditions corresponding to the preset monitoring task to output the human state assessment result. The assessment process data and the human state assessment result are stored or transmitted. This application can enhance the correlation of multi-source physiological signals, improve anti-interference ability, and reduce fusion computational overhead. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart illustrating the multimodal physiological signal fusion method based on wearable devices provided in an embodiment of this application. Figure 2 This is a schematic diagram of the structure of a multimodal physiological signal fusion system based on wearable devices provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0010] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0011] Existing wearable devices typically employ multiple sensing units to collect multi-source signals reflecting human physiological and activity states. These signals are then used for health monitoring, fatigue analysis, and sleep assessment through threshold judgment, trend statistics, or single-modal recognition. While some existing solutions can acquire multiple types of physiological data simultaneously, they usually collect and analyze data separately for different signal types, lacking correlation processing under a unified time reference and failing to fully utilize the complementary relationships between different physiological dimensions.

[0012] Human movement, wearer position shifts, sensor contact status, and environmental changes can all interfere with physiological signals, making single-modal signals prone to fluctuations, distortions, data gaps, or abrupt changes. Existing independent analysis methods cannot differentiate between different modal data based on motion state and signal quality; locally distorted signals may directly affect the determination of human condition. Furthermore, existing multimodal fusion methods typically rely on high computational resources and complex models, making them unsuitable for the limited computing power, storage capacity, and battery life of wearable devices. Therefore, existing technologies suffer from insufficient correlation of multi-source physiological signals, weak anti-interference capabilities, and high computational overhead in fusion.

[0013] To address the aforementioned issues, this application collects multi-source physiological signals and auxiliary signals characterizing human motion states using wearable devices, and divides each signal into corresponding fusion time windows according to a unified time reference. Based on the attributes of different signals and the motion state within the fusion time window, corresponding interference processing parameters are determined to perform noise reduction, anomaly identification, and missing signal repair on the multi-source physiological signals, generating effective physiological signals and signal quality representations. Low-dimensional physiological features associated with the monitoring task are extracted from the effective physiological signals, and a set of features to be fused is formed based on time correspondence. Furthermore, the target fusion strategy is determined by combining the signal quality representation and the operating resource status of the wearable device, configuring the participation status and contribution weight of each low-dimensional physiological feature, associating and combining the set of features to be fused, generating a multimodal physiological fusion representation, and outputting human state assessment results accordingly.

[0014] Through the above technical solutions, this application can establish the correlation between different physiological dimensions within a unified time range, and dynamically adjust the contribution of each modality data to the fusion result according to motion interference and signal quality, thereby reducing the impact of abnormal data on the state assessment result. At the same time, by using a fusion strategy of low-dimensional feature extraction and resource adaptation, the scale of data processing on the terminal side is reduced, thereby enhancing the correlation of multi-source physiological signals, improving the anti-interference capability and assessment accuracy of physiological state monitoring, and reducing the fusion computational overhead.

[0015] The technical solution of this application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0016] Figure 1This is a flowchart illustrating the multimodal physiological signal fusion method based on wearable devices provided in an embodiment of this application. Figure 1 As shown, the method may specifically include: S101 collects multi-source physiological signals representing different physiological dimensions of the human body and auxiliary signals representing the human body's movement state through wearable devices, and divides each signal into the corresponding fusion time window according to a unified time reference. S102, based on the signal attributes of multi-source physiological signals and the motion state within the fusion time window, determines the corresponding interference processing parameters, performs noise reduction, anomaly identification and missing repair on multi-source physiological signals, and generates effective physiological signals and corresponding signal quality characterization. S103, extract low-dimensional physiological features associated with the preset monitoring task from each effective physiological signal, and form a set of features to be fused according to the time correspondence; S104. Based on the signal quality characterization and the operating resource status of the wearable device, determine the target fusion strategy from the preset fusion strategies. The target fusion strategy is used to determine the participation status and contribution weight of the features to be fused. S105, Perform association combination on the set of features to be fused according to the target fusion strategy to generate a multimodal physiological fusion representation; S106 matches the multimodal physiological fusion representation with the state judgment conditions corresponding to the preset monitoring task, outputs the human state assessment results, and stores or transmits the assessment process data and the human state assessment results.

[0017] In some embodiments, wearable devices are used to collect multi-source physiological signals characterizing different physiological dimensions of the human body and auxiliary signals characterizing the human body's movement state. These signals are then divided into corresponding fusion time windows according to a unified time reference, including: The sampling configuration corresponding to each signal is determined according to the preset monitoring task, and the sampling configuration is dynamically adjusted according to the operating resource status of the wearable device and the human motion status. Drive the corresponding sensing unit to collect multi-source physiological signals and auxiliary signals according to the adjusted sampling configuration, and write the time identifier and collection status identifier generated based on the device clock to the collected data. Based on the sampling period and time offset of each sensing unit, the time stamp of the collected data is calibrated to generate a signal sequence under a unified time reference; The signal sequence is segmented into time segments according to a preset window length, and the segment boundaries are adjusted based on the temporal continuity of adjacent acquired data. By associating corresponding signal segments within the same time range to the same fusion time window, a multimodal time series data set carrying sampling configuration and acquisition status identifiers is generated.

[0018] Specifically, wearable devices can be intelligent monitoring terminals worn on the user's wrist. These terminals are equipped with optical physiological sensing units, electrophysiological sensing units, temperature sensing units, and inertial sensing units to collect data reflecting circulatory status, cardiac electrical activity, body surface temperature, and limb activity, respectively. Preset monitoring tasks can include daily health monitoring, fatigue detection, or sleep state analysis. Taking daily health monitoring as an example, the device controller reads the task configuration table to determine the basic sampling frequency, single sampling duration, quantization accuracy, and allowable sleep interval for various signals, and generates corresponding sampling configurations for different sensing units. For example, electrophysiological signals are continuously acquired at a higher sampling frequency, optical physiological signals are periodically acquired at a medium sampling frequency, body temperature signals are acquired at lower sampling frequency intervals, and inertial signals are continuously acquired as needed for activity detection.

[0019] During device operation, the controller periodically acquires remaining battery power, processor utilization, cache usage, and current human motion status. When the human is seated or asleep and the remaining battery power is below a preset threshold, the sampling interval of some sensor units is extended, and the sampling frequency of signals that do not affect task determination is reduced. When the human changes from a static state to walking, running, or strenuous activity, the sampling frequency of inertial signals is increased, and the sampling frequencies of optical and electrophysiological signals that are easily affected by motion interference are appropriately increased to obtain continuous waveforms for subsequent interference identification. When the processor load exceeds a preset load threshold, the sampling configuration of key physiological signals remains unchanged, while the sampling density of non-critical auxiliary signals is reduced first to avoid resource contention between the acquisition task and the fusion computing.

[0020] Each sensing unit initiates data acquisition according to the adjusted sampling configuration. For each set of sampled data acquired, the device controller writes a time identifier based on the unified device clock, and also writes an acquisition status identifier characterizing the sensing unit number, sampling frequency level, contact status, operating mode, and data integrity. For example, when the optical sensing unit detects looseness, it writes a contact abnormality identifier to the corresponding sampled data; when the device is in low-power mode, it writes an interval sampling identifier to the body temperature data. The acquisition status identifier is written to the circular buffer along with the sampled data for subsequent time calibration and signal quality analysis.

[0021] Due to differences in startup delay and sampling period among different sensing units, the device controller establishes a time calibration table based on the device clock, recording the fixed offset and dynamic drift of each sensing unit relative to the reference clock. The controller predicts the theoretical sampling time based on the sampling period, compares the actual time stamp with the theoretical sampling time, performs time remapping on data falling within the allowable deviation range, and configures a timing anomaly flag for data exceeding the allowable deviation range. For signals with different sampling frequencies, the original sampled values ​​are not forcibly changed; instead, a unified timing index is established using the calibrated time stamp to generate a multi-channel signal sequence under a unified time reference.

[0022] Subsequently, the controller segments the multi-channel signal sequence according to a preset window length. For example, a 30-second fusion time window is used for routine health monitoring tasks, with adjacent windows having a 5-second overlap interval. During segmentation, the controller calculates the time interval between adjacent sampled data. When the window boundary falls within a continuous action process, a short-term data gap, or a position where the sensor unit switches sampling levels, the boundary is adjusted forward or backward to a position where the time is continuous and the acquisition status is stable, with the adjustment amount limited to a preset range. For missing segments that cannot be covered by boundary adjustment, the corresponding time position is retained and a missing identifier is written to avoid subsequent incorrect splicing.

[0023] Furthermore, after segmentation, the controller, based on the calibrated time range, associates various physiological signal segments and auxiliary signal segments within the same time period to the same fusion time window. A unique window identifier is assigned to each fusion time window, and the signal source, sampling configuration, acquisition status identifier, and time coverage of each segment are recorded, generating a multimodal time-series data set. This allows for maintaining cross-modal time correspondence under asynchronous sampling and dynamic frequency modulation conditions across different sensing units, providing a continuous and traceable data foundation for subsequent targeted noise reduction, low-dimensional feature extraction, and lightweight fusion, while reducing energy consumption from invalid high-frequency sampling.

[0024] In some embodiments, based on the signal attributes of multi-source physiological signals and the motion state within the fusion time window, corresponding interference processing parameters are determined respectively. Noise reduction, anomaly identification, and missing data repair are then performed on the multi-source physiological signals to generate effective physiological signals and corresponding signal quality characterizations, including: Based on the sampling characteristics and waveform variation features of each multi-source physiological signal, the corresponding signal processing constraints are determined; Motion intensity and attitude change features within the fusion time window are extracted from the auxiliary signal to generate motion disturbance characterization; The signal processing constraints are matched with the motion disturbance characterization to determine the filtering parameters and anomaly judgment parameters corresponding to each multi-source physiological signal, and adaptive noise reduction is performed on each multi-source physiological signal. Based on the continuity, stability, and consistency of the correlation between different physiological dimensions of the noise-reduced signal, abnormal data and missing segments are identified. Temporal compensation is performed on abnormal data and missing segments that meet the repair conditions, and invalidation is configured for data that does not meet the repair conditions to generate valid physiological signals; Based on the noise reduction magnitude, data completeness, and correlation consistency, signal quality characterizations corresponding to each effective physiological signal are generated.

[0025] Specifically, after constructing a multimodal time-series data set, the wearable device reads heart rate, ECG, blood oxygen, body temperature, and motion posture data, using each fusion time window as a processing unit. The device controller first establishes signal processing constraints based on the sampling frequency, amplitude range, effective waveform period, and allowable rate of change for each type of signal. For example, ECG data focuses on constraints related to baseline drift, spike noise, and waveform interval continuity; optical physiological data focuses on constraints related to pulse wave period, perfusion amplitude, and stability of adjacent peaks; and body temperature data focuses on constraints related to slow change trends and upper limits of change per unit time. These signal processing constraints can be pre-stored on the device or adaptively adjusted based on the user's historical stable data.

[0026] For the same fusion time window, the controller calculates the resultant acceleration amplitude, angular velocity change, and attitude transition frequency from the auxiliary signals collected by the inertial sensing unit, and determines the state of rest, light activity, continuous walking, or vigorous movement based on the degree of change at continuous sampling points. The controller further extracts the dominant motion frequency, impact duration, and changes in wearing direction to generate a motion interference characterization. The motion interference characterization records the motion level, main interference frequency band, and interference occurrence interval, used to distinguish between genuine physiological changes and pseudo-changes caused by wrist swinging or device slippage.

[0027] Subsequently, the controller matches the signal processing constraints with motion interference characteristics. When the user is stationary, a narrower baseline correction range is used for ECG and optophysiological data to preserve subtle waveform changes. When the user is walking or running, the filtering bandwidth is adjusted according to the dominant motion frequency, and a motion reference component is constructed using auxiliary signals to weaken interference components that change synchronously with the motion reference component from the original physiological signal. For body temperature data, the smoothing window is adjusted according to posture changes and the degree of environmental abrupt changes. The controller simultaneously determines the amplitude overshoot threshold, abrupt change threshold, period deviation threshold, and continuous distortion duration, and performs adaptive noise reduction and initial anomaly detection accordingly.

[0028] Furthermore, after noise reduction, the controller calculates the continuity of adjacent samples, waveform stability within the window, and effective sampling percentage for each signal. For data exhibiting transient spikes, short-term flatness, or periodic interruptions, anomaly identification is first performed based on the temporal changes before and after the modality, followed by correlation verification using other physiological dimensions. For example, if optical physiological data shows a sudden increase in heart rate without corresponding changes in ECG rhythm and exercise intensity, the spike segment is identified as suspected interference; if blood oxygen levels drop briefly and are accompanied by both strenuous exercise and optical contact anomalies, the corresponding segment is identified as a low-confidence anomaly; if body temperature changes slowly and matches changes in activity intensity, the change is retained as valid data. By cross-verifying different physiological dimensions within the same time window, false rejections caused by relying solely on a single threshold are reduced.

[0029] For outliers or missing segments whose duration is less than the repair threshold and whose preceding and following data maintain the same trend, the controller uses the trend constraint of adjacent valid samples for timing compensation and limits the compensation value to no more than the corresponding signal processing constraint. For periodic signals, missing samples can also be recovered based on the waveform position of adjacent complete cycles. For data whose duration exceeds the repair threshold, whose contact state is continuously abnormal, or whose cross-modal verification results conflict, numerical completion is not performed; instead, an invalidation flag is configured and the original time position is retained. After compensation and marking, valid physiological signals corresponding to each fusion time window are formed.

[0030] Finally, the controller determines the noise reduction amplitude based on the change in interference energy before and after noise reduction, determines the data integrity based on the effective sampling ratio and the proportion of compensation segments, and determines the correlation consistency based on the degree of common support of each physiological dimension for the target state change. The above indicators are then normalized and combined to generate signal quality representations corresponding to each effective physiological signal. Signal quality representations may include quality level and quality score, and are associated with and stored with the signal source, fusion time window, abnormal segments, and repair records, providing a basis for subsequent low-dimensional feature screening, contribution weight configuration, and fusion strategy selection. Through this processing, heterogeneous interference can be dynamically suppressed under conditions of motion and wearing state changes, preventing unrepairable data from entering the fusion process and improving the reliability of subsequent human state assessments.

[0031] In some embodiments, low-dimensional physiological features associated with a preset monitoring task are extracted from each effective physiological signal, and a set of features to be fused is formed according to the time correspondence, including: Based on the state determination requirements corresponding to the preset monitoring tasks, construct task feature constraints to characterize the state of the target human body. Based on the signal attributes of each effective physiological signal, temporal change features and state representation features are extracted within the fusion time window to form candidate physiological features; The task-related degree between candidate physiological features and target human state is determined based on task feature constraints, and the target physiological features are screened based on the redundancy between candidate physiological features. The target physiological features are scaled and compressed to generate low-dimensional physiological features corresponding to each effective physiological signal. A time correspondence is established based on the fusion time window and signal source of the low-dimensional physiological features. The time-corresponding low-dimensional physiological features and the corresponding signal quality characteristics are associated and encapsulated to form a set of features to be fused.

[0032] Specifically, after the wearable device generates valid physiological signals within each fusion time window, the device controller reads the current preset monitoring task and determines the target human state, judgment time scale, and allowed signal types from the task configuration table. Taking fatigue state recognition as an example, the target human state includes awake, mild fatigue, and significant fatigue. Task feature constraints are used to limit the feature categories that can reflect changes in autonomic nervous system, heart rhythm, blood oxygen fluctuations, and reduced activity, while recording the effective time range, minimum signal quality, and upper limit of dimension for each feature. For sleep state analysis tasks, task feature constraints focus on heart rate changes, blood oxygen stability, body surface temperature trends, and limb activity levels during continuous resting periods. Through task feature constraints, it is possible to avoid extracting all signal features unrelated to the current monitoring task, reducing the computational load on the terminal side.

[0033] For each fusion time window, the device controller invokes the corresponding lightweight feature operator according to the signal attributes of each valid physiological signal. For signals with periodic waveforms, the period length, period dispersion, rate of change between adjacent periods, and waveform stability are extracted within the window; for slowly changing signals, the window mean, slope of change, range, and degree of continuous deviation are extracted; for motion posture data, activity intensity, percentage of stillness, frequency of posture transitions, and duration of continuous activity are extracted. Among these features, parameters describing the change process within the window are used as temporal variation features, and parameters describing the current physiological level or activity state are used as state characterization features. The device controller associates and records the feature source, calculation interval, signal quality score, and valid feature identifier for each candidate physiological feature, forming a candidate feature table.

[0034] The device controller performs preliminary matching of the candidate feature table based on task feature constraints, calculating the task correlation degree between candidate physiological features and the target human state. The task correlation degree can be determined comprehensively based on preset feature importance, the degree of deviation of the current feature from the individual's physiological baseline, and the state discrimination ability in historical labeled samples. For example, in a fatigue recognition task, features with decreased heart rate variability, increased sustained rest time, and gradual changes in body surface temperature have high task correlation; short-term amplitude changes occurring alone and lacking other signal support have low task correlation. Candidate physiological features below the correlation threshold are not included in subsequent fusion.

[0035] For candidate physiological features selected based on task relevance, the device controller further calculates the redundancy within the same signal and between different signals. If two features show high consistency in their direction of change, amplitude of change, and state differentiation results across multiple consecutive fusion time windows, features with lower computational complexity, higher signal quality, or greater task relevance are preferentially retained. For example, if the average heart rate and the heart rate level converted from cycle length repeatedly represent the same physiological meaning, only one is retained; if activity intensity and exercise energy show a stable correspondence in the current task, the feature with shorter data length is retained. This process yields the target physiological features and controls the number of features involved in the fusion for each type of signal.

[0036] Subsequently, the device controller performs scaling uniformity based on the preset physiological range or the user's historical stable range for each target physiological feature, converting features of different dimensions to a unified numerical range. For feature groups whose dimensions still exceed the task limit, they are sorted according to task relevance, or the dimensions are compressed using an edge-executable linear projection method to generate low-dimensional physiological features corresponding to each valid physiological signal. During the dimensionality compression process, the feature source index and compression parameters are preserved, allowing subsequent evaluation results to be traced back to the original signal segment.

[0037] Finally, the device controller establishes a time correspondence based on the fusion time window identifier, start and end time, and signal source associated with the low-dimensional physiological features. Low-dimensional physiological features belonging to the same fusion time window are written into the feature container in a preset order, and the corresponding signal quality representation, validity identifier, task relevance, and source index are written simultaneously to form a set of features to be fused. If a signal does not form a valid feature, a missing identifier is written at the corresponding position without using other modal data to replace it. Through this processing, redundant data can be compressed while preserving the complementarity of multidimensional physiological information, reducing the computational burden of subsequent association and combination, and ensuring the traceability of each low-dimensional physiological feature in terms of time and source.

[0038] In some embodiments, determining a target fusion strategy from preset fusion strategies based on signal quality characterization and the operating resource status of the wearable device includes: Based on the signal quality characterization associated with each low-dimensional physiological feature, the corresponding feature confidence level is determined; Obtain the remaining energy and current computing load of wearable devices to generate resource availability that characterizes fusion processing capabilities; Low-dimensional physiological features that meet the participation conditions are selected based on feature credibility level, and the allowed fusion complexity is determined based on resource availability. The selected low-dimensional physiological features, the allowed fusion complexity, and the applicable conditions of the preset fusion strategy are matched to obtain candidate fusion strategies. Based on a comprehensive evaluation of the task suitability and resource consumption of the candidate fusion strategies, the target fusion strategy is determined, and the participation status and contribution weight of each low-dimensional physiological feature are configured.

[0039] Specifically, after the wearable device forms a set of features to be fused, the device controller reads the signal quality score, effective sampling ratio, compensation segment ratio, and cross-modal association consistency associated with each low-dimensional physiological feature according to the fusion time window, and generates a feature credibility level based on preset credibility grading rules. For example, a high credibility level is configured when the signal quality score is not lower than 0.8, the effective sampling ratio is not lower than 90%, and there are no continuous invalid segments; a medium credibility level is configured when the basic integrity condition is met but short-term repair data exists; and a low credibility level is configured when the signal quality score is lower than the participation threshold or the association consistency is abnormal. For multiple low-dimensional physiological features from the same source, the signal quality representation of the effective physiological signal can be used as the basic score, and then corrected by combining the task correlation degree to avoid features with low quality but significant numerical changes receiving excessive fusion contributions.

[0040] The device controller simultaneously invokes the power management unit and task scheduling unit to read remaining battery power, estimated battery life, processor utilization, and current data acquisition task usage. The controller normalizes each resource parameter and configures resource weights according to the importance of the current monitoring task, generating resource availability. Taking all-weather fatigue monitoring as an example, when the remaining battery power is above 60% and the processor utilization is below 40%, resource availability is configured as sufficient; when the remaining battery power is between 20% and 60% or the processor utilization is between 40% and 75%, it is configured as limited; and when the remaining battery power is below 20% or the processor utilization exceeds 75%, it is configured as strained. Resource availability is updated with the fusion time window, and frequent switching between adjacent levels is suppressed through a hysteresis threshold.

[0041] Subsequently, the controller filters low-dimensional physiological features for fusion based on their reliability levels. High-reliability features are directly added to the participation set, medium-reliability features are added when they exhibit consistent changes with other physiological dimensions, and low-reliability features are not included in the current fusion, only their source index and missing positions are retained. If no high-reliability features exist for the physiological dimensions required for the monitoring task, the medium-reliability features with the highest reliability can be selected as restricted participation features, with a contribution cap set. The controller then determines the permissible fusion complexity based on resource availability, including the number of features allowed to participate, the number of association calculations, and the scale of compressed operations. When resources are sufficient, cross-modal association combinations are allowed; when resources are limited, grouped weighted combinations are used; and when resources are scarce, only core features with high task relevance are retained for linear fusion.

[0042] The pre-defined fusion strategy library includes complete association strategies, selective weighting strategies, and core feature strategies. Each fusion strategy associates records applicable to the monitoring task, minimum feature confidence structure, resource requirement range, and expected resource consumption. The controller matches the number of low-dimensional physiological features, their source distribution, confidence level structure, and allowed fusion complexity with the applicable conditions of each fusion strategy. For example, when the ECG rhythm features, heart rate change features, and activity intensity features within the fatigue monitoring time window are all of high confidence and resources are sufficient, the complete association strategy is identified as a candidate; when the optical physiological features are reduced to low confidence due to wrist movement, the selective weighting strategy that excludes the corresponding features is identified as a candidate; when the device enters a low battery state, the core feature strategy is prioritized for matching.

[0043] For candidate fusion strategies, the controller calculates task fit based on the degree to which the candidate fusion strategy covers the necessary physiological dimensions of the task, historical evaluation stability, and current feature reliability structure. It also calculates resource consumption based on the expected number of computations, memory usage, and energy consumption. The controller comprehensively evaluates the task fit and resource consumption according to preset evaluation weights, and selects the strategy with the highest comprehensive evaluation result from among the candidate fusion strategies that meet the minimum task fit requirements as the target fusion strategy.

[0044] Furthermore, after determining the target fusion strategy, the controller configures the participation status for each low-dimensional physiological feature in the feature set to be fused. For features participating in fusion, initial contribution weights are calculated based on feature reliability level, task relevance, and redundancy of similar features, and then normalized. For features with limited participation, the normalization result is limited to a preset contribution upper limit. For features not participating, the contribution weight is configured to zero and the invalidation reason is retained. This allows the fusion process to be dynamically adjusted according to signal quality and equipment resources, ensuring the integrity of fatigue state determination information while reducing the impact of low-reliability data and unnecessary calculations on the fusion result and equipment endurance.

[0045] In some embodiments, the set of features to be fused is associated and combined according to the target fusion strategy to generate a multimodal physiological fusion representation, including: Read the participation status of the target fusion strategy configuration and filter low-dimensional physiological features to participate in the fusion from the set of features to be fused; Based on the temporal correspondence and physiological correlation of each low-dimensional physiological feature, a cross-modal feature association unit is constructed; According to the contribution weights configured in the target fusion strategy, the cross-modal feature association units are weighted and combined, and missing labels are configured for feature locations that do not participate in the fusion. The weighted combination results are scaled and compressed to generate a multimodal physiological fusion representation of the comprehensive physiological state within the fusion time window.

[0046] Specifically, after determining the target fusion strategy, the wearable device uses the set of features to be fused corresponding to the current fusion time window as the processing object, and reads the participation status, contribution weight, signal source, task relevance, and signal quality characterization of each low-dimensional physiological feature. Taking fatigue state monitoring as an example, the participation status can include effective participation, limited participation, and non-participation. The device controller writes the low-dimensional physiological features with effective and limited participation into the current fusion cache, and removes the low-dimensional physiological features without participation from the fusion calculation queue, but retains the corresponding feature position, source index, and invalidation reason. For low-dimensional physiological features with limited participation, the controller limits the corresponding weight according to the contribution upper limit set by the target fusion strategy to avoid features that have been compensated or have large quality fluctuations dominating the fusion result.

[0047] Furthermore, after feature screening, the device controller performs time alignment on the low-dimensional physiological features involved in the fusion based on the fusion time window identifier and the feature acquisition period. Features with consistent start and end times are directly associated; features with slight deviations in time coverage are matched according to the effective coverage ratio and time center position within the fusion time window; features with coverage below a preset ratio are not forcibly associated with other features within the current time window. The controller further invokes physiological association rules to identify feature combinations that can jointly reflect the target human body state across different physiological dimensions. For example, the degree of heart rate change, the stability of electrocardiographic rhythm, and activity intensity are constructed as a circulatory activity association unit, and the degree of blood oxygen fluctuation, rest duration, and body surface temperature change trend are constructed as a recovery state association unit. Each cross-modal feature association unit records the participating features, the degree of time correspondence, and the physiological association type.

[0048] For each cross-modal feature association unit, the device controller weights the participating features according to the contribution weights configured in the target fusion strategy. The contribution weights can be determined by the feature confidence level, task relevance, and redundancy correction coefficient. For example, when the ECG rhythm feature is highly reliable and highly correlated with fatigue, a higher weight is assigned; when the optical physiological feature is affected by motion but still meets the limited participation condition, the corresponding weight is reduced; when the activity intensity feature highly overlaps with continuous static time, one of the features is redundancy-reduced. The controller first performs weighted aggregation within the association unit, and then combines the units according to their adaptability to the current monitoring task to form an initial fusion vector.

[0049] For feature locations not involved in fusion, the controller writes a missing identifier and a missing reason identifier. The missing identifier is used to distinguish feature missingness caused by sensor unit not collecting data, insufficient signal quality, and resource limitations. It does not use fixed values ​​to replace real physiological features, avoiding the subsequent state determination from interpreting placeholder data as normal physiological values. If the target fusion strategy specifies a certain physiological dimension as a necessary dimension, and all corresponding features are missing, then an incomplete dimension identifier is simultaneously written into the initial fusion vector to reduce the evaluation reliability during state confidence calculation.

[0050] Because the output ranges and statistical distributions of each associated unit differ, the device controller performs scale correction on the initial fusion vector based on preset scale parameters or the user's historical stable intervals, mapping the combined results of different physiological dimensions to a unified numerical range. For users with an individual physiological baseline, correction can be performed using the deviation of the current combined result relative to the individual physiological baseline; for users who have not yet formed an individual physiological baseline, correction is performed using the general reference range in the task configuration. After scale correction, the controller performs linear projection or weighted aggregation on the combined components with high correlation according to the output dimensions defined by the target fusion strategy, compressing redundant information and preserving the mapping relationship between each output component and the original low-dimensional physiological features.

[0051] Finally, the device controller encapsulates the compressed fusion vector, dimensional complete state, participating feature sources, and corresponding fusion time window identifiers to generate a multimodal physiological fusion representation. This representation comprehensively describes circulatory activity, oxygenation level, body temperature changes, and movement state within the current time window, and serves as input for subsequent fatigue state category determination and state confidence calculation. By selecting features according to participating states, constructing associated units based on physiological relationships, and implementing reliable weighted fusion, the interference of low-quality data and duplicate features on the fusion results can be reduced, while dimensionality compression controls the computational scale on the terminal side.

[0052] In some embodiments, the multimodal physiological fusion representation is matched with the state determination conditions corresponding to a preset monitoring task to output a human state assessment result, and the assessment process data and the human state assessment result are stored or transmitted, including: The corresponding status judgment conditions are invoked according to the preset monitoring task, and the task judgment parameters are extracted from the multimodal physiological fusion representation. The task judgment parameters are matched with the state judgment conditions to determine the human state category and state confidence level corresponding to the fusion time window. Temporal consistency verification is performed based on the human state category and state confidence of the continuous fusion time window to generate human state assessment results carrying the state change trend. The assessment process data is generated by linking multimodal physiological fusion representation, target fusion strategy, state determination criteria, and human state assessment results. The assessment process data and human condition assessment results are stored locally and transmitted to external devices when preset transmission conditions are met.

[0053] Specifically, after the wearable device generates a multimodal physiological fusion representation, the device controller reads the currently preset monitoring task and retrieves the state determination conditions corresponding to the task type from the task configuration table. Taking fatigue state monitoring as an example, the state determination conditions can include the parameter range, duration requirements, necessary physiological dimensions, and minimum confidence requirements corresponding to wakefulness, mild fatigue, and significant fatigue. Based on the mapping relationship of the output components in the multimodal physiological fusion representation, the device controller extracts task determination parameters reflecting the degree of heart rate change, the stability of electrocardiographic rhythm, the degree of blood oxygen fluctuation, the trend of body temperature change, and the activity level. At the same time, it reads the complete state of the dimensions, the confidence level of the participating features, and the current fusion strategy to form the determination input for the current fusion time window.

[0054] The device controller first matches each task judgment parameter with its corresponding state judgment condition, calculating the degree of parameter deviation and the degree of condition satisfaction. Then, it weights and summarizes the parameters according to their importance in the fatigue monitoring task. For example, when the heart rate change continuously decreases, the ECG rhythm fluctuates more, the activity level decreases, and blood oxygen remains stable, it can be judged as mild fatigue; when multiple parameters simultaneously exceed the significant fatigue range and persist for a preset duration, it can be judged as significant fatigue; when only a single parameter is abnormal and the corresponding signal quality is low, the fatigue level is not directly increased. The state confidence score is generated comprehensively based on the degree of condition satisfaction, the number of participating physiological dimensions, the signal quality of each dimension, and missing identifiers. When necessary physiological dimensions are missing or the target fusion strategy adopts a low-complexity mode, the state confidence score is correspondingly attenuated.

[0055] To avoid state jumps caused by short-term fluctuations within a single fusion time window, the device controller retrieves the human state category and state confidence level from multiple consecutive fusion time windows preceding the current fusion time window and performs a timing consistency check. If the state category remains consistent across adjacent time windows and the state confidence level is consistently higher than the confirmation threshold, the corresponding human state is confirmed. If the state category changes repeatedly between adjacent levels, smoothing is performed by considering the state duration and the direction of parameter changes. If a high-confidence abrupt change occurs in the current time window and multiple physiological dimensions synchronously support this change, the abrupt change result is retained.

[0056] In some examples, taking a 30-second fusion time window as an example, if three consecutive time windows are determined to be mild fatigue, a mild fatigue confirmation result is generated; if the parameter deviation in subsequent consecutive windows continues to increase, the trend of worsening fatigue is recorded. This results in a human state assessment result that includes the current state category, state confidence, state duration, and direction of state change.

[0057] The device controller establishes a correlation between the current multimodal physiological fusion representation, target fusion strategy identifier, participating features and contribution weights, state determination conditions, task determination parameters, condition matching results, and human state assessment results to generate assessment process data. This data is organized according to the fusion time window identifier and device time sequence, and records the reasons for signal loss, weight adjustments, strategy switching, and confidence decay, enabling subsequent backtracking from the assessment results to the corresponding physiological features and original signal segments. For obvious fatigue or continuous low-confidence results, an abnormal event index can also be generated, facilitating rapid location of relevant time periods by mobile terminals.

[0058] During the local storage phase, the device controller writes the human condition assessment results to the long-term results area, writes the multimodal physiological fusion representation and key judgment criteria to the circular cache area, and hierarchically retains historical assessment process data according to storage capacity. Only summary data is saved for normal states, while more complete assessment process data is retained for states that have changed or are abnormal. Preset transmission conditions may include external device connectivity availability, sufficient remaining battery power for transmission, user-initiated viewing requests, or the occurrence of high-priority anomalies.

[0059] When transmission conditions are met, the device controller encapsulates the data to be transmitted in chronological order, writes the device identifier, task type, data integrity identifier, and verification information, and sends it to the mobile terminal or remote health management platform. When communication is unavailable, the data is retained in the local transmission queue and transmission continues after the connection is restored. By using state matching, continuous window verification, and process data association, misjudgments caused by short-term noise can be reduced, improving the stability and traceability of human condition assessment, while also taking into account the storage and communication power consumption of wearable devices.

[0060] In some embodiments, the method further includes: From the evaluation process data corresponding to the continuous fusion time window, stable physiological samples that meet the baseline update conditions in terms of signal quality characterization and state confidence are selected. According to the monitoring task, stable physiological samples are aggregated in time to generate individual physiological baselines that characterize the range of normal physiological changes and the relationship between physiological responses under different exercise states. Calculate the degree of deviation of the current multimodal physiological fusion representation from the individual's physiological baseline, and use the degree of deviation as the basis for correcting the state determination conditions; Based on the confirmed human condition assessment results, the contribution weights of individual physiological baselines, condition determination conditions, and corresponding low-dimensional physiological characteristics are incrementally updated, and assessment process data that do not meet the update confidence conditions are updated and isolated.

[0061] Specifically, during continuous operation, the wearable device continuously generates evaluation process data according to the fusion time window. The device controller reads the signal quality representation, state confidence, human state category, motion state, and multimodal physiological fusion representation within a preset update cycle, and selects stable physiological samples. Stable physiological samples need to meet several conditions, including that there are no consecutive missing main physiological dimensions involved in the fusion, the signal quality score reaches the update threshold, the state confidence is continuously higher than the confirmation threshold, the fusion strategy does not switch frequently within the corresponding time period, and there are no obvious abnormal events.

[0062] For example, taking fatigue monitoring as an example, when a user is in a normal conscious state and continuously maintains low-intensity activity, the heart rate changes, ECG rhythm, blood oxygen level, and activity intensity corresponding to multiple fusion time windows are all within a stable range. The device controller marks the data corresponding to these fusion time windows as candidate stable physiological samples. If a certain time window has a high confidence level but contains a large proportion of compensated data or abnormal wearing contact, it will not be included in the baseline update.

[0063] The device controller groups stable physiological samples according to the monitoring task and motion state. For fatigue monitoring tasks, state groups can be divided into resting, mild activity, and continuous exercise groups, and the central values, normal fluctuation ranges, rate of change ranges, and response relationships between different physiological dimensions of low-dimensional physiological characteristics and multimodal physiological fusion representations are statistically analyzed within each group. For example, when a user transitions from resting to walking, heart rate typically increases, activity intensity increases synchronously, and blood oxygen remains relatively stable. The controller records this coordinated change as a normal physiological response relationship under mild activity. For motion state groups with insufficient sample size, independent baselines are not generated initially; instead, baselines from adjacent motion levels are used and the allowable fluctuation range is expanded. After time-series aggregation, individual physiological baselines are generated, including the normal physiological change range, motion state label, applicable time range, and sample reliability.

[0064] In subsequent state assessments, the device controller selects a matching individual physiological baseline based on the current activity state. It compares the current multimodal physiological fusion representation with the central value and normal fluctuation range of the baseline, calculates the standardized deviation of each fusion component, and generates a comprehensive deviation degree based on the duration of the deviation. For example, if a user is at rest and their heart rate changes and ECG rhythm fluctuations continuously deviate from their resting baseline without an increase in activity intensity, the corresponding deviation degree is used as an enhanced basis for fatigue determination. If an increase in heart rate is accompanied by an increase in exercise intensity, and this conforms to the normal response relationship under exercise conditions, the contribution of this change to fatigue determination is reduced. The device controller incorporates the comprehensive deviation degree into the state determination conditions to correct general thresholds, state confidence levels, and state level transition conditions, enabling differentiated interpretations of the same physiological value for different users and under different activity states.

[0065] Once the human condition assessment results are confirmed via a continuous window or by the user providing status feedback on an external device, the device controller generates a baseline update command. For data confirmed to be in a normal state, a limited step size is used to incorporate the current stable sample into the individual physiological baseline under the corresponding motion state, updating the central value, fluctuation range, and physiological response relationship. For data confirmed to be in a fatigue state, it is not directly written into the normal baseline, but is instead used to adjust the mapping relationship between the degree of deviation and the fatigue level in the state determination criteria.

[0066] In some examples, the controller also fine-tunes the contribution weights of relevant low-dimensional physiological features based on the consistency between the evaluation results and subsequent confirmation results. For example, if multiple consecutive confirmation results indicate that ECG rhythm features have a high ability to distinguish user fatigue states, the corresponding contribution weight is increased within a preset range; if a certain type of feature consistently produces misjudgments, the corresponding weight is decreased.

[0067] For data with insufficient state confidence, missing key physiological dimensions, frequent changes in fusion strategies, or conflicts between user feedback and evaluation results, the device controller will update the isolation flag and write it to an independent isolation area. This data will not be involved in updating individual physiological baselines, state determination conditions, or contribution weights. Isolated data will only be used for subsequent quality review or supplementary data collection analysis, and isolation will only be lifted after the reconfirmation conditions are met.

[0068] By using stable sample screening, aggregation by motion state, and limited incremental updates, the state determination can be gradually adapted to the user's long-term physiological characteristics, while avoiding baseline drift caused by abnormal data and erroneous assessments, thus improving the individual adaptability and stability of human state assessment under continuous monitoring conditions.

[0069] In some embodiments, the method further includes: When the confidence level of the human body status assessment result is lower than the preset threshold, or when there is a correlation conflict between the low-dimensional physiological features corresponding to different physiological dimensions, the target signal to be supplemented is determined based on the quality characterization of each signal. Based on the operating resource status of the wearable device, determine the supplementary sampling duration and sampling configuration corresponding to the target signal, and control the corresponding sensing unit to perform directional supplementary sampling within a limited time. The supplementary sampling signal is subjected to interference processing and low-dimensional feature extraction, and the obtained supplementary sampling features are written into the corresponding feature set to be fused. The multimodal physiological fusion representation is regenerated based on the updated set of features to be fused, and the human state assessment result is determined based on the change in state confidence before and after the reassessment.

[0070] Specifically, after the wearable device completes the human state assessment within the current fusion time window, the device controller reads the state confidence, the contribution direction of each low-dimensional physiological feature, signal quality characterization, and correlation consistency. Preset thresholds can be configured based on the monitoring task; for example, in fatigue state monitoring, a supplementary sampling decision is triggered when the state confidence is below 0.65. Correlation conflicts can manifest as different physiological dimensions providing contradictory support for the same state. For instance, the degree of heart rate change and ECG rhythm characteristics support deepening fatigue, while continuously increasing activity intensity supports normal exercise response; or optical physiological characteristics show decreased blood oxygenation, but ECG rhythm, exercise state, and body temperature changes do not show corresponding abnormalities. The device controller determines the physiological dimension causing the low confidence or conflict based on the source, contribution weight, and signal quality score of the conflicting features.

[0071] For each physiological dimension to be verified, the device controller compares the effective sampling percentage, motion interference level, contact status, and historical stability of the corresponding signal, and determines the target signal by combining the necessary information missing from the current state determination. For example, if the quality of the optical physiological signal decreases due to wrist movement while the ECG signal remains stable, the optical physiological signal is determined as the target signal; if an elevated heart rate cannot distinguish between motion response and fatigue abnormalities, the inertial signal and the ECG signal are jointly determined as the target signal; if multiple physiological dimensions have high quality but conflicting changes, the signal with higher discrimination ability and the ability to verify the conflicting relationship is prioritized for supplementary sampling. The controller also records the reason for supplementary sampling, the source of the target signal, and the state determination conditions to be verified.

[0072] The device controller reads the remaining battery power, processor utilization, cache space, and current sensing tasks to generate supplementary sampling resource constraints. When the remaining battery power is sufficient and the computational load is low, a longer supplementary sampling duration and a higher sampling frequency can be configured for the target signal. When resources are limited, the supplementary sampling duration is shortened, only the sampling frequency of key sensing units is increased, and unnecessary background processing is suspended. Taking fatigue monitoring as an example, when optical physiological signals need to be verified, supplementary sampling can be performed for 20 to 40 seconds after the user remains still, and the optical sampling density is increased. When ECG rhythm needs to be verified, the original sampling accuracy can be maintained and the continuous acquisition time can be extended. If the exercise intensity exceeds the allowable supplementary sampling conditions, the controller delays supplementary sampling and continuously monitors the degree of posture stability until it enters the acquisition state or exceeds the maximum waiting time.

[0073] Within the limited re-sampling time, the controller drives the target sensing unit to perform directional re-sampling, writing the associated identifier, re-sampling batch identifier, and acquisition status identifier of the original fusion time window into the re-sampling data. After the re-sampling is completed, the interference processing parameters are re-determined according to the sampling characteristics of the target signal and the motion state during the re-sampling period. Adaptive noise reduction, anomaly identification, and missing data repair are performed to generate the effective re-sampling signal and the quality characterization of the re-sampling signal. Subsequently, a lightweight feature operator matched to the monitoring task is invoked to extract low-dimensional features that can verify the current state conflict. For example, rhythm stability is extracted from the re-sampling ECG signal, pulse wave stability and blood oxygenation trend are extracted from the re-sampling optical physiological signal, and the percentage of stationary states and attitude change are extracted from the re-sampling inertial signal.

[0074] Based on the time range and signal source of the supplementary features, the device controller writes the supplementary features into the corresponding set of features to be fused. If a low-confidence feature exists at the original location, the supplementary feature replaces the original feature, and the signal quality representation and contribution weight are updated; if the original location is a missing identifier, the supplementary feature is filled in and the corresponding missing state is removed; if the supplementary quality still does not meet the participation conditions, the original missing identifier is retained and the reason for the supplementary failure is recorded. After the update is completed, the controller redetermines the target fusion strategy, performs association combination on the updated set of features to be fused, generates a new multimodal physiological fusion representation, and matches the state determination conditions again.

[0075] Finally, the device controller compares the consistency of state categories, state confidence levels, and different physiological dimensions before and after the reassessment. If the state confidence level after reassessment reaches the confirmation threshold and is higher than before the reassessment, the reassessment result is used as the human state assessment result; if the state category remains unchanged but the confidence level increases, the original state category is retained and the state confidence level is updated; if the state category changes and multiple reassessment features jointly support the new state, the new state category is used; if the state is still below the confirmation threshold after reassessment, a pending confirmation state is output and the anomaly alert level is reduced. Reassessment batches, resource constraints, reassessment features, and differences between pre- and post-assessment data are written into the assessment process data. By using targeted reassessment based on conflict dimensions, the sampling frequency of all sensor units can be avoided, supplementing key evidence while controlling energy consumption and improving the reliability of low-confidence assessment results.

[0076] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.

[0077] Figure 2 This is a schematic diagram of the structure of a multimodal physiological signal fusion system based on wearable devices provided in an embodiment of this application. Figure 2 As shown, the system includes: The acquisition module 201 is used to acquire multi-source physiological signals representing different physiological dimensions of the human body and auxiliary signals representing the human body's motion state through wearable devices, and divide each signal into the corresponding fusion time window according to a unified time reference. The generation module 202 is used to determine the corresponding interference processing parameters based on the signal attributes of multi-source physiological signals and the motion state within the fusion time window, and to perform noise reduction, anomaly identification and missing repair on multi-source physiological signals to generate effective physiological signals and corresponding signal quality characterization. The extraction module 203 is used to extract low-dimensional physiological features associated with the preset monitoring task from each effective physiological signal and form a set of features to be fused according to the time correspondence. The determination module 204 is used to determine a target fusion strategy from the preset fusion strategies based on the signal quality characterization and the operating resource status of the wearable device. The target fusion strategy is used to determine the participation status and contribution weight of the features to be fused. Combination module 205 is used to perform association combination on the set of features to be fused according to the target fusion strategy to generate a multimodal physiological fusion representation; The output module 206 is used to match the multimodal physiological fusion representation with the state judgment conditions corresponding to the preset monitoring task, output the human state assessment result, and store or transmit the assessment process data and the human state assessment result.

[0078] In some embodiments, Figure 2The acquisition module 201 is used to determine the sampling configuration corresponding to each signal according to the preset monitoring task, and dynamically adjust the sampling configuration according to the operating resource status of the wearable device and the human motion status; drive the corresponding sensing unit to acquire multi-source physiological signals and auxiliary signals according to the adjusted sampling configuration, and write time identifiers and acquisition status identifiers generated based on the device clock to the acquired data; calibrate the time identifiers of the acquired data according to the sampling period and time offset of each sensing unit to generate a signal sequence under a unified time reference; divide the signal sequence into time segments according to the preset window length, and adjust the segment boundaries according to the time continuity of adjacent acquired data; associate the signal segments with corresponding time ranges to the same fusion time window to generate a multimodal time series data group carrying sampling configuration and acquisition status identifiers.

[0079] In some embodiments, Figure 2 The generation module 202 is used to determine the corresponding signal processing constraints based on the sampling characteristics and waveform change features of each multi-source physiological signal; extract the motion intensity and posture change features within the fusion time window from the auxiliary signal to generate motion interference characterization; match the signal processing constraints with the motion interference characterization to determine the filtering parameters and anomaly judgment parameters corresponding to each multi-source physiological signal, and perform adaptive noise reduction on each multi-source physiological signal; identify abnormal data and missing segments based on the continuity, stability, and correlation consistency between different physiological dimensions of the denoised signal; perform time-series compensation on abnormal data and missing segments that meet the repair conditions, configure invalid labels for data that do not meet the repair conditions, and generate valid physiological signals; and generate signal quality characterizations corresponding to each valid physiological signal based on the noise reduction amplitude, data integrity, and correlation consistency.

[0080] In some embodiments, Figure 2 The extraction module 203 is used to construct task feature constraints to characterize the state of the target human body according to the state determination requirements corresponding to the preset monitoring task; extract temporal change features and state characterization features within the fusion time window according to the signal attributes of each effective physiological signal to form candidate physiological features; determine the task correlation between the candidate physiological features and the state of the target human body based on the task feature constraints, and screen the target physiological features based on the redundancy between the candidate physiological features; perform scale unification and dimensional compression on the target physiological features to generate low-dimensional physiological features corresponding to each effective physiological signal; establish a time correspondence relationship based on the fusion time window to which the low-dimensional physiological features belong and the signal source, and encapsulate the time-corresponding low-dimensional physiological features and the corresponding signal quality characterization to form a set of features to be fused.

[0081] In some embodiments, Figure 2The determination module 204 is used to determine the corresponding feature confidence level based on the signal quality characterization associated with each low-dimensional physiological feature; obtain the remaining energy and current computing load of the wearable device to generate resource availability characterizing the fusion processing capability; screen low-dimensional physiological features that meet the participation conditions according to the feature confidence level, and determine the allowed fusion complexity based on the resource availability; match the screened low-dimensional physiological features, the allowed fusion complexity, and the applicable conditions of the preset fusion strategy to obtain candidate fusion strategies; and conduct a comprehensive evaluation based on the task adaptability and resource consumption corresponding to the candidate fusion strategies to determine the target fusion strategy and configure the participation status and contribution weight of each low-dimensional physiological feature.

[0082] In some embodiments, Figure 2 The combination module 205 is used to read the participation status of the target fusion strategy configuration, select low-dimensional physiological features to participate in the fusion from the set of features to be fused, construct cross-modal feature association units according to the temporal correspondence and physiological correlation of each low-dimensional physiological feature, perform weighted combination of each cross-modal feature association unit according to the contribution weight configured by the target fusion strategy, and configure missing labels for the feature positions that do not participate in the fusion, perform scale correction and dimensional compression on the weighted combination results, and generate a multimodal physiological fusion representation that represents the comprehensive physiological state within the fusion time window.

[0083] In some embodiments, Figure 2 The output module 206 is used to invoke the corresponding state judgment conditions according to the preset monitoring task and extract the task judgment parameters from the multimodal physiological fusion representation; match the task judgment parameters with the state judgment conditions to determine the human state category and state confidence level corresponding to the fusion time window; perform temporal consistency verification based on the human state category and state confidence level of the continuous fusion time window to generate human state assessment results carrying the state change trend; associate the multimodal physiological fusion representation, target fusion strategy, state judgment basis and human state assessment results to generate assessment process data; store the assessment process data and human state assessment results locally, and transmit them to external devices when the preset transmission conditions are met.

[0084] In some embodiments, Figure 2The output module 206 is used to screen stable physiological samples whose signal quality representation and state confidence meet the baseline update conditions from the evaluation process data corresponding to the continuous fusion time window; to perform time-series aggregation of stable physiological samples according to the monitoring task to generate an individual physiological baseline representing the range of normal physiological changes and physiological response relationships under different motion states; to calculate the degree of deviation of the current multimodal physiological fusion representation from the individual physiological baseline and use the degree of deviation as the correction basis for the state determination conditions; to incrementally update the individual physiological baseline, state determination conditions and corresponding low-dimensional physiological features contribution weights according to the confirmed human state evaluation results, and to update and isolate the evaluation process data that do not meet the update confidence conditions.

[0085] In some embodiments, Figure 2 The determination module 204 is used to determine the target signal to be supplemented based on the quality characterization of each signal when the state confidence of the human state assessment result is lower than a preset threshold, or when there is a correlation conflict between low-dimensional physiological features corresponding to different physiological dimensions; combined with the operating resource status of the wearable device, determine the supplementation duration and sampling configuration corresponding to the target signal, and control the corresponding sensing unit to perform directional supplementation within a limited time; perform interference processing and low-dimensional feature extraction on the supplemented signal, and write the obtained supplemented features into the corresponding set of features to be fused; regenerate the multimodal physiological fusion characterization based on the updated set of features to be fused, and determine the human state assessment result based on the change in state confidence before and after the reassessment.

[0086] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0087] Figure 3 This is a schematic diagram of the electronic device 3 provided in an embodiment of this application. Figure 3 As shown, the electronic device 3 of this embodiment includes: a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, it implements the steps in the various method embodiments described above. Alternatively, when the processor 301 executes the computer program 303, it implements the functions of each module / unit in the various system embodiments described above.

[0088] Electronic device 3 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 3 may include, but is not limited to, processor 301 and memory 302. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or different components.

[0089] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0090] The memory 302 can be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 302 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 3. The memory 302 can also include both internal and external storage units of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0092] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which may be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0093] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for fusing multimodal physiological signals based on wearable devices, characterized in that, include: Wearable devices are used to collect multi-source physiological signals that characterize different physiological dimensions of the human body and auxiliary signals that characterize the human body's movement state. The signals are then divided into corresponding fusion time windows according to a unified time reference. Based on the signal attributes of multi-source physiological signals and the motion state within the fusion time window, the corresponding interference processing parameters are determined respectively. Noise reduction, anomaly identification and missing repair are performed on the multi-source physiological signals to generate effective physiological signals and corresponding signal quality characterization. Low-dimensional physiological features associated with the preset monitoring task are extracted from each effective physiological signal, and a set of features to be fused is formed according to the time correspondence. Based on the signal quality characterization and the operating resource status of the wearable device, a target fusion strategy is determined from the preset fusion strategies. The target fusion strategy is used to determine the participation status and contribution weight of the features to be fused. According to the target fusion strategy, the set of features to be fused is associated and combined to generate a multimodal physiological fusion representation; The multimodal physiological fusion representation is matched with the state determination conditions corresponding to the preset monitoring task to output the human body state assessment result, and the assessment process data and human body state assessment result are stored or transmitted.

2. The method according to claim 1, characterized in that, The process involves collecting multi-source physiological signals characterizing different physiological dimensions of the human body and auxiliary signals characterizing the human body's movement state through wearable devices, and dividing each signal into a corresponding fusion time window according to a unified time reference, including: The sampling configuration corresponding to each signal is determined according to the preset monitoring task, and the sampling configuration is dynamically adjusted according to the operating resource status of the wearable device and the human motion status. Drive the corresponding sensing unit to collect multi-source physiological signals and auxiliary signals according to the adjusted sampling configuration, and write the time identifier and collection status identifier generated based on the device clock to the collected data. Based on the sampling period and time offset of each sensing unit, the time stamp of the collected data is calibrated to generate a signal sequence under a unified time reference; The signal sequence is segmented in time according to a preset window length, and the segment boundaries are adjusted according to the temporal continuity of adjacent acquired data. By associating corresponding signal segments within the same time range to the same fusion time window, a multimodal time series data set carrying sampling configuration and acquisition status identifiers is generated.

3. The method according to claim 1, characterized in that, Based on the signal attributes of multi-source physiological signals and the motion state within the fusion time window, corresponding interference processing parameters are determined respectively. Noise reduction, anomaly identification, and missing data repair are performed on the multi-source physiological signals to generate effective physiological signals and corresponding signal quality characterizations, including: Based on the sampling characteristics and waveform variation features of each multi-source physiological signal, the corresponding signal processing constraints are determined; Motion intensity and attitude change features within the fusion time window are extracted from the auxiliary signal to generate motion disturbance characterization; The signal processing constraints are matched with the motion disturbance characterization to determine the filtering parameters and anomaly judgment parameters corresponding to each multi-source physiological signal, and adaptive noise reduction is performed on each multi-source physiological signal. Based on the continuity, stability, and consistency of the correlation between different physiological dimensions of the noise-reduced signal, abnormal data and missing segments are identified. Temporal compensation is performed on abnormal data and missing segments that meet the repair conditions, and invalidation is configured for data that does not meet the repair conditions to generate valid physiological signals; Based on the noise reduction magnitude, data completeness, and correlation consistency, signal quality characterizations corresponding to each effective physiological signal are generated.

4. The method according to claim 1, characterized in that, The step of extracting low-dimensional physiological features associated with the preset monitoring task from each effective physiological signal and forming a set of features to be fused based on the time correspondence includes: Based on the state determination requirements corresponding to the preset monitoring tasks, construct task feature constraints to characterize the state of the target human body. Based on the signal attributes of each effective physiological signal, temporal change features and state representation features are extracted within the fusion time window to form candidate physiological features; The task-related degree between candidate physiological features and target human state is determined based on task feature constraints, and the target physiological features are screened based on the redundancy between candidate physiological features. The target physiological features are scaled and compressed to generate low-dimensional physiological features corresponding to each effective physiological signal. A time correspondence is established based on the fusion time window and signal source of the low-dimensional physiological features. The time-corresponding low-dimensional physiological features and the corresponding signal quality characteristics are associated and encapsulated to form a set of features to be fused.

5. The method according to claim 1, characterized in that, The step of determining the target fusion strategy from the preset fusion strategies based on the signal quality characterization and the operating resource status of the wearable device includes: Based on the signal quality characterization associated with each low-dimensional physiological feature, the corresponding feature confidence level is determined; Obtain the remaining energy and current computing load of wearable devices to generate resource availability that characterizes fusion processing capabilities; Low-dimensional physiological features that meet the participation conditions are selected based on feature credibility level, and the allowed fusion complexity is determined based on resource availability. The selected low-dimensional physiological features, the allowed fusion complexity, and the applicable conditions of the preset fusion strategy are matched to obtain candidate fusion strategies. Based on a comprehensive evaluation of the task suitability and resource consumption of the candidate fusion strategies, the target fusion strategy is determined, and the participation status and contribution weight of each low-dimensional physiological feature are configured.

6. The method according to claim 1, characterized in that, The step of performing association and combination on the set of features to be fused according to the target fusion strategy to generate a multimodal physiological fusion representation includes: Read the participation status of the target fusion strategy configuration and filter low-dimensional physiological features to participate in the fusion from the set of features to be fused; Based on the temporal correspondence and physiological correlation of each low-dimensional physiological feature, a cross-modal feature association unit is constructed; According to the contribution weights configured in the target fusion strategy, the cross-modal feature association units are weighted and combined, and missing labels are configured for feature locations that do not participate in the fusion. The weighted combination results are scaled and compressed to generate a multimodal physiological fusion representation of the comprehensive physiological state within the fusion time window.

7. The method according to claim 1, characterized in that, The process of matching the multimodal physiological fusion representation with the state determination conditions corresponding to the preset monitoring task, outputting the human state assessment result, and storing or transmitting the assessment process data and the human state assessment result includes: The corresponding status judgment conditions are invoked according to the preset monitoring task, and the task judgment parameters are extracted from the multimodal physiological fusion representation. The task judgment parameters are matched with the state judgment conditions to determine the human state category and state confidence level corresponding to the fusion time window. Temporal consistency verification is performed based on the human state category and state confidence of the continuous fusion time window to generate human state assessment results carrying the state change trend. The assessment process data is generated by linking multimodal physiological fusion representation, target fusion strategy, state determination criteria, and human state assessment results. The assessment process data and human condition assessment results are stored locally and transmitted to external devices when preset transmission conditions are met.

8. The method according to claim 7, characterized in that, The method also includes: From the evaluation process data corresponding to the continuous fusion time window, stable physiological samples that meet the baseline update conditions in terms of signal quality characterization and state confidence are selected. According to the monitoring task, stable physiological samples are aggregated in time to generate individual physiological baselines that characterize the range of normal physiological changes and the relationship between physiological responses under different exercise states. Calculate the degree of deviation of the current multimodal physiological fusion representation from the individual's physiological baseline, and use the degree of deviation as the basis for correcting the state determination conditions; Based on the confirmed human condition assessment results, the contribution weights of individual physiological baselines, condition determination conditions, and corresponding low-dimensional physiological characteristics are incrementally updated, and assessment process data that do not meet the update confidence conditions are updated and isolated.

9. The method according to claim 1, characterized in that, The method also includes: When the confidence level of the human body status assessment result is lower than the preset threshold, or when there is a correlation conflict between the low-dimensional physiological features corresponding to different physiological dimensions, the target signal to be supplemented is determined based on the quality characterization of each signal. Based on the operating resource status of the wearable device, determine the supplementary sampling duration and sampling configuration corresponding to the target signal, and control the corresponding sensing unit to perform directional supplementary sampling within a limited time. The supplementary sampling signal is subjected to interference processing and low-dimensional feature extraction, and the obtained supplementary sampling features are written into the corresponding feature set to be fused. The multimodal physiological fusion representation is regenerated based on the updated set of features to be fused, and the human state assessment result is determined based on the change in state confidence before and after the reassessment.

10. A multimodal physiological signal fusion system based on wearable devices, characterized in that, include: The acquisition module is used to acquire multi-source physiological signals that characterize different physiological dimensions of the human body and auxiliary signals that characterize the human body's motion state through wearable devices, and divide each signal into the corresponding fusion time window according to a unified time reference. The generation module is used to determine the corresponding interference processing parameters based on the signal attributes of multi-source physiological signals and the motion state within the fusion time window, and to perform noise reduction, anomaly identification and missing repair on multi-source physiological signals to generate effective physiological signals and corresponding signal quality characterization. The extraction module is used to extract low-dimensional physiological features associated with the preset monitoring task from each effective physiological signal and form a set of features to be fused according to the time correspondence. The determination module is used to determine a target fusion strategy from a preset fusion strategy based on the signal quality characterization and the operating resource status of the wearable device. The target fusion strategy is used to determine the participation status and contribution weight of the features to be fused. The combination module is used to perform association combination on the set of features to be fused according to the target fusion strategy to generate a multimodal physiological fusion representation; The output module is used to match the multimodal physiological fusion representation with the state judgment conditions corresponding to the preset monitoring task, output the human state assessment result, and store or transmit the assessment process data and the human state assessment result.

Images