Method and system for monitoring and early warning of multi-modal physiological indexes of power operation personnel

Abstract

The application relates to the technical field of intelligent monitoring, and particularly discloses a power operation personnel multi-modal physiological index monitoring and early warning method and system, the method comprising the following steps: collecting physiological data streams of power operation personnel based on an edge terminal in real time; performing time alignment, filtering pretreatment and feature extraction on the physiological data streams, and constructing a multi-modal feature set; processing the multi-modal feature set based on a preset deep fusion analysis model, the deep fusion analysis model being configured to identify abnormal patterns associated with sudden brain diseases and output a comprehensive risk level; triggering corresponding graded early warning and safety interlocking execution instructions according to the comprehensive risk level; the application obtains multi-modal physiological indexes of workers through a wearable device with multiple built-in sensors, comprehensively identifies the multi-modal physiological indexes, and obtains a more comprehensive state determination result, and the identification speed and robustness are extremely high.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring technology, specifically a method and system for monitoring and early warning of multimodal physiological indicators of power workers. Background Technology

[0002] Power workers, including line inspectors, live-line workers, and substation maintenance personnel, are in special working conditions such as high-voltage electric fields, strong electromagnetic interference, working at heights, and confined spaces for extended periods. In addition to facing traditional physical risks such as electric shock and falls, their physical and mental health, especially the sudden abnormalities of the brain and cardiovascular system, has become a key hidden danger that could trigger major safety accidents.

[0003] Currently, safety measures for power workers mainly focus on environmental monitoring, such as video surveillance, hazardous gas detection, and behavioral recognition technology. However, these existing technologies have significant shortcomings. In terms of physiological monitoring, traditional methods rely excessively on single macroscopic indicators, such as heart rate and blood oxygen saturation, lacking the ability to directly perceive deep changes in brain function. This makes it difficult to capture early, specific electrophysiological and metabolic precursor signals of brain diseases, such as epileptic-like discharges and dramatic changes in localized cerebral hemodynamics, resulting in excessively short warning windows and an inability to provide timely and effective risk responses. At the information integration level, different physiological signal acquisition systems operate independently, failing to achieve precise temporal synchronization and deep fusion analysis of pathophysiological mechanisms. For example, stroke precursors may simultaneously involve abnormal fluctuations in EEG signals, rapid changes in brain oxygenation signals, and specific indicators in ECG signals. Single-signal analysis is highly prone to misjudgment or omission, making it impossible to accurately identify complex risk patterns. To address these issues, existing technologies urgently need improvement. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for monitoring and early warning of multimodal physiological indicators of power workers, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for monitoring and early warning of multimodal physiological indicators of power workers, the method comprising: The data stream is based on real-time acquisition of physiological data from power workers at the edge. The physiological data stream includes direct physiological signals from the brain and co-physiological signals. The direct physiological signals from the brain include electroencephalogram (EEG) signals and functional near-infrared brain oxygenation (FIN) signals. The co-physiological signals include electrocardiogram (ECG) signals and electrodermal conductance (EDC) signals. The physiological data stream is time-aligned, filtered, preprocessed, and its features are extracted to construct a multimodal feature set. The multimodal feature set is processed based on a preset deep fusion analysis model, which is configured to identify abnormal patterns associated with sudden brain diseases and output a comprehensive risk level. Based on the comprehensive risk level, corresponding graded early warning and safety interlock execution instructions are triggered; the safety interlock execution instructions are used to adjust the current high-risk power operation tasks.

[0006] As a further aspect of the present invention: the steps of performing time alignment, filtering preprocessing, and feature extraction on the physiological data stream to construct a multimodal feature set include: Based on the same time axis and time step, various types of physiological data are statistically analyzed to obtain data sequences for each type. When analyzing physiological data, time points are determined on the time axis according to the time step, and the physiological data closest to the time point is queried as data in the data sequence. For each type of data sequence, frequency domain transformation is performed, and the frequency domain transformed data sequence is then filtered and preprocessed. Based on a preset time period, the filtered and preprocessed data sequence is extracted, and feature extraction is performed on the extracted data sequence. The feature extraction process includes: Time-frequency domain features are extracted from EEG signals to obtain the power spectral density of a specific frequency band; Extracting time-domain and frequency-domain features of heart rate and heart rate variability from electrocardiogram signals; Extracting skin conductance levels and response amplitude characteristics from electrodermal signals; Differential identification is performed on the functional near-infrared brain oxygenation signal to obtain a differential sequence. Based on the differential sequence, the concentration changes of oxyhemoglobin and deoxyhemoglobin are determined and calculated.

[0007] As a further aspect of the present invention: the deep fusion analysis model is a multimodal temporal model based on deep learning, specifically a spatiotemporal graph convolutional network or a multimodal Transformer model; the abnormal modes include at least: epileptic seizure prodrome mode, stroke prodrome mode, and acute consciousness disorder mode; the comprehensive risk level is at least divided into four levels: normal, attention, warning, and critical.

[0008] As a further aspect of the present invention: the step of triggering the corresponding graded early warning and safety interlock execution command according to the comprehensive risk level includes: When the risk level is "Caution", a Level 1 alert signal is sent to the terminal device worn by the operator. When the risk level is warning, a second-level alarm is sent to the terminals of the operators and the person in charge of the on-site work, and a work suspension prompt message is generated; When the risk level is critical, the third-level safety interlock process is triggered; the safety interlock process includes: automatically sending a distress message containing precise location information to the emergency center, remotely locking the power supply of preset risk equipment, and activating the on-site audible and visual alarm device.

[0009] As a further aspect of the present invention, the method further includes: Query the multimodal feature set of any power worker under normal conditions, and use it as the baseline feature set; Based on the aforementioned benchmark feature set, the multimodal feature set of power workers is normalized and used as real-time status data; The real-time status data of different power workers are compared periodically to calculate the status differences of the power workers. Based on the abnormal status, the power workers are grouped as the actual group set. Query the initial group set of power workers, compare the actual group set with the initial group set, locate the abnormal actual group, and simultaneously locate the abnormal workers.

[0010] As a further aspect of the present invention, the method further includes: For any electrical worker, query the people in the same group as them in the initial group and generate the first relevant group; Each time a group is formed, query the people in the same group as the power worker in the actual group, and generate a second related group; Calculate the crossover ratio between the first and second correlation groups, and adjust the data acquisition frequency at the edge based on the crossover ratio.

[0011] The present invention also provides a multimodal physiological indicator monitoring and early warning system for power workers, the system comprising: The data stream acquisition module is used to acquire physiological data streams of power workers in real time at the edge. The physiological data streams include direct physiological signals and co-physiological signals from the brain. The direct physiological signals from the brain include electroencephalogram (EEG) signals and functional near-infrared cerebral oxygenation (FIN) signals. The co-physiological signals include electrocardiogram (ECG) signals and electrodermal conductance (EDC) signals. The data feature extraction module is used to perform time alignment, filtering preprocessing, and feature extraction on the physiological data stream to construct a multimodal feature set; The feature set processing module is used to process multimodal feature sets based on a preset deep fusion analysis model, wherein the deep fusion analysis model is configured to identify abnormal patterns associated with sudden brain diseases and output a comprehensive risk level. The execution instruction generation module is used to trigger corresponding graded early warning and safety interlock execution instructions based on the comprehensive risk level; the safety interlock execution instructions are used to adjust the current high-risk power operation tasks.

[0012] As a further aspect of the present invention: the data feature extraction module includes: The data sequence acquisition unit is used to statistically analyze various types of physiological data based on the same time axis and time step to obtain a data sequence for each type. Specifically, when analyzing physiological data, a time point is determined on the time axis according to the time step, and the physiological data closest to the time point is queried as the data in the data sequence. The frequency domain processing unit is used to perform frequency domain transformation on each type of data sequence and to perform filtering preprocessing on the frequency domain transformed data sequence; The extraction execution unit is used to extract the filtered and preprocessed data sequence based on a preset time period and perform feature extraction on the extracted data sequence. The feature extraction process includes: Time-frequency domain features are extracted from EEG signals to obtain the power spectral density of a specific frequency band; Extracting time-domain and frequency-domain features of heart rate and heart rate variability from electrocardiogram signals; Extracting skin conductance levels and response amplitude characteristics from electrodermal signals; Differential identification is performed on the functional near-infrared brain oxygenation signal to obtain a differential sequence. Based on the differential sequence, the concentration changes of oxyhemoglobin and deoxyhemoglobin are determined and calculated.

[0013] As a further aspect of the present invention: the deep fusion analysis model is a multimodal temporal model based on deep learning, specifically a spatiotemporal graph convolutional network or a multimodal Transformer model; the abnormal modes include at least: epileptic seizure prodrome mode, stroke prodrome mode, and acute consciousness disorder mode; the comprehensive risk level is at least divided into four levels: normal, attention, warning, and critical.

[0014] As a further aspect of the present invention: the execution instruction generation module includes: The alert unit is used to send a first-level alert signal to the terminal device worn by the operator when the risk level is "caution"; The warning unit is used to send a second-level alarm to the terminals of the operators and the person in charge of the on-site work when the risk level is warning, and to generate a work suspension prompt message; The process control unit is used to trigger the third-level safety interlock process when the risk level is critical; the safety interlock process includes: automatically sending a distress message containing precise location to the emergency center, remotely locking the power supply of preset risk equipment, and activating the on-site audible and visual alarm device.

[0015] Compared with the prior art, the beneficial effects of the present invention are: the present invention acquires the multimodal physiological indicators of workers through wearable devices with multiple built-in sensors, comprehensively identifies the multimodal physiological indicators, and obtains a more comprehensive state judgment result with extremely high recognition speed and robustness. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0017] Figure 1 A flowchart for a multimodal physiological indicator monitoring and early warning method for power workers.

[0018] Figure 2 This is a block diagram of the composition and structure of a multimodal physiological indicator monitoring and early warning system for power workers. Detailed Implementation

[0019] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0020] Figure 1 This is a flowchart illustrating a method for monitoring and issuing early warning of multimodal physiological indicators in power workers. In this embodiment of the invention, a method for monitoring and issuing early warning of multimodal physiological indicators in power workers includes: Step S100: Real-time acquisition of physiological data streams of power workers at the edge terminal; the physiological data streams include direct physiological signals and co-physiological signals of the brain, the direct physiological signals of the brain include electroencephalogram (EEG) signals and functional near-infrared cerebral oxygenation (FNI) signals, and the co-physiological signals include electrocardiogram (ECG) signals and electrodermal conductance (EDC) signals; The edge device refers to an intelligent device capable of acquiring physiological data from power workers. It possesses at least data acquisition, processing, and data transmission capabilities. For example, an electrode cap worn on the worker's head uses conductive gel to contact the scalp and convert changes in the cerebral cortex's electrical potential into electrical signals. Functional near-infrared brain oxygenation (FNI) signal acquisition can utilize wearable devices integrating near-infrared light sources and detectors, placed on the worker's forehead to calculate blood oxygen concentration by measuring the absorption and scattering changes of different wavelengths of near-infrared light in brain tissue. Electrocardiogram (ECG) signal acquisition can employ chest strap or patch-type ECG sensors, fixed to the worker's chest or limbs, to record the heart's electrical activity in real time. Skin conductance signal acquisition can utilize skin conductivity sensors worn on the worker's fingers or wrists to reflect sweat gland activity by measuring changes in skin surface resistance. Through the coordinated operation of these various sensors, physiological information covering multiple dimensions, including the brain, cardiovascular system, and autonomic nervous system, can be acquired, forming a comprehensive multimodal physiological data stream. Acquiring this data is routine under current technological conditions and will not be elaborated upon further.

[0021] Regarding the issue of obtaining the aforementioned signals, the purpose of obtaining the signals is to determine the working status of the operators. It does not involve the user's identity. Furthermore, in practical applications, it is necessary to explicitly obtain the information collection permissions granted by the user in order to collect signals and perform subsequent processing. For the technical solution of this invention, it is assumed that the information collection permissions granted by the user are already in place.

[0022] Step S200: Perform time alignment, filtering preprocessing, and feature extraction on the physiological data stream to construct a multimodal feature set; After acquiring the physiological data stream, since there are many types of physiological data and different acquisition methods, and combined with the fluctuations in the transmission process, these data may differ in the time domain. Therefore, time alignment is required first. Time alignment can be achieved by attaching a precise timestamp to each sensor data packet and performing interpolation or resampling operations on the edge computing node based on these timestamps to ensure that the data points of different modalities correspond precisely on the time axis.

[0023] After time alignment, the resulting information is very regular in the time domain. At this point, filtering is performed to remove some invalid signals, making the time domain data cleaner. The filtering preprocessing can use digital filters to process the original signal. For example, by setting a specific cutoff frequency, noise components such as power frequency interference and motion artifacts can be filtered out to improve the purity of the signal.

[0024] Finally, the processed physiological data stream is simplified using a feature extraction process. For each type of physiological data, after time alignment and filtering, a time-domain sequence is obtained. Feature extraction from this time-domain sequence yields one or more numerical values ​​that characterize the state of the sequence, thus simplifying it into a low-dimensional array. For example, preliminary feature extraction may include calculating the root mean square or peak value of EEG signals within different time windows, calculating the average R-wave interval (RR interval) of ECG signals, calculating the average level of skin conductance of ductus skinis signals, and calculating the average rate of change of the original optical density signal of functional near-infrared brain oxygenation signals. These preliminary features can summarize the basic characteristics of the original signal with low computational complexity, providing a foundation for subsequent in-depth analysis. Each type of physiological data corresponds to a feature set, and feature sets corresponding to multiple physiological data are called multimodal feature sets.

[0025] Step S300: Process the multimodal feature set based on a preset deep fusion analysis model, wherein the deep fusion analysis model is configured to identify abnormal patterns associated with sudden brain diseases and output a comprehensive risk level; The aforementioned multimodal feature set is transmitted to a cloud or local central processing unit and processed by a pre-defined deep fusion analysis model. This deep fusion analysis model is configured to identify abnormal patterns associated with sudden-onset brain diseases and output a comprehensive risk level. For example, the deep fusion analysis model can be an architecture based on a recurrent neural network (RNN) or convolutional neural network (CNN), trained to learn complex temporal dependencies and spatial correlations from the synchronized multimodal feature set. By learning from a large amount of normal and abnormal physiological data, the model can automatically extract specific physiological patterns associated with sudden-onset brain diseases such as epileptic seizures, stroke, or acute loss of consciousness. When the model detects these abnormal patterns, it outputs a quantified comprehensive risk level based on their intensity and duration. For example, it can be simply divided into two levels: "normal" and "abnormal," or more precisely, into "low risk," "medium risk," and "high risk."

[0026] Step S400: Trigger the corresponding graded early warning and safety interlock execution command according to the comprehensive risk level; the safety interlock execution command is used to adjust the current high-risk power operation task.

[0027] Based on the aforementioned comprehensive risk level, corresponding graded early warning and safety interlocking actions are triggered. These actions are used to intervene in or terminate the current high-risk power operation task, completing a closed-loop transmission from physiological state perception to safety control. Specifically, when the comprehensive risk level reaches a preset "abnormal" or "high-risk" threshold, a simple audible and visual alarm can be triggered to alert the worker and those around them. Simultaneously, a vibration alert can be sent to the smartwatch or wristband worn by the worker. In certain situations, such as when the risk level indicates that the worker may lose consciousness or mobility, an emergency notification can be sent to the site supervisor or dispatch center, suggesting manual intervention measures, such as calling the worker via walkie-talkie or dispatching other personnel to the site. These actions aim to respond promptly to physiological abnormalities in workers, thereby reducing the likelihood of accidents.

[0028] As a preferred embodiment of the technical solution of the present invention, compared with the traditional method of monitoring based on only a single physiological indicator (such as heart rate), the present invention constructs a comprehensive multimodal physiological data stream by real-time acquisition of direct brain physiological signals, including at least electroencephalogram (EEG) signals and functional near-infrared brain oxygenation signals, as well as synergistic physiological signals, including at least electrocardiogram (ECG) signals and skin conductance signals. This multi-dimensional data acquisition method can more comprehensively and deeply reflect the physiological state of the operator, especially in identifying early and hidden precursors of sudden brain diseases, which has an unparalleled advantage over single signal monitoring methods. For example, in the above example, if only heart rate is monitored, it may not be possible to detect abnormal slow waves or slight decreases in brain oxygen saturation in user A's EEG signal in time, thus missing the opportunity for early warning. Specifically, the EEG signal is used to detect epileptiform discharges, sudden slow waves, and fatigue EEG characteristic waves.

[0029] Specifically, electroencephalography (EEG) is a non-invasive method for recording the electrical activity of neurons in the brain, reflecting the physiological and pathological state of the cerebral cortex. Its role is to directly capture electrophysiological abnormalities in the brain, providing crucial information for early warning of brain diseases. Specifically, multiple electrodes can be placed on the scalp to collect potential differences in different brain regions, and methods such as Fourier transform can be used to analyze their frequency components and waveform characteristics to identify specific pathological waveforms, such as sharp waves, spikes, and slow waves. Furthermore, high-density EEG systems can be used to increase the number of electrodes to improve spatial resolution, combined with source localization algorithms, to more accurately identify focal epileptiform discharges or slow-wave activity in specific brain regions.

[0030] Functional near-infrared cerebral oxygenation (fNIRS) is used to monitor blood oxygen saturation in the prefrontal cortex in real time to identify acute cerebral hypoxia or abrupt changes in cerebral hemodynamics. FNIRS is an optical imaging technique that non-invasively monitors cortical hemodynamics and oxygen metabolism by measuring changes in the absorption and scattering of near-infrared light in biological tissues. Its role is to provide real-time information on local blood oxygen supply and metabolic status in the brain, which is crucial for identifying ischemic brain injury or abnormal blood flow. Specifically, by wearing a sensor integrating a near-infrared light source and detector in the prefrontal cortex of the worker, the concentration changes of oxyhemoglobin and deoxyhemoglobin can be calculated using the Lambert-Beer law to infer local blood oxygen saturation. Alternatively, multi-channel fNIRS devices covering multiple areas of the prefrontal cortex can be used to obtain a more comprehensive dynamic map of local blood oxygenation, thereby more accurately identifying abrupt drops or abnormal fluctuations in blood oxygen saturation.

[0031] Electrocardiogram (ECG) signals are used to analyze heart rate variability and identify arrhythmias that may lead to cardioembolic stroke. ECG signals are physiological signals that record changes in the heart's electrical activity, reflecting the heart's rhythm, conduction, and excitability. Their role is to provide indirect information about cardiac function, particularly regarding potential cardiogenic risks associated with brain diseases such as cerebral embolism. Specifically, ECG waveforms can be acquired in real time using a chest strap or patch-type ECG sensor, and RR interval analysis algorithms can be used to calculate various time-domain and frequency-domain indices of heart rate variability, assessing the autonomic nervous system's regulatory function on the heart. Furthermore, continuous dynamic ECG monitoring can record ECG data over extended periods, and combined with automated arrhythmia detection algorithms, can identify arrhythmias such as atrial fibrillation and supraventricular tachycardia that may lead to thrombosis and cerebral embolism.

[0032] Electrodermal signals (EDS) are used to assess sympathetic nervous system excitability and sudden stress states. EDS are changes in skin conductance or resistance, primarily caused by sweat gland activity and regulated by the sympathetic nervous system. Their function is to reflect an individual's physiological arousal level and emotional stress state, providing objective evidence for assessing the psychophysiological load and sudden stress response of workers. Specifically, a pair of electrodes can be placed on the skin surface of the fingers or wrist to measure skin conductance levels and the amplitude and frequency of skin conductance response (SCR). These indicators directly reflect the intensity of sympathetic nervous system activity. Furthermore, high-sensitivity EDS sensors, combined with baseline drift correction and artifact removal algorithms, can more accurately capture minute changes in skin conductance, thereby providing a more detailed assessment of workers' stress responses and emotional fluctuations.

[0033] Regarding step S200, the steps of performing time alignment, filtering preprocessing, and feature extraction on the physiological data stream to construct a multimodal feature set include: Based on the same time axis and time step, various types of physiological data are statistically analyzed to obtain data sequences for each type. When analyzing physiological data, time points are determined on the time axis according to the time step, and the physiological data closest to the time point is queried as data in the data sequence. For each type of data sequence, frequency domain transformation is performed, and the frequency domain transformed data sequence is then filtered and preprocessed. The filtered and preprocessed data sequence is truncated based on a preset time period, and features are extracted from the truncated data sequence.

[0034] The above content provides a detailed explanation of the time alignment and filtering preprocessing. First, staff members uniformly determine a time axis and time step. The time step is used to determine a time point at regular intervals. For any type of physiological data, the corresponding value is queried at each time point to obtain the data sequence for each type. Since there is a difference between the time step for determining the time point and the acquisition period of the physiological data, if the time step is large, not every acquired physiological data will become data in the data sequence; only the physiological data closest to the time point can be included in the data sequence. If the acquisition period is large, there will be less physiological data, and in this case, the data sequence may contain a large amount of duplicate data.

[0035] The feature extraction process includes: Time-frequency domain features are extracted from EEG signals to obtain the power spectral density of a specific frequency band; Extracting time-domain and frequency-domain features of heart rate and heart rate variability from electrocardiogram signals; Extracting skin conductance levels and response amplitude characteristics from electrodermal signals; Differential identification is performed on the functional near-infrared brain oxygenation signal to obtain a differential sequence. Based on the differential sequence, the concentration changes of oxyhemoglobin and deoxyhemoglobin are determined and calculated.

[0036] The above content explains the feature extraction process. Combined with the aforementioned related content, it can be understood that feature extraction involves extracting important parameters from a time series as feature parameters. Different signals have different characteristics, and different feature extraction methods are used. Time-frequency domain feature extraction of EEG signals obtains the power spectral density of specific frequency bands. This aims to reveal the dynamic changes of brain electrical activity at different frequencies and times. Time-frequency domain feature extraction can capture the instantaneous frequency components and energy distribution of EEG signals. For example, through short-time Fourier transform or wavelet transform, EEG signals can be decomposed into multiple frequency bands, and the power spectral density of each band can be calculated, thereby quantifying the energy intensity of a specific frequency band. This helps identify specific EEG rhythm abnormalities associated with epileptiform discharges, sudden slow waves, or fatigue.

[0037] Extracting time-domain and frequency-domain features of heart rate and heart rate variability from electrocardiogram (ECG) signals is crucial for comprehensively assessing the autonomic nervous system's regulatory function and potential arrhythmia risks. Heart rate is a fundamental indicator of cardiac function, while heart rate variability reflects the regularity of minute fluctuations in the heartbeat cycle and is an important indicator of autonomic nervous system balance. Time-domain features can include the root mean square of the difference between adjacent RR intervals and the standard deviation of all normal RR intervals; frequency-domain features can include low-frequency power, high-frequency power, and their ratios. These features effectively indicate the activity state of the sympathetic and parasympathetic nervous systems, thereby identifying arrhythmias or stress responses that may trigger cardioembolic stroke.

[0038] This study extracts skin conductance levels and response amplitude characteristics from electrodermal signals (EDS). Specifically, it aims to quantify the sympathetic nervous system excitation level and sudden stress response of power workers. EDS, or skin conductance changes, is closely related to sweat gland activity and is directly regulated by the sympathetic nervous system. Skin conductance levels reflect sympathetic tone at baseline, while the amplitude of the skin conductance response represents the instantaneous intensity of the response to a specific stimulus or stress event. By extracting these characteristics, it is possible to effectively assess the psychological load, anxiety, or sudden stress state of workers.

[0039] The concentration trends of oxyhemoglobin and deoxyhemoglobin were calculated based on the functional near-infrared brain oxygenation signal. Specifically, this calculation was performed to monitor the local blood oxygen metabolism status of the brain in real time. Functional near-infrared spectroscopy, by measuring changes in the absorption of near-infrared light in brain tissue, can non-invasively calculate the relative concentration changes of oxyhemoglobin and deoxyhemoglobin. These concentration trends directly reflect changes in local cerebral blood flow and oxygen consumption, and are key indicators for identifying acute cerebral hypoxia, abrupt changes in cerebral hemodynamics, or precursors to stroke.

[0040] The technical solution of this invention provides the aforementioned specific feature extraction method, ensuring that comprehensive and representative physiological information is obtained from the multimodal physiological data stream. At the edge, after time alignment and filtering preprocessing of the direct physiological signals (such as EEG signals and functional near-infrared brain oxygenation signals) and co-physiological signals (such as ECG signals and skin conductance signals) of the real-time acquired power workers' brains, these preliminary feature extractions are immediately performed. For example, for EEG signals, power spectral density in key frequency bands such as Delta, Theta, Alpha, and Beta is obtained through time-frequency analysis to capture the rhythmic changes of the brain; for ECG signals, time-domain and frequency-domain features such as heart rate, SDNN, RMSSD, and LF / HF are extracted through R-wave detection and RR interval analysis to assess cardiac autonomic nerve function; for skin conductance signals, skin conductance levels and skin conductance response amplitudes are separated and quantified to reflect the excitation state of the sympathetic nervous system; and for functional near-infrared brain oxygenation signals, the concentration change trends of oxyhemoglobin and deoxyhemoglobin are calculated in real time to monitor cerebral blood oxygen supply. These precisely extracted features together constitute a synchronized multimodal feature set, providing high-quality, high-dimensional input for subsequent deep fusion analysis models in the cloud or local central processing unit, thereby enabling more accurate and timely identification of abnormal patterns associated with sudden brain diseases.

[0041] As a preferred embodiment of the technical solution of the present invention, the deep fusion analysis model is a multimodal temporal model based on deep learning, specifically a spatiotemporal graph convolutional network or a multimodal Transformer model; the abnormal modes include at least: epileptic seizure prodrome mode, stroke prodrome mode, and acute consciousness disorder mode; the comprehensive risk level is divided into at least four levels: normal, attention, warning, and critical.

[0042] Deep fusion analysis models aim to process time-series data from different sensors and learn complex patterns and correlations. Their core lies in automatically extracting high-level features from raw data and fusing information from different modalities to capture deep semantics and dynamic changes within the data. One approach is to use spatiotemporal graph convolutional networks (SPCNNs). These networks abstract multimodal physiological data into a graph structure, where different physiological signals (such as EEG and ECG signals) can be considered as nodes in the graph. Graph convolution operations capture the spatial correlations between different signals, while combining this with recurrent neural networks (RNNs) or convolutional neural networks to process time-series data, thus effectively capturing spatiotemporal dynamic features. Another approach is to use multimodal Transformer models. These models utilize self-attention mechanisms to process time-series data from different modalities in parallel, capturing long-range dependencies and complex interactions between modalities. They are particularly adept at handling variable-length sequences and discovering key time points or feature combinations.

[0043] These abnormal patterns represent potential precursors or acute attacks of brain diseases that may occur suddenly during power operations, posing a serious threat to the life and work safety of workers. Identifying these patterns is crucial for early warning and intervention. "Epileptic seizure prodromal patterns" refer to specific electrophysiological or autonomic nervous system changes that may occur in EEG signals, ECG signals, etc., before an epileptic seizure, such as abnormal discharges of brain waves and decreased heart rate variability.

[0044] The "stroke precursor mode" refers to the syndrome of warning signs that may appear before insufficient blood supply to the brain or before a hemorrhage, including a sharp drop in localized cerebral oxygen saturation, focal slow-wave activity, and arrhythmias, as indicated by functional near-infrared cerebral oxygenation signals, electroencephalogram (EEG) signals, and electrocardiogram (ECG) signals. The "acute loss of consciousness mode" refers to a sudden decline in the level of consciousness of workers due to various causes (such as hypoxia, hypoglycemia, or brain lesions), characterized by diffuse slow-wave activity in EEG and stress responses in skin conductance signals. This grading mechanism quantifies the current physiological risk status of workers and provides a basis for decision-making regarding subsequent early warning and intervention measures. Detailed risk levels help the system respond to different levels of situations based on the actual circumstances, avoiding over-intervention or under-intervention. The "normal" level indicates that the worker's physiological state is stable with no obvious abnormalities. The "attention" level indicates the detection of minor abnormalities or potential risk factors, requiring attention. The "warning" level indicates the detection of moderate abnormalities or an increasing risk, potentially requiring suspension of work; the "critical" level indicates the detection of severe abnormalities or warning signs of an impending disease attack, requiring immediate mandatory intervention.

[0045] It should be noted that the above identification process is a model application process. Its purpose is not to directly diagnose the staff or judge their health status. It does not involve a health judgment process. It only judges whether the data meets a certain standard and then generates prompt information. It mainly reflects the staff's current physical condition. Its results cannot be used as a standard for assessing health status.

[0046] Furthermore, regarding the model itself, the deep fusion analysis model can be a spatiotemporal graph convolutional network or a multimodal Transformer model, used for deep processing of these feature sets. These models belong to the existing modeling category and, with their powerful nonlinear modeling capabilities and ability to capture complex correlations between temporal and multimodal data, can learn from massive amounts of physiological data and identify often hidden and complex abnormal patterns associated with sudden brain diseases (such as epileptic seizures, stroke, and acute consciousness disorders). For example, a spatiotemporal graph convolutional network can simultaneously consider the spatial interaction of different physiological signals and their dynamic changes over time, while a multimodal Transformer model can effectively fuse different modal information through a self-attention mechanism and capture long-distance temporal dependencies, thereby more accurately identifying epileptic seizure aura patterns, stroke aura patterns, and acute consciousness disorder patterns. Once these abnormal patterns are identified, the model will output a comprehensive risk level based on their severity and confidence level, which is at least subdivided into four levels: normal, attention, warning, and critical.

[0047] The grading result is the output of the model, which triggers more accurate and appropriate grading warnings and safety interlocking actions in subsequent steps. This avoids the problems of misjudgment, missed judgment and response lag caused by insufficient model generalization ability, vague definition of abnormal mode or rough risk level classification in traditional methods. It realizes more intelligent and refined decision-making in the closed-loop transmission from physiological state perception to safety control.

[0048] Regarding step S400, the step of triggering the corresponding graded early warning and safety interlock execution command according to the comprehensive risk level includes: When the risk level is "Caution", a Level 1 alert signal is sent to the terminal device worn by the operator. When the risk level is warning, a second-level alarm is sent to the terminals of the operators and the person in charge of the on-site work, and a work suspension prompt message is generated; When the risk level is critical, the third-level safety interlock process is triggered; the safety interlock process includes: automatically sending a distress message containing precise location information to the emergency center, remotely locking the power supply of preset risk equipment, and activating the on-site audible and visual alarm device.

[0049] A risk level of "Caution" indicates that, after assessment by a deep fusion analysis model, the worker's physiological state exhibits minor abnormalities, but has not yet reached a level that immediately endangers life or work safety. Examples include signs of fatigue and mild stress responses. The terminal device worn by the worker can be a wearable device such as a smartwatch, smart bracelet, head-mounted display, or a communication module integrated into the safety helmet. This device typically has information receiving, display, and / or vibration alert functions. The first-level alert signal is a non-mandatory, low-intensity reminder designed to draw the worker's attention. For example, it can be achieved through a slight vibration, a low-volume alert sound, or by displaying text information on the screen (such as "Please take a rest" or "Slight fluctuations in physiological indicators").

[0050] A "Warning" risk level indicates that the worker's physiological state has become moderately abnormal, potentially indicating underlying health or operational risks. This requires attention and intervention. Examples include persistent abnormal heart rate variability, moderate stress in the skin conductance signal, or early fatigue waveforms in the electroencephalogram (EEG). The on-site supervisor's terminal can be a smartphone, tablet, walkie-talkie, or dedicated dispatch terminal. This terminal typically has the ability to receive alarm information and display the worker's status and location. A Level 2 alarm is a moderate-intensity alarm designed to alert the worker and their supervisor to the risk and take action. For example, the terminal device emits a continuous vibration, a moderate-volume alarm sound, and displays a clearer warning message on the screen (e.g., "Abnormal physiological indicators, work suspension recommended"). Simultaneously, the on-site supervisor's terminal also receives information including the worker's identity, location, and risk type. "Work suspension recommended" means the system issues a clear suggestion, but the final decision rests with the worker or on-site supervisor. This can be conveyed through a text prompt on the terminal device or a voice announcement.

[0051] A risk level of "critical" indicates that the worker's physiological state has reached a severely abnormal state, and a sudden brain disease or physiological loss of control that may be imminent or ongoing and could endanger life or lead to a major accident is likely to occur. For example, a deep fusion analysis model may identify patterns of epileptic seizure aura, stroke precursors, or acute loss of consciousness. Level 3 safety interlocking is the highest level of mandatory safety intervention, designed to immediately prevent the occurrence or escalation of danger and initiate emergency rescue. It automatically sends a distress message containing precise location information to the emergency center. This can be achieved by obtaining the worker's precise location information through the terminal device's built-in GPS, BeiDou Navigation Satellite System, or base station positioning technology, and then sending the distress message, containing location, personnel identity, risk type, and timestamp, to the pre-set emergency response center or dispatch center via cellular network, satellite communication, or dedicated wireless network. Remotely locking or cutting off the power supply to associated high-risk tools and equipment can be achieved through integration with intelligent tool and equipment management systems or remote control systems at the power work site. For example, when a critical situation is detected, the system can send commands to high-risk tools and equipment such as high-voltage detectors, insulating rods, and power tools, causing them to automatically enter a locked state and become inoperable, or directly cut off their power supply to prevent accidental operation. Activating on-site audible and visual alarm devices can be done via wireless or wired connections to trigger audible and visual alarms near the work site. For example, warning lights and horns installed near substations or aerial work platforms can be activated, emitting high-decibel alarm sounds and flashing warning lights to alert surrounding personnel to the danger and facilitate rescue efforts.

[0052] As a preferred embodiment of the technical solution of the present invention, the method further includes: Query the multimodal feature set of any power worker under normal conditions, and use it as the baseline feature set; Based on the aforementioned benchmark feature set, the multimodal feature set of power workers is normalized and used as real-time status data; The real-time status data of different power workers are compared periodically to calculate the status differences of the power workers. Based on the abnormal status, the power workers are grouped as the actual group set. Query the initial group set of power workers, compare the actual group set with the initial group set, locate the abnormal actual group, and simultaneously locate the abnormal workers.

[0053] In one example of the technical solution of this invention, an extended technical solution based on the original solution is introduced. For any power worker, its multimodal feature set under normal conditions is queried. The normal conditions are the normal working conditions, the period during which no prompt information appears. The multimodal feature set under normal conditions is used as the baseline feature set. Using the elements in the baseline feature set as a benchmark, the multimodal feature set of the power worker is normalized to serve as real-time status data. The simplest normalization method is to directly calculate the ratio, thus making the real-time status data dimensionless. In addition, other normalization processing schemes based on a certain value can be applied to make the data in the multimodal feature set dimensionless, so that all the data are numerical, which is convenient for subsequent processing. Furthermore, the real-time status data of different power workers are compared periodically to calculate the status differences of the power workers. The multimodal feature set is a set of specific numerical values, which can be categorized into some specific types. For matrices with special row and column structures, a matrix comparison scheme can be used for the comparison process. The resulting matrix differences represent the state differences of the power workers. One feasible approach is to sequentially read the corresponding two values ​​from the multi-model feature sets of two power workers, calculate the difference between the two values, and divide the difference by the larger of the two values ​​to obtain a ratio. This ratio reflects the difference between the two power workers on a certain feature. The maximum value of all differences (or the mode, mean, etc.) is selected as the state difference of the power workers. The state difference describes the differences among the power workers and can be understood as distance. Existing clustering algorithms can be used to group the power workers, obtaining the grouping results at each time point (the time of timed comparison), which is called the actual group set. The clustering algorithm can be the K-means algorithm, where K is a pre-set value. Of course, if computing power allows, a clustering algorithm with an unlimited number of classes is also feasible.

[0054] Meanwhile, power workers themselves have an initial group set. The initial group set is grouped according to work type and work scenario. Workers in the same work type and work scenario should have similar states, so they are grouped together. By comparing the actual group set with the initial group set, the combination where the location changes is called the abnormal actual group, and the abnormal workers are located simultaneously. The comparison process only needs to calculate the intersection. Each worker has a unique label, and the number of these labels is extremely limited. If there is an actual group that is different from all the initial groups, then it is an abnormal actual group. The closest initial group is queried, and the different personnel are the abnormal workers. The response speed of the above process is actually faster. It is equivalent to introducing a new identification process after step S200. It does not use a higher-performance model. It only needs to perform simple data comparison and clustering to quickly locate abnormal personnel. It can be used to adjust the identification order of the multimodal feature set in step S300 (corresponding to different workers). For example, for abnormal personnel identified by the above content, the model first identifies the multimodal feature set of the abnormal personnel, making the identification process more orderly.

[0055] It should be noted that there is another reason for using a baseline feature set to normalize the multimodal feature set of power workers: different power workers have different physical conditions, and the resulting multimodal feature sets are inherently different. Normalizing them using their own baseline feature set can largely eliminate the influence of different physical conditions, making subsequent comparisons more meaningful. This normalization process is actually a necessary technical solution.

[0056] As a preferred embodiment of the technical solution of the present invention, the method further includes: For any electrical worker, query the people in the same group as them in the initial group and generate the first relevant group; Each time a group is formed, query the people in the same group as the power worker in the actual group, and generate a second related group; Calculate the crossover ratio between the first and second correlation groups, and adjust the data acquisition frequency at the edge based on the crossover ratio.

[0057] In one example of the technical solution of this invention, an extended scheme regarding battery life is introduced. In the original scheme, the data acquisition frequency at the edge of each power worker is a default static frequency. Generally, different frequencies are used for different types of signals. A basic static frequency can be set first, and a scaling factor can be set for different types. Multiplying the scaling factor by the static frequency yields the data acquisition frequency for different types. For any power worker, the first relevant group is generated by querying the members in the initial group. Then, during timed grouping, the second relevant group is generated by querying the members in the actual group for each power worker. At this point, the data acquisition frequency for each power worker can be obtained. The system checks whether the personnel in the same group have changed (related groups refer to personnel in the same group at different times), calculates the intersection and union ratio of the first and second related groups. The intersection and union ratio reflects the stability of the working status of the power workers. The data acquisition frequency at the edge is adjusted according to the intersection and union ratio. If there is no change, the first and second related groups are the same, with the same intersection and union, and the ratio is 1. If there is a change, the union will increase and the intersection will decrease. At this time, the intersection and union ratio decreases, and the data acquisition frequency must increase. Therefore, the data acquisition frequency is inversely proportional to the intersection and union ratio. In addition, the data acquisition frequency will also be set with a maximum value and a minimum value so that the data acquisition frequency takes a value within a certain range.

[0058] Figure 2 This is a block diagram illustrating the composition of a multimodal physiological indicator monitoring and early warning system for power workers. In this embodiment of the invention, a multimodal physiological indicator monitoring and early warning system for power workers, 10, includes: The data stream acquisition module 11 is used to acquire the physiological data stream of power workers in real time based on the edge terminal; the physiological data stream includes direct physiological signals of the brain and co-physiological signals, the direct physiological signals of the brain include electroencephalogram (EEG) signals and functional near-infrared brain oxygenation signals, and the co-physiological signals include electrocardiogram (ECG) signals and skin conductance signals; The data feature extraction module 12 is used to perform time alignment, filtering preprocessing and feature extraction on the physiological data stream to construct a multimodal feature set; The feature set processing module 13 is used to process the multimodal feature set based on a preset deep fusion analysis model, wherein the deep fusion analysis model is configured to identify abnormal patterns associated with sudden brain diseases and output a comprehensive risk level. The execution instruction generation module 14 is used to trigger corresponding graded early warning and safety interlock execution instructions based on the comprehensive risk level; the safety interlock execution instructions are used to adjust the current high-risk power operation tasks.

[0059] Furthermore, the data feature extraction module 12 includes: The data sequence acquisition unit is used to statistically analyze various types of physiological data based on the same time axis and time step to obtain a data sequence for each type. Specifically, when analyzing physiological data, a time point is determined on the time axis according to the time step, and the physiological data closest to the time point is queried as the data in the data sequence. The frequency domain processing unit is used to perform frequency domain transformation on each type of data sequence and to perform filtering preprocessing on the frequency domain transformed data sequence; The extraction execution unit is used to extract the filtered and preprocessed data sequence based on a preset time period and perform feature extraction on the extracted data sequence. The feature extraction process includes: Time-frequency domain features are extracted from EEG signals to obtain the power spectral density of a specific frequency band; Extracting time-domain and frequency-domain features of heart rate and heart rate variability from electrocardiogram signals; Extracting skin conductance levels and response amplitude characteristics from electrodermal signals; Differential identification is performed on the functional near-infrared brain oxygenation signal to obtain a differential sequence. Based on the differential sequence, the concentration changes of oxyhemoglobin and deoxyhemoglobin are determined and calculated.

[0060] Furthermore, the deep fusion analysis model is a multimodal temporal model based on deep learning, specifically a spatiotemporal graph convolutional network or a multimodal Transformer model; the abnormal patterns include at least: epileptic seizure prodrome pattern, stroke prodrome pattern, and acute consciousness disorder pattern; the comprehensive risk level is divided into at least four levels: normal, attention, warning, and critical.

[0061] Specifically, the execution instruction generation module 14 includes: The alert unit is used to send a first-level alert signal to the terminal device worn by the operator when the risk level is "caution"; The warning unit is used to send a second-level alarm to the terminals of the operators and the person in charge of the on-site work when the risk level is warning, and to generate a work suspension prompt message; The process control unit is used to trigger the third-level safety interlock process when the risk level is critical; the safety interlock process includes: automatically sending a distress message containing precise location to the emergency center, remotely locking the power supply of preset risk equipment, and activating the on-site audible and visual alarm device.

[0062] In this invention, user A wears a smart wearable device integrating an EEG sensor, a functional near-infrared cerebral oxygenation sensor, an ECG sensor, and a skin conductance sensor. These sensors continuously and in real time collect user A's EEG signals, prefrontal cortex functional near-infrared cerebral oxygenation signals, ECG signals, and finger skin conductance signals. Subsequently, an edge computing node carried by user A receives the aforementioned multimodal physiological data stream. This edge computing node first performs time alignment processing on this data to ensure that the signals from different sensors are precisely synchronized on the time axis. Next, the synchronized multimodal feature set is transmitted to a cloud-based central processing unit for deep analysis. The pre-set deep fusion analysis model in the cloud-based central processing unit assesses user A's current physiological state through its internally learned complex patterns and outputs a corresponding comprehensive risk level, such as "low risk" or "attention". Finally, based on the aforementioned comprehensive risk level, corresponding graded warning and safety interlock actions are triggered. Through this series of actions, the present invention realizes a closed-loop transmission from real-time perception of user A's physiological state and intelligent identification of potential risks to timely and effective safety intervention, thereby effectively reducing the risk of accidents caused by sudden physiological abnormalities of workers in high-risk power operations.

[0063] This invention provides a multimodal physiological indicator monitoring and early warning system for power workers. This system effectively addresses the problems of insufficient real-time performance, information silos, and lack of closed-loop control in existing power operation safety monitoring. The multimodal physiological signal acquisition module comprehensively acquires direct and co-sensory physiological signals from the brain, providing a rich data foundation for accurately judging the physiological state of workers. The edge computing and fusion processing module performs time alignment, filtering preprocessing, and preliminary feature extraction near the data source, significantly reducing data transmission volume and latency, ensuring the system's real-time response capability to emergencies, and solving the time synchronization problem between different signals, avoiding information silos. The cloud-based intelligent analysis and decision-making module utilizes powerful computing capabilities and deep fusion analysis models to identify complex abnormal patterns related to sudden brain diseases that are difficult to detect using traditional methods, thereby significantly improving the accuracy and foresight of risk identification. The graded early warning and safety interlocking execution module directly transforms the perceived physiological state into actual safety intervention measures, realizing closed-loop control from early warning to mandatory intervention, effectively preventing the occurrence or escalation of accidents, and fundamentally improving the safety protection level of power workers and the inherent safety of the power grid.

[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal physiological indicator monitoring and early warning system for power workers, characterized in that, The system includes: The data stream acquisition module is used to acquire physiological data streams of power workers in real time at the edge. The physiological data streams include direct physiological signals and co-physiological signals from the brain. The direct physiological signals from the brain include electroencephalogram (EEG) signals and functional near-infrared cerebral oxygenation (FIN) signals. The co-physiological signals include electrocardiogram (ECG) signals and electrodermal conductance (EDC) signals. The data feature extraction module is used to perform time alignment, filtering preprocessing, and feature extraction on the physiological data stream to construct a multimodal feature set; The system's operations also include: Query the multimodal feature set of any power worker under normal conditions, and use it as the baseline feature set; Based on the aforementioned benchmark feature set, the multimodal feature set of power workers is normalized and used as real-time status data; The real-time status data of different power workers are compared periodically to calculate the status differences of the power workers. Based on the abnormal status, the power workers are grouped as the actual group set. Query the initial group set of power workers, compare the actual group set with the initial group set, locate abnormal actual groups, and simultaneously locate abnormal workers; The feature set processing module is used to process multimodal feature sets based on a preset deep fusion analysis model, wherein the deep fusion analysis model is configured to identify abnormal patterns associated with sudden brain diseases and output a comprehensive risk level. The execution instruction generation module is used to trigger corresponding graded early warning and safety interlock execution instructions based on the comprehensive risk level; the safety interlock execution instructions are used to adjust the current high-risk power operation tasks.

2. The multimodal physiological indicator monitoring and early warning system for power workers according to claim 1, characterized in that, The data feature extraction module includes: The data sequence acquisition unit is used to statistically analyze various types of physiological data based on the same time axis and time step to obtain a data sequence for each type. When analyzing physiological data, the time point is determined on the time axis according to the time step, and the physiological data closest to the time point is queried as the data in the data sequence. The frequency domain processing unit is used to perform frequency domain transformation on each type of data sequence and to perform filtering preprocessing on the frequency domain transformed data sequence; The extraction execution unit is used to extract the filtered and preprocessed data sequence based on a preset time period and perform feature extraction on the extracted data sequence. The feature extraction process includes: Time-frequency domain features are extracted from EEG signals to obtain the power spectral density of a specific frequency band; Extracting time-domain and frequency-domain features of heart rate and heart rate variability from electrocardiogram signals; Extracting skin conductance levels and response amplitude characteristics from electrodermal signals; Differential identification is performed on the functional near-infrared brain oxygenation signal to obtain a differential sequence. Based on the differential sequence, the concentration changes of oxyhemoglobin and deoxyhemoglobin are determined and calculated.

3. The multimodal physiological indicator monitoring and early warning system for power workers according to claim 1, characterized in that, The deep fusion analysis model is a multimodal temporal model based on deep learning, specifically a spatiotemporal graph convolutional network or a multimodal Transformer model; the abnormal patterns include at least: epileptic seizure prodrome pattern, stroke prodrome pattern, and acute consciousness disorder pattern; the comprehensive risk level is divided into at least four levels: normal, attention, warning, and critical.

4. The multimodal physiological indicator monitoring and early warning system for power workers according to claim 1, characterized in that, The execution instruction generation module includes: The alert unit is used to send a first-level alert signal to the terminal device worn by the operator when the risk level is "caution"; The warning unit is used to send a second-level alarm to the terminals of the operators and the person in charge of the on-site work when the risk level is warning, and to generate a work suspension prompt message; The process control unit is used to trigger the third-level safety interlock process when the risk level is critical; the safety interlock process includes: automatically sending a distress message containing precise location to the emergency center, remotely locking the power supply of preset risk equipment, and activating the on-site audible and visual alarm device.

Images