A cognitive failure early warning method fusing task awareness and multi-modal physiological features

Abstract

To address the shortcomings of existing high-risk industrial production environments, such as the reliance on single indicators for monitoring personnel's physiological state leading to false alarms, the lack of stress coupling to extreme physical environments, and the inability to perceive task hazards due to the use of static fixed thresholds, this paper proposes a cognitive failure early warning method that integrates task perception and multimodal physiological characteristics. By constructing a nonlinear cross-coupling model of environmental penalty factors and task perception, the method achieves transient quantitative assessment of cognitive failure in workers. Furthermore, through an innovative dynamic reverse threshold adaptive adjustment mechanism, the method enables alerts to be set according to specific events, eliminating fatal mismatches between state and task from the algorithmic level.

Description

Technical Field

[0001] This invention relates to the fields of industrial safety automation, human factors engineering and artificial intelligence pattern recognition, and specifically to a cognitive failure early warning method that integrates task perception and multimodal physiological characteristics. Background Technology

[0002] In high-risk and highly sensitive process industries such as nuclear power, petrochemicals, aerospace, and deep-sea engineering, the inherent safety of a system ultimately depends on the accurate decision-making and execution of frontline operators in the face of abnormal transients. Psychological and human factors engineering research shows that when personnel are under extreme fatigue, high-pressure stress, or facing sudden emergencies, they are highly susceptible to the "cognitive tunneling effect" or "cognitive failure." At this time, operators' focus narrows dramatically, mechanically concentrating only on a specific local clue (such as a particular parameter or minor alarm), while ignoring crucial global crisis information (such as the failure rate of core systems or red lights on other main control panels). This is a core human factor contributing to major industrial disasters such as the Three Mile Island and Chernobyl nuclear accidents.

[0003] In recent years, with the development of wearable devices, some high-risk industrial sites have begun to try introducing personnel physiological status monitoring systems. However, facing the complex and ever-changing heavy industrial production environment, existing status monitoring algorithms and early warning systems have the following fatal limitations: (1) The physiological indicators have a single dimension of mapping and lack in-depth quantification of "cognitive state" (which can easily lead to the "boy who cried wolf" effect).

[0004] Most existing commercial wearable devices or conventional industrial monitoring systems rely solely on single heart rate or absolute blood oxygen levels for alarms. However, in industrial settings, purely abnormal physiological indicators (such as a spike in heart rate caused by heavy physical labor like lifting heavy objects or climbing scaffolding) do not equate to "mental overload" or "cognitive impairment" in operators. Current algorithms lack mathematical models that cross-modally fuse "surface physiological data" (such as heart rate) with "deep psychological stress" (such as sympathetic / parasympathetic imbalance), making them highly susceptible to triggering numerous "false alarms" during routine physical labor. This ultimately leads to "alarm fatigue" among on-site management personnel, rendering the system completely unusable.

[0005] (2) It is in an “environmental island” state and fails to quantify the nonlinear stress amplification effect of extreme physical environment.

[0006] High-risk industrial sites are often accompanied by extreme high temperature, high humidity, or high radiation environments (such as confined spaces during nuclear power plant overhauls). Existing physiological assessment algorithms typically model "humans" in an ideal normal temperature environment, completely ignoring the exponential amplification effect of harsh external physical environments on operators' psychological panic and physiological limits, leading to severely distorted risk assessment results under extreme conditions.

[0007] (3) Lack of "task context" awareness and the use of static thresholds lead to fatal mismatch between "state and task" (core pain point).

[0008] This is the most fatal flaw in existing technology. In actual industrial production, the systemic risks of the same level of fatigue when performing "routine ground cleaning" (low fault tolerance requirements) are drastically different from those when performing "manual insertion and removal of nuclear reactor control rods" and "quality assurance witness point (H-point) operation" (zero fault tolerance requirements). Existing monitoring algorithms mainly rely on fixed static alarm thresholds (e.g., setting an alarm when the heart rate exceeds 120 beats / min). This "one-size-fits-all" static logic is completely isolated from information. In modern high-risk process industries, instructions from the production site are uniformly issued by the Distributed Control System (DCS). Existing physiological monitoring algorithms fail to connect the "production information layer" and the "personnel perception layer," resulting in a disconnect between system status assessment and the difficulty of the current task. This invention innovatively introduces a task criticality factor κ(t) derived from the DCS to characterize the fault tolerance and risk level of the current operation, thus providing an objective basis for subsequent adaptive dynamic early warning. In summary, developing a cognitive failure early warning method that can integrate multimodal physiological and environmental data and perform "adaptive dynamic threshold adjustment" based on the current risk level of production tasks, thus achieving "emergency alerts based on events," has become a critical technological bottleneck that urgently needs to be overcome in the field of industrial automation and safety engineering. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing monitoring methods for personnel's physiological state in high-risk industrial production environments. These methods suffer from limitations such as reliance on single indicators leading to false alarms, lack of stress coupling to extreme physical environments, and the inability to perceive "task hazard" due to the use of static, fixed thresholds. This invention provides a cognitive failure early warning method that integrates task perception and multimodal physiological characteristics. It aims to achieve transient quantitative assessment of "cognitive failure (cognitive tunneling effect)" in workers by constructing a nonlinear cross-coupling model of environmental penalty factors and task perception. Furthermore, through a pioneering dynamic reverse threshold adaptive adjustment mechanism, it achieves "event-based warning," eliminating the fatal mismatch between state and task at the algorithmic level.

[0010] The technical solution of this invention is as follows: A cognitive failure early warning method integrating task perception and multimodal physiological characteristics, comprising the following steps: S1: Acquire real-time multimodal physiological characteristic signals of high-risk workers and environmental stress parameters of the work site. The multimodal physiological characteristic signals include electrocardiogram (ECG) and electrodermal conductance (EDA) signals. The environmental stress parameters include ambient temperature and radiation dose rate. S2: Perform frequency domain analysis on both ECG and EDA signals to extract the low-to-high frequency power ratio (HRV) of heart rate variability. LF / HF (t) and the skin conductance response amplitude SCR(t), and then normalized respectively; S3: Constructing the nonlinear environmental penalty factor E pen (t): in: Temp(t) is the real-time temperature. max It is the upper limit of the specified safe temperature; Rad(t) is the real-time radiation dose rate. max It is a safe radiation limit.

[0011] ρ is the coupling coefficient, and τ is the exposure time accumulation factor; S4: Low-to-high frequency power ratio (HRV) of the normalized heart rate variability in S2 LF / HF The baseline stress load (BSL) at time t is calculated by fusing the skin conductance response amplitude (SCR)(t) with the environmental penalty factor constructed by S3: Norm represents the normalization operator; w1 and w2 are weight coefficients. S5: Obtain the task criticality factor κ(t), and perform a nonlinear cross-mapping between the baseline stress load BSL(t) in S4 and the task criticality factor κ(t) to obtain the cognitive failure index. Where λ is the task sensitivity adjustment coefficient; S6: Generate the current cognitive failure alarm threshold Th(t): in: Th base Baseline security threshold μ: Threshold tightening coefficient; S7: Compare CFI(t) and Th(t) in real time. When it is determined that CFI(t) > Th(t), generate an early warning signal and send it out.

[0012] In S1, the real-time physiological signals of high-risk workers and environmental stress parameters of the work site are simultaneously acquired through an intrinsically safe wearable device and an on-site environmental monitoring sensor network at a preset sampling rate.

[0013] Intrinsically safe wearable devices include safety helmets and smart chest patches with dry electrodes or photoplethysmography (PPG) enhancement algorithms.

[0014] In S3, under ideal, stress-free, normal temperature and environment, the penalty factor E pen When (t) is 1, it means that no additional amplification is applied to the multimodal physiological characteristic signals, and the stress of the personnel comes entirely from the work itself.

[0015] The coupling coefficient ρ is used to adjust the weight of the overall environmental stress on the final result, and is fitted based on historical data of a specific industrial scenario.

[0016] In S4, w1 and w2 are fitted and their values ​​are determined based on historical data from actual industrial scenarios.

[0017] In S5, the current task criticality factor κ(t) is issued through the distributed control system DCS.

[0018] In S5, the task criticality factor κ(t) ranges from (0,1). The larger κ is, the more dangerous the task.

[0019] In S6, Th base This indicates the initial maximum tolerance threshold set by the system when the operator is at rest or in an absolutely safe state.

[0020] In step S7, a forced freeze of the corresponding operation interface is triggered. The beneficial effects of this invention are: (1) It breaks through the bottleneck of "alarm fatigue" caused by a single indicator.

[0021] This invention innovatively fuses electrocardiogram frequency domain features and skin conductance features, which reflect the state of the autonomic nervous system, across modalities, and introduces an environmental penalty factor E. pen (t). This mechanism accurately quantifies the amplifying effect of harsh physical environments on personnel stress, effectively distinguishing between "heart rate increase caused by simple heavy physical labor" and "mental exhaustion caused by high-pressure stress," and significantly reducing the false alarm rate in industrial sites.

[0022] (2) It pioneered a nonlinear evaluation paradigm based on "task awareness".

[0023] Traditional algorithms separate the assessment of a person's state from the task being performed. This invention introduces the task criticality κ(t) issued by the DCS into the model through an exponential coupling formula, mathematically revealing the objective law that "the more dangerous the task, the more easily even small physiological fluctuations can trigger cognitive collapse," thus achieving a leap from superficial physiological monitoring to deep cognitive reliability assessment.

[0024] (3) A mechanism for preventing false alarms by setting up alarms based on the situation has been implemented.

[0025] This invention breaks through the limitations of traditional static alarm thresholds by designing an inverse function adjustment mechanism between task criticality and alarm threshold. Specifically, the more dangerous the current task (the larger κ(t)), the lower the threshold Th(t) for triggering the system alarm is, and the more stringent the system's review of personnel's cognitive state. This ensures both extreme sensitivity during critical operations (such as nuclear power plant quality assurance H-point operations) and smooth operation for routine tasks. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the application process.

[0027] Figure 2 A schematic diagram of a dynamic early warning threshold and cognitive failure index determination logic curve based on task criticality is provided for this embodiment. Detailed Implementation

[0028] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0029] The terminology used in one or more embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this application. The singular forms “a,” “the,” and “the” used in one or more embodiments of this application and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more associated listed items.

[0030] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this application, and similarly, second may also be referred to as first.

[0031] It should be noted that in this document, the terms "comprising," "including," or any other term thereof are used interchangeably. A cognitive failure early warning method integrating task awareness and multimodal physiological characteristics includes the following execution steps: S1: Synchronous acquisition of heterogeneous physiological and environmental data across multiple modalities.

[0032] Through intrinsically safe wearable devices and an on-site environmental monitoring sensor network, real-time physiological signals of high-risk workers and environmental stress parameters of the work site are simultaneously acquired at a preset sampling rate. The multimodal physiological signals include at least electrocardiogram (ECG) and electrical conductance analysis (EDA) signals; the environmental stress parameters include at least ambient temperature and radiation dose rate.

[0033] S2: Multimodal feature extraction and baseline stress load (BSL) calculation.

[0034] Frequency domain analysis was performed on the electrocardiogram signal to extract the low-to-high frequency power ratio (HRV) of heart rate variability. LF / HF (t); Time-domain analysis was performed on the skin conductance signal to extract the skin conductance response amplitude SCR(t), and normalization was performed on each.

[0035] HRV (Heart Rate Variability) is the analysis of small fluctuations in the RR interval of an electrocardiogram (ECG) signal. By performing frequency domain analysis such as Fast Fourier Transform (FFT) on the time series, low frequency (LF, which usually reflects the mixed activity of the sympathetic and parasympathetic nervous systems, but mainly reflects the activity of the sympathetic nervous system under severe stress) and high frequency (HF, which mainly reflects the parasympathetic nervous system activity, i.e., the relaxed state) can be separated.

[0036] Currently available industrial-grade intrinsically safe smart wearable devices (such as safety helmets and smart chest patches equipped with dry electrodes or photoplethysmography (PPG) enhancement algorithms) have sufficient chip computing power and sampling rate to calculate this frequency domain ratio in real time at the edge. This indicator is recognized as the gold standard for measuring "mental stress load" and "autonomic nervous system balance" in medical and military human factors engineering.

[0037] S3: Constructing the nonlinear environmental penalty factor E pen (t) (representing the environmental penalty factor at time t), its mathematical model is: Temp max Rad max Here, ρ represents the environmental safety limit, τ represents the coupling coefficient, and τ represents the exposure time accumulation factor. Under ideal, stress-free, normal temperature and environment, the penalty factor E pen When (t) is 1, it means that no additional amplification of physiological data is applied, and the stress of the personnel comes entirely from the work itself.

[0038] The coupling coefficient ρ is used to adjust the weight of the overall environmental stress term on the final result, and it usually needs to be fitted based on historical data of a specific industrial scenario.

[0039] Temp(t) is the real-time temperature. max It is the upper limit of the specified safe temperature. This is the temperature stress ratio. The closer the current temperature is to the limit, the closer this value is to 1.

[0040] Rad(t) is the real-time radiation dose rate. max It is a safe radiation limit. This is the radiation stress ratio. This is especially critical in flaw detection operations involving the nuclear industry or radioactive sources.

[0041] τ is the cumulative exposure time factor, representing the length of time personnel work continuously in harsh environments.

[0042] S4: Normalized physiological characteristics (including heart rate variability, low-to-high frequency power ratio HRV) LF / HF The baseline stress load (BSL) at time t is calculated by fusing the skin conductance response amplitude (SCR(t)) with the environmental penalty factor: in: BSL(t): Represents the base stress load at time t, used to quantify the basic stress level of operators as manifested by physiological responses under specific physical conditions.

[0043] Norm(): Represents the normalization operator. Because the physical dimensions of ECG frequency domain data and skin conductance amplitude data are completely different, they must be normalized to map them to a uniform numerical range (e.g., [0, 1]). This step is a key mathematical operation to eliminate dimensional differences and ensure that data from different modalities can be weighted and fused.

[0044] w1 and w2: These represent the fusion weighting coefficients for electrocardiogram (ECG) and electrodermal (ED) characteristics, respectively. They are used to adjust the proportion of these two different dimensions of physiological signals in the overall load calculation, and are usually fitted and valued based on historical data from actual industrial scenarios.

[0045] HRV LF / HF (t): Represents the low-to-high frequency power ratio of heart rate variability at time t, extracted from the electrocardiogram (EDG) signal, and is used to characterize the activity of the sympathetic nervous system.

[0046] SCR(t): Represents the amplitude of skin conductance response at time t, extracted from the skin conductance signal (EDA), used to characterize the emotional arousal of the individual.

[0047] E pen (t): Represents the nonlinear environmental penalty factor. As a product term, it is used to quantify the exponential amplification effect of extreme physical environments (such as high temperature, high radiation, and the cumulative exposure time) on human physiological stress.

[0048] S5: Calculation of the Cognitive Failure Index (CFI) based on task awareness.

[0049] The task criticality factor κ(t) (κ(t)∈(0,1]) currently issued by the distributed control system (DCS) is acquired in real time. The basic stress load BSL(t) and the task criticality factor κ(t) are nonlinearly cross-mapped to calculate the cognitive failure index CFI(t) that quantifies the operator's current state. in: CFI(t) is the cognitive failure index at time t. The higher the value, the greater the probability that the human brain "shuts down" or experiences the "tunneling effect".

[0050] BSL(t) is the previously calculated baseline stress load. It represents the stress that the individual experiences purely physical and environmental.

[0051] κ(t) is the mission criticality factor (from DCS), and its value range is usually set to (0,1). The larger κ is, the more dangerous the mission (for example, κ=0.1 for ordinary ground inspection; while κ=0.9 for "zero-tolerance" operations such as reactor main pump alignment). λ is the task sensitivity adjustment coefficient. This is an empirical constant used to control the steepness of the exponential curve.

[0052] S6: Adaptive dynamic threshold generation and over-limit intervention.

[0053] A dynamic threshold inverse function is constructed with the task criticality factor κ(t) as the independent variable to generate the current cognitive failure alarm threshold Th(t) in real time: Th base This is the baseline safety threshold. This is the initial maximum tolerance level set by the system when the operator is at rest or in an absolutely safe state.

[0054] κ(t): This is also a mission criticality factor transmitted from DCS.

[0055] μ: Threshold tightening coefficient. Used to adjust the rate at which this warning line descends.

[0056] Inverse function logic (inverse proportional relationship): As the task becomes increasingly critical (κ(t) increases from 0), the denominator (1 + μ•κ(t)) increases, causing the overall value of the fraction Th(t) to decrease. This means that the alarm threshold is deliberately lowered by the system when performing dangerous tasks.

[0057] S7: The system compares CFI(t) and Th(t) in real time. When it determines that CFI(t) > Th(t), it generates a "cognitive tunnel" warning signal and sends it to the industrial management terminal, triggering the forced freeze or dual-person review process on the corresponding operation interface. Taking the "reactor main pump sealing assembly operation during nuclear power plant overhaul" as an example, this paper illustrates the effect of dynamic threshold mechanism in preventing false alarms and missed alarms.

[0058] When operators complete the preceding "material handling" process, their basal stress load (BSL(t)) increases significantly due to physical exertion; however, the criticality κ of the task issued by the DCS is extremely low at this time (e.g., κ=0.1). According to the inverse function formula in step S4, the system alarm threshold Th(t) remains at a high level. Therefore, although the operator's heart rate is high and BSL is high, CFI(t) is still below Th(t), and the system determines it as safe "physical fatigue," without issuing invalid interference alarms (filtering out false alarms).

[0059] Subsequently, the operator proceeded to the core "main pump sealing surface alignment and bolt tightening" process (H-point operation, zero tolerance for error). At this point, the DCS issued a task criticality κ=0.95 for this process. According to the inverse function formula of step S4, the system's alarm threshold Th(t) was instantly and significantly reduced (the system became extremely demanding). Simultaneously, due to the high-temperature radiation environment (increased cumulative τ) and high-pressure operation, although the operator appeared calm to the naked eye, their skin conductance (SCR) and heart rate (HRV) were significantly reduced. LF / HF (t) exhibits slight fluctuations in sympathetic overload. According to the exponential amplification formula in step S3, its CFI(t) spikes exponentially.

[0060] In t x At that moment, the CFI(t) curve broke through the suppressed dynamic threshold line Th(t), and the system accurately determined that the operator had fallen into the "cognitive tunneling effect" (i.e., extreme focus but loss of overall judgment). The system triggered an interlocking block within milliseconds, forcibly freezing the torque feedback interface of the tightening wrench and prompting "Personnel's cognitive load exceeds the limit, please force a 15-second pause and have a supervisor review the operation." This effectively avoided fatal assembly errors caused by excessive focus on a local area.

[0061] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0062] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0063] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0064] The preferred embodiments disclosed above are merely illustrative of this application. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this application. These embodiments are selected and specifically described in this application to better explain the principles and practical applications of this application, thereby enabling those skilled in the art to better understand and utilize this application.

Claims

1. A cognitive failure early warning method integrating task perception and multimodal physiological characteristics, characterized in that: Includes the following steps: S1: Acquire real-time multimodal physiological characteristic signals of high-risk workers and environmental stress parameters of the work site. The multimodal physiological characteristic signals include electrocardiogram (ECG) and electrodermal conductance (EDA) signals. The environmental stress parameters include ambient temperature and radiation dose rate. S2: Frequency domain analysis of ECG and EDA, respectively, to extract the low-to-high frequency power ratio of heart rate variability, HRV LF / HF (t) and the skin conductance response amplitude SCR(t), and normalize them respectively; S3: Constructing the nonlinear environmental penalty factor E pen (t): in: Temp(t) is the real-time temperature. max It is the upper limit of the specified safe temperature; Rad(t) is the real-time radiation dose rate. max These are safe radiation limits; ρ is the coupling coefficient, and τ is the exposure time accumulation factor; S4: Low-to-high frequency power ratio (HRV) of the normalized heart rate variability in S2 LF / HF The baseline stress load (BSL) at time t is calculated by fusing the skin conductance response amplitude (SCR)(t) with the environmental penalty factor constructed by S3: Norm represents the normalization operator; w1 and w2 are weight coefficients. S5: Obtain the task criticality factor κ(t), and perform a nonlinear cross-mapping between the baseline stress load BSL(t) in S4 and the task criticality factor κ(t) to obtain the cognitive failure index. Where λ is the task sensitivity adjustment coefficient; S6: Generate the current cognitive failure alarm threshold Th(t): in: Th base Baseline security threshold μ: Threshold tightening coefficient; S7: Compare CFI(t) and Th(t) in real time. When it is determined that CFI(t) > Th(t), generate an early warning signal and send it out.

2. The cognitive failure early warning method integrating task perception and multimodal physiological characteristics according to claim 1, characterized in that: In S1, the real-time physiological signals of high-risk workers and environmental stress parameters of the work site are simultaneously acquired through an intrinsically safe wearable device and an on-site environmental monitoring sensor network at a preset sampling rate.

3. The cognitive failure early warning method integrating task perception and multimodal physiological characteristics according to claim 2, characterized in that: Intrinsically safe wearable devices include safety helmets and smart chest patches with dry electrodes or photoplethysmography (PPG) enhancement algorithms.

4. The cognitive failure early warning method integrating task perception and multimodal physiological characteristics according to claim 1, characterized in that: In S3, under ideal, stress-free, normal temperature and environment, the penalty factor E pen When (t) is 1, it means that no additional amplification is applied to the multimodal physiological characteristic signals, and the stress of the personnel comes entirely from the work itself.

5. The cognitive failure early warning method integrating task perception and multimodal physiological characteristics according to claim 1, characterized in that: The coupling coefficient ρ is used to adjust the weight of the overall environmental stress on the final result, and is fitted based on historical data of a specific industrial scenario.

6. The cognitive failure early warning method integrating task perception and multimodal physiological characteristics according to claim 1, characterized in that: In S4, w1 and w2 are fitted and their values ​​are determined based on historical data from actual industrial scenarios.

7. The cognitive failure early warning method integrating task perception and multimodal physiological characteristics according to claim 1, characterized in that: In S5, the current task criticality factor κ(t) is issued through the distributed control system DCS.

8. The cognitive failure early warning method integrating task perception and multimodal physiological characteristics according to claim 7, characterized in that: In S5, the task criticality factor κ(t) has a value range of (0,1).

9. The cognitive failure early warning method integrating task perception and multimodal physiological characteristics according to claim 1, characterized in that: In S6, Th base This indicates the initial maximum tolerance threshold set by the system when the operator is at rest or in an absolutely safe state.

10. The cognitive failure early warning method integrating task perception and multimodal physiological characteristics according to claim 1, characterized in that: In step S7, a forced freeze is triggered on the corresponding operation interface.

Images