A human-computer collaborative perception operation fatigue state evaluation method and device

Abstract

This invention provides a method and apparatus for assessing operational fatigue state through human-machine collaborative perception, relating to the field of data processing technology. The method includes: acquiring multimodal physiological and behavioral data of the operator, performing time synchronization and noise suppression processing to form a multimodal observation dataset; extracting multidimensional feature parameters based on the multimodal observation dataset, and combining them with a constructed biomathematical fatigue assessment model to generate a fatigue characterization quantity of the operator; performing data-driven modeling processing on the multidimensional feature parameters to form a fusion characterization model, and jointly mapping the fatigue characterization quantity with the fusion characterization model to construct a human-machine collaborative perception characterization space; and comprehensively calculating the collaborative consistency and response efficiency during the interaction between the operator and the system based on the human-machine collaborative perception characterization space to generate a state analysis result. This invention can improve the accuracy of operational fatigue state analysis by incorporating the dynamic adaptation degree during the human-machine collaboration process.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a method and apparatus for assessing operational fatigue state through human-machine collaborative perception. Background Technology

[0002] Existing human-machine interface fatigue monitoring technologies are mainly applied to scenarios such as vehicle driving, aviation control, industrial control console operation, and complex equipment operation. They typically identify and warn of fatigue levels by collecting information such as the operator's heart rate, EEG, eye movements, EMG, facial expressions, posture, and operation logs. Current solutions mostly employ single physiological signal threshold discrimination methods or establish statistical models based on indicators such as behavior frequency, operation interval, and number of misoperations. Some technologies also incorporate machine learning methods to fuse and analyze multi-source data to improve the accuracy of fatigue identification. As the complexity of intelligent equipment and human-machine interaction systems continues to increase, evaluation methods that rely solely on local signals or single-dimensional indicators are gradually evolving towards comprehensive evaluation methods that integrate physiological states, behavioral patterns, and interaction characteristics.

[0003] However, existing technologies generally suffer from a significant drawback: in the process of multimodal data fusion, they focus only on the fatigue characteristics of the individual operator, neglecting the crucial factors of the coordination and consistency of human-system interaction and response efficiency. This leads to a discrepancy between fatigue assessment results and actual operational performance. Specifically, existing methods typically fail to construct a high-dimensional representation space that can uniformly describe the coupling relationship between physiological states, behavioral characteristics, and system responses. This makes it difficult for fatigue assessment results to reflect the dynamic adaptation in the human-machine collaboration process, thus hindering the accurate characterization of fatigue states in complex operating environments. Summary of the Invention

[0004] This invention provides a method and apparatus for assessing operational fatigue state through human-machine collaborative perception, which can improve the accuracy of operational fatigue state analysis by combining the degree of dynamic adaptation in the human-machine collaboration process.

[0005] A first aspect of the present invention provides a method for assessing operational fatigue state through human-machine collaborative perception, the method comprising: Acquire multimodal physiological and behavioral data of the operating subject, and perform time synchronization and noise suppression processing to form a multimodal observation dataset; Multidimensional feature parameters are extracted based on the multimodal observation dataset, and combined with the constructed biomathematical fatigue assessment model to generate fatigue characterization of the operating subject. Data-driven modeling processing is performed on the multidimensional feature parameters to form a fusion representation model, and the fatigue representation quantity is jointly mapped with the fusion representation model to construct a human-machine collaborative perception representation space. Based on the human-machine collaborative perception representation space, the collaborative consistency and response efficiency during the interaction between the operating subject and the system are comprehensively calculated to generate state analysis results.

[0006] In a second aspect of the invention, a human-machine collaborative sensing operation fatigue state assessment device is provided. The device is used to execute a human-machine collaborative sensing operation fatigue state assessment method as described in any of the above embodiments. The device includes an acquisition module, a processing module, and an output module, wherein: The acquisition module is used to acquire multimodal physiological and behavioral data of the operating subject, and perform time synchronization processing and noise suppression processing to form a multimodal observation data set; The processing module is used to extract multidimensional feature parameters based on the multimodal observation data set, and combine them with the constructed biomathematical fatigue assessment model to generate the fatigue characterization quantity of the operating subject. The processing module is used to perform data-driven modeling processing on the multidimensional feature parameters to form a fusion representation model, and to jointly map the fatigue representation quantity with the fusion representation model to construct a human-machine collaborative perception representation space. The output module is used to perform comprehensive calculations on the coordination consistency and response efficiency of the interaction between the operating subject and the system based on the human-machine collaborative perception representation space, and generate state analysis results.

[0007] In a third aspect of the invention, an electronic device is provided, including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any of the preceding embodiments.

[0008] In a fourth aspect of the invention, a non-transitory computer-readable storage medium is provided, the computer-readable storage medium storing instructions that, when executed, perform the method as described in any of the preceding claims.

[0009] In summary, one or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: This invention generates a fatigue characterization quantity of the operator by constructing a biomathematical fatigue assessment model based on multimodal observation data. This model is then jointly mapped to a human-machine collaborative perception characterization space using a data-driven modeling process that combines the model with multidimensional feature parameters. This allows for a unified representation of the operator's physiological state, behavioral characteristics, and system interaction requirements within the same high-dimensional space. In this space, by simultaneously calculating collaborative consistency, response efficiency, and silent task adaptability, it is possible to distinguish between a low-interaction but highly alert silent state and a true fatigue state. This quantifies the operator's dynamic adaptability at different task stages, significantly improving the accuracy of fatigue state analysis and its adaptability to complex operating environments. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating a human-machine collaborative perception method for assessing operational fatigue state, as disclosed in an embodiment of the present invention. Figure 2 This is a schematic diagram of a head-mounted EEG acquisition device disclosed in an embodiment of the present invention; Figure 3 This is a schematic diagram of a human-machine collaborative perception-based operational fatigue state assessment device disclosed in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention.

[0011] Explanation of reference numerals in the attached drawings: 301, acquisition module; 302, processing module; 303, output module; 401, processor; 402, communication bus; 403, user interface; 404, network interface; 405, memory. Detailed Implementation

[0012] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0013] In the description of the embodiments of the present invention, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0014] In the description of the embodiments of the present invention, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0015] While existing human-machine fatigue monitoring technologies have evolved from single physiological signal discrimination to multimodal fusion analysis, they still primarily focus on the physiological and behavioral characteristics of individual operators. They lack modeling and characterization of the consistency and response efficiency of human-system interaction, and fail to construct a unified high-dimensional representation space that reflects the coupling relationship between physiological state, behavioral patterns, and system response. This results in fatigue assessment results that are difficult to accurately reflect the dynamic adaptation in the human-machine collaboration process, leading to insufficient assessment accuracy and deviation from actual operational performance in complex operating environments.

[0016] This invention discloses a human-machine collaborative perception method for assessing operational fatigue, which is applied to a server. The server includes, but is not limited to, electronic devices such as mobile phones, tablets, wearable devices, and PCs (Personal Computers), and can also be a backend server running the human-machine collaborative perception method for assessing operational fatigue. The server can be a standalone server or a server cluster composed of multiple servers.

[0017] This embodiment discloses a human-machine collaborative perception method for assessing operational fatigue state, referring to... Figure 1 It includes the following steps: S110: Acquire multimodal physiological and behavioral data of the operating subject, and perform time synchronization processing and noise suppression processing to form a multimodal observation data set.

[0018] S120 extracts multidimensional feature parameters based on a multimodal observation dataset and combines them with a constructed biomathematical fatigue assessment model to generate a fatigue characterization of the operating subject.

[0019] S130 performs data-driven modeling processing on multidimensional feature parameters to form a fusion representation model, and jointly maps the fatigue representation quantity with the fusion representation model to construct a human-machine collaborative perception representation space.

[0020] S140, based on the human-machine collaborative perception representation space, comprehensively calculates the collaborative consistency and response efficiency of the interaction process between the operating subject and the system, and generates state analysis results.

[0021] In practical implementation, the operator refers to the personnel who undertake monitoring, judgment, confirmation, control, or emergency response tasks within the target human-machine system. Multimodal physiological data includes electroencephalogram (EEG) data, heart rate data, heart rate variability data, eye movement data, pupillary change data, eye closure data, electromyography (EMG) data, electrodermal conductance (EDA) data, respiratory data, and facial expression data. This data reflects the operator's neural activity, cardiovascular response, visual attention state, muscle tension state, autonomic nervous system arousal state, and respiratory rhythm changes. Behavioral data includes key press records, touch trajectories, mouse or joystick displacement, command confirmation records, response latency, error correction records, head posture, trunk posture, hand movement trajectories, and interface attention areas. This data reflects the actual interaction process between the operator and the system. During data collection, physiological data acquisition devices, behavioral data acquisition devices, and system log recording units are configured in the same task environment. This ensures that both multimodal physiological and behavioral data are associated with task stage identifiers, operator identifiers, and device identifiers, preventing the inability to determine the corresponding task state and operator object for a specific data segment during subsequent analysis.

[0022] Electroencephalogram (EEG) data can be acquired using head-mounted EEG acquisition devices, focusing on recording changes in electrical activity in brain regions related to attention, fatigue, and visual processing, such as the frontal, parietal, and occipital lobes. (See reference...) Figure 2The head-mounted EEG acquisition device adopts a wearable, wraparound frame structure, consisting of a head support skeleton, electrode array, acquisition and control unit, power supply and communication unit, and adjustment and fixing mechanism. This structure uses a longitudinal support beam at the top of the head and a circumferential headband to form a spatial support frame, ensuring a stable fit to different head shapes and reliable contact between the acquisition electrodes and the scalp. The device employs a multi-point support layout, providing multi-channel EEG coverage in the frontal, parietal, occipital, and bilateral temporal lobes, for acquiring EEG signals related to attention, fatigue, cognitive load, and visual processing. Structurally, the top of the head features an arc-shaped support beam and cross-connecting rods, with multiple connection nodes using a rotatable joint structure, allowing for radial and axial fine-tuning of the electrode placement. This design adapts to different head circumferences, scalp curvatures, and hair volumes, avoiding the instability issues found in traditional fixed EEG caps in certain areas. The headband uses a ring-shaped flexible support that wraps around the head circumferentially. Adjustable knobs on both sides change the inner diameter of the headband, creating a uniform clamping force that ensures stability while reducing localized pressure during prolonged wear. Frontal lobe electrodes are preferentially placed in the forehead region to collect frontal EEG activity highly relevant to fatigue monitoring, such as the low-frequency enhancement characteristic commonly seen during attention decline. Parietal and central region electrodes supplement signals related to cognitive processing and task execution. Reference electrodes are used to establish a stable potential baseline and are typically placed behind the ear, in the mastoid region, or in low-interference areas. Grounding electrodes suppress common-mode noise and environmental electromagnetic interference, improving the signal-to-noise ratio of EEG signals. Differential acquisition allows for the separation of weak EEG signals from skin potentials and environmental noise. Each electrode employs a flexible contact structure, with the lower end potentially fitted with a spring probe, conductive silicone tip, or wet electrode conductive medium. For short-term, rapid deployment scenarios, a dry electrode structure can be used to reduce conductive gel preparation time; for high-precision acquisition scenarios, a semi-dry or wet electrode structure can be used to improve skin contact stability. Electrodes are connected to the rear acquisition module via wires. These wires are arranged along the inside of the support beam or in slots, and their direction is constrained by fixing clips to prevent motion artifacts caused by swaying during wear. The rear acquisition module is located at the back or lower back of the head, where status indicator lights, a data interface, and adjustment knobs are located. Positioning the acquisition module at the back of the head serves three purposes: first, it balances the weight of the forehead area, reducing the burden of forward head tilt; second, it is away from the frontal lobe electrodes, reducing coupling interference from the main control circuit to the weak signals at the front end; and third, it facilitates the integration of the battery, wireless communication unit, and storage unit. Status indicator lights display the power supply status, Bluetooth or wireless network connection status, acquisition operation status, and electrode contact quality status. The acquisition control unit typically includes a front-end amplifier circuit, an analog-to-digital converter circuit, a main control processor, and a data buffer circuit. The front-end amplifier circuit amplifies the microvolt-level EEG signals with high input impedance to prevent signal attenuation. The analog-to-digital converter circuit converts continuous analog EEG waveforms into digital sampling sequences.The main control processor performs filtering, channel management, impedance detection, timestamp writing, and data packaging. A data caching circuit temporarily stores data during short-term fluctuations in the wireless link to prevent data loss. If the device is used in a fatigue assessment system, it can simultaneously collect EEG data, posture data, eye-tracking data, or behavior log data locally, and then upload them to a host computer. The device supports various channel configurations, such as 16, 32, or 64 channels. 16 channels are suitable for portable fatigue monitoring and routine on-call assessment; 32 channels are suitable for professional applications that balance spatial resolution and portability; and 64 channels are suitable for high-precision scientific research analysis. The sampling rate can be configured between 250Hz and 1000Hz; lower sampling rates are suitable for routine rhythm analysis, while higher sampling rates are suitable for event-related potentials and high-frequency EEG research. An input impedance greater than 100MΩ reduces measurement errors caused by changes in contact impedance, and a common-mode rejection ratio greater than 100dB effectively suppresses power frequency interference. In practical use, first adjust the headband size according to the user's head circumference, then wear the device to the target position, aligning the frontal lobe electrodes with the forehead area, the parietal lobe electrodes with the top of the head, and the reference electrode and ground electrode with the preset position. After starting the device, perform an electrode impedance self-test. If a channel has poor contact, improve the contact quality by rotating the electrode adjustment structure or fine-tuning the headband pressure. After completing the self-test, enter continuous acquisition mode, transmitting EEG data in real time to the fatigue assessment platform for joint analysis with heart rate data, eye movement data, and behavioral data. When used for human-machine collaborative fatigue monitoring, this device can focus on extracting theta wave power, alpha wave suppression degree, beta wave activity degree, left and right frontal lobe asymmetry characteristics, frequency band energy ratio, and brain network connectivity characteristics. When the user experiences drowsiness, attention decline, or cognitive overload, the EEG pattern will show significant changes. Combined with the headband structure, it enables long-term continuous wear, continuously outputting EEG state information in scenarios such as vehicle driving, underwater vehicle monitoring, aviation operation, industrial control console monitoring, and complex equipment operation, providing a core data source for subsequent fatigue characterization calculations and state analysis results generation.

[0023] Heart rate and heart rate variability data can be acquired using electrocardiogram (ECG) sensors or photoplethysmography (PPG) sensors to characterize the sympathetic and parasympathetic nervous system regulation. Eye movement (EMG) data, pupillary change data, and eye closure data can be acquired using eye trackers or visual imaging devices to characterize fixation stability, saccade behavior, blink frequency, and drowsiness trends. Electromyography (EMG) data can be acquired using surface EMG sensors to characterize muscle tension and movement fatigue trends during actions. Electrodermal activity (EDA) data can be acquired using EDA sensors to characterize changes in autonomic arousal caused by stimuli. Respiratory data can be acquired using chest straps, nasal airflow sensors, or non-contact respiratory detection devices to characterize respiratory rate and respiratory stability. These data from different sources collectively constitute multimodal physiological data, enabling fatigue status to be determined without relying on a single physiological signal, but rather allowing for cross-verification of the subject's state changes from multiple physiological channels.

[0024] Behavioral data is acquired synchronously through the input devices, display interface, console, control devices, and task logs of the human-computer interaction system. Key press records characterize whether the operator triggers the target control action as required; touch trajectories, mouse displacement, or joystick displacement characterize the operation path, movement amplitude, and continuity; command confirmation records characterize the operator's understanding and confirmation of system prompts; response latency characterizes the time interval between the appearance of a system prompt and the operator's effective operation; error correction records characterize whether the operator mis-touches, missed operations, repeated confirmations, or undo / redos during execution; head posture, torso posture, and hand movement trajectories characterize the operator's body stability, attention direction, and movement lag; and interface focus area characterizes whether the operator continuously focuses on key display areas. Through this behavioral data, fatigue state can be correlated with actual operational ability, avoiding the judgment of fatigue solely based on physiological changes while ignoring true interactive performance.

[0025] Time synchronization processing refers to mapping data from different acquisition devices, with different sampling frequencies, and different recording formats to the same time reference. Since EEG data, heart rate data, eye movement data, posture data, and operation logs have different sampling frequencies, and the internal clocks of each device may deviate, a timestamp is first written for each type of data. Then, the data streams are aligned according to a unified clock signal, network time synchronization signal, task start trigger signal, or system synchronization pulse. For high-sampling-frequency data, it is segmented according to the evaluation window; for low-sampling-frequency data, interpolation, hold, or window statistics are used to fill in the corresponding time window; for event-based behavioral data, the time of the event is mapped to the corresponding evaluation window, and the event duration and sequence are recorded. After time synchronization processing, each evaluation window contains physiological state changes and behavioral interaction changes within the same time range, enabling subsequent fatigue analysis to determine whether a certain physiological change is related to a certain operational delay, misoperation, or system prompt.

[0026] Noise suppression processing refers to the identification and reduction of non-target disturbances, acquisition artifacts, and anomalous records in multimodal physiological and behavioral data. For EEG data, bandpass filtering, power frequency notch filtering, and artifact separation can be performed to reduce the impact of eye movements, electromyography (EMG), electrode loosening, and electromagnetic interference on EEG characteristics. For heart rate and heart rate variability data, abnormal pulsation points can be removed and short-term missing segments can be repaired to reduce abrupt changes caused by poor sensor contact. For eye movement data, invalid points caused by blinking obstruction, loss of vision, and glare can be removed. For EMG data, baseline drift and transient spikes can be suppressed. For ESC data, slow baseline changes and event-related responses can be separated. For respiratory data, waveform drift caused by body movement can be suppressed. For behavioral data, anomalous records caused by device jitter, repeated triggering, system log delays, or invalid input can be removed, while retaining the actual operational actions and their temporal relationships. The goal of noise suppression processing is not simply to delete anomalous data, but to improve the usability and consistency of data across modalities without disrupting the true changes in fatigue state.

[0027] When forming a multimodal observation dataset, the multimodal physiological and behavioral data, after time synchronization and noise suppression processing, are structured according to the operator identifier, task stage identifier, evaluation window identifier, and timestamp. Each evaluation window corresponds to an observation unit, which includes at least a physiological observation field, a behavioral observation field, a task context field, and a data quality field. The physiological observation field is used to store processed data fragments or statistical results such as EEG, heart rate, eye movement, EMG, ESC, and respiration; the behavioral observation field is used to store interactive information such as operation actions, response latency, error correction, posture changes, and interface attention areas; the task context field is used to store the current task stage, system prompt status, interaction goals, and operation constraints; and the data quality field is used to store the effectiveness rate, noise level, missing data ratio, and synchronization error of each modality.

[0028] In one possible implementation, multidimensional feature parameters are extracted based on a multimodal observation dataset, and combined with a constructed biomathematical fatigue assessment model to generate fatigue characterization parameters for the operator. Specifically, this includes: extracting sleep-wake data, task execution time, continuous wakefulness duration, sleep duration, and task duration parameters from the multimodal observation dataset, and constructing fatigue time-series data; identifying whether the operator is awake or asleep based on the fatigue time-series data, and generating corresponding sleep homeostasis components and sleep homeostasis recovery components; and calculating the first day based on the task execution time and task duration parameters. The system identifies the nocturnal and second-day circadian rhythm components; it identifies the transition from sleep to wakefulness based on fatigue time-series data and generates a sleep inertia component; it performs individualized corrections on the sleep homeostasis component, first-day circadian rhythm component, second-day circadian rhythm component, and sleep inertia component based on multidimensional feature parameters, generating model correction coefficients; it calculates a comprehensive alertness score based on the model correction coefficients and the sleep homeostasis component, first-day circadian rhythm component, second-day circadian rhythm component, and sleep inertia component, and maps it to generate a fatigue characterization quantity; it outputs the fatigue characterization quantity to a biomathematical fatigue assessment model for joint mapping.

[0029] Specifically, when extracting sleep-wake data from a multimodal observation dataset, the user's state segments—sleep, wakefulness, short-term eye closure, low-activity standby, or task execution—can be identified from EEG data, eye movement data, posture data, heart rate variability data, respiratory data, and behavioral logs, and each state segment is bound to a unified timestamp. Sleep-wake data describes the user's sleep onset, sleep end, wakefulness onset, wakefulness duration, and sleep interruptions, providing a temporal basis for subsequent biomathematical fatigue assessment models of fatigue accumulation and recovery. Task execution time refers to the point in time when the user begins task execution, receives system prompts, completes key operations, or enters a specific task phase, used to determine the correspondence between fatigue state and task phase. Continuous wakefulness duration refers to the length of time the user remains awake from the end of their most recent sleep to the current task execution time, characterizing the degree of accumulated sleep pressure. Sleep duration refers to the effective sleep time during the user's most recent sleep cycle, characterizing the degree of sleep recovery. The task duration parameter refers to the duration of the current task phase, the cumulative on-duty time, the passive monitoring duration, or the continuous operation duration, used to characterize the impact of task exposure on fatigue accumulation. The above data is structured according to the operator identifier, task phase identifier, and unified timestamp to form fatigue time-series data, ensuring that each assessment moment corresponds to the most recent sleep status, current wakefulness duration, and task execution duration.

[0030] Fatigue time series data can be represented as: in, This represents the fatigue time series data corresponding to the current evaluation moment; This indicates the duration of continuous wakefulness corresponding to the current assessment time, which is determined by the difference between the current assessment time and the time when the most recent sleep ended. This indicates the duration of the most recent effective sleep, which is obtained by accumulating the effective sleep segments between the start and end times of the most recent sleep. This indicates the time when the task is executed, which is determined by the time the task begins, the system prompts, or the task phase transition. This represents the task duration parameter, which is determined by the difference between the current evaluation time and the task start time or the duration of the current task phase. This represents the sleep-wake state identifier corresponding to the current assessment moment, used to distinguish between the awake state, the sleep state, and the sleep-to-wake transition state. This expression integrates sleep recovery, wakefulness accumulation, and task exposure into the same temporal framework through a unified temporal structure, enabling subsequent fatigue components to be correlated and calculated at the same assessment moment.

[0031] When identifying whether an operator is awake or asleep based on fatigue time-series data, a comprehensive judgment can be made based on sleep-wake state markers, EEG rhythm characteristics, eye movement event density, postural activity intensity, and task operation logs. A awake state refers to a state where the operator is capable of performing tasks, receiving system prompts, and responding; a sleep state refers to a state where the operator is in a continuous resting or sleep recovery process. The sleep homeostasis component characterizes the loss of alertness during wakefulness as the duration of continuous wakefulness increases, while the sleep homeostasis recovery component characterizes the recovery of alertness during sleep as the duration of sleep increases. If the current assessment time is during wakefulness, the homeostasis level at the end of the most recent sleep episode is used as the initial level, and the sleep homeostasis component is calculated based on the duration of continuous wakefulness. If the current assessment time is during sleep, the homeostasis level at the time of the most recent fall-off of sleep is used as the initial level, and the sleep homeostasis recovery component is calculated based on the duration of sleep. This processing allows the fatigue assessment results to simultaneously reflect the fatigue accumulation caused by wakefulness and the fatigue recovery caused by sleep.

[0032] The homeostatic components of sleep during wakefulness can be represented as: in, This represents the sleep homeostasis component at the current assessment time; This represents the low-level asymptote, used to define the lower limit to which the homeostatic components of sleep gradually approach after prolonged wakefulness. Its value can be set to 2.0 to 3.0. It represents the initial steady-state level corresponding to the time when the most recent sleep ended, and it is determined by the sleep steady-state recovery component at the end of sleep; This represents the sobriety attenuation coefficient, which takes a value less than 0 and can be set to -0.06 to -0.01. This represents the duration of continuous alertness at the current assessment moment. This formula describes the depletion of alertness level due to continuous alertness using an exponential decay relationship; the longer the duration of continuous alertness, the higher the alertness level. The closer This indicates that the operator's basic alertness is lower.

[0033] The sleep homeostasis recovery component during sleep can be expressed as: in, This indicates the sleep homeostasis recovery component at the current assessment moment; This represents the high-level asymptote, used to define the upper limit to which the steady-state recovery level gradually approaches after sufficient sleep. The value can be set from 13.0 to 15.0. The initial steady-state level corresponds to the most recent time of falling asleep, and is determined by the sleep homeostasis components before falling asleep; g represents the sleep recovery coefficient, which takes a value less than... It can be set to to ; This represents the duration of sleep corresponding to the most recent sleep episode. This formula describes the restorative effect of sleep on alertness levels using an exponential recovery relationship; the longer the sleep duration, the better. The closer This indicates that the more fully the sleep is restored.

[0034] When calculating the first and second circadian rhythm components based on the task execution time and task duration parameters, the task execution time can be converted into the phase position of the current assessment time within the intraday cycle, and the phase position can be corrected by combining the task duration parameter. The first circadian rhythm component is used to characterize the principal periodic alertness fluctuations caused by the operator's 24-hour intrinsic biological rhythm, while the second circadian rhythm component is used to characterize the secondary periodic alertness fluctuations caused by the operator's 12-hour semi-diurnal rhythm. The task execution time is used to determine whether the operator performs the task during the day, night, early morning trough, or other rhythmic positions, and the task duration parameter is used to reflect the superimposed effects of long-term continuous tasks on rhythmic phase perception and fatigue accumulation. Calculating the first and second circadian rhythm components together avoids the insufficient expression of afternoon troughs, late night troughs, and fatigue changes during long-term duty due to using only a single 24-hour rhythm.

[0035] The first 24-hour rhythm component can be represented as: in, This represents the first 24-hour rhythm component at the current assessment time; This represents the median of the first 24-hour rhythm components, used to determine the baseline level of rhythm fluctuations, and can be set to -1.0 to 1.0; The amplitude of the first 24-hour rhythm component is used to determine the intensity of the influence of the 24-hour main circadian rhythm on alertness, and its value can be set from 1.5 to 3.5. This indicates the current assessment time within a daily cycle, which is calculated from the task execution time. This represents the phase correction coefficient for mission duration, and its value can be set to... to ; represents the task duration parameter; p represents the circadian rhythm phase, used to characterize the offset of an individual's rhythm peak or trough relative to intraday time, with a value ranging from 0 to 24. This formula simulates the periodic regulation of alertness by the 24-hour rhythm through a cosine function, and through... Incorporate the impact of long-duration tasks on rhythmic fatigue perception into phase position.

[0036] The second circadian rhythm component can be represented as: in, This represents the second diurnal rhythm component at the current assessment time; This represents the median of the second diurnal rhythm components, used to determine the baseline level of semi-diurnal rhythm fluctuations, and can be set to -1.0 to 0. The amplitude of the second diurnal rhythm component is used to determine the intensity of the influence of the twelve-hour sub-cycle rhythm on alertness, and its value can be set from 0.2 to 1.0. This indicates the current assessment time position within a daily cycle; This represents the phase correction coefficient for mission duration, and its value can be set to... to ; This represents the task duration parameter; p represents the circadian rhythm phase; This represents the phase compensation value of the second circadian rhythm component relative to the first circadian rhythm component, and can be set to 1 to 5. This formula uses a twelve-hour cycle to supplement the semi-diurnal fluctuations that the main cycle rhythm cannot express, enabling the model to characterize the secondary decrease in alertness that occurs during silent duty, night shifts, and long-duration tasks.

[0037] When identifying the transition from sleep to wakefulness based on fatigue time-series data, the awakening time can be determined by the time point when the sleep-wake state indicator switches from sleep to wakefulness, and the time difference between the current assessment time and the awakening time can be calculated. The sleep inertia component characterizes the short-term impact on cognitive response, judgment speed, and operational accuracy immediately after the operator transitions from sleep to wakefulness, even though they are already awake. This component mainly functions in the initial stage of awakening and gradually diminishes with increasing wakefulness time. For scenarios such as shift work, taking over a shift after short naps, and waking up at night to perform tasks, the sleep inertia component can effectively reflect the potential operational risks in the initial awakening stage.

[0038] The sleep inertial component can be expressed as: in, This represents the sleep inertia component at the current assessment moment; This indicates the degree of decrease in alertness during the initial stage of awakening, with a value less than 0, and can be set to -8.0 to -2.0; This represents the sleep inertial decay coefficient, with a value less than 0, and can be set to -3.0 to -0.5; This represents the time elapsed since the current assessment moment and the most recent awakening moment. This formula describes the rapid decay process of sleep inertia using an exponential decay relationship. When smaller, It significantly weakens alertness, as Increase, The value gradually approaches 0, indicating that the delayed effects after awakening gradually disappear.

[0039] When performing individualized corrections on the sleep homeostasis component, first circadian rhythm component, second circadian rhythm component, and sleep inertia component based on multidimensional feature parameters, one can first screen for EEG features, eye movement features, heart rate variability features, respiratory stability features, postural stability features, and behavioral response features related to fatigue state from the multidimensional feature parameters. Then, these features are compared with the individual baseline of the operator to obtain the real-time deviation. Individualized correction refers to adaptively adjusting the general biomathematical fatigue assessment model based on the long-term baseline of the operator's own physiological and behavioral characteristics, avoiding deviations in fatigue representation caused by differences in baseline heart rate, eye movement habits, EEG rhythm, and postural behavior among different operators. The model correction coefficient is used to uniformly represent the overall correction strength of the multidimensional feature parameters on the biomathematical components. When real-time multidimensional feature parameters show decreased attention, increased drowsiness, or delayed reaction, the model correction coefficient enhances the fatigue effect; when real-time multidimensional feature parameters show stable alertness, the model correction coefficient suppresses misjudgments.

[0040] The model correction coefficients can be expressed as: in, This represents the model correction coefficient at the current evaluation moment; m represents the number of multidimensional feature parameters involved in the individualized correction. Let represent the weight coefficient corresponding to the j-th multidimensional feature parameter, with a value ranging from 0 to 1, and all . The sum does not exceed 1; This represents the j-th multidimensional feature parameter at the current evaluation time; This represents the mean value of the j-th multidimensional feature parameter under the baseline state of the individual operator; This represents the fluctuation scale of the j-th multidimensional feature parameter under the baseline state of the individual operating subject; Represents the stability constant, which can be taken as... to This formula first converts each multidimensional feature parameter into the degree of deviation relative to the individual baseline, then uses the hyperbolic tangent function to limit the influence of abnormal peaks on the model correction coefficient, and finally forms a unified correction amount by superimposing weight coefficients, so that the model correction coefficient retains the real-time state changes of the individual while avoiding drastic fluctuations in fatigue characterization caused by a single abnormal signal.

[0041] When calculating the comprehensive alertness score based on the model correction coefficients and the sleep homeostasis component, the first circadian rhythm component, the second circadian rhythm component, and the sleep inertia component, the sleep homeostasis component can be used as the basic alertness support, the first and second circadian rhythm components as periodic alertness regulation, and the sleep inertia component as the short-term wakefulness lag effect. The model correction coefficients are then used to uniformly regulate the four components. The comprehensive alertness score is an intermediate state quantity used to characterize the overall alertness level of the operator at the current assessment moment. A higher score indicates that the operator is more likely to maintain better attention, judgment, and response capabilities under the current physiological rhythm and task conditions; a lower score indicates a higher risk of fatigue.

[0042] The overall alertness score can be expressed as: in, This represents the overall alertness score at the current assessment moment; Indicates the model correction coefficient; It represents the sleep homeostasis component, which is calculated from the duration of continuous wakefulness in the waking state and updated from the sleep homeostasis recovery component in the sleep state; Indicates the first diurnal rhythm component; Indicates the second diurnal rhythm component; Indicates the sleep inertia component; , , and These represent the weighting coefficients for the sleep homeostasis component, the first circadian rhythm component, the second circadian rhythm component, and the sleep inertia component, respectively. Each coefficient ranges from 0 to 1, and the sum of all four is 1. This formula unifies fatigue accumulation, circadian rhythm, and wakefulness lag into a comprehensive alertness score through a weighted summation method. Furthermore, it incorporates real-time physiological and behavioral deviations of the individual through model correction coefficients, enabling the comprehensive alertness score to possess both biorhythmic interpretability and multimodal data adaptability.

[0043] When mapping the comprehensive alertness score to generate a fatigue characterization quantity, a logical mapping can be used to convert the comprehensive alertness score into a fatigue state index within a defined range. The fatigue characterization quantity is a quantitative result used to describe the current fatigue level of the operator; its value can be set between 0 and 1, with a higher value indicating a higher degree of fatigue. Since a higher comprehensive alertness score indicates stronger alertness, the fatigue characterization quantity should have an inverse relationship with the comprehensive alertness score. Simultaneously, a task duration parameter can be introduced to appropriately increase the fatigue characterization quantity when the continuous task time is long, thereby reflecting the fatigue accumulation caused by long-term duty or continuous operation.

[0044] The fatigue characterization quantity can be expressed as: in, This represents the fatigue characterization value at the current evaluation moment, with a value ranging from 0 to 1; This indicates the fatigue reference offset, which can range from -3 to 3. This represents the inhibition coefficient corresponding to the overall alertness score, and its value is greater than 0, ranging from 0.1 to 2.0. This represents the overall alertness score; This represents the enhancement coefficient corresponding to the task duration parameter, with a value greater than 0, and can be set from 0.1 to 2.0; This represents the normalized task duration parameter, with a value ranging from 0 to 1. The formula uses a logic function to constrain the fatigue estimation results within a stable range, and also... A higher overall alertness score indicates a lower level of fatigue. The longer the task lasts, the higher the fatigue level.

[0045] When fatigue characteristics are output to a biomathematical fatigue assessment model for joint mapping, the fatigue characteristics at the current assessment moment and the fatigue time-series data at the next assessment moment can be jointly stored. This allows the biomathematical fatigue assessment model to correct the initial level of sleep homeostasis, the intensity of individualized correction, and the trend of state change at the next moment based on the fatigue results at the previous moment. Joint mapping means that fatigue characteristics are not only used as output results, but also as constraint variables for subsequent model recursion and the construction of the human-machine collaborative perception representation space, so that the fatigue state can be continuously transmitted along the time dimension. Through this process, the biomathematical fatigue assessment model can form a closed-loop assessment process, avoiding the neglect of the continuity of fatigue accumulation due to independent calculations at each assessment moment. The biomathematical fatigue assessment model is a time-series predictive model used to describe the effects of accumulated fatigue, sleep recovery, circadian rhythm fluctuations, and wakefulness lag on the operator. Its structure typically employs a coupled mechanistic and correction layer. The mechanistic layer consists of a sleep homeostasis sub-model, a circadian rhythm sub-model, and a sleep inertia sub-model, used to calculate the baseline alertness level based on parameters such as continuous wakefulness duration, sleep duration, task execution time, and task duration. The correction layer comprises a multimodal feature mapping network, used to generate individualized correction coefficients based on EEG characteristics, eye movement characteristics, heart rate variability characteristics, posture characteristics, and behavioral response characteristics, thus forming a fatigue output structure that combines physiological interpretability and individual adaptability. During model construction, sleep logs, duty logs, task logs, and multimodal observation data are first collected, and time alignment, anomaly removal, and status labeling are performed. A sleep-wake timeline is then established, and sleep homeostasis parameters, circadian rhythm parameters, and sleep inertia parameters are initialized as group prior values ​​and calibrated based on individual historical data, so that different operators have independent parameter sets. During training, a two-stage optimization approach is adopted. First, the mechanistic layer parameters are fitted using publicly available fatigue experimental data, historical duty data, or alertness test data. Then, the mechanistic layer or semi-frozen mechanistic layer is fixed, and supervised learning is used to train the correction layer to minimize the error between the predicted fatigue representation and subjective fatigue scores, reaction time delay, error rate, and continuous attention test results. At the same time, time smoothing constraints, individual consistency constraints, and scene transfer constraints are added. Finally, the model parameters are continuously corrected through online incremental updates, so that the model maintains the accuracy of fatigue assessment in different scenarios such as shift duty, silent monitoring, and high-load tasks during long-term operation.

[0046] The recursive fatigue characterization after joint mapping can be expressed as: in, This represents the recursive fatigue characterization quantity at the current evaluation moment; This represents the fatigue characterization quantity at the current evaluation moment; This represents the recursive fatigue characterization quantity at the previous evaluation time. This represents the recursive smoothing coefficient, with a value range of [value missing]. to This formula weakens misjudgments caused by single noise, short-term action abnormalities, or instantaneous physiological fluctuations by weighted fusion of the current fatigue characterization quantity and the recursive fatigue characterization quantity at the previous evaluation time. This enables the recursive fatigue characterization quantity to stably reflect the fatigue evolution trend of the operator as the task progresses, and provides continuous and reliable fatigue constraints for subsequent fusion characterization models and human-machine collaborative perception characterization spaces.

[0047] In one possible implementation, data-driven modeling processing is performed on multidimensional feature parameters to form a fusion representation model, and fatigue representation quantities are jointly mapped with the fusion representation model to construct a human-machine collaborative perception representation space. Specifically, this includes: binding multidimensional feature parameters according to timestamps, task stage identifiers, and operating subject identifiers to form a multidimensional feature parameter sequence; constructing cognitive load feature subsequences, physiological response feature subsequences, and behavioral dynamic feature subsequences based on the standardized multidimensional feature parameter sequence, and performing time-series encoding processing on each to form cognitive load encoding vectors, physiological response encoding vectors, and behavioral dynamic encoding vectors; calculating cross-modal correlation weights based on the cognitive load encoding vector, physiological response encoding vector, and behavioral dynamic encoding vector; forming a fusion representation vector based on the cross-modal correlation weights; generating fatigue gating parameters based on fatigue representation quantities, the fusion representation vector, and task load parameters, and performing modulation processing on the fusion representation vector based on the fatigue gating parameters to form a fatigue modulation representation vector; and performing joint projection processing on the fatigue modulation representation vector and the system interaction state vector to construct a human-machine collaborative perception representation space.

[0048] Specifically, when binding multidimensional feature parameters according to timestamps, task stage identifiers, and operator identifiers, the cognitive load-related parameters, physiological response-related parameters, and behavioral dynamic-related parameters extracted within each evaluation window can first be uniformly assigned to the same time reference and associated with the current task stage and the corresponding operator. The timestamp indicates the specific evaluation moment when the feature was generated; the task stage identifier indicates whether the feature corresponds to a task stage such as monitoring, confirmation, control, emergency response, silent duty, or task switching; and the operator identifier distinguishes the individual states of different personnel. Through this binding process, the multidimensional feature parameters of the same operator at different task stages can be arranged continuously in chronological order, forming a multidimensional feature parameter sequence that describes the state evolution process, avoiding data mixing between different personnel, different task stages, or different time windows.

[0049] The multidimensional feature parameter sequence can be represented as: in, This represents the multidimensional feature parameter sequence unit corresponding to the current evaluation time. The set of cognitive load characteristics at the current assessment moment consists of EEG band energy, changes in cerebral blood oxygenation, attentional maintenance characteristics, and task comprehension stress characteristics. It represents the set of physiological response characteristics at the current assessment moment, consisting of heart rate variability, skin conductance response, respiratory rhythm, eye movement response, and electromyographic response; It represents the set of dynamic behavioral characteristics at the current evaluation moment, consisting of operation response delay, action continuity, number of error corrections, interaction interruption status, and posture changes; This indicates the timestamp corresponding to the current evaluation moment; This indicates the task stage identifier corresponding to the current evaluation moment; This represents the identifier of the operating entity corresponding to the current evaluation moment. By organizing feature content, time location, task semantics, and personnel identity, this expression enables subsequent data-driven modeling to continuously analyze the task process of the same operating entity.

[0050] When standardizing multidimensional feature parameter sequences, a unified scale transformation can be performed on parameters from different sources, with different numerical ranges, and different fluctuation characteristics, based on the individual baseline of the operator and the baseline of the task stage. Standardization refers to transforming parameters that cannot be directly compared, such as EEG power, eye movement frequency, heart rate variability, operation latency, and error correction counts, into a comparable numerical range. This prevents subsequent time-series coding processing from excessively affecting model learning due to the large numerical range of a particular parameter. For continuous parameters, a standardization method based on mean and fluctuation scale can be used; for proportional parameters, interval normalization can be used; and for event-based parameters, window statistics followed by normalization can be used. After standardization, the multidimensional feature parameter sequence can maintain its original trend while possessing the scale consistency required for cross-modal fusion.

[0051] Standardization can be represented as: in, This represents the j-th standardized feature parameter at the current evaluation time; This represents the j-th original feature parameter at the current evaluation time; This represents the baseline mean of the j-th feature parameter under the corresponding operation subject and the corresponding task stage; This represents the baseline fluctuation scale of the j-th feature parameter under the corresponding operational subject and the corresponding task stage; Indicates the task phase identifier; u indicates the operation entity identifier; Represents the stability constant, which can be taken as... to This formula constrains the feature scaling process simultaneously by using individual baselines and task stage baselines, ensuring that the standardized feature parameters reflect both the relative degree of anomaly and reduce biases caused by individual and task stage differences.

[0052] When constructing cognitive load feature subsequences, physiological response feature subsequences, and behavioral dynamic feature subsequences based on standardized multidimensional feature parameter sequences, they can be grouped according to the state source reflected by the features. The cognitive load feature subsequence describes changes in brain functional load during task comprehension, attention maintenance, information processing, and decision-making, and may include features such as EEG theta wave energy, alpha wave inhibition level, beta wave activity level, cerebral blood oxygenation changes, fixation load, and task information density. The physiological response feature subsequence describes changes in the operator's autonomic nervous system, cardiopulmonary response, and physical arousal level, and may include features such as heart rate variability, skin conductance response, respiratory rate, respiratory stability, electromyographic tension, and pupillary changes. The behavioral dynamic feature subsequence describes the explicit interaction between the operator and the system, and may include features such as response latency, key press intervals, manipulation trajectory, action continuity, number of error corrections, interaction interruption states, and posture changes. Through this grouping method, the fusion representation model can learn the changing patterns of brain functional states, physiological arousal states, and behavioral execution states separately.

[0053] The characteristic subsequences of cognitive load, physiological response, and behavioral dynamics can be represented as follows: in, This represents the characteristic subsequence of cognitive load consisting of the current assessment time up to the previous q assessment times; This represents the physiological response feature subsequence consisting of the current assessment time up to the previous q assessment times; This represents the subsequence of behavioral dynamic features formed from the current evaluation time to the previous q evaluation times; This represents the time backtracking length, and its value can be set to 1 to... ; , and These represent the standardized cognitive load feature vector, physiological response feature vector, and behavioral dynamic feature vector at the current assessment moment, respectively. The above expressions preserve the continuous relationship between the current state and historical states using a sliding window approach, enabling subsequent temporal coding processing to identify the evolutionary trends of fatigue accumulation, attentional decay, and response lag.

[0054] When performing temporal coding on cognitive load feature subsequences, physiological response feature subsequences, and behavioral dynamic feature subsequences respectively, a corresponding temporal coding function can be configured for each type of feature subsequence. This allows different feature sources to be converted into a unified-dimensional coding vector while preserving their own variation patterns. Temporal coding refers to compressing the feature changes within a continuous evaluation window, preserving both the feature intensity at the current moment and the trend, duration, and fluctuation pattern of changes over several past moments. The cognitive load coding vector represents the continuous state of the operator in cognitive processing, the physiological response coding vector represents the continuous state of the operator in physiological arousal, and the behavioral dynamic coding vector represents the continuous state of the operator in interactive behavior. Temporal coding functions can be implemented using gated recurrent networks, temporal convolutional networks, self-attention coding networks, or combinations thereof.

[0055] The cognitive load encoding vector, physiological response encoding vector, and behavioral dynamic encoding vector can be represented as follows: in, This represents the cognitive load encoding vector at the current assessment moment; This represents the physiological response encoding vector at the current assessment moment; This represents the dynamic encoding vector of the behavior at the current evaluation moment; Represents the temporal coding function for cognitive load; Represents a physiological response timing coding function; Represents a dynamic timing coding function for behavior; , and These represent the trainable parameters of the corresponding temporal coding functions. The above expressions, by modeling different feature subsequences separately, enable the fusion representation model to avoid mixing physiological noise, behavioral actions, and cognitive load into a single, uninterpretable feature, while preserving the independent contributions of different state sources.

[0056] When calculating cross-modal association weights based on cognitive load encoding vectors, physiological response encoding vectors, and behavioral dynamics encoding vectors, the importance of each state source at the current assessment moment can be determined by utilizing the consistency relationship between the current task stage, task load, and each encoding vector. Cross-modal association weights are dynamic weights used to measure the contribution of cognitive load, physiological response, and behavioral dynamics to the assessment of the current operational state. If the current task stage primarily involves continuous monitoring and information judgment, the weight of the cognitive load encoding vector can be increased; if the current task stage involves significant physiological stress or fatigue accumulation, the weight of the physiological response encoding vector can be increased; if the current task stage requires high-frequency manipulation or rapid confirmation, the weight of the behavioral dynamics encoding vector can be increased. Through cross-modal association weights, the fusion representation model can dynamically adjust the participation level of different feature sources according to task stage and state changes.

[0057] Cross-modal association weights can be expressed as: in, represents the cross-modal association weight corresponding to the r-th class encoding vector at the current evaluation time, with a value ranging from 0 to 1, and the sum of the cross-modal association weights of all classes is 1; r represents the feature source type, with values ​​including , and These correspond to cognitive load, physiological response, and behavioral dynamics, respectively. Represents the r-th class encoding vector; The task stage encoding vector representing the current evaluation moment is obtained by mapping the task stage identifier, task complexity, and interaction requirements. The weight mapping parameters represent the r-th class encoding vector; The weight mapping parameters represent the encoding vectors at each task stage; This represents the weight bias parameter for the r-th class; This represents the cross-modal weight projection parameters. This formula calculates the dynamic contributions of different encoding vectors through an attention normalization mechanism, enabling the fusion representation model to automatically select more reliable feature sources based on the current task semantics and multimodal state.

[0058] When forming a fusion representation vector from cognitive load encoding vector, physiological response encoding vector, and behavioral dynamics encoding vector based on cross-modal correlation weights, each encoding vector can be weighted and superimposed according to its cross-modal correlation weights, and then unified to a preset dimension as needed through linear or nonlinear mapping. The fusion representation vector is a unified vector that comprehensively represents the cognitive load state, physiological response state, and behavioral dynamics state. Its function is to provide a unified expression of the operational subject's state for subsequent fatigue modulation and human-machine collaborative perception representation space construction. Compared with simple concatenation, the fusion method based on cross-modal correlation weights can reduce the influence of noisy modalities or irrelevant features and enhance state sources more relevant to the current task stage.

[0059] The fusion representation vector can be expressed as: in, This represents the fusion representation vector at the current evaluation moment; This represents the cross-modal association weights corresponding to the r-th class encoding vector; Let represent the encoding vector of the r-th class. This formula forms a unified representation through weighted summation, enabling the fusion of cognitive load, physiological response, and behavioral dynamics in the same vector space, and the contribution of each class of state sources is dynamically controlled by cross-modal association weights.

[0060] When generating fatigue gating parameters based on fatigue characterization quantities, fused characterization vectors, and task load parameters, the fatigue characterization quantities can be used as fatigue degree constraints, the task load parameters as task pressure constraints, and the fused characterization vector as the basic expression of the current state, all mapped together to fatigue gating parameters. Fatigue gating parameters are control parameters used to adjust the relationship between fatigue-sensitive and non-fatigue-stable components in the fused characterization vector; their values ​​can be set between 0 and 1. Task load parameters represent the current task complexity, information density, response time, risk level, and interaction intensity. When both the fatigue characterization quantity and the task load parameter are high, the fatigue gating parameters will enhance the modulation effect on fatigue-related features; when the fatigue characterization quantity or the task load parameter is low, the fatigue gating parameters will reduce over-modulation of the fused characterization vector.

[0061] The fatigue gating parameter can be expressed as: in, This represents the fatigue gating parameter at the current evaluation moment, with the elements in the vector ranging from 0 to 1; This represents a logical activation function used to constrain the gating result to between 0 and 1; This represents the fatigue gating weight parameter; Indicates the fatigue gating bias parameter; Represents the fusion representation vector; This represents a fatigue characterization quantity, with a value ranging from 0 to 1; This represents the task load parameter, with a value range of 0 to 1; This represents a vector concatenation operation. This formula maps the current fusion state, fatigue level, and task load of the operating entity simultaneously to the gating space, enabling the fatigue effect to be differentially adjusted according to the feature dimensions.

[0062] When modulating the fused representation vector based on fatigue gating parameters, the fatigue gating parameters can be applied element-wise to the fused representation vector, and the individual baseline representation vector can be introduced as a stable reference to form a fatigue-modulated representation vector. The fatigue-modulated representation vector refers to the expression of the operator's state after superimposing the fatigue effect on the fused representation vector. Its function is to ensure that the subsequent human-machine collaborative perception representation space not only reflects the current cognitive, physiological, and behavioral states, but also reflects the significance of these states under fatigue conditions. The individual baseline representation vector is used to represent the normal characteristics of the operator in non-fatigue or stable task phases. When the fatigue gating parameter is low, the fatigue-modulated representation vector is closer to the individual baseline representation vector; when the fatigue gating parameter is high, the fatigue-modulated representation vector highlights the fatigue-sensitive changes in the current fused representation vector.

[0063] The fatigue modulation characterization vector can be represented as: in, This represents the fatigue modulation representation vector at the current evaluation moment; Indicates fatigue gating parameters; Represents the fusion representation vector; 1 represents the individual baseline representation vector corresponding to the operating subject, which is generated by the multidimensional feature parameters of the operating subject under the historical stable state; 1 represents a one vector with the same dimension as the fatigue gating parameter. This indicates element-wise multiplication. The formula adaptively adjusts the current fused representation vector and the individual baseline representation vector using fatigue gating parameters, enabling the fatigue-modulated representation vector to simultaneously retain both the individual's normal reference and current fatigue-related changes.

[0064] When performing joint projection processing based on the fatigue modulation representation vector and the system interaction state vector, the operator-side state and the system-side state can be placed in the same mapping space, while preserving their differences and cooperative relationships. The system interaction state vector is a vectorized representation of the interaction requirements and feedback states proposed by the system to the operator at the current evaluation moment. It can include system prompt intensity, prompt frequency, task objectives, response time limits, feedback delays, automation intervention levels, alarm levels, interface complexity, and task constraint states. Joint projection processing involves connecting the fatigue modulation representation vector and the system interaction state vector, calculating their differences and interactions, and then mapping them to a unified representation space through a projection function. This processing ensures that the human-machine collaborative perception representation space includes not only the operator's fatigue state but also the system interaction requirements, thereby supporting the comprehensive calculation of subsequent collaborative consistency and response efficiency.

[0065] The human-machine collaborative perception representation space can be represented as: in, This represents the human-machine collaborative perception representation space at the current evaluation moment; To represent a nonlinear mapping function, a hyperbolic tangent function, a modified linear function, or a multilayer sensing mapping function can be used. Indicates the joint projection weight parameters; Indicates the joint projection bias parameters; Represents the fatigue modulation characterization vector; Represents the system interaction state vector; This indicates the difference between the state on the operator side and the state on the system side; This indicates the coordination relationship between the state on the operator side and the state on the system side; Indicates a vector concatenation operation; This indicates element-wise multiplication. By simultaneously expressing the subject's state, system state, state differences, and state coordination, this formula enables the human-machine collaborative perception representation space to reflect the dynamic matching degree between the operating subject and the system's interaction requirements under the influence of fatigue.

[0066] Once the human-machine collaborative perception representation space is constructed, it can serve as a unified representation basis for subsequent state analysis processes, used to calculate collaborative consistency, response efficiency, interaction stability, and the degree of fatigue impact. Collaborative consistency can be determined based on the similarity and difference between the operator's state and the system's state; response efficiency can be determined based on necessary response events, response delays, error correction, and system feedback loop conditions; and interaction stability can be determined based on the fluctuation of the representation space within the continuous evaluation window. Since the human-machine collaborative perception representation space simultaneously includes the temporal expression of multidimensional feature parameters, the modulation results of fatigue representation quantities, and the system interaction state, the subsequent state analysis results can avoid being solely determined by physiological fatigue or behavioral frequency, but rather reflect the operator's true adaptation state in specific system interaction scenarios.

[0067] In one possible implementation, the system comprehensively calculates the coordination consistency and response efficiency during the interaction between the operator and the system based on the human-machine collaborative perception representation space, generating state analysis results. Specifically, this includes: binding the human-machine collaborative perception representation space with system interaction data to form interaction evaluation data; extracting the operator-side state vector and system-side state vector from the interaction evaluation data and calculating coordination consistency parameters; calculating response efficiency parameters based on the system prompt time, operation response time, system feedback time, and error correction data extracted from the interaction evaluation data; performing fatigue impact correction on the coordination consistency parameters and response efficiency parameters based on fatigue characterization quantities to form coordination correction parameters and efficiency correction parameters; calculating the fluctuation degree of coordination correction parameters, efficiency correction parameters, and fatigue characterization quantities within a continuous evaluation window based on the interaction evaluation data to form interaction stability parameters; calculating comprehensive operation state data based on the coordination correction parameters, efficiency correction parameters, interaction stability parameters, and fatigue characterization quantities; and performing hierarchical judgment based on the comprehensive operation state data, coordination correction parameters, and efficiency correction parameters to generate state analysis results.

[0068] Specifically, when binding the human-machine collaborative perception representation space with system interaction data, a correspondence can be established according to a unified timestamp, task stage identifier, operator identifier, and interaction event identifier. This ensures that the human-machine collaborative perception representation space within each evaluation window corresponds to system prompts, operation responses, system feedback, and error correction within the same time range. System interaction data refers to the interaction records generated by the system during task execution, including system prompt content, system prompt time, operation response time, system feedback time, operation result, error correction data, alarm level, task objective, and automation intervention status. Interaction evaluation data refers to structured data that binds the operator's state expression with system interaction records. Its function is to organize the fatigue-modulated operator's state, system-side interaction requirements, and actual interaction results in the same data unit, providing a basis for the joint calculation of collaborative consistency and response efficiency.

[0069] Interactive evaluation data can be represented as: in, This represents the interactive evaluation data corresponding to the current evaluation moment; This represents the human-machine collaborative perception representation space at the current evaluation moment; This represents the system interaction data within the current evaluation window; This indicates the timestamp corresponding to the current evaluation moment; This indicates the task stage identifier corresponding to the current evaluation moment; Indicates the identifier of the operating entity corresponding to the current assessment moment; This represents the identifier of the interaction event within the current evaluation window. By binding the representation space, system interaction records, time information, task semantics, and personnel identity together, this expression enables subsequent calculations to accurately locate the interaction behavior corresponding to a certain state change.

[0070] When extracting the operator-side state vector and system-side state vector from the interactive evaluation data, dimensions related to the operator's state and dimensions related to the system's interaction requirements can be separated from the human-machine collaborative perception representation space. The operator-side state vector characterizes the operator's cognitive load, physiological response, behavioral dynamics, and fatigue modulation state at the current evaluation moment, reflecting whether the operator possesses the physiological and behavioral basis to complete the current interactive task. The system-side state vector characterizes the system's prompt intensity, response time, task complexity, feedback status, automation intervention level, and alarm level at the current evaluation moment, reflecting the system's interaction requirements for the operator. By simultaneously extracting both types of vectors, the collaborative consistency calculation can be transformed from a simple fatigue judgment into a matching judgment between the operator's state and system requirements.

[0071] The operator-side state vector and the system-side state vector can be represented as: in, This represents the operational subject-side state vector at the current evaluation moment; This represents the system-side state vector at the current evaluation moment; This represents the state mapping function on the operator side, used to extract the operator's state representation from the human-machine collaborative perception representation space; This represents the system-side state mapping function, used to extract the system interaction requirements from the human-machine collaborative perception representation space; The trainable parameters represent the state mapping function on the operator side; These represent the trainable parameters of the system-side state mapping function; This represents the human-machine collaborative perception representation space at the current evaluation moment. The above mapping enables the operator-side state and the system-side state to have comparable vector representations within the same evaluation moment.

[0072] The coordination consistency parameter represents the degree of matching between the operator's state and the system's state, and can simultaneously consider their directional similarity, spatial deviation, and the completion of the interaction goal. Directional similarity determines whether the operator's current state change direction aligns with the system's interaction requirements; spatial deviation determines whether the distance between them in the representation space is excessive; and the completion of the interaction goal determines whether the operator's actual operation result meets the system's expectations. The coordination consistency parameter increases when the operator's state vector and the system's state vector are close in direction and small in distance, and the operation result matches the system's interaction goal; conversely, the coordination consistency parameter decreases when the operator's state deviates from the system requirements or the operation result does not meet the goal requirements.

[0073] The consensus parameter can be expressed as: in, This represents the consensus parameter at the current evaluation moment, with a value ranging from 0 to 1; This represents a logical activation function used to constrain the calculation result to a range of 0 to 1; Represents the state vector on the operator's side; Represents the system-side state vector; Represents the stability constant, which can be taken as... to ; This represents the interaction target matching parameter, with a value ranging from 0 to 1, and is determined by the matching relationship between the operation result and the system interaction target; This represents the directional similarity weight coefficient, with a value ranging from 0 to 1; This represents the spatial deviation penalty coefficient, with a value ranging from 0 to 1; This represents the interaction target matching weight coefficient, with a value ranging from 0 to 1, and satisfying the following conditions: This formula enhances the matching state through a directional similarity term, penalizes state deviations through a spatial deviation term, and supplements result constraints through an interactive target matching term, enabling the collaborative consistency parameter to simultaneously reflect process matching and result matching.

[0074] When extracting system prompt moments, operation response moments, system feedback moments, and error correction data from interaction evaluation data, the starting point, response point, closed-loop point, and correction point of each interaction event can be identified first. The system prompt moment refers to the time when the system issues a display prompt, voice prompt, alarm prompt, or control suggestion to the operator. The operation response moment refers to the time when the operator generates a valid key press, touch, confirmation, manipulation, or voice feedback. The system feedback moment refers to the time when the system confirms, executes, rejects, or provides feedback on the operator's response. Error correction data refers to data on erroneous operations, omissions, duplicate operations, undoing and redoing, or secondary confirmations that occur during the interaction process. By extracting the above timing and error information, response efficiency parameters can be established based on the actual interaction closed loop, rather than solely relying on the number of operations or interaction frequency.

[0075] Response efficiency parameters characterize the timeliness, closed-loop nature, and accuracy of an operator's effective interaction after receiving system prompts. Timeliness is represented by the response delay between the system prompt and the operation response time; closed-loop nature is represented by the feedback loop duration between the system prompt and the system feedback time; and accuracy is represented by error correction data. Response efficiency parameters increase when response delay is short, feedback loop duration is short, and error corrections are infrequent; conversely, they decrease when there is prolonged delay, extended feedback loop duration, or frequent error corrections.

[0076] The response efficiency parameter can be expressed as: in, This parameter represents the response efficiency at the current evaluation moment, and its value ranges from 0 to 1. This indicates the response delay between the system notification time and the operation response time; This indicates the reference response latency corresponding to the current task phase, which is determined by historical stable interaction records or task specification requirements. This indicates the feedback loop duration between the system prompt time and the system feedback time. This indicates the reference feedback loop duration corresponding to the current task phase; Indicates the number of error corrections within the current evaluation window; This indicates the reference number of error corrections for the current task phase; This represents the weighting coefficient corresponding to the response delay, with a value ranging from 0 to 1; The weighting coefficient represents the duration of the feedback loop, and its value ranges from 0 to 1. The weighting coefficient represents the number of error corrections, ranging from 0 to 1, and satisfies the following conditions: ; This represents the stability constant. This formula uses an exponential decay method to uniformly transform the increase in time delay, the lengthening of the closed-loop circuit, and the increase in error correction into a decrease in response efficiency, allowing the response efficiency parameter to simultaneously reflect both response speed and operational quality.

[0077] When applying fatigue-based correction to coordination consistency and response efficiency parameters, the degree of fatigue of the operator can be determined first based on the fatigue characteristics, and then the intensity of the fatigue's impact on interaction performance can be determined by combining fatigue sensitivity deviation data. Fatigue-based correction involves introducing fatigue state constraints into the coordination consistency and response efficiency parameters, enabling the model to distinguish between normal operational fluctuations and fatigue-induced decreased coordination, delayed response, or increased errors. Fatigue sensitivity deviation data can be obtained by fusing EEG fatigue characteristics, eye closure characteristics, abnormal reaction time, postural sluggishness characteristics, and heart rate variability deviation characteristics, and is used to determine whether fatigue characteristics have already had a significant impact on interaction performance.

[0078] The fatigue sensitivity deviation parameter can be expressed as: in, The parameter represents the fatigue sensitivity deviation at the current evaluation moment, with a value ranging from 0 to 1; m represents the number of features involved in the fatigue sensitivity deviation calculation. This represents the weight coefficient corresponding to the j-th fatigue-sensitive feature, with a value ranging from 0 to 1, and all values ​​are equal. The sum of them is ; This represents the j-th fatigue-sensitive feature at the current evaluation time; This represents the baseline mean of the j-th fatigue sensitivity feature under the stable state of the corresponding operating entity; This represents the baseline fluctuation scale of the j-th fatigue sensitivity feature under the stable state of the corresponding operating entity; Represents the stability constant; This represents the logistic activation function. The formula quantifies the deviation of fatigue-sensitive characteristics from the individual baseline, ensuring that fatigue impact correction does not rely solely on fatigue characterization quantities, but rather incorporates real-time performance anomalies for judgment.

[0079] The co-correction parameter and the efficiency correction parameter can be expressed as: in, This indicates the co-correction parameters at the current evaluation moment; This represents the efficiency correction parameter at the current evaluation moment; Indicates the parameters of collaborative consistency; This represents the response efficiency parameter; This represents a fatigue characterization quantity, with a value ranging from 0 to 1; This represents the fatigue sensitivity deviation parameter, with a value range of 0 to 1; The fatigue impact coefficient corresponding to the coordination and consistency is represented, and its value ranges from 0 to 1; The fatigue influence coefficient, representing the response efficiency, ranges from 0 to 1. The above formula is derived from... and The product of the values ​​determines the intensity of the fatigue effect. When the fatigue characterization is high and the fatigue sensitivity deviation is significant, the synergistic correction parameter and the efficiency correction parameter are reduced. When the fatigue characterization is low or the fatigue sensitivity deviation is not significant, the correction amplitude is reduced, thereby avoiding excessive punishment of normal state fluctuations.

[0080] When calculating the fluctuations of the collaborative correction parameters, efficiency correction parameters, and fatigue characterization quantities within a continuous evaluation window based on interactive evaluation data, several continuous evaluation windows can be selected prior to the current evaluation moment to calculate the variation amplitudes of the collaborative correction parameter sequence, efficiency correction parameter sequence, and fatigue characterization quantity sequence, respectively. The interactive stability parameter characterizes whether the interaction process between the operator and the system remains stable over a period of time. If the collaborative correction parameters, efficiency correction parameters, and fatigue characterization quantities fluctuate drastically in a short period, it indicates that the state analysis results may be affected by sudden task changes, short-term interference, or physiological abnormalities; if the three change smoothly, it indicates that the state is more stable and reliable. By introducing the interactive stability parameter, the impact of accidental anomalies at a single evaluation moment on the final state analysis results can be reduced.

[0081] The interaction stability parameter can be expressed as: in, The interaction stability parameter represents the current evaluation time and ranges from 0 to 1. This represents the variance of the collaborative correction parameters from the current evaluation time to the previous r evaluation times; This represents the variance of the efficiency correction parameter from the current evaluation time to the previous r evaluation times; This represents the variance of the fatigue characterization from the current evaluation time to the previous r evaluation times; This indicates the length of the continuous evaluation window, and its value can be set from 2 to 30. This represents the weighting coefficient corresponding to the coordinated fluctuation, with a value ranging from 0 to 1; The weighting coefficient corresponding to efficiency fluctuations ranges from 0 to 1. The weighting coefficient representing fatigue fluctuation ranges from 0 to 1 and satisfies the following conditions: This formula measures state fluctuations by the variance within a continuous window. The greater the fluctuation, the lower the interaction stability parameter, indicating that the current state analysis results need to be carefully judged in conjunction with stability.

[0082] When calculating comprehensive operational status data based on collaborative correction parameters, efficiency correction parameters, interaction stability parameters, and fatigue characterization quantities, the degree of collaborative matching, response completion efficiency, and interaction stability can be considered as positive factors, while fatigue characterization quantities can be considered as negative factors. Comprehensive operational status data is a quantitative result describing the overall operational capability and fatigue risk of the operator in the current task phase. Collaborative correction parameters reflect whether the operator's state matches system requirements; efficiency correction parameters reflect whether the operator can complete interactions promptly and accurately; interaction stability parameters reflect whether the state remains stable; and fatigue characterization quantities reflect the degree to which fatigue weakens operational capability. Through comprehensive calculation, the results of state analysis can be avoided from being overly determined by a single indicator.

[0083] The overall operational status data can be represented as follows: in, This represents the comprehensive operational status data at the current assessment moment; Indicates the co-correction parameters; Indicates the efficiency correction parameter; Indicates the interaction stability parameter; This represents a fatigue characterization quantity; This represents the weighting coefficients corresponding to the collaborative correction parameters, with values ​​ranging from 0 to 1; This represents the weighting coefficient corresponding to the efficiency correction parameter, with a value ranging from 0 to 1; This represents the weighting coefficient corresponding to the interaction stability parameter, with a value ranging from 0 to 1; This represents the deduction weighting coefficient corresponding to the fatigue characterization quantity, with a value ranging from 0 to 1, and satisfying the following conditions: This formula positively accumulates interaction performance and state stability, and negatively deducts fatigue level, so that the comprehensive operational state data can directly reflect the overall ability of the operator to work collaboratively with the system under the influence of fatigue.

[0084] When performing tiered judgments based on comprehensive operational status data, collaborative correction parameters, and efficiency correction parameters, multiple state thresholds can be determined according to the safety requirements of the current task stage and historical stable states, and the judgment is made in conjunction with the minimum safety requirements of the collaborative correction parameters and efficiency correction parameters. State analysis results can include normal collaborative state, mild fatigue impact state, moderate fatigue impact state, and high-risk fatigue mismatch state. If the comprehensive operational status data is high, and both the collaborative correction parameters and efficiency correction parameters meet the requirements of the task stage, it is judged as a normal collaborative state; if the comprehensive operational status data decreases slightly, the fatigue characterization quantity increases, but the collaborative correction parameters are still within an acceptable range, it is judged as a mild fatigue impact state; if the comprehensive operational status data decreases significantly, and the efficiency correction parameters continue to decrease or error corrections increase, it is judged as a moderate fatigue impact state; if the comprehensive operational status data is below the minimum threshold, or the collaborative correction parameters and efficiency correction parameters are both below the safety threshold, it is judged as a high-risk fatigue mismatch state.

[0085] The classification can be expressed as: in, This indicates the state analysis result at the current evaluation moment; Indicates a normal collaborative state; This indicates a state affected by mild fatigue. This indicates a state of moderate fatigue. This indicates a high-risk fatigue mismatch state; This represents comprehensive data on operational status. Indicates the first state threshold; Indicates the second state threshold; Represents the third state threshold, and satisfies ; Indicates the co-correction parameters; Indicates the efficiency correction parameter; Indicates the collaborative correction safety threshold; This represents the efficiency correction safety threshold. The formula determines the overall state level using comprehensive operational state data and sets safety constraints through collaborative correction parameters and efficiency correction parameters. This ensures that the state analysis results reflect the overall trend while preventing misjudgments of acceptable states even when key interactive capabilities fall below safety requirements.

[0086] Furthermore, when deep-sea submersibles perform continuous navigation, covert monitoring, or complex control missions, the operators are in a confined space for extended periods, with weakened circadian rhythms, limited sleep during shifts, and high-pressure decision-making environments. Conventional fatigue detection methods cannot simultaneously reflect the accumulation of sleep debt and the risk of human-machine interaction mismatch. Existing response efficiency evaluation logic assumes that the more frequent the interaction events, the more frequent the operational responses, and the more continuous the action triggers, the more efficient the operator is working. However, the mission objectives during the silent monitoring phase in the deep sea are exactly the opposite. Covert monitoring or passive sonar listening emphasizes low exposure, low disturbance, and continuous observation. The operator's primary responsibility is not to frequently manipulate the system, but to maintain continuous vigilance against key acoustic signals, abnormal target signs, and changes in system status under conditions of minimal prompts, commands, and actions over extended periods. Therefore, low interaction density in this scenario does not equate to low responsiveness, but may actually be the normal working state under mission constraints.

[0087] If response efficiency parameters are still calculated using explicit behavioral indicators such as the number of operations per unit time, interaction completion frequency, action trigger rate, or interface click density, the model will obtain low behavioral activity when the system does not issue prompts for a long time, the operator does not need to actively control, and sonar monitoring is in a passive monitoring state. Since these indicators are usually associated with sluggishness, decreased attention, or fatigue in typical high-interaction tasks, the model will incorrectly apply this experience, interpreting the normal low-activity state during silent monitoring as fatigue-induced low response.

[0088] Furthermore, high levels of alertness during silent monitoring in the deep sea are not necessarily manifested as frequent key presses, frequent confirmations, or continuous operations. Rather, they are more often characterized by stable gaze, normal micro-eye movements, stable EEG attentional characteristics, no significant descent of posture, and timely response when necessary alarms occur. If the model lacks the ability to recognize these silent alertness characteristics and only measures response efficiency using the frequency of explicit operations, then even if the operator is awake, focused, and ready to respond at any time, they may still be judged as having insufficient response efficiency.

[0089] Furthermore, when the response efficiency parameter is incorrectly suppressed, subsequent comprehensive calculations often integrate it with the coordination consistency parameter, fatigue characterization, and interaction stability parameter, leading to a decrease in the overall operational status data. If the model further reinforces the low response efficiency with the fatigue characterization, a chain of misjudgments will form: low interaction density will be interpreted as low response efficiency, low response efficiency will be interpreted as increased fatigue, increased fatigue will further suppress the coordination evaluation, and ultimately, normal silent standby will be output as a fatigue-affected state or a coordination mismatch state.

[0090] Furthermore, because the silent monitoring phase often lasts for a long time, involves few external events and minimal human intervention, the system struggles to obtain sufficient positive evidence from traditional interaction logs to prove that the operator is in an effective monitoring state. Meanwhile, the deep-sea environment is characterized by its enclosed space, low light levels, low stimuli, and long shift work. Both normal silent states and fatigue-induced sluggish states may manifest as low-movement, minimal interaction, and prolonged posture maintenance in overt behavior. Therefore, relying solely on behavioral frequency weakens the model's ability to distinguish between task-constrained stillness and fatigue-induced sluggishness.

[0091] In one possible implementation, multidimensional feature parameters are extracted based on a multimodal observation dataset, and combined with a constructed biomathematical fatigue assessment model to generate a fatigue characterization quantity of the operator. Specifically, this includes: identifying whether the current task stage is in a silent monitoring phase by combining the multimodal observation dataset and task stage data, generating a silent monitoring identification result; extracting multidimensional feature parameters based on the silent monitoring identification result from the multimodal observation dataset; performing silent monitoring correction on the multidimensional feature parameters to generate a silent monitoring feature parameter set; calculating silent alertness maintenance parameters based on the silent monitoring feature parameter set; and calculating the silent alertness maintenance parameter based on sleep homeostasis components and daytime / nighttime data. The basic fatigue driving force is calculated from the nocturnal rhythm component and the sleep inertia component. Based on the silent watch identification results and silent alertness maintenance parameters, the basic fatigue driving force is corrected for silent watch, generating the silent-corrected fatigue driving force. The fatigue characterization is calculated based on the silent-corrected fatigue driving force, silent watch task load parameters, abnormal physiological deviation parameters, passive monitoring duration parameters, and silent alertness maintenance parameters. The silent consistency is verified between the fatigue characterization and the silent alertness maintenance parameters, abnormal physiological deviation parameters, and passive monitoring duration parameters, generating silent consistency parameters. The fatigue characterization is then output to the biomathematical fatigue assessment model based on the silent consistency parameters.

[0092] Specifically, when identifying whether the current task stage is in a silent monitoring phase by combining multimodal observation datasets and task stage data, we can first extract system cue events, operational action events, gaze area changes, postural activity amplitude, EEG attentional maintenance characteristics, and physiological response changes from the multimodal observation datasets. Then, we can extract sonar monitoring status, covert constraint markers, system cue strategies, necessary response event markers, and task stage risk levels from the task stage data. The silent monitoring phase refers to a state where the operator is in covert monitoring, passive listening, low command triggering, or minimal interaction standby. The task requires the operator to maintain continuous observation and responsiveness, rather than proving task execution effectiveness through frequent operations. The silent monitoring identification results are used to distinguish between a normal low-interaction state caused by the task mechanism and an abnormally low-response state caused by fatigue, ensuring that subsequent fatigue characterization calculations do not directly interpret fewer operations, fewer action triggers, or fewer system cuees as increased fatigue.

[0093] The result of silent monitoring identification can be represented as: in, This represents the silent monitoring identification result at the current evaluation moment, with a value ranging from 0 to 1. The larger the value, the more the current task stage matches the characteristics of the silent monitoring stage. This represents the covert constraint flag, which is set to 1 if there is a covert monitoring or passive sonar listening requirement, and 0 if there is no requirement. This indicates the sparsity of system prompts, calculated from the interval between system prompt events and the number of system prompt events within the current evaluation window, with a value ranging from 0 to 1. This represents the passive monitoring duration parameter, which is obtained by normalizing the current passive monitoring duration, and its value ranges from 0 to 1. This represents the normalized result of the risk level at each stage of the task, with a value ranging from 0 to 1. This represents the density of necessary response events, which is obtained by normalizing the number of events that must be responded to within the current evaluation window, and its value ranges from 0 to 1. This indicates the activity level of the operation, which is obtained by normalizing the frequency of button presses, touch controls, joystick movements, or confirmation actions, and the value ranges from 0 to 1. , , , , and These represent the weight coefficients of the corresponding parameters, with values ​​ranging from 0 to 1, and the sum of all weight coefficients is 1. This represents the logical activation function. The formula improves the silent monitoring identification result by using hidden constraints, sparse system prompts, continuous passive monitoring, and task risk, while reducing the silent monitoring identification result by increasing the density of necessary response events and the activity level of operational actions. This avoids misidentifying stages with a large number of necessary operations as silent monitoring stages.

[0094] When extracting multidimensional feature parameters based on silent vigilance recognition results from multimodal observation datasets, features reflecting silent alertness should be prioritized when the silent vigilance recognition results are high, rather than prioritizing the number of responses per unit time, action trigger frequency, or interaction completion frequency. Multidimensional feature parameters include continuous gaze stability parameters, micro-eye movement change parameters, low-frequency EEG fatigue parameters, EEG attention maintenance parameters, heart rate variability stability parameters, respiratory rhythm stability parameters, head micro-movement stability parameters, posture maintenance parameters, passive monitoring continuity parameters, and necessary response effectiveness parameters. The sustained gaze stability parameter describes whether the operator continuously focuses on the key monitoring area; the micro-eye movement change parameter distinguishes between normal scanning observation and prolonged staring rigidity; the EEG low-frequency fatigue parameter characterizes the trend of increasing drowsiness; the EEG attention retention parameter characterizes whether attentional resources are continuously invested; the heart rate variability stability parameter and respiratory rhythm stability parameter characterize whether physiological arousal is stable; the head micro-motion stability parameter and posture retention parameter distinguish between normal stillness and fatigue drooping; the passive monitoring duration parameter characterizes the cumulative time of silent tasks; and the necessary response effectiveness parameter characterizes whether the operator still has the ability to respond in a timely manner when necessary response events occur.

[0095] Multidimensional feature parameters can be expressed as: in, This represents the multidimensional feature parameters extracted at the current evaluation moment for the silent monitoring phase; This represents the stability parameter for continuous fixation, and its value ranges from 0 to 1; This represents the micro-eye movement parameter, with a value range of 0 to 1; This represents a low-frequency fatigue parameter in brainwaves, with a value range of 0 to 1. This indicates the EEG attention retention parameter, with a value range of 0 to 1; This represents the heart rate variability stability parameter, with a value ranging from 0 to 1; This represents a parameter for stabilizing respiratory rhythm, with a value ranging from 0 to 1; This represents the head micro-motion stability parameter, with a value range of 0 to 1; This represents the attitude maintenance parameter, with a value range of 0 to 1; This represents a passive monitoring parameter, with a value range of 0 to 1. This represents the effective parameter for necessary responses, with a value ranging from 0 to 1. This expression organizes multi-source information that can characterize continuous vigilance, stable observation, and necessary response capabilities during the silent monitoring phase into a unified feature vector, enabling subsequent correction and fatigue calculations to be based on the actual state of the silent task.

[0096] When performing silent guarding correction on multidimensional feature parameters, the participation intensity of various feature parameters can be adjusted according to the silent guarding recognition results. For the silent guarding phase, fewer system interaction events and fewer action triggers are task characteristics and should not be considered evidence of fatigue enhancement; therefore, the weight of interaction frequency features needs to be reduced. Sustained gaze stability, micro-eye movement changes, EEG attention maintenance, physiological rhythm stability, and posture maintenance better reflect the true level of alertness; therefore, the weight of these features needs to be increased. The silent guarding feature parameter set refers to the feature set adjusted by the silent guarding recognition results. Its function is to transform the explicit action evaluation logic in conventional high-interaction tasks into a responsive readiness evaluation logic in silent guarding tasks.

[0097] The set of silent guarding feature parameters can be represented as: in, This represents the set of silent, guarded characteristic parameters at the current evaluation moment; This represents the multidimensional feature parameters extracted at the current evaluation moment for the silent monitoring phase; This indicates the result of silent monitoring identification; This represents the silent correction weight vector. Each element in the vector takes a value between 0 and 2. A value greater than 1 indicates that the contribution of the corresponding feature parameter is enhanced, and a value less than 1 indicates that the contribution of the corresponding feature parameter is suppressed. Indicates and A uniformly one vector with consistent dimensions; This indicates element-wise multiplication. This formula largely preserves the original feature parameters when the silent-guarded recognition result is low, and adjusts the feature contribution according to the silent-guarded correction weight vector when the silent-guarded recognition result is high, thereby reducing the risk of misjudgment caused by low interaction density.

[0098] When calculating silent vigilance maintenance parameters based on a set of silent watch characteristic parameters, parameters such as sustained gaze stability, micro-eye movement changes, EEG attention maintenance, heart rate variability stability, respiratory rhythm stability, head micro-movement stability, posture maintenance, and necessary response effectiveness can be fused together. Simultaneously, a negative influence is applied to EEG low-frequency fatigue parameters and abnormal lag characteristics. Silent vigilance maintenance parameters represent whether the operator maintains effective monitoring and responsiveness under conditions of low interaction, minimal cues, and prolonged monitoring. A high silent vigilance maintenance parameter indicates stable gaze, moderate micro-eye movements, stable EEG attention characteristics, stable physiological rhythms, no posture drooping, and effective response when necessary events occur. Conversely, a low silent vigilance maintenance parameter indicates fixed gaze rigidity, reduced eye movements, increased low-frequency EEG, posture drooping, or failure to respond to necessary events.

[0099] The silent alertness maintenance parameter can be expressed as: in, This represents the silent vigilance maintenance parameter at the current assessment moment, with a value ranging from 0 to 1. The larger the value, the stronger the vigilance maintenance capability during the silent watch phase. This indicates the number of feature parameters involved in the positive silent alertness calculation; This represents the weight coefficient corresponding to the k-th positive feature parameter, with a value ranging from 0 to 1, and the sum of all positive weight coefficients does not exceed 1; This represents the k-th positive feature parameter in the set of silent, conserved feature parameters at the current evaluation moment; This represents the negative weighting coefficient corresponding to the low-frequency fatigue parameter of EEG, with a value range of 0 to 1; This indicates low-frequency fatigue parameters in brainwaves; This represents the logical activation function. The formula uses the accumulation of positive characteristic parameters to represent sustained alertness support capacity, and uses negative subtraction based on low-frequency EEG fatigue parameters to enable the silent alertness maintenance parameter to distinguish between normal silent standby and fatigue-induced retardation.

[0100] When calculating the basal fatigue drive based on the sleep homeostasis component, circadian rhythm component, and sleep inertia component, the basal components in the biomathematical fatigue assessment model can be converted into fatigue drive expressions in the same direction. The sleep homeostasis component represents the alertness basis jointly determined by sleep recovery and continuous wakefulness; the circadian rhythm component represents the periodic regulation of alertness by the current intraday phase; and the sleep inertia component represents the effect of incomplete alertness recovery shortly after wakefulness. The basal fatigue drive refers to the fatigue tendency formed by insufficient sleep, circadian rhythm troughs, and wakefulness lag before considering silent guard correction; a higher value indicates a higher risk of basal fatigue in the operator.

[0101] The basic fatigue driving force can be expressed as: in, This represents the basic fatigue driving force at the current evaluation moment, with a value ranging from 0 to 1; This represents the normalized sleep homeostasis component, with a value ranging from 0 to 1. A larger value indicates that sleep homeostasis is more conducive to maintaining alertness. This represents the normalized circadian rhythm component, with a value ranging from 0 to 1. The larger the value, the more conducive the current circadian rhythm phase is to maintaining alertness. This represents the normalized sleep inertia component, with a value ranging from 0 to 1. The larger the value, the stronger the influence of sleep inertia. The weighting coefficients corresponding to the sleep homeostasis components range from 0 to 1. The weighting coefficients corresponding to the diurnal rhythm components range from 0 to 1. The weighting coefficients corresponding to the sleep inertia component range from 0 to 1 and satisfy the following conditions: This formula unifies insufficient sleep homeostasis, unfavorable circadian rhythm, and enhanced sleep inertia into fatigue-driving directions, providing a basic fatigue estimate for subsequent silent maintenance correction.

[0102] When performing silent standby correction on the baseline fatigue drive based on silent standby identification results and silent alertness maintenance parameters, it can suppress fatigue amplification indirectly introduced by low interaction density when it is confirmed that the current state is in a silent standby phase and the silent alertness maintenance parameter is high; conversely, it can preserve or enhance the baseline fatigue drive when the current state is in a silent standby phase but the silent alertness maintenance parameter is low. Silent-corrected fatigue drive refers to the fatigue drive expression corrected based on the baseline fatigue drive based on the actual alertness maintenance capability during the silent standby phase. Its purpose is to avoid misjudging normal silent standby as fatigue, while still being able to identify attentional decline and insufficient response readiness caused by genuine fatigue.

[0103] The silent correction fatigue driving force can be expressed as: in, This indicates the silent correction fatigue drive at the current evaluation moment; Indicates the basic fatigue driving force; This represents the silent misjudgment suppression coefficient, with a value ranging from 0 to 1; This indicates the result of silent monitoring identification; Indicates the parameter for maintaining silent alertness; This represents the silent fatigue compensation coefficient, with a value ranging from 0 to 1. This formula reduces the basic fatigue drive when the silent vigilance recognition result is high and the silent alertness maintenance parameter is high, and increases fatigue drive compensation when the silent vigilance recognition result is high but the silent alertness maintenance parameter is low, thereby distinguishing between normal silent state and fatigue-induced low response state.

[0104] When calculating fatigue characterization parameters based on silent-corrected fatigue drive quantity, silent monitoring task load parameters, abnormal physiological deviation parameters, passive monitoring duration parameters, and silent vigilance maintenance parameters, the silent-corrected fatigue drive quantity can be used as the core fatigue basis. The silent monitoring task load parameters represent the impact of task stress, the abnormal physiological deviation parameters represent real-time evidence of anomalies, the passive monitoring duration parameters represent the impact of long-term low-stimulus exposure, and the silent vigilance maintenance parameters represent fatigue inhibitory factors. The silent monitoring task load parameters characterize the number of monitored targets, alarm sensitivity level, sonar information complexity, and task risk level. The abnormal physiological deviation parameters characterize the degree of abnormality in EEG, eye movement, heart rate variability, respiratory rhythm, and posture relative to the individual's silent baseline. The passive monitoring duration parameters characterize the impact of prolonged silent monitoring on fatigue accumulation. The fatigue characterization quantity is the final quantitative result representing the current fatigue level of the operator; a higher value indicates a higher degree of fatigue.

[0105] The fatigue characterization quantity can be expressed as: in, This represents the fatigue characterization value at the current evaluation moment, with a value ranging from 0 to 1; This indicates the fatigue reference offset, which can range from -3 to 3. This represents the weighting coefficient corresponding to the silent correction fatigue driving quantity, with a value range of 0 to 1; This indicates the amount of fatigue drive that is silently corrected. This represents the weighting coefficient corresponding to the load parameters of the silent monitoring task, with a value ranging from 0 to 1; This represents the load parameter for the silent monitoring task, with a value range of 0 to 1; This represents the weighting coefficient corresponding to the abnormal physiological deviation parameter, with a value range from 0 to 1; This represents a parameter indicating abnormal physiological deviation, with a value range of 0 to 1; This represents the weighting coefficient corresponding to the passive monitoring continuous parameters, with a value range of 0 to 1; This represents a passive monitoring parameter, with a value range of 0 to 1. This represents the inhibition weight coefficient corresponding to the silent alertness maintenance parameter, with a value ranging from 0 to 1; Indicates the parameter for maintaining silent alertness; This represents the logical activation function. The formula positively accumulates the effects of fatigue drive, task load, physiological abnormalities, and passive monitoring, while negatively suppressing them through a silent vigilance maintenance parameter. This ensures that the fatigue characterization reflects the true fatigue risk, rather than simply reflecting low interaction density.

[0106] Abnormal physiological deviation parameters can be calculated based on an individual's silent baseline, and can be expressed as: in, This indicates the abnormal physiological deviation parameter at the current assessment time; This indicates the number of physiological characteristics involved in the calculation of abnormal physiological deviations; This represents the weight coefficient corresponding to the j-th physiological feature, with a value ranging from 0 to 1, and all values ​​are equal. The sum is 1; This represents the j-th physiological characteristic at the current evaluation time; This represents the baseline mean of the j-th physiological characteristic under the normal silent monitoring state of the corresponding operating subject; This represents the baseline fluctuation scale of the j-th physiological characteristic under the normal silent monitoring state of the corresponding operating subject; Represents the stability constant, which can be taken as... to ; This represents the logical activation function. The formula calculates the degree of physiological deviation from an individual's normal silent baseline, enabling abnormal physiological deviation parameters to identify genuine drowsiness, low arousal, or postural lethargy, rather than treating silent conservatism itself as abnormal.

[0107] When performing a silent consistency check based on fatigue characterization parameters, silent vigilance maintenance parameters, abnormal physiological deviation parameters, and passive monitoring persistence parameters, it can be determined whether the increase in fatigue characterization parameters is supported by genuine evidence under silent tasks. Silent consistency check involves jointly verifying the trend of fatigue characterization parameter changes with a decrease in silent vigilance maintenance parameters, an increase in abnormal physiological deviation parameters, and an increase in passive monitoring persistence parameters. If fatigue characterization parameters increase while silent vigilance maintenance parameters decrease, abnormal physiological deviation parameters increase, or passive monitoring persistence parameters continue to increase, then the increase in fatigue characterization parameters is reasonable. If fatigue characterization parameters increase but silent vigilance maintenance parameters remain stable, abnormal physiological deviation parameters are not significant, and only low-interaction events are observed, then the fatigue characterization parameters may be affected by low-interaction density interference during silent tasks, requiring suppression or smoothing.

[0108] The silent consistency parameter can be expressed as: in, The parameter represents the silence consistency at the current evaluation moment, and its value ranges from 0 to 1. The larger the value, the more consistent the fatigue characterization is with the actual fatigue evolution in the silent stage. This represents the magnitude of change in fatigue characterization from the current assessment time to the previous r assessment times; This represents the magnitude of change in the silent vigilance maintenance parameter from the current evaluation time to the previous r evaluation times; This indicates the magnitude of change in abnormal physiological deviation parameters from the current assessment time to the previous r assessment times; This indicates the magnitude of change of the passively monitored continuous parameter from the current assessment time to the previous r assessment times; This indicates the length of the silent consistency evaluation window, with a value ranging from 2 to 30. The weighting coefficient represents the magnitude of change in fatigue characterization parameters, and its value ranges from 0 to 1. This represents the weighting coefficient corresponding to the magnitude of change in the silent alertness maintenance parameter, with a value ranging from 0 to 1; The weighting coefficient represents the magnitude of change in abnormal physiological deviation parameters, and its value ranges from 0 to 1. This represents the weighting coefficient corresponding to the magnitude of change in the passively monitored continuous parameter, with a value ranging from 0 to 1, and satisfying the following conditions: This formula judges the reliability of fatigue characterization through multi-parameter trend relationships. Among them, the decrease in the silent alertness maintenance parameter will improve fatigue consistency, the increase in the abnormal physiological deviation parameter and the increase in the passive monitoring continuous parameter will also improve fatigue consistency, thereby preventing low interaction density from driving fatigue judgment alone.

[0109] When outputting fatigue representations to the biomathematical fatigue assessment model based on the silent consistency parameter, the silent consistency parameter can be used as a confidence constraint on the fatigue representations. The fatigue representations can be retained, smoothed, or processed by callback before being output to the recursive stage of the biomathematical fatigue assessment model. If the silent consistency parameter is high, it indicates that the increase in fatigue representations is supported by multi-source evidence, and the current fatigue representation can be directly retained. If the silent consistency parameter is low, it indicates that the fatigue representation may be affected by the silent low-interaction state, and it should be smoothed by combining the fatigue results from the previous assessment time. Through this processing, the biomathematical fatigue assessment model can maintain continuous, stable, and task-adaptive fatigue estimation during the silent monitoring phase.

[0110] The fatigue characterization quantity output to the biomathematical fatigue assessment model can be expressed as: in, This represents the silent verification fatigue characterization quantity output to the biomathematical fatigue assessment model at the current assessment moment. Indicates the silent consistency parameter; This represents the fatigue characterization quantity at the current evaluation moment; This represents the smoothing coefficient, with a value ranging from 0.1 to 0.9; This represents the silent verification fatigue representation quantity output to the biomathematical fatigue assessment model at the previous assessment time. This formula increases the output proportion of the current fatigue representation quantity when the silent consistency parameter is high, and enhances the continuation effect of historical fatigue results when the silent consistency parameter is low. This reduces the sudden misjudgment of the fatigue representation quantity caused by the ultra-low interaction density during the silent monitoring phase, and enables the biomathematical fatigue assessment model to continue performing subsequent joint mappings based on stable fatigue results.

[0111] In one possible implementation, data-driven modeling processing is performed on the multidimensional feature parameters to form a fusion representation model, and the fatigue representation quantity is jointly mapped with the fusion representation model to construct a human-machine collaborative perception representation space. Specifically, this further includes: binding the multidimensional feature parameters, fatigue representation quantity, and silent guarding identification results to form silent guarding modeling data; performing feature grouping processing on the silent guarding modeling data to form a silent alertness feature group, a silent stability feature group, a silent persistence feature group, and an interactive event feature group; performing contribution suppression processing on the interactive event feature group based on the silent guarding identification results, and performing contribution enhancement processing on the silent alertness feature group, the silent stability feature group, and the silent persistence feature group to generate a silent correction function. Positive feature sequence; based on the silent correction feature sequence, perform temporal coding processing to generate a silent guarding coding vector; calculate low interaction confidence parameters based on silent guarding identification results, hidden constraint identifiers, system prompt sparsity, passive monitoring persistence parameters, and response missing parameters; generate silent interaction gating parameters based on the low interaction confidence parameters, and perform fusion processing on the silent guarding coding vector, interaction event coding vector, and silent baseline coding vector according to the silent interaction gating parameters to form a silent fusion representation vector; jointly map the fatigue representation quantity with the silent fusion representation vector to generate a silent fatigue modulation representation vector; construct a human-machine collaborative perception representation space based on the silent fatigue modulation representation vector and the system silent task state vector.

[0112] Specifically, when identifying whether the current task stage is in a silent monitoring phase by combining multimodal observation datasets and task stage data, we can first extract system cue events, operational action events, gaze area changes, postural activity amplitude, EEG attentional maintenance characteristics, and physiological response changes from the multimodal observation datasets. Then, we can extract sonar monitoring status, covert constraint markers, system cue strategies, necessary response event markers, and task stage risk levels from the task stage data. The silent monitoring phase refers to a state where the operator is in covert monitoring, passive listening, low command triggering, or minimal interaction standby. The task requires the operator to maintain continuous observation and responsiveness, rather than proving task execution effectiveness through frequent operations. The silent monitoring identification results are used to distinguish between a normal low-interaction state caused by the task mechanism and an abnormally low-response state caused by fatigue, ensuring that subsequent fatigue characterization calculations do not directly interpret fewer operations, fewer action triggers, or fewer system cuees as increased fatigue.

[0113] The result of silent monitoring identification can be represented as: in, This represents the silent monitoring identification result at the current evaluation moment, with a value ranging from 0 to 1. The larger the value, the more the current task stage matches the characteristics of the silent monitoring stage. This represents the covert constraint flag, which is set to 1 if there is a covert monitoring or passive sonar listening requirement, and 0 if there is no requirement. This indicates the sparsity of system prompts, calculated from the interval between system prompt events and the number of system prompt events within the current evaluation window, with a value ranging from 0 to 1. This represents the passive monitoring duration parameter, which is obtained by normalizing the current passive monitoring duration, and its value ranges from 0 to 1. This represents the normalized result of the risk level at each stage of the task, with a value ranging from 0 to 1. This represents the density of necessary response events, which is obtained by normalizing the number of events that must be responded to within the current evaluation window, and its value ranges from 0 to 1. This indicates the activity level of the operation, which is obtained by normalizing the frequency of button presses, touch controls, joystick movements, or confirmation actions, and the value ranges from 0 to 1. , , , , and These represent the weight coefficients of the corresponding parameters, with values ​​ranging from 0 to 1, and the sum of all weight coefficients is 1. This represents the logical activation function. The formula improves the silent monitoring identification result by using hidden constraints, sparse system prompts, continuous passive monitoring, and task risk, while reducing the silent monitoring identification result by increasing the density of necessary response events and the activity level of operational actions. This avoids misidentifying stages with a large number of necessary operations as silent monitoring stages.

[0114] When extracting multidimensional feature parameters based on silent vigilance recognition results from multimodal observation datasets, features reflecting silent alertness should be prioritized when the silent vigilance recognition results are high, rather than prioritizing the number of responses per unit time, action trigger frequency, or interaction completion frequency. Multidimensional feature parameters include continuous gaze stability parameters, micro-eye movement change parameters, low-frequency EEG fatigue parameters, EEG attention maintenance parameters, heart rate variability stability parameters, respiratory rhythm stability parameters, head micro-movement stability parameters, posture maintenance parameters, passive monitoring continuity parameters, and necessary response effectiveness parameters. The sustained gaze stability parameter describes whether the operator continuously focuses on the key monitoring area; the micro-eye movement change parameter distinguishes between normal scanning observation and prolonged staring rigidity; the EEG low-frequency fatigue parameter characterizes the trend of increasing drowsiness; the EEG attention retention parameter characterizes whether attentional resources are continuously invested; the heart rate variability stability parameter and respiratory rhythm stability parameter characterize whether physiological arousal is stable; the head micro-motion stability parameter and posture retention parameter distinguish between normal stillness and fatigue drooping; the passive monitoring duration parameter characterizes the cumulative time of silent tasks; and the necessary response effectiveness parameter characterizes whether the operator still has the ability to respond in a timely manner when necessary response events occur.

[0115] Multidimensional feature parameters can be expressed as: in, This represents the multidimensional feature parameters extracted at the current evaluation moment for the silent monitoring phase; This represents the stability parameter for continuous fixation, and its value ranges from 0 to 1; This represents the micro-eye movement parameter, with a value range of 0 to 1; This represents a low-frequency fatigue parameter in brainwaves, with a value range of 0 to 1. This indicates the EEG attention retention parameter, with a value range of 0 to 1; This represents the heart rate variability stability parameter, with a value ranging from 0 to 1; This represents a parameter for stabilizing respiratory rhythm, with a value ranging from 0 to 1; This represents the head micro-motion stability parameter, with a value range of 0 to 1; This represents the attitude maintenance parameter, with a value range of 0 to 1; This represents a passive monitoring parameter, with a value range of 0 to 1. This represents the effective parameter for necessary responses, with a value ranging from 0 to 1. This expression organizes multi-source information that can characterize continuous vigilance, stable observation, and necessary response capabilities during the silent monitoring phase into a unified feature vector, enabling subsequent correction and fatigue calculations to be based on the actual state of the silent task.

[0116] When performing silent guarding correction on multidimensional feature parameters, the participation intensity of various feature parameters can be adjusted according to the silent guarding recognition results. For the silent guarding phase, fewer system interaction events and fewer action triggers are task characteristics and should not be considered evidence of fatigue enhancement; therefore, the weight of interaction frequency features needs to be reduced. Sustained gaze stability, micro-eye movement changes, EEG attention maintenance, physiological rhythm stability, and posture maintenance better reflect the true level of alertness; therefore, the weight of these features needs to be increased. The silent guarding feature parameter set refers to the feature set adjusted by the silent guarding recognition results. Its function is to transform the explicit action evaluation logic in conventional high-interaction tasks into a responsive readiness evaluation logic in silent guarding tasks.

[0117] The set of silent guarding feature parameters can be represented as: in, This represents the set of silent, guarded characteristic parameters at the current evaluation moment; This represents the multidimensional feature parameters extracted at the current evaluation moment for the silent monitoring phase; This indicates the result of silent monitoring identification; This represents the silent correction weight vector. Each element in the vector ranges from 0 to 2. A value greater than 1 indicates enhancement of the corresponding feature parameter contribution, while a value less than 1 indicates suppression of the corresponding feature parameter contribution; 1 indicates no change. A uniformly one vector with consistent dimensions; This indicates element-wise multiplication. This formula largely preserves the original feature parameters when the silent-guarded recognition result is low, and adjusts the feature contribution according to the silent-guarded correction weight vector when the silent-guarded recognition result is high, thereby reducing the risk of misjudgment caused by low interaction density.

[0118] When calculating silent vigilance maintenance parameters based on a set of silent watch characteristic parameters, parameters such as sustained gaze stability, micro-eye movement changes, EEG attention maintenance, heart rate variability stability, respiratory rhythm stability, head micro-movement stability, posture maintenance, and necessary response effectiveness can be fused together. Simultaneously, a negative influence is applied to EEG low-frequency fatigue parameters and abnormal lag characteristics. Silent vigilance maintenance parameters represent whether the operator maintains effective monitoring and responsiveness under conditions of low interaction, minimal cues, and prolonged monitoring. A high silent vigilance maintenance parameter indicates stable gaze, moderate micro-eye movements, stable EEG attention characteristics, stable physiological rhythms, no posture drooping, and effective response when necessary events occur. Conversely, a low silent vigilance maintenance parameter indicates fixed gaze rigidity, reduced eye movements, increased low-frequency EEG, posture drooping, or failure to respond to necessary events.

[0119] The silent alertness maintenance parameter can be expressed as: in, This represents the silent vigilance maintenance parameter at the current assessment moment, with a value ranging from 0 to 1. The larger the value, the stronger the vigilance maintenance capability during the silent vigilance phase; K represents the number of feature parameters involved in the positive silent vigilance calculation. This represents the weight coefficient corresponding to the k-th positive feature parameter, with a value ranging from 0 to 1, and the sum of all positive weight coefficients does not exceed 1; This represents the k-th positive feature parameter in the set of silent, conserved feature parameters at the current evaluation moment; This represents the negative weighting coefficient corresponding to the low-frequency fatigue parameter of EEG, with a value range of 0 to 1; This indicates low-frequency fatigue parameters in brainwaves; This represents the logical activation function. The formula uses the accumulation of positive characteristic parameters to represent sustained alertness support capacity, and uses negative subtraction based on low-frequency EEG fatigue parameters to enable the silent alertness maintenance parameter to distinguish between normal silent standby and fatigue-induced retardation.

[0120] When calculating the basal fatigue drive based on the sleep homeostasis component, circadian rhythm component, and sleep inertia component, the basal components in the biomathematical fatigue assessment model can be converted into fatigue drive expressions in the same direction. The sleep homeostasis component represents the alertness basis jointly determined by sleep recovery and continuous wakefulness; the circadian rhythm component represents the periodic regulation of alertness by the current intraday phase; and the sleep inertia component represents the effect of incomplete alertness recovery shortly after wakefulness. The basal fatigue drive refers to the fatigue tendency formed by insufficient sleep, circadian rhythm troughs, and wakefulness lag before considering silent guard correction; a higher value indicates a higher risk of basal fatigue in the operator.

[0121] The basic fatigue driving force can be expressed as: in, This represents the basic fatigue driving force at the current evaluation moment, with a value ranging from 0 to 1; This represents the normalized sleep homeostasis component, with a value ranging from 0 to 1. A larger value indicates that sleep homeostasis is more conducive to maintaining alertness. This represents the normalized circadian rhythm component, with a value ranging from 0 to 1. The larger the value, the more conducive the current circadian rhythm phase is to maintaining alertness. This represents the normalized sleep inertia component, with a value ranging from 0 to 1. The larger the value, the stronger the influence of sleep inertia. The weighting coefficients corresponding to the sleep homeostasis components range from 0 to 1. The weighting coefficients corresponding to the diurnal rhythm components range from 0 to 1. The weighting coefficients corresponding to the sleep inertia component range from 0 to 1 and satisfy the following conditions: This formula unifies insufficient sleep homeostasis, unfavorable circadian rhythm, and enhanced sleep inertia into fatigue-driving directions, providing a basic fatigue estimate for subsequent silent maintenance correction.

[0122] When performing silent standby correction on the baseline fatigue drive based on silent standby identification results and silent alertness maintenance parameters, it can suppress fatigue amplification indirectly introduced by low interaction density when it is confirmed that the current state is in a silent standby phase and the silent alertness maintenance parameter is high; conversely, it can preserve or enhance the baseline fatigue drive when the current state is in a silent standby phase but the silent alertness maintenance parameter is low. Silent-corrected fatigue drive refers to the fatigue drive expression corrected based on the baseline fatigue drive based on the actual alertness maintenance capability during the silent standby phase. Its purpose is to avoid misjudging normal silent standby as fatigue, while still being able to identify attentional decline and insufficient response readiness caused by genuine fatigue.

[0123] The silent correction fatigue driving force can be expressed as: in, This indicates the silent correction fatigue drive at the current evaluation moment; Indicates the basic fatigue driving force; This represents the silent misjudgment suppression coefficient, with a value ranging from 0 to 1; This indicates the result of silent monitoring identification; Indicates the parameter for maintaining silent alertness; This represents the silent fatigue compensation coefficient, with a value ranging from 0 to 1. This formula reduces the basic fatigue drive when the silent vigilance recognition result is high and the silent alertness maintenance parameter is high, and increases fatigue drive compensation when the silent vigilance recognition result is high but the silent alertness maintenance parameter is low, thereby distinguishing between normal silent state and fatigue-induced low response state.

[0124] When calculating fatigue characterization parameters based on silent-corrected fatigue drive quantity, silent monitoring task load parameters, abnormal physiological deviation parameters, passive monitoring duration parameters, and silent vigilance maintenance parameters, the silent-corrected fatigue drive quantity can be used as the core fatigue basis. The silent monitoring task load parameters represent the impact of task stress, the abnormal physiological deviation parameters represent real-time evidence of anomalies, the passive monitoring duration parameters represent the impact of long-term low-stimulus exposure, and the silent vigilance maintenance parameters represent fatigue inhibitory factors. The silent monitoring task load parameters characterize the number of monitored targets, alarm sensitivity level, sonar information complexity, and task risk level. The abnormal physiological deviation parameters characterize the degree of abnormality in EEG, eye movement, heart rate variability, respiratory rhythm, and posture relative to the individual's silent baseline. The passive monitoring duration parameters characterize the impact of prolonged silent monitoring on fatigue accumulation. The fatigue characterization quantity is the final quantitative result representing the current fatigue level of the operator; a higher value indicates a higher degree of fatigue.

[0125] The fatigue characterization quantity can be expressed as: in, This represents the fatigue characterization value at the current evaluation moment, with a value ranging from 0 to 1; This indicates the fatigue reference offset, which can range from -3 to 3. This represents the weighting coefficient corresponding to the silent correction fatigue driving quantity, with a value range of 0 to 1; This indicates the amount of fatigue drive that is silently corrected. This represents the weighting coefficient corresponding to the load parameters of the silent monitoring task, with a value ranging from 0 to 1; This represents the load parameter for the silent monitoring task, with a value range of 0 to 1; This represents the weighting coefficient corresponding to the abnormal physiological deviation parameter, with a value range from 0 to 1; This represents a parameter indicating abnormal physiological deviation, with a value range of 0 to 1; This represents the weighting coefficient corresponding to the passive monitoring continuous parameters, with a value range of 0 to 1; This represents a passive monitoring parameter, with a value range of 0 to 1. This represents the inhibition weight coefficient corresponding to the silent alertness maintenance parameter, with a value ranging from 0 to 1; Indicates the parameter for maintaining silent alertness; This represents the logical activation function. The formula positively accumulates the effects of fatigue drive, task load, physiological abnormalities, and passive monitoring, while negatively suppressing them through a silent vigilance maintenance parameter. This ensures that the fatigue characterization reflects the true fatigue risk, rather than simply reflecting low interaction density.

[0126] Abnormal physiological deviation parameters can be calculated based on an individual's silent baseline, and can be expressed as: in, This indicates the abnormal physiological deviation parameter at the current assessment time; This indicates the number of physiological characteristics involved in the calculation of abnormal physiological deviations; This represents the weight coefficient corresponding to the j-th physiological feature, with a value ranging from 0 to 1, and all values ​​are equal. The sum is 1; This represents the j-th physiological characteristic at the current evaluation time; This represents the baseline mean of the j-th physiological characteristic under the normal silent monitoring state of the corresponding operating subject; This represents the baseline fluctuation scale of the j-th physiological characteristic under the normal silent monitoring state of the corresponding operating subject; Represents the stability constant, which can be taken as... to ; This represents the logical activation function. The formula calculates the degree of physiological deviation from an individual's normal silent baseline, enabling abnormal physiological deviation parameters to identify genuine drowsiness, low arousal, or postural lethargy, rather than treating silent conservatism itself as abnormal.

[0127] When performing a silent consistency check based on fatigue characterization parameters, silent vigilance maintenance parameters, abnormal physiological deviation parameters, and passive monitoring persistence parameters, it can be determined whether the increase in fatigue characterization parameters is supported by genuine evidence under silent tasks. Silent consistency check involves jointly verifying the trend of fatigue characterization parameter changes with a decrease in silent vigilance maintenance parameters, an increase in abnormal physiological deviation parameters, and an increase in passive monitoring persistence parameters. If fatigue characterization parameters increase while silent vigilance maintenance parameters decrease, abnormal physiological deviation parameters increase, or passive monitoring persistence parameters continue to increase, then the increase in fatigue characterization parameters is reasonable. If fatigue characterization parameters increase but silent vigilance maintenance parameters remain stable, abnormal physiological deviation parameters are not significant, and only low-interaction events are observed, then the fatigue characterization parameters may be affected by low-interaction density interference during silent tasks, requiring suppression or smoothing.

[0128] The silent consistency parameter can be expressed as: in, The parameter represents the silence consistency at the current evaluation moment, and its value ranges from 0 to 1. The larger the value, the more consistent the fatigue characterization is with the actual fatigue evolution in the silent stage. This represents the magnitude of change in fatigue characterization from the current assessment time to the previous r assessment times; This represents the magnitude of change in the silent vigilance maintenance parameter from the current evaluation time to the previous r evaluation times; This indicates the magnitude of change in abnormal physiological deviation parameters from the current assessment time to the previous r assessment times; This represents the magnitude of change in the passively monitored continuous parameter from the current evaluation time to the previous r evaluation times; r represents the length of the silent consistency evaluation window, with a value ranging from 2 to 30. The weighting coefficient represents the magnitude of change in fatigue characterization parameters, and its value ranges from 0 to 1. This represents the weighting coefficient corresponding to the magnitude of change in the silent alertness maintenance parameter, with a value ranging from 0 to 1; The weighting coefficient represents the magnitude of change in abnormal physiological deviation parameters, and its value ranges from 0 to 1. This represents the weighting coefficient corresponding to the magnitude of change in the passively monitored continuous parameter, with a value ranging from 0 to 1, and satisfying the following conditions: This formula judges the reliability of fatigue characterization through multi-parameter trend relationships. Among them, the decrease in the silent alertness maintenance parameter will improve fatigue consistency, the increase in the abnormal physiological deviation parameter and the increase in the passive monitoring continuous parameter will also improve fatigue consistency, thereby preventing low interaction density from driving fatigue judgment alone.

[0129] When outputting fatigue representations to the biomathematical fatigue assessment model based on the silent consistency parameter, the silent consistency parameter can be used as a confidence constraint on the fatigue representations. The fatigue representations can be retained, smoothed, or processed by callback before being output to the recursive stage of the biomathematical fatigue assessment model. If the silent consistency parameter is high, it indicates that the increase in fatigue representations is supported by multi-source evidence, and the current fatigue representation can be directly retained. If the silent consistency parameter is low, it indicates that the fatigue representation may be affected by the silent low-interaction state, and it should be smoothed in conjunction with the fatigue results at the previous assessment time. Through this processing, the biomathematical fatigue assessment model can maintain continuous, stable, and task-adaptive fatigue estimation during the silent monitoring phase.

[0130] The fatigue characterization quantity output to the biomathematical fatigue assessment model can be expressed as: in, This represents the silent verification fatigue characterization quantity output to the biomathematical fatigue assessment model at the current assessment moment. Indicates the silent consistency parameter; This represents the fatigue characterization quantity at the current evaluation moment; This represents the smoothing coefficient, with a value ranging from 0.1 to 0.9; This represents the silent verification fatigue representation quantity output to the biomathematical fatigue assessment model at the previous assessment time. This formula increases the output proportion of the current fatigue representation quantity when the silent consistency parameter is high, and enhances the continuation effect of historical fatigue results when the silent consistency parameter is low. This reduces the sudden misjudgment of the fatigue representation quantity caused by the ultra-low interaction density during the silent monitoring phase, and enables the biomathematical fatigue assessment model to continue performing subsequent joint mappings based on stable fatigue results.

[0131] In one possible implementation, the system comprehensively calculates the coordination consistency and response efficiency during the interaction between the operator and the system based on the human-machine collaborative perception representation space, generating state analysis results. Specifically, this includes: parsing the task stage semantics in the human-machine collaborative perception representation space and generating silent interaction judgment results; calculating silent coordination consistency parameters based on the operator's silent state vector and the system's silent task vector extracted from the human-machine collaborative perception representation space; calculating silent response efficiency parameters based on necessary response event data; calculating alternative response efficiency parameters based on continuous gaze stability, EEG attention maintenance, posture maintenance, and system scan coverage when no necessary response events exist in the current evaluation window; selecting silent response efficiency parameters or alternative response efficiency parameters based on the number of necessary response events to form silent stage response efficiency parameters; calculating silent stability parameters based on silent coordination consistency parameters, silent stage response efficiency parameters, and fatigue characterization within the continuous evaluation window; calculating silent state comprehensive data based on silent coordination consistency parameters, silent stage response efficiency parameters, silent stability parameters, and fatigue characterization; and performing state grading judgment based on the silent state comprehensive data to generate state analysis results.

[0132] Specifically, when analyzing the task phase semantics in the human-machine collaborative perception representation space, representational dimensions related to task phase identifiers, sonar monitoring status, concealment constraint identifiers, system prompt sparsity, necessary response event density, passive monitoring duration, and task risk level can be extracted from the human-machine collaborative perception representation space. These representational dimensions are then mapped to silent interaction judgment results. Task phase semantics refers to the meaning of the task attributes corresponding to the current evaluation moment, used to explain which interaction mode the current task belongs to, such as high-frequency operation, routine confirmation, emergency response, silent monitoring, or task switching. Silent interaction judgment results are used to determine whether the response efficiency calculation logic of the silent monitoring phase should be adopted for the current evaluation window, avoiding misjudging a low number of system interaction events as low response from the operating entity during concealed monitoring or passive sonar monitoring phases.

[0133] The expression for the result of silent interaction determination can be represented as: in, This indicates the result of the silent interaction judgment at the current evaluation moment. The value ranges from 0 to 1. The larger the value, the more the current evaluation window conforms to the interaction characteristics of the silent guarding phase. This indicates the result of silent monitoring identification; This represents the covert constraint flag, which is set to 1 if there is a covert monitoring or passive sonar listening requirement, and 0 if there is no requirement. This indicates the sparsity of system prompts, which is obtained by normalizing the system prompt interval and the number of system prompts. This represents the passive monitoring duration parameter, which is obtained by normalizing the current passive monitoring duration. This indicates the result of normalized task risk level; This represents the density of necessary response events, which is obtained by normalizing the number of events that must be responded to within the current evaluation window; , , , , and These represent the weight coefficients of the corresponding parameters, with values ​​ranging from 0 to 1, and the sum of all weight coefficients is 1. This represents the logical activation function. The formula improves the silent interaction judgment result through silent task evidence and reduces the silent interaction judgment result through the density of necessary response events, ensuring that stages with a large number of necessary operations are not incorrectly included in the silent monitoring evaluation logic.

[0134] When extracting the silent state vector of the operator and the silent task vector of the system from the human-machine collaborative perception representation space, the dimensions related to the operator's state in the human-machine collaborative perception representation space can be mapped to the silent state vector of the operator, and the dimensions related to the system task requirements can be mapped to the silent task vector of the system. The silent state vector of the operator is used to represent the operator's sustained gaze stability state, EEG attention maintenance state, micro-eye movement change state, physiological rhythm stability state, posture maintenance state, and fatigue modulation state under low interaction conditions. The silent task vector of the system is used to represent the monitoring target coverage requirements, necessary response requirements, alarm sensitivity level, concealment constraint requirements, system scan coverage requirements, and task risk requirements proposed by the system to the operator during the silent duty phase. By extracting these two types of vectors, the silent collaborative consistency calculation can be shifted from focusing on the frequency of operation to focusing on whether the operator maintains a responsive and ready state that matches the silent task requirements.

[0135] The silent state vector of the operating entity and the silent task vector of the system can be represented as: in, This represents the silent state vector of the operating entity at the current evaluation moment; This represents the system's silent task vector at the current evaluation moment; This represents the function that maps the silent state of the operating entity. This represents the system's silent task mapping function; This represents the human-machine collaborative perception representation space at the current evaluation moment; The trainable parameters represent the mapping function of the silent state of the operating subject; The trainable parameters represent the system's silent task mapping function. This mapping enables comparable vector representations of the operator's internal state during the silent monitoring phase and the system's silent task requirements.

[0136] The silent coordination consistency parameter represents the degree of matching between the silent state vector of the operator and the silent task vector of the system. If the operator maintains continuous observation, stable attentional resources, stable physiological rhythms, and normal posture, and this state meets the current silent monitoring requirements of the system, the silent coordination consistency parameter increases. Conversely, if the operator exhibits attentional slippage, postural stagnation, decreased EEG attention, or a mismatch with the system's key monitoring requirements, the silent coordination consistency parameter decreases. This parameter emphasizes that coordination in silent monitoring scenarios is not a high-frequency operational collaboration, but rather an adaptation relationship between the human body's readiness state and the system's task constraints under low-interaction conditions.

[0137] The expression for the silent collaborative consistency parameter can be represented as: in, This represents the silent collaborative consistency parameter at the current evaluation moment, with a value ranging from 0 to 1; This represents the silent state vector of the operating entity; Represents the system's silent task vector; Represents the stability constant; This represents a fatigue characterization quantity, with a value ranging from 0 to 1; This represents the silent alertness maintenance parameter, with a value ranging from 0 to 1; This represents the directional similarity weight coefficient, with a value ranging from 0 to 1; This represents the spatial difference penalty coefficient, with a value ranging from 0 to 1; This represents the fatigue penalty coefficient, with a value ranging from 0 to 1; This represents the silent alertness enhancement coefficient, with a value ranging from 0 to 1; This represents the logical activation function. The formula evaluates the matching relationship between the silent state of the operator and the silent task requirements of the system through directional similarity and spatial difference, while using fatigue characterization quantities for negative constraints and silent alertness maintenance parameters for positive compensation.

[0138] When calculating silent response efficiency parameters based on necessary response event data, events that truly require an operator's response within the current evaluation window should be selected from the system interaction data first, excluding ordinary waiting, continuous listening, or silent monitoring segments from the efficiency calculation. Necessary response event data refers to event data that still requires a valid response during the silent monitoring phase, including sonar anomaly alarms, safety confirmation prompts, track correction prompts, equipment anomaly prompts, manual confirmation requests, and rule-triggered confirmation events. Silent response efficiency parameters are used to evaluate whether the operator can respond promptly, accurately, and with low error rates when necessary response events occur, rather than evaluating whether the number of responses per unit time is sufficient.

[0139] The expression for the silent response efficiency parameter can be represented as: in, This parameter represents the silent response efficiency at the current evaluation moment, and its value ranges from 0 to 1. This indicates the average response delay between the triggering of a necessary response event and the effective operational response within the current evaluation window; The reference response time for necessary response events is determined by historical stable monitoring records or task specifications. This indicates the number of events that were not responded to in a timely manner among the necessary response events within the current assessment window; This indicates the total number of necessary response events within the current evaluation window; This indicates the number of erroneous responses among the necessary response events within the current evaluation window; This represents the response delay weighting coefficient, with a value ranging from 0 to 1; This represents the weighting coefficient for the proportion of missed responses, with a value ranging from 0 to 1; This represents the error response proportional weighting coefficient, with a value ranging from 0 to 1, and satisfying the following conditions: ; This represents the stability constant. This formula evaluates response delay, missed responses, and erroneous responses only when the necessary response events exist, thus avoiding the incorrect judgment of inefficiency due to zero response counts during the event-free waiting phase.

[0140] When no necessary response event exists within the current evaluation window, alternative response efficiency parameters can be used to characterize the operator's responsive readiness state. Responsive readiness state refers to the operator maintaining continuous observation of key areas, attention to the monitored object, coverage of the system scan range, and a stable posture, even when no explicit response is required by the system. This ensures a timely response should a necessary event occur. Continuous gaze stability indicates whether the operator continuously focuses on key monitoring areas; EEG attention maintenance indicates whether the operator maintains attentional resources; posture maintenance indicates whether the operator exhibits head drooping, trunk relaxation, or delayed posture changes; and system scan coverage indicates whether the operator's line of sight or monitoring focus covers the key areas required by the system. By using alternative response efficiency parameters, the operator's effective operational readiness can still be evaluated even when no necessary response event occurs.

[0141] The expression for the alternative response efficiency parameter can be expressed as: in, The parameter representing the alternative response efficiency at the current evaluation moment ranges from 0 to 1. This indicates a steady state of continuous observation, with a value ranging from 0 to 1; This indicates the state of attention maintained during brain activity, with a value ranging from 0 to 1; This indicates the attitude hold state, with a value ranging from 0 to 1; Indicates the system scan coverage status, with a value ranging from 0 to 1; This represents a fatigue characterization quantity, with a value ranging from 0 to 1; This represents a parameter indicating abnormal physiological deviation, with a value range of 0 to 1; , , , , and These represent the weight coefficients of the corresponding parameters, with values ​​ranging from 0 to 1, and the sum of all weight coefficients is 1. This represents the logical activation function. The formula positively evaluates the responsive readiness state through gaze, EEG, posture, and scan coverage, and negatively constrains it through fatigue representation parameters and abnormal physiological deviation parameters, ensuring that a reasonable efficiency measure can still be obtained for unnecessary response events.

[0142] When selecting a silent response efficiency parameter or an alternative response efficiency parameter based on the number of necessary response events, first determine if the total number of necessary response events within the current evaluation window is greater than zero. If the total number of necessary response events is greater than zero, it indicates that there are clear response requirements in the current evaluation window, and the silent response efficiency parameter should be used to evaluate the actual response performance of the operating entity to the necessary events. If the total number of necessary response events is equal to zero, it indicates that the current evaluation window belongs to a pure silent monitoring or waiting phase, and the alternative response efficiency parameter should be used to evaluate the responsiveness and readiness status of the operating entity. The silent phase response efficiency parameter is an efficiency indicator uniformly used in the silent monitoring phase. It automatically switches the calculation source based on the presence or absence of necessary response events, thereby solving the problem of traditional response efficiency parameters being suppressed due to ultra-low interaction density.

[0143] The expression for the response efficiency parameter during the silent phase can be represented as: in, This represents the response efficiency parameter during the silent phase at the current evaluation moment; This indicates an indicator function that takes the value 1 if the condition is met, and 0 otherwise. This indicates the total number of necessary response events within the current evaluation window; This represents the silent response efficiency parameter; This represents the alternative response efficiency parameter. The formula selects different efficiency calculation logics based on the necessary number of response events, allowing the evaluation of the true response efficiency during the event phase and the evaluation of the responsive readiness state during the event-free phase, thus avoiding directly evaluating the efficiency of silent monitoring based on the density of interactive events.

[0144] When calculating the silent stability parameter based on the silent coordination consistency parameter, silent phase response efficiency parameter, and fatigue characterization quantity within a continuous evaluation window, multiple continuous evaluation windows can be selected backward from the current evaluation time, and the fluctuation degree of the silent coordination consistency parameter sequence, silent phase response efficiency parameter sequence, and fatigue characterization quantity sequence can be calculated separately. The silent stability parameter is used to indicate whether the state of the operator is stable during a long period of silent monitoring. If the silent coordination consistency, silent phase response efficiency, and fatigue characterization quantity all maintain a smooth change, the silent stability parameter is high; if the three fluctuate sharply, it indicates that there may be risks of attention slippage, sudden fatigue, short-term interference, or state instability.

[0145] The expression for the silent stability parameter can be represented as: in, This represents the silent stability parameter at the current evaluation moment, with a value ranging from 0 to 1; This represents the variance of the silent collaborative consistency parameter from the current evaluation time to the previous r evaluation times; This represents the variance of the response efficiency parameter during the silent phase from the current evaluation time to the previous r evaluation times; This represents the variance of the fatigue characterization from the current evaluation time to the previous r evaluation times; This indicates the length of the continuous evaluation window, with a value ranging from 2 to 30. This represents the silent collaborative fluctuation weighting coefficient, with a value ranging from 0 to 1; This represents the weighting coefficient for fluctuations in silent efficiency, with a value ranging from 0 to 1. This represents the fatigue fluctuation weighting coefficient, with a value ranging from 0 to 1, and satisfying the following conditions: This formula measures the stability of the silent state by the variance within a continuous window. The greater the fluctuation, the lower the silent stability parameter, thus avoiding the direct determination of the state analysis results by the accidental high or low scores of a single evaluation window.

[0146] When calculating the comprehensive data of the silent state based on the silent coordination consistency parameter, silent phase response efficiency parameter, silent stability parameter, and fatigue characterization quantity, the silent coordination consistency parameter, silent phase response efficiency parameter, and silent stability parameter can be used as positive evaluation factors, while the fatigue characterization quantity can be used as a negative evaluation factor. The comprehensive data of the silent state is used to uniformly represent the overall duty capability of the operator during the silent duty phase. It no longer uses the number of interactions per unit time as the core evaluation criterion, but comprehensively considers whether the operator matches the system's silent task requirements, whether it maintains the necessary response capability or a responsive readiness state, whether the state is stable, and whether fatigue has increased.

[0147] The expression for the comprehensive data of the silent state can be represented as: in, This represents the comprehensive data of the silent state at the current evaluation moment; Indicates the parameters for silent collaborative consistency; This represents the response efficiency parameter during the silent phase; Indicates the silent stability parameter; This represents a fatigue characterization quantity; This represents the weighting coefficient corresponding to the silent collaborative consistency parameter, with a value ranging from 0 to 1; This represents the weighting coefficient corresponding to the response efficiency parameter during the silent phase, with a value ranging from 0 to 1; This represents the weighting coefficient corresponding to the silent stability parameter, with a value ranging from 0 to 1; This represents the deduction weighting coefficient corresponding to the fatigue characterization quantity, with a value ranging from 0 to 1, and satisfying the following conditions: This formula positively accumulates silent coordination, silent efficiency, and silent stability, and negatively deducts fatigue level, enabling the comprehensive data of the silent state to reflect the actual operational capability during the silent duty phase.

[0148] When performing state classification judgment based on comprehensive silent state data, a joint judgment can be made by combining the safety thresholds of silent coordination consistency parameters and silent phase response efficiency parameters. State analysis results can include normal silent coordination state, decreased alertness state, fatigue accumulation state, and high-risk silent mismatch state. A normal silent coordination state indicates that the operator, although in a low-interaction state, still maintains good silent alertness and responsiveness; a decreased alertness state indicates that the operator experiences a slight decrease in attention or silent efficiency, but no significant risk has yet materialized; a fatigue accumulation state indicates an increase in fatigue characteristics, a decrease in silent stability parameters, or a weakening of responsiveness; a high-risk silent mismatch state indicates a significant mismatch between the operator's silent state and the system's silent task requirements, or the occurrence of missed, erroneous, or delayed responses to necessary response events.

[0149] The state classification can be expressed as: in, This indicates the state analysis result at the current evaluation moment; This indicates a normal, silent collaborative state. This indicates a state of decreased alertness; This indicates a state of accumulated fatigue. This indicates a high-risk silent mismatch state; This represents the comprehensive data for the silent state; This represents the threshold for the first silent state; This indicates the threshold for the second silent state; This represents the threshold for the third silent state, and satisfies... ; Indicates the parameters for silent collaborative consistency; This represents the response efficiency parameter during the silent phase; Indicates the safety threshold for silent collaboration; This represents the silent efficiency safety threshold. The formula first divides the overall state level by integrating silent state data, and then sets bottom-line constraints by using the silent collaboration safety threshold and the silent efficiency safety threshold, so that the state analysis results can still be accurately output under low interaction density during the silent monitoring phase.

[0150] This embodiment also discloses a human-machine collaborative perception-based operational fatigue state assessment device, referring to... Figure 3 The device includes an acquisition module 301, a processing module 302, and an output module 303. It is used to perform any of the above-described human-machine collaborative perception methods for assessing operational fatigue states, wherein: The acquisition module 301 is used to acquire multimodal physiological and behavioral data of the operating subject, and perform time synchronization processing and noise suppression processing to form a multimodal observation data set; The processing module 302 is used to extract multi-dimensional feature parameters based on the multimodal observation data set, and combine them with the constructed biomathematical fatigue assessment model to generate the fatigue characterization of the operating subject. The processing module 302 is used to perform data-driven modeling processing on multi-dimensional feature parameters to form a fusion representation model, and to jointly map the fatigue representation quantity with the fusion representation model to construct a human-machine collaborative perception representation space. Output module 303 is used to perform comprehensive calculations on the coordination consistency and response efficiency of the interaction between the operating subject and the system based on the human-machine collaborative perception representation space, and generate state analysis results.

[0151] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0152] This embodiment also discloses an electronic device, as shown in the reference. Figure 4 The electronic device may include: at least one processor 401, at least one communication bus 402, user interface 403, network interface 404, and at least one memory 405.

[0153] The communication bus 402 is used to enable communication between these components.

[0154] The user interface 403 may include a display screen and a camera. Optionally, the user interface 403 may also include a standard wired interface and a wireless interface.

[0155] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0156] The processor 401 may include one or more processing cores. The processor 401 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 405, and by calling data stored in memory 405. Optionally, the processor 401 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 401 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 401.

[0157] The memory 405 may include random access memory (RAM) or read-only memory. Optionally, the memory may include a non-transitory computer-readable storage medium. The memory 405 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 405 may also be at least one storage device located remotely from the aforementioned processor 401. As a computer storage medium, the memory 405 may include an operating system, a network communication module, a user interface 403 module, and an application program for a human-machine collaborative perception-based operational fatigue state assessment method.

[0158] exist Figure 4In the electronic device shown, the user interface 403 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 401 can be used to call an application program stored in the memory 405 that is a human-machine collaborative perception operation fatigue state assessment method. When executed by one or more processors 401, the electronic device executes one or more methods as described in the above embodiments.

[0159] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, as some steps can be performed in other orders or simultaneously according to the present invention. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0160] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0161] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0162] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0163] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0164] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 405 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned memory 405 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.

[0165] The present invention also discloses a non-transitory computer-readable storage medium storing instructions. When executed by one or more processors 401, these instructions cause an electronic device to perform one or more methods as described in the above embodiments.

[0166] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This invention is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for assessing operational fatigue state through human-machine collaborative perception, characterized in that, The method includes: Acquire multimodal physiological and behavioral data of the operating subject, and perform time synchronization and noise suppression processing to form a multimodal observation dataset; Multidimensional feature parameters are extracted based on the multimodal observation dataset, and combined with the constructed biomathematical fatigue assessment model to generate fatigue characterization of the operating subject. Data-driven modeling processing is performed on the multidimensional feature parameters to form a fusion representation model, and the fatigue representation quantity is jointly mapped with the fusion representation model to construct a human-machine collaborative perception representation space. Based on the human-machine collaborative perception representation space, the collaborative consistency and response efficiency during the interaction between the operating subject and the system are comprehensively calculated to generate state analysis results.

2. The method for assessing operational fatigue state through human-machine collaborative perception according to claim 1, characterized in that, The process of extracting multidimensional feature parameters based on the multimodal observation dataset and combining them with the constructed biomathematical fatigue assessment model to generate fatigue characterization parameters for the operating subject specifically includes: Based on the multimodal observation data set, sleep-wake data, task execution time, continuous wakefulness duration, sleep duration, and task duration parameters are extracted, and fatigue time series data are constructed. Based on the fatigue time series data, the operator is identified as being awake or asleep, and corresponding sleep homeostasis components and sleep homeostasis recovery components are generated. Calculate the first circadian rhythm component and the second circadian rhythm component based on the task execution time and the task duration parameter; Based on the fatigue time series data, the transition process from sleep to wakefulness is identified, and a sleep inertia component is generated. Based on the multidimensional feature parameters, individualized corrections are performed on the sleep homeostasis component, the first circadian rhythm component, the second circadian rhythm component, and the sleep inertia component to generate model correction coefficients; The comprehensive alertness score is calculated based on the model correction coefficient, the sleep homeostasis component, the first circadian rhythm component, the second circadian rhythm component, and the sleep inertia component, and then mapped to generate the fatigue characterization quantity. The fatigue characterization parameters are output to the biomathematical fatigue assessment model for joint mapping.

3. The method for assessing operational fatigue state through human-machine collaborative perception according to claim 1, characterized in that, The step of performing data-driven modeling processing on the multidimensional feature parameters to form a fusion representation model, and jointly mapping the fatigue representation quantity with the fusion representation model to construct a human-machine collaborative perception representation space, specifically includes: The multidimensional feature parameters are bound according to timestamp, task stage identifier, and operation subject identifier to form a multidimensional feature parameter sequence; Based on the standardized multidimensional feature parameter sequence, construct cognitive load feature subsequence, physiological response feature subsequence, and behavioral dynamic feature subsequence, and perform time-series coding processing respectively to form cognitive load coding vector, physiological response coding vector, and behavioral dynamic coding vector; Calculate the cross-modal association weights based on the cognitive load encoding vector, the physiological response encoding vector, and the behavioral dynamics encoding vector; Based on the cross-modal association weights, a fusion representation vector is formed from the cognitive load encoding vector, the physiological response encoding vector, and the behavioral dynamic encoding vector; Fatigue gating parameters are generated based on the fatigue characterization quantity, the fused characterization vector, and the task load parameters. The fused characterization vector is then modulated based on the fatigue gating parameters to form a fatigue modulation characterization vector. The human-machine collaborative perception representation space is constructed by performing joint projection processing based on the fatigue modulation representation vector and the system interaction state vector.

4. The method for assessing operational fatigue state through human-machine collaborative perception according to claim 1, characterized in that, The process of comprehensively calculating the coordination consistency and response efficiency during the interaction between the operating subject and the system based on the human-machine collaborative perception representation space, and generating state analysis results, specifically includes: The human-machine collaborative perception representation space is bound to the system interaction data to form interactive evaluation data; Based on the interactive evaluation data, extract the operation subject-side state vector and the system-side state vector, and calculate the collaborative consistency parameter; Based on the system prompt time, operation response time, system feedback time, and error correction data extracted from the interaction evaluation data, the response efficiency parameter is calculated. Based on the fatigue characterization parameters, fatigue influence correction is performed on the coordination consistency parameters and the response efficiency parameters to form coordination correction parameters and efficiency correction parameters. Based on the interactive evaluation data, the fluctuation degree of the collaborative correction parameter, the efficiency correction parameter, and the fatigue characterization quantity within the continuous evaluation window is calculated to form the interactive stability parameter; Calculate comprehensive operational state data based on the collaborative correction parameters, the efficiency correction parameters, the interactive stability parameters, and the fatigue characterization parameters; The status analysis results are generated by performing a hierarchical determination based on the comprehensive operational status data, the collaborative correction parameters, and the efficiency correction parameters.

5. The method for assessing operational fatigue state through human-machine collaborative perception according to claim 1, characterized in that, The step of extracting multidimensional feature parameters based on the multimodal observation dataset and combining them with the constructed biomathematical fatigue assessment model to generate fatigue characterization quantities for the operating subject specifically includes: By combining the multimodal observation dataset and task phase data, it is determined whether the current task phase is in a silent monitoring phase, and a silent monitoring identification result is generated. Multidimensional feature parameters based on the silent watch identification results are extracted from the multimodal observation data set. Silent value conservation correction is performed on the multidimensional feature parameters to generate a set of silent value conservation feature parameters; Calculate the silent vigilance maintenance parameters based on the set of silent watch characteristic parameters; The basic fatigue driving force is calculated based on the sleep homeostasis component, the circadian rhythm component, and the sleep inertia component. Based on the silent monitoring identification result and the silent alertness maintenance parameter, the basic fatigue driving quantity is corrected by silent monitoring to generate the silent corrected fatigue driving quantity. The fatigue characterization quantity is calculated based on the silent correction fatigue driving quantity, silent duty load parameters, abnormal physiological deviation parameters, passive monitoring continuity parameters, and silent alertness maintenance parameters. Perform a silent consistency check based on the fatigue characterization quantity, the silent alertness maintenance parameter, the abnormal physiological deviation parameter, and the passive monitoring continuity parameter to generate silent consistency parameters; The fatigue characterization quantity is output to the biomathematical fatigue assessment model based on the silent consistency parameter.

6. The method for assessing operational fatigue state through human-machine collaborative perception according to claim 1, characterized in that, The step of performing data-driven modeling processing on the multidimensional feature parameters to form a fusion representation model, and jointly mapping the fatigue representation quantity with the fusion representation model to construct a human-machine collaborative perception representation space, specifically includes: The multidimensional feature parameters, the fatigue characterization quantity, and the silent guarding identification result are bound together to form silent guarding modeling data; The silent monitoring modeling data is subjected to feature grouping processing to form silent alertness feature group, silent stability feature group, silent duration feature group and interactive event feature group; Based on the silent monitoring identification results, contribution suppression processing is performed on the interactive event feature group, and contribution enhancement processing is performed on the silent alertness feature group, the silent stability feature group, and the silent persistence feature group to generate a silent correction feature sequence. Based on the silent correction feature sequence, perform temporal coding processing to generate a silent guarded coding vector; Calculate the low interaction confidence parameter based on the silent monitoring identification result, hidden constraint identifier, system prompt sparsity, passive monitoring persistence parameter, and response missing parameter. Based on the low interaction confidence parameter, a silent interaction gating parameter is generated, and the silent guarding encoding vector, the interaction event encoding vector, and the silent baseline encoding vector are fused according to the silent interaction gating parameter to form a silent fusion representation vector. The fatigue characterization quantity is jointly mapped with the silent fusion characterization vector to generate a silent fatigue modulation characterization vector. Based on the silent fatigue modulation representation vector and the system silent task state vector, the human-machine collaborative perception representation space is constructed.

7. The method for assessing operational fatigue state through human-machine collaborative perception according to claim 1, characterized in that, The step of comprehensively calculating the coordination consistency and response efficiency during the interaction between the operating subject and the system based on the human-machine collaborative perception representation space to generate state analysis results specifically includes: The semantics of the task stages in the human-machine collaborative perception representation space are analyzed, and a silent interaction judgment result is generated. Calculate the silent collaboration consistency parameter based on the silent state vector of the operating subject and the silent task vector of the system extracted from the human-machine collaborative perception representation space. Calculate the silent response efficiency parameters based on the necessary response event data; When there is no necessary response event within the current evaluation window, alternative response efficiency parameters are calculated based on the sustained gaze steady state, EEG attentional maintenance state, posture maintenance state, and system scan coverage state. Select the silent response efficiency parameter or the alternative response efficiency parameter according to the number of necessary response events to form the silent phase response efficiency parameter; The silent stability parameter is calculated based on the silent coordination consistency parameter, the silent phase response efficiency parameter, and the fatigue characterization quantity within the continuous evaluation window. Calculate the comprehensive data of the silent state based on the silent coordination consistency parameter, the silent phase response efficiency parameter, the silent stability parameter, and the fatigue characterization quantity; Based on the comprehensive data of the silent state, a state classification determination is performed, and the state analysis result is generated.

8. A human-machine collaborative sensing device for assessing operational fatigue status, characterized in that, The device is used to perform a human-machine collaborative perception method for assessing operational fatigue state as described in any one of claims 1-7. The device includes an acquisition module, a processing module, and an output module, wherein: The acquisition module is used to acquire multimodal physiological and behavioral data of the operating subject, and perform time synchronization processing and noise suppression processing to form a multimodal observation data set; The processing module is used to extract multidimensional feature parameters based on the multimodal observation data set, and combine them with the constructed biomathematical fatigue assessment model to generate the fatigue characterization quantity of the operating subject. The processing module is used to perform data-driven modeling processing on the multidimensional feature parameters to form a fusion representation model, and to jointly map the fatigue representation quantity with the fusion representation model to construct a human-machine collaborative perception representation space. The output module is used to perform comprehensive calculations on the coordination consistency and response efficiency of the interaction between the operating subject and the system based on the human-machine collaborative perception representation space, and generate state analysis results.

9. An electronic device, characterized in that, The device includes a processor, a communication bus, a user interface, a network interface, and a memory. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The communication bus is used to enable communication between the components within the electronic device. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-7.

Images