Operator performance measurement method and system based on resting difference

Abstract

The invention discloses an operator performance measurement method and system based on a resting difference value, and relates to the field of human factor engineering, and the method comprises the steps: enabling an operator to rest in a non-interference environment for a set time, and collecting a physiological signal; according to the physiological signals, multiple items of resting data are obtained through calculation; calculating a baseline mean value and a standard deviation of each piece of resting data; obtaining a signal standardization value according to the resting data, the baseline mean value and the baseline standard deviation; calculating an in-window mean value of the signal standardization value according to the sliding window to obtain a task period signal; calculating a dynamic difference value between a task period signal in a certain sliding window and the baseline mean value; obtaining the weight of each piece of resting data, and obtaining a comprehensive difference value index in a certain sliding window according to the weight and the dynamic difference value; and inputting the comprehensive difference index in a certain sliding window into a trained Gaussian process regression model, and outputting a performance score in a certain sliding window by the trained Gaussian process regression model. And dynamic fusion and real-time analysis of the multi-modal physiological signals are realized.

Description

Technical Field

[0001] This application belongs to the fields of human factors engineering, performance evaluation and biological signal processing technology, specifically involving an operator performance quantification method based on dynamic modeling of resting-state difference of multimodal physiological signals, which is applicable to real-time status monitoring and performance feedback in high-precision operation scenarios such as industrial control, medical surgery, and aviation driving. Background Technology

[0002] In complex operating environments (such as nuclear power plant controls, operating rooms, or aircraft cockpits), operator performance directly impacts mission safety and efficiency. Current technologies primarily rely on subjective ratings, task completion time, or error rates for operator performance evaluation, but these methods have significant limitations, as detailed below.

[0003] (1) Subjectivity and lag: It relies on manual scoring or statistics after the task is completed, which is difficult to standardize and cannot reflect the dynamic changes of the operator's state in real time. For example, although the inventor Liu Yuzhong's patent CN113367695B authorized in 2022, "A fatigue monitoring method and device based on EEG signals", proposed a fatigue monitoring method based on EEG signals, it still relies on analysis after the task is completed and lacks real-time early warning capabilities.

[0004] (2) Limitations of Single Signal: Traditional methods often rely on only a single physiological signal (such as heart rate, skin conductance, etc.), which is easily affected by environmental interference and lacks reliability. Although the inventor Liu Yuzhong's patent CN113367695B, authorized in 2022, "A Fatigue Monitoring Method and Device Based on EEG Signals", proposed a fatigue monitoring method based on EEG signals, it did not integrate multimodal signals such as heart rate variability (HRV) and skin conductance activity (EDA), resulting in poor anti-interference ability and insufficient reliability.

[0005] (3) Individual baseline differences: Resting physiological values ​​(such as EEG alpha wave energy) vary significantly among different operators, and directly using absolute value thresholds can lead to assessment bias. This issue was pointed out by Raja Parasuraman, author of the paper "Neuroergonomics: research and practice" published in the journal Frontiers In Neuroergonomics, who emphasized the impact of individual differences on assessment accuracy.

[0006] (4) Lack of closed-loop feedback mechanism: Existing methods are mostly limited to the monitoring stage and do not combine the evaluation results with real-time intervention, thus failing to achieve dynamic feedback and intervention of the evaluation results.

[0007] In summary, existing technologies for operator performance evaluation suffer from issues such as subjectivity, lag, limitations of single signals, individual baseline differences, and a lack of closed-loop feedback mechanisms. This paper focuses on integrating multimodal signals, considering individual differences, and achieving real-time evaluation and dynamic feedback to improve the accuracy and practicality of operator performance evaluation. Summary of the Invention

[0008] The technical problem solved by this application is to overcome the shortcomings of the prior art and provide a method for measuring operator performance that solves the problem of individual baseline differences, realizes dynamic fusion and real-time analysis of multimodal physiological signals, and provides a closed-loop feedback mechanism to adjust the operator's state in a timely manner.

[0009] The technical solution provided in this application is as follows:

[0010] Firstly, a method for measuring operator performance based on resting time difference is provided, including:

[0011] The operator rests in an undisturbed environment for a set time and collects physiological signals.

[0012] Based on the collected physiological signals, multiple resting data points were calculated.

[0013] Calculate the baseline mean μ of each resting data point. i and baseline standard deviation σ i ;

[0014] Based on various resting data and the corresponding baseline mean μ i and baseline standard deviation σ i Calculate the normalized value of the signal

[0015] Based on the signal standardization value of the sliding window Calculate the mean within the window to obtain the task period signal.

[0016]

[0017] Calculate the task duration signal within each sliding window. Dynamic difference ΔS from baseline mean i ;

[0018] Obtain the weights w of each resting data point i According to the weight w i and dynamic difference ΔS i Obtain the comprehensive difference index D within each sliding window. total ;

[0019] A Gaussian process regression model is used to map the comprehensive difference index to performance scores. The Gaussian process regression model is trained using historical task data to obtain the trained model. The historical task data includes: the input is the sequence of comprehensive difference indices for the historical task, and the output is the expert-annotated performance score corresponding to each comprehensive difference index in the sequence.

[0020] The comprehensive difference index D within a certain sliding window total Input the trained model, and the trained model outputs a performance score within a certain sliding window.

[0021] Furthermore, the physiological signals include electroencephalogram (EEG), heart rate variability, and skin conductance.

[0022] Resting-state data includes the energy ratio R of alpha and beta waves in electroencephalography (EEG). base The low-frequency to high-frequency power ratio of heart rate variability Q base and skin electrical conductivity average C base .

[0023] Furthermore, the baseline mean μ of each resting data point i and baseline standard deviation σ i They are respectively:

[0024]

[0025] Where i = 1 - M, M is the number of items in the resting data, M = 3; S i (t) represents the value of the i-th resting data at time point t, and N represents the total number of sampling points during the resting period.

[0026] Furthermore, the signal normalization value for:

[0027]

[0028] Where, μ i σ is the baseline mean; i S represents the baseline standard deviation; i (t) represents the value of the i-th resting data point at time t.

[0029] Furthermore, the task period signal for:

[0030]

[0031] in, The signal is standardized; the time range of a certain sliding window is [k, k+T]; T is the length of the sliding window.

[0032] Furthermore, the dynamic difference ΔS ifor:

[0033]

[0034] in, denoted as , where μi is the baseline mean.

[0035] Furthermore, the comprehensive difference index D total for:

[0036]

[0037] Where i = 1 - M, M is the number of items in the resting data, and M = 3.

[0038] Furthermore, the method of mapping the comprehensive difference index to performance scores using a Gaussian process regression model includes:

[0039] P = f(D) total )+∈

[0040] Wherein, the performance score P∈[0,1]; ∈ represents Gaussian noise.

[0041] Furthermore, when the performance score P < 0.7 within a certain sliding window output by the trained model, a tiered intervention is triggered;

[0042] The tiered intervention includes highlighting key parameters on the interface when 0.5 ≤ P < 0.7;

[0043] When P < 0.5, reduce task complexity;

[0044] If P < 0.5 for three consecutive sliding windows, the emergency operation protocol interface will be displayed.

[0045] Secondly, a resting-state difference-based operator performance measurement system is provided, including:

[0046] The acquisition module is used to collect physiological signals from the operator during a set resting time in an undisturbed environment.

[0047] The calculation module calculates resting data based on the collected physiological signals; it denoises each resting data point and calculates the baseline mean μ for each resting data point. i and baseline standard deviation σ i Based on various resting data and the corresponding baseline mean μ i and baseline standard deviation σ i ; calculate the normalized value of the signal. Based on the signal standardization value of the sliding window Calculate the mean within the window to obtain the task period signal. Calculate the task duration signal within a certain sliding window. Dynamic difference ΔS from baseline mean i The weights w of each resting data point are obtained using the entropy weight method. i According to the weight w i and dynamic difference ΔS i Obtain the comprehensive difference index D within a certain sliding window total ;

[0048] The model module employs a Gaussian process regression model, trained using historical task data. The input is a sequence of comprehensive difference indicators from the historical tasks, and the output is expert-annotated performance scores. The comprehensive difference indicator D within a sliding window... total Input the model module, and the model module outputs the performance score within a sliding window;

[0049] The judgment and execution module determines whether to trigger tiered intervention based on the performance score output by the model module, and executes it according to the judgment result.

[0050] The advantages of this invention compared to the prior art are:

[0051] 1. Individual adaptability: By standardizing the resting baseline and comparing the resting baseline rather than the absolute threshold, the physiological characteristics of different operators are adapted to eliminate individual differences (experimental data show that the assessment error is reduced by 45%).

[0052] 2. Anti-interference capability: Multimodal entropy weight fusion enables the system to maintain high robustness under environmental noise (accuracy improved to 92% in the case of skin signal interference).

[0053] 3. Real-time performance: The entire process from signal acquisition to feedback response has a delay of less than 3 seconds, meeting the requirements of high-risk scenarios.

[0054] 4. Closed-loop optimization: By combining real-time data with historical models, a "monitoring-evaluation-intervention" closed loop is achieved, dynamically adjusting the difficulty of tasks and improving operator performance by an average of 25% (based on nuclear power plant control room simulation test data). Attached Figure Description

[0055] Figure 1 This is a flowchart of an operator performance measurement method based on resting difference, as shown in the embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments disclosed in this application will be described in further detail below with reference to the accompanying drawings.

[0057] This application discloses an operator performance measurement method based on resting difference, such as... Figure 1 As shown, it includes:

[0058] 1. Resting baseline modeling and normalization

[0059] (1) The operator rests in an undisturbed environment for at least 10 minutes and collects the following signals:

[0060] Electroencephalography (EEG): Extracting the energy ratio of alpha waves (8-13Hz) to beta waves (14-30Hz).

[0061] Heart rate variability (HRV): Calculate the power ratio of low-frequency (LF, 0.04-0.15Hz) to high-frequency (HF, 0.15-0.4Hz) frequencies.

[0062] Electrical skin activity (EDA): Measured average skin conductivity C base .

[0063] (2) After denoising the resting data, calculate the baseline mean and standard deviation of each physiological signal:

[0064]

[0065] Among them, S i (t) represents the value of the i-th type of signal at time point t, N is the total number of sampling points during the resting period; i = 1 - M, M = 3, M is the number of physiological indicators (physiological indicators include EEG, HRV and EDA).

[0066] 2. Dynamic standardization of signals during the mission period

[0067] (1) Calculate the signal standardization value in real time while the operator is performing the task:

[0068]

[0069] (2) Calculate the mean value within the sliding window (length T = 60 seconds, for example, time window is 0-60S, 61-120S, 121-180S...) on the standardized signal to obtain the signal during the mission period.

[0070] The time range of a certain window is [k, k+T];

[0071]

[0072] 3. Dynamic fusion of resting difference

[0073] (1) Calculate the dynamic difference between the task period signal and the resting baseline within a certain window:

[0074]

[0075] (2) Assigning weights w to multiple signals using the entropy weighting methodi Generate a comprehensive difference index D within a certain window. total : in

[0076] (3) Weight calculation is based on the information entropy H of each signal. i The historical data is divided into K intervals, k∈[1,K], where p k Let i be the probability that signal i falls into the k-th interval in the historical data.

[0077]

[0078] Among them, H i Let 1 be the entropy value of each physiological signal, and 1 - entropy value is the difference coefficient. The larger the difference coefficient, the greater the information content of the indicator, and the higher the weight. The weight is obtained by dividing the difference index of each indicator by the sum.

[0079] 4. Non-linear mapping of performance ratings

[0080] (1) Using the Gaussian process regression (GPR) model to transform D total Mapped to performance ratings P∈[0,1]:

[0081]

[0082] Where ∈ represents Gaussian noise; σ 2 This represents the noise variance.

[0083] (2)f(D total A kernel function is used, which combines radial basis function (RBF) with white noise.

[0084]

[0085] Where x represents the training data; x′ represents the test data; l and σ noise is a hyperparameter; exp represents an exponential function with the natural constant e as its base.

[0086] (3) During model training, the input is the D of the historical task. total The sequence outputs expert-annotated performance scores (such as quantitative values ​​of surgical success rate), and the hyperparameters l and σ are optimized using a historical dataset (containing 1000 sets of operator task data). noise .

[0087] 5. Dynamic feedback control: When P < 0.7 within the window, tiered intervention is triggered.

[0088] (1) Level 1 warning (0.5≤P<0.7): The interface highlights key parameters.

[0089] (2) Secondary intervention (P<0.5): Automatically reduce task complexity (e.g., limit the sensitivity of the aircraft control stick).

[0090] (3) Level 3 intervention (when P < 0.5 in 3 consecutive windows): The system automatically brings up the emergency operation protocol interface.

[0091] This embodiment also discloses an operator performance measurement system based on resting difference, including:

[0092] The acquisition module is used to collect physiological signals from the operator during a set resting time in an undisturbed environment.

[0093] The calculation module calculates resting data based on the collected physiological signals; it denoises each resting data point and calculates the baseline mean μ for each resting data point. i and baseline standard deviation σ i Based on various resting data and the corresponding baseline mean μ i and baseline standard deviation σ i ; calculate the normalized value of the signal. Based on the signal standardization value of the sliding window Calculate the mean within the window to obtain the task period signal. Calculate the task duration signal within a certain sliding window. Dynamic difference ΔS from baseline mean i The weights w of each resting data point are obtained using the entropy weight method. i According to the weight w i and dynamic difference ΔS i Obtain the comprehensive difference index D within a certain sliding window total ;

[0094] The model module employs a Gaussian process regression model, trained using historical task data. The input is a sequence of comprehensive difference indicators from the historical tasks, and the output is expert-annotated performance scores. The comprehensive difference indicator D within a sliding window... total Input the model module, and the model module outputs the performance score within a sliding window;

[0095] The judgment and execution module determines whether to trigger tiered intervention based on the performance score output by the model module, and executes it according to the judgment result.

[0096] In this application's usage method, it is necessary to first process historical data from the actual application scenario, pre-train the GPR model, and assign weights to each physiological signal in this scenario, as shown in the attached document. Figure 1 As shown below, examples of common human safety scenarios will be used.

[0097] Example 1: Taking a nuclear power plant control room operator as an example

[0098] 1. Training the model and assigning weights

[0099] (1) Training the model: The historical dataset (physiological signals and performance scores) from the nuclear power plant control room was used as input to train the GPR model. Physiological signals included electroencephalography (EEG), heart rate variability (HRV), and electrical skin activity (EDA).

[0100] (2) Weighting: 1000 historical data points from the nuclear power plant control room were processed using the entropy weighting method, and the signal weight w was derived based on the signal volatility. EEG =0.4,w HRV =0.3,w EDA =0.3.

[0101] 2. Task Process

[0102] (1) Resting modeling: Before starting work, operators rest for 10 minutes in an undisturbed environment and collect physiological data such as electroencephalogram (EEG) and heart rate variability (HRV) as required.

[0103] (2) Signal preprocessing: The acquired signals are preprocessed (denoised) to obtain the baseline mean μ of each signal. EEG =2.1,μ HRV =1.8,μ EDA =4.3μS.

[0104] (3) Data collection during the task period: Physiological signals are collected in real time while the operator is performing the task (a sliding window is formed every 5 seconds), and the average signal value within the sliding time window is calculated.

[0105] (4) Weight allocation: The signal weight w is derived based on the signal volatility. EEG =0.4,w HRV =0.3,w EDA =0.3.

[0106] (5) Difference Calculation: Calculate the dynamic difference (ΔS) between the signal during the mission period and the resting baseline. i ).

[0107] The dynamic difference between the mission-period signal and the resting baseline is combined with multi-signal weighting to generate a comprehensive difference index (D). total For example: In an operational task, the dynamic difference ΔS of a certain window is calculated in real time. EEG =-0.5,ΔS HRV =0.7,ΔS EDA =1.2, then the comprehensive difference index is D. total =0.4×(-0.5)+0.3×0.7+0.3×1.2=0.37.

[0108] (6) Model prediction: The comprehensive difference index D total Inputting the pre-trained GPR model, the output real-time performance score P = 0.63.

[0109] (7) Feedback mechanism: When a Level 1 warning is triggered, the system will automatically bring up the auxiliary decision-making interface and notify the duty supervisor.

[0110] Example 2: Taking minimally invasive surgery as an example

[0111] (1) Resting baseline modeling: Physicians were at rest before surgery, and μ was measured. HRV =1.5,μ EDA =5.2μS.

[0112] (2) Monitoring during the mission: Intraoperative D total =0.42, GPR model output P=0.58, triggering secondary intervention.

[0113] (3) Feedback execution: The surgical navigation system automatically magnifies the field of view by 3 times and shortens the instrument response delay to 50ms.

[0114] Example 3: Taking an aviation piloting scenario as an example

[0115] (1) Resting baseline modeling: The pilot rests in the cockpit for 10 minutes and collects μ data. EEG =1.8,μ HRV =2.3,μ EDA =3.7μS.

[0116] (2) Mission monitoring: Real-time calculation of ΔS during flight EEG = -0.6 (β wave energy decrease), triggering an early warning.

[0117] (3) Feedback execution: The system automatically reduces the sensitivity of the joystick and prompts the co-driver to take over some operations.

[0118] The contents not described in detail in this application specification are common knowledge to those skilled in the art.

[0119] The present application has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present application. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and implementation methods of the present application without departing from the spirit and scope of the present application, and all such modifications and improvements fall within the scope of the present application. The scope of protection of the present application is determined by the appended claims.

Claims

1. A method for measuring operator performance based on resting time difference, characterized in that, include: The operator rests in an undisturbed environment for a set time and collects physiological signals. Based on the collected physiological signals, multiple resting data points were calculated. Calculate the baseline mean μ of each resting data point. i and baseline standard deviation σ i ; Based on various resting data and the corresponding baseline mean μ i and baseline standard deviation σ i Calculate the normalized value of the signal Based on the signal standardization value of the sliding window Calculate the mean within the window to obtain the task period signal. Calculate the task duration signal within each sliding window. Dynamic difference ΔS from baseline mean i ; Obtain the weights w of each resting data point i According to the weight w i and dynamic difference ΔS i Obtain the comprehensive difference index D within each sliding window. total ; A Gaussian process regression model is used to map the comprehensive difference index to a performance score; the Gaussian process regression model is trained using historical task data to obtain the trained model; The comprehensive difference index D within a certain sliding window total Input the trained model, and the trained model outputs a performance score within a certain sliding window.

2. The operator performance measurement method based on resting difference according to claim 1, characterized in that, The physiological signals include electroencephalography (EEG), heart rate variability, and skin conductance. Resting-state data includes the energy ratio R of alpha and beta waves in electroencephalography (EEG). base The low-frequency to high-frequency power ratio of heart rate variability Q base and skin electrical conductivity average C base .

3. A method for measuring operator performance based on resting time difference according to claim 1 or 2, characterized in that, The baseline mean μ of each resting data point i and baseline standard deviation σ i They are respectively: Where i = 1 - M, M is the number of items in the resting data, M = 3; S i (t) represents the value of the i-th resting data at time point t, and N represents the total number of sampling points during the resting period.

4. The operator performance measurement method based on resting difference according to claim 1, characterized in that, The signal normalization value for: Where, μ i σ is the baseline mean; i S represents the baseline standard deviation; i (t) represents the value of the i-th resting data point at time t.

5. The operator performance measurement method based on resting difference according to claim 1, characterized in that, The task period signal for: in, The signal is standardized; the time range of a certain sliding window is [k, k+T]; T is the length of the sliding window.

6. The operator performance measurement method based on resting difference according to claim 1, characterized in that, The dynamic difference ΔS i for: in, denoted as , where μi is the baseline mean.

7. The operator performance measurement method based on resting difference according to claim 1, characterized in that, The comprehensive difference index D total for: Where i = 1 - M, M is the number of items in the resting data, and M = 3.

8. The operator performance measurement method based on resting difference according to claim 1, characterized in that, The method of mapping the comprehensive difference index to performance scores using a Gaussian process regression model includes: P=f(D total )+∈ Wherein, the performance score P∈[0,1]; ∈ represents Gaussian noise.

9. The operator performance measurement method based on resting difference according to claim 1, characterized in that: When the trained model outputs a performance score P < 0.7 within a certain sliding window, a tiered intervention is triggered. The tiered intervention includes highlighting key parameters on the interface when 0.5 ≤ P < 0.7; When P < 0.5, reduce task complexity; If P < 0.5 for three consecutive sliding windows, the emergency operation protocol interface will be displayed.

10. An operator performance measurement system based on resting time difference, characterized in that, include: The acquisition module is used to collect physiological signals from the operator during a set resting time in an undisturbed environment. The calculation module calculates resting data based on the collected physiological signals; Denoise the resting data points and calculate the baseline mean μ for each resting data point. i and baseline standard deviation σ i Based on various resting data and the corresponding baseline mean μ i and baseline standard deviation σ i ; calculate the normalized value of the signal. Based on the signal standardization value of the sliding window Calculate the mean within the window to obtain the task period signal. Calculate the task duration signal within each sliding window. Dynamic difference ΔS from baseline mean i The weights w of each resting data point are obtained using the entropy weight method. i According to the weight w i and dynamic difference ΔS i Obtain the comprehensive difference index D within each sliding window. total ; The model module employs a Gaussian process regression model, trained using historical task data. The input is a sequence of comprehensive difference indicators from the historical tasks, and the output is expert-annotated performance scores. The comprehensive difference indicator D within a sliding window... total Input the model module, and the model module outputs the performance score within a sliding window; The judgment and execution module determines whether to trigger tiered intervention based on the performance score output by the model module, and executes it according to the judgment result.

Images