Method and system for identifying emotional state of operator in main control room of nuclear power plant

Abstract

The invention discloses a method and system for identifying the emotional state of an operator in a main control room of a nuclear power plant, and belongs to the technical field of nuclear power safety, and the method for identifying the emotional state of the operator in the main control room of the nuclear power plant comprises the steps: collecting multi-modal original signals of the operator in the main control room and carrying out data preprocessing and anti-interference processing; obtaining task context information of a current operator; calculating a fatigue degree index and an emotional state level of the operator locally on a console by using an emotional state recognition model based on the preprocessed multi-modal signal features and task context information; and a grading risk score is generated based on the fatigue degree index and the emotional state grade, the current state risk grade of the operator is judged based on the grading risk score, corresponding operation is executed according to the risk grade, and the risk score and associated feature summary information are encrypted and sent to an on-duty management terminal. The method is high in anti-interference capability, stable and reliable in signal, high in recognition accuracy and low in misjudgment rate, and privacy and data security are fundamentally guaranteed.

Description

Technical Field

[0001] This invention belongs to the field of nuclear power safety technology, specifically relating to a method and system for recognizing the emotional state of operators in the main control room of a nuclear power plant. Background Technology

[0002] The statements herein provide only background information in relation to this invention and do not necessarily constitute prior art.

[0003] In the safe operation of nuclear power plants, the mental state of the main control room operators (such as fatigue, tension, anger, etc.) is a key factor affecting personnel reliability. Currently, the nuclear power industry mainly relies on manual observation and institutional management to monitor the operators' condition, lacking objective, continuous, and quantitative monitoring methods.

[0004] Although fatigue and emotion recognition technologies based on multimodal physiological signals (such as facial images, heart rate, and skin conductance) already exist in fields such as automotive driving and industrial safety, the inventors have discovered that these technical solutions face the following inherent defects and adaptation challenges when directly applied to the special environment of a nuclear power plant's main control room: (1) Environmental interference leads to signal inaccuracy: The main control room has complex and variable lighting (such as large screen glare and local shadows) and strong electromagnetic interference, which makes it easy for vision-based image analysis to produce errors. The weak physiological signals (such as rPPG heart rate signals) have low signal-to-noise ratio, which seriously reduces the recognition accuracy and reliability of the existing technology.

[0005] (2) Special working conditions lead to misjudgment of behavior: The operator’s operating posture in front of the fixed structure control console is diverse (such as leaning forward to operate, leaning back to monitor, and talking sideways). Existing technology uniformly regards it as a “driver” sitting posture for analysis, which makes it easy to misjudge normal operating behavior as fatigue or emotional abnormality due to changes in posture.

[0006] (3) Data security and privacy compliance risks: Nuclear power plants have extremely high requirements for data security, privacy protection and audit traceability. Existing technical solutions mostly use cloud or central servers to process raw biometric data, which poses a risk of data leakage and is difficult to meet the compliance requirements for sensitive data processing in nuclear safety regulations. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for recognizing the emotional state of operators in the main control room of a nuclear power plant. This method and system have strong anti-interference capabilities, stable and reliable signals, high recognition accuracy, low misjudgment rate, and fundamental protection of privacy and data security.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solution: In a first aspect, the technical solution of the present invention provides a method for recognizing the emotional state of operators in the main control room of a nuclear power plant, including: Collect multimodal raw signals from operators in the main control room and perform data preprocessing and anti-interference processing; Obtain the current operator's task context information; Based on the preprocessed multimodal signal features and task context information, the operator's fatigue level and emotional state level are calculated locally at the control console using an emotion state recognition model. The emotion state recognition model includes a fatigue discrimination model and an emotion state level recognition model. The weights of each modality signal in the fatigue discrimination model are dynamically adjusted based on the task context information. A graded risk score is generated based on fatigue level indicators and emotional state levels. The risk score is used to determine the current risk level of the operator in the main control room, and corresponding operations are performed according to the risk level. At the same time, the risk score and associated feature summary information are encrypted and sent to the duty management terminal.

[0009] In at least one embodiment, the multimodal raw signals include an image of the operator's face, a near-infrared image, a heart rate signal, skin conductance signals, seat pressure distribution data, and data on changes in the pressure applied to operating buttons or a mouse.

[0010] In at least one embodiment, data preprocessing and anti-interference processing specifically include: Multi-scale illumination compensation algorithms are applied to facial and near-infrared images to address the complex lighting conditions in the main control room. Bandpass filtering and frequency domain noise suppression are performed on the heart rate signal; A short-time baseline adaptive calibration algorithm is performed on skin conductance signals, seat pressure distribution data, and data on changes in the pressure of operating buttons or mice to eliminate the influence of individual physiological differences among operators. The raw signals of the multimodal status of the main control room operator are collected and protected against electromagnetic interference through hardware shielding and grounding structures.

[0011] In at least one embodiment, the task context information includes task type, operating condition parameters, and alarm level information; wherein, the task type includes shift handover and administrative tasks, steady-state monitoring, planned operations, abnormal operating condition response, and design basis accident response.

[0012] In at least one embodiment, the fatigue discrimination model is specifically represented as follows:

[0013] In the formula, Values ​​representing the degree of fatigue; Indicates physiological characteristics; Indicates behavioral characteristics; Indicates operational characteristics; , and These are the weights corresponding to physiological characteristics, behavioral characteristics, and operational characteristics, respectively. Indicates task context modification items; The emotion state level recognition model adopts a two-dimensional model of "valence-arousal". It takes the preprocessed multimodal signal as input, calculates valence and arousal respectively, and uses the valence value and arousal value as the x-coordinate and y-coordinate of the emotion state point, respectively. It determines the quadrant in which the emotion state point falls, and the number of quadrants in which it falls is used as the value of the emotion state level. Based on the emotion state level, it outputs the emotion state label and the emotion state risk level.

[0014] In at least one embodiment, the weights of each modal signal in the fatigue discrimination model are dynamically adjusted based on task context information. Specifically, this includes: automatically increasing the weights of physiological and operational characteristics when the task type is abnormal working condition response or design baseline accident response, in order to quickly identify high-load conditions; and increasing the weights of seat pressure distribution and posture characteristics when the task type is steady-state monitoring, planned operation, shift handover and administrative tasks, in order to focus on identifying persistent fatigue.

[0015] In at least one embodiment, the fatigue level index includes low fatigue, moderate fatigue, and high fatigue; the emotional state labels include a low-risk calm / focused label, a low-to-medium-risk excited / positive label, a medium-to-high-risk fatigue / depressed label, and a high-risk anxiety / tension label.

[0016] In at least one embodiment, a graded risk score is generated based on fatigue level indicators and emotional state levels, specifically as follows: A graded risk score is generated based on fatigue level indicators and emotional state risk levels, and is represented as follows:

[0017] In the formula, This indicates the final risk score. Values ​​representing the degree of fatigue; This indicates the risk level of the emotional state; Indicates the severity of the task; Indicates fatigue weight; Indicates sentiment weight; This represents the environmental weight.

[0018] In at least one embodiment, the risk level of the current status of the operator in the main control room is determined based on a graded risk score, and corresponding operations are performed according to the risk level, specifically including: When the risk level is low, there is no early warning. The remote duty management terminal only records data for long-term trend analysis and reporting. When the risk level is medium, a soft warning is issued to the operator through the local warning device, and an encrypted real-time data packet is sent to the duty management terminal, suggesting that the duty manager conduct remote observation. When the risk level is medium, a mandatory early warning is issued to the operator through the local warning device, and an encrypted real-time data packet is immediately sent, prompting the operator to implement intervention decision-making processes such as shift change or rest. The encrypted real-time data packet includes a risk score, task context, and associated feature summary information; the feature summary information includes blink frequency and eye closure ratio, heart rate variability index, seat pressure change amplitude, and operation pressure change frequency data, accompanied by timestamps, operator identification, and task context summary.

[0019] Secondly, the technical solution of the present invention also provides a system for recognizing the emotional state of operators in the main control room of a nuclear power plant, comprising: The data acquisition and preprocessing module is configured to: acquire multimodal raw signals from the main control room operator and perform data preprocessing and anti-interference processing; The task acquisition module is configured to: acquire the task context information of the current operator; The recognition module is configured to: calculate the operator's fatigue level index and emotional state level locally on the control panel based on the preprocessed multimodal signal features and task context information using an emotion state recognition model; wherein, the emotion state recognition model includes a fatigue discrimination model and an emotion state level recognition model; and dynamically adjust the weights of each modality signal in the fatigue discrimination model based on task context information. The early warning module is configured to: generate a graded risk score based on fatigue level indicators and emotional state level; determine the current risk level of the operator in the main control room based on the graded risk score; execute corresponding operations according to the risk level; and simultaneously encrypt and send the risk score and associated feature summary information to the duty management terminal.

[0020] The beneficial effects of the above-described technical solution of the present invention are as follows: 1) The method for recognizing the emotional state of operators in the main control room of a nuclear power plant, as described in this invention, effectively overcomes the impact of the complex environment and electromagnetic interference in the main control room of a nuclear power plant on signal quality through specialized data preprocessing and anti-interference steps, providing a reliable data foundation for accurate judgment; by introducing task context information and dynamically adjusting model weights, it effectively distinguishes between normal operating postures and real fatigue and emotional abnormalities, significantly reducing the misjudgment rate; adopting a core architecture of "local processing without uploading raw data," it only uploads encrypted feature summaries, eliminating the risk of leakage of operators' biometric privacy data from the technical source, and fully meeting the compliance requirements of nuclear safety regulations. 2) This invention also has the functions of multimodal consistency verification and long-term risk trend analysis, which not only improves the reliability of single judgment, but also provides managers with scientific and traceable decision support, and comprehensively improves the human safety level of nuclear power operation. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a schematic diagram of the method for recognizing the emotional state of operators in the main control room of a nuclear power plant, as disclosed in Embodiment 1 of the present invention. Figure 2 This is a schematic diagram of the emotional state recognition system for operators in the main control room of a nuclear power plant, as disclosed in Embodiment 2 of the present invention. Detailed Implementation

[0023] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] As described in the background section, the purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for recognizing the emotional state of operators in the main control room of a nuclear power plant. This method and system have strong anti-interference capabilities, stable and reliable signals, high recognition accuracy, low misjudgment rate, and fundamental protection of privacy and data security.

[0025] Example 1 In a typical embodiment of the present invention, such as Figure 1 As shown in the figure, this embodiment discloses a method for recognizing the emotional state of operators in the main control room of a nuclear power plant, including: S1. Collect raw signals of multimodal status of operators in the main control room and perform data preprocessing and anti-interference processing; S2. Obtain the current operator's task context information; S3. Based on the preprocessed multimodal signal features and task context information, the operator's fatigue level index and emotional state level are calculated locally on the control panel using an emotion state recognition model; the emotion state recognition model includes a fatigue discrimination model and an emotion state level recognition model; the weights of each modality signal in the fatigue discrimination model are dynamically adjusted based on task context information. S4. Generate a graded risk score based on fatigue level indicators and emotional state level. Determine the current risk level of the operator in the main control room based on the graded risk score, and perform corresponding operations according to the risk level. At the same time, encrypt and send the risk score and associated feature summary information to the duty management terminal.

[0026] The above-described method for recognizing the emotional state of operators in the main control room of a nuclear power plant will be described in detail below with reference to specific implementation methods.

[0027] S1. Collect raw signals of multimodal status of operators in the main control room and perform data preprocessing and anti-interference processing.

[0028] In this step, the operator's multimodal status raw signals include facial images, near-infrared images, heart rate signals, skin conductance signals, seat or backrest pressure distribution data, and data on changes in the pressure applied to control handles or buttons. These raw status signals are acquired synchronously by multimodal sensors fixedly mounted on the main control room console or its auxiliary structures.

[0029] Specifically, a dual-spectrum camera module located at the upper edge of the control panel acquires visible light and near-infrared images of the operator's face in real time; heart rate signals are acquired based on photoplethysmography (rPPG); pressure sensors and touch capacitive sensors integrated into the control panel buttons or handles acquire data on changes in the pressure applied to the control handles or buttons in real time; flexible pressure distribution sensing pads deployed on the seat cushion and backrest acquire data on the pressure distribution of the seat or backrest in real time; and miniature physiological sensors acquire data on the operator's skin conductivity and skin temperature in real time.

[0030] In this step, after acquiring the raw multimodal status signals of the operator in the main control room, it is necessary to perform noise reduction, illumination compensation, electromagnetic interference filtering, and individual baseline adaptive calibration on the raw signals to ensure that effective signals can still be stably acquired under complex lighting and interference environments.

[0031] Specifically, a multi-scale illumination compensation algorithm is executed on the visible light and near-infrared images of the operator's face acquired by the dual-spectrum camera module to cope with the complex lighting conditions in the main control room; bandpass filtering and frequency domain noise suppression processing are performed on the photoplethysmography pulse wave signal; and a short-time baseline adaptive calibration algorithm is executed on the skin conductance signal, seat pressure distribution data, and data on changes in the pressure of operating buttons or mouse to eliminate the influence of individual physiological differences of the operator; the raw signals of the multimodal status of the operator in the main control room acquired above are protected against electromagnetic interference through hardware shielding and grounding structure.

[0032] Different modal sensors may capture the same physical event at the same time point or within a time window. If the image signal and physiological signal do not change within the corresponding time window, an anomaly may exist. Therefore, in this step, the preprocessed facial features, photoplethysmography (PPG) pulse wave signal, and pressure distribution signal are compared for consistency. This checks whether the signals from different sensors are semantically or logically mutually supportive. When significant inconsistencies occur between signals from different modalities, sensor anomaly markers are output, and the corresponding discrimination results are filtered.

[0033] Specifically, given the known correlation between signal characteristics of different modalities under specific physiological or psychological states of the operator, the system first defines a characteristic pattern for a "high load" state for each modality. Then, in real-time analysis, if one modality (such as heart rate) strongly indicates "high load," but other modalities (such as facial expression, skin conductance) show "low load" or "relaxation," it is determined that there is a significant inconsistency between the signals of different modalities, and sensor anomaly markers are output and the corresponding discrimination results are filtered.

[0034] Finally, the signals of different modes are standardized to eliminate the influence of dimensions between the signals of different modes.

[0035] S2. Obtain the task context information of the current operator.

[0036] In this step, the task context acquisition unit, connected to the distributed control system (DCS) and operation management system, acquires the current operator's task context information in real time, including task type, operating condition parameters, and alarm level information. Task types include shift handover and administrative tasks, steady-state monitoring, planned operations, abnormal condition responses, and design-basis accident responses. Alarm level information is based on the nuclear power plant's alarm warning system, categorized from highest to lowest severity as Level 1, Level 2, Level 3, and Level 4 alarms. Level 1 alarms have the highest urgency and severity, requiring immediate action.

[0037] S3. Based on the preprocessed multimodal signal features and task context information, the operator's fatigue level and emotional state level are calculated locally on the control panel using an emotion state recognition model.

[0038] In this step, an emotion state recognition model is first constructed, which includes a fatigue discrimination model and an emotion state level recognition model, respectively outputting the operator's fatigue level index and emotion state level calculated locally on the control panel.

[0039] Specifically, the fatigue discrimination model can be expressed as follows:

[0040] In the formula, Values ​​representing the degree of fatigue; Indicates physiological characteristics, such as HRV, Waves, etc.; Indicates behavioral characteristics, such as blinking, posture, etc.; Indicates operational characteristics, such as changes in pressure; , and These are the weights corresponding to physiological characteristics, behavioral characteristics, and operational characteristics, respectively. This indicates a task context modification.

[0041] In this step, the weights of each modal signal in the fatigue discrimination model are dynamically adjusted based on contextual information under task context awareness. Specifically, when the task type is abnormal operating condition response or design baseline accident response, the weights of physiological and operational characteristics are automatically increased to quickly identify high-load conditions; when the task type is steady-state monitoring, planned operation, shift handover, or administrative tasks, the weights of seat pressure distribution and posture characteristics are increased to focus on identifying persistent fatigue.

[0042] Based on a fatigue discrimination model, fatigue level index values ​​are output, and the fatigue levels are divided into three levels: low fatigue, moderate fatigue, and high fatigue, according to the magnitude of the fatigue level index values. Among these, when... If the output is low fatigue, then the output is low fatigue. If the output is medium fatigue, then the output is medium fatigue. When this happens, the output will be "high fatigue".

[0043] In this step, the emotion state level recognition model adopts a two-dimensional "valence-arousal" model. Using preprocessed multimodal signals as input, it identifies the operator's key emotions on the control panel during the working state and calculates the emotion state level. It also outputs the emotional state label and the emotional state risk level.

[0044] Specifically, the valence is first calculated based on the preprocessed facial images, seat pressure, and operational pressure. It mainly reflects the inner feelings and tendencies of emotions, and can be expressed as:

[0045] In the formula, This indicates the first modal signal extracted from the preprocessed multimodal signal. Each feature is represented; Indicates the first Each feature represents the effect on valence. The contribution of the feature is as follows: the higher the contribution, the closer the absolute value is to 1. In this step, facial micro-expressions are extracted based on the preprocessed facial images. When negative expressions appear, such as drooping corners of the mouth and tucked-in eyebrows, they directly reflect unhappiness, anxiety, or frustration, and the contribution of this feature is negative. When positive expressions appear, such as upturned corners of the mouth, the contribution of this feature is positive. Based on the preprocessed seat pressure, features such as the frequency and amplitude of center of gravity shift are extracted to judge body stability. If body instability is present, it indicates negative emotions such as anxiety or irritability, and the contribution of this feature is negative. If the body is relaxed, does not move randomly, and the frequency of posture shift decreases, it indicates body stability, and the contribution of this feature is positive. Based on the preprocessed operation pressure, features such as the variance of pressure intensity are extracted to judge operation stability. If the operation accuracy decreases and the force is unstable, it reflects hesitation, frustration, or anxiety leading to operation instability, and the contribution of this feature is negative. If the operation delay decreases, the operation is confident, and the task is executed smoothly, it indicates operation stability, and the contribution of this feature is positive. In this step, the contribution of the feature to the valence is calculated using massive amounts of data to arrive at a preliminary optimal value, which is then fine-tuned and solidified by evaluation experts based on practical experience.

[0046] In this step, when the mood is pleasant and positive, the valence is high. The value approaches When feeling frustrated or anxious, valence... The value approaches .

[0047] Then, arousal level is calculated based on preprocessed skin conductance signals, heart rate signals, near-infrared images, and operational pressure. It mainly reflects the body's energy level and physiological stress response, and can be expressed as:

[0048] In the formula, This indicates the first modal signal extracted from the preprocessed multimodal signal. Each feature is represented; Indicates the first Each feature represents the relationship between arousal level and wakefulness. The higher the contribution, the closer the absolute value is to 1; This is an external correction term, with a value of 1 or 2; 2 for Level 1 alarm and 1 for others. In this step, features such as the number of GSR peaks and transient response amplitude are extracted from the preprocessed skin conductance signal to determine the activation level of the autonomic nervous system. Larger GSR fluctuations and higher frequencies indicate stronger autonomic nervous system activation and higher arousal (tension, anxiety), with a positive contribution. Reduced transient responses and lack of interference indicate no sudden stimulus causing physiological activation, with a negative contribution. The LF / HF ratio and HRVrMSSD heart rate pattern of heart rate variability (HRV) are extracted from the preprocessed heart rate signal. An increased ratio indicates physiological stress and increased arousal. The feature representation has the following characteristics: a positive contribution indicates a positive heart rate, while a negative contribution indicates relaxation or rest, suggesting a normal heart rate and physiological recovery. Based on the preprocessed near-infrared image, features such as pupillary distance diameter, blink frequency, and facial muscle characteristics are extracted. Pupil dilation and increased blink frequency are typical signs of cognitive load or stress, resulting in a positive contribution. Relaxed facial muscles and no frowning indicate a negative contribution. Based on the preprocessed pressure applied during operation, a pressure intensity feature is extracted. A significantly increased average pressure intensity indicates muscle tension, resulting in a positive contribution; otherwise, it is negative. In this step, the contribution of these features to arousal is calculated using massive amounts of data to arrive at a preliminary optimal value, which is then fine-tuned and solidified by evaluation experts based on practical experience.

[0049] In this step, arousal is increased when experiencing tension or anger. The value approaches When calm and relaxed, arousal level is high. The value approaches .

[0050] Valence Value and wake-up rate The values ​​are used as the x and y coordinates of the emotional state point, respectively. The quadrant in which the emotional state point falls is determined, and the number of quadrants it falls into is used as the emotional state level. The value of . When the emotional state point Falling into the first quadrant, If the emotional state is calm / focused, the output label will be calm / focused, and the risk level will be low; when the emotional state point is... Falling into the second quadrant, If the emotional state is positive, an excited / positive label will be output, with a risk level of low to medium risk; when the emotional state point is positive... Falling into the third quadrant, If the emotional state is low, the output label will be fatigue / depressed, and the risk level will be medium to high. Falling into the fourth quadrant, If so, the anxiety / tension label will be output, and the risk level will be high.

[0051] S4. Generate a graded risk score based on fatigue level indicators and emotional state level. Determine the current risk level of the operator in the main control room based on the graded risk score, and perform corresponding operations according to the risk level. At the same time, encrypt and send the risk score and associated feature summary information to the duty management terminal.

[0052] In this step, a graded risk score is generated based on the fatigue level index and emotional state risk level output from step S3, which can be specifically represented as:

[0053] In the formula, This represents the final risk rating; the higher the score, the greater the perceived risk of a safety incident due to human error. The fatigue level index is used to evaluate the operator's fatigue level; the higher the value, the greater the risk. This indicates the risk level of an emotional state; the higher the level, the greater the risk. It indicates the severity of the task and is used to assess the current danger of the external environment of the nuclear power plant. The higher the level, the more serious the consequences. This indicates the fatigue weight, which determines the importance of the fatigue level index value in the overall risk. This represents the emotional weight, which determines the importance of emotional state in the overall risk. This represents the environmental weight, which determines the importance of the degree of external environmental risk in the total risk.

[0054] Among them, the severity of the task Specifically, this refers to the score of task type in alarms at all levels, as shown in Table 1.

[0055] Table 1 Task Severity Classification

[0056] Based on risk scoring Determine the current risk level of the operator in the main control room and execute corresponding operations based on that risk level. Specifically, when... When the risk level is low, there is no early warning; the remote duty management terminal only records data for long-term trend analysis and reporting. At this time, the risk level is medium risk. In this situation, a soft alert is issued to the operator via a local notification device, such as HMI edge flashing or a slight audible alert, and an encrypted real-time data packet is sent to the duty management terminal. It is recommended that the duty manager conduct remote observation. When the risk level is medium, a mandatory early warning is issued to the operator via a local alert device, such as an audible and visual alarm or flashing of key mission parameters. An encrypted real-time data packet is also immediately sent to prompt the operator to implement intervention decision-making processes such as shift change or rest.

[0057] In this step, the encrypted real-time data packet includes a risk score, task context, and associated feature summary information. The feature summary information includes blink frequency and eye-closing ratio (PERCLOS value), heart rate variability (HRV) index, seat pressure change amplitude, and operator pressure change frequency data, accompanied by timestamps, operator identification, and a summary of the task context, to facilitate subsequent auditing.

[0058] In this step, the fatigue level index of consecutive shifts is cumulatively calculated to generate a fatigue accumulation curve, and management recommendations are generated when the long-term risk threshold is exceeded.

[0059] Example 2 In a typical embodiment of the present invention, such as Figure 2 As shown, this embodiment discloses a system for recognizing the emotional state of operators in the main control room of a nuclear power plant, including: The data acquisition and preprocessing module is configured to: acquire multimodal raw signals from the main control room operator and perform data preprocessing and anti-interference processing; The task acquisition module is configured to: acquire the task context information of the current operator; The recognition module is configured to: calculate the operator's fatigue level index and emotional state level locally on the control panel based on the preprocessed multimodal signal features and task context information using an emotion state recognition model; wherein, the emotion state recognition model includes a fatigue discrimination model and an emotion state level recognition model; and dynamically adjust the weights of each modality signal in the fatigue discrimination model based on task context information. The early warning module is configured to: generate a graded risk score based on fatigue level indicators and emotional state level; determine the current risk level of the operator in the main control room based on the graded risk score; execute corresponding operations according to the risk level; and simultaneously encrypt and send the risk score and associated feature summary information to the duty management terminal.

[0060] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for recognizing the emotional state of operators in the main control room of a nuclear power plant, characterized in that, include: Collect multimodal raw signals from operators in the main control room and perform data preprocessing and anti-interference processing; Obtain the current operator's task context information; Based on the preprocessed multimodal signal features and task context information, the operator's fatigue level and emotional state level are calculated locally at the control console using an emotion state recognition model. The emotion state recognition model includes a fatigue discrimination model and an emotion state level recognition model. The weights of each modality signal in the fatigue discrimination model are dynamically adjusted based on the task context information. A graded risk score is generated based on fatigue level indicators and emotional state levels. The risk score is used to determine the current risk level of the operator in the main control room, and corresponding operations are performed according to the risk level. At the same time, the risk score and associated feature summary information are encrypted and sent to the duty management terminal.

2. The method for recognizing the emotional state of operators in the main control room of a nuclear power plant as described in claim 1, characterized in that, The multimodal raw signals include the operator's facial image, near-infrared image, heart rate signal, skin conductance signal, seat pressure distribution data, and data on changes in the pressure applied to the operating buttons or mouse.

3. The method for recognizing the emotional state of operators in the main control room of a nuclear power plant as described in claim 2, characterized in that, Data preprocessing and anti-interference processing specifically include: Multi-scale illumination compensation algorithms are applied to facial and near-infrared images to address the complex lighting conditions in the main control room. Bandpass filtering and frequency domain noise suppression are performed on the heart rate signal; A short-time baseline adaptive calibration algorithm is performed on skin conductance signals, seat pressure distribution data, and data on changes in the pressure of operating buttons or mice to eliminate the influence of individual physiological differences among operators. The raw signals of the multimodal status of the main control room operator are collected and protected against electromagnetic interference through hardware shielding and grounding structures.

4. The method for recognizing the emotional state of operators in the main control room of a nuclear power plant as described in claim 1, characterized in that, The task context information includes task type, operating condition parameters, and alarm level information; among which, task types include shift handover and administrative tasks, steady-state monitoring, planned operations, abnormal operating condition response, and design basis accident response.

5. The method for recognizing the emotional state of operators in the main control room of a nuclear power plant as described in claim 1, characterized in that, The fatigue discrimination model is specifically represented as follows: In the formula, Values ​​representing the degree of fatigue; Indicates physiological characteristics; Indicates behavioral characteristics; Indicates operational characteristics; , and These are the weights corresponding to physiological characteristics, behavioral characteristics, and operational characteristics, respectively. Indicates task context modification items; The emotional state level recognition model adopts a two-dimensional "valence-arousal" model. It takes the preprocessed multimodal signal as input, calculates valence and arousal respectively, and uses the valence value and arousal value as the x-coordinate and y-coordinate of the emotional state point, respectively. It determines the quadrant in which the emotional state point falls, and the number of quadrants it falls into is used as the value of the emotional state level. The model outputs an emotional state label and an emotional state risk level.

6. The method for recognizing the emotional state of operators in the main control room of a nuclear power plant as described in claim 4, characterized in that, The weights of each modal signal in the fatigue discrimination model are dynamically adjusted based on task context information. Specifically, when the task type is abnormal working condition response or design baseline accident response, the weights of physiological and operational characteristics are automatically increased to quickly identify high-load conditions. When the task type is steady-state monitoring, planned operation, shift handover and administrative tasks, the weights of seat pressure distribution and posture characteristics are increased to focus on identifying persistent fatigue.

7. The method for recognizing the emotional state of operators in the main control room of a nuclear power plant as described in claim 1, characterized in that, Fatigue level indicators include low fatigue, moderate fatigue, and high fatigue; emotional state labels include low-risk calm / focused labels, low-to-medium risk excited / positive labels, medium-to-high risk fatigue / depressed labels, and high-risk anxiety / tension labels.

8. The method for recognizing the emotional state of operators in the main control room of a nuclear power plant as described in claim 1, characterized in that, A graded risk score is generated based on fatigue level indicators and emotional state levels, specifically: A graded risk score is generated based on fatigue level indicators and emotional state risk levels, and is represented as follows: In the formula, This indicates the final risk score. Values ​​representing the degree of fatigue; This indicates the risk level of the emotional state; Indicates the severity of the task; Indicates fatigue weight; Indicates sentiment weight; This represents the environmental weight.

9. The method for recognizing the emotional state of operators in the main control room of a nuclear power plant as described in claim 1, characterized in that, The risk level of the operator in the main control room is determined based on a tiered risk score, and corresponding operations are performed according to the risk level, specifically including: When the risk level is low, there is no early warning. The remote duty management terminal only records data for long-term trend analysis and reporting. When the risk level is medium, a soft warning is issued to the operator through the local warning device, and an encrypted real-time data packet is sent to the duty management terminal, suggesting that the duty manager conduct remote observation. When the risk level is medium, a mandatory early warning is issued to the operator through the local warning device, and an encrypted real-time data packet is immediately sent, prompting the operator to implement intervention decision-making processes such as shift change or rest. The encrypted real-time data packet includes a risk score, task context, and associated feature summary information; the feature summary information includes blink frequency and eye closure ratio, heart rate variability index, seat pressure change amplitude, and operation pressure change frequency data, accompanied by timestamps, operator identification, and task context summary.

10. A system for recognizing the emotional state of operators in the main control room of a nuclear power plant, characterized in that: include: The data acquisition and preprocessing module is configured to: acquire multimodal raw signals from the main control room operator and perform data preprocessing and anti-interference processing; The task acquisition module is configured to: acquire the task context information of the current operator; The recognition module is configured to: calculate the operator's fatigue level index and emotional state level locally on the control panel based on the preprocessed multimodal signal features and task context information using an emotion state recognition model; wherein, the emotion state recognition model includes a fatigue discrimination model and an emotion state level recognition model; and dynamically adjust the weights of each modality signal in the fatigue discrimination model based on task context information. The early warning module is configured to: generate a graded risk score based on fatigue level indicators and emotional state level; determine the current risk level of the operator in the main control room based on the graded risk score; execute corresponding operations according to the risk level; and simultaneously encrypt and send the risk score and associated feature summary information to the duty management terminal.

Images