Virtual reality-based nuclear power plant human factor engineering verification and training method and system

Abstract

The invention belongs to the technical field of human factor engineering verification, and discloses a nuclear power plant human factor engineering verification and training method and system based on virtual reality, and the method comprises the steps: constructing a virtual reality intelligent interaction scene; synchronously collecting multi-dimensional data of the student in the virtual reality intelligent interaction scene, wherein the multi-dimensional data comprises electroencephalogram data, physiological data, operation data and eye movement data; performing data preprocessing and feature extraction on the multi-dimensional data to obtain a plurality of key features, and performing fusion analysis on the plurality of key features; and performing multi-dimensional evaluation and personalized feedback based on the fusion analysis result, generating a multi-dimensional evaluation report, and performing personalized training of the trainee according to the personalized feedback. According to the invention, by constructing a high-simulation virtual nuclear environment and synchronously collecting multi-modal data of an operator, scientific verification of a human-computer interface, environment design and operation regulations is realized, and personalized training of students is realized.

Description

Technical Field

[0001] This invention relates to the field of human factors engineering verification technology, and in particular to a method and system for human factors engineering verification and training in nuclear power plants based on virtual reality. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In the field of nuclear power plant operation and maintenance, human reliability is crucial to system safety. From 1991 to 2022, my country's nuclear power plants experienced 1031 operational events, of which 501 (48.6%) were caused by human error, including operational errors, decision-making delays, and misinterpretations of information. Therefore, the design verification of human-machine interfaces (HMIs) in nuclear power plants, the optimization of operating procedures, and personnel training have always been key areas of focus for the industry.

[0004] Currently, human factors engineering verification and training in nuclear power plants mainly rely on the following methods: 1. Physical Simulation System. A physical simulation chamber identical to the actual control room is built, enabling operational training through physical control consoles, instruments, and simulated signals. For example, a nuclear power plant uses a 1:1 replica of its control room simulation chamber for operator qualification certification training, but its construction cost is as high as tens of millions of yuan, and it cannot simulate the physical environment (such as high temperature and radiation alarms) of extreme accident scenarios (such as loss of coolant accidents, i.e., LOCA).

[0005] 2. Traditional Virtual Reality (VR) Training Systems. These systems utilize VR technology to construct virtual control rooms, enabling interactive operation through peripherals such as controllers. While these systems reduce costs, they have significant limitations: they can only collect operational behavior data (such as click location and reaction time), failing to quantify the operator's cognitive state (such as inattention or cognitive overload). This results in training evaluations remaining focused on "whether the operation was correct," rather than delving into "why the error occurred."

[0006] 3. Human Factors Engineering Validation Methods. Existing validation methods rely on subjective expert evaluation (such as the SHERPA method) or small-scale human trials, collecting feedback through questionnaires and post-event interviews. This approach suffers from problems such as small sample size, data lag, inability to reproduce stress scenarios, and difficulty in accurately identifying design flaws such as console layout and information display methods.

[0007] These existing technologies have the following drawbacks: 1. Insufficient environmental realism. Physical simulation cabins cannot simulate the dynamic changes in accident scenarios (such as sound and light alarms, sudden changes in parameters), and traditional VR lacks multi-sensory stimulation (such as vibration, temperature changes), resulting in a decrease in the operator's adaptability in real accidents.

[0008] 2. Limited data collection. Focusing only on operational results while neglecting cognitive processes (such as EEG signals), physiological responses (such as heart rate), and visual attention allocation makes it difficult to establish a correlation model between "environment-cognition-behavior".

[0009] 3. Poor objectivity in evaluation. Reliance on subjective feedback and lack of quantitative indicators (such as cognitive load threshold and fixation time in key areas) result in a lack of data support for human-computer interface design optimization.

[0010] 4. Insufficient training relevance. The training fails to provide personalized guidance based on the operator's physiological and cognitive characteristics; for example, it does not provide simplified process training for trainees who are prone to cognitive overload. Summary of the Invention

[0011] To address the aforementioned issues, this invention proposes a method and system for verifying and training human factors engineering in nuclear power plants based on virtual reality. By constructing a highly realistic virtual nuclear environment and simultaneously collecting multimodal data from operators, it enables scientific verification of human-machine interfaces, environmental design, and operating procedures, as well as personalized training for trainees.

[0012] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for verification and training of human factors engineering in nuclear power plants based on virtual reality, comprising the following steps: Constructing virtual reality intelligent interactive scenarios; Simultaneously collect multi-dimensional data of trainees in virtual reality intelligent interaction scenarios, including EEG data, physiological data, operational data, and eye-tracking data; Data preprocessing and feature extraction are performed on multi-dimensional data to obtain multiple key features, and these key features are then fused and analyzed. Based on the results of the fusion analysis, multi-dimensional assessments and personalized feedback are conducted to generate multi-dimensional assessment reports, and personalized training is provided to trainees based on the personalized feedback.

[0013] As an alternative implementation, a global positioning system and a local clock are used to synchronize timestamps, ensuring that all data acquisition devices are aligned to the system's master clock and that data acquisition is synchronized.

[0014] As an alternative implementation method, several key features include cognitive load index, attention concentration, stress index, and performance characteristics.

[0015] As an alternative implementation method, a fusion analysis of multiple key features is performed, specifically: A deep learning model was used to establish a "cognition-physiology-operation" correlation model, and the correlation model was used to perform fusion analysis on multiple key features.

[0016] As an alternative implementation method, multi-dimensional evaluation includes: Human-computer interface evaluation, which is the evaluation of the console layout, calculates the attention of each area through eye-tracking data. If a tester fails to focus on a certain area beyond a set threshold, then that area is determined to be an easily neglected area and needs to be optimized. Information display evaluation involves analyzing the visual scanning path. If the average scanning time of the visual scanning path exceeds a set threshold, it indicates poor information relevance, and the way the information is presented needs to be adjusted. Environmental parameter assessment involves comparing operational error rates and cognitive load under different environmental parameters to determine the optimal environmental parameters. The clarity of operating procedures is assessed by counting the number of times a student hesitates about each step of the procedure. When the reaction time exceeds a set threshold, it is considered a hesitation. If the hesitation rate exceeds a set percentage, the procedure is deemed to be vague and needs to be revised. The stress robustness assessment is conducted under high-stress scenarios. If the trainee's process compliance drops below a set threshold, it is marked as a process that needs optimization.

[0017] As an alternative implementation method, personalized feedback includes a "cognition-operation" matching degree. Based on the matching degree of the feedback, training programs are recommended for the trainees' weak points, and personalized training is conducted.

[0018] Secondly, the present invention provides a virtual reality-based nuclear power plant human factors engineering verification and training system, comprising: The virtual environment construction module is configured to: construct virtual reality intelligent interactive scenes; The multimodal data acquisition module is configured to simultaneously acquire multi-dimensional data of trainees in a virtual reality intelligent interaction scenario, including EEG data, physiological data, operational data, and eye-tracking data. The data processing and analysis module is configured to: perform data preprocessing and feature extraction on multi-dimensional data to obtain multiple key features, and perform fusion analysis on multiple key features; The assessment and feedback module is configured to: conduct multi-dimensional assessments and personalized feedback based on the fusion analysis results, generate multi-dimensional assessment reports, and provide personalized training for trainees based on the personalized feedback.

[0019] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0020] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0021] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention presents a groundbreaking method for human factors engineering verification and training in nuclear power plants based on virtual reality. It innovatively integrates virtual reality technology, brain-computer interface technology, and multimodal data acquisition technology, applying them to the field of human factors engineering verification and high-fidelity training in nuclear power plants. By constructing a highly realistic virtual nuclear environment using VR technology, the training and verification scenarios closely approximate the actual operation of a nuclear power plant. Brain-computer interface technology collects electroencephalogram (EEG) data to gain insights into the operator's deep cognitive state. Multimodal data acquisition technology simultaneously acquires multi-dimensional information such as eye movements, physiological data, and operational data, achieving comprehensive and accurate capture of personnel status. This provides a novel approach and solution to many challenges existing in traditional nuclear power plant human factors engineering verification and training.

[0023] This invention presents a virtual reality-based method for human factors engineering verification and training in nuclear power plants, developing a series of feature extraction algorithms for EEG, eye-tracking, physiological, and operational data. These multi-dimensional feature extraction algorithms can extract key information from complex data, providing strong support for subsequent evaluation and analysis. A deep learning model is used to establish a "cognition-physiology-operation" correlation model, discovering the intrinsic connections between different types of data through learning and analysis of large amounts of data. The establishment and application of this correlation model breaks through the limitations of traditional data analysis that only focuses on a single dimension, providing a new perspective and method for a comprehensive and in-depth understanding of operator behavior and status.

[0024] This invention presents a virtual reality-based human factors engineering verification and training method for nuclear power plants. It designs a complete human-machine interface (HMI) evaluation process, starting with multimodal data acquisition. Through eye-tracking data analysis, it controls the attention levels of different areas of the control panel / station, evaluates information display methods based on visual scanning paths, and compares operational error rates and cognitive load under different environmental parameters to determine the optimal environmental parameters. This multi-dimensional data-driven HMI evaluation process provides a scientific and objective basis for optimizing HMIs in nuclear power plants. A verification method for nuclear power plant operating procedures is also established. This method assesses clarity by statistically analyzing the number of times operators hesitate regarding procedure steps, evaluates stress robustness by testing procedure compliance under high-pressure scenarios, and identifies human error points by analyzing the correlation between operational errors and cognitive load. This innovative operating procedure verification method can effectively identify problems in existing operating procedures, providing a reliable reference for the optimization and improvement of procedures.

[0025] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0026] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0027] Figure 1 This is a framework diagram of the virtual reality-based nuclear power plant human factors engineering verification and training system of the present invention; Figure 2 This is a schematic diagram of the data acquisition device of the present invention; Figure 3 This is a flowchart of the data analysis process of the present invention; Figure 4 This is a flowchart illustrating the evaluation and feedback process of the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0029] It should be noted that the following detailed description is exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0030] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but includes other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0032] Example 1 This embodiment provides a method for verification and training of human factors engineering in nuclear power plants based on virtual reality, including the following steps: Constructing virtual reality intelligent interactive scenarios; Simultaneously collect multi-dimensional data of trainees in virtual reality intelligent interaction scenarios, including EEG data, physiological data, operational data, and eye-tracking data; Data preprocessing and feature extraction are performed on multi-dimensional data to obtain multiple key features, and these key features are then fused and analyzed. Based on the results of the fusion analysis, multi-dimensional assessments and personalized feedback are conducted to generate multi-dimensional assessment reports, and personalized training is provided to trainees based on the personalized feedback.

[0033] The specific solution of the present invention is as follows: The connection relationships of the modules in this invention are as follows: Figure 1 As shown. The virtual environment construction module is the foundation of the entire system's scene, responsible for generating highly realistic core scenes. Its hardware includes a graphics workstation and a VR rendering server, while the software components consist of a 3D modeling engine, a physics engine, and a scene database. The multimodal data acquisition module plays a crucial role in data collection, consisting of sensing devices and a synchronization controller, specifically designed for collecting EEG, eye-tracking, physiological, and operational data. The data processing and analysis module is the core of data analysis, relying on edge computing units and cloud servers to complete a series of operations such as data preprocessing, feature extraction, and quantitative analysis. The evaluation and feedback module includes human-computer interface and environment evaluation, procedure verification, and training feedback, capable of generating visual evaluation reports to provide a basis for subsequent optimization and training. The user interaction module consists of a VR headset, data gloves, and force feedback devices, supporting immersive interaction between users and the virtual environment.

[0034] The modules are connected via high-speed Ethernet (1Gbps) and Time-Sensitive Network (TSN). This connection method ensures that the data transmission latency can be controlled within 10ms and the synchronization accuracy can be within 1ms, providing a solid network foundation for the efficient and stable operation of the system.

[0035] Virtual environment building module: This module constructs a highly realistic nuclear power plant environment through a series of precise steps. In the 3D modeling phase, laser scanning technology is used to scan the main control room, auxiliary control room, and reactor hall, among other areas, to obtain accurate dimensional data with an error controlled within 5mm. Subsequently, Blender and 3ds Max software are used to construct a 1:1 3D model containing over 1200 equipment models, including monitors, control handles, and indicator lights. The material library includes over 20 categories of physical property parameters for materials such as metal, plastic, and glass. The reflectivity of metal is set to 0.8, plastic to 0.3, and glass to 0.9. Substance Painter software is used to process these materials, achieving a high degree of detail reproduction; for example, the text labels on equipment buttons are clearly visible.

[0036] In terms of physics engine integration, Unreal Engine 5's Chaos physics engine is used to simulate the physical feedback during device operation. The damping of the knob rotation is set to 5 N·m, the button pressing stroke to 2 mm, and the switch switching force to 3 N. These parameters are designed to closely resemble the feel of real device operation. To study the impact of these parameters on user operation, they can be customized. Simultaneously, the engine can also achieve dynamic scene interaction. For example, in the LOCA accident scenario, it can simulate the visual, auditory, and temperature sensations of water jets when a pipe ruptures, mimicking a temperature of 80°C during steam diffusion, accompanied by a 110dB audible and visual alarm and a 2Hz red light flashing effect, comprehensively recreating the realistic scene of an accident.

[0037] The scenario library includes three basic scenarios: normal operation scenarios (e.g., power regulation), abnormal scenarios (e.g., abnormal coolant temperature), and accident scenarios (e.g., LOCA, main pump failure). It also supports dynamic adjustment of scenario parameters; control room illuminance can be continuously adjusted between 50-1000 lux, background noise ranges from 40-90 dB, and equipment failure probability can be set from 0-100% to meet different verification and training needs.

[0038] Multimodal data acquisition module: This module achieves multi-dimensional data synchronous acquisition through integrated equipment (see attached). Figure 2(Schematic diagram of the data acquisition equipment). The equipment is rich and sophisticated. The VR headset has a resolution of 2880×1600 and a refresh rate of 120Hz. It also integrates an eye-tracking module with a sampling rate of 120Hz and an accuracy of 0.5°. The EEG sensor is a multi-channel dry electrode (at least 8 channels) with a sampling rate set to 250Hz. The electrodes are placed in the prefrontal cortex, central sulcus, parietal lobe, etc., to monitor different bands of brain waves. Among them, alpha waves (8-13Hz) represent a relaxed state, beta waves (13-30Hz) reflect cognitive activity, and theta waves (4-8Hz) are related to fatigue.

[0039] The physiological sensors include a heart rate chest strap and a skin conductance sensor. The heart rate chest strap has a sampling rate of 1Hz and a measurement range of 40-200 bpm; the skin conductance sensor has a sampling rate of 10Hz and an impedance range of 100kΩ-10MΩ. The motion capture equipment consists of a data glove and motion capture base stations. The data glove is equipped with 16 sensors with a sampling rate of 100Hz; there are 6 motion capture base stations, and the positioning accuracy can reach within 1mm.

[0040] The synchronous acquisition mechanism is rigorous and reliable, employing a global positioning system and a local clock for timestamp synchronization to ensure all devices are aligned with the system's master clock, with an error controlled within 0.5ms. Data is transmitted in frames, each containing EEG data, eye-tracking data, physiological data, and operational data, and is uploaded to the data processing module in real time via UDP protocol, guaranteeing data timeliness and integrity.

[0041] The data collection was comprehensive and targeted. EEG data reflected cognitive load and stress response, with cognitive load assessed based on the β / α wave ratio and stress response measured by theta wave power. Eye-tracking data included fixation point coordinates, saccade amplitude, and fixation on key areas; effective attention was defined as fixation duration of 1 second or more on key areas (such as alarm lights). Physiological data covered heart rate variability and skin conductance levels. Operational data included operation sequences (e.g., "press button A → rotate knob B"), reaction time (the interval from the event to the first operation), and error records, categorized as missed operations, incorrect sequence, and misoperation.

[0042] Data processing and analysis module: This module performs multi-dimensional analysis on the collected data, and the process is as follows: Figure 3 As shown. In the data preprocessing stage, noise reduction is performed first. EEG data are processed using independent component analysis to remove electrooculography artifacts, while physiological data are processed using Kalman filtering to smooth noise and improve data quality. Subsequently, standardization is performed to normalize data such as heart rate and skin conductance to the [0,1] interval, where 0 represents the baseline and 1 represents the maximum value, which facilitates subsequent analysis and comparison.

[0043] During the feature extraction phase, several key features are extracted. Cognitive features include cognitive load index (cognitive load index = β-wave energy / α-wave energy) and attention concentration (attention concentration = fixation time in key areas / total task duration); physiological features are mainly stress index (stress index = 0.6 × standardized skin conductance value + 0.4 × standardized heart rate value); operational features include error rate (error rate = number of errors / total number of operations) and process compliance (process compliance = number of operations that comply with the procedure / total number of operations).

[0044] In the fusion analysis phase, a deep learning model is used to establish a "cognition-physiology-operation" correlation model. For example, when the cognitive load index is greater than 1.8 and the stress index is greater than 0.7, the probability of an increase in the operational error rate will exceed 85%. At the same time, a time series heatmap is generated to intuitively show the distribution of attention at different task stages. For example, in a LOCA accident, it can be clearly seen whether the operator prioritizes attention to the pressure gauge.

[0045] The above evaluation indicators will be adjusted adaptively according to individual differences.

[0046] Assessment and Feedback Module: This module is as follows Figure 4 As shown, multi-dimensional evaluation and personalized feedback are achieved based on the analysis results. In terms of human-computer interface evaluation, for console layout evaluation, eye-tracking data is used to calculate the attention level of each area. If 30% of testers do not look at a certain button, then that area is determined to be easily overlooked and needs optimization. Information display evaluation analyzes the visual scanning path, such as the average scan time from the alarm light to the parameter table. If it exceeds 2 seconds, it indicates poor information correlation, and the information presentation method needs adjustment. Environmental parameter evaluation compares the operational error rate and cognitive load under different illuminance levels to determine the optimal illuminance range.

[0047] In the operational procedure verification phase, clarity is assessed by counting the number of times operators hesitate regarding the procedure steps. A reaction time exceeding 5 seconds is considered hesitation, and if the hesitation rate exceeds 20%, the procedure is deemed ambiguous and requires modification. Stress robustness assessment is conducted under high-stress scenarios (stress index greater than 0.8). If process compliance decreases by more than 30%, it is marked as requiring optimization. Human error identification identifies potential risks by analyzing the correlation between operational errors and cognitive load. For example, in step-intensive phases, high load can easily lead to missed operations; these situations are highlighted.

[0048] The training feedback process generates detailed individual reports, including "cognitive-operational" matching scores (e.g., "high error rate under high load, suggesting improved attention management under pressure") and team collaboration scores (based on the synchronization of multiple learners' operations). Simultaneously, training programs are recommended to address learners' weaknesses; for example, learners with insufficient attention spans will receive specialized visual search training to improve their related abilities.

[0049] System workflow: Taking the LOCA accident emergency training for nuclear power plant operators as an example, the system's operation process proceeds in an orderly manner according to the following steps: First, the administrator accesses the system scenario configuration interface through the user interaction module, selects the "LOCA accident scenario" from the scenario library, and then adjusts the virtual environment parameters according to training needs, including the light intensity and background noise level in the control room, completing the scenario initialization settings. Before entering the training, trainees must sequentially put on the VR headset, EEG sensor, physiological sensor, and operation capture device. After the equipment is complete, trainees enter the virtual reactor hall scenario through the VR headset. At this time, the system automatically starts the equipment calibration program to debug the signal transmission and time synchronization functions of all acquisition devices to ensure the accuracy of data acquisition; at the same time, the system will collect the trainees' physiological baseline data in a resting state, including basic physiological indicators such as heart rate and skin conductance, providing a reference standard for the analysis of physiological state changes during subsequent training.

[0050] Next, the system triggers a LOCA (Local Occurrence-Oriented Compulsory Assessment) accident scenario according to preset logic, presenting a complete accident characterization in the virtual environment: reactor-related pipelines rupture, and the supporting pressure monitoring system provides real-time feedback on the sudden pressure drop and triggers an alarm mechanism. Trainees must perform emergency response operations in the virtual scenario according to the nuclear power plant LOCA accident emergency operating procedures, including closing the corresponding pipeline isolation valves to stop the medium leakage and starting the standby pump to maintain system circulation. Throughout the trainees' operations, the multimodal data acquisition module remains continuously active, synchronously recording various key data: EEG sensors focus on monitoring changes in brainwaves in the central sulcus region of the trainees' brains; when beta waves show a significant increasing trend, it reflects an increase in the trainee's cognitive load; the eye-tracking module captures the trainee's visual focus movement trajectory in real time, recording the time when their attention is first focused on the pressure gauges to determine the timeliness of the trainee's attention to key equipment parameters; physiological sensors dynamically collect the trainee's heart rate and skin conductance data, tracking the trainee's physiological stress response during the accident response process; and the operation capture device records in detail the trainee's every operational action sequence, especially whether key operational steps were completed according to procedures, such as whether there were any omissions in the pressure confirmation step after the standby pump was started.

[0051] After the trainees complete the emergency operation procedures, all the collected multimodal data is transmitted to the data processing module. This module first preprocesses the raw data, removes interference noise, and completes data standardization transformation. Then, it uses feature extraction algorithms to extract key feature indicators from EEG, eye movement, physiological, and operational data, including core assessment parameters such as cognitive load-related index, stress response index, and compliance with operation procedures, ultimately forming a complete data analysis result.

[0052] Finally, the evaluation and feedback module generates a detailed, multi-dimensional evaluation report based on the analysis results output by the data processing module. At the human-machine interface design evaluation level, by combining the attention distribution of each operating area in the eye-tracking data, it identifies areas with weak attention (such as the isolation valve operating area) and proposes targeted optimization suggestions. At the operating procedure verification level, based on the frequency of hesitation and operational coherence during the trainee's operation, it analyzes steps in the procedure that are complex in expression or unclear in logic, and provides suggestions for simplification and adjustment. At the individual training feedback level, by combining the correlation between changes in the trainee's cognitive load, physiological stress response, and operational performance, it identifies the trainee's shortcomings in high-pressure scenarios (such as attention allocation problems) and recommends matching specialized training programs. Through this complete process, it not only achieves systematic training in the emergency response capabilities for nuclear power plant operators in LOCA accidents, but also provides a scientific verification basis for the optimization of nuclear power plant human-machine interface design and the improvement of operating procedures.

[0053] The alternative to the present invention is as follows: 1. Diversified data acquisition module: Solution: Use near-infrared imaging acquisition equipment to collect EEG data.

[0054] Feasibility: Using near-infrared imaging acquisition equipment to collect more comprehensive EEG data will facilitate the collection of more quantitative data on cognitive, stress, and emotional levels, and better establish a personalized database.

[0055] 2. Construction of Mixed Reality (AR) Environments: Solution: Use AR glasses to replace VR headsets, and overlay virtual devices onto the physical control console.

[0056] Feasibility: It retains the realistic operating feel (such as physical buttons) while displaying virtual parameters (such as pipeline pressure) through AR, making it suitable for training scenarios that require the integration of physical equipment (such as valve mechanical operation).

[0057] 3. Introduce deep learning prediction models: Solution: Add an LSTM neural network to the data processing module to predict the probability of operator errors in specific scenarios based on historical data.

[0058] Feasibility: By predicting high-risk operations (such as "Trainee A has a 70% probability of missing a step in a LOCA scenario"), early intervention (such as highlighting prompts in a virtual environment) can enhance the foresight of training.

[0059] 4. Expand the team collaboration assessment scheme: Variations: Added a multi-user synchronous interaction module (supporting collaborative operation by 4-6 people) and a team behavior analysis algorithm. Feasibility: Collect data on team members' operational synchronization (such as command response delay) and information exchange frequency to evaluate team collaboration efficiency. This is applicable to control room team training and expands the system's application scenarios.

[0060] Example 2 This embodiment provides a virtual reality-based human factors engineering verification and training system for nuclear power plants, including: The virtual environment construction module is configured to: construct virtual reality intelligent interactive scenes; The multimodal data acquisition module is configured to simultaneously acquire multi-dimensional data of trainees in a virtual reality intelligent interaction scenario, including EEG data, physiological data, operational data, and eye-tracking data. The data processing and analysis module is configured to: perform data preprocessing and feature extraction on multi-dimensional data to obtain multiple key features, and perform fusion analysis on multiple key features; The assessment and feedback module is configured to: conduct multi-dimensional assessments and personalized feedback based on the fusion analysis results, generate multi-dimensional assessment reports, and provide personalized training for trainees based on the personalized feedback.

[0061] It should be noted that the above modules correspond to the steps in Embodiment 1, and the examples and application scenarios implemented by the above modules and their corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules can be executed in a computer system as part of the system.

[0062] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0063] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0064] A computer-readable storage medium for storing computer instructions that, when executed by a processor, perform the method of Embodiment 1.

[0065] The method in Example 1 can be directly executed by a hardware processor, or it can be executed by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0066] A computer program product includes a computer program that, when executed by a processor, implements the method in Embodiment 1.

[0067] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0068] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0069] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0070] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0071] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A virtual reality based nuclear power plant human factors validation and training method, characterized in that, The method comprises the following steps: constructing a virtual reality intelligent interaction scene; synchronously collecting multi-dimensional data of the trainee in the virtual reality intelligent interaction scene, the multi-dimensional data comprising electroencephalogram data, physiological data, operation data and eye movement data; respectively performing data preprocessing and feature extraction on the multi-dimensional data to obtain multiple key features, and performing fusion analysis on the multiple key features; performing multi-dimensional evaluation and personalized feedback based on the fusion analysis result, generating a multi-dimensional evaluation report, and performing personalized training of the trainee according to the personalized feedback.

2. The virtual reality based nuclear power plant human factors validation and training method of claim 1, wherein, The global positioning system and the local clock are used to realize timestamp synchronization, so that all data collection devices are unified to the system master clock, and the synchronization of data collection is ensured.

3. The virtual reality based nuclear power plant human factors validation and training method of claim 1, wherein, The multiple key features include a cognitive load index, an attention concentration degree, a stress index and an operation feature.

4. The virtual reality based nuclear power plant human factors validation and training method of claim 1, wherein, The fusion analysis on the multiple key features comprises: a deep learning model is used to establish a "cognition-physiology-operation" correlation model, and the correlation model is used to perform fusion analysis on the multiple key features.

5. The virtual reality based nuclear power plant human factors validation and training method of claim 1, wherein, The multi-dimensional evaluation comprises: human-computer interface evaluation, that is, console layout evaluation, the attention degree of each region is calculated through eye movement data, if a certain region has more than a set threshold of testers who do not gaze, then it is determined that the region is an easy-to-overlook region and needs to be optimized; information display evaluation, the visual scanning path is analyzed, if the average saccade time of the visual scanning path exceeds a set threshold, it is indicated that the information correlation is poor, and the presentation mode of the information needs to be adjusted; environment parameter evaluation, the operation error rate and the cognitive load under different environment parameters are compared to determine the optimal environment parameter; operation procedure clarity evaluation, the hesitation times of the trainee to the procedure steps are counted, when the reaction time exceeds a set threshold, it is regarded as hesitation, if the hesitation rate exceeds a set proportion, it is determined that the procedure expression is ambiguous and needs to be modified; stress robustness evaluation, which is performed in a high-pressure scene, if the process compliance of the trainee decreases by more than a set threshold, it is marked as a link that needs to be optimized.

6. The virtual reality based nuclear power plant human factors validation and training method of claim 1, wherein, The personalized feedback includes a "cognition-operation" matching degree, according to the matching degree of the feedback, a training scheme is recommended for the weak link of the trainee for personalized training.

7. A virtual reality based nuclear power plant human factors validation and training system characterized in that, The method comprises: a virtual environment construction module configured to construct a virtual reality intelligent interaction scene; a multi-modal data collection module configured to synchronously collect multi-dimensional data of the trainee in the virtual reality intelligent interaction scene, the multi-dimensional data comprising electroencephalogram data, physiological data, operation data and eye movement data; a data processing and analysis module configured to respectively perform data preprocessing and feature extraction on the multi-dimensional data to obtain multiple key features, and perform fusion analysis on the multiple key features; an evaluation and feedback module configured to perform multi-dimensional evaluation and personalized feedback based on the fusion analysis result, generate a multi-dimensional evaluation report, and perform personalized training of the trainee according to the personalized feedback.

8. An electronic device, comprising: The computer program product comprises a memory and a processor, and computer instructions stored in the memory and run on the processor, when the computer instructions are run by the processor, the method of any one of claims 1-6 is completed.

9. A computer-readable storage medium, characterized in that, A computer program product for storing computer instructions which, when executed by a processor, perform the method of any one of claims 1-6.

10. A computer program product, characterised in that, A computer program for implementing the method of any one of claims 1-6 when executed by a processor.

Images