Virtual reality closed isolated environment multi-person cooperative training system and method based on inter-brain synchronism measurement

Abstract

The invention discloses a virtual reality closed isolated environment multi-person cooperative training system and method based on inter-brain synchronism measurement. The system comprises a virtual closed isolated environment construction module, a multi-person cooperative task control module, an inter-brain synchronism real-time detection module, a multi-modal data synchronous acquisition module and a neural feedback control module. A military closed and isolated operation environment is constructed through a virtual closed and isolated environment construction module, and a user executes a multi-user cooperation task in the closed and isolated operation environment through a multi-user cooperation task control module. An inter-brain synchronism real-time detection module is used for collecting electroencephalogram signals of the users in the multi-user cooperation task process, and inter-brain synchronism evaluation is carried out according to the electroencephalogram signals to obtain synchronism indexes; performing data synchronization on events, behavior data and electroencephalogram signals in a multi-person cooperation task through a multi-modal data synchronous acquisition module; and adjusting the difficulty of the multi-person cooperation task according to the synchronism index through a neural feedback control module.

Description

Technical Field

[0001] This invention belongs to the field of virtual reality simulation training technology, and particularly relates to a multi-person collaborative training system and method for virtual reality closed and isolated environments based on brain-to-brain synchronization measurement. Background Technology

[0002] With the development of military training technology, the collaborative capabilities of combat teams in closed and isolated environments are receiving increasing attention. In modern closed and isolated environment warfare, efficient collaboration among combat teams is a key factor in completing combat missions. Traditional closed and isolated environment training mainly relies on physical simulators. These simulators can realistically reproduce the operational environment of closed and isolated environments, but they suffer from problems such as high construction costs, fixed scenarios, and difficulty in standardized measurement. A complete closed and isolated environment physical simulator often costs tens of millions of yuan, and once completed, adjusting its training scenarios and difficulty is relatively difficult, failing to meet personalized training needs.

[0003] In recent years, virtual reality technology has been widely used in military training. Existing VR military training systems can provide immersive training experiences, support the simulation of various combat scenarios, and have a cost advantage compared to physical simulators. However, existing VR military training systems mainly focus on individual skill training and lack in-depth analysis of multi-person collaboration mechanisms. Especially in scenarios requiring high teamwork, such as combat in closed and isolated environments, existing systems cannot objectively quantify the synchronicity of collaboration among team members, relying mainly on instructors' subjective observations and evaluations to assess training effectiveness, leading to issues with the consistency and scientific rigor of the evaluation results.

[0004] In the field of electroencephalography (EEG) monitoring technology, existing EEG systems are already well-suited for monitoring and evaluating individual cognitive states. Some studies have applied EEG technology to military training, assessing training effectiveness by monitoring indicators such as trainees' attention levels and cognitive load. Patent application CN202010345678.9 discloses an EEG-based cognitive load assessment system that can monitor a single person's cognitive state in real time, but it is limited to individual measurements and does not involve analysis of brain synchronization among multiple individuals. Patent application CN201910234567.8 discloses a closed, isolated environment simulation training system based on VR technology. This system constructs a virtual, closed, isolated environment, but only supports single-person training and lacks multi-person collaborative functionality.

[0005] International research on related technologies mainly focuses on the development of multi-user VR systems. US Patent 10,456,789 discloses a multi-user virtual reality training system that supports collaborative training among multiple users in a virtual environment, but it does not address EEG signal monitoring or objective assessment of collaborative synchronization. European Patent EP3,456,789 discloses a neurofeedback-based training system that can adjust training content based on the user's EEG signals, but it primarily targets single-person training and lacks consideration for multi-person collaboration.

[0006] The main problems with existing technologies include: First, the lack of an objective multi-person collaborative assessment mechanism. Traditional closed and isolated training environments rely mainly on instructors' subjective observations, which cannot quantify the synchronicity of collaboration among participants, resulting in poor accuracy and consistency of assessment results. Second, high training costs and insufficient flexibility. Physical simulators are extremely expensive to build, and the scenarios are relatively fixed, making it difficult to quickly adjust them according to training needs. Third, the lack of standardized collaborative cognitive measurement paradigms. Existing systems cannot systematically measure the cognitive synchronicity of teams in key collaborative aspects such as attention allocation, risk decision-making, and execution coordination. Finally, the inability to achieve adaptive training based on neurofeedback. Existing systems cannot dynamically adjust training content based on trainees' real-time cognitive states and collaborative performance, resulting in lower training efficiency. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention proposes a virtual reality closed and isolated environment multi-person collaborative training system and method based on brain-to-brain synchronization measurement, thereby resolving the issues existing in the prior art.

[0008] To achieve the above objectives, this invention provides a multi-person collaborative training system and method for a closed, isolated virtual reality environment based on brain-brain synchronization measurement, comprising:

[0009] The module includes a virtual closed and isolated environment construction module, a multi-person collaborative task control module, a real-time brain-to-brain synchronization detection module, a multimodal data synchronous acquisition module, and a neurofeedback control module.

[0010] The system employs a virtual closed and isolated environment construction module to create a military-style closed and isolated operating environment. A multi-user collaborative task control module performs collaborative tasks within this environment. A real-time brain-brain synchronization detection module collects users' electroencephalogram (EEG) signals during the collaborative tasks and assesses brain-brain synchronization based on these signals to obtain synchronization indices. A multimodal data synchronization acquisition module synchronizes event and behavioral data, along with EEG signals, within the collaborative tasks. Finally, a neurofeedback control module adjusts the difficulty of the collaborative tasks based on the synchronization indices.

[0011] Optionally, in the multi-user collaborative task control module, the multi-user collaborative task includes an attention synchronization measurement task, a risk decision synchronization measurement task, and an execution coordination synchronization measurement task. The attention synchronization measurement task is designed based on the CPT method. In the attention synchronization measurement task, different users perform sonar monitoring tasks, four-quadrant information monitoring tasks, and weapon-related dual tasks. The weapon-related dual tasks are performed by one user and include weapon system readiness maintenance and 2-back digital matching. The risk decision synchronization measurement task is set based on the Iowa gambling task. The risk decision synchronization measurement task includes deciding on different decision schemes. Different decision schemes are selected through individual user evaluation, information sharing, and group decision-making. The execution coordination synchronization measurement task is designed based on a multi-user Simon task. In the execution coordination synchronization measurement task, different users coordinate responses under different timing requirements.

[0012] Optionally, in the real-time brain synchronicity detection module, the specific process of assessing brain synchronicity based on the EEG signals includes:

[0013] Key EEG data in different frequency bands are extracted from the EEG signal. The key EEG data is then calculated using the phase-locked value method to obtain a synchronization index. Specifically, the key EEG data is processed using the Hilbert transform method to obtain the instantaneous phase. Based on the instantaneous phase, the phase difference of the EEG between different users is calculated to obtain the synchronization index.

[0014] Optionally, in the neurofeedback control module, the process of adjusting the difficulty of multi-person collaborative tasks based on synchronization metrics includes:

[0015] In the real-time brain synchronization detection module, synchronization matrices of different frequency bands are constructed based on the synchronization index;

[0016] In the neurofeedback control module, different cooperative states are obtained by classifying according to the synchronization matrix. The cooperative states include: high synchronization state, medium synchronization state and low synchronization state. The corresponding difficulty adjustment schemes are: increasing task difficulty, maintaining task difficulty and decreasing task difficulty.

[0017] Optionally, in the real-time brain synchronization detection module, the key EEG data includes EEG signals in the Alpha, Beta, and Gamma bands, which respectively correspond to the calculation of synchronization indicators for attention synchronization measurement tasks, risk decision synchronization measurement tasks, and execution coordination synchronization measurement tasks.

[0018] Optionally, in the multimodal data synchronization acquisition module, data synchronization of event and behavioral data and EEG signals in multi-person collaborative tasks includes:

[0019] Collaborative events in multi-person collaborative tasks are tagged, including task phase transitions, individual operational behaviors, and collaborative interaction behaviors. Data synchronization of event and behavioral data and EEG signals in multi-person collaborative tasks is achieved through hardware clock synchronization, software timestamp calibration, and delay compensation methods.

[0020] Optionally, in the neural feedback control module, the synchronization matrix is ​​classified using a real-time collaborative state recognition method, wherein the real-time collaborative state recognition method employs a machine learning approach.

[0021] Optionally, in the real-time brain synchronization detection module, before assessing brain synchronization based on the EEG signal, the module further includes preprocessing the EEG signal, wherein the preprocessing includes bandpass filtering, removal of power line interference, and removal of artifacts from electrooculography and electromyography.

[0022] Optionally, the neurofeedback control module can also be used to calculate individual and team metrics for different tasks, analyze brain-to-brain synchronization, and assess collaborative abilities.

[0023] On the other hand, the present invention also provides a method for multi-person collaborative training in a closed and isolated virtual reality environment based on brain-brain synchronization measurement. Based on the above-mentioned system, multi-person collaborative training in a closed and isolated virtual reality environment is carried out.

[0024] Compared with the prior art, the present invention has the following advantages and technical effects:

[0025] Regarding the objectivity of collaborative assessment, this invention achieves, for the first time, objective neurometrics of three-person collaboration in a closed, isolated environment. Through interbrain synchronization indicators, the quality of collaboration among personnel A, B, and C can be objectively quantified, with an assessment accuracy approximately 45% higher than traditional subjective evaluation methods. This objective assessment method not only avoids the bias of instructors' subjective judgments but also identifies collaboration problems that are difficult to detect using traditional observation methods, providing a reliable basis for the scientific evaluation of training effectiveness.

[0026] In terms of training cost and efficiency, this invention significantly reduces the cost of training in closed, isolated environments and improves training efficiency. The construction cost of the virtual reality training system is only about 5% of that of a traditional physical simulator, and the operating cost is reduced by about 80%. The standardized 18-minute training process is about 60% shorter than traditional training time, greatly improving the cost-effectiveness of training. At the same time, the system supports multiple teams training simultaneously, increasing training capacity by more than 3 times, further improving the utilization efficiency of training resources.

[0027] Regarding training standardization, this invention establishes a standardized measurement system for collaborative cognition in closed and isolated environments. Based on a three-module task design using a psychological experimental paradigm, it provides a scientific and repeatable measurement tool for collaborative training in closed and isolated environments, with the repeatability of evaluation results exceeding 98%. This standardized measurement system not only supports tracking training effectiveness within the same unit but also supports comparative analysis of collaborative capabilities between different units, providing strong support for the standardized management of military training.

[0028] In terms of personalized training, this invention achieves personalized training based on neurofeedback. The system can automatically adjust the training difficulty according to real-time brain-to-brain synchronization, identify weak links in collaboration, and provide targeted training suggestions. Compared with the traditional "one-size-fits-all" training mode, the personalized training method improves the training effect by about 35% and can better meet the training needs of teams with different experience levels.

[0029] In terms of technological innovation, this invention has achieved breakthroughs in several technical fields. Regarding brain-to-brain synchronization analysis technology, it is the first to realize real-time neural synchronization measurement in a three-person collaborative environment; regarding multimodal data synchronization technology, it solves the problem of high-precision time synchronization for multiple people and multiple devices in a VR environment; and regarding adaptive training control technology, it is the first to use brain-to-brain synchronization as a feedback signal for training control. These technological innovations not only promote the development of related technical fields but also provide a reference for collaborative training in other fields. Attached Figure Description

[0030] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0031] Figure 1 This is a system overall architecture diagram according to an embodiment of the present invention;

[0032] Figure 2 This is a flowchart of real-time calculation of interbrain synchronization according to an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of multimodal data synchronization according to an embodiment of the present invention;

[0034] Figure 4 This is a layout diagram of the VR collaborative environment according to an embodiment of the present invention;

[0035] Figure 5 This is a design diagram of the training task module according to an embodiment of the present invention;

[0036] Figure 6 This is a flowchart of the brain-brain synchronization analysis algorithm according to an embodiment of the present invention;

[0037] Figure 7This is an adaptive training control logic diagram according to an embodiment of the present invention;

[0038] Figure 8 This is a data management and feedback system architecture diagram according to an embodiment of the present invention. Detailed Implementation

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0041] To address the problems existing in existing closed and isolated environment collaborative training technologies, the purpose of this invention is to provide a virtual reality closed and isolated environment multi-person collaborative training system and method based on brain-to-brain synchronization measurement, which solves the technical problems of lack of objective collaborative assessment, high training costs, and inability to achieve adaptive training in existing technologies.

[0042] Specifically, the technical problems this invention aims to solve include: how to achieve objective quantitative evaluation of three-person collaboration in a virtual, closed, and isolated environment; how to establish a standardized collaborative cognitive measurement system; how to achieve high-precision synchronous acquisition of multimodal data; and how to achieve adaptive training control based on interbrain synchronization. By solving these technical problems, this invention aims to provide a scientific, efficient, and economical collaborative training solution for combat teams in closed and isolated environments.

[0043] like Figure 1 As shown, the technical solution provided by this invention is a virtual reality closed-environment multi-person collaborative training system based on brain-to-brain synchronization measurement. This system is specifically designed for three-person collaborative scenarios involving personnel A, B, and C in closed-environment combat. The system adopts a modular architecture, mainly including a virtual closed-environment construction module (…). Figure 1 The VR environment rendering module and the three-person collaborative task control module are shown. Figure 1 The multi-user interactive synchronization module and the real-time monitoring module for interbrain synchronization are shown. Figure 1 The EEG data acquisition module and brain-brain synchronization analysis module shown), multimodal data synchronization acquisition module (integrated into other modules), and neurofeedback control module ( Figure 1 The data management and feedback module (shown) consists of five core components.

[0044] The virtual closed-loop isolated environment construction module is responsible for building a virtual closed-loop isolated environment command cabin with dimensions of 12m × 4m × 2.5m in virtual space, setting up three professional workstations: Workstation A, Command Center B, and Control Console C. This module uses a CAVE four-sided projection system to achieve 360-degree immersive display, supporting real-time physical simulation and multi-user interaction. Through high-precision 3D modeling and real-time rendering technology, it provides users with a highly realistic closed-loop isolated environment operation environment.

[0045] like Figure 5 As shown, the three-person collaborative task control module is the core of the system, responsible for managing the standardized 18-minute training process. This module contains three sub-modules: the attention synchronization measurement task module (lasting 6 minutes), the risk decision synchronization measurement task module (lasting 8 minutes), and the execution coordination synchronization measurement task module (lasting 4 minutes), which execute the attention synchronization measurement task, the risk decision synchronization measurement task, and the execution coordination synchronization measurement task sequentially, respectively. Among them, the attention synchronization measurement task is designed based on the CPT (Continuous Performance Test) paradigm. Personnel A performs an 8×8 grid sonar monitoring task, which needs to detect 20% of the red target signals and suppress 80% of the blue interference signals within a 500ms interval; Personnel B performs a four-quadrant information monitoring task, which needs to simultaneously monitor sonar information, radar information, communication information, and system information, and handle abnormal information with a 10% probability of occurrence (including sonar anomalies, radar anomalies, communication anomalies, and system anomalies); Personnel C performs a dual-task paradigm, with the primary task being to maintain the weapon system's readiness state and the secondary task being to perform 2-back digital matching. The risk decision-making synchronous measurement task, adapted from the Iowa gambling mission, offers four closed, isolated environment combat options for the team to choose from: immediate attack, evasive maneuver, tracking and monitoring, and surfacing for distress. Each option has a different probability of success and risk-reward ratio, requiring the team to complete the decision-making process through three stages: individual assessment, information sharing, and group decision-making. The execution coordination synchronous measurement task, based on the multi-person Simon mission design, requires three personnel to coordinate their responses according to strict timing requirements: Personnel A detects and confirms the target (within 500ms), Personnel B determines the attack direction (within 300ms), and Personnel C executes preparatory actions (within 200ms).

[0046] Grid size and VR rendering method:

[0047] Physical specifications:

[0048] Grid size: Each grid cell is 100×100 pixels, and the total display area is 800×800 pixels;

[0049] VR headset display: Occupies a 30°×30° field of view in the virtual environment;

[0050] Display location: Located in the center of the main display of the sonar workstation, 2.5 meters away from the user's virtual eye position.

[0051] Presentation method:

[0052] Grid background: Dark blue ocean background (RGB: 0, 20, 40);

[0053] Grid lines: Semi-transparent white lines, 2 pixels wide;

[0054] Signal point: Circular light spot, 20 pixels in diameter, with a central luminous effect.

[0055] Refresh mechanism: The entire grid is cleared every 500ms, and new signal points are displayed at random locations.

[0056] Red target signal:

[0057] Color: Bright red (RGB: 255, 0, 0);

[0058] Shape: Circular pulse with an outward-spreading halo effect;

[0059] Sound effect: High-frequency "beep" sound, frequency 1500Hz, duration 200ms;

[0060] Probability of occurrence: 20% (there is a 20% chance of it occurring within every 500ms period);

[0061] Required action: The user needs to respond quickly by clicking or pressing a key.

[0062] Blue interference signal:

[0063] Color: Dark blue (RGB: 0, 100, 255);

[0064] Shape: Square flashing with blurred edges;

[0065] Sound effect: Low-frequency "humming" sound, frequency 300Hz, lasting 100ms;

[0066] Probability of occurrence: 80%;

[0067] Required action: The user needs to suppress the response and not perform any operation.

[0068] Abnormal information category:

[0069] Sonar anomalies: unknown acoustic signals, Doppler anomalies, echo interference;

[0070] Radar anomalies: electromagnetic interference alarm, target identification error, system calibration deviation;

[0071] Communication errors: signal interruption, channel conflict, encryption error;

[0072] System anomalies: power fluctuations, overheating, equipment fault codes;

[0073] Processing requirements: Users need to identify the anomaly type within 2 seconds and select the appropriate processing solution (ignore / report / process immediately). The system records the processing time and accuracy.

[0074] The real-time brain synchronization monitoring module uses a 64-lead EEG device to simultaneously monitor the EEG signals of three individuals at a sampling rate of 1000Hz. This module can calculate the real-time brain synchronization between individuals A and B, B and C, and A and C, focusing on analyzing attentional coordination in the Alpha band (8-12Hz), decision-making coordination in the Beta band (13-30Hz), and executive coordination in the Gamma band (31-50Hz). Figure 6 As shown, the system uses the Phase Lock Value (PLV) algorithm to calculate brain-brain synchrony, obtains the instantaneous phase through Hilbert transform, calculates the phase difference, and finally obtains the synchrony index.

[0075] The multimodal data synchronization acquisition module establishes a unified hardware clock reference, ensuring microsecond-level time synchronization accuracy for VR environment events, EEG data, and behavioral data. This module employs a specialized event tagging and coding system to automatically tag key collaborative events, including task phase transitions, individual operational behaviors, and collaborative interaction behaviors. Through hardware clock synchronization, software timestamp calibration, and latency compensation algorithms, the module ensures the temporal consistency of multimodal data. Multimodal data time alignment is achieved as follows: Figure 3 As shown, where:

[0076] Timeline: A unified millisecond-level time base;

[0077] VR events: key events such as mission start, target appearance, and user interaction;

[0078] EEG data: continuous blocks of brainwave data, each block representing a fixed time window;

[0079] Behavioral data: timestamps of user actions;

[0080] Synchronization marker (△): Precise time marker for key events

[0081] Data alignment method: All data streams are based on a unified master clock, and microsecond-level alignment is achieved through hardware triggering and software timestamps. Further fine alignment is then achieved through interpolation algorithms.

[0082] The neurofeedback control module intelligently adjusts the training difficulty based on real-time brain synchronicity indicators, achieving adaptive training control. For example... Figure 7As shown, when a high level of synchronization is detected among the three brains, the system automatically increases the task difficulty; when the synchronization level is low, the system reduces the task difficulty and provides targeted training suggestions. This module can also identify weak points in collaboration and generate personalized training reports and improvement plans.

[0083] For the above solution, the key technical details are described in detail:

[0084] 1. In the three-person collaborative task control module, the design of collaborative tasks in a closed and isolated environment based on the psychological paradigm is as follows:

[0085] Attention Synchronization Measurement Task (Based on CPT Continuous Performance Testing Paradigm)

[0086] Personnel A performs an 8×8 grid sonar monitoring task: a signal appears every 500ms, of which 20% are red target signals that need to be responded to, and 80% are blue interference signals that need to be suppressed;

[0087] Personnel B performs a four-quadrant information monitoring task: simultaneously monitoring sonar information (40% attention), radar information (30%), communication information (20%), and system information (10%), updating every 2 seconds, with a 10% probability of encountering abnormal information that needs to be classified;

[0088] The combatant performs a dual-task paradigm: the primary task is to maintain the weapon system in a green ready state (maintenance needs to be clicked every 30 seconds), and the secondary task is to perform 2-back number matching (a number is displayed every 1.5 seconds, and a key needs to be pressed when the current number is the same as the number two steps ago).

[0089] Risk Decision-Making Synchronous Measurement Task (Adapted from the Iowa Gambling Task)

[0090] Design four combat scenarios for closed and isolated environments: Scheme A (Immediate attack: 60% success rate, losses -100 to gains +200), Scheme B (Evasion maneuver: 85% success rate, losses -50 to gains +100), Scheme C (Tracking and monitoring: 70% success rate, losses -75 to gains +150), and Scheme D (Float up and call for help: 95% success rate, losses -10 to gains +50).

[0091] Three-stage decision-making process: Individual independent assessment (2 minutes) → Structured information sharing (3 minutes) → Group discussion and decision-making (3 minutes);

[0092] Real-time monitoring of the dynamic changes in the synchronization of the three individuals' brains during the decision-making process.

[0093] Perform coordinated and synchronized measurement tasks (based on a multi-user Simon task):

[0094] The screen randomly displays targets on the left or right, requiring the three people to coordinate their response according to a strict time sequence.

[0095] Timing requirements: Personnel A detects and confirms target (within 500ms) → Personnel B determines attack direction (within 300ms) → Operator executes corresponding preparatory actions (within 200ms);

[0096] Set up consistency conditions (target location and attack direction are consistent) and inconsistency conditions (target location and attack direction are inconsistent) to measure cooperative synchronization under the Simon conflict effect.

[0097] 2. Real-time synchronous computing technology between three brains:

[0098] Phase Lock Value (PLV) Calculation Method:

[0099] For two EEG signals x1(t) and x2(t):

[0100] 1. Obtain the instantaneous phases φ1(t) and φ2(t) using the Hilbert transform;

[0101] 2. Calculate the phase difference: Δφ(t) = φ1(t) - φ2(t);

[0102] 3. Calculate PLV: PLV = |1 / N*Σ(exp(j*Δφ(t)))|, where PLV represents the phase lock value, N represents the number of sampling points in the calculation window, used to calculate the total number of data points for PLV, Δ represents the instantaneous phase difference between the two EEG signals at time point t, i.e., φ1(t) - φ2(t), j represents the imaginary unit, j² = -1 in complex number operations; exp(): exponential function, used to convert the phase difference into a point on the unit circle in complex form; |·|: represents the modulus (absolute value) of the complex number, used to calculate the length of the complex vector.

[0103] Construction of a three-person synchronization network:

[0104] Construct a 3×3 synchronization matrix S:

[0105] S=

[0106] Where: S_AB = PLV(Personnel A, Personnel B), S_AC = PLV(Personnel A, Combatant), S_BC = PLV(Personnel B, Combatant).

[0107] Different EEG signals are categorized as follows, and task-specific frequency band analysis is performed:

[0108] Attention task: Focus on analyzing the synchronicity of the prefrontal cortex region in the alpha band (8-12Hz);

[0109] Decision-making task: Focus on analyzing the synchronicity of the prefrontal cortex and anterior cingulate cortex in the Beta band (13-30Hz);

[0110] Task: Focus on analyzing the synchronicity of the motor cortex and cerebellum in the Gamma band (31-50Hz);

[0111] 3. Data synchronization and marking technology for combat procedures in closed and isolated environments:

[0112] Event tagging coding system:

[0113] Task phase markers (1-10): 1-Task start, 2-Module transition, 3-Task end, etc.;

[0114] Personnel A Event Marking (11-20): 11-Target signal detection, 12-Threat level assessment, 13-Location information reporting, etc.;

[0115] Personnel B Event Markers (21-30): 21-Information Reception Confirmation, 22-Comprehensive Situation Analysis, 23-Issuance of Decision-Making Instructions, etc.;

[0116] Personnel C event markers (31-40): 31-Weapon system preparation, 32-Target parameter setting, 33-Attack action execution, etc.;

[0117] Collaboration event markers (41-50): 41 - Start of information transmission, 42 - Decision discussion, 43 - Synchronized action, etc.;

[0118] Microsecond-level time synchronization achieved:

[0119] Hardware clock synchronization: The EEG sampling clock is synchronized with the VR system's master clock at the hardware level;

[0120] Software timestamp calibration: Multi-machine time calibration is performed using the Network Time Protocol (NTP);

[0121] Delay compensation algorithm: Real-time estimation of network latency and adaptive compensation; the delay compensation algorithm includes: 1. Network latency measurement: RTT measurement: sending timestamp packets every 100ms to measure round-trip time; latency trend analysis: calculating the moving average of the last 10 measurements; jitter detection: identifying the fluctuation range of network latency; 2. Predictive compensation: building a predictive model based on historical latency data; using the Kalman filter algorithm to predict the latency of the next cycle; adjusting the sending time of event markers in advance; 3. Adaptive clock synchronization: GPS / NTP master clock synchronization, accuracy ±1ms; local clock drift detection and correction for each device; dynamically adjusting the synchronization interval (adjustable from 1 to 10 seconds).

[0122] Event triggering mechanism: Key collaborative events automatically trigger time markers, such as... Figure 4 as shown

[0123] 4. Neural Feedback-based Adaptive Training Control Technology

[0124] Real-time Collaboration State Recognition Algorithm:

[0125] It includes the following steps:

[0126] Step 1: Feature extraction;

[0127] Alpha-band PLV: Calculate the phase-locking value between three pairs of users in the 8 - 12 Hz band; Beta-band PLV: Calculate the synchrony in the 13 - 30 Hz band; Gamma-band PLV: Calculate the synchrony in the 31 - 50 Hz band; Time window: 1-second sliding window, 0.5-second step size.

[0128] Step 2: Feature fusion;

[0129] Construct a feature vector: [PLV_AB_alpha, PLV_AC_alpha, PLV_BC_alpha, PLV_AB_beta, PLV_AC_beta, PLV_BC_beta, PLV_AB_gamma, PLV_AC_gamma, PLV_BC_gamma]; Calculate the network-level metric: Network density = (sum of all PLV values) / 9.

[0130] Step 3: State classification:

[0131] High synchrony state: Network density > 0.7;

[0132] Medium synchrony state: 0.4 ≤ Network density ≤ 0.7;

[0133] Low synchrony state: Network density < 0.4.

[0134] Step 4: State smoothing;

[0135] Adopt a 3-point moving average to avoid frequent state jumps, and at the same time set a hysteresis mechanism for state switching (switching requires meeting the conditions continuously twice).

[0136] Collaboration state = classify(PLV_alpha, PLV_beta, PLV_gamma);

[0137] State classification: High synchrony state: PLV > 0.7, the task difficulty can be appropriately increased; Medium synchrony state: 0.4 < PLV < 0.7, maintain the current difficulty; Low synchrony state: PLV < 0.4, the task difficulty needs to be reduced.

[0138] Dynamic Difficulty Adjustment Mechanism:

[0139] Sonar monitoring task: Adjust the frequency of target signal occurrence and noise level;

[0140] Adjust parameters:

[0141] Target signal frequency:

[0142] Difficulty reduced: Target occurrence rate reduced from 20% to 15%, interference rate adjusted accordingly;

[0143] Increased difficulty: Target spawn rate increased to 25%, with added false target interference.

[0144] Noise level:

[0145] Reduced difficulty: Background noise reduced by 20%, signal contrast increased;

[0146] Increased difficulty: Adds 30% random noise, reducing signal clarity.

[0147] Decision-making task: Adjusting time pressure and information complexity;

[0148] Adjust parameters:

[0149] Time pressure:

[0150] Reduce difficulty: Individual assessment time extended to 3 minutes, decision-making time extended to 4 minutes;

[0151] Increased difficulty: Assessment time reduced to 1.5 minutes, decision-making time reduced to 2 minutes.

[0152] Information complexity:

[0153] Reduce difficulty: Reduce the number of options to three and simplify the risk description;

[0154] Increase the difficulty: Increase the number of options to 5, and add dynamically changing risk parameters.

[0155] Execution task: Adjust timing window and consistency condition ratio;

[0156] Adjust parameters:

[0157] Timing window:

[0158] Reduced difficulty: The reaction window has been widened to 600ms / 400ms / 300ms;

[0159] Increase difficulty: Tighten to 400ms / 200ms / 100ms.

[0160] Consistency condition:

[0161] Reduced difficulty: The proportion of consistency conditions increased to 70%;

[0162] Increased difficulty: The proportion of inconsistent conditions has increased to 70%.

[0163] Personalized training program generation:

[0164] Establish personal collaboration ability profiles based on historical training data;

[0165] Personal ability profile data structure: {

[0166] "User ID": "USER_001",

[0167] "Historical Performance": {

[0168] "Attention Task": [Accuracy sequence, Reaction time sequence],

[0169] "Decision-making task": [Decision quality sequence, Decision time sequence],

[0170] "Task Execution": [Coordination success rate sequence, time series accuracy sequence]

[0171] },

[0172] "Brain synchronous history": {

[0173] "Alpha band": Historical PLV data,

[0174] "Beta band": Historical PLV data,

[0175] "Gamma band": PLV historical data},

[0176] "Learning curve": Fitting parameters}

[0177] Identify the weaknesses and strengths of team collaboration;

[0178] 1. Relative ranking analysis:

[0179] - Compare individual metrics with group averages

[0180] - Ability dimension to identify values ​​below 1 standard deviation from the mean

[0181] 2. Collaborative Network Analysis:

[0182] - Calculate the node centrality of an individual in a three-person network.

[0183] - Identify weak connections in collaborative networks

[0184] 3. Analysis of Progress Trends:

[0185] -The speed of improvement in various capabilities using linear regression analysis

[0186] -Ability to identify slow or stagnant progress

[0187] Generate targeted training suggestions and improvement plans.

[0188] Generation rules:

[0189] If attentional synchronicity < 0.4, then the recommendation is: "Focus on strengthening attentional coordination training and increasing alpha-band neural feedback."

[0190] If the difference in decision-making synchronicity is > 0.3, then the recommendation is: "There are significant differences in the team's decision-making styles; it is recommended to increase communication and coordination training."

[0191] If individual reaction time > group mean + 2*standard deviation, then the suggestion is: "Individual reaction speed requires specific training; it is recommended to train reaction time separately."

[0192] The key technical points of this invention mainly include the following aspects:

[0193] Firstly, this invention employs a closed-environment collaborative task design technique based on psychological experimental paradigms. It innovatively combines classic psychological experimental paradigms such as the CPT (Continuous Performance Test), the Iowa gambling task, and the Simon task with a virtual military closed-environment combat environment, forming a complete measurement system for attention synchronization, decision-making synchronization, and execution synchronization. This design not only ensures the scientific rigor and standardization of the measurements but also guarantees a high degree of relevance to actual closed-environment combat scenarios.

[0194] Secondly, there is the real-time computation technology for three-person brain synchronization. This invention develops a brain synchronization analysis algorithm specifically for three-person collaboration, capable of constructing a 3×3 synchronization network matrix and calculating the neural synchronization among the three individuals in real time. The system employs sliding window technology and frequency domain analysis methods, enabling the analysis of brain synchronization in different frequency bands across various cognitive tasks, providing objective neurophysiological indicators for collaborative assessment.

[0195] Thirdly, there is the multimodal data synchronization technology in virtual, closed, and isolated environments. This invention solves the technical challenge of achieving high-precision time synchronization among multiple users, devices, and data sources in complex VR environments. By establishing a unified hardware clock reference, an automatic event labeling system, and an adaptive latency compensation algorithm, it ensures the accurate correspondence between EEG data and VR environment events, providing a reliable foundation for subsequent data analysis.

[0196] Fourth is the adaptive training control technology based on neurofeedback. This invention is the first to use brain-to-brain synchronization as a feedback signal for training control, realizing the intelligence and personalization of the training system. The system can automatically adjust training parameters based on real-time monitored collaboration status, which not only improves training efficiency but also provides differentiated training programs for teams with different experience levels.

[0197] It should also be noted that:

[0198] While the core technical solution of this invention is unique, alternative solutions exist for certain technical aspects. In terms of brain-to-brain synchronization calculation, in addition to the Phase-Locked Value (PLV) algorithm, methods such as coherence analysis, cross-correlation analysis, or wavelet coherence analysis can also be considered. These methods each have advantages in different application scenarios, but the Phase-Locked Value method performs better in terms of real-time performance and stability, making it more suitable for the application requirements of this invention.

[0199] In terms of virtual environment display technology, besides the CAVE four-sided projection system, head-mounted VR displays or LED display arrays can also be used. Head-mounted VR displays offer a better sense of immersion, but may have inconvenient interaction issues in multi-person collaborative scenarios. LED display arrays are relatively inexpensive, but their visual effects and immersion are not as good as projection systems.

[0200] In terms of EEG signal acquisition, in addition to traditional wet electrode EEG systems, dry electrode EEG systems or near-infrared spectroscopy (fNIRS) technology can also be considered. Dry electrode systems have advantages in ease of use, but may be insufficient in signal quality and stability. fNIRS technology performs better in resisting motion artifacts, but is inferior to EEG systems in terms of temporal resolution and frequency analysis capabilities.

[0201] In designing collaborative tasks, besides classic paradigms such as CPT, the Iowa gambling task, and the Simon task, other cognitive testing paradigms, such as the Stroop task and working memory tasks, can also be considered. However, the three paradigms selected in this invention comprehensively cover the three core collaborative dimensions of attention, decision-making, and executive coordination, and have a good theoretical foundation and practical application value.

[0202] The hardware implementation scheme for the above technical solution is described below:

[0203] The system hardware implementation scheme of the present invention is as follows: Figure 1 As shown, the entire system adopts a distributed architecture, mainly consisting of four parts: a display subsystem, an EEG monitoring subsystem, a computing and processing subsystem, and a network communication subsystem.

[0204] The display subsystem employs CAVE four-sided projection technology to construct a virtual, enclosed, and isolated environment. The specific configuration includes four high-resolution laser projectors, model EPSON EB-PU1007W, with a 4K resolution and supporting a 90Hz refresh rate. The projection screens are four 3m × 2.5m rear-projection hard screens made of high-gain screen material, providing uniform brightness distribution and excellent viewing angle characteristics. The image processing system uses professional edge blending equipment, supporting geometric and color correction to ensure seamless stitching between the four projection surfaces.

[0205] The EEG monitoring subsystem employs three 64-lead EEG devices, model Brain Products ActiCHamp. Each device has a sampling rate of 1000Hz, a frequency response range of DC-200Hz, a dynamic range of ±3.2mV, and a resolution of 24 bits. Ag / AgCl wet electrodes are used, with an electrode impedance requirement of less than 5kΩ. The system is equipped with dedicated signal amplifiers and filters to effectively suppress power line interference and motion artifacts.

[0206] The computing subsystem employs a high-performance server cluster architecture. The main server is configured with an Intel i9-13900K processor, 64GB of DDR5 memory, and dual RTX 4090 graphics cards for VR environment rendering and real-time image processing. The EEG data processing server is configured with an Intel Xeon Gold 6248 processor and 128GB of DDR4 memory, specifically for real-time processing and analysis of EEG signals. The storage subsystem uses an NVMe SSD array with a total capacity of 10TB and read / write speeds exceeding 3GB / s, capable of meeting the storage requirements of multiple high-speed data streams.

[0207] The network communication subsystem adopts a gigabit Ethernet architecture, with all devices interconnected through a dedicated switch, and network latency controlled within 1ms. The system uses a dedicated time synchronization protocol to ensure that the time base of all devices is consistent, with synchronization accuracy reaching the microsecond level.

[0208] The relevant software environment and usage are also explained for the above technical solutions:

[0209] The software system adopts a modular design, mainly including a VR environment engine (equivalent to...). Figure 1 The VR environment rendering module shown), and the EEG signal processing engine (equivalent to...) Figure 1 The brain-to-brain synchronization analysis module (shown), task control engine (equivalent to) Figure 1 The multi-user interactive synchronization module shown) and the data management engine (equivalent to...) Figure 1 It consists of four core modules, including the data management and feedback module shown.

[0210] The VR environment engine is developed based on Unity 2023.3 LTS and uses the Universal Render Pipeline, supporting high-quality real-time rendering and physical simulation. The enclosed, isolated environment model is created using high-precision 3D modeling technology, including detailed internal structures, instruments, and user interfaces. The system supports multiple users online simultaneously and uses the MirrorNetworking network framework to achieve multi-user collaboration.

[0211] The EEG signal processing engine is developed using a hybrid Python and C++ approach, employing the MNE-Python library for signal preprocessing, including filtering, denoising, and artifact removal. Interbrain synchronization calculations utilize multi-threaded parallel processing technology, enabling real-time calculation of the power spectral density (PLV) values ​​among three individuals. The system supports multiple frequency band analyses, including power spectral analysis and phase analysis in the Alpha, Beta, and Gamma bands.

[0212] The task control engine manages the entire training process, including task sequence control, parameter settings, and event flagging. The system employs a state machine design, enabling flexible switching between different task modules. Each task module has an independent parameter configuration file, supporting customized settings based on training requirements.

[0213] The data management engine uses HDF5 format to store multimodal data, supporting efficient storage and retrieval of large datasets. The system implements a complete data backup and recovery mechanism to ensure the security and integrity of training data. The data analysis module provides rich statistical analysis functions, supporting in-depth analysis of training effects and report generation. For example... Figure 8 As shown, it includes a four-layer architecture:

[0214] Data acquisition layer: Real-time acquisition of EEG data, VR event data, and user behavior data;

[0215] Data processing layer: data cleaning, time synchronization, and format standardization;

[0216] Data analysis layer: statistical analysis, machine learning algorithms, and visualization processing;

[0217] Feedback Display Layer: Real-time feedback display, training report generation, and improvement suggestion output.

[0218] In response to the system described above, this invention provides a related training method, the training process of which includes the following:

[0219] like Figure 2 As shown, the entire training process is divided into five stages, totaling approximately 20 minutes. Each stage has clear task objectives and evaluation metrics, and the system can automatically record and analyze training data.

[0220] The first phase is the system initialization phase, lasting approximately 2 minutes. During this phase, three trainees respectively enter workstation A, command center B, and operating console C. The system performs hardware checks and signal calibration. Electrode impedance checks of the EEG equipment ensure signal quality, and the VR environment is personalized, including viewpoint adjustments and user habit settings. The system also records 3 minutes of resting-state EEG data to establish an individual baseline, providing a reference for subsequent brain-brain synchronization analysis.

[0221] The second phase is the attention synchronization measurement phase, lasting 6 minutes. Personnel A faces an 8×8 grid sonar monitoring interface, where a signal point appears every 500ms. 20% of these are red target signals requiring rapid button presses, while 80% are blue interference signals requiring suppression. Personnel B faces a four-quadrant information monitoring interface, simultaneously monitoring four information panels: sonar, radar, communication, and systems. Information is updated every 2 seconds, and any abnormal information with a 10% probability requires classification and processing. Personnel C executes a dual-task paradigm: the primary task is monitoring the weapon system status and performing regular maintenance, while the secondary task is performing a 2-back digit matching task. The system monitors the Alpha band brain synchronicity of the three personnel in real time to assess their attention coordination ability.

[0222] The third phase is the risk decision-making synchronization measurement phase, lasting 8 minutes. The system simulates a typical combat decision-making scenario: Personnel A discovers a suspicious target, requiring the team to jointly decide on the next course of action. The system provides four operational plans, each with different risk-reward characteristics. The decision-making process is divided into three sub-phases: individual assessment phase (2 minutes), where three individuals independently evaluate each plan; information sharing phase (3 minutes), where information is exchanged in a prescribed order; and group decision-making phase (3 minutes), where a consensus is reached through discussion. The system focuses on monitoring brain synchronization in the Beta frequency band and analyzing the decision-making coordination mechanism.

[0223] The fourth phase is the coordinated and synchronized measurement phase, which lasts for 4 minutes. For example... Figure 4 As shown, the system executes a multi-person Simon task. Target points are randomly displayed on the screen (left or right), requiring three people to coordinate their responses according to a strict time sequence. Personnel A needs to confirm the target location within 500ms, Personnel B needs to determine the attack direction within the following 300ms, and Personnel C needs to perform the corresponding preparatory actions within the final 200ms. The task consists of 40 trials, with 50% under consistent conditions and 50% under inconsistent conditions. The system focuses on monitoring brain-to-brain synchronization in the Gamma band to assess coordination ability. Figure 4As shown, the system includes a front screen, a middle screen, a rear screen, and a bottom screen; the front screen displays the main combat situation and sonar scan results; the middle screen displays the user interface and mission instructions; the rear screen displays the overall battlefield situation and threat assessment information; and the bottom screen monitors equipment status and displays system parameters.

[0224] The three workstations are arranged at a 120° angle in the center of the CAVE, allowing each user to see the content of all four screens and achieve a personalized perspective through head tracking.

[0225] The fifth stage is the results evaluation and feedback stage, lasting 2 minutes. The system automatically analyzes the training data and generates performance reports for individuals and teams. The reports include objective indicators for each task, brain-to-brain synchronization analysis results, collaborative ability assessments, and improvement suggestions. The system will also adjust individual ability profiles based on performance in this training session, providing personalized parameter settings for the next training session.

[0226] Among them, the performance indicators are calculated as follows:

[0227] Individual metrics: Attention retention = (Accuracy × 0.6) + (Reciprocal of reaction time × 0.4) - Decision quality = (Expected return × 0.7) + (Reciprocal of decision time × 0.3) - Execution coordination = (Success rate × 0.8) + (Time series accuracy × 0.2).

[0228] Team metrics: Attention Synchronization = Average PLV value of Alpha band - Decision Synchronization = Average PLV value of Beta band - Execution Synchronization = Average PLV value of Gamma band - Overall Collaboration Index = (Attention Synchronization + Decision Synchronization + Execution Synchronization) / 3.

[0229] Feedback report generation:

[0230] Real-time performance curve: Displays performance changes over 18 minutes;

[0231] Brain synchronization heatmap: showing the synchronization network among the three individuals;

[0232] Personal strengths analysis: Ability assessment based on historical data;

[0233] Team improvement suggestions: Training recommendations targeting weaknesses.

[0234] In response to the above, the present invention provides the following core content:

[0235] Inter-brain synchronization calculation is the core content of this system, and the specific implementation process is as follows: First, preprocess the 64-channel EEG signals of three people, including band-pass filtering (1-50 Hz), removing power frequency interference, and removing electrooculogram and electromyogram artifacts. Then, perform Hilbert transform on the preprocessed signals to obtain the instantaneous phase information of each electrode point. Next, calculate the phase difference between corresponding electrode points between any two people, and calculate the phase locking value through the sliding window technique. Finally, construct a 3×3 inter-brain synchronization matrix to achieve visual analysis of the three-person collaboration network.

[0236] The adaptive training control algorithm adjusts the training difficulty based on real-time inter-brain synchronization indicators. The system sets three synchronization thresholds: high synchronization threshold (PLV>0.7), medium synchronization threshold (0.4<PLV<0.7), and low synchronization threshold (PLV<0.4). When a high synchronization state is detected, the system automatically increases the task difficulty, such as increasing the frequency of the target signal appearance and increasing the decision-making time pressure. When a low synchronization state is detected, the system reduces the task difficulty, such as reducing interference information and extending the response time. The system also learns the ability characteristics of individuals and teams based on historical training data to achieve more accurate personalized control.

[0237] The above is only a preferred specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in this application should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A virtual reality closed-isolated environment multi-player cooperative training system based on inter-brain synchrony measurement, characterized in that, The system comprises a virtual closed isolated environment construction module, a multi-person cooperative task control module, an inter-brain synchrony real-time detection module, a multi-modal data synchronous acquisition module, and a neural feedback control module. The military closed isolated operation environment is constructed by the virtual closed isolated environment construction module, the multi-person cooperative task is executed by the user in the closed isolated operation environment by the multi-person cooperative task control module, the electroencephalogram of the user is collected during the multi-person cooperative task by the inter-brain synchrony real-time detection module, the inter-brain synchrony is evaluated according to the electroencephalogram to obtain a synchrony index, the event and behavior data and the electroencephalogram in the multi-person cooperative task are synchronously acquired by the multi-modal data synchronous acquisition module, and the difficulty of the multi-person cooperative task is adjusted according to the synchrony index by the neural feedback control module.

2. The system of claim 1, wherein in the multi-person cooperative task control module, the multi-person cooperative task comprises an attention synchrony measurement task, a risk decision synchrony measurement task, and an execution coordination synchrony measurement task, the attention synchrony measurement task is designed based on a CPT method, different users perform a sonar monitoring task, a four-quadrant information monitoring task, and a weapon-related double task in the attention synchrony measurement task, the weapon-related double task is performed by one user and comprises weapon system readiness state maintenance and 2-back digital matching, the risk decision synchrony measurement task is set based on an Iowa gambling task, the risk decision synchrony measurement task comprises different decision schemes, and different decision schemes are selected by individual evaluation, information sharing, and group decision of the user, and the execution coordination synchrony measurement task is designed based on a multi-person Simon task, different users perform coordinated responses under different timing requirements in the execution coordination synchrony measurement task.

3. The system of claim 1, wherein in the inter-brain synchrony real-time detection module, the specific process of evaluating the inter-brain synchrony according to the electroencephalogram comprises: in the electroencephalogram, key electroencephalogram data in different frequency bands are extracted, the key electroencephalogram data are calculated by a phase locking value method to obtain a synchrony index, the key electroencephalogram data are processed by a Hilbert transform method to obtain an instantaneous phase, the phase difference between the electroencephalograms of different users is calculated according to the instantaneous phase, and the synchrony index is obtained.

4. The system of claim 1, wherein in the neural feedback control module, the process of adjusting the difficulty of the multi-person cooperative task according to the synchrony index comprises: in the inter-brain synchrony real-time detection module, a synchrony matrix of different frequency bands is constructed according to the synchrony index; in the neural feedback control module, the synchrony matrix is classified to obtain different cooperative states, the cooperative states comprise a high synchrony state, a medium synchrony state, and a low synchrony state, and the corresponding difficulty adjustment schemes are: increasing the task difficulty, maintaining the task difficulty, and reducing the task difficulty.

5. The system of claim 3, wherein ​ In the brain inter-synchronism real-time detection module, the key brain electrical data includes brain electrical signals of Alpha frequency band, Beta frequency band and Gamma frequency band, which respectively correspond to the synchronism indexes of attention synchronism measurement task, risk decision synchronism measurement task and execution coordination synchronism measurement task.

6. The system of claim 1, wherein, In the multi-modal data synchronous acquisition module, the data synchronization of the event and behavior data and the brain electrical signals in the multi-person cooperation task includes: The cooperation events in the multi-person cooperation task are marked, wherein the cooperation events include task stage conversion, individual operation behavior and cooperation interaction behavior, and the data synchronization of the event and behavior data and the brain electrical signals in the multi-person cooperation task is performed through hardware clock synchronization, software time stamp calibration and delay compensation method.

7. The system of claim 4, wherein, In the neural feedback control module, the real-time cooperation state recognition method is used for classification according to the synchronism matrix, wherein the real-time cooperation state recognition method adopts a machine learning method.

8. The system of claim 1, wherein, In the brain inter-synchronism real-time detection module, the brain electrical signals are preprocessed before the brain inter-synchronism evaluation according to the brain electrical signals, wherein the preprocessing includes band-pass filtering, removing power frequency interference, removing electrooculogram and electromyogram artifacts.

9. The system of claim 1, wherein, The individual indexes and team indexes of different tasks, the brain inter-synchronism analysis and the cooperation ability evaluation are also performed through the neural feedback control module.

10. A method for multi-player cooperative training in a virtual reality closed-isolated environment based on brain inter-synchrony measurement, characterized in that, The system according to any one of claims 1-9 is used for multi-person cooperation training in a virtual reality closed isolated environment.

Images