Simulation system, acquisition system and interaction system of closed environment

Abstract

The invention relates to a simulation system, an acquisition system and an interaction system of a closed environment, the interaction system comprises the simulation system, the acquisition system and a configuration system of the closed environment, and the configuration system is used for configuring related data according to experiment requirements; the closed environment simulation system is used for simulating a preset virtual environment according to the collected relevant data of the real closed environment and the stimulation source event, and collecting behavior data of a user in the preset virtual environment to adjust the state of the preset virtual environment so as to construct a target virtual environment; the acquisition system is used for acquiring electroencephalogram signals of a user in a target virtual environment when the acquisition normal forms are executed to obtain target electroencephalogram signal samples, and performing feature coding on the target electroencephalogram signal samples obtained by executing each type of acquisition normal forms to generate an electroencephalogram signal data set for analyzing the mental state of the user. Therefore, safe, reliable and immersive closed environment simulation can be realized, and reliable data support is provided for collection of electroencephalogram signals.

Description

Technical Field

[0001] This application mainly relates to the field of electroencephalogram acquisition, and particularly to a simulation system, an acquisition system and an interaction system for a closed environment. Background Art

[0002] When humans work in a closed environment for a long time, they may experience negative emotions such as anxiety, loneliness, fear, etc., and even have serious physiological reactions, such as excessive stress, inattention, etc. How to maintain a good mental state in a tense closed environment is very meaningful for improving work efficiency and the physical and mental health of workers.

[0003] By simulating a closed environment, researchers can better understand the effects of different environmental conditions on human psychology, cognitive abilities and physiological reactions. However, traditional closed environment experiments usually face problems such as high cost of actual environment reconstruction, limited experimental samples, limited data acquisition, etc. In addition, for the assessment of mental states such as sleep quality, anxiety level, attention, etc., currently mainly using the self-rating scale method, this method overly relies on the subjective evaluation of the subjects, and the assessment results lack objective and quantitative biological index support.

[0004] With the rapid development of brain-computer interface technology, brain-computer interface technology has shown great potential in mental state assessment tools, and the market demand for systems that can acquire high-quality electroencephalogram data for mental state assessment is increasing day by day. However, in a closed environment, there are still many problems in collecting high-quality data for mental state assessment, such as insufficient scene authenticity, low diversification degree, poor interaction and lack of feedback mechanism, etc. In addition, the electroencephalogram quality in a closed environment is poor, which also poses high requirements for the feature coding scheme of mental problems. Summary of the Invention

[0005] An object of this application is to provide a simulation system, an acquisition system and an interaction system for a closed environment, so as to solve the problems of high cost of closed environment reconstruction, limited data acquisition, poor electroencephalogram quality and poor interaction in the prior art.

[0006] According to one aspect of this application, a simulation system for a closed environment is provided. The simulation system includes: a three-dimensional scene reconstruction module, an environment interaction control module, a scene presentation module and a user interaction module; wherein,

[0007] The three-dimensional scene reconstruction module is used to collect relevant data of a real closed environment and reconstruct a three-dimensional virtual scene based on the relevant data of the real closed environment;

[0008] The environment interaction control module is used to set different types of stimulus events and simulate a preset virtual environment according to the scene performance of the three-dimensional virtual scene for each stimulus event;

[0009] The scene presentation module is used to present the preset virtual environment on the virtual interface;

[0010] The user interaction module is used to collect the behavior data of the user in the preset virtual environment, and adjust the state of the target virtual environment according to the behavior data to construct a target virtual environment.

[0011] Optionally, the three-dimensional scene reconstruction module is used to collect relevant data of the real closed environment through different acquisition devices, perform feature matching on the relevant data, extract key features and align them, and construct a three-dimensional virtual scene according to the aligned features.

[0012] Optionally, the three-dimensional scene reconstruction module is used to perform virtualization adjustment according to simulation requirements, wherein the virtualization adjustment includes adjustment of physical environment parameters, adjustment of elements in the three-dimensional virtual scene, and editing and modification of virtual objects and interaction behaviors.

[0013] Optionally, the environment interaction control module is used to adjust the intensity and duration of the stimulation event according to the feedback information of the user on the stimulation event.

[0014] Optionally, the user interaction module is used to identify the collected behavior data, determine the operation instructions of the user on the objects in the preset virtual environment, and adjust the parameters of the objects according to the operation instructions, wherein the behavior data includes body posture information, limb movement information, and eye movement trajectory information, and the parameters include position information, rotation information, and state information of the objects.

[0015] Optionally, the user interaction module configures event trigger buttons and corresponding event effects, and is used to track the perspective position of the user according to the received selection operation of the user on the event trigger buttons, and display the corresponding event effects at the perspective position, so as to adjust the display content of the target virtual environment according to the event effects.

[0016] According to another aspect of the present application, there is also provided an acquisition system based on a simulation system, and the acquisition system includes: a signal acquisition module, a data processing module, and a feature encoding module, wherein,

[0017] The signal acquisition module is used to receive an acquisition instruction carrying an acquisition paradigm type, and acquire electroencephalogram signals of the user in the virtual environment provided by the simulation system when executing the acquisition paradigm;

[0018] The data processing module is used to screen the electroencephalogram signals to obtain target electroencephalogram signal samples; the feature encoding module is used to perform feature encoding on the target electroencephalogram signal samples obtained by executing each type of acquisition paradigm;

[0019] The data processing module is used to determine the mental state evaluation result of the user under each type of acquisition paradigm according to the encoded features, so as to generate an EEG signal dataset for analyzing the user's mental state. Among them, the mental state evaluation result includes the state type and intensity.

[0020] Optionally, the data processing module is used to label the target EEG signal samples according to the mental state evaluation result, the basic information of the user, and the closed operation time, so as to generate an EEG signal dataset for analyzing the user's mental state.

[0021] Optionally, the data processing module is used to determine the mental state evaluation result of each household under each type of acquisition paradigm according to the encoded features and the mental state comparison table, generate an evaluation scale, and track the mental state change according to the evaluation scale and the subsequently collected EEG signals.

[0022] Optionally, the acquisition paradigm includes a resting state paradigm, an attention paradigm, and a sustained attention task paradigm. The signal acquisition module is used to receive a request carrying an end tag after the end of each type of acquisition paradigm, so as to start the next stage of the acquisition paradigm and disable the current stage of the acquisition paradigm.

[0023] According to another aspect of the present application, an interaction system is further provided. The interaction system includes a simulation system of a closed environment, an acquisition system, and a configuration system, where

[0024] The configuration system is used to configure relevant data according to experimental requirements. Among them, the relevant data includes different types of acquisition paradigms, stimulus source event types and intensity levels, and transmission configuration information of the acquisition system;

[0025] The simulation system of the closed environment is used to simulate a preset virtual environment according to the relevant data of the real closed environment collected and the stimulus source event, and collect the behavior data of the user in the target virtual environment to adjust the state of the preset virtual environment to construct a target virtual environment;

[0026] The acquisition system is used to receive an acquisition instruction carrying the type of acquisition paradigm issued by the configuration system, collect the EEG signals of the user in the virtual environment provided by the simulation system when performing the acquisition paradigm, and perform screening to obtain target EEG signal samples, and perform feature encoding on the target EEG signal samples obtained by performing each type of acquisition paradigm, so as to generate an EEG signal dataset for analyzing the user's mental state.

[0027] Compared with the prior art, a simulation system, a collection system, and an interaction system for a closed environment provided by the present application can achieve safe, reliable, three-dimensional, and immersive closed environment simulation, and collect high-quality electroencephalogram data for multiple scenarios. At the same time, through the adjustable design of the collection paradigm, the type of stimulation event, and the stimulation intensity, efficient feature encoding of mental problems is realized, providing reliable data support for mental health assessment, intervention, and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to make the above objects, features, and advantages of the present application more obvious and understandable, the following detailed description of the specific embodiments of the present application will be given in conjunction with the accompanying drawings, where:

[0029] Figure 1 FIG. shows a schematic structural diagram of a simulation system for a closed environment provided according to one aspect of the present application;

[0030] Figure 2 FIG. shows a schematic structural diagram of a collection system based on the simulation system provided according to another aspect of the present application;

[0031] Figure 3 FIG. shows a schematic structural diagram of an interaction system provided according to still another aspect of the present application.

[0032] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] In order to make the above objects, features, and advantages of the present application more obvious and understandable, the following detailed description of the specific embodiments of the present application will be given in conjunction with the accompanying drawings.

[0034] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application may be implemented in other ways different from those described herein. Therefore, the present application is not limited by the specific embodiments disclosed below.

[0035] As shown in the present application and the claims, unless the context clearly indicates otherwise, the words "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0036] Figure 1 FIG. shows a schematic structural diagram of a simulation system for a closed environment provided according to one aspect of the present application. The simulation system 1 includes: a three-dimensional scene reconstruction module 11, an environmental interaction control module 12, a scene presentation module 13, and a user interaction module 14; wherein,

[0037] The three-dimensional scene reconstruction module 11 is used to collect relevant data of a real closed environment and reconstruct a three-dimensional virtual scene based on the relevant data of the real closed environment. Herein, the three-dimensional scene reconstruction module 11 includes laser scanning, unmanned aerial vehicle remote sensing, and meteorological data collection devices. Through these collection devices, real relevant data of the closed environment is obtained, and the relevant data includes, but is not limited to, geographical features, climate, ecology, air pressure, humidity, etc. The collected relevant data is combined with virtual reality technology to generate a three-dimensional virtual scene with a high sense of realism.

[0038] The environmental interaction control module 12 is used to set different types of stimulus events and simulate a preset virtual environment according to the scene performance of the three-dimensional virtual scene for each stimulus event. Herein, the stimulus events include natural disasters, equipment failures, etc. Natural disasters such as snowstorms, ice layer ruptures, etc. By setting different stimulus events, the performance of the three-dimensional virtual scene is affected to help users better enter the experimental state and simulate the required preset virtual environment. Among them, the user is the subject when constructing the simulation system, so that subsequently, the required closed environment is constructed through the virtual experience of the subject and the virtual mapping of the physical environment.

[0039] The scene presentation module 13 is used to present the preset virtual environment on the virtual interface. Herein, the scene presentation module is used for virtual reality display and presentation. A virtual reality head-mounted device, such as Oculus Rift or HTC Vive, can be used to present the preset virtual scene of the closed environment to the user through the virtual interface of the virtual reality device, ensuring that the user can immerse in a highly realistic virtual environment. In this module, high-resolution graphics and real-time rendering are used to provide a virtual interface for the user, so that the user can see a high-precision closed environment scene, and the dynamic changes in the scene (such as weather changes, equipment failures, etc.) can match the real environment, enhancing the user's sense of reality. In addition, this module also supports haptic feedback, thereby providing more sensory stimuli to further enhance the user's immersive experience.

[0040] The user interaction module 14 is used to collect the behavior data of the user in the preset virtual environment, and adjust the state of the preset virtual environment according to the behavior data to construct a target virtual environment. Thus, a safe, reliable, three-dimensional and immersive closed environment simulation can be realized. Herein, the user interaction module is used for user behavior tracking and interaction with the user. It collects the behavior data of the user in the virtual environment in real time through a head motion sensor, a gesture recognition device, an eye tracker, etc. The behavior data includes head motion, hand movements, etc., and interacts with the virtual environment in real time according to the behavior data, adjusts the state of the virtual environment according to the behavior data, so as to obtain the required target virtual environment for experiments. The target virtual environment is constructed through the virtual experience of the subject and the virtual mapping of the physical environment, which better meets the experimental requirements and provides conditional support for collecting real EEG signals.

[0041] In some embodiments of the present application, the three-dimensional scene reconstruction module 11 is used to collect the relevant data of the real closed environment through different acquisition devices, perform feature matching on the relevant data, extract key features and align them, and construct a three-dimensional virtual scene according to the aligned features. Herein, the three-dimensional scene reconstruction module provides a highly realistic virtual space for the simulation system. In this module, by collecting and processing the sensor data of the real environment, using technologies such as laser scanning, photogrammetry, and depth images, a high-precision three-dimensional model is constructed; the relevant data of the real closed environment can be collected through multiple perspectives and perception devices, so as to ensure the accuracy and integrity of the three-dimensional model. An algorithm can be built into the module to perform precise matching, alignment, and optimization processing on the collected raw data, eliminate possible noise or errors, and generate a final three-dimensional digital model available for the simulation system. The specific implementation steps of the algorithm built into the module are as follows: Step 1, use a feature matching method (such as SIFT) to perform a preliminary match on different data sources, extract key features and align them. Step 2, use the ICP algorithm to precisely align the data from different perspectives or sensors, and convert it into spatial data in a unified coordinate system. Step 3, remove noise and errors through a Kalman filter to ensure the accuracy of the data. Step 4, use the least squares method to optimize the three-dimensional model, further refine and refine the data, and ensure that the finally generated digital model has high precision and can be effectively used by the system.

[0042] In some embodiments of the present application, the three-dimensional scene reconstruction module 11 is used to perform virtualization adjustment according to simulation requirements. Among them, the virtualization adjustment includes the adjustment of physical environment parameters, the adjustment of elements in the three-dimensional virtual scene, and the editing and modification of virtual objects and interaction behaviors. Here, the three-dimensional scene reconstruction module 11 not only supports the reconstruction of the physical environment, but also can perform virtualization adjustment on the environment according to simulation requirements. By editing and modifying the model, users can simulate different physical characteristics and interaction behaviors according to their needs in the virtual environment, thereby providing a more flexible experimental or application scenario, ensuring a high degree of reduction of the virtual environment, and making the user experience in the virtual reality environment more real. Among them, when performing virtualization adjustment, it includes the adjustment of physical environment parameters, the adjustment of elements in the three-dimensional virtual scene, and the editing and modification of virtual objects and interaction behaviors; the physical environment parameters include physical environment parameters such as weather and wind speed; the adjustment of elements in the three-dimensional virtual scene includes the modification of various elements, physical properties, behaviors and interaction methods in the environment. By adjusting the appearance or layout of objects, buildings, terrain and other elements in the environment, it can be reconstructed according to simulation requirements; at the same time, physical properties such as gravity, light intensity, and material reflection characteristics can also be modified to simulate different environmental conditions. Virtual objects include virtual characters and objects. The adjustment of the behavior patterns and interaction methods of the characters or objects in the virtualized scene also includes the modification of the time scale and space dimension. Among them, the modification on the time scale is such as day-night cycle and seasonal change, and the modification on the space dimension is such as space-time distortion and 3D space scaling. Through the above virtualization adjustment, a virtual environment that accurately simulates specific requirements or scenarios is finally realized.

[0043] It should be noted that the above editing and modification of the model refer to the adjustment of the objects, characters and their interaction behaviors in the virtual environment by the operator through the configuration interface to meet specific experimental requirements or application scenarios. The specific process includes: 1) Modify the appearance, material, size and shape of the object, adjust physical properties such as gravity, friction and elasticity, and simulate different physical environments; 2) Set the interaction logic of the object and the character, and define how they respond to user operations or interact with each other; 3) Adjust dynamic elements such as weather changes and day-night cycles to create a variable virtual world; 4) Modify the behavior model of the character and set its reaction or behavior pattern in a specific situation. Through these edits and modifications, users can flexibly simulate various physical characteristics and interaction behaviors according to their needs, providing more possibilities for experimental or application scenarios.

[0044] In some embodiments of the present application, the environmental interaction control module 12 is used to adjust the dynamic state of the three-dimensional virtual scene according to different stimulus events, so as to determine the new behaviors and visual effects of the objects in the three-dimensional virtual scene. Herein, the environmental interaction control module 12 affects the scene performance in the virtual environment by setting different stimulus events to help the user enter a preset experiment or scenario. The environmental interaction control module 12 adjusts the dynamic state of the virtual environment by simulating events such as natural disasters or equipment failures, thereby affecting the object behaviors and visual effects in the environment. For example, when simulating a snowstorm event, the environmental interaction control module 12 can change parameters such as the weather, wind speed, and visibility in the virtual environment in real time, thereby affecting the user's perception and interaction with the scene. In this interaction process, the user's experience will be more immersive, thus improving the realism and reliability of the experiment. Through the method provided by the environmental interaction control module 12, not only the realism and immersion of the virtual environment are enhanced, but also more flexible and dynamic control can be provided according to the experimental requirements to help the user enter a specific virtual scenario.

[0045] In some embodiments of the present application, the environmental interaction control module 12 is used to adjust the intensity and duration of the stimulus event according to the feedback information of the user on the stimulus event. Herein, the environmental interaction control module 12 has the function of dynamically adjusting the stimulus intensity and can adjust the intensity and duration of the stimulus event according to the user's reaction, so as to simulate emergency responses in complex environments.

[0046] In some embodiments of the present application, the scene presentation module 13 is used to track the user's perspective and adjust the virtual objects in the target virtual environment according to the user's perspective to update the display content in real time. Herein, the scene presentation module 13 is responsible for presenting the visual information of the virtual environment to the user. This module converts the three-dimensional model and virtual scene into an image that the user can perceive through high-quality image rendering technology, and through cooperation with hardware devices, ensures that the virtual environment can be naturally and smoothly displayed in the user's field of view. The hardware devices include, but are not limited to, VR helmets, display screens, eye trackers, etc. The scene presentation module 13 uses a powerful rendering engine to realize the construction and rendering of the three-dimensional environment, supports technologies such as ray tracing and shadow processing to enhance the realism of the virtual environment; the objects and characters in the virtual scene can be dynamically adjusted according to the user's perspective, so as to ensure that the objects in the field of view always maintain a real sense of space. Through eye tracking and head tracking technologies, the scene presentation module 13 can update the display content in real time according to the change of the user's perspective, making the interaction between the virtual environment and the real world seamlessly connected.

[0047] In some embodiments of the present application, the user interaction module 14 is used to identify the collected behavior data, determine the operation instructions of the user on the objects in the preset virtual environment, and adjust the parameters of the objects according to the operation instructions. Wherein, the behavior data includes body posture information, limb movement information, and eye movement trajectory information, and the parameters include the position information, rotation information, and state information of the objects.

[0048] The main function of the user interaction module 14 is to track the user's actions in the virtual environment in real time and adjust the state of the virtual environment according to these actions. This module integrates a variety of sensors and devices, such as motion capture systems, gesture recognition, VR controllers, eye trackers, etc. It uses the integrated sensors and devices to capture data such as the user's body posture, hand movements, and eye movement trajectories, and converts them into operation instructions in the virtual environment. The user interaction module 14 can accurately identify the user's gestures and actions and support multiple interaction methods, such as gesture control, object grasping and releasing, etc. For example, when the user manipulates a virtual object by reaching out or waving, the system will adjust the position, rotation, or state of the object in real time according to the user's actions. Among them, gesture recognition can not only support direct physical interaction, but also execute more complex commands, such as operating virtual devices, changing environmental settings, etc. The eye tracking method is used in this module, making the interaction experience more natural and intuitive. By monitoring the user's line of sight, the system can judge the user's focus of attention and react based on the fixation behavior. For example, the user can select a virtual object by gazing at it, or activate certain functions through eye movement control. The user no longer needs to rely on hand or voice input and can perform effective operations simply through eye movement, greatly improving the convenience and immersion of the interaction.

[0049] In some embodiments of the present application, the user interaction module 14 configures event trigger buttons and corresponding event effects, and is used to track the user's perspective position according to the received selection operation of the user on the event trigger buttons, and display the corresponding event effects at the perspective position to adjust the display content of the target virtual environment according to the event effects. Here, the user interaction module 14 configures event trigger buttons on the VR screen. For example, two buttons are configured, namely "fire" and "power outage", and corresponding event effects are configured for the events. For example, a red ray display effect is configured for "fire". In a specific scenario, when entering the event judgment operation, the "fire" and "power outage" buttons will be displayed on the VR screen. The user selects the corresponding event special effect by pressing the trigger keys of the left and right handles. After the correct selection, it enters the fault point query stage. In the fault point query operation, the user's gaze position is identified by an eye tracker. When the user presses the trigger key, the handle will emit a red ray pointing to the special effect position. When the ray points to the correct fault point, it turns green, and the user presses the trigger key again to eliminate it. When the ray does not point to the fault point, it remains red.

[0050] Figure 2 The structural schematic diagram of an acquisition system based on a simulation system provided according to another aspect of the present application is shown. The acquisition system 2 includes: a signal acquisition module 21, a data processing module 22, and a feature encoding module 23. Through the adjustable design of the acquisition paradigm, efficient feature encoding of mental problems is achieved, providing reliable data support for mental health assessment, intervention, and treatment. Among them,

[0051] The signal acquisition module 21 is configured to receive an acquisition instruction carrying the type of acquisition paradigm, and acquire the electroencephalogram (EEG) signals of the user in the virtual environment provided by the simulation system when the acquisition paradigm is executed; here, the virtual environment provided by the simulation system may be the target virtual environment constructed by the above simulation for more accurate assessment of the user's mental state in the future. When the user is in the virtual environment provided by the simulation system, the user interacts with the virtual environment, and during the interaction process, the EEG signals of the user are acquired. The EEG signals are neural signals generated when the user faces the virtual environment stimuli, which can reflect the interaction situation, such as interaction time, reaction speed, and interaction accuracy, etc., and thus can better reflect the mental state of the user in response to the closed environment.

[0052] The data processing module 22 is configured to screen the EEG signals to obtain target EEG signal samples; here, after the EEG signals are acquired, the signals are processed such as cleaning, denoising, and storing, and then qualified EEG signal samples are screened out, and the screened EEG signal samples are classified and sorted to obtain the required data set.

[0053] The feature encoding module 23 is configured to perform feature encoding on the target EEG signal samples obtained by executing each type of acquisition paradigm. The data processing module 22 is configured to determine the mental state assessment results of the user under each type of acquisition paradigm according to the encoded features to generate an EEG signal data set for analyzing the user's mental state, where the mental state assessment results include state types and intensities. Here, the feature encoding is the feature encoding of the user's mental problems. Different types of acquisition paradigms are configured for the acquired EEG signals, and then the data processing module 22 can effectively analyze and classify the effects of different mental problems on the EEG signals, evaluate the user's mental state, and determine the state types and intensities. The state types include different mental states or mental problems, cognitive ability states, etc. The intensity can be divided by scores or by grades.

[0054] In some embodiments of the present application, the acquisition paradigm includes a resting-state paradigm, an attention paradigm, and a sustained attention task paradigm. The signal acquisition module is configured to receive a request carrying an end tag after the end of each type of acquisition paradigm, so as to initiate the next stage of the acquisition paradigm and disable the current stage of the acquisition paradigm. Here, each paradigm corresponds to a different mental state or mental problem. The resting-state paradigm records EEG signals in a resting state without any external stimuli. This paradigm is used to analyze the basic activity pattern of the brain when not under any interference and can serve as a benchmark for mental health status; the attention paradigm (ANT) requires the subject to maintain attention in a specific cognitive task, such as a choice reaction task (Go / No-Go), to evaluate their cognitive control ability and attention level; the sustained attention task paradigm (SART) is used to evaluate an individual's performance in a sustained attention task. The task requires the subject to continuously focus on a specific stimulus and respond within a period of time. By recording the characteristics of EEG fluctuations during the task completion process, the individual's attention maintenance ability, anxiety level, etc. can be further analyzed. Since the entire experimental process includes multiple paradigms, the system will send an end tag to the COM port at the end of each paradigm, thereby enabling the button for the next stage paradigm and disabling the button for the current stage paradigm.

[0055] In some embodiments of the present application, continuing to refer to Figure 2 , the signal acquisition module 21 includes an EEG device 211 and an amplifier 212. The EEG device 211 is configured to collect the EEG signals of the user through an electrode array, and the amplifier 212 is configured to amplify the EEG signals. Here, the EEG device 211 can be an EEG cap, which is convenient for the user to wear. It contacts the scalp of the subject through the electrode array and records the electrical activities of the cerebral cortex neurons in real time. Through precise electrode arrangement, the EEG device can collect signals covering different brain regions, thereby providing complete brain electrical activity data; the amplifier 212 is responsible for amplifying and preliminarily processing the signals to ensure that the collected signals can meet the requirements of subsequent analysis. The signal acquisition module 21 can stably and efficiently acquire real-time EEG data and is the basic module of the entire system.

[0056] In some embodiments of the present application, the data processing module 22 is configured to denoise the EEG signals, screen out target EEG signal samples from the denoised EEG signals according to a set threshold, and classify the target EEG signal samples according to the mental state and cognitive ability of the user.

[0057] In the data preprocessing stage, methods such as band-pass filtering, blind source separation, and empirical mode decomposition are used to filter out the noise in the EEG samples, thereby improving the signal-to-noise ratio of the signal. In the data screening and sample classification stage, thresholds of statistical quantities such as the signal-to-noise ratio and standard deviation are set. For example, if the signal-to-noise ratio is set to 5 dB, it means that the effective part of the signal should be more than 5 times that of the noise in order to consider the signal clear enough. The standard deviation is set to 50 μV, indicating that the range of signal fluctuations does not exceed 50 μV to ensure the stability of the signal. Thus, qualified EEG signal samples are automatically screened out by setting the thresholds of the signal-to-noise ratio and standard deviation. Subsequently, the EEG samples are classified and sorted according to factors such as the mental state and cognitive ability of the subjects. For example, classifications such as healthy / unhealthy in insomnia, healthy / unhealthy with distracted attention, and healthy / unhealthy in anxiety are sorted out.

[0058] In some embodiments of the present application, the data processing module 22 is used to label the target EEG signal samples according to the mental state evaluation result, the basic information of the user, and the closed operation time, and generate an EEG signal data set for analyzing the mental state of the user. Here, the mental state evaluation result is evaluation result information such as the mental health state and cognitive ability state, and is labeled by combining various data such as the basic information of the user's age, gender, etc. and the closed operation time in a closed environment, and then an EEG signal data set with labels is formed. The data set labeled with labels is stored in the database. These labeled data provide an important basis for subsequent analysis, ensuring that the system can make an accurate mental state evaluation according to the individual differences of the user.

[0059] In some embodiments of the present application, the data processing module 22 is configured to determine the mental state assessment results of each household under each type of acquisition paradigm according to the encoded features and the mental state comparison table, generate an assessment scale, and track the change of the mental state according to the assessment scale and the subsequently acquired electroencephalogram signals. Here, in the self-assessment scale part, the Pittsburgh Sleep Quality Index (PSQI) and the Hamilton Anxiety Rating Scale (HAMA) can be used to conduct a control score for the user, calculate the overall mental condition score, conduct a quantitative assessment, obtain a self-quantification form, and conduct mental state tracking; the tracking method can be to regularly collect the scale and calculate the mental state score, and combine the mental state assessment of the user obtained from the subsequently collected electroencephalogram signals to achieve the tracking of the change of the mental state. It should be noted that the Pittsburgh Sleep Quality Index is used to measure the sleep quality and sleep disorders of an individual in the past month, covering multiple aspects of sleep, including sleep duration, sleep onset time, sleep efficiency, sleep quality, and the frequency of sleep disorders. The Hamilton Anxiety Rating Scale is used to evaluate the anxiety symptoms and their severity of an individual, covering symptoms such as anxiety, tension, fear, insomnia, and depression. Each item has a score from 0 to 4 to indicate the severity of the symptoms. The Pittsburgh Sleep Quality Index and the Hamilton Anxiety Rating Scale cover a wide range and can provide a relatively accurate assessment of the user's mental state.

[0060] Figure 3 FIG. shows a schematic structural diagram of an interaction system provided according to another aspect of the present application. The interaction system includes a simulation system 1 of a closed environment, an acquisition system 2, and a configuration system 3, so as to realize a safe, reliable, three-dimensional and immersive simulation of the closed environment and collect high-quality electroencephalogram data of multiple scenarios. At the same time, through the adjustable design of the acquisition paradigm, the type of stimulation event, and the stimulation intensity, an efficient feature encoding of mental problems is realized, providing reliable data support for mental health assessment, intervention, and treatment.

[0061] The simulation system 1 of the closed environment is used to simulate a preset virtual environment according to the relevant data of the real closed environment and the stimulation source events, and collect the behavior data of the user in the target virtual environment to adjust the state of the preset virtual environment to construct a target virtual environment. Here, the simulation system 1 includes a three-dimensional scene reconstruction module 11, an environment interaction control module 12, a scene presentation module 13, and a user interaction module 14. Among them, the three-dimensional scene reconstruction module 11 is used to collect the relevant data of the real closed environment and reconstruct a three-dimensional virtual scene based on the relevant data of the real closed environment. The environment interaction control module 12 is used to set different types of stimulation events and simulate a preset virtual environment according to the influence performance of each stimulation event on the three-dimensional virtual scene. The scene presentation module 13 includes a virtual reality head-mounted device, and the virtual reality head-mounted device is used to present the preset virtual environment on a virtual interface. The user interaction module 14 is used to collect the behavior data of the user in the preset virtual environment and adjust the state of the preset virtual environment according to the behavior data, so as to construct a target virtual environment. By simulating a closed environment through virtual reality technology, various environmental changes and emergency events, such as natural disasters and equipment failures, can be realistically reproduced, improving the immersion and real experience of the subjects. By simulating complex environmental emergency responses, it helps the subjects better enter the experimental state. In addition, through the dynamic adjustment and personalized design of the stimulation events, the system effectively improves the accuracy and reliability of the mental problem feature coding, can provide customized experimental scenarios for different users, and thus enhances the scientificity and accuracy of the mental state assessment.

[0062] The acquisition system 2 is used to receive the acquisition instruction carrying the acquisition paradigm type sent by the configuration system, collect the electroencephalogram signals of the user in the virtual environment provided by the simulation system when executing the acquisition paradigm, and perform screening to obtain the target electroencephalogram signal samples, and perform feature coding on the target electroencephalogram signal samples obtained by executing each type of acquisition paradigm to generate an electroencephalogram signal dataset for analyzing the mental state of the user. Here, the acquisition system 2 includes a signal acquisition module 21, a data processing module 22, and a feature coding module 23. Among them, the signal acquisition module 21 includes an electroencephalogram device 211 and an amplifier 212. The electroencephalogram device 211 is used to collect the electroencephalogram signals of the user through an electrode array, and the amplifier 212 is used to amplify the electroencephalogram signals. The data processing module is used to denoise the electroencephalogram signals, screen out the target electroencephalogram signal samples from the denoised electroencephalogram signals according to the set threshold, and classify the target electroencephalogram signal samples according to the mental state and cognitive ability of the user. The data processing module 22 is also used to calculate the mental condition score of the user, perform quantitative evaluation to obtain a self-quantification form, and perform mental state tracking.

[0063] The acquisition system 2 obtains the user's EEG signals in real time and uses advanced signal processing techniques, such as band-pass filtering, blind source separation, and empirical mode decomposition, to remove noise and significantly improve the signal-to-noise ratio of the signals, thus ensuring the high quality of EEG data. Through the feature encoding module, it can accurately analyze the EEG signals and identify mental states such as anxiety, attention, and stress. Thresholds are set according to statistics such as the signal-to-noise ratio and standard deviation of the data for sample screening, and sample annotation is carried out by combining various data such as self-check or inspection scales, the user's age, and the closed operation time, constructing a high-quality database. Through the design of various paradigms such as the resting-state paradigm, attention paradigm, and sustained attention task paradigm, the characteristics of different mental states and problems are comprehensively covered, providing a reliable biological basis for subsequent mental health assessment and intervention, and providing a convenient data management platform for subsequent long-term data tracking and analysis.

[0064] The configuration system 3 is used to configure relevant data according to experimental requirements. Among them, the relevant data includes different types of acquisition paradigms, stimulus source event types and intensity levels, and the transmission configuration information of the acquisition system; here, the configuration system provides an operation interface for the operator. When conducting an experiment, according to the current experimental requirements, the operator configures the relevant data on the operation interface; among them, the stimulus source types include fire, power outage, snowstorm, ice rupture, etc.; the transmission configuration information includes the configuration of the port and baud rate. The port is COM and the baud rate is 9600, so as to realize the interaction with the simulation system and the acquisition system. Among them, the acquisition paradigms in the relevant data include the resting-state paradigm, attention paradigm, and sustained attention task paradigm. The acquisition paradigm selected by the operator on the operation interface is obtained, so as to carry out personalized customization for the individual user and conduct individual mental state assessment. For example, taking the resting-state paradigm as an example, the system will send corresponding commands to the COM port of the EEG device according to the user's open or closed eye state to start or collect the process.

[0065] In some embodiments of the present application, the user wears devices such as an EEG device and a VR glasses to construct an immersive closed environment to simulate and collect corresponding EEG signals; in addition, devices such as an eye tracker, a handle, and a keyboard are equipped to ensure the detection of the user's behavior, realize human-computer interaction and give real-time feedback. When the user wears the devices, the operation process of the interaction system is as follows:

[0066] The configuration system obtains the personal information of the subject (user) input by the operator and selects the required EEG signal acquisition paradigm according to the experimental requirements. After selecting the acquisition paradigm, the system will automatically pop up the corresponding configuration interface to facilitate the operator to perform specific configurations, including configuring the subject's information, IP, paradigm selection, disease, specific stage time, test method, scene, weather, event, etc. Taking the resting-state EEG paradigm as an example, the system will send corresponding commands to the COM port of the EEG device according to the subject's state (open eyes or closed eyes) to start or end the acquisition process. In the "open eyes" state, the system will start timing and initiate EEG acquisition; in the "closed eyes" state, the system will reconfigure and collect data. In addition to the resting-state paradigm, the system also supports other task paradigms, such as the ANT paradigm and the SART paradigm. The ANT paradigm evaluates the subject's attention level by designing an attention interference task, while the SART paradigm evaluates the subject's sustained attention by requiring the subject to continuously maintain attention to the stimulus. Since the entire experimental process includes multiple paradigms, at the end of each paradigm, the system will send an end tag to the COM port, thereby enabling the next-stage paradigm button and disabling the previous-stage paradigm button.

[0067] After the configuration is completed, the simulation system will present different intensities and types of virtual event stimuli to the subject through the scene display module (head-mounted display device), such as blizzards, ice cracks, heating failures, power outages, animal intrusions, and fires, etc., to help the subject quickly enter the corresponding mental state. Among them, the display of events is carried out in the set time sequence to enhance the subject's immersion.

[0068] It should be noted that during the entire experimental process, the operator uses the process control interface to monitor the experimental progress and ensure the smooth switching between paradigms. At the end of each paradigm, an end tag needs to be sent to the COM port, thereby enabling the next-stage paradigm button and disabling the previous-stage paradigm button.

[0069] The head-mounted display device is responsible for displaying virtual scenes, including standby scenes, cut-scenes, event test scenes, and mood soothing scenes. For example, in the cut-scene stage, the system shows the weather changes of the polar research station, then switches to the ice exploration scene, and then enters the polar biological observation scene. In the event test stage, the system will pop up judgment buttons, such as "fire" and "power outage", and the subject selects the event type by pressing the trigger key of the handle. The system enters the next stage according to the correctness of the selection. In the fault point query operation stage, the system will emit a red ray pointing to the fault point. When the ray points to the correct position, it turns green, prompting the subject to press the trigger key to eliminate the fault point.

[0070] All experimental data are transmitted to the acquisition system in real time. Noise is removed through signal processing techniques such as band-pass filtering and blind source separation to improve the quality and accuracy of EEG signals. Meanwhile, the acquisition system also screens for abnormal and outlier samples. For the possible abnormal or outlier samples in the training set, the acquisition system screens them by setting thresholds of statistical quantities such as signal-to-noise ratio and standard deviation to ensure the data quality. In addition, the acquisition system evaluates the subject labels based on various factors such as the self-check or inspection scale related to the cognitive ability of the synchronously collected subjects, the subject's age, and the closed operation time, classifies and sorts the EEG samples, and all the data will be sorted and stored in the specified file location for subsequent data analysis and archiving.

[0071] Throughout the process, the interaction system also implements an error detection and interaction feedback mechanism to ensure that each operation can be accurately executed, and at the same time feedbacks the current operation status and results to the user through the interface; this kind of feedback is crucial for ensuring that the user has a clear understanding and sense of control over the system operation. Through the above steps, the system not only provides a convenient, realistic and diverse method for simulating a closed environment, but also realizes an efficient and accurate EEG signal acquisition platform for a closed environment through an advanced mental problem feature coding scheme, providing reliable data support and technical guarantee for mental health assessment, intervention and treatment.

[0072] It should be understood that the embodiments described above are illustrative only. The embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processor can be implemented within one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and / or other electronic units designed to perform the functions described herein, or in combination therewith.

[0073] Some aspects of the present application may be executed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above-mentioned hardware or software may be referred to as a "data block", "module", "engine", "unit", "component", or "system". The processor may be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. In addition, aspects of the present application may be embodied as a computer product located on one or more computer-readable media, which includes computer-readable program code. For example, the computer-readable media may include, but is not limited to, magnetic storage devices (such as hard disks, floppy disks, magnetic tapes...), optical disks (such as compact disks CD, digital versatile disks DVD...), smart cards, and flash memory devices (such as cards, sticks, key drives...).

[0074] The computer-readable medium may include a propagated data signal having computer program code embodied therein, for example, on a baseband or as part of a carrier wave. The propagated signal may take various forms, including electromagnetic forms, optical forms, etc., or suitable combinations thereof. The computer-readable medium may be any computer-readable medium other than a computer-readable storage medium, which can communicate, propagate, or transport a program for use by connecting to an instruction execution system, apparatus, or device. The program code located on the computer-readable medium may be propagated through any appropriate medium, including radio, cable, fiber optic cable, radio frequency signal, or similar media, or any combination of the above media.

[0075] The basic concepts have been described above. Obviously, for those skilled in the art, the above invention disclosure is only an example and does not constitute a limitation to the present application. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to the present application. Such modifications, improvements, and corrections are proposed in the present application, so such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of the present application.

[0076] Meanwhile, the present application uses specific terms to describe the embodiments of the present application. Such as "one embodiment", "an embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that the "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of the present application may be appropriately combined.

[0077] In some embodiments, numbers are used to describe components and the quantity of attributes. It should be understood that such numbers used in the description of embodiments are, in some examples, modified by the modifiers "about", "approximately" or "substantially". Unless otherwise stated, "about", "approximately" or "substantially" indicate that the said numbers allow a variation of ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, and such approximate values may vary according to the characteristics required by individual embodiments. In some embodiments, the numerical parameters should consider the specified significant digits and adopt the method of retaining the general number of digits. Although the numerical ranges and parameters used in some embodiments of the present application to confirm the breadth of their scope are approximate values, in specific embodiments, the setting of such numerical values is as precise as possible within the feasible range.

Claims

1. A simulation system for a closed environment, characterized in that The simulation system includes: a three-dimensional scene reconstruction module, an environment interaction control module, a scene presentation module, and a user interaction module; wherein, The three-dimensional scene reconstruction module is used to collect relevant data of a real closed environment and reconstruct a three-dimensional virtual scene based on the relevant data of the real closed environment; The environment interaction control module is used to set different types of stimulus events and simulate a preset virtual environment for the scene performance of the three-dimensional virtual scene according to each stimulus event; The scene presentation module is used to present the preset virtual environment on a virtual interface; The user interaction module is used to collect the behavior data of the user in the preset virtual environment and adjust the state of the preset virtual environment according to the behavior data to construct a target virtual environment.

2. The simulation system according to claim 1, wherein The three-dimensional scene reconstruction module is used to collect relevant data of a real closed environment through different acquisition devices, perform feature matching on the relevant data, extract key features and align them, and construct a three-dimensional virtual scene according to the aligned features.

3. The simulation system according to claim 1, wherein The three-dimensional scene reconstruction module is used to perform virtualization adjustment according to simulation requirements, wherein the virtualization adjustment includes adjustment of physical environment parameters, adjustment of elements in the three-dimensional virtual scene, and editing and modification of virtual objects and interaction behaviors.

4. The simulation system according to claim 1, wherein The environment interaction control module is used to adjust the intensity and duration of the stimulus event according to the feedback information of the user on the stimulus event.

5. The analog system according to claim 1, characterized in that, The user interaction module is used to identify the collected behavior data, determine the operation instructions of the user on the objects in the preset virtual environment, and adjust the parameters of the objects according to the operation instructions, wherein the behavior data includes body posture information, limb movement information, and eye movement trajectory information, and the parameters include position information, rotation information, and state information of the objects.

6. The simulation system according to claim 1, wherein The user interaction module configures an event trigger button and a corresponding event effect, and is used to track the perspective position of the user according to the received selection operation of the user on the event trigger button, and display the corresponding event effect at the perspective position to adjust the display content of the target virtual environment according to the event effect.

7. An acquisition system using the analog system according to any one of claims 1 to 6, characterized in that, The acquisition system includes: a signal acquisition module, a data processing module, and a feature encoding module, wherein, The signal acquisition module is used to receive an acquisition instruction carrying the type of acquisition paradigm and acquire the electroencephalogram signals of the user in the virtual environment provided by the simulation system when performing the acquisition paradigm; The data processing module is used to screen the electroencephalogram signals to obtain target electroencephalogram signal samples; the feature encoding module is used to perform feature encoding on the target electroencephalogram signal samples obtained by performing each type of acquisition paradigm; The data processing module is used to determine the mental state evaluation result of the user under each type of acquisition paradigm according to the encoded features to generate an electroencephalogram signal dataset for analyzing the mental state of the user, wherein the mental state evaluation result includes the state type and intensity.

8. The acquisition system according to claim 7, characterized in that, The data processing module is used to label the target electroencephalogram signal samples according to the mental state evaluation result, the basic information of the user, and the closed operation time to generate an electroencephalogram signal dataset for analyzing the mental state of the user.

9. The acquisition system according to claim 7, characterized in that, The data processing module is used to determine the mental state evaluation results of each household under each type of acquisition paradigm according to the encoded features and the mental state comparison table, generate an evaluation scale, and track the changes in the mental state according to the evaluation scale and the subsequently collected EEG signals.

10. The acquisition system according to claim 7, characterized in that, The acquisition paradigms include a resting state paradigm, an attention paradigm, and a sustained attention task paradigm. The signal acquisition module is used to receive a request carrying an end tag after the end of each type of acquisition paradigm, so as to start the next stage of the acquisition paradigm and disable the current stage of the acquisition paradigm.

11. An interaction system, characterized in that, The interaction system includes a simulation system for a closed environment, an acquisition system, and a configuration system, where The configuration system is used to configure relevant data according to experimental requirements, where the relevant data includes different types of acquisition paradigms, stimulus source event types and intensity levels, and transmission configuration information of the acquisition system; The simulation system for the closed environment is used to simulate a preset virtual environment according to the relevant data of the real closed environment collected and the stimulus source events, and collect the behavior data of the user in the preset virtual environment to adjust the state of the preset virtual environment to construct a target virtual environment; The acquisition system is used to receive an acquisition instruction carrying the type of acquisition paradigm issued by the configuration system, collect the EEG signals of the user in the virtual environment provided by the simulation system when performing the acquisition paradigm, and perform screening to obtain target EEG signal samples, and perform feature encoding on the target EEG signal samples obtained by performing each type of acquisition paradigm to generate an EEG signal dataset for analyzing the mental state of the user.

Images