Astronaut weightlessness training evaluation method and system based on virtual reality

Abstract

The invention relates to the technical field of virtual reality, and discloses an astronaut weightlessness training evaluation method and system based on virtual reality, and the method comprises the steps: collecting the eye movement data and vestibular response data of an astronaut in a virtual weightlessness environment; calculating an eye movement vestibular coupling degree index based on the eye movement data and the vestibular reaction data; multi-modal fusion of the eye movement data and the vestibular reaction data is realized by utilizing a self-attention mechanism; constructing an individualized adaptation model, and predicting illusion resistance of the astronaut in different weightlessness states; the performance of the astronaut in the virtual weightlessness environment is comprehensively evaluated, and the adaptive capacity of the astronaut in the actual space task is predicted; the astronaut weightlessness training comprehensive evaluation method is constructed through the multi-modal data fusion and deep learning technology, and compared with a traditional single-index evaluation method, the evaluation precision of the adaptive capacity of the astronaut in the weightlessness environment is improved.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and more specifically, it relates to a method and system for evaluating the weightlessness training of astronauts based on virtual reality. Background Art

[0002] Astronauts face visual-vestibular system conflicts caused by the weightlessness environment during space missions, which affect their spatial orientation, balance control, and operation ability. Such conflicts can lead to problems for astronauts such as spatial disorientation, unstable attitude control, and decreased operation accuracy, seriously affecting the mission execution efficiency and safety.

[0003] Existing ground training methods mainly rely on devices such as rotating chairs and tilting platforms to simulate partial weightlessness, but these methods are difficult to comprehensively simulate the complex visual-vestibular conflict situations in the space environment. Although virtual reality technology provides new possibilities for simulating weightlessness environments on the ground, the existing VR weightlessness training evaluation systems have the following technical problems: Existing evaluation systems mainly rely on the monitoring of single physiological indicators, such as eye movement data or attitude stability indicators, and cannot accurately quantify the adaptation ability and illusion resistance of astronauts in the visual-vestibular conflict environment; existing systems are difficult to identify and quantify the attention resource allocation patterns of astronauts when coping with visual-vestibular conflicts, lacking an accurate assessment of the coordinated adaptation ability between the vestibular system and the visual system; existing systems cannot provide accurate evaluations and personalized training programs for individual differences among astronauts, resulting in low training efficiency and difficulty in specifically improving the adaptation ability of astronauts.

[0004] Therefore, there is a need for a method and system that can comprehensively evaluate the adaptation ability of astronauts in the visual-vestibular conflict environment to improve the accuracy and effectiveness of astronaut weightlessness training. Summary of the Invention

[0005] The present invention provides a method and system for evaluating the weightlessness training of astronauts based on virtual reality, which solves the technical problem in related technologies that the adaptation ability of astronauts in the visual-vestibular conflict environment cannot be accurately quantified.

[0006] The present invention provides a method for evaluating the weightlessness training of astronauts based on virtual reality, including: Collecting the eye movement data and vestibular response data of astronauts in a virtual weightlessness environment; Calculating the oculovestibular coupling index based on the eye movement data and vestibular response data to quantify the adaptation rate and stability of astronauts in the visual-vestibular conflict environment; Using the self-attention mechanism to achieve multimodal fusion of the eye movement data and vestibular response data to analyze the attention resource allocation pattern of astronauts; Based on eye movement data, vestibular responses, the oculovestibular coupling index, classification of attention resource allocation patterns, and evaluation metrics, an individualized adaptation model is constructed to predict the illusion resistance of astronauts in different weightless states; Based on the eye movement data, vestibular responses, the oculovestibular coupling index, classification of attention resource allocation patterns, evaluation metrics, and the prediction results of the individualized adaptation model, comprehensively evaluate the performance of astronauts in a virtual weightless environment and predict their adaptation ability in actual space missions.

[0007] Furthermore, the collection of eye movement data and vestibular response data of astronauts in a virtual weightless environment includes: Collect eye movement parameters through an eye tracking device integrated in a virtual reality headset, including the eye movement trajectory, fixation point distribution, and pupil size change; Collect the three-axis linear acceleration data, three-axis angular velocity data, and head pose data of the astronaut's head through an inertial sensor fixed on a head-mounted virtual reality device; Use timestamp alignment technology to ensure the precise synchronization of eye movement data and vestibular response data.

[0008] Furthermore, the calculation of the oculovestibular coupling index based on eye movement data and vestibular response data includes: Extract eye movement feature vectors from the eye movement data, including saccade velocity, fixation duration, fixation point distribution density, and pupil diameter change rate; Extract vestibular feature vectors from the vestibular response data, including head angular velocity, linear acceleration, and attitude change rate; Calculate the oculovestibular coupling index based on the extracted feature vectors, and draw an adaptation ability curve according to the coupling index values within a continuous time window, and extract the initial adaptation rate, steady-state coupling value, adaptation time, and fluctuation amplitude.

[0009] Furthermore, the implementation of multi-modal fusion of eye movement data and vestibular response data using the self-attention mechanism includes: Convert the eye movement data and vestibular response data into feature matrices, and extract temporal features, frequency domain features, and spatial features; Apply the self-attention mechanism to calculate the attention weight matrix to identify the important relationships between different features; Capture the temporal dependence relationship of multi-modal data based on a bidirectional long short-term memory network; Identify the attention resource allocation pattern of the astronaut by analyzing the self-attention weight matrix and the output of the bidirectional long short-term memory network.

[0010] Furthermore, the attention resource allocation patterns include visual preference type, vestibular preference type, balance integration type, and random switching type.

[0011] Further, the construction of the individualized adaptation model includes: Using a deep convolutional neural network to extract features from eye movement data and vestibular response data; Combining the personal information of the astronaut with multi-modal features to construct an individual feature vector through a residual network; Constructing an illusion resistance prediction model based on the double Q-network algorithm; Continuously optimizing the parameters of the individualized adaptation model based on the performance data of the astronaut during the training process.

[0012] Further, the comprehensive evaluation of the astronaut's performance in the virtual weightlessness environment includes: Calculating the comprehensive adaptation ability score of the astronaut based on multiple indicators; Dividing the adaptation ability of the astronaut into five levels from A to E according to the comprehensive adaptation ability score; Displaying the performance and training progress of the astronaut in each dimension through various visualization methods; Predicting the adaptation performance of the astronaut in the actual space mission based on the support vector regression model; Generating targeted personalized training suggestions based on the evaluation results and predictive analysis.

[0013] Further, the comprehensive adaptation ability score is calculated based on the following parameters: the steady-state value of the oculovestibular coupling degree index, the adaptation time of the astronaut, the fluctuation amplitude of the coupling degree index, the task completion score, and the attention resource allocation mode score.

[0014] Further, the virtual weightlessness environment includes: a rotating scene where visual information is inconsistent with vestibular sensations, a spatial positioning task scene under simulated weightlessness, and an in-capsule operation task scene under simulated space conditions.

[0015] The present invention provides a virtual reality-based astronaut weightlessness training evaluation system for implementing the above-mentioned virtual reality-based astronaut weightlessness training evaluation method, including: A multi-modal data acquisition module for collecting eye movement data and vestibular response data of the astronaut in the virtual weightlessness environment; An oculovestibular coupling degree calculation module for calculating the oculovestibular coupling degree index to quantify the adaptation rate and stability of the astronaut in the illusion environment; An attention resource allocation analysis module for realizing multi-modal data fusion using the self-attention mechanism and analyzing the attention resource allocation mode of the astronaut; An individualized adaptation model establishment module for realizing real-time fusion analysis of visual and vestibular data based on a deep convolutional neural network and constructing an illusion resistance prediction model using a deep reinforcement learning algorithm; The training effect evaluation and prediction module is used to evaluate the performance of astronauts in a virtual weightless environment, predict their adaptability in actual space missions, and provide personalized training suggestions.

[0016] The beneficial effects of the present invention are as follows: By using multi-modal data fusion and deep learning technologies, a comprehensive evaluation method for astronaut weightlessness training is constructed. Compared with traditional single-index evaluation methods, the evaluation accuracy of astronauts' adaptability in a weightless environment is improved; through the oculovestibular coupling index and the attention resource allocation vector, the adaptability and attention resource allocation mode of astronauts in a visual-vestibular conflict environment can be accurately quantified; Through an individualized balance ability adaptation model, it is possible to quantify the illusion resistance and adaptability of astronauts by analyzing their unique response patterns in a visual-vestibular conflict environment, providing a scientific basis for personalized training programs; Based on multi-dimensional evaluation indicators and a deep reinforcement learning model, the training parameters and difficulty can be adjusted to maximize the training effect, reduce the training time, lower resource consumption, and improve training efficiency.

[0017] Through a comprehensive evaluation index system and a space environment performance prediction model, the performance of astronauts in actual space missions can be predicted, and the prediction accuracy is higher than that of traditional evaluation methods, providing a more reliable decision-making basis for astronaut selection and mission assignment; By discovering and targeting the adaptation weaknesses of astronauts in advance for targeted training, the operation risks caused by poor adaptation of astronauts in actual space missions can be reduced. Description of the Drawings

[0018] Figure 1 It is a flowchart of a method for evaluating astronaut weightlessness training based on virtual reality in the present invention.

[0019] Figure 2 It is a flowchart of step 1 of the present invention.

[0020] Figure 3 It is a flowchart of step 2 of the present invention.

[0021] Figure 4 It is a flowchart of step 3 of the present invention.

[0022] Figure 5 It is a flowchart of step 4 of the present invention.

[0023] Figure 6 It is a flowchart of step 5 of the present invention. Detailed Embodiments

[0024] Reference will now be made to example embodiments to discuss the subject matter described herein. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and thus implement the subject matter described herein. Without departing from the scope of protection of the content of this specification, changes can be made to the functions and arrangements of the elements discussed. Each example can omit, substitute, or add various processes or components as needed. Additionally, the features described in some examples can also be combined in other examples.

[0025] In at least one embodiment of the present invention, a method for evaluating the weightlessness training of astronauts based on virtual reality is disclosed, as Figures 1 to 6 shown, including the following steps: Step 1, collect the eye movement data and vestibular response data of the astronaut in the virtual weightlessness environment; In this step, a dual-channel data acquisition system for eye movement tracking and head micro-movement detection is constructed to synchronously record the eye movement trajectory, fixation point distribution, and vestibular system response data of the astronaut in a controllable visual-vestibular illusion induction scenario.

[0026] Step 1.1, construct a visual-vestibular illusion induction scenario; Create a series of scenarios specifically for inducing visual-vestibular conflicts in the VR environment, including environments where visual motion is inconsistent with body perception, gravity direction illusion scenarios, scenarios where rotation and environmental rotation alternate, etc. These scenarios simulate the perceptual conflicts that astronauts may encounter in the space weightlessness environment by precisely controlling the inconsistency between the visual flow and vestibular stimuli.

[0027] Step 1.2, configure a multi-modal data acquisition system; Integrate a high-precision eye tracker into the VR headset to collect eye movement data such as fixation point coordinates, eye movement speed, pupil size, etc.; at the same time, collect micro-movement data such as head position, angular velocity, and acceleration through the inertial measurement unit sensor integrated into the VR headset to indirectly measure the response of the vestibular system.

[0028] Step 1.3, achieve synchronous data acquisition and preprocessing; Adopt a timestamp synchronization mechanism to ensure the time alignment of the eye movement data and the head micro-movement data. The sampling frequency is set to 120Hz for the eye movement data and 1000Hz for the head micro-movement data. Denoise the collected raw data, including median filtering to remove the blinking interference in the eye movement data, wavelet transform to remove the high-frequency noise in the head micro-movement data, and perform data normalization processing.

[0029] Step 1.4, extract multi-modal feature vectors; Extract the eye movement feature vector and vestibular response feature vector from the preprocessed data. The eye movement feature vector includes features such as the sequence of fixation point coordinates, the eye movement velocity spectrum, and the fixation point dwell time distribution; the vestibular response feature vector includes features such as the head angular velocity spectrum, the acceleration change rate, and the head posture stability index.

[0030] After these multi-modal data are aligned by timestamp and preprocessed, they lay the foundation for the subsequent calculation of the eye movement-vestibular coupling degree, and can accurately characterize the visual-vestibular coordination state of astronauts in the virtual weightlessness environment.

[0031] Step 2: Calculate the eye movement-vestibular coupling degree index based on the eye movement data and vestibular response data to quantify the adaptation rate and stability of astronauts in the visual-vestibular conflict environment; This step is based on the eye movement feature vector and vestibular response feature vector obtained in Step 1, and uses the correlation analysis method to calculate the eye movement-vestibular coupling degree index to quantify the adaptation rate and stability of astronauts in the illusion environment. This step converts the multi-modal raw data collected in Step 1 into a quantifiable adaptation ability index, and establishes an evaluation basis for the coordination ability of the astronaut's visual-vestibular system.

[0032] Step 2.1: Construct a time window analysis model; Divide the collected continuous data stream into windows of a fixed time length (such as 2 seconds). Each window contains the eye movement feature vector and vestibular response feature vector synchronized in time, and the overlapping rate of adjacent windows is set to 50% to ensure the continuous analysis of the data.

[0033] Step 2.2: Calculate the eye movement-vestibular covariance matrix; For the eye movement feature vector and vestibular response feature vector in each time window, calculate their covariance , which represents the correlation between the two feature vectors. The covariance matrix calculation formula is: ; where represents the covariance between the eye movement feature vector and the vestibular response feature vector , is the number of sampling points in the time window, and represent the average values of the eye movement feature and vestibular response feature in the time window respectively, represents the -th component of the eye movement feature vector at time point , represents the -th component of the vestibular response feature vector at time point , is the sampling point index in the time window; Represents the summation symbol.

[0034] Step 2.3, calculate the oculovestibular coupling index; Based on the covariance matrix, calculate the oculovestibular coupling index, which reflects the coordination degree between eye movement and vestibular response. The calculation formula is: ; Where is the oculovestibular coupling index, is the feature dimension, representing the dimension of the feature vector, and represent the variances of eye movement features and vestibular response features respectively, represents the covariance between eye movement features and vestibular response features, represents the summation over all feature dimensions, represents the square root operation; The value range of

[0035] is [-1, 1]. The closer the value is to 1, the higher the coordination between eye movement and vestibular response. The closer the value is to -1, the higher the reverse coordination between them. The value close to 0 indicates the lack of coordination between the two. Based on the value change in the time series, construct an evaluation model for the adaptation rate and stability of astronauts. The adaptation rate is calculated by the change speed of value from low to high, that is: ; Where represents the adaptation rate, represents the derivative of with respect to time, that is, the change rate, is the time point when obvious changes start to occur, represents calculating the derivative value at the time point .

[0036] The stability is calculated by the volatility of value in the stable stage, that is: ; Where represents the stability index, is the standard deviation of value in the stable stage, representing the degree of fluctuation, is the average value of value in the stable stage, Denote the coefficient of variation, which measures relative volatility; The closer the value is to 1, the higher the stability.

[0037] After step 2, the output is: the time series of the oculovestibular coupling degree index , adaptation rate , stability index , adaptation ability curve; These indicators together constitute a quantitative assessment of the astronaut's visual vestibular system adaptation ability, providing basic data support for subsequent attention resource allocation analysis and individual adaptation model establishment. In particular, the oculovestibular coupling degree index , as the core quantitative index of the astronaut's adaptation ability, will be widely used in subsequent steps.

[0038] Step 3, use the self-attention mechanism to achieve multimodal fusion of eye movement data and vestibular response data, and analyze the attention resource allocation pattern of astronauts; This step is based on the multimodal data obtained in step 1 (eye movement feature vectors and vestibular response feature vectors and the oculovestibular coupling degree index calculated in step 2 . Use the self-attention algorithm to achieve multimodal data fusion, combine the recurrent neural network to capture the temporal feature changes, and evaluate the attention resource allocation strategy of astronauts in different weightless states. This step deeply analyzes the cognitive mechanism of astronauts' response to visual vestibular conflicts from the perspective of attention resource allocation, and is a further expansion and deepening of the coupling degree index analysis in step 2.

[0039] Step 3.1, construct a self-attention model; Introduce the self-attention algorithm to process multimodal data. The self-attention calculation formula is: ; where is the self-attention function, , , respectively represent the query matrix, key matrix, and value matrix mapped from the original input data. is 's dimension, represents the matrix multiplied by the transpose of the matrix , represents 's square root, which is used to scale the dot product result. is the normalization function, which converts the input into a probability distribution. is the value matrix, which is multiplied by the attention weights to obtain the final output; In this model, the eye movement feature vector and the vestibular response feature vector as inputs, which are mapped through a linear transformation to , , matrices: ; ; ; where , , are weight matrices to be trained, which are used to generate query, key, and value matrices respectively, represents concatenating the eye movement feature vector and the vestibular response feature vector on the feature dimension to form a joint feature representation.

[0040] Step 3.2, construct a multi-head attention model; To capture relationships at different levels, a multi-head attention model is constructed to map the input data to different subspaces: ; ; where is the multi-head attention function, , , respectively represent the th, the 1st, and the th attention heads, represents the number of attention heads (set to 8 in this embodiment), , , are the query, key, and value matrices of the th attention head respectively, represents the concatenation operation, which concatenates the outputs of all attention heads on the feature dimension, is the output mapping matrix, which is used to map the concatenated features to the desired output dimension; The multi-head attention model can capture the relationships between eye movement and vestibular response features from different subspaces and provide richer attention distribution information.

[0041] Step 3.3, integrate a bidirectional long short-term memory network; The output of the multi-head attention model is fed into the bidirectional long short-term memory network algorithm to capture the long-term dependencies in the time-series data. The Bi-LSTM consists of a forward and a backward LSTM layer, which can consider the context information of both the past and the future simultaneously. The core calculations of the LSTM unit include: ; ; ; ; ; ; where , , represent the forget gate, input gate, and output gate respectively, controlling the flow of information, represents the cell state, storing long-term memory, represents the candidate cell state, i.e., the new information that may be added to the cell state at the current time step, represents the cell state at the previous time step, represents the hidden state, serving as the output at the current time step, represents the hidden state at the previous time step, represents the input data at the current time step, , , , represent the weight matrices of the forget gate, input gate, cell state, and output gate respectively, corresponding to the parameters of each gate, , , , represent the bias terms of the forget gate, input gate, cell state, and output gate respectively, represents concatenating the hidden state at the previous time step with the current input , represents the sigmoid activation function, with an output range of [0,1], represents the hyperbolic tangent activation function, with an output range of [-1,1]; represents the current time step index.

[0042] The output of the bidirectional LSTM is obtained by concatenating the forward and backward hidden states: ; where is the hidden state output of the Bi-LSTM, represents the hidden state of the forward LSTM, processing information from the past to the current, represents the hidden state of the backward LSTM, processing information from the future to the current, represents the concatenation operation in the feature dimension.

[0043] Thus, more complete temporal context information can be captured.

[0044] Step 3.4: Construct an attention resource allocation model; Based on the output of the Bi-LSTM network, construct an attention resource allocation model for quantitatively evaluating the attention allocation strategy of astronauts in a weightless environment: ; ; ; where 、 、 represent the proportions of attention resources allocated to visual tasks, vestibular balance tasks, and motor control tasks respectively. is the Softmax activation function to ensure that the sum of the three is 1, converting the original scores into a probability distribution. 、 、 are the weight matrices for visual, vestibular balance, and motor control tasks respectively, used to calculate the attention allocation scores for each task. 、 、 are the bias terms for visual, vestibular balance, and motor control tasks respectively. is the hidden state output of the Bi-LSTM, containing temporal information.

[0045] By comparing the attention resource allocation patterns of different difficulty tasks, different stages, and different individuals, identify the strategy differences and adaptation processes of astronauts in coping with visual-vestibular conflicts. The attention allocation model also calculates the following evaluation metrics: Attention switching frequency: ; where is the attention switching frequency, represents the attention allocation vector at time point , represents the total length of the sequence, represents the absolute difference between adjacent attention allocation vectors, represents the sum over all adjacent time point pairs, represents the average value.

[0046] Attention stability index (ASI): ; where is the attention stability index, is the average attention allocation vector, Represents the deviation of the attention allocation vector from the average value, Represents the square of the deviation, Represents the sum over all time points, Represents the calculation of the average value, Represents the square root operation, Represents the ratio of the standard deviation to the average value, i.e., the coefficient of variation.

[0047] Task Priority Index (TPI): ; Where is the task priority index, represents the time point of the task performance metric, represents the time point assigned to the th task's proportion of attention resources, represents the weighted product of attention resources and task performance, represents the sum over all time points, represents the weighted average proportion of attention allocation.

[0048] Output after Step 3: Time series of attention resource allocation vectors, classification of attention resource allocation patterns, attention evaluation metrics, attention weight matrix; Among them, based on the time characteristics and distribution characteristics, the attention resource allocation patterns of astronauts are divided into four categories: Visual preference type: Mainly relies on visual information for spatial orientation and balance control; Vestibular preference type: Mainly relies on vestibular sensations for spatial orientation and balance control; Balance integration type: Effectively integrates visual and vestibular information, with balanced attention resource allocation; Random switching type: Randomly switches between visual and vestibular information, lacking a stable attention allocation strategy; Attention evaluation metrics include attention switching frequency, attention stability index, and task priority index, quantifying different dimensions of astronauts' attention regulation ability; The attention weight matrix shows the importance relationship between different modal features, revealing the feature selection preferences of astronauts when dealing with visual-vestibular conflicts.

[0049] These data deeply reveal the cognitive mechanisms and strategy selections of astronauts in dealing with visual-vestibular conflicts, and together with the oculovestibular coupling index in Step 2, constitute a comprehensive assessment of astronauts' adaptability. In particular, the classification of attention resource allocation patterns provides important feature inputs for subsequent construction of individualized adaptation models, and can accurately depict the cognitive preference and adaptation strategy differences of different astronauts.

[0050] Step 4: Based on the eye movement data, vestibular responses, oculovestibular coupling index, classification of attention resource allocation patterns, and evaluation metrics, construct an individualized adaptation model to predict the illusion resistance of astronauts in different weightless states. This step integrates the core outputs of the previous three steps: the multimodal raw data (eye movement feature vectors and vestibular response feature vectors ) from Step 1, the oculovestibular coupling index from Step 2 and the classification of attention resource allocation patterns and evaluation metrics from Step 3. Based on a deep convolutional neural network, perform real-time fusion analysis of visual-vestibular data, use a deep reinforcement learning algorithm to construct an illusion resistance prediction model, and establish an individualized balance ability adaptation model for astronauts. This step further integrates the analysis results of the previous steps to construct a personalized adaptation model for each astronaut's characteristics, achieving the ability to predict the performance of astronauts in space missions.

[0051] Step 4.1: Construct a deep convolutional neural network model. Design a multi-channel deep convolutional neural network (Convolutional Neural Networks, CNN) structure to process eye movement data and vestibular response data separately. The CNN structure includes the following layers: Input layer: Receive the eye movement feature vectors and vestibular response feature vectors respectively, and reconstruct the time series data into a two-dimensional feature map form. Convolutional layer: Use multi-scale convolutional kernels (3×3, 5×5, 7×7) to extract feature patterns at different scales. The convolution operation is expressed as: ; where represents the -th feature map of the -th layer, that is, the output of the convolutional layer; represents the convolutional kernel between the -th feature map of the -th layer and the -th feature map of the -th layer, that is, the convolutional weight; represents the bias term; represents the activation function (using the ReLU function, that is, ); represents convolving and summing over all feature maps of the previous layer; represents the number of feature maps in the -th layer; Pooling layer: Use the max pooling method to reduce the dimension and retain the features. Batch normalization layer: Accelerates network convergence and improves stability; Fully connected layer: Maps the flattened feature vectors to the prediction space.

[0052] Step 4.2, Implement visual-vestibular data fusion; Adopt a feature-level fusion strategy to fuse the eye movement features and vestibular response features extracted by the CNN in the feature space: ; where represents the fused features, and respectively represent the eye movement features and vestibular response features extracted by the CNN, represents the feature concatenation operation, which connects the two features in the feature dimension, represents the fusion function (implemented by the fully connected layer), which maps the concatenated features to a unified feature space.

[0053] Step 4.3, Construct a deep reinforcement learning model; Based on the fused features , construct a deep reinforcement learning model to predict the astronauts' illusion resistance under different interference conditions. Adopt a Double Deep Q-Network (DDQN) structure, including a state space , an action space and a reward function : State space : Represented by the fused features , reflecting the current visual-vestibular coordination state of the astronaut; Action space : Represents possible resistance strategies, such as "visual dependence", "vestibular dependence", and "multimodal fusion", etc.; Reward function : Based on the performance score of the astronaut during the execution of a specific task, such as attitude stability, task completion accuracy, etc.

[0054] The Q-value update formula is: ; where represents the expected cumulative return value of executing action in state , is the learning rate, controlling the update step size, is the discount factor, determining the importance of future rewards, with a value range of [0,1]; is the immediate reward, representing the feedback obtained after executing the action, and respectively represent the current state and the actions taken, represent the execution of an action and the next state to which the transfer occurs after the action, represent the maximum Q-value that may be obtained in the next state, that is, the maximum expected return that can be obtained among all possible actions in state and the maximum expected return that can be obtained among all possible actions, represent the temporal difference error, that is, the difference between the actual return and the expected return.

[0055] Step 4.4, establish an individualized balance ability adaptation model; Based on the training results of deep reinforcement learning, construct an individualized balance ability adaptation model, which includes the following components: Adaptability scoring function: ; where represents the adaptability score of the astronaut at time point , , and respectively represent the weight coefficients of the oculovestibular coupling degree, visual attention resources, and optimal policy return, is the oculovestibular coupling degree index, represents the proportion of attention resources allocated to the visual channel; represents the maximum Q-value in the current state, reflecting the expected return of the optimal policy.

[0056] Adaptation curve fitting: Use an exponential function to fit the adaptation curve of the astronaut: ; where represents the maximum adaptability score, that is, the upper limit of the score in the long-term stable state, represents the individual adaptation rate constant, which determines the steepness of the adaptation curve, represents the exponential decay term, represents the training time.

[0057] Quantification of individual differences: By comparing the values and values of different astronauts, quantify the adaptation ability differences among individuals, and provide a basis for personalized training programs.

[0058] Output of Step 4: The individualized balance ability adaptation model, the illusion resistance prediction model, and the adaptability scoring function constitute an accurate description of the individual adaptation characteristics of astronauts, integrating the analysis results of Steps 1 to 3 into a unified individualized model, laying a foundation for the subsequent training effect evaluation and prediction. In particular, the individualized balance ability adaptation model can not only describe the current adaptation state of astronauts but also predict their performance under different training intensities and different space mission scenarios, providing the core basis for the comprehensive evaluation and personalized training suggestions in Step 5.

[0059] Step 5: Based on the eye movement data, vestibular responses, oculovestibular coupling index, classification and evaluation indicators of attention resource allocation patterns, and prediction results of the individualized adaptation model, comprehensively evaluate the performance of astronauts in the virtual weightless environment and predict their adaptation ability in actual space missions; This step comprehensively utilizes all the output results of the previous four steps: the multimodal raw data in Step 1, the oculovestibular coupling index and adaptation ability curve in Step 2, the attention resource allocation pattern and evaluation indicators in Step 3, and the individualized adaptation model and illusion resistance prediction results in Step 4 to construct a comprehensive evaluation and prediction system. According to the multidimensional performance data of astronauts in the virtual weightless environment, predict their adaptation ability in actual space missions and provide personalized training suggestions. This step is the final integration stage of the entire method, transforming all the previous analyses into practical evaluation results and training guidance.

[0060] Step 5.1: Construct a comprehensive evaluation index system; Based on the multi-dimensional evaluation indicators obtained from the previous steps, construct a comprehensive evaluation index system, including: Vestibular-visual adaptability index: Based on the time-varying characteristics of the oculovestibular coupling index calculate the vestibular-visual adaptability index: ; where represents the vestibular-visual adaptability index, is the evaluation period, i.e., the total evaluation time, represents the integral of over the entire evaluation period, i.e., the average coupling degree, is the normalization factor, is the adaptation rate, is the rate weight factor, controlling the influence degree of the adaptation rate on the total score,

[0061] Attention regulation index: Based on the dynamic change characteristics of the attention resource allocation vector evaluate the attention regulation ability of astronauts: ; where represents the attention regulation index, is the entropy value of , representing the uncertainty of attention allocation, the smaller it is, the more targeted the attention allocation is, represents the attention concentration, represents the integral of the attention concentration within the entire evaluation period, represents the normalization factor.

[0062] Illusion resistance index: The effect of the illusion resistance strategy predicted based on the deep reinforcement learning model: ; where represents the illusion resistance index, is the number of test state samples, represents the th test state, is the maximum Q value in state , representing the expected return of the optimal strategy in this state, represents the sum of the maximum Q values of all test states, represents taking the average.

[0063] Step 5.2, training effect evaluation model; Based on the comprehensive evaluation index, construct a training effect evaluation model to calculate the overall adaptability score of the astronaut : ; where represents the overall adaptability score, , and respectively represent the weight coefficients of vestibular-visual adaptability, attention regulation ability, and illusion resistance, respectively representing the importance of vestibular-visual adaptability, attention regulation ability, and illusion resistance in the total score, and are determined through statistical analysis of historical data.

[0064] Step 5.3, construct a space environment performance prediction model; Based on the comparative analysis of the ground VR training data and the existing astronaut space mission performance data, construct a performance prediction model: ; where is the predicted value of the space environment performance, representing the expected performance of the astronaut in the actual space mission, is the overall adaptability score, Training data features, including features such as the progress curve and stability during training, is the evaluation period, represents the non-linear mapping function. The Gradient Boosting Decision Tree (GBDT) model is used to implement the non-linear mapping function and the model parameters are obtained through training with historical data.

[0065] Step 5.4, generate personalized training suggestions; Based on the evaluation results and the prediction model, personalized training suggestions are automatically generated for the characteristics of different astronauts: Identification of weaknesses in adaptability: By comparing 、 and for the three indicators, identify the relative weaknesses of astronauts in visual-vestibular adaptation, attention regulation, or illusion resistance; Optimization of training intensity: Based on the adaptation curve: ; where represents the adaptability score of the astronaut at time point , represents the maximum adaptability score, represents the individual adaptation rate constant, represents the time point. Calculate the optimal training intensity increase rate : ; where represents the optimal training intensity increase rate, that is, the proportion by which the training difficulty should increase, represents the individual adaptation rate constant, represents the maximum adaptability score, is the current adaptability score, that is, the current adaptation level of the astronaut, is the correction factor used to adjust the training intensity growth rate according to individual characteristics; Personalization of training content: According to the weak index, recommend targeted training content, such as visual-vestibular conflict strengthening training, attention distribution training, or illusion environment adaptation training, etc., and configure the corresponding training parameters.

[0066] Output of Step 5: Comprehensive adaptability score, adaptability level classification, multi-dimensional performance data, space mission adaptability prediction report, and personalized training plan suggestions; Among them, the adaptability of astronauts is divided into five levels according to the TAS score: Level A (90 - 100 points): Excellent adaptability, capable of performing the most complex space missions; Level B (80 - 89 points): Excellent adaptability, suitable for most space missions; Level C (70 - 79 points): Good adaptability, requiring intensive training for weak points; Level D (60 - 69 points): Basic adaptability, requiring systematic intensive training; Level E (below 60 points): Insufficient adaptability, requiring re - evaluation of space mission adaptability.

[0067] A virtual - reality - based astronaut weightlessness training evaluation system for implementing the above - mentioned virtual - reality - based astronaut weightlessness training evaluation method, comprising: A multi - modal data acquisition module for acquiring the eye movement data and vestibular response data of astronauts in a virtual weightlessness environment; An eye movement - vestibular coupling degree calculation module for calculating the eye movement - vestibular coupling degree index to quantify the adaptation rate and stability of astronauts in an illusory environment; An attention resource allocation analysis module for using the self - attention mechanism to achieve multi - modal data fusion and analyzing the attention resource allocation pattern of astronauts; An individualized adaptation model establishment module for implementing real - time fusion analysis of visual - vestibular data based on a deep convolutional neural network and constructing an illusion resistance prediction model using a deep reinforcement learning algorithm; A training effect evaluation and prediction module for evaluating the performance of astronauts in a virtual weightlessness environment, predicting their adaptability in actual space missions, and providing personalized training suggestions.

[0068] The embodiments of the present invention have been described above. However, these embodiments are not limited to the above - mentioned specific implementation manners. The above - mentioned specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of this embodiment, those of ordinary skill in the art can also make more equivalent embodiments in various forms, all of which fall within the protection scope of this embodiment.

Claims

1. A virtual reality-based astronaut weightlessness training evaluation method, characterized in that: include: Collect eye movement data and vestibular response data of astronauts in a virtual weightless environment; The oculovestibular coupling index is calculated based on the eye movement data and vestibular response data to quantify the adaptation rate and stability of astronauts in the visual-vestibular conflict environment. Use the self-attention mechanism to achieve multimodal fusion of eye movement data and vestibular response data, and analyze the astronauts' attention resource allocation pattern; Based on eye movement data and vestibular response, eye-vestibular coupling index, attention resource allocation pattern classification and evaluation indicators, an individualized adaptation model is constructed to predict astronauts' resistance to illusions under different weightlessness conditions. Based on eye movement data and vestibular responses, eye-vestibular coupling index, attention resource allocation pattern classification and evaluation indicators, and individualized adaptation model prediction results, the astronauts' performance in a virtual weightless environment is comprehensively evaluated and their adaptability in actual space missions is predicted.

2. The virtual reality-based astronaut weightlessness training evaluation method according to claim 1, characterized in that: The collecting of the eye movement data and vestibular response data of the astronauts in the virtual weightlessness environment includes: The eye tracking device integrated in the virtual reality headset collects eye movement parameters, including eye movement trajectory, gaze point distribution, and pupil size changes; The inertial sensors fixed on the head-mounted virtual reality device collect the three-axis linear acceleration data, three-axis angular velocity data and head posture data of the astronaut's head; Timestamp alignment technology is used to ensure accurate synchronization of eye movement data and vestibular response data.

3. The virtual reality-based astronaut weightlessness training evaluation method according to claim 1, characterized in that: The calculating of the eye-vestibular coupling index based on the eye movement data and the vestibular response data comprises: Extract eye movement feature vectors from eye movement data, including saccade velocity, fixation duration, fixation point distribution density and pupil diameter change rate; Extracting vestibular feature vectors from vestibular response data, including head angular velocity, linear acceleration, and posture change rate; The oculovestibular coupling index was calculated based on the extracted feature vectors, and the adaptation ability curve was drawn according to the coupling index value in the continuous time window to extract the initial adaptation rate, steady-state coupling value, adaptation time and fluctuation amplitude.

4. The virtual reality-based astronaut weightlessness training evaluation method according to claim 1, characterized in that: The method of using the self-attention mechanism to realize the multimodal fusion of eye movement data and vestibular response data includes: The eye movement data and vestibular response data are converted into feature matrices to extract temporal features, frequency domain features and spatial features; Apply the self-attention mechanism to calculate the attention weight matrix and identify the importance relationship between different features; Capturing the temporal dependencies of multimodal data based on bidirectional long short-term memory networks; By analyzing the self-attention weight matrix and the output of the bidirectional long short-term memory network, the astronauts' attention resource allocation pattern is identified.

5. The virtual reality-based astronaut weightlessness training evaluation method according to claim 4, characterized in that: The attention resource allocation patterns include visual preference type, vestibular preference type, balance integration type and random switching type.

6. The virtual reality-based astronaut weightlessness training evaluation method according to claim 1, characterized in that: The constructing of the individualized adaptation model comprises: A deep convolutional neural network is used to extract features from eye movement data and vestibular response data; Combining the astronauts’ personal information with multimodal features, the individual feature vector is constructed through the residual network; Construct an illusion resistance prediction model based on the double Q network algorithm; Based on the performance data of astronauts during training, the parameters of the individualized adaptation model are continuously optimized.

7. The virtual reality-based astronaut weightlessness training evaluation method according to claim 1, characterized in that: The comprehensive assessment of astronauts' performance in a virtual weightless environment includes: Calculate the astronauts' comprehensive adaptability score based on multiple indicators; According to the comprehensive adaptability score, the adaptability of astronauts is divided into five levels from A to E; Use multiple visualization methods to display astronauts’ performance and training progress in various dimensions; Predicting astronauts' adaptation performance in actual space missions based on support vector regression model; Generate targeted, personalized training recommendations based on assessment results and predictive analysis.

8. The virtual reality-based astronaut weightlessness training evaluation method according to claim 7, characterized in that: The comprehensive adaptability score is calculated based on the following parameters: the steady-state value of the oculovestibular coupling index, the astronaut's adaptation time, the fluctuation amplitude of the coupling index, the task completion score and the attention resource allocation pattern score.

9. The virtual reality-based astronaut weightlessness training evaluation method according to claim 1, characterized in that: The virtual weightlessness environment includes: a rotating scene in which visual information is inconsistent with vestibular sensation, a space positioning task scene simulating a weightlessness state, and a task scene simulating an operation in a space capsule.

10. A virtual reality-based astronaut weightlessness training assessment system, characterized in that: A virtual reality-based astronaut weightlessness training assessment method for executing any one of claims 1 to 9, comprising: Multimodal data acquisition module, used to collect eye movement data and vestibular response data of astronauts in a virtual weightless environment; The oculovestibular coupling calculation module is used to calculate the oculovestibular coupling index and quantify the adaptation rate and stability of astronauts in the illusion environment; The attention resource allocation analysis module is used to realize multimodal data fusion using the self-attention mechanism and analyze the attention resource allocation pattern of astronauts; Individualized adaptation model building module, which is used to realize real-time fusion analysis of visual vestibular data based on deep convolutional neural network and build an illusion resistance prediction model using deep reinforcement learning algorithm; The training effect evaluation and prediction module is used to evaluate the performance of astronauts in a virtual weightless environment, predict their adaptability in actual space missions, and provide personalized training recommendations.

Images