Mental health state assessment method and assessment device based on brain-computer interface and eye movement tracking, computer storage medium and electronic equipment

Abstract

The invention discloses a mental health state assessment method and assessment device based on a brain-computer interface and eye movement tracking, a computer storage medium and electronic equipment. The method comprises the steps that eye movement data parameters and electroencephalogram signals of a target user are acquired; judging the emotional state of the target user based on the eye movement data parameters, and judging the pressure state of the target user based on the electroencephalogram signals; fusing the emotional state and the pressure state based on a preset algorithm, and outputting a health state score; and evaluating the mental health state of the target user based on the health state score. Therefore, according to the method, the mental state of the target user is evaluated according to the eye movement data parameters and the electroencephalogram signals obtained in the virtual reality scene, and the mental state evaluation accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of mental assessment technology, and in particular to a method for assessing mental health status based on brain-computer interface and eye tracking, a device for assessing mental health status based on brain-computer interface and eye tracking, a computer-readable storage medium, and an electronic device. Background Art

[0002] Mental health status assessment refers to a systematic evaluation process of an individual's psychological functions and emotional state. Its purpose is to help users objectively understand their mental health status. By collecting and analyzing the individual's subjective experience and behavioral manifestations, it provides their mental state assessment results. This not only helps in the early detection and intervention of psychological problems, but also provides a scientific basis for the formulation of personalized treatment plans, thereby improving treatment outcomes and quality of life.

[0003] In the related art, the health status of participants is assessed by professional evaluation and psychological assessment scales. However, the evaluation results of this evaluation method depend on the professional knowledge and experience of professionals and the accuracy of the participants' self-assessment, and there is a great deal of uncertainty. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the first purpose of the present application is to propose a method for evaluating mental health status based on brain-computer interface and eye tracking, which evaluates the mental state of the target user according to the eye movement data parameters and EEG signals obtained in a virtual reality scene, thereby improving the accuracy of mental state evaluation.

[0005] The second purpose of this application is to propose a mental health status assessment device based on brain-computer interface and eye tracking.

[0006] The third object of the present application is to provide a computer-readable storage medium.

[0007] The fourth objective of the present application is to provide an electronic device.

[0008] To achieve the above-mentioned objectives, the first aspect of the present application proposes a method for assessing mental health status based on brain-computer interface and eye tracking, the method comprising: obtaining eye movement data parameters and EEG signals of the target user; determining the emotional state of the target user based on the eye movement data parameters, and determining the stress state of the target user based on the EEG signals; fusing the emotional state and stress state based on a preset algorithm, and outputting a health status score; and assessing the mental health status of the target user based on the health status score.

[0009] According to the mental health status assessment method based on brain-computer interface and eye tracking in the embodiment of the present application, first, the eye movement data parameters and EEG signals of the target user are obtained, the emotional state of the target user is determined based on the eye movement data parameters, and the stress state of the target user is determined based on the EEG signals, then, the emotional state and stress state are integrated based on a preset algorithm, a health status score is output, and the mental health status of the target user is assessed based on the health status score. Thus, the method assesses the mental state of the target user based on the eye movement data parameters and EEG signals obtained in a virtual reality scene, thereby improving the accuracy of mental state assessment.

[0010] In addition, the mental health status assessment method based on brain-computer interface and eye tracking according to the above embodiment of the present application may also have the following additional technical features:

[0011] According to one embodiment of the present application, the mental health status of the target user is evaluated based on the health status score, including: determining the score interval according to the health status score, and determining the mental health status level corresponding to the score interval, that is, the mental health status of the target user.

[0012] According to one embodiment of the present application, the eye movement data parameters include at least one of gaze time, glance speed and pupil diameter, wherein the emotional state of the target user is determined based on the eye movement data parameters, including: when the gaze time is greater than or equal to a preset time threshold, or the glance speed is less than or equal to a preset speed threshold, or the pupil diameter is greater than or equal to a preset diameter threshold, determining that the emotional state of the target user is a positive emotion; when the gaze time is less than the preset time threshold, the glance speed is greater than the preset speed threshold, and the pupil diameter is less than the preset diameter threshold, determining that the emotional state of the target user is a negative emotion.

[0013] According to one embodiment of the present application, the EEG signal includes at least one of a change in an EEG spectrum and an activity intensity of a specific brain area, and the activity intensity of the specific brain area includes an alpha wave of the prefrontal EEG. wherein determining the stress state of the target user based on the EEG signal includes: determining that the target user is in a stressed state when the change in the EEG spectrum is a beta wave, or the alpha wave of the prefrontal EEG is asymmetric and the right side of the prefrontal lobe is more dominant than the left side; determining that the target user is in a relaxed state when the change in the EEG spectrum is an alpha wave, or the alpha wave of the prefrontal EEG is symmetric, or the alpha wave of the prefrontal EEG is asymmetric and the left side of the prefrontal lobe is more dominant than the right side.

[0014] According to one embodiment of the present application, eye movement data parameters include at least one of blinking frequency, fixation time, scanning speed and pupil diameter, and EEG signals include at least one of changes in EEG spectrum and activity intensity of specific brain areas. The mental health status assessment method based on brain-computer interface and eye movement tracking includes: using the principal component analysis method to perform dimensionality reduction extraction on the eye movement data parameters and EEG signals respectively to obtain the extracted eye movement features and EEG features; inputting the eye movement features into a deep learning network, and concatenating the neuron hidden layer states obtained by the first layer of the deep learning network with the EEG features and inputting them into the second layer of the deep learning network for fusion to obtain fused features; inputting the fused features into the fully connected layer of the deep learning network to obtain a health status score.

[0015] According to one embodiment of the present application, the mental health status assessment method based on brain-computer interface and eye tracking also includes: when the health status score is less than a first preset score threshold, displaying the target user's mental health status as abnormal; when the health status score is greater than a second preset score threshold, displaying the target user's mental health status as normal.

[0016] According to one embodiment of the present application, the mental health status assessment method based on brain-computer interface and eye tracking also includes: displaying multiple virtual reality relaxation scenes for target users to select.

[0017] To achieve the above-mentioned objectives, the second embodiment of the present application proposes a mental health status assessment device based on brain-computer interface and eye tracking, the device comprising: an acquisition module, used to obtain eye movement data parameters and EEG signals of the target user; a determination module, used to determine the emotional state of the target user based on the eye movement data parameters, and to determine the stress state of the target user based on the EEG signals; a prediction module, used to fuse the emotional state and the stress state based on a preset algorithm, and output a health status score; an evaluation module, used to evaluate the mental health status of the target user based on the health status score.

[0018] According to the mental health status assessment device based on brain-computer interface and eye tracking in the embodiment of the present application, the acquisition module acquires the eye movement data parameters and EEG signals of the target user, the determination module determines the emotional state of the target user based on the eye movement data parameters, and determines the stress state of the target user based on the EEG signals, the prediction module fuses the emotional state and the stress state based on the preset algorithm, outputs the health status score, and the assessment module assesses the mental health status of the target user based on the health status score. Thus, the device assesses the mental state of the target user based on the eye movement data parameters and EEG signals acquired in the virtual reality scene, thereby improving the accuracy of the mental state assessment.

[0019] To achieve the above-mentioned objectives, the third aspect embodiment of the present application proposes a computer-readable storage medium, on which a mental health status assessment program based on brain-computer interface and eye tracking is stored. When the mental health status assessment program based on brain-computer interface and eye tracking is executed by a processor, the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking is implemented.

[0020] According to the computer-readable storage medium of the embodiment of the present application, when the mental health status assessment program based on brain-computer interface and eye tracking stored thereon is executed by the processor, the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking is implemented. Based on the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking, the mental state of the target user is assessed according to the eye movement data parameters and electroencephalogram signals obtained in the virtual reality scene, thereby improving the accuracy of the mental state assessment.

[0021] To achieve the above-mentioned objectives, the fourth aspect embodiment of the present application proposes an electronic device, comprising: a memory, a processor, and a mental health status assessment program based on brain-computer interface and eye tracking stored in the memory and executable on the processor. When the processor executes the mental health status assessment program based on brain-computer interface and eye tracking, the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking is implemented.

[0022] According to the electronic device of the embodiment of the present application, when the processor executes the mental health status assessment program based on brain-computer interface and eye tracking, the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking is implemented. Based on the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking, the mental state of the target user is assessed according to the eye movement data parameters and EEG signals obtained in the virtual reality scenario, thereby improving the accuracy of the mental state assessment.

[0023] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of a method for assessing mental health status based on brain-computer interface and eye tracking according to an embodiment of the present application;

[0025] Figure 2 It is a flowchart of a method for assessing mental health status based on brain-computer interface and eye tracking according to a specific embodiment of the present application;

[0026] Figure 3 This is a connection diagram of a mental health status assessment device based on brain-computer interface and eye tracking according to an embodiment of the present application;

[0027] Figure 4 Schematic diagram of a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0029] The following describes, with reference to the accompanying drawings, a method for assessing mental health status based on brain-computer interface and eye tracking, a device for assessing mental health status based on brain-computer interface and eye tracking, a computer-readable storage medium, and an electronic device proposed in an embodiment of the present application.

[0030] Figure 1 This is a flowchart of a method for assessing mental health status based on brain-computer interface and eye tracking according to an embodiment of the present application.

[0031] like Figure 1 As shown, the mental health status assessment method based on brain-computer interface and eye tracking in the embodiment of the present application may include:

[0032] S1, obtain the target user's eye movement data parameters and EEG signals;

[0033] S2, determining the target user's emotional state based on eye movement data parameters, and determining the target user's stress state based on EEG signals;

[0034] S3, integrates the emotional state and stress state based on a preset algorithm and outputs a health status score;

[0035] S4, assessing the mental health status of the target user based on the health status score.

[0036] Specifically, the mental health status assessment method based on brain-computer interface and eye tracking is applied to a virtual reality scene or a real scene, without any specific limitation. Taking the mental health status assessment method based on brain-computer interface and eye tracking applied in a virtual reality scene as an example, the method is described in detail.

[0037] First, a virtual reality scene library is pre-set based on virtual reality technology, and target users can select corresponding virtual reality scenes according to their assessment needs to conduct mental health status assessment.

[0038] During the mental health status assessment process, the target user wears a VR (Virtual Reality) headset, which can be integrated with an eye movement acquisition unit and an EEG signal acquisition unit to synchronously collect the user's eye movement data parameters and EEG signals in the virtual reality scene.

[0039] For example, the eye movement acquisition unit uses a high-precision eye tracking device to capture the user's eye movement, gaze point, scan and other eye movement data parameters in real time. The eye movement data parameters correspond to the user's psychological state such as the degree of concentration and emotional changes, so as to determine the user's emotional state based on the eye movement data parameters. For example, based on the gaze time and gaze point distribution, the user's area of ​​interest and the target user's concentration on a specific task can be identified; based on the eye movement pattern and gaze distribution, the user's emotional state can be evaluated, for example, the target user's emotions can be judged by the gaze duration and pupil diameter.

[0040] The EEG signal acquisition unit collects the user's EEG signals through a wearable EEG device, and uses signal processing technology to extract stress-related features (such as changes in the EEG spectrum, the activity intensity of specific brain regions, etc.) from the EEG signals, so as to judge the stress state of the target user based on the extracted features. Exemplarily, the power spectrum density of different frequency bands of EEG waves corresponds to different stress states, for example, beta waves (13-30Hz) are usually associated with alertness and concentration, while alpha waves (8-13Hz) are associated with a relaxed state; the user's stress level is evaluated by analyzing the power spectrum density and the activities of specific brain regions (such as the asymmetry of alpha waves in the prefrontal EEG).

[0041] The two indicators of emotion and stress are integrated through a preset algorithm to output the health status score of the target user. For example, the weight values ​​corresponding to the emotional state and the stress state can be preset, and the health status is scored based on the corresponding weight values. Among them, the preset algorithm can be a neural network model obtained based on training. Then, the mental health status of the target user is evaluated according to the health status score. For example, the corresponding relationship between the health status score and the mental health status can be preset. After obtaining the health status score, the mental health status of the target user is determined by calling the preset relationship.

[0042] This embodiment evaluates the mental health status of the target user based on eye movement data parameters and EEG signals, adopts objective parameters to evaluate the mental health status, thereby improving the accuracy of the mental health status assessment, and at the same time, the application of virtual reality scenarios based on brain-computer interfaces and eye movement tracking improves the application flexibility of the mental health status.

[0043] In one embodiment of the present application, the mental health status of the target user is evaluated based on the health status score, including: determining the score interval according to the health status score, and determining the mental health status level corresponding to the score interval, that is, the mental health status of the target user.

[0044] That is to say, multiple mental health states and the scoring intervals corresponding to each mental health state are pre-set, and the mapping relationship is stored. After the health status score of the target user is obtained based on the eye movement data parameters and the EEG signal, the stored mapping relationship is called, and the mental health state corresponding to the health status score interval is used as the mental health state of the target user.

[0045] In one embodiment of the present application, the eye movement data parameters include at least one of gaze time, glance speed and pupil diameter, wherein the emotional state of the target user is determined based on the eye movement data parameters, including: when the gaze time is greater than or equal to a preset time threshold, or the glance speed is less than or equal to a preset speed threshold, or the pupil diameter is greater than or equal to a preset diameter threshold, determining that the emotional state of the target user is a positive emotion; when the gaze time is less than the preset time threshold, the glance speed is greater than the preset speed threshold, and the pupil diameter is less than the preset diameter threshold, determining that the emotional state of the target user is a negative emotion.

[0046] Specifically, positive emotions refer to a pleasant feeling that makes people excited, hopeful and optimistic. This emotion can enhance people's self-confidence, make people live and work in a happy mood, and maintain a state of happiness and pleasure. For example, happiness, interest, satisfaction, love, pride and gratitude.

[0047] Negative emotions refer to emotions that are not conducive to continuing work or normal thinking due to external or internal factors in a certain behavior, such as sadness, grief, anger, tension, anxiety, pain, fear and hatred.

[0048] When people are interested in a stimulus or feel positive emotions, they tend to stare at the stimulus for a longer time. Therefore, this concentration of attention reflects the individual's positive evaluation and interest in the stimulus. The longer the gaze time, the more positive the emotion. In addition, there is a certain correlation between eye movement speed and emotional state. When individuals are in a positive emotional state, they process the stimulus more carefully and deeply, resulting in a slower scanning speed. In addition, the change in pupil diameter is also related to the emotional state. When individuals experience positive emotions, it can cause pupil dilation and the pupil diameter tends to increase, while negative emotions may cause pupil constriction. Therefore, the positive and negative emotions of the target user are divided based on the gaze time, the scanning speed and the pupil diameter. Specifically, the target user's emotional state is judged to be positive when the gaze time is long; the target user's emotional state is judged to be positive when the scanning speed is slow; the target user's emotional state is judged to be positive when the pupil diameter is large; when the gaze time is less than the preset time threshold, the scanning speed is greater than the preset speed threshold, and the pupil diameter is less than the preset diameter threshold, the target user is judged to be negative.

[0049] In one embodiment of the present application, the EEG signal includes at least one of a change in an EEG spectrum and an activity intensity of a specific brain area, and the activity intensity of the specific brain area includes an alpha wave of the prefrontal EEG. wherein determining the stress state of the target user based on the EEG signal includes: determining that the target user is in a stressed state when the change in the EEG spectrum is a beta wave, or the alpha wave of the prefrontal EEG is asymmetric and the right side of the prefrontal lobe is more dominant than the left side; determining that the target user is in a relaxed state when the change in the EEG spectrum is an alpha wave, or the alpha wave of the prefrontal EEG is symmetric, or the alpha wave of the prefrontal EEG is asymmetric and the left side of the prefrontal lobe is more dominant than the right side.

[0050] Specifically, the stress state of the target user can be judged according to the power density of different frequency bands of brain waves. For example, when the brain wave spectrum changes to beta waves (13-30Hz), since beta waves are related to alertness and concentration, the target user is judged to be in a state of stress; when the brain wave spectrum changes to alpha waves (8-13Hz), since alpha waves are related to a state of relaxation, the target user is judged to be in a state of relaxation.

[0051] In addition, the user's stress level can also be assessed based on the power spectral density of a specific brain region. For example, the power spectral density of the alpha wave corresponding to the frequency of the prefrontal brain region is used to measure the activity intensity of the brain region. When the alpha wave of the prefrontal EEG is asymmetric and the right side of the prefrontal lobe is more dominant than the left side, the target user is judged to be in a state of stress; when the alpha wave of the prefrontal EEG is asymmetric and the left side of the prefrontal lobe is more dominant than the right side, the target user is judged to be in a state of relaxation. Among them, the alpha wave asymmetry of the prefrontal EEG refers to the phenomenon that there is a difference in the alpha wave of the left and right frontal lobe EEG when performing certain cognitive tasks. This phenomenon reflects the functional division of the left and right hemispheres of the brain when processing specific information. In terms of emotional regulation, the activity of the left frontal lobe is related to positive emotional states, while the activity of the right frontal lobe is related to negative emotional states. Therefore, when the alpha waves of the prefrontal EEG are asymmetric and the right side of the prefrontal lobe is more dominant than the left side, it can be considered that the target user is in a stressed state; when the alpha waves of the prefrontal EEG are symmetric, or when the alpha waves of the prefrontal EEG are asymmetric and the left side of the prefrontal lobe is more dominant than the right side, it can be considered that the target user is in a relaxed state.

[0052] In one embodiment of the present application, the preset algorithm includes but is not limited to one of weighted average, principal component analysis and deep learning network.

[0053] That is to say, the two indicators of emotion and stress are fused using methods such as weighted averaging, principal component analysis, and deep learning networks, so as to fuse the eye movement data parameters reflecting the emotional state and the EEG signals reflecting the stress state, and use the fused final data to evaluate the user's mental health state. This embodiment learns from multiple data sources, reduces the noise and bias that may be caused by a single data source, and improves the accuracy and robustness of the evaluation.

[0054] In one embodiment of the present application, eye movement data parameters include at least one of blinking frequency, fixation time, scanning speed and pupil diameter, and EEG signals include at least one of changes in EEG spectrum and activity intensity of specific brain areas. The mental health status assessment method based on brain-computer interface and eye movement tracking includes: using the principal component analysis method to perform dimensionality reduction extraction on the eye movement data parameters and EEG signals respectively to obtain the extracted eye movement features and EEG features; inputting the eye movement features into a deep learning network, and concatenating the hidden layer states of neurons obtained by the first layer of the deep learning network with the EEG features and inputting them into the second layer of the deep learning network for fusion to obtain fused features; inputting the fused features into the fully connected layer of the deep learning network to obtain a health status score.

[0055] Specifically, the data collected from the target user includes eye movement data parameters (such as fixation time and scanning speed) and EEG parameters (such as the power spectrum density of beta and alpha waves). First, the principal component analysis (PCA) algorithm is used to perform dimensionality reduction extraction on the eye movement data parameters and EEG signals to reduce the dimension of the data and extract the features that best represent the emotions and stress states, that is, to obtain the extracted eye movement features and EEG features.

[0056] Then, a deep learning network, such as Convolutional Neural Networks (CNN) or Recurrent Neural Networks (RNN), is used to learn the complex relationship between eye movement and EEG data and predict the user's emotions and stress state. Specifically, first, the extracted eye movement features are input into the first layer of the deep learning network to obtain the neuron hidden layer state, that is, the user's emotional state; then, the neuron hidden layer state and the EEG features are spliced ​​and input into the second layer of the deep learning network for fusion. For example, in the second layer of the deep learning network, the user's stress state can be obtained based on the EEG features, and then the stress state and the emotional state are feature fused to obtain the fused features; the fused features are input into the fully connected layer to obtain the final prediction score, that is, the output health status score. In this way, the eye movement data parameters and EEG signals can be comprehensively considered to provide more accurate evaluation results.

[0057] This method can comprehensively consider the eye movement data parameters and EEG features to give a more accurate evaluation result. Classifiers trained based on machine learning models such as support vector machines and multi-layer perceptrons are used to calculate the user's health status score based on the features of eye movement and EEG data, thereby evaluating the user's mental health status, and using feature selection methods to select eye movement and EEG signal features that are more important for evaluating mental health status. For example, after dimensionality reduction and feature extraction of user data, the feature components with the highest weights are the "features that are more important for evaluating mental health status" of the user. This process has user differences and requires training of user data to obtain a model that is "customized" for a certain user and has the best judgment effect.

[0058] In one embodiment of the present application, the mental health status assessment method based on brain-computer interface and eye tracking also includes: when the health status score is less than a first preset score threshold, displaying the target user's mental health status as abnormal; when the health status score is greater than a second preset score threshold, displaying the target user's mental health status as normal.

[0059] In other words, the health status score is positively correlated with the user's mental health status. The higher the health status score, the more normal the target user's mental health status is; the lower the health status score, the more abnormal the target user's mental health status is.

[0060] This embodiment uses a first preset scoring threshold and a second preset scoring threshold as the criteria for judging the mental health status of the target user. Specifically, when the health status score of the target user is less than the first preset scoring threshold, the mental health status of the target user is considered abnormal, and the target user is informed on the evaluation result display interface that he is in an abnormal mental health state, such as a stress state. When the health status score of the target user is greater than the second preset scoring threshold, the mental health status of the target user is determined to be normal, and the target user is informed on the evaluation result display interface that his mental health status is normal / relaxed.

[0061] In one embodiment of the present application, the mental health status assessment method based on brain-computer interface and eye tracking also includes: displaying multiple virtual relaxation scenes for target users to select.

[0062] Specifically, when the health status score is lower than the first preset score threshold, the evaluation result display interface informs the user that he is in a stressful state, i.e., an abnormal mental health state, and provides the user with multiple VR relaxation scenes for relaxation through the interface. The user can choose which relaxation scene to use, and different scenes are in a peer relationship. Thus, relaxation adjustment can be performed by providing the user with interactive VR scenes.

[0063] In addition, when the health status score is higher than the first preset score threshold, the evaluation result display interface informs the user that his / her status is normal / relaxed, that is, normal mental health status, and also provides the user with the option of using VR scenes to relax. That is, if the target user believes that there is a need for relaxation at the moment, he / she can also choose a scene to relax.

[0064] Furthermore, the multiple virtual reality relaxation scenes displayed may correspond to the health status score or assessed mental health status of the target user, so as to provide virtual reality adjustment scenes adapted to different mental states, thereby achieving relaxation needs that match the target user.

[0065] For target users with low health status scores (health status scores lower than the first preset score threshold), after relaxation adjustment through the virtual reality relaxation scene, the mental health status of the target users can be evaluated again, and a feedback report of "adjustment is effective" or "adjustment is ineffective and realistic adjustment method suggestions / medical advice" can be provided to the target users.

[0066] As a specific embodiment of the present application, Figure 2As shown, the method for assessing mental health status based on brain-computer interface and eye tracking may include the following steps:

[0067] S101, obtaining eye movement data parameters of a target user.

[0068] S102: Determine the emotional state of the target user based on the eye movement data parameters.

[0069] S103, obtaining an electroencephalogram signal of a target user.

[0070] S104: Determine the stress state of the target user based on the EEG signal.

[0071] S105, integrating the emotional state and the stress state based on a preset algorithm, and outputting a health status score.

[0072] S106, determining whether the health status score is less than a first preset score threshold. If yes, executing step S107; if no, executing step S109.

[0073] S107, displaying that the mental health status of the target user is abnormal.

[0074] S108, displaying multiple virtual reality relaxation scenes for the target user to select.

[0075] Through interactive VR scenes, users with low evaluation scores can be relaxed and adjusted. After the adjustment, the user's mental health status can be evaluated again, and feedback reports of "effective adjustment" or "ineffective adjustment and realistic adjustment method suggestions / medical advice" can be provided to the user.

[0076] S109, displaying that the mental health status of the target user is normal. In addition, further status division can also be performed by setting a threshold.

[0077] This embodiment uses eye movement and EEG technology to integrate indicators to evaluate the mental health status of the target user, and provides a variety of virtual reality scenarios for adjusting the user's mental health status based on the health status. It also supports secondary evaluation after use, and ultimately provides the user with adjustment effects and medical advice or more personalized services.

[0078] In summary, according to the mental health status assessment method based on brain-computer interface and eye tracking in the embodiment of the present application, first, the eye movement data parameters and EEG signals of the target user are obtained, the emotional state of the target user is determined based on the eye movement data parameters, and the stress state of the target user is determined based on the EEG signals, and then, the emotional state and stress state are integrated based on the preset algorithm, and the health status score is output, and the mental health status of the target user is evaluated based on the health status score. Therefore, the method evaluates the mental state of the target user based on the eye movement data parameters and EEG signals obtained in the virtual reality scene, thereby improving the accuracy of the mental state assessment.

[0079] Corresponding to the above embodiments, the present application also proposes a mental health status assessment device based on brain-computer interface and eye tracking.

[0080] like Figure 3 As shown, the mental health status assessment device based on brain-computer interface and eye tracking in an embodiment of the present application may include: an acquisition module 10, a determination module 20, a prediction module 30 and an assessment module 40.

[0081] Among them, the acquisition module 10 is used to obtain the eye movement data parameters and EEG signals of the target user. The determination module 20 is used to determine the emotional state of the target user based on the eye movement data parameters, and to determine the stress state of the target user based on the EEG signals. The prediction module 30 is used to fuse the emotional state and the stress state based on a preset algorithm and output a health status score. The evaluation module 40 is used to evaluate the mental health state of the target user based on the health status score.

[0082] According to one embodiment of the present application, the evaluation module 40 evaluates the mental health status of the target user based on the health status score, and is specifically used to: determine the score interval according to the health status score, and determine the mental health status level corresponding to the score interval, that is, the mental health status of the target user.

[0083] According to one embodiment of the present application, the eye movement data parameters include at least one of gaze time, glance speed and pupil diameter, wherein the determination module 30 determines the emotional state of the target user based on the eye movement data parameters, specifically for: when the gaze time is greater than or equal to a preset time threshold, or the glance speed is less than or equal to a preset speed threshold, or the pupil diameter is greater than or equal to a preset diameter threshold, determining that the emotional state of the target user is a positive emotion; when the gaze time is less than the preset time threshold, the glance speed is greater than the preset speed threshold, and the pupil diameter is less than the preset diameter threshold, determining that the emotional state of the target user is a negative emotion.

[0084] According to one embodiment of the present application, the EEG signal includes at least one of a change in the EEG spectrum and the activity intensity of a specific brain area, and the activity intensity of the specific brain area includes the alpha wave of the prefrontal EEG, wherein the determination module 30 determines the stress state of the target user based on the EEG signal, specifically for: determining that the target user is in a stressful state when the change in the EEG spectrum is a beta wave, or the alpha wave of the prefrontal EEG is asymmetric and the right side of the prefrontal lobe is more dominant than the left side; determining that the target user is in a relaxed state when the change in the EEG spectrum is an alpha wave, or the alpha wave of the prefrontal EEG is symmetric, or the alpha wave of the prefrontal EEG is asymmetric and the left side of the prefrontal lobe is more dominant than the right side.

[0085] According to one embodiment of the present application, the eye movement data parameters include at least one of blinking frequency, fixation time, scanning speed and pupil diameter, and the EEG signal includes at least one of changes in the EEG spectrum and activity intensity of a specific brain area. The prediction module 30 is also used to: use the principal component analysis method to perform dimensionality reduction extraction on the eye movement data parameters and the EEG signal respectively to obtain the extracted eye movement features and EEG features; input the eye movement features into the deep learning network, and concatenate the neuron hidden layer states obtained by the first layer of the deep learning network with the EEG features and input them into the second layer of the deep learning network for fusion to obtain fused features; input the fused features into the fully connected layer of the deep learning network to obtain a health status score.

[0086] According to one embodiment of the present application, the evaluation module 40 is also used to: when the health status score is less than a first preset score threshold, display the mental health status of the target user as abnormal; when the health status score is greater than a second preset score threshold, display the mental health status of the target user as normal.

[0087] According to one embodiment of the present application, the mental health status assessment device based on brain-computer interface and eye tracking also includes: a display module for displaying multiple virtual reality relaxation scenes for target users to select.

[0088] It should be noted that for details not disclosed in the mental health status assessment device based on brain-computer interface and eye tracking in the embodiment of the present application, please refer to the details disclosed in the mental health status assessment method based on brain-computer interface and eye tracking in the above embodiment of the present application, and the details will not be repeated here.

[0089] According to the mental health status assessment device based on brain-computer interface and eye tracking in the embodiment of the present application, the acquisition module acquires the eye movement data parameters and EEG signals of the target user, the determination module determines the emotional state of the target user based on the eye movement data parameters, and determines the stress state of the target user based on the EEG signals, the prediction module fuses the emotional state and the stress state based on the preset algorithm, outputs the health status score, and the assessment module assesses the mental health status of the target user based on the health status score. Thus, the device assesses the mental state of the target user based on the eye movement data parameters and EEG signals acquired in the virtual reality scene, thereby improving the accuracy of the mental state assessment.

[0090] Corresponding to the above embodiments, the present application also proposes a computer-readable storage medium.

[0091] The computer-readable storage medium of an embodiment of the present application stores a mental health status assessment program based on brain-computer interface and eye tracking. When the mental health status assessment program based on brain-computer interface and eye tracking is executed by a processor, the mental health status assessment method based on brain-computer interface and eye tracking is implemented.

[0092] According to the computer-readable storage medium of the embodiment of the present application, when the mental health status assessment program based on brain-computer interface and eye tracking stored thereon is executed by the processor, the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking is implemented. Based on the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking, the mental state of the target user is assessed according to the eye movement data parameters and EEG signals obtained in the virtual reality scene, thereby improving the accuracy of the mental state assessment.

[0093] Corresponding to the above embodiment, the present application also proposes an electronic device.

[0094] like Figure 4 As shown, the electronic device 100 of an embodiment of the present application includes: a memory 110, a processor 120, and a mental health status assessment program based on brain-computer interface and eye tracking stored in the memory 110 and executable on the processor 120. When the processor 120 executes the mental health status assessment program based on brain-computer interface and eye tracking, the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking is implemented.

[0095] According to the electronic device of the embodiment of the present application, when the processor executes the mental health status assessment program based on brain-computer interface and eye tracking, the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking is implemented. Based on the above-mentioned mental health status assessment method based on brain-computer interface and eye tracking, the mental state of the target user is assessed according to the eye movement data parameters and EEG signals obtained in the virtual reality scenario, thereby improving the accuracy of the mental state assessment.

[0096] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0097] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0098] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0099] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0100] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0101] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for assessing mental health status based on brain-computer interface and eye tracking, characterized in that: The method comprises: Obtain the target user's eye movement data parameters and EEG signals; Determining the target user's emotional state based on the eye movement data parameters, and determining the target user's stress state based on the EEG signal; The emotional state and the stress state are integrated based on a preset algorithm to output a health status score; The mental health status of the target user is evaluated based on the health status score.

2. The method for assessing mental health status based on brain-computer interface and eye tracking according to claim 1, characterized in that: Evaluating the mental health status of the target user based on the health status score includes: The scoring interval is determined according to the health status score, and the mental health status level corresponding to the scoring interval is determined, that is, the mental health status of the target user.

3. The method for assessing mental health status based on brain-computer interface and eye tracking according to claim 1, characterized in that: The eye movement data parameter includes at least one of gaze time, scanning speed and pupil diameter, wherein determining the emotional state of the target user based on the eye movement data parameter includes: When the gaze time is greater than or equal to a preset time threshold, or the glance speed is less than or equal to a preset speed threshold, or the pupil diameter is greater than or equal to a preset diameter threshold, determining that the emotional state of the target user is a positive emotion; When the gaze time is less than the preset time threshold, the scanning speed is greater than the preset speed threshold, and the pupil diameter is less than the preset diameter threshold, it is determined that the emotional state of the target user is a negative emotion.

4. The method for assessing mental health status based on brain-computer interface and eye tracking according to claim 1, characterized in that: The EEG signal includes at least one of a change in an EEG spectrum and an activity intensity of a specific brain region, wherein the activity intensity of the specific brain region includes an alpha wave of a prefrontal EEG, wherein determining the stress state of the target user based on the EEG signal includes: When the change of the brain wave spectrum is beta wave, or the alpha wave of the prefrontal lobe brain wave is asymmetric and the right side of the prefrontal lobe is more dominant than the left side, it is determined that the target user is in a stressful state; When the change of the brain wave spectrum appears to be alpha wave, or the alpha wave of the prefrontal electroencephalogram is symmetrical, or the alpha wave of the prefrontal electroencephalogram is asymmetric and the left side of the prefrontal lobe is more dominant than the right side, it is determined that the target user is in a relaxed state.

5. The method for assessing mental health status based on brain-computer interface and eye tracking according to claim 1, characterized in that: The eye movement data parameter includes at least one of blink frequency, fixation time, scanning speed and pupil diameter, the EEG signal includes at least one of changes in EEG spectrum and activity intensity of a specific brain area, and the method includes: Using principal component analysis method to perform dimensionality reduction extraction on the eye movement data parameters and the EEG signal respectively, so as to obtain extracted eye movement features and EEG features; Input the eye movement feature into a deep learning network, and splice the neuron hidden layer state obtained by the first layer network in the deep learning network with the EEG feature and input it into the second layer network of the deep learning network for fusion to obtain a fusion feature; The fused features are input into the fully connected layer of the deep learning network to obtain the health status score.

6. The method for assessing mental health status based on brain-computer interface and eye tracking according to any one of claims 1 to 5, characterized in that: The method further comprises: In the case where the health status score is less than a first preset score threshold, displaying that the mental health status of the target user is abnormal; When the health status score is greater than a second preset score threshold, the mental health status of the target user is displayed as normal.

7. The method for assessing mental health status based on brain-computer interface and eye tracking according to claim 6, characterized in that: The method further comprises: Display multiple VR relaxation scenes for target users to choose from.

8. A device for assessing mental health status based on brain-computer interface and eye tracking, characterized in that: The device comprises: An acquisition module is used to acquire eye movement data parameters and EEG signals of the target user; A determination module, configured to determine the emotional state of the target user based on the eye movement data parameters, and determine the stress state of the target user based on the EEG signal; A prediction module, used to fuse the emotional state and the stress state based on a preset algorithm and output a health state score; An evaluation module is used to evaluate the mental health status of the target user based on the health status score.

9. A computer-readable storage medium, characterized in that: A program is stored thereon, which, when executed by a processor, implements the mental health status assessment method based on brain-computer interface and eye tracking according to any one of claims 1-7.

10. An electronic device, characterized in that: include: A memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the method for assessing mental health status based on brain-computer interface and eye tracking according to any one of claims 1 to 7 is implemented.

Images