Eye movement feature-based emotional disorder evaluation system and method

Abstract

The invention discloses an emotional disorder assessment system and method based on eye movement features, and relates to the field of emotional disorder assessment, and the system comprises an eye movement video collection module, an eye movement feature extraction module and an emotional disorder assessment module. The eye movement video collection module collects eye movement videos when a user watches emotional disorder assessment tasks, the emotional disorder assessment tasks comprise a free watching task reflecting attention deviation degree, a staring task reflecting attention stability and anti-interference capability and an attention transfer task reflecting attention transfer efficiency, and attention characteristics of the user are reflected in multiple dimensions. The eye movement feature extraction module extracts eye movement features corresponding to each task from the eye movement video, and affection caused by subjective factors can be reduced by performing emotional disorder assessment based on objective eye movement features. The emotional disorder assessment module objectively determines an emotional disorder assessment result from multiple dimensions by using the emotional disorder assessment model and the eye movement features corresponding to the tasks, is not limited by psychological medical resources and regions, and can be applied on a large scale.

Description

Technical Field

[0001] The present invention relates to the field of emotional disorder assessment, and particularly to an emotional disorder assessment system and method based on eye movement characteristics. Background Art

[0002] Emotional disorder is a mental problem mainly including depressive disorder and anxiety disorder. At present, the methods for assessing emotional disorder mainly include doctor's inquiry and scale assessment, but both of these two methods have certain defects. Doctor's inquiry requires the doctor to communicate fully with the user. On the one hand, it is limited by the current shortage of psychological medical resources, resulting in the user's inquiry needs not being fully met; on the other hand, for users in remote areas, there is also the problem of inconvenient inquiry, and it cannot be popularized to a wider user group; moreover, the doctor's diagnosis is easily affected by factors such as their own experience and the user's expression ability, lacking objectivity. Scale assessment is easily affected by the user's subjective will, social desirability effect, and cultural differences, and cannot accurately and objectively assess the user's emotional state. And scale assessment can only reflect the user's emotional state at a specific time and cannot capture the dynamic changes of the user's emotions. Eye movement tracking technology is the most important monitoring tool for attention response and stability. Multiple empirical studies based on eye movement have found that patients with emotional disorders have specific processing mechanisms for emotional stimuli, manifested in the attention, response, and understanding of emotional stimuli. In addition, patients with emotional disorders are also different from ordinary people in terms of attention stability. These studies on cognitive processing mechanisms provide the possibility for the objective assessment of patients with emotional disorders. Summary of the Invention

[0003] The purpose of the present invention is to provide an emotional disorder assessment system and method based on eye movement characteristics, which can objectively determine the emotional disorder assessment result from multiple dimensions, is not restricted by psychological medical resources and regions, and can be widely applied.

[0004] To solve the above technical problems, the present invention provides an emotional disorder assessment system based on eye movement characteristics, including:

[0005] An eye movement video acquisition module, configured to acquire an eye movement video of a user when watching an emotional disorder assessment task; wherein, the emotional disorder assessment task includes a free viewing task for reflecting the attention bias degree of the user to content of different emotions, a fixation task for reflecting the attention stability and anti-interference ability of the user, and an attention transfer task for reflecting the attention transfer efficiency of the user between contents of different emotions;

[0006] An eye movement feature extraction module, configured to extract free viewing eye movement features from the eye movement video of the user when watching the free viewing task, extract fixation eye movement features from the eye movement video of the user when watching the fixation task, and extract attention transfer eye movement features from the eye movement video of the user when watching the attention transfer task;

[0007] An emotional disorder assessment module, configured to input the free viewing eye movement features, the fixation eye movement features, and the attention transfer eye movement features into an emotional disorder assessment model, and output an emotional disorder assessment result determined by the emotional disorder assessment model based on the free viewing eye movement features, the fixation eye movement features, and the attention transfer eye movement features.

[0008] Optionally, the free viewing task includes:

[0009] Displaying task guidance information on the screen and / or playing the task guidance information by voice, where the task guidance information is information for prompting the user to freely view a picture of a human face displayed on the screen;

[0010] After receiving a task start instruction, displaying a current task progress page on the screen and lasting for a first preset duration, after the display of the current task progress page ends, displaying an eye movement correction page on the screen and lasting for a second preset duration, and after the display of the eye movement correction page ends, displaying a current picture of a human face on the screen and lasting for a third preset duration;

[0011] After the display of the current picture of a human face ends, taking the next picture after the current picture of a human face in the collection of pictures of human faces as the new current picture of a human face, updating the current task progress page, and entering the step of displaying the current task progress page on the screen and lasting for a first preset duration until all the pictures in the collection of pictures of human faces have been displayed;

[0012] Among them, the collection of pictures of human faces includes positive-neutral face pictures and negative-neutral face pictures. The positive-neutral face pictures refer to pictures that simultaneously display positive emotion face pictures and neutral emotion face pictures on the same picture. The negative-neutral faces refer to pictures that simultaneously display negative emotion face pictures and neutral emotion face pictures on the same picture, and the positive emotion face pictures and neutral emotion face pictures in the positive-neutral face pictures are distributed according to a preset distribution rule, and the negative emotion face pictures and neutral emotion face pictures in the negative-neutral face pictures are distributed according to a preset distribution rule.

[0013] Optionally, extracting free viewing eye movement features from the eye movement video when the user views the free viewing task includes:

[0014] Extract the positive face fixation time and the number of positive face fixations when the user gazes at all the positive emotion face pictures, the negative face fixation time and the number of negative face fixations when gazing at all the negative emotion face pictures, the neutral face fixation time and the number of neutral face fixations when gazing at all the neutral emotion face pictures, and the first fixation latency corresponding to each of the positive emotion face pictures, the negative emotion face pictures, and the neutral emotion face pictures, where the first fixation latency is the time interval from when each face picture appears on the screen until the user first gazes at the face picture;

[0015] Determine the positive bias index and the negative bias index based on the positive face fixation time, the negative face fixation time, and the neutral face fixation time, where the positive bias index = positive face fixation time / (positive face fixation time + neutral face fixation time), and the negative bias index = negative face fixation time / (negative face fixation time + neutral face fixation time);

[0016] Take each face fixation time, each face fixation count, the first fixation latency corresponding to each face picture, the positive bias index, and the negative bias index as the free viewing eye movement features.

[0017] Optionally, the gaze task includes:

[0018] Display task guidance information on the screen and / or play the task guidance information by voice, where the task guidance information is information prompting the user to gaze at the marker in the center of the screen;

[0019] After receiving the task start instruction, display an eye movement correction page on the screen for a fourth preset duration, and after the display of the eye movement correction page ends, display a gaze task picture on the screen for a fifth preset duration and then end; where the center point of the gaze task picture displays the marker, and distractors are displayed around the marker.

[0020] Optionally, the extracting of the gaze eye movement features from the eye movement video when the user views the gaze task includes:

[0021] Extract the fixation hold time when the user gazes at the marker in the gaze task picture, the number of saccades during the completion of the gaze task, the average value of the saccade amplitude during the completion of the gaze task, and the average value of the gaze deviation distance of the user's fixation point from the marker;

[0022] Take the fixation hold time, the number of saccades, the average value of the saccade amplitude, and the average value of the gaze deviation distance as the gaze eye movement features.

[0023] Optionally, the attention shift task includes:

[0024] Display task guidance information on the screen and / or play the task guidance information by voice, where the task guidance information is information for prompting the user to gaze at the human face selected by the cue box;

[0025] After receiving the task start instruction, display the current task progress page on the screen for a sixth preset duration, after the display of the current task progress page ends, display the eye movement correction page on the screen for a seventh preset duration, after the display of the eye movement correction page ends, display the current attention shift test picture on the screen for an eighth preset duration, and during the display of the current attention shift test picture, control the cue box to successively select non-neutral emotion face pictures and neutral emotion face pictures in the current attention shift test picture according to a preset display rule;

[0026] After the display of the current attention shift test picture ends, use the next picture in the attention shift test picture set that is after the current attention shift test picture as the new current attention shift test picture, update the current task progress page, and enter the step of displaying the current task progress page on the screen for a seventh preset duration until all the pictures in the attention shift test picture are displayed;

[0027] Among them, the attention shift test pictures include both the non-neutral emotion face pictures and the neutral emotion face pictures, and the non-neutral emotion face pictures and the neutral emotion face pictures are distributed according to a preset distribution rule, and the non-neutral emotion face pictures are positive emotion face pictures or negative emotion face pictures.

[0028] Optionally, extracting attention shift eye movement features from the eye movement video of the user watching the attention shift task includes:

[0029] Extracting the transfer latency from the start of the cue box by the user to the start of the saccade of the user, the transfer time from the start of the saccade of the user to the user's gaze at the new emotion face picture, the fixation time of the user's gaze at each emotion face picture, the fixation times of the user's gaze at each emotion face picture, the average speed and the highest speed of the saccade of the user during the process of completing the attention shift task, and the complexity of the eye movement trajectory of the user during the process of completing the gaze transfer task;

[0030] Taking each of the transfer latencies, the transfer times, the fixation times, the fixation times, the average speed and the highest speed of the saccade, and the complexity of the eye movement trajectory as the attention shift eye movement features.

[0031] Optionally, the emotion disorder assessment system further includes:

[0032] The behavioral feature acquisition module is used to acquire and output the behavioral features of the user when watching the attention transfer task. Among them, the behavioral features include reaction time and accuracy rate. The reaction time is the time from the appearance of each cue box to the user's fixation on the emotional face picture selected by the cue box. The accuracy rate is the ratio of the number of times the user correctly fixates on the emotional face picture selected by each cue box to the total number of times each cue box selects each emotional face picture.

[0033] Optionally, the emotional disorder assessment model determines the emotional disorder assessment result based on the free viewing eye movement features, the fixation eye movement features, and the attention transfer eye movement features, including:

[0034] The emotional disorder assessment model uses the logistic regression formula determined during the training process, the free viewing eye movement features and their corresponding weights, the fixation eye movement features and their corresponding weights, and the attention transfer eye movement features and their corresponding weights to determine the logistic regression result;

[0035] Determine the prediction probability according to the logistic regression result, and map the prediction probability to the score range to obtain the score of this emotional disorder assessment of the user.

[0036] To solve the above technical problems, the present invention also provides an emotional disorder assessment method based on eye movement features, including:

[0037] Collect the eye movement video of the user when watching the emotional disorder assessment task; among them, the emotional disorder assessment task includes a free viewing task for reflecting the attention bias degree of the user to the content of different emotions, a fixation task for reflecting the attention stability and anti-interference ability of the user, and an attention transfer task for reflecting the attention transfer efficiency of the user between the content of different emotions;

[0038] Extract the free viewing eye movement features from the eye movement video of the user when watching the free viewing task, extract the fixation eye movement features from the eye movement video of the user when watching the fixation task, and extract the attention transfer eye movement features from the eye movement video of the user when watching the attention transfer task;

[0039] Input the free viewing eye movement features, the fixation eye movement features, and the attention transfer eye movement features into the emotional disorder assessment model, and output the emotional disorder assessment result determined by the emotional disorder assessment model based on the free viewing eye movement features, the fixation eye movement features, and the attention transfer eye movement features.

[0040] The beneficial effects of the present invention are to provide an emotional disorder assessment system and method based on eye movement characteristics, including an eye movement video acquisition module, an eye movement feature extraction module, and an emotional disorder assessment module. The eye movement video acquisition module acquires the eye movement video of the user when watching the emotional disorder assessment task. Among them, the emotional disorder assessment task includes a free viewing task reflecting the attention bias, a fixation task reflecting the attention stability and anti-interference ability, and an attention transfer task reflecting the attention transfer efficiency, which reflects the user's attention characteristics from multiple dimensions. The eye movement feature extraction module extracts the eye movement features corresponding to each task from the eye movement video. Subsequently, the emotional disorder assessment based on objective eye movement features can reduce the influence of subjective factors on the assessment results. The emotional disorder assessment module uses the emotional disorder assessment model and the eye movement features corresponding to each task to objectively determine the emotional disorder assessment results from multiple dimensions, and is not restricted by psychological medical resources and regions, and can be widely applied. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the prior art and the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0042] Figure 1 The first structural schematic diagram of an emotional disorder assessment system based on eye movement characteristics provided by the present invention;

[0043] Figure 2 The second structural schematic diagram of an emotional disorder assessment system based on eye movement characteristics provided by the present invention;

[0044] Figure 3 The display schematic diagram of a free viewing task provided by the present invention;

[0045] Figure 4 The display schematic diagram of a fixation task provided by the present invention;

[0046] Figure 5 The display schematic diagram of an attention transfer task provided by the present invention;

[0047] Figure 6 The system architecture diagram of an emotional disorder assessment system based on eye movement characteristics provided by the present invention;

[0048] Figure 7 The work flow chart of an emotional disorder assessment system based on eye movement characteristics provided by the present invention;

[0049] Figure 8Flowchart of an emotional disorder assessment method based on eye movement characteristics provided by the present invention. Detailed implementation manners

[0050] The core of the present invention is to provide an emotional disorder assessment system and method based on eye movement characteristics, which can objectively determine the emotional disorder assessment result from multiple dimensions, is not restricted by psychological medical resources and regions, and can be widely applied.

[0051] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0052] Please refer to Figure 1 , Figure 1 which is a schematic structural diagram of an emotional disorder assessment system based on eye movement characteristics provided by the present invention. The emotional disorder assessment system includes:

[0053] An eye movement video acquisition module 101, configured to acquire an eye movement video of a user when watching an emotional disorder assessment task; wherein, the emotional disorder assessment task includes a free viewing task for reflecting the attention bias degree of the user to contents of different emotions, a gaze task for reflecting the attention stability and anti-interference ability of the user, and an attention transfer task for reflecting the attention transfer efficiency of the user between contents of different emotions.

[0054] An eye movement feature extraction module 102, configured to extract free viewing eye movement features from the eye movement video of the user when watching the free viewing task, extract gaze eye movement features from the eye movement video of the user when watching the gaze task, and extract attention transfer eye movement features from the eye movement video of the user when watching the attention transfer task.

[0055] An emotional disorder assessment module 103, configured to input the free viewing eye movement features, the gaze eye movement features and the attention transfer eye movement features into an emotional disorder assessment model, and output an emotional disorder assessment result determined by the emotional disorder assessment model based on the free viewing eye movement features, the gaze eye movement features and the attention transfer eye movement features.

[0056] In consideration of the fact that users with emotional disorders often exhibit attention deficits such as inattention and being easily attracted by negative stimuli in addition to emotional problems such as anxiety and depression, the present invention sets up an emotional disorder assessment task that can reflect the attention characteristics of users, and this emotional disorder assessment task includes tasks that comprehensively reflect the attention characteristics of users from multiple dimensions. Specifically, the free viewing task can reflect the attention bias of users, the gaze task can reflect the attention stability and anti-interference ability of users, and the attention transfer task can reflect the attention transfer efficiency of users.

[0057] As mentioned above, users with emotional disorders are easily attracted by negative stimuli and are more likely to fixate on content with negative emotions. For example, when negative emotion pictures and neutral emotion pictures are simultaneously displayed on the screen, the time and frequency of users with emotional disorders fixating on negative emotion pictures are more than those of fixating on neutral emotion pictures. Therefore, the present invention sets up a free viewing task for reflecting the attention bias of users towards content with different emotions. When the user views the free viewing task, the eye movement video acquisition module 101 acquires the eye movement video of the user; the eye movement feature extraction module 102 extracts free viewing eye movement features that can reflect the attention bias of the user from the eye movement video corresponding to the free viewing task; and the emotional disorder assessment module 103 uses the free viewing eye movement features as a dimension during emotional disorder assessment.

[0058] Since users with emotional disorders are more likely to fixate on content with negative emotions, there are also obvious differences in their attention transfer efficiency under different emotional content. For example, when positive emotion content and neutral emotion content are simultaneously displayed on the screen, record the transfer latency and transfer time when the user moves the fixation point from the positive emotion content to the neutral emotion content; then when negative emotion content and neutral emotion content are simultaneously displayed on the screen, also record the transfer latency and transfer time when the user moves the fixation point from the negative emotion content to the neutral emotion content. For users with emotional disorders, there will be obvious differences in the transfer latency and transfer time in these two situations. Based on this, the present invention sets up an attention transfer task for reflecting the attention transfer efficiency of users between content with different emotions. The eye movement video acquisition module 101 also acquires the eye movement video of the user when viewing the attention transfer task, and then the eye movement feature extraction module 102 extracts attention transfer eye movement features that can reflect the attention transfer efficiency from the eye movement video corresponding to the attention transfer task, and uses the attention transfer eye movement features as a dimension during emotional disorder assessment.

[0059] As described above, users with emotional disorders often have problems with inattention and are more susceptible to external interference. For example, when markers and distractors are displayed on the screen simultaneously, users with emotional disorders are easily affected by the distractors, resulting in a relatively short fixation time on the markers. Therefore, the present invention sets up a fixation task for reflecting the attention stability and anti-interference ability of the user. The eye movement video acquisition module 101 acquires the eye movement video of the user when watching the fixation task, and then the eye movement feature extraction module 102 extracts the fixation eye movement features from the eye movement video corresponding to the fixation task, which can reflect the attention stability and anti-interference ability of the user, and uses the fixation eye movement features as a dimension in the evaluation of emotional disorders.

[0060] The free viewing task, the fixation task, and the attention shift task all belong to viewing tasks, but their respective presentation methods and corresponding eye movement features are different. The specific presentation methods and the specific contents of the eye movement features will be illustrated by examples later, and will not be elaborated here for the time being.

[0061] On the basis of the above steps, the emotional disorder evaluation module 103 inputs the free viewing eye movement features, the fixation eye movement features, and the attention shift eye movement features into a pre-trained emotional disorder evaluation model, so that the emotional disorder evaluation model can comprehensively and multi-dimensionally determine the emotional disorder evaluation result based on the free viewing eye movement features, the fixation eye movement features, and the attention shift eye movement features. Finally, the emotional disorder evaluation module 103 outputs the emotional disorder evaluation result, for example, pushing it to the customer's mobile terminal in a visual way. In addition, the emotional disorder evaluation model can also provide customized mental health suggestions and risk warnings for the user based on the emotional disorder evaluation result, and feedback them to the user by the emotional disorder evaluation module 103 to achieve precise intervention for emotional disorders.

[0062] The emotional disorder evaluation result in the present invention can be a numerical value (for example, the emotional disorder evaluation result is a numerical value between 0 and 10, and the larger the numerical value, the higher the risk of emotional disorder), or it can be text (for example, no obvious emotional disorder, mild emotional disorder, moderate emotional disorder, and severe emotional disorder). The present invention does not make a special limitation on the manifestation form of the finally obtained emotional disorder evaluation result.

[0063] So far, the emotional disorder assessment system based on eye movement features provided by the present invention has completed the emotional disorder assessment of the user. On the one hand, aiming at the specific attention defects of users with emotional disorders, a free viewing task reflecting the user's attention bias, a gaze task reflecting the user's attention stability and anti-interference ability, and an attention transfer task reflecting the user's attention transfer efficiency are set, expanding the dimension of emotional disorder assessment. On the other hand, it is difficult for users to guess the assessment purpose during the process of completing the emotional disorder assessment task, which solves to a certain extent the problem that users deliberately conceal to cover their true emotional state. The eye movement video of the user when watching the emotional disorder assessment task and the eye movement features extracted from the eye movement video are all objective data, and the subsequent emotional disorder assessment result determined based on the eye movement features of the user when completing the emotional disorder assessment task is also objective, ensuring the objectivity and accuracy of the emotional disorder assessment result. On the other hand, when conducting emotional disorder assessment through the emotional disorder assessment system provided by the present invention, it is not restricted by psychological medical resources and regions. Users can complete the emotional disorder assessment through a mobile terminal (including but not limited to mobile phones, tablets, etc.), expanding the application scenario.

[0064] In addition, when the eye movement video acquisition module 101 is deployed on a mobile terminal, the above-mentioned various emotional disorder assessment tasks can be displayed on the screen of the mobile terminal, and the eye movement video of the user when watching the emotional disorder assessment task can be acquired through the front camera of the mobile terminal. It can not only solve the problem of difficult medical treatment for users with inconvenient mobility or living in remote areas, as well as the problem that it is difficult for users to obtain professional assessment in a timely manner due to insufficient mental and psychological medical resources, but also overcome the problems of high equipment price, complex use, and the need for professional personnel to operate when using a traditional eye movement instrument to acquire eye movement videos. It popularizes emotional disorder assessment into daily life scenarios, fully covering users of different ages, different regions, and different social backgrounds, and improving the coverage and utilization rate of emotional disorder assessment technology.

[0065] The following will detail the presentation method of the free viewing task and the specific process of extracting free viewing eye movement features from the eye movement video corresponding to the free viewing task.

[0066] As an optional embodiment, when presenting the free viewing task, first display task guidance information on the screen and / or play task guidance information by voice. The task guidance information is information prompting the user to freely view the picture of a human face displayed on the screen.

[0067] After receiving the task start instruction, display the current task progress page on the screen and last for the first preset duration. After the display of the current task progress page ends, display the eye movement correction page on the screen and last for the second preset duration. After the display of the eye movement correction page ends, display the current picture of the human face on the screen and last for the third preset duration.

[0068] After the display of the current human face picture ends, the next picture in the human face picture collection that comes after the current human face picture is used as the new current human face picture, the current task progress page is updated, and the step of displaying the current task progress page on the screen and lasting for the first preset duration is entered until all the pictures in the human face picture collection have been displayed.

[0069] Among them, the human face picture collection includes positive-neutral face pictures and negative-neutral face pictures. The positive-neutral face picture means that a positive emotion face picture and a neutral emotion face picture are simultaneously displayed on the same picture, and the negative-neutral face means that a negative emotion face picture and a neutral emotion face picture are simultaneously displayed on the same picture. Moreover, the positive emotion face picture and the neutral emotion face picture in the positive-neutral face picture are distributed according to a preset distribution rule, and the negative emotion face picture and the neutral emotion face picture in the negative-neutral face picture are distributed according to a preset distribution rule.

[0070] Specifically, at the start of the free viewing task, task guidance information is first displayed on the screen to prompt the user about what needs to be done when completing the free viewing task, enabling the user to understand the free viewing task in advance so as to make timely responses after the task officially starts and ensure objectivity. For example, the following task guidance information is displayed on the screen: "Here is an album of human expressions, and all you need to do is freely view these human expression pictures", and the above task guidance information is played by voice.

[0071] After the user clicks the "OK" button through the touch screen, keyboard or mouse, a task start instruction triggered by the button is received, and the content in the free viewing task is officially started to be displayed. First, the current task progress page is displayed on the screen. For example, the characters X / Y are displayed on the current task progress page, where X represents the serial number of the current human face picture, and Y represents the total number of pictures included in the human face picture collection. Please refer to Figure 3 , Figure 3 FIG. is a schematic diagram of the display of a free viewing task provided by the present invention. As Figure 3 shown, the current task progress page displays 2 / 16, indicating that the serial number of the current human face picture is 2, and the human face picture collection altogether includes 16 human face pictures.

[0072] The current task progress page is continuously displayed for the first preset duration. After the display ends, a eye movement correction page is presented, and the eye movement correction page is continuously displayed for the second preset duration. Take Figure 3For example, the current task progress page is displayed for 1 second, and then a cross icon (i.e., the eye movement correction page) is presented, which is also displayed for 1 second. Then, the current person's face picture starts to be displayed. The current person's face picture is displayed for 3.5 seconds. After the display ends, the next picture in the collection of person's face pictures is used as the current person's face picture, and the current task progress page is updated. The above display process is repeated until all the pictures in the collection of person's face pictures have been displayed, and then the free viewing task ends.

[0073] In the present invention, a collection of person's face pictures is preset, which includes positive-neutral face pictures and negative-neutral face pictures. The so-called positive-neutral face picture means that on the same picture, there are both positive emotion face pictures and neutral emotion face pictures, and these two kinds of emotion face pictures are distributed according to a preset distribution rule; the negative-neutral face picture means that on the same picture, there are both negative emotion face pictures and neutral emotion face pictures, and these two kinds of emotion face pictures are distributed according to a preset distribution rule. Taking Figure 3 as an example, the total number of positive-neutral face pictures and negative-neutral face pictures included in the collection of person's face pictures is 16, and each picture is displayed for 3.5 seconds. The preset distribution rule will balance the left and right of each emotion face picture. For example, among the 8 positive-neutral face pictures, in 4 pictures, the positive emotion face picture is on the left and the neutral emotion face picture is on the right; in the other 4 pictures, the positive emotion face picture is on the right and the neutral emotion face picture is on the left. Among the 8 negative-neutral face pictures, in 4 pictures, the negative emotion face picture is on the left and the neutral emotion face picture is on the right; in the other 4 pictures, the negative emotion face picture is on the right and the neutral emotion face picture is on the left. By setting the preset distribution rule to balance the positions of each emotion face picture, the objectivity of the result can be further ensured.

[0074] On this basis, free viewing eye movement features are extracted from the eye movement video when the user views the free viewing task, including:

[0075] extracting the positive face fixation time and positive face fixation times for the user's fixation on all positive emotion face pictures, the negative face fixation time and negative face fixation times for the user's fixation on all negative emotion face pictures, the neutral face fixation time and neutral face fixation times for the user's fixation on all neutral emotion face pictures, as well as the first fixation latency corresponding to the positive emotion face pictures, negative emotion face pictures, and neutral emotion face pictures respectively, where the first fixation latency is the time interval from when each face picture appears on the screen to when the user first fixates on that face picture.

[0076] Determine the positive bias index and the negative bias index based on the fixation time on positive faces, the fixation time on negative faces, and the fixation time on neutral faces. Among them, the positive bias index = fixation time on positive faces / (fixation time on positive faces + fixation time on neutral faces), and the negative bias index = fixation time on negative faces / (fixation time on negative faces + fixation time on neutral faces).

[0077] Take the fixation time on positive faces, the number of fixations on positive faces, the fixation time on negative faces, the number of fixations on negative faces, the fixation time on neutral faces, the number of fixations on neutral faces, the positive bias index, the negative bias index, and the first fixation latency corresponding to each of the positive emotion face pictures, negative emotion face pictures, and neutral emotion face pictures as free viewing eye movement features.

[0078] It should be noted that the above fixation time on positive faces can be the sum of the fixation times of the user on all positive emotion face pictures, or the average fixation time of the user on all positive emotion face pictures. The same is true for other free viewing eye movement features, as long as the calculation methods of all free viewing eye movement features are unified. In addition, a fixation point distribution heat map of the user on the positive emotion face pictures, negative emotion face pictures, and neutral emotion face pictures can be generated and output to the mobile terminal to assist in reflecting the user's emotional disorder status.

[0079] The following will detail the presentation method of the gaze task and the specific process of extracting gaze eye movement features from the eye movement video corresponding to the gaze task.

[0080] As an optional embodiment, when presenting the gaze task, first display task guidance information on the screen and / or play the task guidance information by voice. The task guidance information is information that prompts the user to fixate on the marker in the center of the screen. After receiving the task start instruction, display an eye movement correction page on the screen for a fourth preset duration, and after the display of the eye movement correction page ends, display a gaze task picture on the screen for a fifth preset duration and then end. Among them, a marker is displayed at the center point of the gaze task picture, and distractors are displayed around the marker.

[0081] Specifically, at the beginning of the gaze task, first display task guidance information on the screen to prompt the user what to do when completing the gaze task, so that the user can understand the gaze task in advance and make timely responses after the task officially starts, ensuring objectivity. For example, at the beginning of the gaze task, first display the task guidance information on the screen: "This is an interesting gaze task. Please stare at the red dot in the center of the screen and keep your attention until the task ends. If your attention is concentrated enough, you will find that the circle around the red dot will disappear", and play the above task guidance information by voice.

[0082] After the user clicks the "OK" button, a task start instruction triggered by the button is received, and the content in the gaze task starts to be officially displayed. Please refer to Figure 4 , Figure 4 which is a display schematic diagram of a gaze task provided by the present invention. First, an eye movement correction page is displayed on the screen and lasts for a fourth preset duration. Figure 4 The page with a cross icon presented in the middle as shown is the eye movement correction page. After the cross icon is displayed for 1 second (i.e., the fourth preset duration), the gaze task picture starts to be displayed. A marker is displayed at the center point of the gaze task picture, and distractors are displayed around the marker. Figure 4 The marker of the gaze task picture is a dot, and the distractors are the rings around the dot. After the gaze task picture is continuously displayed for 20 seconds (i.e., the fifth preset duration), the gaze task ends.

[0083] On this basis, gaze eye movement features are extracted from the eye movement video when the user views the gaze task, including: extracting the fixation hold time of the user's fixation on the marker in the gaze task picture, the number of saccades during the completion of the gaze task, the average value of the saccade amplitude during the completion of the gaze task, and the average value of the fixation deviation distance of the user's fixation point from the marker. The fixation hold time, the number of saccades, the average value of the saccade amplitude, and the average value of the fixation deviation distance are used as gaze eye movement features.

[0084] The following will detail the presentation method of the attention transfer task and the specific process of extracting attention transfer eye movement features from the eye movement video corresponding to the attention transfer task.

[0085] As an optional embodiment, when presenting the attention transfer task, task guidance information is first displayed on the screen and / or the task guidance information is played by voice. The task guidance information is information for prompting the user to fixate on the face of the person selected in the cue box.

[0086] After receiving the task start instruction, a current task progress page is displayed on the screen and lasts for a sixth preset duration. After the display of the current task progress page ends, an eye movement correction page is displayed on the screen and lasts for a seventh preset duration. After the display of the eye movement correction page ends, a current attention transfer test picture is displayed on the screen and lasts for an eighth preset duration. And during the display of the current attention transfer test picture, the cue box is controlled to sequentially select non-neutral emotion face pictures and neutral emotion face pictures in the current attention transfer test picture according to a preset display rule.

[0087] After the display of the current attention transfer test picture ends, the next picture in the attention transfer test picture collection that is after the current attention transfer test picture is used as the new current attention transfer test picture, the current task progress page is updated, and the step of displaying the current task progress page on the screen and lasting for the seventh preset duration is entered until all the pictures in the attention transfer test pictures are displayed.

[0088] Among them, the attention transfer test pictures include both non-neutral emotion face pictures and neutral emotion face pictures, and the non-neutral emotion face pictures and neutral emotion face pictures are distributed according to a preset distribution rule, and the non-neutral emotion face pictures are positive emotion face pictures or negative emotion face pictures.

[0089] Specifically, at the beginning of the attention transfer task, task guidance information is first displayed on the screen to prompt the user what to do when completing the attention transfer task, so that the user can understand the attention transfer task in advance, so as to make a timely response after the task officially starts and ensure objectivity. For example, at the beginning of the attention transfer task, the task guidance information is first displayed on the screen: "This is an attention transfer task to test reaction ability. After the task officially starts, two face pictures and a cue box will appear on the screen at the same time. Please look at the face picture selected by the cue box. When the face picture selected by the cue box changes, please look at the new face picture selected by the cue box", and the above task guidance information is played by voice, so that the user can understand the content of the attention transfer task in advance and make an accurate response after the attention transfer task starts.

[0090] After the user clicks the "OK" button, a task start instruction triggered by the button is received, and the content of the attention transfer task is officially started to be displayed. First, the current task progress page is displayed on the screen. For example, the characters X / Y are displayed on the current task progress page, where X represents the serial number of the current attention transfer test picture, and Y represents the total number of pictures included in the attention transfer test picture collection. Please refer to Figure 5 , Figure 5 which is a display schematic diagram of an attention transfer task provided by the present invention. As Figure 5 shown, the current task progress page shows 2 / 8, indicating that the serial number of the current attention transfer test picture is 2, and the attention transfer test picture collection includes a total of 8 attention transfer test pictures.

[0091] The current task progress page is continuously displayed for the sixth preset duration. After the display ends, an eye movement correction page is presented, and the eye movement correction page is continuously displayed for the seventh preset duration. Take Figure 5For example, the current task progress page is displayed for 1 second, then the eye movement correction page is continuously displayed for 1 second, and then the current attention transfer test picture is displayed and continuously displayed for 3.5 seconds. After the display of the current attention transfer test picture ends, the next picture in the attention transfer test picture collection is used as the new current attention transfer test picture, and the current task progress page is updated, and the above display process is repeated until all the pictures in the attention transfer test picture collection are displayed.

[0092] It should be noted that the present invention pre-sets an attention transfer test picture collection, which includes a certain number of attention transfer test pictures. Each attention transfer test picture includes both non-neutral emotion face pictures and neutral emotion face pictures, and the non-neutral emotion face pictures and neutral emotion face pictures are distributed according to a preset distribution rule. The non-neutral emotion face pictures are positive emotion face pictures or negative emotion face pictures.

[0093] It should also be noted that the clue box needs to be controlled to select the non-neutral emotion face pictures and neutral emotion face pictures in the current attention transfer test picture successively according to a preset display rule. The preset display rule can stipulate the starting position and transfer direction of the clue box, and it is necessary to balance different emotion face pictures.

[0094] As Figure 5 shown, the second picture in the attention transfer test picture collection is used as the current attention transfer test picture. This picture includes both a positive emotion face picture and a neutral emotion face picture. The clue box first selects the positive emotion face picture on the right and continuously displays it for about one to two seconds, and then selects the neutral emotion face picture on the left and continuously displays it for two seconds. Taking the next new current attention transfer test picture that also includes a positive emotion face and a neutral emotion face picture as an example, the clue box can first select the neutral emotion face picture on the left and continuously display it for about one to two seconds, and then select the positive emotion face on the right and continuously display it for about two seconds. By balancing the starting position and moving direction of the clue box, the attention transfer eye movement features extracted from the eye movement video when the user watches the attention transfer task are made more objective, laying a foundation for subsequent emotion disorder assessment.

[0095] On this basis, attention transfer eye movement features are extracted from the eye movement videos of users when they perform the attention transfer task, including: the transfer latency from the start of the cue box to the start of the saccade of the user, the transfer time from the start of the saccade to the user's fixation on the new emotional face picture, the fixation time of the user on each emotional face picture, the fixation count of the user on each emotional face picture, the average speed and the maximum speed of the saccades during the user's completion of the attention transfer task, and the complexity of the eye movement trajectory during the user's completion of the fixation transfer task. The transfer latency, transfer time, fixation time, fixation count, average speed and maximum speed of the saccades, and the complexity of the eye movement trajectory are used as attention transfer eye movement features.

[0096] In summary, the emotional disorder assessment system provided by the present invention sets up a free viewing task, a fixation task, and an attention transfer task, and these three tasks reflect the user's attention status from different perspectives. Among them, the free viewing task reflects the attention bias, the fixation task reflects the attention maintenance ability and the anti-interference ability, and the attention transfer task reflects the attention transfer efficiency. By extracting the free viewing eye movement features, fixation eye movement features, and attention transfer eye movement features from the eye movement videos corresponding to these three tasks, the emotional disorder risk of the user can be reflected more comprehensively and accurately.

[0097] Furthermore, the present invention can also set up a behavioral feature acquisition module in the emotional disorder assessment system. The behavioral feature acquisition module is used to acquire and output the behavioral features of the user when viewing the attention transfer task. Among them, the behavioral features include the reaction time and the correct rate. The reaction time is the time from the appearance of each cue box to the user's fixation on the emotional face picture selected by the cue box, and the correct rate is the ratio of the number of times the user correctly fixes on the emotional face picture selected by each cue box to the total number of times each cue box selects each emotional face picture. By combining the eye movement features and the behavioral features of the user when completing the emotional disorder assessment task, the emotional disorder status is jointly evaluated, and the emotional disorder risk is identified more comprehensively.

[0098] In addition to evaluating emotional disorders based on eye movement features and behavioral features, multimodal data such as facial expressions, speech features, and physiological signals (such as heart rate and skin conductance) when the user completes the emotional disorder assessment task can also be combined. Through multimodal data fusion algorithms (such as ensemble learning or hybrid neural network models), the accuracy and reliability of emotion recognition can be further improved. This extended solution can provide a more comprehensive and accurate assessment, improve the accuracy and robustness of emotion recognition, in cases where the data quality is low or a single mode cannot fully reflect the user's emotional state. The emotional disorder assessment technology of the present invention can also be integrated into wearable devices such as smart glasses and VR / AR helmets to achieve a more natural and immersive emotional disorder risk assessment. In specific scenarios (such as hospitals, research institutions, etc.), professional eye tracking devices can be combined to improve the precision and accuracy of data collection.

[0099] The process of the emotional disorder assessment model determining the emotional disorder assessment result will be described below.

[0100] As an optional embodiment, the emotional disorder assessment model determines the emotional disorder assessment result based on free viewing eye movement features, fixation eye movement features, and attentional shift eye movement features, including:

[0101] The emotional disorder assessment model uses the logistic regression formula determined during the training process, the free viewing eye movement features and their corresponding weights, the fixation eye movement features and their corresponding weights, and the attentional shift eye movement features and their corresponding weights to determine the logistic regression result. The predicted probability is determined according to the logistic regression result and mapped to the score range to obtain the score of this emotional disorder assessment for the user.

[0102] Before conducting the assessment of mood disorders, a mood disorder assessment model is established and trained in advance. During the model establishment process, first, the eye movement characteristics of users with mood disorders and healthy control groups are collected. Then, differential statistical analysis is performed on the collected eye movement characteristics, and the eye movement characteristics with significant differences are selected to further establish the model. For the eye movement characteristics with significant differences, different models are established using different algorithms, such as decision trees, support vector machines, convolutional neural networks, logistic regression, etc. After the model is established, a new dataset is used for verification, and the area under the ROC curve (Receiver Operating Characteristic curve) of each model is calculated. When the area is the largest, it indicates that the current model has the greatest effectiveness in distinguishing users with mood disorders from normal users, that is, this model is selected as the final model. Through data analysis, when using the logistic regression algorithm, the area under the ROC curve is the largest, and the specific algorithm is as follows: 5826 +- 0.3919 * row['Average speed of negative saccades'] +- 0.5244 * row['Positive bias'] +- 0.1902 * row['Number of fixation intervals'] +- 0.0187 * row['Highest saccade speed'] +- 0.1289 * row['Highest saccade speed of positive terminal gain'] + 0.7653 * row['Saccade intrusion'] + 0.5226 * row['Negative correct saccade latency (after correction)'].

[0103] As a specific embodiment, the mood disorder assessment model includes an input layer, a feature extraction layer, a feature fusion layer, a fully connected layer, and an output layer. The input layer receives and processes free viewing eye movement characteristics, fixation eye movement characteristics, and attention transfer eye movement characteristics. The feature extraction layer may include a long short-term memory network layer and a convolutional neural network layer. Among them, the long short-term memory network layer can extract the time series characteristics of free viewing eye movement characteristics, fixation eye movement characteristics, and attention transfer eye movement characteristics, and the convolutional neural network layer can extract the spatial characteristics of the eye movement trajectory image. The feature fusion layer fuses the free viewing eye movement characteristics, fixation eye movement characteristics, and attention transfer eye movement characteristics, so that the fully connected layer can perform mapping based on the fused eye movement characteristics to obtain the mood disorder risk level or mood disorder risk score. Finally, the mood disorder risk level or mood disorder risk score is output through the output layer.

[0104] The training process of the emotional disorder assessment model mainly includes eye movement feature preprocessing (including but not limited to data cleaning, feature extraction, calculation of feature means, handling missing values, and data merging), feature standardization, feature selection, data partitioning, model training and evaluation, output of the logistic regression equation, determination of the optimal classification threshold, and generation of the logistic regression model, etc. As a specific embodiment, the logistic regression equation is as follows: 5826 +- 0.3919 * row['average negative saccade velocity'] +- 0.5244 * row['positive bias index'] +- 0.1902 * row['number of fixation intervals'] +- 0.0187 * row['highest saccade velocity'] +- 0.1289 * row['highest saccade velocity of positive terminal gain'] + 0.7653 * row['saccade intrusion'] + 0.5226 * row['negative correct saccade latency (after correction)']. Among them, the average negative saccade velocity refers to the average of the saccade velocities when the user transfers attention from a negative emotion face picture to a neutral emotion face picture during the attention transfer task; the positive bias index is a parameter when the user completes the free viewing task; the number of fixation intervals is the number of times the user's eyes switch between fixation points during the fixation task; the highest saccade velocity refers to the maximum value of the velocity when the user's eyes move rapidly between different fixation points during the fixation task; the highest saccade velocity of positive terminal gain refers to the maximum velocity of the eyeball during saccade movement when the user transfers from a positive emotion face picture to a neutral emotion face picture during the attention transfer task; saccade intrusion refers to the situation where saccade data is interfered or contaminated due to external factors or device problems when the user completes the attention transfer task; the negative correct saccade latency refers to the time interval between the moment when the user decides to make a saccade and the moment when the saccade actually starts during the process of transferring from a negative emotion face picture to a neutral emotion face picture. After calculating the logistic regression result using the logistic regression equation and eye movement features, the predicted probability is calculated through 1 / (1 + exp(-logistic regression result)), and finally, the emotional disorder risk assessment result of the user is mapped to the range of 0 to 10 by multiplying the predicted probability by 10, obtaining the score of this emotional disorder assessment for the user. The larger the score, the higher the risk of emotional disorder.

[0105] It should be noted that the above logistic regression equation is only one of the implementation methods determined in actual applications. In the application process, logistic regression formulas can also be established for other eye movement features that show obvious differences between users with emotional disorders and normal users. The present invention does not make specific limitations on this.

[0106] In addition, other algorithms can be introduced to continuously optimize and improve the personalized emotional disorder assessment model. By using reinforcement learning algorithms, the emotional disorder assessment model can be continuously optimized according to users' feedback to achieve adaptive assessment and personalized intervention. Transfer learning can utilize pre-trained models on other related tasks (such as emotion recognition, attention deficit recognition, etc.) to accelerate the training process of the emotional disorder assessment model in the present invention and improve the generalization ability of the model. Graph neural networks can also be used to construct the social relationships, behavior patterns, etc. between users into a graph structure, and use graph neural networks to mine the mutual influence between users to improve the accuracy of emotional disorder risk assessment. Generative adversarial networks can be used to generate more realistic emotional stimulus materials for the design of cognitive assessment tasks to improve the ecological validity of the assessment.

[0107] The following further introduces the deployment method of the emotional disorder assessment system based on eye movement features provided by the present invention.

[0108] Please refer to Figure 2 , Figure 2 which is the second structural schematic diagram of an emotional disorder assessment system based on eye movement features provided by the present invention. When the eye movement video acquisition module 101 is deployed on a mobile terminal, the above-mentioned various emotional disorder assessment tasks can be displayed through the screen of the mobile terminal, and the eye movement video of the user when watching the emotional disorder assessment tasks can be acquired through the front camera of the mobile terminal. A data preprocessing module can be further set on the mobile terminal, and the data preprocessing module preprocesses the original eye movement video acquired by the eye movement video acquisition module 101, including but not limited to time alignment, data denoising, filtering, smoothing, feature extraction, and eye movement video encoding and compression, etc., to facilitate the eye movement feature extraction module 102 to extract eye movement features from the processed eye movement video.

[0109] The eye movement feature extraction module 102 and the emotional disorder assessment module 103 are deployed on the cloud server. Data storage modules for storing information such as eye movement features, behavior features, and emotional disorder assessment results are respectively set on both the cloud service and the mobile terminal, as well as a result presentation and feedback module for presenting the eye movement features, behavior features, and emotional disorder assessment results in a visual manner on the client side.

[0110] In addition, a data security and privacy protection module is set up, adopting strict data security and privacy protection measures to ensure the security of user data during the data interaction between the mobile terminal and the cloud server, and enhancing user trust. When the mobile terminal and the cloud server conduct data interaction, they both utilize the end-to-end encryption and anonymization processing algorithm mechanism to ensure the security and privacy of the user's eye movement characteristics and the results of emotional disorder assessment, meeting the requirements of relevant laws and regulations, improving the user's trust and acceptance of the system, and overcoming the problem of insufficient data security and privacy protection in the existing technology. Blockchain technology can also be used to enhance data security and decentralization. Through blockchain, a decentralized and immutable record system can be created to store and manage the user's eye movement data and emotional assessment results. Through federated learning, multiple devices or institutions are allowed to collaboratively train models without sharing the original data, further enhancing privacy protection.

[0111] It should also be noted that the present invention can also deploy edge computing nodes and deploy more powerful computing resources on the edge computing nodes to achieve more complex model calculations and data analyses. Migrate some tasks originally executed on the cloud server (such as deep learning model inference and eye movement feature extraction, etc.) to the edge computing nodes to further reduce latency and improve the real-time performance of emotional disorder assessment. At the same time, model updates and federated learning are implemented on the edge computing nodes to protect user data privacy.

[0112] Please refer to Figure 6 and Figure 7 , Figure 6 which is the system architecture diagram of an emotional disorder assessment system based on eye movement characteristics provided by the present invention. Figure 7The following is a flowchart of the operation of an emotional disorder assessment system based on eye movement features provided by the present invention. The mobile device is used to provide a user interaction interface, capture a face video through a front camera, and preprocess the face video (including but not limited to denoising, filtering, and feature extraction, etc.) through a processing unit built into the mobile device, and send the preprocessed face video to the edge computing node to reduce data transmission latency. The edge computing node module is a computing node deployed near the user, used for data preprocessing and preliminary analysis, thereby reducing data transmission latency and improving the response speed of the emotional disorder assessment system. The edge computing node conducts preliminary analysis and classification on the preprocessed face video, localizes some simple emotion recognition tasks, reduces the burden of data transmission to the cloud server, and returns the preliminary analysis results to the client or uploads them to the cloud server for further analysis. The cloud server module receives the preliminary analysis data from the edge node and conducts multi-dimensional emotional state analysis and emotional disorder risk assessment using deep learning and machine learning algorithms. It is equivalent to the eye movement feature extraction module 102 and the emotional disorder assessment module 103 being deployed on the edge computing node and the cloud server to jointly complete the emotional disorder assessment. The cloud server also returns the finally determined emotional disorder assessment results, health suggestions, and health warnings to the mobile device to prompt the user.

[0113] For some special scenarios (such as when the mobile network is unstable or the user's privacy requirements are relatively high), the processing capacity of the edge computing node can be enhanced, the computing resources for local data storage and processing of the device can be increased, and a more powerful local processing capacity can be achieved, thereby reducing the dependence on data transmission, improving the system response speed, and protecting user privacy. In addition, in scenarios where the network connection is unstable or there are high data privacy requirements, the emotional disorder assessment system can run entirely on the mobile terminal or the edge computing node without relying on the cloud server. This alternative solution can ensure local data processing and improve the privacy and security of the system.

[0114] It should also be noted that the emotional disorder assessment system provided by the present invention can also be extended to other application scenarios in addition to emotional disorder assessment. The following is an introduction to some application scenarios.

[0115] Based on the emotional disorder assessment system, personal users' daily emotion monitoring and management are realized. Specifically, users can regularly conduct self-tests for emotional disorder risks through a smartphone APP to understand the changing trends of their own emotional states. The APP can provide personalized emotion management suggestions according to the assessment results, such as relaxation training, meditation guidance, and psychological counseling recommendations. The APP can record users' emotion diaries to help users identify emotion triggers and cultivate positive emotion regulation strategies.

[0116] An intelligent eye-brain health management platform is established based on the emotional disorder assessment system. Specifically, the platform can provide users with comprehensive emotional disorder risk assessment services, including online tests, result interpretation, expert consultations, etc. The platform can recommend personalized intervention plans according to the user's assessment results, such as cognitive behavioral therapy, drug therapy, physical therapy, etc. The platform can establish user health records, track the changes in the user's emotional state in the long term, and detect potential risks in a timely manner. The platform can provide data support for doctors and researchers to promote the early diagnosis and treatment of emotional disorders.

[0117] In the scenarios of telemedicine and mental health management, doctors can conduct remote emotional disorder assessment and diagnosis on patients through the remote video conferencing system in combination with the eye movement tracking technology of the present invention. Psychologists can provide remote psychological counseling and intervention services for patients through the online platform and monitor the changes in the emotional state of patients by using the eye movement tracking technology of the present invention. The present invention can provide convenient mental health services for patients in remote areas or with inconvenient mobility.

[0118] In the field of education, teachers can use the eye movement tracking technology of the present invention to monitor the attention and emotional state of students in the classroom and detect students with learning difficulties or emotional problems in a timely manner. The online learning platform can integrate the eye movement tracking function, adjust the teaching content and rhythm according to the learning state of students, and improve the learning effect. The present invention can be used to evaluate the learning styles and cognitive characteristics of students and provide references for personalized teaching.

[0119] In the field of human resources and workplace mental health management, enterprises can use the eye movement tracking technology of the present invention to evaluate the emotional state and psychological pressure of employees and detect potential mental health problems in a timely manner. Enterprises can provide mental health training and consulting services for employees to improve the psychological quality and work efficiency of employees. It can also be used for talent recruitment and selection to evaluate the emotional stability, stress resistance, etc. of candidates.

[0120] The system of the present invention can also be integrated into smart home devices, such as smart mirrors, smart TVs, smart speakers and other devices, to detect the emotional state of family members and automatically adjust the home environment (such as lights, temperature, music) to improve the mood and comfort. At the same time, it can provide early warnings when the emotions of family members are abnormal, helping family members better understand each other's emotional health status.

[0121] The technology of the present invention can also be applied to in-vehicle emotion monitoring systems to monitor the emotional and attention states of drivers in real time. If it detects that the emotional state of the driver is unstable or the attention has declined (such as fatigue or anxiety), the system can issue a warning signal or suggest that the driver take a rest, thereby improving driving safety.

[0122] In gaming and virtual reality applications, the system provided by the present invention can be used to monitor players' emotional responses in real time, dynamically adjust game scenarios and difficulty settings, and enhance the user experience. For example, in horror games, the system can adjust the game atmosphere and sound effects in real time according to the players' emotional states to provide a more immersive experience.

[0123] The system provided by the present invention can also be used for large-scale social emotion monitoring. For example, in emergencies or public crises, it can collect and analyze a large amount of users' emotional data in real time to help the government and relevant agencies promptly understand the changing trends of social emotions and formulate more effective public policies and crisis response measures.

[0124] Please refer to Figure 8 , Figure 8 which is a flowchart of an emotional disorder assessment method based on eye movement features provided by the present invention. The method includes:

[0125] S801. Collect an eye movement video of a user when watching an emotional disorder assessment task; wherein, the emotional disorder assessment task includes a free viewing task for reflecting the attention bias degree of the user towards content with different emotions, a fixation task for reflecting the user's attention stability and anti-interference ability, and an attention transfer task for reflecting the attention transfer efficiency of the user between content with different emotions.

[0126] S802. Extract free viewing eye movement features from the eye movement video of the user when watching the free viewing task, extract fixation eye movement features from the eye movement video of the user when watching the fixation task, and extract attention transfer eye movement features from the eye movement video of the user when watching the attention transfer task.

[0127] S803. Input the free viewing eye movement features, fixation eye movement features, and attention transfer eye movement features into an emotional disorder assessment model, and output an emotional disorder assessment result determined by the emotional disorder assessment model based on the free viewing eye movement features, fixation eye movement features, and attention transfer eye movement features.

[0128] For a detailed introduction to the emotional disorder assessment method based on eye movement features provided by the present invention, please refer to the embodiments of the emotional disorder assessment system based on eye movement features above, and the present invention will not elaborate herein.

[0129] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. It should also be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, article or device comprising the element.

[0130] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. An emotional disorder assessment system based on eye movement characteristics, characterized in that, Including: An eye movement video acquisition module, configured to acquire an eye movement video of a user when watching an emotional disorder assessment task; wherein, the emotional disorder assessment task includes a free viewing task for reflecting the attention bias of the user towards content of different emotions, a gaze task for reflecting the attention stability and anti-interference ability of the user, and an attention transfer task for reflecting the attention transfer efficiency of the user between content of different emotions; An eye movement feature extraction module, configured to extract free viewing eye movement features from the eye movement video of the user when watching the free viewing task, extract gaze eye movement features from the eye movement video of the user when watching the gaze task, and extract attention transfer eye movement features from the eye movement video of the user when watching the attention transfer task; An emotional disorder assessment module, configured to input the free viewing eye movement features, the gaze eye movement features, and the attention transfer eye movement features into an emotional disorder assessment model, and output an emotional disorder assessment result determined by the emotional disorder assessment model based on the free viewing eye movement features, the gaze eye movement features, and the attention transfer eye movement features.

2. The emotional disorder assessment system based on eye movement features according to claim 1, wherein The free viewing task includes: Displaying task guiding information on the screen and / or playing the task guiding information by voice, where the task guiding information is information for prompting the user to freely view a picture of a human face displayed on the screen; After receiving a task start instruction, displaying a current task progress page on the screen for a first preset duration, after the display of the current task progress page ends, displaying an eye movement correction page on the screen for a second preset duration, and after the display of the eye movement correction page ends, displaying a current picture of a human face on the screen for a third preset duration; After the display of the current picture of a human face ends, using the next picture after the current picture of a human face in the human face picture collection as the new current picture of a human face, updating the current task progress page, and entering the step of displaying the current task progress page on the screen for a first preset duration until all pictures in the human face picture collection have been displayed; Wherein, the human face picture collection includes positive-neutral face pictures and negative-neutral face pictures, the positive-neutral face pictures refer to pictures that simultaneously display positive emotion face pictures and neutral emotion face pictures on the same picture, the negative-neutral faces refer to pictures that simultaneously display negative emotion face pictures and neutral emotion face pictures on the same picture, and the positive emotion face pictures and neutral emotion face pictures in the positive-neutral face pictures are distributed according to a preset distribution rule, and the negative emotion face pictures and neutral emotion face pictures in the negative-neutral face pictures are distributed according to a preset distribution rule.

3. The emotional disorder assessment system based on eye movement characteristics according to claim 2, wherein Extracting free viewing eye movement features from the eye movement video of the user when watching the free viewing task includes: Extract the positive face fixation time and the number of positive face fixations when the user gazes at all the positive emotion face pictures, the negative face fixation time and the number of negative face fixations when gazing at all the negative emotion face pictures, the neutral face fixation time and the number of neutral face fixations when gazing at all the neutral emotion face pictures, as well as the first fixation latency corresponding to each of the positive emotion face pictures, the negative emotion face pictures, and the neutral emotion face pictures, where the first fixation latency is the time interval from when each face picture appears on the screen until the user first gazes at the face picture; Determine the positive bias index and the negative bias index based on the positive face fixation time, the negative face fixation time, and the neutral face fixation time, where the positive bias index = positive face fixation time / (positive face fixation time + neutral face fixation time), and the negative bias index = negative face fixation time / (negative face fixation time + neutral face fixation time); Take each face fixation time, each face fixation count, the first fixation latency corresponding to each face picture, the positive bias index, and the negative bias index as the free viewing eye movement features.

4. The emotional disorder assessment system based on eye movement features according to claim 1, characterized in that, The gaze task includes: Display task guidance information on the screen and / or play the task guidance information by voice, where the task guidance information is information prompting the user to gaze at the marker in the center of the screen; After receiving the task start instruction, display an eye movement correction page on the screen for a fourth preset duration, and after the display of the eye movement correction page ends, display a gaze task picture on the screen for a fifth preset duration and then end; where the center point of the gaze task picture displays the marker, and distractors are displayed around the marker.

5. The emotion disorder assessment system based on eye movement characteristics according to claim 4, characterized in that The extraction of the gaze eye movement features from the eye movement video when the user views the gaze task includes: Extract the fixation hold time when the user gazes at the marker in the gaze task picture, the number of saccades during the completion of the gaze task, the average value of the saccade amplitude during the completion of the gaze task, and the average value of the fixation deviation distance of the user's fixation point from the marker; Take the fixation hold time, the number of saccades, the average value of the saccade amplitude, and the average value of the fixation deviation distance as the gaze eye movement features.

6. The emotional disorder assessment system based on eye movement characteristics according to claim 1, wherein The attention shift task includes: Display task guidance information on the screen and / or play the task guidance information by voice, where the task guidance information is information prompting the user to gaze at the human face selected by the cue box; After receiving the task start instruction, display the current task progress page on the screen for a sixth preset duration, after the display of the current task progress page ends, display an eye movement correction page on the screen for a seventh preset duration, after the display of the eye movement correction page ends, display the current attention shift test picture on the screen for an eighth preset duration, and during the display of the current attention shift test picture, control the cue box to sequentially select non - neutral emotion face pictures and neutral emotion face pictures in the current attention shift test picture according to a preset display rule; After the display of the current attention transfer test picture ends, the next picture in the attention transfer test picture collection that comes after the current attention transfer test picture is used as the new current attention transfer test picture. Update the current task progress page and enter the step of displaying the current task progress page on the screen and continuing for the seventh preset duration until all the pictures in the attention transfer test pictures are displayed; Among them, the attention transfer test pictures include both the non-neutral emotion face pictures and the neutral emotion face pictures, and the non-neutral emotion face pictures and the neutral emotion face pictures are distributed according to a preset distribution rule. The non-neutral emotion face pictures are positive emotion face pictures or negative emotion face pictures.

7. The emotional disorder assessment system based on eye movement characteristics according to claim 6, characterized in that Extract the attention transfer eye movement features from the eye movement video when the user views the attention transfer task, including: Extract the transfer latency when the user starts to transfer from the cue box until the user starts to saccade, the transfer time from the start of saccade to the user's fixation on the new emotion face picture, the fixation time when the user fixates on each emotion face picture, the number of fixation times when the user fixates on each emotion face picture, the average speed and the highest speed of saccades during the user's completion of the attention transfer task, and the complexity of the eye movement trajectory during the user's completion of the fixation transfer task; Take each of the transfer latency, the transfer time, the fixation time, the number of fixation times, the average speed and the highest speed of saccades, and the complexity of the eye movement trajectory as the attention transfer eye movement features.

8. The emotional disorder assessment system based on eye movement features according to claim 6, wherein The emotional disorder assessment system further includes: A behavioral feature acquisition module for acquiring and outputting the behavioral features when the user views the attention transfer task. Among them, the behavioral features include the reaction time and the correct rate. The reaction time is the time from the appearance of each cue box to the user's fixation on the emotion face picture selected by the cue box, and the correct rate is the ratio of the number of times the user correctly fixates on the emotion face pictures selected by each cue box to the total number of times each cue box selects the emotion face pictures.

9. The emotional disorder assessment system based on eye movement characteristics according to claim 1, characterized in that The emotional disorder assessment model determines the emotional disorder assessment result based on the free viewing eye movement features, the fixation eye movement features, and the attention transfer eye movement features, including: The emotional disorder assessment model uses the logistic regression formula determined during the training process, the free viewing eye movement features and their corresponding weights, the fixation eye movement features and their corresponding weights, and the attention transfer eye movement features and their corresponding weights to determine the logistic regression result; Determine the predicted probability according to the logistic regression result and map the predicted probability to the scoring range to obtain the score of this emotional disorder assessment of the user.

10. An emotional disorder assessment method based on eye movement characteristics, characterized in that, Include: Collect the eye movement video when the user views the emotional disorder assessment task; among them, the emotional disorder assessment task includes a free viewing task for reflecting the attention bias degree of the user towards content with different emotions, a fixation task for reflecting the attention stability and anti-interference ability of the user, and an attention transfer task for reflecting the attention transfer efficiency of the user between content with different emotions. Extract free-viewing eye movement features from the eye movement video when the user watches the free-viewing task, extract fixation eye movement features from the eye movement video when the user watches the fixation task, and extract attention transfer eye movement features from the eye movement video when the user watches the attention transfer task; Input the free-viewing eye movement features, the fixation eye movement features, and the attention transfer eye movement features into the mood disorder assessment model, and output the mood disorder assessment result determined by the mood disorder assessment model based on the free-viewing eye movement features, the fixation eye movement features, and the attention transfer eye movement features.

Images