Method, device, equipment and medium for evaluating brain nerve fatigue

Abstract

The present invention provides a method, device, equipment and medium for assessing brain nerve fatigue, which relates to the field of smart medical technology. The specific implementation method includes: in the process of visually guiding the assessment subject to perform multiple eye movement tests, collecting the assessment subject's multi-dimensional eye movement characteristics, and the multi-dimensional eye movement characteristics are controlled by different functional structures of the brain; inputting the multi-dimensional eye movement characteristics into a fatigue assessment neural network model, and the neural network model outputs a brain nerve fatigue assessment value. The fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relationship of eye movement activity characteristics-brain structure-fatigue manifestations, and the fatigue manifestation is the change in eye movement activity characteristics when the brain nerves are fatigued. The present invention solves the problems of low efficiency of existing fatigue assessment methods, single assessment dimension, and one-sided judgment basis, improves the accuracy and reliability of the assessment, and realizes comprehensive, systematic and efficient fatigue monitoring.

Description

Technical Field

[0001] The present application relates to the field of computer technology, specifically to the field of smart medical technology, and in particular to a method, device, equipment and medium for evaluating brain nerve fatigue. Background Art

[0002] The human eye movement system is closely connected to multiple regions of the cerebral cortex, such as the frontal, parietal, and occipital lobes, which are crucial for higher-level brain functions. Furthermore, eye movement is closely linked to brain activity, such as default mode network activity and changes in neurotransmitters. Therefore, eye movement can serve as a window into the functional state of the brain. Fatigue, a typical manifestation of brain function, is often accompanied by changes in eye movement. Fatigue is closely related to physiological processes such as neurotransmitter depletion in brain structures, disrupted energy metabolism, and changes in cortical neurons. These changes can affect the brain's control of eye movement, leading to abnormalities in eye movement. For example, fatigue can lead to delayed visual information processing, slower eye saccades, weakened pupillary light reflexes, and decreased convergence ability.

[0003] Currently, for scenarios requiring fatigue monitoring and assessment, such as assessing the neurological fatigue status of high-intensity mental workers, pilots, aviation personnel, and athletes to help prevent fatigue-related accidents and health problems, existing traditional methods rely primarily on interviews and subjective observations, relying on the physician's personal experience and making it difficult to obtain objective and accurate results. While recent innovative solutions have improved objectivity through the use of digital technology, such as saccade assessment methods that use LED screen light guidance and eye movement measurement to capture and identify minor anomalies in eye movement behavior, they often still require a physician's visual observation for comprehensive judgment.

[0004] In addition, existing methods still have major flaws. On the one hand, they fail to fully utilize the eyes, the organ closest to the brain. On the other hand, they fail to test eye movement activities with unified standards. Therefore, there are great limitations in efficiency and accuracy: because they rely on people's subjective evaluation, they lack objectivity and uniformity, and the evaluation results cannot be quantified; they are also time-consuming, and the amount of information provided by tests of dozens of minutes is relatively small. Fundamentally, the limitations of the current methods mainly come from two points: first, there is a lack of an integrated guidance and monitoring plan for comprehensive fatigue manifestations, and second, there is a lack of a comprehensive and integrated judgment method for performance characteristics. Summary of the Invention

[0005] In response to the problems of existing fatigue assessment methods such as inefficiency, single assessment dimension, one-sided judgment basis and inability to quantify assessment results, a brain nerve fatigue assessment method, device, equipment and medium are provided.

[0006] According to a first aspect, a method for assessing cranial nerve fatigue is provided, comprising:

[0007] In the process of visually guiding the subject to perform a variety of eye movement tests, collecting multi-dimensional eye movement characteristics of the subject, wherein the multi-dimensional eye movement characteristics are controlled by different functional structures of the brain;

[0008] The multi-dimensional eye movement characteristics are input into a fatigue assessment neural network model, and the fatigue assessment neural network model outputs a brain nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relationship between eye movement activity characteristics-brain structure-fatigue manifestations, and the fatigue manifestations are changes in eye movement activity characteristics when brain nerves are fatigued.

[0009] According to a second aspect, a device for assessing cranial nerve fatigue is provided, comprising:

[0010] An eye movement feature acquisition module, configured to acquire multi-dimensional eye movement features of an assessment subject during a process of visually guiding the assessment subject to perform various eye movement tests, wherein the multi-dimensional eye movement features are controlled by different functional structures of the brain;

[0011] The brain nerve fatigue assessment module is used to input the multi-dimensional eye movement characteristics into the fatigue assessment neural network model, and the fatigue assessment neural network model outputs a brain nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on the mesh mapping relationship of eye movement activity characteristics-brain structure-fatigue manifestations, and the fatigue manifestations are changes in eye movement activity characteristics when brain nerves are fatigued.

[0012] According to the third aspect, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method as in any embodiment of the brain nerve fatigue assessment method.

[0013] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method of any embodiment of the brain nerve fatigue assessment method is implemented.

[0014] According to the solution of this application, the existing technology is optimized from three levels: comprehensiveness, systematicness, and efficiency: through standardized guidance and feature collection of multi-dimensional eye movements, neural fatigue is comprehensively evaluated, thereby improving the accuracy and reliability of the evaluation; standardized testing and analysis methods are established, providing an integrated fatigue assessment solution; a fatigue assessment neural network model is constructed based on the mesh mapping relationship between the human body's eye movement characteristics-brain structure-fatigue manifestations, and an in-depth assessment of neural fatigue is performed through the neural network model to obtain a quantitative neural fatigue assessment value, which takes into account comprehensiveness and systematicness while also achieving rapid and accurate fatigue monitoring and evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0016] Figure 1 is an exemplary system architecture diagram to which some embodiments of the present application may be applied;

[0017] Figure 2 is a flow chart of an embodiment of a method for assessing cranial nerve fatigue according to the present application;

[0018] Figure 3 is a schematic structural diagram of an embodiment of a brain nerve fatigue assessment device according to the present application;

[0019] Figure 4 3 is a block diagram of an electronic device used to implement the brain nerve fatigue assessment method of an embodiment of the present application. DETAILED DESCRIPTION

[0020] The following description of exemplary embodiments of the present application is made in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0022] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the brain nerve fatigue assessment method or brain nerve fatigue assessment device of the present application can be applied.

[0023] like Figure 1As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0024] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as video applications, live broadcast applications, instant messaging tools, email clients, social platform software, etc.

[0025] The terminal devices 101, 102, and 103 here can be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with display screens, including but not limited to smartphones, tablet computers, e-book readers, laptop computers, and desktop computers, etc. When the terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, multiple software or software modules for providing distributed services), or they can be implemented as a single software or software module. No specific limitation is made here.

[0026] The server 105 may be a server that provides various services, such as a background server that provides support to the terminal devices 101, 102, and 103. The background server may analyze and process the received data such as eye movement characteristics, and feed back the processing results to the terminal device.

[0027] It should be noted that the brain nerve fatigue assessment method provided in the embodiment of the present application can be executed by the server 105 or the terminal devices 101, 102, 103, and accordingly, the brain nerve fatigue assessment device can be set in the server 105 or the terminal devices 101, 102, 103.

[0028] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0029] Continue to refer Figure 2 , shows a process 200 of an embodiment of a method for assessing brain nerve fatigue according to the present application. The method for assessing brain nerve fatigue comprises the following steps:

[0030] Step 201 : In the process of visually guiding the evaluation subject to perform a variety of eye movement tests, multi-dimensional eye movement characteristics of the evaluation subject are collected. The multi-dimensional eye movement characteristics are controlled and realized by different functional structures of the brain.

[0031] In step 202, the multi-dimensional eye movement characteristics are input into a fatigue assessment neural network model, and the fatigue assessment neural network model outputs a brain nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relationship of eye movement activity characteristics-brain structure-fatigue manifestations, and the fatigue manifestations are changes in eye movement activity characteristics when brain nerves are fatigued.

[0032] In this embodiment, the visual brain nerve fatigue assessment method is executed on the execution subject (eg Figure 1 The server or terminal device shown in the figure can visually guide the evaluation object to perform a variety of eye movement tests. It can not only complete the standardized guidance of various eye movement behaviors, but also analyze various characteristics, map them to the structure of the brain through neural networks, and then locate the mechanism of neural fatigue, and perform a deep, three-dimensional and reliable assessment of fatigue. This embodiment uses standardized guidance and feature collection of multi-dimensional eye movements, combined with a neural network model to perform an in-depth assessment of neural fatigue, thereby quantifying the fatigue assessment results, improving the accuracy and reliability of the assessment, and realizing comprehensive, systematic and efficient fatigue monitoring.

[0033] In some optional implementations of this embodiment, during the process of visually guiding the subject to perform multiple eye movement tests, multi-dimensional eye movement characteristics of the subject are collected, including:

[0034] During the process of visually guiding the evaluation subject to perform a pupil light reflex test, a forward saccade test, a backward saccade test, and a convergence test, the eye movement characteristics corresponding to the evaluation subject in each test are collected respectively to obtain a multi-dimensional eye movement characteristic of the evaluation subject, wherein the multi-dimensional eye movement characteristic includes pupil light reflex characteristics, forward saccade characteristics, backward saccade characteristics, and convergence reaction characteristics.

[0035] During specific implementation, the evaluation subject is visually guided, the eye image of the evaluation subject is captured, and feature extraction is performed on the eye image to obtain multi-dimensional eye movement characteristics of the evaluation subject; wherein, the visual guidance includes: light spot guidance of eye saccades, quantitative change guidance of light, and change in depth distance of the guide object.

[0036] In some optional implementations of this embodiment, a virtual reality device can be used to provide visual guidance and collect multi-dimensional eye movement characteristics of the evaluation subject. The device can display content in a virtual reality (VR) manner and has an infrared video module that can capture and record images of the test subject's eyes. Virtual reality devices, such as VR helmets with eye movement capture modules, use virtual reality technology to provide comprehensive visual guidance and can record real-time eye movement characteristics such as independent gaze direction, eye movement, and pupil changes of both eyes.

[0037] This embodiment conducts eye movement testing on subjects undergoing virtual reality (VR) assessment. Specifically, through light spot guidance of eye saccades, quantitative changes in light, and changes in the depth and distance of the guide, quantitative and standardized measurement and analysis of multiple types of eye movements are completed. Then, using a feature extraction method based on neural mechanisms, the structure and function of the brain are mapped, further reflecting the impact of fatigue on the brain.

[0038] In a specific implementation, the eye image of the evaluation object is captured by an infrared-based pupil corneal reflection method, and features are extracted from the eye image to obtain multi-dimensional eye movement features of the evaluation object.

[0039] The infrared pupil and corneal reflection method (IPCC) uses an infrared light source and a highly sensitive camera to measure the pupil and corneal reflections of the eye. This method is an effective tool for ophthalmic diagnosis and vision research, providing important information about eye health and visual function through precise measurements of pupil and corneal reflections.

[0040] This embodiment proposes a test set that can comprehensively and quickly reflect fatigue status, covering pupil light reflex, forward saccade, backward saccade, and convergence, specifically including:

[0041] (1) The subject is located in the device acquisition area, completes binocular gaze direction calibration, and enters the test process. The calibration method can be the common 5-point or 9-point calibration method.

[0042] (2) First, a pure black background is presented to the subject for a period of time to allow the subject's pupil to adapt to the current light conditions. Then, a white light flash is presented at a fixed intensity, and the characteristics of the pupil light stimulation are recorded.

[0043] (3) A target bright spot with a diameter of 1.5° is displayed on a pure black background to guide eye movement. The subject is asked to move his eyes as quickly as possible to follow the bright spot. The bright spot first appears in the center of the visual field and stays there briefly. Then it flashes to a fixed position in a random direction up, down, left, or right and stays there before disappearing. After 2 seconds, it reappears in the center. This is repeated several times, and the characteristics of the eye movements during this process are recorded.

[0044] (4) Continue to display a target bright spot with a diameter of 1.5° on a pure black background to guide eye movement. The subject is required to observe the bright spot and move the eye as quickly as possible. The bright spot first appears in the center of the visual field and stays for a short time. Then it will flash to a fixed position in a random direction of up, down, left, or right and stay there before disappearing. At this time, the subject is required to look in the opposite direction of the bright spot. After 2 seconds, it will reappear in the center position. This will be repeated several times. The characteristics of the reverse eye movement during this process are recorded.

[0045] (5) Using the display method of virtual reality, a red three-dimensional ball is presented in a simulated three-dimensional space. The ball will flash randomly within a range of 20 cm to 100 cm in front of the subject's eyes, and each position will last for 1 second. After 1 second, it will continue to flash at random positions, and continue to repeat several times. The subject needs to actively focus through the convergence reaction. This process is called the convergence test.

[0046] The four tasks described above comprehensively cover the entire visual-oculomotor pathway of the human body. The pupillary light stimulus response corresponds to the pupil's contraction and expansion in response to light. Forward saccades reflect the function of eye movement (passive execution). Reverse saccades are associated with the visual field and active execution of eye movement. Convergence tests correspond to the eye's perception of depth information. Guided by virtual reality technology, this ensures both uniform content (full-field display ensures consistent light intensity for all viewers) and the rich visual content presented in two-dimensional (planar information) and three-dimensional space (the presence of depth information). It also avoids the compensatory problem of unconscious head movement during eye movement.

[0047] During the eye movement guidance process, the subjects completed a total of four tasks, during which a large number of characteristic parameters were collected. The details are as follows:

[0048] (1) In the pupillary light reflex test, in the pupillary light stimulation reaction, the average pupil size before the light appears is collected as the baseline value, and the time it takes for the pupil to contract when the strong light appears is recorded as the latency; the maximum velocity mcv and the maximum contraction value mc during the contraction process are recorded; the time it takes for the pupil to return to stability is recorded as st; the pupil size after stabilization is recorded as sa and the difference from the baseline value sid.

[0049] (2) In the forward saccade test, the saccade velocity sv, saccade latency sl, saccade task completion time sf, and task accuracy sar are recorded based on the changes in the eye gaze direction from the center to the surrounding area.

[0050] (3) In the antisaccade test, the antisaccade latency al and antisaccade accuracy ac are recorded based on the changes in the eye gaze direction from the center position to the opposite direction of the surrounding area.

[0051] (4) In the convergence test, based on the process of the eyes fixating on the red ball at a random position, the convergence velocity cv, the convergence latency cl, and the convergence judgment accuracy ca are recorded and collected.

[0052] Finally, the basic information of the subjects, including gender, age, education level, vision, etc., was recorded.

[0053] In some optional implementations of this embodiment, the multi-dimensional eye movement features are input into a fatigue assessment neural network model, and the fatigue assessment neural network model outputs a cranial nerve fatigue assessment value, including:

[0054] The fatigue assessment neural network model is based on the correlation between the multi-dimensional eye movement characteristics and the functional structure of the brain, and determines the neural fatigue status of each functional structure of the brain according to the changes in the multi-dimensional eye movement characteristics;

[0055] According to the neural fatigue status of each functional structure of the brain, the brain neural fatigue assessment value is output.

[0056] In specific implementation, the fatigue assessment neural network model is constructed based on the mesh mapping relationship between eye movement characteristics, brain structure, and fatigue manifestations. First, the mesh mapping relationship between eye movement characteristics, brain structure, and fatigue manifestations needs to be established, as follows:

[0057] (1) Eye movement characteristics:

[0058] a. Pupillary light reflex: The speed and amplitude of the pupil's response to light stimulation are mainly regulated by the oculomotor nucleus in the brainstem.

[0059] b. Forward saccade: A rapid eye movement that shifts the focus from one location to another, primarily controlled by the frontal and parietal eye fields of the cerebral cortex.

[0060] c. Antisaccade: The ability to inhibit the impulse of autonomous eye movements and turn in the opposite direction of the stimulus, reflecting the brain's inhibitory control ability and is mainly related to the frontal cortex.

[0061] D. Convergence: The ability of the two eyes to coordinate and focus on objects at different distances, mainly related to the convergence center in the midbrain.

[0062] (2) Brain structure:

[0063] a. Brainstem: Responsible for basic life functions, including pupillary light reflex.

[0064] b. Frontal lobe: The center of higher cognitive functions, including decision-making, planning, executive control, and inhibitory control, and is closely related to saccades and countersaccades.

[0065] c. Parietal lobe: integrates sensory information, guides spatial attention and eye movements, and is closely related to saccades.

[0066] d. Midbrain: coordinates eye movements, including convergence.

[0067] (3) Fatigue symptoms:

[0068] a. Pupillary light reflex: Fatigue may cause the latency of the pupillary light reflex to be prolonged, the contraction speed to be slowed down, and the contraction amplitude to be reduced.

[0069] b. Eye saccades: Fatigue may cause slower eye saccade speed, decreased accuracy, and longer latency.

[0070] c. Countersaccade: Fatigue may lead to an increase in the error rate of countersaccades.

[0071] d. Convergence: Fatigue may lead to slower convergence speed, decreased accuracy, and longer latency.

[0072] After determining the mesh mapping relationship between eye movement characteristics, brain structure, and fatigue manifestations, a fatigue assessment neural network model is established. This neural network model determines the neural fatigue status of each functional brain structure corresponding to each eye movement characteristic based on the changes in the multi-dimensional eye movement characteristics. Based on the neural fatigue status of each functional brain structure, the model then outputs a final brain neural fatigue assessment value, thereby achieving standardized, comprehensive, and quantified fatigue assessment results.

[0073] In some optional implementations of this embodiment, the fatigue assessment neural network model includes: an input layer, a hidden layer, a shared layer, a task-specific layer and an output layer; the input layer is used to receive the multi-dimensional eye activity feature values of the assessment object after standardization; the hidden layer is used to extract and learn key features in the input data based on the mesh mapping relationship, and the key features are the remaining features after eliminating abnormal feature data; the shared layer is used to extract common features, and the common features include basic information of the assessment object; the task-specific layer is used to determine the neural fatigue status of each functional structure of the brain according to changes in eye activity characteristics of different dimensions; and the output layer is used to output a brain neural fatigue assessment value based on the neural fatigue status of each functional structure of the brain.

[0074] In practice, this embodiment establishes an artificial intelligence neural network model for fatigue assessment. Taking into account comprehensive multi-dimensional input parameters and multiple probabilistic results, the solution uses a multi-task learning (MTL) architecture. The details are as follows:

[0075] (1) Parameter standardization: All collected eye movement feature parameters are organized into a consistent format and standardized so that the network can process them effectively.

[0076] (2) Configure the network:

[0077] a. Input layer: Design an input layer that can receive all the parameters that have been standardized in the above steps, such as latency, mcv, sf, etc.

[0078] b. Hidden layers and shared layers: Multiple hidden layers are configured to extract and learn key features from the data. Some layers serve as shared layers to extract common features. Common features in the shared layers include basic information about the subject, such as age and gender. Hidden layers contain mappings between eye movement features, brain structure, and fatigue manifestations.

[0079] c. Task-specific layer: A specific layer is configured for fatigue, which is used to determine the neural fatigue status of each functional structure of the brain based on the changes in eye movement characteristics in different dimensions.

[0080] d. Output layer: Outputs the brain neural fatigue assessment value based on the neural fatigue status of each functional structure of the brain.

[0081] In some optional implementations of this embodiment, a softmax activation function is used in the output layer to output the fatigue probability of the functional structure of the brain associated with different eye movement characteristics; the fatigue probability of the functional structure of the brain associated with different eye movement characteristics is subjected to principal component analysis to obtain a comprehensive brain nerve fatigue assessment value.

[0082] The Softmax activation function is a commonly used function in machine learning and deep learning. It is used in the output layer of a neural network to convert the raw scores output by the neural network into a probability distribution. The output of the Softmax function is a normalized probability distribution, meaning that the sum of all output values is 1. The aforementioned principal component analysis is a dimensionality reduction technique that combines fatigue probabilities from multiple tasks into a comprehensive indicator for assessing overall fatigue.

[0083] In a specific implementation, principal component analysis is performed on the fatigue probabilities of the different eye movement features to obtain a comprehensive fatigue assessment result, including:

[0084] (1) Standardizing data: standardizing the fatigue probability values of the functional structures of the brain associated with the different eye movement characteristics to obtain standardized data;

[0085] (2) Calculating the covariance matrix and eigendecomposition: Calculating the covariance matrix of the standardized data, performing eigendecomposition on the covariance matrix to obtain eigenvectors and eigenvalues;

[0086] (3) Selecting the principal component: selecting a principal eigenvector from the eigenvectors based on the eigenvalues, projecting the standardized data onto the principal eigenvectors, and obtaining a comprehensive fatigue assessment value.

[0087] Finally, the model is trained: typical fatigue population and normal people are selected to conduct fatigue tests to complete the training of the fatigue assessment neural network model.

[0088] Once the model is established, it can be integrated into a neural fatigue assessment system, enabling a comprehensive and objective analysis of neural fatigue using four simple tasks in a virtual reality setting. After the subject completes all tasks, the model directly outputs an overall probability of fatigue (ranging from 0-100%).

[0089] This embodiment provides a method for assessing brain fatigue. It uses virtual reality technology for guidance (including the presentation of visual information, guidance of eye movements, changes in light, and changes in the depth and distance of the guide object). It uses an infrared-based pupil-corneal reflection method to collect eye information (including gaze direction, eye movement, pupil diameter changes, and convergence activity). All parameters of the eye information are integrated to establish a multi-parameter model to provide the assessment results. This proposal addresses the inefficiency of traditional methods and the single evaluation dimension and one-sided judgment basis of new innovative solutions in recent years. It improves the accuracy and reliability of the assessment and enables rapid and accurate fatigue monitoring and assessment.

[0090] Further references Figure 3 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a brain nerve fatigue assessment device. Figure 2 Corresponding to the method embodiment shown, in addition to the features described below, the device embodiment may also include Figure 2 The device can be applied to various electronic devices.

[0091] like Figure 3As shown, the brain nerve fatigue assessment device 300 of this embodiment includes: an eye movement feature acquisition module 301 and a brain nerve fatigue assessment module 302. The eye movement feature acquisition module 301 is used to collect the multi-dimensional eye movement features of the assessment object in the process of visually guiding the assessment object to perform various eye movement tests. The multi-dimensional eye movement features are controlled and realized by different functional structures of the brain. The brain nerve fatigue assessment module 302 is used to input the multi-dimensional eye movement features into the fatigue assessment neural network model, and the fatigue assessment neural network model outputs a brain nerve fatigue assessment value. The fatigue assessment neural network model is a neural network model constructed based on the mesh mapping relationship of eye movement activity features-brain structure-fatigue manifestations, and the fatigue manifestations are changes in eye movement activity features when the brain nerves are fatigued.

[0092] In this embodiment, the specific processing of the eye movement feature acquisition module 301 and the brain nerve fatigue assessment module 302 of the brain nerve fatigue assessment device 300 and the technical effects thereof can be referred to respectively. Figure 2 The relevant descriptions of step 201 and step 202 in the corresponding embodiment are not repeated here.

[0093] In some optional implementations of this embodiment, the eye movement feature acquisition module 301 collects multi-dimensional eye movement features of the evaluation object during the process of visually guiding the evaluation object to perform pupil light reflex test, forward saccade test, backward saccade test and convergence test. The multi-dimensional eye movement features include pupil light reflex features, forward saccade features, backward saccade features and convergence reaction features.

[0094] In some optional implementations of this embodiment, the fatigue assessment neural network model in the brain nerve fatigue assessment module 302 is based on the correlation between the multi-dimensional eye movement characteristics and the functional structure of the brain, and determines the neural fatigue status of each functional structure of the brain according to the changes in the multi-dimensional eye movement characteristics; and outputs a brain nerve fatigue assessment value according to the neural fatigue status of each functional structure of the brain.

[0095] Among them, the fatigue assessment neural network model includes: an input layer, a hidden layer, a shared layer, a task-specific layer and an output layer; the input layer is used to receive the multi-dimensional eye activity feature values of the assessment object after standardization processing, the hidden layer is used to extract and learn the key features in the input data based on the mesh mapping relationship, the key features are the remaining features after eliminating abnormal feature data, the shared layer is used to extract common features, and the common features include basic information of the assessment object, the task-specific layer is used to determine the neural fatigue status of each functional structure of the brain according to the changes in eye activity characteristics of different dimensions, and the output layer is used to output the brain neural fatigue assessment value according to the neural fatigue status of each functional structure of the brain.

[0096] In some optional implementations of this embodiment, a softmax activation function is used in the output layer to output the fatigue probability of the functional structure of the brain associated with different eye movement characteristics; the fatigue probability of the functional structure of the brain associated with different eye movement characteristics is subjected to principal component analysis to obtain a comprehensive brain nerve fatigue assessment value.

[0097] Among them, the fatigue probability values of the functional structures of the brain associated with the different eye movement characteristics are standardized to obtain standardized data; the covariance matrix of the standardized data is calculated, and the covariance matrix is eigendecomposed to obtain eigenvectors and eigenvalues; according to the eigenvalues, the main eigenvectors are selected from the eigenvectors, and the standardized data are projected onto the main eigenvectors to obtain a comprehensive brain nerve fatigue assessment value.

[0098] In some optional implementations of this embodiment, the eye movement feature acquisition module 301 may be a virtual reality device, which visually guides the evaluation subject to perform various eye movement tests and acquire the multi-dimensional eye movement features.

[0099] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0100] like Figure 4 , is a block diagram of an electronic device according to a method for assessing brain nerve fatigue according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0101] like Figure 4 As shown, the electronic device includes: one or more processors 401, a memory 402, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 401 is taken as an example.

[0102] Memory 402 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the neurological fatigue assessment method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the neurological fatigue assessment method provided in this application.

[0103] The memory 402 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the brain nerve fatigue assessment method in the embodiment of the present application (for example, the attached Figure 3 The processor 401 executes the non-transient software programs, instructions, and modules stored in the memory 402 to execute various functional applications and data processing of the server, thereby implementing the brain nerve fatigue assessment method in the above method embodiment.

[0104] The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the electronic device of the brain nerve fatigue assessment method, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely located relative to the processor 401, and these remote memories may be connected to the electronic device of the brain nerve fatigue assessment method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0105] The electronic device of the brain nerve fatigue assessment method may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403 and the output device 404 may be connected via a bus or other means. Figure 4 The bus connection is taken as an example.

[0106] The input device 403 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device of the cranial nerve fatigue assessment method, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, an indicator stick, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 404 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0107] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0109] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0110] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0111] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0112] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0113] The units involved in the embodiments described in this application can be implemented by software or hardware. The units described can also be set in a processor. For example, it can be described as: a processor includes an eye movement feature acquisition module and a brain nerve fatigue assessment module. Among them, the names of these units do not constitute a limitation of the units themselves in some cases. For example, the eye movement feature acquisition module unit can also be described as a "data acquisition module."

[0114] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the device, the device: guides through virtual reality technology (including the presentation of visual information, guidance of eye movements, changes in light, and changes in the depth and distance of the guided object), collects eye information (including gaze direction, eye movement, pupil diameter change, and convergence activity) through an infrared-based pupil corneal reflection method, and fuses all parameters of the eye information to establish a multi-parameter model to provide an evaluation result.

[0115] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for evaluating brain nerve fatigue, characterized in that: include: In the process of visually guiding the subject to perform a variety of eye movement tests, collecting multi-dimensional eye movement characteristics of the subject, wherein the multi-dimensional eye movement characteristics are controlled by different functional structures of the brain; Inputting the multi-dimensional eye movement features into a fatigue assessment neural network model, the fatigue assessment neural network model outputting a cranial nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relationship between eye movement features, brain structure, and fatigue manifestations, and the fatigue manifestations are changes in the eye movement features when cranial nerve fatigue occurs; In the process of visually guiding the evaluation subject to perform a variety of eye movement tests, the multi-dimensional eye movement characteristics of the evaluation subject are collected, including: During the process of visually guiding the evaluation subject to perform a pupil light reflex test, a forward saccade test, a backward saccade test, and a convergence test, respectively collecting eye movement characteristics corresponding to the evaluation subject in each test to obtain multi-dimensional eye movement characteristics of the evaluation subject, the multi-dimensional eye movement characteristics including pupil light reflex characteristics, forward saccade characteristics, backward saccade characteristics, and convergence response characteristics; The multi-dimensional eye movement features are input into a fatigue assessment neural network model, and the fatigue assessment neural network model outputs a cranial nerve fatigue assessment value, including: The fatigue assessment neural network model is based on the correlation between the multi-dimensional eye movement characteristics and the functional structure of the brain, and determines the neural fatigue status of each functional structure of the brain according to the changes in the multi-dimensional eye movement characteristics; According to the neural fatigue status of each functional structure of the brain, the brain neural fatigue assessment value is output.

2. The method for evaluating cranial nerve fatigue according to claim 1, wherein: The fatigue assessment neural network model includes: an input layer, a hidden layer, a shared layer, a task-specific layer and an output layer; The input layer is used to receive the multi-dimensional eye movement feature values of the evaluation object after standardization processing, the hidden layer is used to extract and learn the key features in the input data based on the mesh mapping relationship, and the key features are the remaining features after eliminating abnormal feature data. The shared layer is used to extract common features, and the common features include basic information of the evaluation object. The task-specific layer is used to determine the neural fatigue status of each functional structure of the brain according to the changes in the eye movement characteristics of different dimensions. The output layer is used to output the brain neural fatigue evaluation value according to the neural fatigue status of each functional structure of the brain.

3. The method for evaluating cranial nerve fatigue according to claim 2, wherein: The method further comprises: A softmax activation function is used in the output layer to output fatigue probabilities of brain functional structures associated with different eye movement features; The fatigue probability of the functional structure of the brain associated with the different eye movement characteristics is subjected to principal component analysis to obtain a comprehensive brain nerve fatigue assessment value.

4. The method for evaluating cranial nerve fatigue according to claim 3, wherein: The fatigue probability of the functional structure of the brain associated with the different eye movement characteristics is subjected to principal component analysis to obtain a comprehensive brain nerve fatigue assessment value, including: Normalizing the fatigue probability values of the functional structures of the brain associated with the different eye movement characteristics to obtain standardized data; Calculating the covariance matrix of the standardized data, performing eigendecomposition on the covariance matrix to obtain eigenvectors and eigenvalues; According to the eigenvalues, a main eigenvector is selected from the eigenvectors, and the standardized data is projected onto the main eigenvector to obtain a comprehensive cranial nerve fatigue assessment value.

5. The method for evaluating cranial nerve fatigue according to claim 1, wherein: During the process of visually guiding the subject to perform various eye movement tests, multi-dimensional eye movement characteristics of the subject are collected, including: The evaluation subject is visually guided to perform a variety of eye movement tests and collect the multi-dimensional eye movement characteristics through virtual reality equipment.

6. A device for assessing brain fatigue, characterized in that: include: An eye movement feature acquisition module, configured to acquire multi-dimensional eye movement features of an assessment subject during a process of visually guiding the assessment subject to perform various eye movement tests, wherein the multi-dimensional eye movement features are controlled by different functional structures of the brain; The eye movement feature acquisition module collects eye movement features corresponding to the evaluation subject in each test during the process of visually guiding the evaluation subject to perform the pupil light reflex test, the forward saccade test, the backward saccade test, and the convergence test, thereby obtaining multi-dimensional eye movement features of the evaluation subject, wherein the multi-dimensional eye movement features include pupil light reflex features, forward saccade features, backward saccade features, and convergence response features; a cranial nerve fatigue assessment module, configured to input the multi-dimensional eye movement characteristics into a fatigue assessment neural network model, the fatigue assessment neural network model outputting a cranial nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relationship between eye movement characteristics, brain structure, and fatigue manifestations, wherein the fatigue manifestations are changes in eye movement characteristics when cranial nerve fatigue occurs; Among them, the fatigue assessment neural network model in the brain nerve fatigue assessment module is based on the correlation between the multi-dimensional eye movement characteristics and the functional structure of the brain, and determines the neural fatigue status of each functional structure of the brain according to the changes in the multi-dimensional eye movement characteristics; and outputs the brain nerve fatigue assessment value according to the neural fatigue status of each functional structure of the brain.

7. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images