Cranial nerve fatigue assessment method, device, equipment and medium

Abstract

The invention provides a cranial nerve fatigue assessment method and device, equipment and a medium, and relates to the technical field of smart medical treatment, and the specific implementation mode comprises the steps: collecting multi-dimensional eyeball movement features of an assessment object in the process of performing visual guidance on the assessment object to execute a plurality of eye movement tests, the multi-dimensional eyeball activity features are controlled and realized by different functional structures of the brain; the multi-dimensional eyeball movement features are input into a fatigue assessment neural network model, the neural network model outputs a cranial nerve fatigue assessment value, and the fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relation of the eyeball movement feature-brain structure-fatigue performance. The fatigue manifestation is a change condition of eye movement activity characteristics during cranial nerve fatigue. According to the method, the problems of low efficiency, single evaluation dimension, one-sided judgment basis and the like of an existing fatigue evaluation method are solved, the evaluation accuracy and reliability are improved, and comprehensive, systematic and efficient fatigue monitoring is realized.

Description

Technical Field

[0001] The present application relates to the field of computer technology, specifically to the field of smart medical technology, and in particular to a method, device, equipment and medium for evaluating cranial nerve fatigue. Background Art

[0002] The human eye movement system is closely related to many areas of the cerebral cortex, such as the frontal lobe, parietal lobe, occipital lobe, etc., and these brain areas are precisely the important areas responsible for higher-level brain functions; at the same time, eye movement is also closely related to brain activity, such as default mode network activity, neurotransmitter changes, etc. Therefore, eye movement can be used as a window to reflect the functional state of the brain, and fatigue, as a typical manifestation of the functional state of the brain, is often accompanied by changes in eye movement. The occurrence of fatigue is closely related to physiological processes such as neurotransmitter consumption in brain structure, energy metabolism disorders, and changes in cortical neurons. Changes in these physiological processes will affect the brain's control over eye movement, leading to abnormal eye movement. For example, fatigue can lead to slow processing of visual information, slower eye saccades, weakened pupil light reflex, and decreased convergence ability.

[0003] At present, for scenarios that require fatigue monitoring and assessment, such as assessing the neural fatigue status of high-intensity mental workers, pilots, aviation personnel, athletes, etc., to help them prevent accidents and health problems caused by fatigue, the existing traditional methods mainly rely on inquiries and subjective observations, and rely on the personal experience of doctors, making it difficult to obtain objective and accurate results. Although new innovative solutions in recent years have improved objectivity with the help of digital technology, such as the eye movement assessment method that can capture slight abnormalities in eye movement behavior and help identify them through LED screen light spot guidance and eye movement measurement, it often still requires comprehensive judgment with the naked eye observation of doctors.

[0004] In addition, the existing methods still have major flaws. On the one hand, they fail to fully utilize the eyes, the organ closest to the brain; on the other hand, they fail to test eye movement activities with a unified standard. Therefore, there are great limitations in efficiency and accuracy: because they rely on people's subjective evaluation, they lack objectivity and uniformity, and the evaluation results cannot be quantified; in addition, they take a long time, and the amount of information provided by the test of dozens of minutes is relatively small. Fundamentally, the limitations of the current methods mainly come from two points: first, there is a lack of an integrated guidance and monitoring program for comprehensive fatigue performance; second, there is a lack of a comprehensive and integrated judgment method for performance characteristics. Summary of the invention

[0005] In view of the problems of inefficiency of existing fatigue assessment methods, single assessment dimension, one-sided judgment basis and inability to quantify assessment results, a cranial nerve fatigue assessment method, device, equipment and medium are provided.

[0006] According to a first aspect, a method for assessing cranial nerve fatigue is provided, comprising:

[0007] In the process of visually guiding the evaluation subject to perform a variety of eye movement tests, collecting multi-dimensional eye movement characteristics of the evaluation subject, wherein the multi-dimensional eye movement characteristics are controlled and realized by different functional structures of the brain;

[0008] The multi-dimensional eye movement characteristics are input into a fatigue assessment neural network model, and the fatigue assessment neural network model outputs a cranial nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relationship of eye movement activity characteristics-brain structure-fatigue manifestations, and the fatigue manifestations are changes in eye movement activity characteristics when cranial nerves are fatigued.

[0009] According to a second aspect, a cranial nerve fatigue assessment device is provided, comprising:

[0010] An eye movement feature acquisition module, used to acquire multi-dimensional eye movement features of the evaluation subject during the process of visually guiding the evaluation subject to perform various eye movement tests, wherein the multi-dimensional eye movement features are controlled and realized by different functional structures of the brain;

[0011] A cranial nerve fatigue assessment module is used to input the multi-dimensional eye movement characteristics into a fatigue assessment neural network model, and the fatigue assessment neural network model outputs a cranial nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relationship between eye movement activity characteristics-brain structure-fatigue manifestations, and the fatigue manifestations are changes in eye movement activity characteristics when cranial nerves are fatigued.

[0012] According to a third aspect, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method as in any embodiment of the method for assessing brain nerve fatigue.

[0013] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, a method as in any embodiment of the cranial nerve fatigue assessment method is implemented.

[0014] According to the scheme of the present application, the existing technology is optimized from three levels: comprehensiveness, systematicness and efficiency: through standardized guidance and feature collection of multi-dimensional eye movements, neural fatigue is comprehensively evaluated, thereby improving the accuracy and reliability of the evaluation; standardized testing and analysis methods are established, providing an integrated fatigue assessment solution; a fatigue assessment neural network model is constructed based on the mesh mapping relationship between the human body's eye movement characteristics-brain structure-fatigue manifestations, and an in-depth assessment of neural fatigue is performed through the neural network model to obtain a quantitative neural fatigue assessment value, which takes into account comprehensiveness and systematicness while also achieving rapid and accurate fatigue monitoring and evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0016] Figure 1 is an exemplary system architecture diagram to which some embodiments of the present application may be applied;

[0017] Figure 2 is a flow chart of an embodiment of a method for assessing cranial nerve fatigue according to the present application;

[0018] Figure 3 is a schematic structural diagram of an embodiment of a cranial nerve fatigue assessment device according to the present application;

[0019] Figure 4 It is a block diagram of an electronic device used to implement the cranial nerve fatigue assessment method of an embodiment of the present application. DETAILED DESCRIPTION

[0020] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0021] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0022] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of a method or device for evaluating cranial neural fatigue of the present application can be applied.

[0023] like Figure 1As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0024] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as video applications, live broadcast applications, instant messaging tools, email clients, social platform software, etc.

[0025] The terminal devices 101, 102, 103 here can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, multiple software or software modules for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0026] The server 105 may be a server that provides various services, such as a background server that provides support for the terminal devices 101, 102, and 103. The background server may analyze and process the received data such as eye movement characteristics, and feed back the processing results to the terminal device.

[0027] It should be noted that the brain nerve fatigue assessment method provided in the embodiment of the present application can be executed by the server 105 or the terminal devices 101, 102, 103, and accordingly, the brain nerve fatigue assessment device can be set in the server 105 or the terminal devices 101, 102, 103.

[0028] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0029] Continue to refer Figure 2 , shows a process 200 of an embodiment of a method for assessing cranial nerve fatigue according to the present application. The method for assessing cranial nerve fatigue comprises the following steps:

[0030] Step 201 , in the process of visually guiding the evaluation subject to perform a variety of eye movement tests, multi-dimensional eye movement characteristics of the evaluation subject are collected, and the multi-dimensional eye movement characteristics are controlled and realized by different functional structures of the brain.

[0031] Step 202: input the multi-dimensional eye movement characteristics into a fatigue assessment neural network model, and the fatigue assessment neural network model outputs a cranial nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relationship of eye movement activity characteristics-brain structure-fatigue manifestations, and the fatigue manifestations are changes in eye movement activity characteristics when cranial nerves are fatigued.

[0032] In this embodiment, the visual brain nerve fatigue assessment method is executed on the execution subject (for example Figure 1 The server or terminal device shown in the figure can visually guide the evaluation object to perform a variety of eye movement tests, which can not only complete the standardized guidance of various eye movement behaviors, but also analyze various features, map them to the structure of the brain through neural networks, and then locate the mechanism of neural fatigue, and perform a deep, three-dimensional and reliable evaluation of fatigue. This embodiment uses standardized guidance and feature collection of multi-dimensional eye movements, combined with a neural network model to perform an in-depth evaluation of neural fatigue, thereby quantifying the fatigue evaluation results, improving the accuracy and reliability of the evaluation, and realizing comprehensive, systematic and efficient fatigue monitoring.

[0033] In some optional implementations of this embodiment, in the process of visually guiding the evaluation object to perform multiple eye movement tests, multi-dimensional eye movement features of the evaluation object are collected, including:

[0034] In the process of visually guiding the evaluation object to perform pupil light reflex test, forward saccade test, reverse saccade test and convergence test, the eye movement characteristics corresponding to the evaluation object in each test are collected respectively to obtain multi-dimensional eye movement characteristics of the evaluation object, and the multi-dimensional eye movement characteristics include pupil light reflex characteristics, forward saccade characteristics, reverse saccade characteristics and convergence reaction characteristics.

[0035] In a specific implementation, visual guidance is performed on the evaluation object, the eye image of the evaluation object is captured, and feature extraction is performed in the eye image to obtain multi-dimensional eye movement characteristics of the evaluation object; wherein the visual guidance includes: light spot guidance of eye saccades, quantitative change guidance of light, and change in depth distance of the guide object.

[0036] In some optional implementations of this embodiment, the evaluation object can be visually guided and multi-dimensional eye movement feature collection can be performed through a virtual reality device. The device can display content in a virtual reality (VR) manner, and has an infrared video module that can capture and record images of the tester's eyes. Virtual reality devices, such as VR helmets with eye movement capture modules, present comprehensive visual guidance through virtual reality technology, and can record real-time eye movement features such as independent gaze directions of both eyes, eye movements, pupil changes, etc.

[0037] This embodiment conducts eye movement tests on the subject of virtual reality (VR) assessment, and specifically completes quantitative standardized measurement and analysis of multiple types of eye activities through light spot guidance of eye saccades, quantitative changes in light, and changes in the depth distance of the guide. Then, the brain structure and function are mapped using a feature extraction method based on neural mechanisms, further reflecting the impact of fatigue on the brain.

[0038] In a specific implementation, the eye image of the evaluation object is captured by a pupil-cornea reflection method based on infrared, and feature extraction is performed in the eye image to obtain multi-dimensional eye movement features of the evaluation object.

[0039] The infrared pupil and corneal reflection method (IPCC) is a method that uses an infrared light source and a high-sensitivity camera to measure the pupil and corneal reflection of the eye. This method is an effective tool for ophthalmic diagnosis and visual research. By accurately measuring the pupil and corneal reflection, it can provide important information for eye health and visual function.

[0040] This embodiment proposes a test set that can comprehensively and quickly reflect the fatigue state, covering pupil light reflex, forward saccade, reverse saccade, and convergence, specifically including:

[0041] (1) The subject is located in the device collection area, completes the calibration of binocular gaze direction, and enters the test process. The calibration method can be the common 5-point or 9-point calibration method.

[0042] (2) First, a pure black background is presented to the subject for a period of time to allow the subject's pupil to adapt to the current light conditions. Then, white light flashes with a fixed intensity, and the characteristics of the pupil light stimulation are recorded.

[0043] (3) A target bright spot with a diameter of 1.5° is displayed on a pure black background to guide eye movement, and the subjects are required to move their eyes as quickly as possible to follow the bright spot. The bright spot first appears in the center of the visual field and stays there for a short time, then flashes to a fixed position in a random direction in the upper, lower, left, or right direction, stays there, and then disappears. After 2 seconds, it reappears in the center position, and continues to repeat several times. The characteristics of eye saccades during this process are recorded.

[0044] (4) Continue to display a target bright spot with a diameter of 1.5° on a pure black background to guide eye movement, and ask the subject to move the eyeballs as quickly as possible to observe the bright spot. The bright spot first appears in the center of the visual field and stays for a short time, then flashes to a fixed position in a random direction of up, down, left, or right, stays there, and then disappears. At this time, the subject should look in the opposite direction of the bright spot. After 2 seconds, it will reappear in the center position. Continue to repeat several times, and record the characteristics of reverse eye movement during this process.

[0045] (5) Using the display method of virtual reality, a red three-dimensional ball is presented in a simulated three-dimensional space. The ball will flash randomly within a range of 20 cm to 100 cm in front of the subject's eyes, and each position will last for 1 second. After 1 second, it will continue to flash at random positions, and continue to repeat several times. The subject needs to actively focus through the convergence reaction. This process is the convergence test.

[0046] The above four tasks comprehensively cover the entire visual-eye movement pathway of the human body. The pupil light stimulation response corresponds to the contraction and expansion of the pupil to light, the positive saccade reflects the function of eye movement (passive execution), the reverse saccade is related to the field of vision and the active execution of eye movement, and the convergence test corresponds to the eye's perception of depth information. Guided by virtual reality technology, it not only ensures the uniformity of content (the full-field display ensures that the intensity of the stimulus light is consistent for everyone), but also ensures the rich visual content presentation of two-dimensional (plane information) + three-dimensional space (the existence of depth information), and avoids the compensation problem of people unconsciously moving their heads during eye movement.

[0047] During the process of guiding eye movements, the subjects completed a total of four tasks, during which a large number of characteristic parameters were collected. The details are as follows:

[0048] (1) In the pupil light reflex test, in the pupil light stimulation reaction, the average pupil size before the light appears is collected as the baseline value, and the time it takes for the pupil to contract when the strong light appears is recorded as the latency; the maximum speed mcv and the maximum contraction value mc during the contraction process are recorded; the time it takes for the pupil to return to stability is recorded, and the duration is st; the pupil size after stabilization is recorded as sa and the difference from the baseline value sid.

[0049] (2) In the forward saccade test, based on the changes in the eye gaze direction from the center to the surrounding area, the saccade velocity sv, saccade eye movement latency sl, saccade task completion time sf, and task accuracy sar were recorded and collected.

[0050] (3) In the antisaccade test, the antisaccade latency al and antisaccade accuracy ac are recorded based on the change in the direction of eye gaze from the central position to the opposite direction of the surrounding area.

[0051] (4) In the convergence test, based on the process of the eyes fixating on a red ball at a random position, the convergence velocity cv, convergence latency cl, and convergence judgment accuracy ca are recorded and collected.

[0052] Finally, the basic information of the subjects, namely gender, age, education level, vision, etc., was recorded.

[0053] In some optional implementations of this embodiment, the multi-dimensional eye movement features are input into a fatigue assessment neural network model, and the fatigue assessment neural network model outputs a cranial nerve fatigue assessment value, including:

[0054] The fatigue assessment neural network model determines the neural fatigue status of each functional structure of the brain based on the correlation between the multi-dimensional eye movement characteristics and the functional structure of the brain and according to the changes in the multi-dimensional eye movement characteristics;

[0055] According to the neural fatigue status of each functional structure of the brain, the brain neural fatigue assessment value is output.

[0056] In specific implementation, the fatigue assessment neural network model is constructed based on the mesh mapping relationship between eye movement features, brain structure and fatigue performance. First, the mesh mapping relationship between eye movement features, brain structure and fatigue performance needs to be established, as follows:

[0057] (1) Eye movement characteristics:

[0058] a. Pupillary light reflex: The speed and amplitude of the pupil's reaction to light stimulation are mainly regulated by the oculomotor nucleus of the brainstem.

[0059] b. Positive saccade: A rapid eye movement that shifts the focus from one location to another, primarily controlled by the frontal and parietal eye areas of the cerebral cortex.

[0060] c. Countersaccade: The ability to inhibit the impulse of autonomous eye movements and turn to the opposite direction of the stimulus, reflecting the brain's inhibitory control ability and is mainly related to the frontal cortex.

[0061] D. Vergence: The ability of both eyes to coordinately focus on objects at different distances, which is mainly related to the convergence center in the midbrain.

[0062] (2) Brain structure:

[0063] a. Brainstem: Responsible for basic life functions, including pupillary light reflex.

[0064] b. Frontal lobe: The center of higher cognitive functions, including decision-making, planning, executive control and inhibitory control, and is closely related to saccades and antisaccades.

[0065] c. Parietal lobe: integrates sensory information, guides spatial attention and eye movements, and is closely related to eye movements.

[0066] d. Midbrain: coordinates eye movements, including convergence.

[0067] (3) Fatigue symptoms:

[0068] a. Pupillary light reflex: Fatigue may cause the latency of the pupillary light reflex to be prolonged, the contraction speed to be slowed down, and the contraction amplitude to be reduced.

[0069] b. Eye saccades: Fatigue may cause eye saccades to slow down in speed, decrease in accuracy, and prolong in latency.

[0070] c. Countersaccade: Fatigue may lead to an increase in the error rate of countersaccades.

[0071] d. Convergence: Fatigue may lead to slower convergence speed, decreased accuracy, and longer latency period.

[0072] After determining the mesh mapping relationship between eye movement characteristics, brain structure and fatigue performance, a fatigue assessment neural network model is established. The neural network model needs to determine the neural fatigue of each functional structure of the brain corresponding to each eye movement feature according to the changes in the multi-dimensional eye movement characteristics; and then output the final brain neural fatigue assessment value according to the neural fatigue of each functional structure of the brain, so as to achieve the purpose of standardizing, comprehensive and quantifying the fatigue assessment results.

[0073] In some optional implementations of the present embodiment, the fatigue assessment neural network model includes: an input layer, a hidden layer, a shared layer, a task-specific layer and an output layer; the input layer is used to receive the multi-dimensional eye activity feature values ​​of the assessment object after standardization, the hidden layer is used to extract and learn key features in the input data based on the mesh mapping relationship, the key features are the remaining features after eliminating abnormal feature data, the shared layer is used to extract common features, and the common features include basic information of the assessment object, the task-specific layer is used to determine the neural fatigue status of each functional structure of the brain according to changes in eye activity characteristics of different dimensions, and the output layer is used to output brain neural fatigue assessment values ​​according to the neural fatigue status of each functional structure of the brain.

[0074] In the specific implementation, this embodiment establishes an artificial intelligence neural network model for fatigue assessment. Considering the comprehensive multi-dimensional input parameters and multiple probability results, the solution uses a multi-task learning (MTL) architecture. The specific details are as follows:

[0075] (1) Parameter standardization: All collected eye movement feature parameters are organized into a consistent format and standardized so that the network can process them effectively.

[0076] (2) Configure the network:

[0077] a. Input layer: Design an input layer that can receive all the parameters that have been standardized in the above steps, such as latency, mcv, sf, etc.

[0078] b. Hidden layers and shared layers: Multiple hidden layers are configured to extract and learn key features in the data. Some layers are used as shared layers to extract common features. Common features in the shared layers include basic information of the subjects, such as age and gender. The hidden layers contain the mapping relationship between eye movement features, brain structure, and fatigue performance.

[0079] c. Task-specific layer: A specific layer is configured for fatigue, which is used to determine the neural fatigue of each functional structure of the brain based on the changes in eye movement characteristics in different dimensions.

[0080] d. Output layer: Outputs the brain neural fatigue assessment value based on the neural fatigue status of each functional structure of the brain.

[0081] In some optional implementations of the present embodiment, a softmax activation function is used in the output layer to output the fatigue probability of the functional structure of the brain associated with different eye movement characteristics; the fatigue probability of the functional structure of the brain associated with different eye movement characteristics is subjected to principal component analysis to obtain a comprehensive brain nerve fatigue assessment value.

[0082] The Softmax activation function is a commonly used function in machine learning and deep learning. It is used in the output layer of a neural network to convert the raw scores output by the neural network into a probability distribution. The output of the Softmax function is a normalized probability distribution, that is, the sum of all output values ​​is 1. The above principal component analysis is a dimensionality reduction technique that can combine the fatigue probability values ​​of multiple tasks into a comprehensive indicator for evaluating the overall fatigue situation.

[0083] In a specific implementation, the fatigue probabilities of the different eye movement features are subjected to principal component analysis to obtain a comprehensive fatigue assessment result, including:

[0084] (1) Standardizing data: standardizing the fatigue probability values ​​of the functional structures of the brain associated with the different eye movement characteristics to obtain standardized data;

[0085] (2) Calculating the covariance matrix and eigendecomposition: Calculating the covariance matrix of the standardized data, performing eigendecomposition on the covariance matrix, and obtaining eigenvectors and eigenvalues;

[0086] (3) Selecting the principal component: selecting a main eigenvector from the eigenvectors according to the eigenvalues, projecting the standardized data onto the main eigenvectors, and obtaining a comprehensive fatigue assessment value.

[0087] Finally, the model is trained: typical fatigue population and normal people are selected to conduct fatigue tests to complete the training of the fatigue assessment neural network model.

[0088] After the model is built, it is embedded in the neural fatigue assessment system, and a comprehensive and objective analysis of neural fatigue can be performed through four simple tasks in a virtual reality scene. After the subject completes all the tasks, the model can directly give an output value - the overall probability of fatigue (between 0-100%).

[0089] A method for assessing cranial nerve fatigue in this embodiment is guided by virtual reality technology (including the presentation of visual information, the guidance of eye movements, the change of light, and the change of the depth distance of the guide object), and collects eye information (including gaze direction, eye movement, pupil diameter change, and convergence activity) through an infrared-based pupil corneal reflection method, and integrates all parameters of eye information to establish a multi-parameter model to give the assessment results. This proposal solves the inefficiency problem of traditional methods in the past, and also solves the problem of single evaluation dimension and one-sided judgment basis of new innovative solutions in recent years, improves the accuracy and reliability of the assessment, and realizes fast and accurate fatigue monitoring and assessment.

[0090] Further references Figure 3 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a brain nerve fatigue assessment device, which is similar to Figure 2 Corresponding to the method embodiment shown in FIG. 1 , in addition to the features described below, the device embodiment may also include Figure 2 The device can be applied to various electronic devices.

[0091] like Figure 3As shown, the brain nerve fatigue assessment device 300 of this embodiment includes: an eye movement feature acquisition module 301 and a brain nerve fatigue assessment module 302. The eye movement feature acquisition module 301 is used to collect the multi-dimensional eye movement features of the assessment object in the process of visually guiding the assessment object to perform various eye movement tests, and the multi-dimensional eye movement features are controlled by different functional structures of the brain. The brain nerve fatigue assessment module 302 is used to input the multi-dimensional eye movement features into the fatigue assessment neural network model, and the fatigue assessment neural network model outputs the brain nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on the mesh mapping relationship of eye movement activity features-brain structure-fatigue performance, and the fatigue performance is the change of eye movement activity features when the brain nerves are fatigued.

[0092] In this embodiment, the specific processing of the eye movement feature acquisition module 301 and the cranial nerve fatigue assessment module 302 of the cranial nerve fatigue assessment device 300 and the technical effects thereof can be referred to in Figure 2 The relevant descriptions of step 201 and step 202 in the corresponding embodiment are not repeated here.

[0093] In some optional implementations of the present embodiment, the eye movement feature collection module 301 collects multi-dimensional eye movement features of the evaluation object during the process of visually guiding the evaluation object to perform pupil light reflex test, forward saccade test, reverse saccade test and convergence test. The multi-dimensional eye movement features include pupil light reflex features, forward saccade features, reverse saccade features and convergence reaction features.

[0094] In some optional implementations of the present embodiment, the fatigue assessment neural network model in the brain nerve fatigue assessment module 302 is based on the correlation between the multi-dimensional eye movement characteristics and the functional structure of the brain, and determines the neural fatigue status of each functional structure of the brain according to the changes in the multi-dimensional eye movement characteristics; and outputs a brain nerve fatigue assessment value according to the neural fatigue status of each functional structure of the brain.

[0095] Among them, the fatigue assessment neural network model includes: an input layer, a hidden layer, a shared layer, a task-specific layer and an output layer; the input layer is used to receive the multi-dimensional eye activity feature values ​​of the assessment object after standardization processing, the hidden layer is used to extract and learn the key features in the input data based on the mesh mapping relationship, the key features are the remaining features after eliminating abnormal feature data, the shared layer is used to extract common features, and the common features include basic information of the assessment object, the task-specific layer is used to determine the neural fatigue status of each functional structure of the brain according to the changes in eye activity characteristics of different dimensions, and the output layer is used to output the brain neural fatigue assessment value according to the neural fatigue status of each functional structure of the brain.

[0096] In some optional implementations of the present embodiment, a softmax activation function is used in the output layer to output the fatigue probability of the functional structure of the brain associated with different eye movement characteristics; the fatigue probability of the functional structure of the brain associated with different eye movement characteristics is subjected to principal component analysis to obtain a comprehensive brain nerve fatigue assessment value.

[0097] Among them, the fatigue probability values ​​of the functional structures of the brain associated with the different eye movement characteristics are standardized to obtain standardized data; the covariance matrix of the standardized data is calculated, and the covariance matrix is ​​eigendecomposed to obtain eigenvectors and eigenvalues; according to the eigenvalues, the main eigenvectors are selected from the eigenvectors, and the standardized data are projected onto the main eigenvectors to obtain a comprehensive brain nerve fatigue assessment value.

[0098] In some optional implementations of this embodiment, the eye movement feature acquisition module 301 may be a virtual reality device, which visually guides the evaluation object to perform a variety of eye movement tests and acquire the multi-dimensional eye movement features.

[0099] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0100] like Figure 4 , is a block diagram of an electronic device according to a method for assessing cranial nerve fatigue according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0101] like Figure 4 As shown, the electronic device includes: one or more processors 401, a memory 402, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to an interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 401 is taken as an example.

[0102] The memory 402 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to execute the cranial nerve fatigue assessment method provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to execute the cranial nerve fatigue assessment method provided in the present application.

[0103] The memory 402 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the cranial nerve fatigue assessment method in the embodiment of the present application (for example, the attached Figure 3 The processor 401 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 402, that is, the brain nerve fatigue assessment method in the above method embodiment is implemented.

[0104] The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device of the cranial nerve fatigue assessment method, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely arranged relative to the processor 401, and these remote memories may be connected to the electronic device of the cranial nerve fatigue assessment method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0105] The electronic device of the cranial nerve fatigue assessment method may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403 and the output device 404 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0106] The input device 403 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device of the cranial nerve fatigue assessment method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 404 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0107] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0109] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0110] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0111] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The units involved in the embodiments described in the present application can be implemented by software or hardware. The described units can also be set in a processor. For example, it can be described as: a processor includes an eye movement feature acquisition module and a cranial nerve fatigue assessment module. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances. For example, the eye movement feature acquisition module unit can also be described as a "data acquisition module".

[0114] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the device, the device: guides through virtual reality technology (including the presentation of visual information, the guidance of eye movements, the change of light, the change of the depth distance of the guide object), collects eye information (including gaze direction, eye movement, pupil diameter change, convergence activity) through an infrared-based pupil corneal reflection method, and fuses all parameters of the eye information to establish a multi-parameter model to give an evaluation result.

[0115] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A method for evaluating cranial nerve fatigue, characterized in that: include: In the process of visually guiding the evaluation subject to perform a variety of eye movement tests, collecting multi-dimensional eye movement characteristics of the evaluation subject, wherein the multi-dimensional eye movement characteristics are controlled and realized by different functional structures of the brain; The multi-dimensional eye movement characteristics are input into a fatigue assessment neural network model, and the fatigue assessment neural network model outputs a cranial nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relationship of eye movement activity characteristics-brain structure-fatigue manifestations, and the fatigue manifestations are changes in eye movement activity characteristics when cranial nerves are fatigued.

2. The cranial nerve fatigue assessment method according to claim 1, characterized in that: In the process of visually guiding the evaluation subject to perform a variety of eye movement tests, the multi-dimensional eye movement characteristics of the evaluation subject are collected, including: In the process of visually guiding the evaluation object to perform pupil light reflex test, forward saccade test, reverse saccade test and convergence test, the eye movement characteristics corresponding to the evaluation object in each test are collected respectively to obtain multi-dimensional eye movement characteristics of the evaluation object, and the multi-dimensional eye movement characteristics include pupil light reflex characteristics, forward saccade characteristics, reverse saccade characteristics and convergence reaction characteristics.

3. The cranial nerve fatigue assessment method according to claim 1, characterized in that: Inputting the multi-dimensional eye movement features into a fatigue assessment neural network model, and the fatigue assessment neural network model outputting a cranial nerve fatigue assessment value, includes: The fatigue assessment neural network model determines the neural fatigue status of each functional structure of the brain based on the correlation between the multi-dimensional eye movement characteristics and the functional structure of the brain and according to the changes in the multi-dimensional eye movement characteristics; According to the neural fatigue status of each functional structure of the brain, the brain neural fatigue assessment value is output.

4. The cranial nerve fatigue assessment method according to claim 3, characterized in that: The fatigue assessment neural network model comprises: an input layer, a hidden layer, a shared layer, a task-specific layer and an output layer; The input layer is used to receive the multi-dimensional eye movement feature values ​​of the evaluation object after standardization. The hidden layer is used to extract and learn the key features in the input data based on the mesh mapping relationship. The key features are the remaining features after eliminating abnormal feature data. The shared layer is used to extract common features, and the common features include basic information of the evaluation object. The task-specific layer is used to determine the neural fatigue status of each functional structure of the brain according to the changes in the eye movement characteristics of different dimensions. The output layer is used to output the brain neural fatigue evaluation value according to the neural fatigue status of each functional structure of the brain.

5. The cranial nerve fatigue assessment method according to claim 4, characterized in that: The method further comprises: A softmax activation function is used in the output layer to output fatigue probabilities of brain functional structures associated with different eye movement features; The fatigue probability of the functional structure of the brain associated with the different eye movement characteristics is subjected to principal component analysis to obtain a comprehensive cranial nerve fatigue assessment value.

6. The cranial nerve fatigue assessment method according to claim 5, characterized in that: The fatigue probability of the functional structure of the brain associated with the different eye movement characteristics is subjected to principal component analysis to obtain a comprehensive cranial nerve fatigue assessment value, including: Standardizing the fatigue probability values ​​of the functional structures of the brain associated with the different eye movement characteristics to obtain standardized data; Calculating the covariance matrix of the standardized data, performing eigendecomposition on the covariance matrix to obtain eigenvectors and eigenvalues; According to the eigenvalues, a main eigenvector is selected from the eigenvectors, and the standardized data is projected onto the main eigenvector to obtain a comprehensive cranial nerve fatigue assessment value.

7. The cranial nerve fatigue assessment method according to claim 1, characterized in that: In the process of visually guiding the evaluation subject to perform a variety of eye movement tests, the multi-dimensional eye movement characteristics of the evaluation subject are collected, including: The virtual reality device is used to visually guide the evaluation subject to perform a variety of eye movement tests and collect the multi-dimensional eye movement characteristics.

8. A device for assessing cranial nerve fatigue, characterized in that: include: An eye movement feature acquisition module, used to acquire multi-dimensional eye movement features of the evaluation subject during the process of visually guiding the evaluation subject to perform various eye movement tests, wherein the multi-dimensional eye movement features are controlled and realized by different functional structures of the brain; A cranial nerve fatigue assessment module is used to input the multi-dimensional eye movement characteristics into a fatigue assessment neural network model, and the fatigue assessment neural network model outputs a cranial nerve fatigue assessment value, wherein the fatigue assessment neural network model is a neural network model constructed based on a mesh mapping relationship between eye movement activity characteristics-brain structure-fatigue manifestations, and the fatigue manifestations are changes in eye movement activity characteristics when cranial nerves are fatigued.

9. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images