A method for evaluating the subject's sustained attention state based on fNIRS and EEG dual modality

Abstract

The present application provides a method for evaluating the sustained attention state of a subject based on fNIRS and EEG dual modalities, and the method relates to the technical field of sustained attention state evaluation. The method includes presenting a sequence of a first predetermined number of numbers on the screen in sequence in a manner that each number is presented on the screen individually for a first predetermined time; receiving a confirmation operation performed by the subject to identify the target sequence; obtaining the brain fNIRS blood oxygen data and electroencephalogram data of the subject during a second predetermined time period before and after the presentation of each target sequence; and evaluating the sustained attention state of the subject based on the acquired brain fNIRS blood oxygen data of the subject and the electroencephalogram data of the subject. In this way, the sustained attention state of the subject is evaluated by combining the brain fNIRS blood oxygen data and electroencephalogram data, and the evaluation result has high reliability, which helps to provide a more accurate reference for the diagnosis of clinical mental illness.

Description

Technical Field

[0001] The present application relates to the technical field of sustained attention state assessment, and in particular to a method for assessing a subject's sustained attention state based on fNIRS and EEG dual modalities. Background Art

[0002] Patients with attention deficit hyperactivity disorder (ADHD) have obvious attention impairment. In behavioral studies, the correct rate or reaction time of behavioral responses in sustained attention tasks has a greater degree of differentiation for ADHD patients. However, behavioral response indicators (correct rate or reaction time of behavioral responses) are easily affected by subjective factors. Behavioral response indicators often rely on the subjective reports or behavioral performance of the subjects, and are easily affected by factors such as the subject's level of consciousness, emotional state, and motivation, thereby generating introspective bias and reducing the accuracy of assessing the subject's sustained attention state.

[0003] At present, technologies that can directly measure brain activity include near-infrared functional brain imaging technology (fNIRS) and electroencephalogram (EEG). Among them, the blood oxygen signal collected based on fNIRS is a slow signal, which generally requires a longer interval between stimulations; the EEG signal collected based on EEG is a fast signal, which requires the evaluation of the EEG signal within 1 second caused by each stimulus, and at the same time requires random jitter in the time interval between different stimuli. Therefore, related technologies often use fNIRS or EEG single-modality measurement methods to evaluate the subject's sustained attention state, and cannot combine fNIRS and EEG to evaluate the subject's sustained attention state, and the accuracy of the evaluation needs to be improved. Summary of the invention

[0004] This application is proposed to solve the above technical problems in the prior art. This application aims to provide a method for evaluating the sustained attention state of a subject based on fNIRS and EEG dual modalities, which can obtain fNIRS and EEG dual modal brain function biological indicators by asking the subject to complete the fNIRS and EEG dual modal task paradigm, and based on the biological indicators, it is helpful to improve the reliability and accuracy of the evaluation results of the sustained attention state of the subject.

[0005] According to a first scheme of the present application, a method for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities is provided, wherein the subject may include an ADHD patient or a subject with a tendency to attention deficit, and the method comprises using a processor to perform the following steps: presenting a sequence of a first predetermined number of numbers on a screen in sequence in a manner in which each number is presented on the screen individually for a first predetermined time, wherein a target sequence of preset numbers appearing successively presents a number of numbers randomly selected from a range of 4-30 between each other, and the first predetermined time is selected within a range of 600±100ms; receiving a confirmation operation performed by the subject to identify the target sequence; acquiring fNIRS blood oxygen data and electroencephalogram data of the subject's brain during a second predetermined time period before and after the presentation of each target sequence; and evaluating the subject's sustained attention state based on the acquired fNIRS blood oxygen data of the subject's brain and the electroencephalogram data of the subject.

[0006] According to a second scheme of the present application, a device for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities is provided, wherein the subject may include an ADHD patient or a subject with a tendency to attention deficit. The device includes a processor, and the processor is configured to execute a method for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities as described in various embodiments of the present application.

[0007] According to the third scheme of the present application, a computer program product is provided, which evaluates the sustained attention state of a subject based on fNIRS and EEG dual modalities. The subject may include an ADHD patient or a subject with attention deficit tendency. The computer program product includes computer executable instructions. When the computer executable instructions are executed by a processor, a method for evaluating the sustained attention state of a subject based on fNIRS and EEG dual modalities as described in each embodiment of the present application is implemented.

[0008] According to the fourth scheme of the present application, a system for evaluating the sustained attention state of a subject based on fNIRS and EEG dual modalities is provided, the system comprising: a brain fNIRS acquisition module, for acquiring brain fNIRS blood oxygen data of the subject; an EEG acquisition module, for acquiring EEG data of the subject; an interface, which is configured to acquire the brain fNIRS blood oxygen data of the subject from the brain fNIRS acquisition module, and the EEG data of the subject from the EEG acquisition module; and a processor, which is configured to execute the method for evaluating the sustained attention state of a subject based on fNIRS and EEG dual modalities as described in each embodiment of the present application.

[0009] Compared with the prior art, the beneficial effects of the embodiments of the present application are:

[0010] The method for evaluating the sustained attention state of the subject based on fNIRS and EEG dual modality provided in the embodiment of the present application is to present a sequence of a first predetermined number of numbers on the screen in sequence in a manner that each number is presented on the screen separately for a first predetermined time, and the target sequence of preset numbers that appear successively presents a number of numbers randomly selected from the range of 4-30, and the presentation time of each number is 600±100ms. In this way, the time interval between the target sequences can allow the fNIRS signal caused by the target sequence before the current time to be restored to the baseline level to a certain extent when the current target sequence is presented. In addition, since the time interval between the target sequences is randomly changed, the fNIRS signal also has good statistical validity in the subsequent analysis stage; on the other hand, the EEG data can be used to analyze the EEG response caused by each target sequence, and the time interval between the target sequences can also meet the extraction requirements of the EEG index. In this way, in the process of the subject performing the fNIRS and EEG dual modality task paradigm, the fNIRS blood oxygen data of the subject's brain and the EEG data of the subject can be obtained, thereby realizing the extraction of fNIRS and EEG dual modality brain function data.

[0011] Specific lesions in patients with mental illness may appear in differences in brain functional activities in different brain regions, or in differences in neural patterns of rapid processing of stimuli. The brain fNIRS blood oxygen data obtained by the method provided in the embodiment of the present application can reflect the activity status of brain regions related to sustained attention. Combined with the real-time neural activity status reflected by the subject's EEG data, it can improve the accuracy and reliability of the assessment of the subject's sustained attention state. The sustained attention state assessed by the method of the present application can be used as a highly specific indicator for ADHD patients or those with a tendency to attention deficit, thereby providing a reliable reference for doctors to diagnose mental illness in subjects.

[0012] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above description and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. Similar reference numerals with letter suffixes or different letter suffixes may represent different examples of similar components. The accompanying drawings generally illustrate various embodiments by way of example and not by way of limitation, and together with the description and claims, are used to illustrate the disclosed embodiments. Such embodiments are illustrative and exemplary and are not intended to be exhaustive or exclusive embodiments of the present method or non-transitory computer-readable medium having instructions for implementing the method.

[0014] Figure 1 A flowchart of a method for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities according to an embodiment of the present application is shown.

[0015] Figure 2 It shows various presentation interfaces displayed to the subject through the screen within 5 minutes according to an embodiment of the present application.

[0016] Figure 3 A flow chart of evaluating a subject's sustained attention state based on brain fNIRS blood oxygen characteristics and EEG data characteristics according to an embodiment of the present application is shown.

[0017] Figure 4 A schematic diagram of the structure of a system for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation examples. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation examples, but are not intended to limit the present application.

[0019] The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish. The words "include" or "comprises" and similar words used in this application mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of covering other elements. In this application, the arrows shown in the figures of each step are only examples of the execution order, not limitations. The technical solution of this application is not limited to the execution order described in the embodiments. The various steps in the execution order can be combined, decomposed, or the order can be changed, as long as the logical relationship of the execution content is not affected.

[0020] All terms (including technical terms or scientific terms) used in this application have the same meaning as those understood by ordinary technicians in the field to which this application belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined here. Techniques and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, the techniques and equipment should be considered as part of the specification.

[0021] Figure 1 A flowchart of a method for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities according to an embodiment of the present application is shown. Specifically, a processor is used to execute the method shown in steps S101 to S104. In this embodiment, the arrows shown in the figure for each step are merely examples of the execution order, not limitations. The technical solution of the present application is not limited to the execution order described in the embodiment. The steps in the execution order can be combined, decomposed, or the order can be changed, as long as the logical relationship of the execution content is not affected.

[0022] Figure 2 The various presentation interfaces displayed to the subject through the screen within 5 minutes according to the embodiment of the present application are shown below. Figure 2 Provide specific instructions.

[0023] In this embodiment, the subject includes an ADHD patient or a subject with an attention deficit tendency. Common mental illnesses associated with attention deficit, such as learning disabilities, autism spectrum disorders, post-traumatic stress disorder, etc., are only used as examples and do not constitute a limitation on the specific scheme. The task type of the fNIRS and EEG dual-modality sustained attention task in this application can be a rapid visual information processing (RVP) task.

[0024] In step S101, a sequence of a first predetermined number of numbers is presented on the screen in sequence in such a manner that each number is presented on the screen individually for a first predetermined time, wherein a target sequence in which preset numbers appear successively presents a number of numbers randomly selected from the range of 4-30 between each other, and the first predetermined time is selected within the range of 600±100ms.

[0025] The screen may be from a smart phone, a tablet computer, a desktop computer or other visualization device, and is not limited thereto. The number presented on the screen may be any one of 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, and 15, and the presentation order of the numbers presented on the screen may be set by the user or matched according to the information of the subject.

[0026] In this embodiment, each number may be presented individually in the first area on the screen. For example, only one single-digit number may be presented individually at a time in the first area on the screen, and each number may be presented individually for a first predetermined time before the next number is presented.

[0027] There is no limitation on the position where a single number is presented on the screen and the background color of the screen. For example, the number can be presented in the center, left or right position of the screen, and the background color of the screen can be black, dark blue or other dark colors. The user can set the presentation position of the number and the background color of the screen by himself, as long as the subject can easily observe the number and it will not distract the subject's attention.

[0028] In a preferred embodiment of the present application, a small frame with light lines is presented in the center of a black background, and the single number is presented in the center of the small frame with light lines, and the color of the single number is white. Using black as the background color reduces the visual impact of the background color on the examinee, reduces the degree of interference of the background color on the examinee's attention, and the single white number is in the center of the black background, which makes it easier for the examinee to notice the number.

[0029] Specifically, Figure 2 As shown, the background color of each presentation interface is a black background, and a small frame with light-colored lines is presented in the center. The area shown by the small frame with light-colored lines is the first area 202, and the colors of the numbers presented in the center of the small frame with light-colored lines are all white. A single white number "7" is presented in the center of the first area 202 of the first presentation interface 201. After the single number "7" is presented for 600ms, the next number set in sequence is presented. Moreover, the line of the single white number "7" is wider than the small frame with light-colored lines. In other embodiments, the line of the single number is 1.5 times wider than the small frame with light-colored lines to avoid distracting the subject's attention due to the excessively wide small frame with light-colored lines.

[0030] In this way, by setting a black background color, white numbers, and limiting the line width of the numbers, the acceptance of subjects with attention deficit tendencies can be improved, they are less likely to feel tired, and avoid being distracted by other content, which is conducive to allowing the subjects to maintain good concentration and complete the task.

[0031] In this embodiment, a sequence of a first predetermined number of numbers is sequentially presented on the screen, wherein the sequence includes a target sequence in which preset numbers appear in succession. In other embodiments, the target sequence is a sequence of numbers consisting of three numbers appearing in succession.

[0032] For example, the consecutive occurrences of "3, 5, 7" are preset as a target sequence, or the consecutive occurrences of "2, 4, 6" are preset as a target sequence, or the consecutive occurrences of "1, 4, 8" are preset as a target sequence. Figure 2 As shown, the numbers “3”, “5”, and “7” displayed in sequence on the second presentation interface 204 , the third presentation interface 205 , and the fourth presentation interface 206 are a target sequence.

[0033] This is only an example and does not constitute a specific limitation on the target sequence.

[0034] Setting the target sequence as a digital sequence consisting of three numbers appearing in succession is conducive to adjusting the fNIRS and EEG dual-modal sustained attention task to a difficulty suitable for the subjects by flexibly adjusting the three numbers appearing in succession according to the cognitive level of different subjects.

[0035] The target sequence can be set by the user. For example, the user can set the target sequence according to the subject's attention status. Specifically, if the doctor believes that the current subject's attention is poor, the target sequence can be set to a regular target sequence such as "1, 3, 5" or "2, 4, 6" to reduce the difficulty of the task, so that the fNIRS and EEG dual-modal sustained attention task that the subject is about to perform is adapted to the subject's actual condition.

[0036] For the target sequence in which preset numbers appear successively, the number of single-digit numbers between each other can be selected from the range of 4-30, so as to avoid that too many numbers increase the difficulty of the subject to perform the task, thereby causing reduced attention and inability to complete the task, and at the same time avoid that too few numbers cause the subject to perform the task too simply, thereby failing to obtain valid biological data.

[0037] Specifically, the target sequence presents 4, 15, 20 or 30 single-digit numbers between each other, which is not limited and can also be set by the user.

[0038] For example, the way in which numbers are displayed on the screen can be preset to a first mode, a second mode with a higher task difficulty than the first mode, and a third mode with a higher task difficulty than the second mode. For example, in the first mode, the target sequence presented on the screen presents 4-9 randomly selected single-digit numbers between each other, in the second mode, the target sequence presented on the screen presents 10-20 randomly selected single-digit numbers between each other, and in the third mode, the target sequence presented on the screen presents 21-30 randomly selected single-digit numbers between each other. This is only used as an example and does not constitute a limitation on the specific solution.

[0039] The doctor can use other medical methods to pre-judge the subject's sustained attention state. If the subject's sustained attention state is poor, the doctor can choose to configure the first mode for the subject so that the subject can adapt to the task intensity and complete the fNIRS and EEG dual-modal sustained attention task. This is only an exemplary description and does not constitute a limitation on the specific solution.

[0040] In some other embodiments, the dominant perspective of each number presentation interface presents the target sequence at a size smaller than the number. Figure 2 As shown, the second area 203 of the first presentation interface 201 displays a target sequence, and there are three target sequences, namely "3, 5, 7", "2, 4, 6" and "4, 6, 8". The second area 203 is located in the upper right corner of the first presentation interface 201, and the position of the second area 203 is the dominant perspective of the first presentation interface 201. The three target sequences are presented in the upper right corner of the first presentation interface 201 to reduce the working memory burden of the subject's brain and only call on the resources related to sustained attention in the subject's brain.

[0041] in, Figure 2 Only one dominant perspective display mode is shown, and the user can set the dominant perspective according to the subject's dominant hand or other habits. For example, assuming that the subject's dominant hand is the left hand, the dominant perspective can be set at the upper left of the presentation interface, or assuming that the subject's peripheral vision is more accustomed to focusing on the lower left, the dominant perspective can be set at the lower left of the presentation interface.

[0042] There is no limitation on this. Presenting the target sequence at the advantageous visual angle of the presentation interface is helpful for the examinee to see the target sequence with peripheral vision, thereby avoiding distraction.

[0043] In addition, Figure 2Taking the first presentation interface 201 as an example, the sizes of the numbers in the target sequences "3, 5, 7", "2, 4, 6" and "4, 6, 8" are smaller than the size of the single number "7" presented alone in the first area 202. In this way, the interference of the target sequence on the subject's attention can be further reduced.

[0044] and, Figure 2 The target sequences "3, 5, 7", "2, 4, 6" and "4, 6, 8" displayed in the second area 203 are displayed in an array manner, so that the distraction of the subject's attention by the target sequence can be further reduced, so that the subject can easily and quickly scan the entire target sequence through the peripheral vision. That is to say, as a preferred embodiment, while each number is presented separately on the black background, the target sequence is presented in an array manner in the dominant perspective of the black background.

[0045] In this embodiment, for a single assessment, the first predetermined number is 300-1200 numbers, the ratio of the target sequence is 10-40%, there are 1-4 target sequences, and the numbers in each target sequence have regularity, so that the requirement for sustained attention is lower than the random appearance of each number. By collaboratively setting the first predetermined number, the ratio of the target sequence and the number of target sequences, the time for the subject to perform the task is kept at 5-10 minutes, which can collaboratively ensure that the subject's sustained attention load is within an acceptable range, thereby making the acquired biological data have higher reliability.

[0046] With the development of relevant brain areas such as the frontal lobe, adults have better sustained attention than teenagers. Therefore, at least one of the first predetermined number, the ratio of the target sequence, and the number of target sequences can be adjusted according to the age of the subject. For example, for teenagers, the first predetermined number can be 300-700 numbers, the ratio of the target sequence can be 30%-40%, and the number of target sequences can be 1-2. For adult subjects, the first predetermined number can be 700-1200 numbers, the ratio of the target sequence can be 10%-30%, and the number of target sequences can be 3-4. In this way, the fNIRS and EEG dual-modal sustained attention tasks can be better adjusted to a difficulty that is more suitable for the subject.

[0047] Secondly, the numbers in the target sequence can be set regularly according to the age of the subject. For example, for adult subjects, the number difference in the target sequence can be set to 2, such as the target sequence is "3, 5, 7"; for adolescent subjects, the number difference in the target sequence can be set to 1, such as the target sequence is "1, 2, 3". This is conducive to maintaining the subject's high attention to complete the fNIRS and EEG dual-modal sustained attention task, and then obtaining reliable biological data indicators.

[0048] In addition, the numbers in the target sequence can be random, that is, they can be irregular. For example, the target sequence can be "1, 4, 9". However, for ADHD patients or subjects with impaired attention, the irregularity of the numbers in the target sequence will increase the requirements for the subjects' sustained attention, which is not conducive to the subjects completing the entire fNIRS and EEG dual-modal sustained attention task with high attention. When the numbers in the target sequence are regular, the requirements for the subjects' sustained attention are reduced, and the subjects are more likely to accept it. Therefore, the numbers in each target sequence can be set to have regularity, so that the requirements for sustained attention are lower than those for random numbers.

[0049] Back to this embodiment, in step S102, a confirmation operation performed by the subject after recognizing the target sequence is received. In other embodiments, the confirmation operation includes an indication operation performed on an indicator immediately after recognizing the tail of the target sequence. The indicator may be a touch screen, a remote controller, a button, a mouse, a voice input conversion device, or other device capable of transmitting action instructions, which is not limited thereto.

[0050] In some embodiments, Figure 2 As shown, after the first area 202 in the first presentation interface 201 presents the number "7" for 600ms, the second presentation interface 204, the third presentation interface 205 and the fourth presentation interface 206 are entered respectively, and the numbers "3", "5" and "7" are presented in the second presentation interface 204, the third presentation interface 205 and the fourth presentation interface 206 respectively. In the case where the subject recognizes that the number "7" displayed in the presentation interface 204 is the last digit of the target sequence "3, 5, 7", the confirmation operation can be performed by pressing a key, touching the current presentation interface 204 or other methods.

[0051] In this embodiment, only the first presentation interface 201 is described in detail, and the display methods of the other presentation interfaces are the same as the display method of the first presentation interface 201, and the display methods of the other presentation interfaces are not described in detail. Figure 2 The “…” in the textbox is used to indicate the remaining parts of the presentation interface that are not displayed.

[0052] In step S103, the fNIRS blood oxygen data and EEG data of the subject's brain are obtained during the second predetermined time period before and after the presentation of each target sequence. The second predetermined time period can be set according to the first predetermined time, without specific limitation. For example, assuming that the first predetermined time is 600ms, the second predetermined time period can be from 50-600ms before the presentation of each target sequence to 600ms after the presentation of each target sequence, which is only used as an exemplary illustration.

[0053] The time intervals between target sequences vary randomly, which can avoid interference from other interference factors on fNIRS blood oxygen data and EEG data of the brain.

[0054] In step S104, the subject's sustained attention state is evaluated based on the acquired fNIRS blood oxygen data and EEG data of the subject's brain. By setting a dual-modality sustained attention task suitable for fNIRS and EEG, the subject's brain fNIRS blood oxygen data and the subject's EEG data can be collected simultaneously, and the collected brain fNIRS blood oxygen data and EEG data can accurately reflect the brain function activity status and real-time neural activity status of attention-related brain areas.

[0055] By combining and analyzing the subject's brain fNIRS blood oxygen data and the subject's EEG data, the subject's sustained attention state can be accurately evaluated, and the evaluation results obtained after the evaluation can be used as a highly specific indicator for ADHD patients or those with a tendency to attention deficit.

[0056] In some other embodiments of the present application, the sustained attention state of the subject is evaluated based on the acquired fNIRS blood oxygen data of the subject's brain and the EEG data of the subject's target electrode position. For the fNIRS and EEG dual-modality head cap, the EEG electrode position used to collect EEG data and the probe position used to collect fNIRS blood oxygen data of the brain are mutually exclusive, and for different mental illnesses, due to differences in active brain areas, the setting positions of EEG electrodes are also very different. Therefore, the target electrode position can be predetermined based on prior knowledge of activated brain areas of ADHD patients or those with attention deficit tendencies.

[0057] Alternatively, the location of the activated brain area can be determined based on the acquired fNIRS blood oxygen data of the subject's brain, and the target electrode position can be determined based on the location of the activated brain area. In other words, the target electrode position is determined based on the acquired fNIRS blood oxygen data of the subject's brain. Specifically, the maximum channel position of the target sequence blood oxygen response can indicate the brain area related to sustained attention, and the target electrode position selected by the EEG data is selected based on this, so that specific EEG data reflecting sustained attention can be extracted, which is conducive to the accurate assessment of the sustained attention state.

[0058] In this way, the target electrode position can be determined in advance based on prior knowledge of activated brain areas in ADHD patients or those with attention deficit tendencies. The EEG data of the effective target electrode position can be used to evaluate the subject's sustained attention state in combination with the fNIRS blood oxygen data of the brain. Compared with processing the collected EEG data of each electrode position, the processing load of the processor can be reduced and the processing efficiency can be improved.

[0059] In some other embodiments of the present application, Figure 3 , the processor can evaluate the subject's sustained attention state by executing steps S301 to S304.

[0060] In step S301, a reference model is determined based on the convolution of the target sequence presentation time and the blood oxygen response kinetic function. Specifically, the blood oxygen response kinetic function can be obtained by pre-testing or simulation modeling, or reference can be made to the benchmark blood oxygen response kinetic function provided in "Modeling the hemodynamic response to brain activation" or "Prevalence of mental disorders in China: a cross-sectional epidemiological study".

[0061] In step S302, the degree of fit between the acquired brain fNIRS blood oxygen data and the reference model is determined as the brain fNIRS blood oxygen feature. In other words, the acquired brain fNIRS blood oxygen data is similarly fitted with the reference model, and the fit can be used as the brain fNIRS blood oxygen feature. The higher the fit, the more it tends to be activated.

[0062] In step S303, the EEG data within 50-600ms before each target sequence is presented is used as the baseline EEG data, and the EEG data within 600ms from the start of each target sequence to the end of the complete presentation is used as the target EEG data to extract the attribute characteristics of the change curve of the target EEG data relative to the baseline EEG data as the EEG data characteristics. In step S304, the subject's sustained attention state is evaluated based on the brain fNIRS blood oxygen characteristics in conjunction with the EEG data characteristics.

[0063] The attribute feature may be an EEG response amplitude feature, a phase feature, a spatial feature, etc., which is not limited. In this embodiment, the time period for acquiring EEG data and the time period for acquiring brain fNIRS blood oxygen data are substantially overlapped, and the brain fNIRS blood oxygen feature and the EEG data feature both reflect the brain response caused by the same stimulus.

[0064] The processor may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0065] In some embodiments of the present application, a device for evaluating the sustained attention state of a subject based on fNIRS and EEG dual modalities is provided, wherein the subject may include an ADHD patient or a subject with a tendency to attention deficit, and the device includes a processor, wherein the processor is configured to execute a method for evaluating the sustained attention state of a subject based on fNIRS and EEG dual modalities according to various embodiments of the present application. Based on this device, the fNIRS blood oxygen data of the subject's brain and the EEG data of the subject can be obtained during the subject's execution of fNIRS and EEG dual modal tasks, thereby realizing the extraction of fNIRS and EEG dual modal brain function data. Based on the evaluated sustained attention state, it can be used as a highly specific indicator for ADHD patients or those with a tendency to attention deficit, thereby providing a reliable reference for doctors to diagnose mental illness in subjects.

[0066] In some embodiments of the present application, a computer program product is provided, which evaluates the sustained attention state of a subject based on fNIRS and EEG dual modalities, and the subject can include an ADHD patient or a subject with a tendency to attention deficit, and the computer program product includes computer executable instructions, and when the computer executable instructions are executed by a processor, a method for evaluating the sustained attention state of a subject based on fNIRS and EEG dual modalities according to each embodiment of the present application is implemented. In this way, it is used to obtain fNIRS and EEG dual-modal brain function biological indicators, which are beneficial to improve the reliability and accuracy of the evaluation results of evaluating the sustained attention state of the subject based on the biological indicators.

[0067] In some embodiments of the present application, a system for evaluating the sustained attention state of a subject based on fNIRS and EEG dual modalities is provided, wherein the subject may include an ADHD patient or a subject with a tendency to attention deficit, specifically, Figure 4As shown, the system 400 includes a brain fNIRS acquisition module 401, an EEG acquisition module 402, an interface 403 and a processor 404, wherein the brain fNIRS acquisition module 401 is used to acquire the brain fNIRS blood oxygen data of the subject; the EEG acquisition module 402 is used to acquire the EEG data of the subject; the interface 403 is configured to acquire the brain fNIRS blood oxygen data of the subject from the brain fNIRS acquisition module, and the EEG data of the subject from the EEG acquisition module; and the processor 404 is configured to execute the method for evaluating the sustained attention state of the subject based on fNIRS and EEG dual modalities as described in various embodiments of the present application.

[0068] Based on the system 400, by comprehensively considering the characteristics of near-infrared signals and EEG signals, fNIRS and EEG dual-modal tasks can be provided. In the process of the subject performing fNIRS and EEG dual-modal tasks, the subject's brain fNIRS blood oxygen data and the subject's EEG data can be obtained, thereby realizing the extraction of fNIRS and EEG dual-modal brain function data to evaluate the subject's sustained attention state. In addition, the evaluated sustained attention state can be used as a highly specific indicator for ADHD patients or those with attention deficit tendencies.

[0069] This application describes various operations or functions that may be implemented as software codes or instructions or defined as software codes or instructions. Such content may be source code or differential code ("incremental" or "patch" code) that can be directly executed ("object" or "executable" form). Software codes or instructions may be stored in a computer-readable storage medium, and when executed, may cause a machine to perform the described functions or operations, and include any mechanism for storing information in a form accessible to a machine (e.g., a computing device, an electronic system, etc.), such as recordable or non-recordable media (e.g., read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

[0070] The exemplary methods described in this application can be implemented at least in part by a machine or computer. In some embodiments, a computer-readable storage medium stores computer program instructions, which, when executed by a processor, cause the processor to perform a method for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities as described in various embodiments of this application.

[0071] The implementation of such a method may include software code, such as microcode, assembly language code, high-level language code, etc. Various software programming techniques may be used to create various programs or program modules. For example, a program portion or program module may be designed with or by means of Java, Python, C, C++, assembly language, or any known programming language. One or more of such software portions or modules may be integrated into a computer system and / or a computer-readable medium. Such software code may include computer-readable instructions for executing various methods. The software code may form a part of a computer program product or a computer program module. In addition, in an example, the software code may be tangibly stored on one or more volatile, non-temporary or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media may include, but are not limited to, hard disks, removable disks, removable optical disks (such as optical disks and digital video disks), cassettes, memory cards or memory sticks, random access memories (RAMs), read-only memories (ROMs), etc.

[0072] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the execution of the application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the claims and the full scope of their equivalents.

[0073] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more schemes thereof) may be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features may be grouped together to simplify the present application. This should not be interpreted as an intention that a disclosed feature that is not required to be protected is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a particular disclosed embodiment. Thus, the claims are incorporated herein as examples or embodiments in the specific embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments may be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the attached claims and the full scope of equivalent forms granted by these claims.

[0074] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities, characterized in that: The subject may include an ADHD patient or a subject with attention deficit tendency. The method includes using a processor to execute the following steps: A sequence of a first predetermined number of numbers is sequentially presented on the screen in a manner that each number is presented on the screen individually for a first predetermined time, wherein a number of numbers randomly selected from a range of 4-30 are presented between each other in a target sequence of preset numbers, and the first predetermined time is selected within a range of 600±100 ms; receiving a confirmation operation performed by the subject when recognizing the target sequence; Acquiring fNIRS blood oxygen data and EEG data of the subject's brain during a second predetermined time period before and after presentation of each target sequence, thereby extracting fNIRS and EEG dual-modality brain function data; and The sustained attention state of the subject is evaluated based on the acquired fNIRS blood oxygen data of the subject's brain and the EEG data of the subject.

2. The method according to claim 1, characterized in that Based on the acquired fNIRS blood oxygen data of the subject's brain and the subject's EEG data, the subject's sustained attention state is evaluated, specifically including: The sustained attention state of the subject is evaluated based on the acquired fNIRS blood oxygen data of the subject's brain and the EEG data of the subject's target electrode position, wherein the target electrode position is determined according to any of the following methods: Determining a target electrode position based on the acquired fNIRS blood oxygen data of the subject's brain; The target electrode locations are predetermined based on prior knowledge of activated brain regions in ADHD patients or those prone to attention deficit.

3. The method according to claim 1, characterized in that The target sequence is a sequence of three numbers appearing in succession.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises, using the processor: The dominant visual angle in the presentation interface of each number presents the target sequence at a size smaller than the number.

5. The method according to any one of claims 1 to 3, characterized in that: The processor is further configured such that, for a single evaluation, the first predetermined number is 300-1200 numbers, the ratio of the target sequence is 10-40%, there are 1-4 target sequences, and the numbers in each target sequence are regular, so that the requirement for sustained attention is lower than that of random numbers.

6. The method according to any one of claims 1 to 3, characterized in that: The processor is further configured to: Determine a reference model based on the convolution of the target sequence presentation time and the blood oxygen response dynamics function; Determining the degree of fit between the acquired brain fNIRS blood oxygen data and the reference model as the brain fNIRS blood oxygen feature; The EEG data within 50-600ms before each target sequence is presented is used as the baseline EEG data, and the EEG data within 600ms from the start of each target sequence presentation to the end of the complete presentation is used as the target EEG data to extract the attribute characteristics of the change curve of the target EEG data relative to the baseline EEG data as the EEG data features; The sustained attention state of the subject is evaluated based on the fNIRS blood oxygen characteristics of the brain in conjunction with the EEG data characteristics.

7. The method according to any one of claims 1 to 3, characterized in that: The method of presenting the first predetermined time on the screen with each number separately specifically includes: A small frame with light lines is presented in the center of the black background, and a single number is presented in the center of the small frame with light lines, so that the line of the single number is 1.5 times wider than the small frame with light lines.

8. The method according to claim 7, characterized in that The processor is further configured to present the target sequence in the following manner: While each digit was presented individually on a black background, the target sequence was presented in an array in the dominant visual field against a black background.

9. The method according to any one of claims 1 to 3, characterized in that: The confirming operation includes performing an indicating operation on an indicator immediately after the tail of the target sequence is identified.

10. A device for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities, characterized in that: The subject may include an ADHD patient or a subject with a tendency to attention deficit, and the device includes a processor configured to execute a method for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities according to any one of claims 1-9.

11. A computer program product, characterized in that It evaluates the sustained attention state of the subject based on fNIRS and EEG dual modalities. The subject may include ADHD patients or subjects with attention deficit tendencies. The computer program product contains computer executable instructions. When the computer executable instructions are executed by a processor, a method for evaluating the sustained attention state of the subject based on fNIRS and EEG dual modalities according to any one of claims 1 to 9 is implemented.

12. A system for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities, characterized in that: The subject may include an ADHD patient or a subject with a tendency to attention deficit, and the system includes: The brain fNIRS acquisition module is used to collect the brain fNIRS blood oxygen data of the subject; An EEG acquisition module is used to collect EEG data of the subject; An interface configured to acquire brain fNIRS blood oxygen data of the subject from the brain fNIRS acquisition module and EEG data of the subject from the EEG acquisition module; and A processor configured to execute the method for evaluating a subject's sustained attention state based on fNIRS and EEG dual modalities according to any one of claims 1 to 9.

Images