Brain function evaluation method and system based on fNIRS and eye movement feature fusion

Abstract

The invention provides a brain function evaluation method and system based on fNIRS and eye movement feature fusion, and the method comprises the steps: testing a to-be-tested person under a pre-constructed social scene normal form, and obtaining test data which comprises functional near infrared spectrum data and eye movement data; extracting time dynamic characteristics of brain function activation and brain function network space organization mode characteristics from the functional near infrared spectrum data, and constructing a near infrared brain function characteristic set; for a near-infrared brain function feature set corresponding to the functional near-infrared spectrum data and an eye movement space-time feature set corresponding to the eye movement data, constructing features in each near-infrared brain function feature set and features in the eye movement space-time feature set into feature pairs, and calculating a correlation coefficient of each feature pair, feature fusion and screening are realized based on correlation coefficients; and inputting the screened features into a classification evaluation model to obtain the severity of the behavioral dysfunction and a brain function analysis value.

Description

Technical Field

[0001] The present invention relates to the technical field of brain function rehabilitation assessment, and in particular to a brain function assessment method and system based on the fusion of fNIRS and eye movement features. Background Art

[0002] Autism, also known as autism, is a subtype of autism spectrum disorder (ASD). Its core disorder characteristics include social interaction disorders, language communication disorders and repetitive stereotyped behaviors. It is the most representative disease among children's pervasive developmental disorders.

[0003] The development of neuroimaging technology provides objective technical means for the diagnosis and assessment of children with ASD. Among them, magnetic resonance imaging and electroencephalography are often used to assess abnormal brain function in ASD. However, in clinical practice, due to the weak ability of these imaging technologies to resist motion artifacts, most studies are limited to resting-state assessments of children with ASD and are not suitable for recording brain functional activities under specific task paradigms. In addition, ASD children have serious comorbidities, especially those involving functional impairments including speech, cognitive control, and social interaction. Therefore, a single indicator is not enough to reveal the abnormal brain function characteristics of children with autism. In addition, most studies only focus on ASD children and healthy controls, and have not yet established a connection with the severity of ASD functional impairments. It is difficult to describe the abnormal brain function characteristics of ASD children with different severities under task performance, and thus cannot provide effective guidance for clinical rehabilitation assessment and intervention. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a brain function evaluation method based on the fusion of fNIRS and eye movement features to eliminate or improve one or more defects in the prior art.

[0005] One aspect of the present invention provides a method for evaluating brain function based on the fusion of fNIRS and eye movement features, the method comprising the following steps:

[0006] Testing the subject under a pre-constructed social scenario task paradigm to obtain test data, wherein the test data includes functional near-infrared spectroscopy data and eye movement feature data;

[0007] Extracting temporal dynamics characteristics of functional activation and spatial organizational pattern characteristics of brain functional networks from the functional near-infrared spectroscopy data, and constructing the temporal dynamics characteristics and the spatial organizational pattern characteristics of brain functional networks into a near-infrared brain functional feature set, wherein the temporal dynamics characteristics of functional activation include the average intensity and dynamic change rate of functional activity, and the organizational pattern characteristics of brain functional networks include modularity, small-world characteristics, and hemispheric asymmetry characteristics of the corresponding undirected weighted functional network;

[0008] For the near-infrared brain function feature set corresponding to the functional near-infrared spectroscopy data and the eye movement spatiotemporal feature set corresponding to the eye movement feature data, constructing feature pairs with the features in each near-infrared brain function feature set and the features in the eye movement spatiotemporal feature set, calculating the correlation coefficient of each feature pair, and screening features based on the correlation coefficient;

[0009] The features in the screened near-infrared brain function feature set and the features in the eye movement spatiotemporal feature set are input into the classification evaluation model to obtain the severity of behavioral dysfunction and brain function analysis values.

[0010] Adopting the above scheme, this scheme first tests the subjects under a pre-constructed social scene task paradigm to collect initial functional near-infrared spectral data and eye movement feature data, and obtains multiple features by analyzing the two types of data, and constructs them into near-infrared brain function feature sets and eye movement spatiotemporal feature sets. The features in the two feature sets are then constructed into feature pairs, and the features that can effectively characterize the brain function characteristics are analyzed and screened pair by pair. Finally, the severity of behavioral dysfunction and brain function analysis value are given through a classification evaluation model. This scheme can provide an effective reference for clinical rehabilitation evaluation and intervention through brain function analysis value.

[0011] In some embodiments of the present invention, the subject is tested under a pre-constructed social scenario task paradigm, and in the step of obtaining test data, a near-infrared brain functional imaging device equipped with a 40-channel fNIRS acquisition head cap is used to cover the bilateral prefrontal, temporal, parietal and occipital brain regions of the subject based on a 10-10 positioning system, and hemodynamic response information is collected as functional near-infrared spectral data at a sampling frequency of 10 Hz; a desktop eye tracker is used to track the subject's eye movements through an infrared camera to record the subject's eye movement feature data during the test.

[0012] In some embodiments of the present invention, the eye movement feature data includes gaze point position, gaze time and scan path.

[0013] In some embodiments of the present invention, a subset of fNIRS brain function features and eye movement features that are significantly correlated with the functional status of the subject is extracted to achieve preliminary feature screening; in the step of constructing feature pairs from each feature in the near-infrared brain function feature set and the features in the eye movement spatiotemporal feature set, and calculating the correlation coefficient of each feature pair:

[0014] Calculating an objective function value based on the features in the feature pair and determining an optimal coefficient vector using an alternating minimization method;

[0015] Calculate the correlation coefficient of the feature pair based on the optimal coefficient vector.

[0016] In some embodiments of the present invention, the step of obtaining the near-infrared brain function feature set corresponding to the functional near-infrared spectral data and the eye movement spatiotemporal feature set corresponding to the eye movement feature data also includes extracting functional near-infrared spectral data and eye movement feature data that are significantly correlated with the functional state of the subject to achieve preliminary feature screening.

[0017] In some embodiments of the present invention, in the step of calculating the objective function value based on the features in the feature pair and determining the optimal coefficient vector using the alternating minimization method, the objective function value is calculated based on the following formula:

[0018]

[0019] Among them, y represents the objective function value, C XY Represents the covariance matrix constructed by the X feature and Y feature in the feature pair, λ1 and λ2 are regularization parameters, ||α||1 and ||β||1 are the L1 norms of the optimal coefficient vector α and the optimal coefficient vector β respectively, α T represents the transpose of the optimal coefficient vector α.

[0020] In some embodiments of the present invention, in the step of calculating the correlation coefficient of the feature pair based on the optimal coefficient vector, the correlation coefficient is calculated using the following formula:

[0021] ρ=α T C XY β;

[0022] Where ρ represents the correlation coefficient.

[0023] In some embodiments of the present invention, in the step of screening features based on correlation coefficients, for two features in a feature pair, after the correlation coefficients are calculated, the dimensions of the values in each feature are adjusted a preset number of times, and the correlation coefficients are calculated again a preset number of times. The correlation coefficients calculated again are compared with the correlation coefficients calculated for the first time to determine whether the features in the feature pair are screened as calculated features.

[0024] In some embodiments of the present invention, in the step of comparing the recalculated correlation coefficient with the first calculated correlation coefficient to determine whether the feature in the feature pair is screened as the calculated feature, the difference between the recalculated correlation coefficient and the first calculated correlation coefficient is calculated, and the absolute value is calculated. If the absolute value is greater than a preset threshold, the feature in the feature pair is screened as the calculated feature.

[0025] In some embodiments of the present invention, in the step of extracting the temporal dynamic characteristics of functional activation and the spatial organizational pattern characteristics of the brain functional network from the functional near-infrared spectroscopy data, the average amplitude of each fNIRS channel time series data in the functional near-infrared spectroscopy data over the entire time series is calculated as the average intensity of functional activity; the rate of change of the fNIRS signal amplitude at adjacent time points in the functional near-infrared spectroscopy data is calculated as the dynamic rate of change; Pearson correlation analysis is used to calculate the values between fNIRS channels and construct a brain functional network matrix. For the brain functional network matrix, a graph theory algorithm is used to calculate the values of modularity, small-world characteristics, and hemispheric asymmetry characteristics.

[0026] In some embodiments of the present invention, in the step of calculating the average amplitude of each fNIRS channel time series data in the functional near-infrared spectroscopy data over the entire time series as the average intensity of the functional activity, the average amplitude is calculated using the following formula: in, represents the average amplitude, T represents the number of time points, and X(t) represents the amplitude at time point t. In the step of calculating the rate of change of the fNIRS signal amplitude at adjacent time points in the functional near-infrared spectroscopy data as the dynamic change rate, the dynamic change rate is calculated using the following formula: Where R(t) represents the dynamic change rate at time point t, and X(t+1) represents the amplitude at time point t+1.

[0027] A second aspect of the present invention further provides a brain function evaluation system based on the fusion of fNIRS and eye movement features, the system comprising a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, the processor being configured to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps implemented by the method described above.

[0028] The third aspect of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned brain function evaluation method based on the fusion of fNIRS and eye movement features.

[0029] Additional advantages, objects, and features of the present invention will be described in part in the following description and will become apparent to those skilled in the art after studying the following or may be learned by practice of the present invention. The objects and other advantages of the present invention may be particularly pointed out and attained in the description and drawings.

[0030] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0032] Figure 1 Schematic diagram of an embodiment of the brain function evaluation method based on the fusion of fNIRS and eye movement features of the present invention;

[0033] Figure 2 Schematic diagram of the processing architecture of the brain function evaluation method based on the fusion of fNIRS and eye movement features of the present invention;

[0034] Figure 3 Schematic diagram of the overall architecture of the brain function evaluation method based on the fusion of fNIRS and eye movement features of the present invention;

[0035] Figure 4 This is the framework diagram for calculating the multi-dimensional spatiotemporal features of this scheme;

[0036] Figure 5 This is the feature analysis framework diagram for this solution;

[0037] Figure 6 This is a schematic diagram of the processing flow of the classification and evaluation model for this solution;

[0038] Figure 7 A schematic diagram of the final evaluation results. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0040] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.

[0041] like Figure 1-3 As shown, the present invention proposes a brain function evaluation method based on the fusion of fNIRS and eye movement features, the method comprising the following steps:

[0042] Step S100, testing the subject under a pre-constructed social scenario task paradigm to obtain test data, wherein the test data includes functional near-infrared spectroscopy data and eye movement feature data;

[0043] Specifically, the pre-constructed social scenario task paradigm adopts the social scenario task paradigm related to the functional impairment of the test subjects.

[0044] Step S200, extracting temporal dynamics characteristics of functional activation and spatial organizational pattern characteristics of brain functional networks from the functional near-infrared spectroscopy data, and constructing the temporal dynamics characteristics and the spatial organizational pattern characteristics of brain functional networks into a near-infrared brain functional feature set, wherein the temporal dynamics characteristics of functional activation include the average intensity and dynamic change rate of functional activity, and the organizational pattern characteristics of brain functional networks include modularity, small-world characteristics, and hemispheric asymmetry characteristics of the corresponding undirected weighted functional network;

[0045] Specifically, in an undirected weighted functional network, the edges of the network (i.e., connections between brain regions) are non-directional, indicating that the functional connectivity between two nodes (such as fNIRS channels) is bidirectional and symmetrical. For example, the Pearson correlation coefficient between channel A and channel B is equivalent to the correlation coefficient between channel B and channel A, regardless of direction.

[0046] like Figure 4 Specifically, before step S200, the method further includes preprocessing the functional near-infrared spectroscopy data to obtain a preprocessed dataset, including filtering and removing head motion artifacts and physiological noise interference. A 0.01-0.1 Hz bandpass filter is used to remove low-frequency noise related to heartbeat and respiration, as well as high-frequency measurement noise. A time derivative repair distribution algorithm is used to correct head motion artifacts and improve the signal-to-noise ratio. After preprocessing, a preprocessed dataset of functional near-infrared spectroscopy data for each acquisition channel is obtained. In the preprocessing of the eye movement feature data, the raw data is segmented based on the different stimulus contents in the face recognition task, obtaining eye movement data segments under different stimulus windows after segmentation.

[0047] Step S300, for the near-infrared brain function feature set corresponding to the functional near-infrared spectral data and the eye movement spatiotemporal feature set corresponding to the eye movement feature data, constructing feature pairs with the features in each near-infrared brain function feature set and the features in the eye movement spatiotemporal feature set, calculating the correlation coefficient of each feature pair, and screening features based on the correlation coefficient;

[0048] Specifically, in the step of extracting functional near-infrared spectroscopy data and eye movement feature data that are significantly correlated with the functional status of the subjects and realizing preliminary feature screening, the features in the near-infrared brain function feature set and the eye movement spatiotemporal feature set are preliminarily selected. For the near-infrared brain function feature set and the eye movement spatiotemporal feature set, the fNIRS brain function feature and eye movement feature subsets that are significantly correlated with the autism functional status (CARS scale score) are extracted through correlation analysis. The features with weak explanatory power for the target variable are preliminarily removed, and the preliminary dimensionality reduction of the indicator features is performed; the feature fusion regression analysis is performed as follows: Figure 4 shown.

[0049] Specifically, the sparse canonical correlation analysis, based on the aforementioned preliminary feature dimensionality reduction, uses sparse canonical correlation analysis to perform a fusion analysis between brain function features and eye movement behavior performance, determining the correlation between each brain function feature and eye movement behavior performance. The selected features are sorted by correlation, and significant eye movement behavior parameters and brain function features are selected to determine the subset of fNIRS brain function features and eye movement fusion features related to the functions of children with autism. This completes feature screening and constructs a multi-feature subset.

[0050] Specifically, the brain function characteristics and eye movement behavior characteristics data were prepared, and the related feature matrices were vectorized and converted into feature vectors. Among them, the dimension of each subject's near-infrared brain function spatiotemporal characteristics is p, so the brain function feature matrix is X(n×p); the dimension of each subject's eye movement spatiotemporal characteristics is q, so the eye movement behavior feature matrix is Y(n×p), where n represents the number of subjects.

[0051] Specifically, the brain function feature matrix X(n×p) and the eye movement behavior feature matrix Y(n×p) are standardized so that the mean of each feature is 0 and the variance is 1, eliminating the influence of different feature scales.

[0052] Specifically, brain function and eye movement behavior feature data were prepared by vectorizing the relevant feature matrices and converting them into feature vectors. The dimension of each subject's near-infrared brain function spatiotemporal features is p, resulting in a brain function feature matrix X(n×p); the dimension of each subject's eye movement spatiotemporal features is q, resulting in an eye movement behavior feature matrix Y(n×p), where n represents the number of subjects.

[0053] Step S400: Input the filtered features in the near-infrared brain function feature set and the features in the eye movement spatiotemporal feature set into a classification evaluation model to obtain the severity of behavioral dysfunction and brain function analysis value.

[0054] The severity of the behavioral dysfunction is determined based on the brain function analysis value. Specifically, the severity of the behavioral dysfunction is determined by comparing a threshold with the brain function analysis value.

[0055] In the specific implementation process, the classification evaluation model adopts the KNN algorithm, takes the filtered features as the feature subset as input, uses the autism severity scale score as the category label, and determines the optimal K value through ten-fold cross validation to construct a classification evaluation model;

[0056] Specifically, first perform Z-score normalization on each feature data x in the feature subset Where x is the original eigenvalue, μ is the mean of the feature, and σ is the standard deviation of the feature;

[0057] like Figure 6 As shown in the figure, specifically, in the multi-feature space, the Euclidean distance is used to measure the similarity between samples. For two m-dimensional samples W = (w1, w2, ..., w m ) and U=(u1,u2,…,u m ), based on the calculated Euclidean distance, sort the distances and find the K samples closest to it (K nearest neighbor samples). Classification decisions are made using a weighted average method, with the inverse of the distance as the weight. Classification decisions are made based on the weighted number of category occurrences. The samples to be classified are classified into the category with the highest number of occurrences. Using the multi-feature subsets and corresponding labels in the training set, the model is trained according to the KNN algorithm steps. Through ten-fold cross-validation, the model parameters are continuously adjusted, and the model accuracy, precision, and recall rate indicators are calculated to evaluate the model classification performance. The optimal K value is determined, the model is trained and optimized, and a classification evaluation model is constructed.

[0058] Specifically, the autism severity scale score is used as the category label. According to the Child Autism Rating Scale (CARS, total score 60 points), a total score of less than 30 points is defined as a category without autism; a total score in the range of 30-37 points is defined as a category with mild to moderate autism; and a total score in the range of 37-60 points is defined as a category with severe autism.

[0059] Adopting the above scheme, this scheme first tests the subjects under a pre-constructed social scene task paradigm to collect initial functional near-infrared spectral data and eye movement feature data, and obtains multiple features by analyzing the two types of data, and constructs them into near-infrared brain function feature sets and eye movement spatiotemporal feature sets. The features in the two feature sets are then constructed into feature pairs, and the features that can effectively characterize the brain function characteristics are analyzed and screened pair by pair. Finally, the brain function analysis value is given through the classification evaluation model. This scheme can provide an effective reference for clinical rehabilitation evaluation and intervention through brain function analysis values.

[0060] In some embodiments of the present invention, the subject is tested under a pre-constructed social scenario task paradigm, and in the step of obtaining test data, a near-infrared brain functional imaging device equipped with a 40-channel fNIRS acquisition head cap is used to cover the bilateral prefrontal, temporal, parietal and occipital brain regions of the subject based on a 10-10 positioning system, and hemodynamic response information is collected as functional near-infrared spectral data at a sampling frequency of 10 Hz; a desktop eye tracker is used to track the subject's eye movements through an infrared camera to record the subject's eye movement feature data during the test.

[0061] In its implementation, this protocol uses a face recognition task as a social context paradigm. The core functional impairments of children with autism are manifested in social interaction, primarily in face recognition, emotion perception, and attention to faces. Therefore, by presenting different types of faces, the brain function characteristics and eye movement patterns of the children are simultaneously observed during this process. This task paradigm is highly compatible and suitable for children with varying severity levels. Therefore, the face recognition task used in this protocol as a functional assessment paradigm has broader applicability.

[0062] Specifically, the child sits upright on a chair at an appropriate height and distance, with his hands placed naturally. After the "start" command, pictures of faces including familiar faces (such as parents, caregivers), unfamiliar faces, and different types of faces with different emotions (joy, anger, sadness, happiness, etc.) are displayed in random order on the desktop monitor. The pictures are presented on the screen one by one, each presentation lasting 5 seconds, and a total of 120 face pictures are randomly presented.

[0063] Specifically, the near-infrared functional brain imaging device is equipped with a multi-channel fNIRS acquisition head cap, which covers the bilateral prefrontal, temporal, parietal, and occipital regions of interest based on the 10-10 system and collects hemodynamic response information at a frequency of 10 Hz;

[0064] Specifically, eye tracking technology uses a desktop eye tracker to track eye movements through an infrared camera, recording eye movement data such as the gaze point position, gaze duration, and scanning path of children when they view facial images.

[0065] In some embodiments of the present invention, the eye movement feature data includes gaze point position, gaze time and scan path.

[0066] In some embodiments of the present invention, in the step of constructing a feature pair from each feature in the near-infrared brain function feature set and a feature in the eye movement spatiotemporal feature set, and calculating the correlation coefficient of each feature pair:

[0067] Calculating an objective function value based on the features in the feature pair and determining an optimal coefficient vector using an alternating minimization method;

[0068] Calculate the correlation coefficient of the feature pair based on the optimal coefficient vector.

[0069] In some embodiments of the present invention, in the step of calculating the objective function value based on the features in the feature pair and determining the optimal coefficient vector using the alternating minimization method, the objective function value is calculated based on the following formula:

[0070]

[0071] Among them, y represents the objective function value, C XY Represents the covariance matrix constructed by the X feature and Y feature in the feature pair, λ1 and λ2 are regularization parameters, ||α||1 and ||β||1 are the L1 norms of the optimal coefficient vector α and the optimal coefficient vector β respectively, α T represents the transpose of the optimal coefficient vector α.

[0072] Specifically, by alternately minimizing this method, we can find α and β respectively and bring them into the objective function to maximize them.

[0073] In some embodiments of the present invention, in the step of calculating the correlation coefficient of the feature pair based on the optimal coefficient vector, the correlation coefficient is calculated using the following formula:

[0074] ρ=α T C XY β;

[0075] Where ρ represents the correlation coefficient.

[0076] like Figure 5 As shown, in some embodiments of the present invention, in the step of screening features based on correlation coefficients, for two features in a feature pair, after the correlation coefficients are calculated, the dimensions of the values in each feature are adjusted a preset number of times, and the correlation coefficients are calculated again a preset number of times. The correlation coefficients calculated again are compared with the correlation coefficients calculated for the first time to determine whether the features in the feature pair are screened as calculated features.

[0077] Specifically, the above steps involve performing a statistical significance analysis on the actual calculated correlation coefficient and the correlation coefficient calculated after scrambling (for example, scrambling 1000 times will produce 1000 correlation coefficients). If the actual calculated value is significantly greater than the distribution of the 1000 values, it is considered significant.

[0078] Using the above scheme, a permutation test method was used to identify significantly correlated canonical variable features for extracting a fusion feature subset. First, the original data was randomly permuted multiple times to disrupt the corresponding relationships, and the canonical correlation coefficient was recalculated to obtain a null distribution. The actual calculated canonical correlation coefficient was compared with the null distribution. If the actual value was significantly greater than the value in the null distribution, the corresponding canonical variable pair was considered to have a significant correlation. Indicators that clearly showed a significant correlation between brain function and eye movement behavior parameters could be used as fusion feature subsets to ensure the relevance of feature screening.

[0079] In some embodiments of the present invention, in the step of comparing the recalculated correlation coefficient with the first calculated correlation coefficient to determine whether the feature in the feature pair is screened as the calculated feature, the difference between the recalculated correlation coefficient and the first calculated correlation coefficient is calculated, and the absolute value is calculated. If the absolute value is greater than a preset threshold, the feature in the feature pair is screened as the calculated feature.

[0080] In some embodiments of the present invention, in the step of extracting the temporal dynamic characteristics of functional activation and the spatial organizational pattern characteristics of the brain functional network from the functional near-infrared spectroscopy data, the average amplitude of each fNIRS channel time series data in the functional near-infrared spectroscopy data over the entire time series is calculated as the average intensity of functional activity; the rate of change of the fNIRS signal amplitude at adjacent time points in the functional near-infrared spectroscopy data is calculated as the dynamic rate of change; Pearson correlation analysis is used to calculate the values between fNIRS channels and construct a brain functional network matrix. For the brain functional network matrix, a graph theory algorithm is used to calculate the values of modularity, small-world characteristics, and hemispheric asymmetry characteristics.

[0081] like Figure 4 As shown, in the specific implementation process, modularization utilizes the Louvain algorithm to divide the modules in the network by optimizing the modularity (Q value). The higher the Q value, the more significant the modularity of the functional network. The small-world characteristic analysis is described by calculating the average path length (L) and clustering coefficient (C) of the network. Among them, the average path length refers to the average value of the shortest path length between any two nodes in the network, reflecting the efficiency of information propagation in the network; the clustering coefficient measures the degree of interconnection between the neighboring nodes of the node, reflecting the local connection characteristics of the network. Among them, hemispheric asymmetry defines the hemispheric autonomy coefficient based on the difference between the intra-hemispheric connection and the inter-hemispheric connection of the functional connection network N i represents the sum of the functional connectivity strengths between channel i and the ipsilateral hemisphere; N c represents the sum of functional connections between channel i and the contralateral hemisphere; T i With T cThey represent the sum of functional connections of the ipsilateral and contralateral hemispheres respectively; the HA value can be used to reflect the imbalance of functional connections within and between hemispheres. The larger the HA, the stronger the connection strength of the channel and within the hemisphere is than that between the hemispheres. By comparing the HA sizes of the left and right hemispheres, the cortical lateralization can be indirectly reflected.

[0082] In some embodiments of the present invention, in the step of calculating the average amplitude of each fNIRS channel time series data in the functional near-infrared spectroscopy data over the entire time series as the average intensity of the functional activity, the average amplitude is calculated using the following formula: in, represents the average amplitude, T represents the number of time points, and X(t) represents the amplitude at time point t. In the step of calculating the rate of change of the fNIRS signal amplitude at adjacent time points in the functional near-infrared spectroscopy data as the dynamic change rate, the dynamic change rate is calculated using the following formula: Where R(t) represents the dynamic change rate at time point t, and X(t+1) represents the amplitude at time point t+1.

[0083] Furthermore, the mean and standard deviation of the functional near-infrared spectroscopy data were further calculated to characterize the response of the functional activity changes of the channel during the monitoring period.

[0084] Using the above scheme, the average amplitude reflects the overall activity intensity level of the fNIRS channel during the monitoring period.

[0085] In the specific implementation process, the eye movement feature data is segmented and processed to extract the gaze behavior characteristics of the eye movement information under different face tasks, and the behavior patterns of eye movements in time and space dimensions are analyzed;

[0086] Among them, the eye movement spatial characteristics include the distribution of gaze point positions; by dividing the face into facial area and non-facial area, the facial area is further divided into eye, nose, mouth and ear interest areas, and the number of gaze points falling within the interest area is analyzed in a targeted manner to characterize the spatial distribution characteristics of the eye movement gaze point position.

[0087] The eye movement time features include fixation duration, saccade velocity, and blink frequency. Fixation duration refers to the length of time the eyes remain fixed on a given area; saccade velocity refers to the time required to complete a face scan, and changes in saccade duration can reflect the speed and efficiency of an individual's information acquisition. Blink frequency refers to the number of blinks per unit time and can be used to indicate the degree of attentional focus.

[0088] Through the above analysis, we obtain the spatiotemporal feature set of eye movements under different facial stimulus contents. Specifically, the eye movement information features include the multi-stimulus task-spatiotemporal feature set under the stimulation of familiar faces, unfamiliar faces and faces with different emotions.

[0089] Specifically, after the brain function analysis value is finally calculated, the functional status of the child is determined by comparison and a multi-dimensional spatiotemporal feature visualization is performed, such as Figure 5 、 6 As shown in Figure 7. Visual displays are used to provide feedback on the temporal and spatial characteristics of brain function and eye movement information under the task paradigm. The temporal and spatial characteristics of brain function are displayed in the form of functional activation and bar graphs. Eye movement behavior characteristics are displayed by drawing eye movement hotspot maps, using color depth to indicate the density of fixations, highlighting the key areas of individual attention, and visually displaying the eye movement characteristics of autistic children for feedback to medical staff and guardians.

[0090] Specifically, during actual use, assessors can use the model output scores and specific brain function characteristics and eye movement information characteristics as a reference to conduct a comprehensive and objective assessment of the functional status of autistic children, and can longitudinally track the effectiveness of rehabilitation training and make real-time adjustments.

[0091] In summary, the present invention uses face recognition tasks as a social function assessment paradigm, based on multi-channel near-infrared brain function covering the bilateral prefrontal, temporal, occipital and parietal lobes and non-contact eye tracking equipment, and effectively integrates multi-level brain function and eye movement information by extracting multi-dimensional spatiotemporal features and fusion regression analysis methods, to achieve effective assessment of the functional status of children with autism, and provide a method for assessors and guardians to develop personalized rehabilitation training plans.

[0092] An embodiment of the present invention also provides a brain function evaluation system based on the fusion of fNIRS and eye movement features. The system includes a computer device, the computer device including a processor and a memory, the memory storing computer instructions, and the processor being configured to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented in the method described above.

[0093] Embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the steps of the aforementioned method for evaluating brain function based on the fusion of fNIRS and eye movement features. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0094] It should be understood by those skilled in the art that the various exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0095] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0096] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0097] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations to the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A brain function evaluation method based on the fusion of fNIRS and eye movement features, characterized in that: The steps of the method include: Testing the subject under a pre-constructed social scene task paradigm to obtain test data, wherein the test data includes functional near-infrared spectroscopy data and eye movement feature data; Extracting temporal dynamics characteristics of functional activation and spatial organizational pattern characteristics of brain functional networks from the functional near-infrared spectroscopy data, and constructing the temporal dynamics characteristics and the spatial organizational pattern characteristics of brain functional networks into a near-infrared brain functional feature set, wherein the temporal dynamics characteristics of functional activation include the average intensity and dynamic change rate of functional activity, and the organizational pattern characteristics of brain functional networks include modularity, small-world characteristics, and hemispheric asymmetry characteristics of the corresponding undirected weighted functional network; For the near-infrared brain function feature set corresponding to the functional near-infrared spectroscopy data and the eye movement spatiotemporal feature set corresponding to the eye movement feature data, constructing feature pairs with the features in each near-infrared brain function feature set and the features in the eye movement spatiotemporal feature set, calculating the correlation coefficient of each feature pair, and screening features based on the correlation coefficient; The features in the screened near-infrared brain function feature set and the features in the eye movement spatiotemporal feature set are input into the classification evaluation model to obtain the severity of behavioral dysfunction and brain function analysis values.

2. The brain function evaluation method based on fNIRS and eye movement feature fusion according to claim 1, characterized in that: The subjects were tested under a pre-constructed social scenario task paradigm. In the step of obtaining test data, a near-infrared brain functional imaging device equipped with a 40-channel fNIRS acquisition head cap was used to cover the bilateral prefrontal, temporal, parietal and occipital brain regions of the subjects based on the 10-10 positioning system, and hemodynamic response information was collected as functional near-infrared spectral data at a sampling frequency of 10 Hz; a desktop eye tracker was used to track the subject's eye movements through an infrared camera, and the subject's eye movement characteristic data during the test was recorded.

3. The brain function evaluation method based on fNIRS and eye movement feature fusion according to claim 1, characterized in that: In the step of constructing feature pairs from each near-infrared brain function feature set and the features from the eye movement spatiotemporal feature set, and calculating the correlation coefficient of each feature pair: Calculating an objective function value based on the features in the feature pair and determining an optimal coefficient vector using an alternating minimization method; Calculate the correlation coefficient of the feature pair based on the optimal coefficient vector.

4. The brain function evaluation method based on fNIRS and eye movement feature fusion according to claim 3, characterized in that: The steps for the near-infrared brain function feature set corresponding to the functional near-infrared spectral data and the eye movement spatiotemporal feature set corresponding to the eye movement feature data also include extracting functional near-infrared spectral data and eye movement feature data that are significantly correlated with the functional state of the subject to achieve preliminary feature screening.

5. The brain function evaluation method based on fNIRS and eye movement feature fusion according to claim 4, characterized in that: In the step of calculating the correlation coefficient of the feature pair based on the optimal coefficient vector, the correlation coefficient is calculated using the following formula: p=a T C XY b; Where ρ represents the correlation coefficient.

6. The brain function evaluation method based on fNIRS and eye movement feature fusion according to any one of claims 1 to 5, characterized in that: In the step of filtering features based on correlation coefficients, for the two features in a feature pair, after the correlation coefficients are calculated, the dimension where the values in each feature are located is adjusted a preset number of times, and the correlation coefficients are calculated again a preset number of times. The correlation coefficients calculated again are compared with the correlation coefficients calculated for the first time to determine whether the features in the feature pair are filtered as calculated features.

7. The brain function evaluation method based on fNIRS and eye movement feature fusion according to claim 6, characterized in that: In the step of comparing the recalculated correlation coefficient with the first calculated correlation coefficient to determine whether the feature in the feature pair is screened as the calculated feature, the difference between the recalculated correlation coefficient and the first calculated correlation coefficient is calculated, and the absolute value is calculated. If the absolute value is greater than a preset threshold, the feature in the feature pair is screened as the calculated feature.

8. The brain function evaluation method based on fNIRS and eye movement feature fusion according to claim 1, characterized in that: In the step of extracting the temporal dynamic characteristics of functional activation and the spatial organizational pattern characteristics of the brain functional network from the functional near-infrared spectroscopy data, the average amplitude of the time series data of each fNIRS channel in the functional near-infrared spectroscopy data over the entire time series is calculated as the average intensity of functional activity; the rate of change of the fNIRS signal amplitude at adjacent time points in the functional near-infrared spectroscopy data is calculated as the dynamic rate of change; Pearson correlation analysis is used to calculate the values between fNIRS channels and construct a brain functional network matrix. For the brain functional network matrix, a graph theory algorithm is used to calculate the values of modularity, small-world characteristics, and hemispheric asymmetry characteristics.

9. The brain function evaluation method based on fNIRS and eye movement feature fusion according to claim 8, characterized in that: For each fNIRS channel time series data in the functional near-infrared spectroscopy data, the average amplitude over the entire time series is calculated as the average intensity of the functional activity. The average amplitude is calculated using the following formula: in, represents the average amplitude, T represents the number of time points, and X(t) represents the amplitude at time point t. In the step of calculating the rate of change of the fNIRS signal amplitude at adjacent time points in the functional near-infrared spectroscopy data as the dynamic change rate, the dynamic change rate is calculated using the following formula: Where R(t) represents the dynamic change rate at time point t, and X(t+1) represents the amplitude at time point t+1.

10. A brain function evaluation system based on the fusion of fNIRS and eye movement features, characterized in that: The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions. The processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented by the method according to any one of claims 1 to 9.

Images