A brain function evaluation method and system based on fNIRS and eye movement feature fusion

Abstract

The application provides a brain function evaluation method and system based on fNIRS and eye movement feature fusion, and the steps of the method comprise: testing a to-be-tested person under a pre-constructed social scene paradigm, and acquiring test data, wherein the test data comprises functional near-infrared spectroscopy data and eye movement data; extracting time dynamics features of brain function activation and brain function network spatial organization mode features from the functional near-infrared spectroscopy data, and constructing the features into a near-infrared brain function feature set; constructing each feature in each near-infrared brain function feature set and each feature in an eye movement space-time feature set corresponding to the eye movement data into a feature pair, calculating a correlation coefficient of each feature pair, and realizing feature fusion and screening based on the correlation coefficient; and inputting the screened features into a classification evaluation model to obtain a behavioral dysfunction severity and a brain function analysis value.

Description

Technical Field

[0001] This invention relates to the field of brain function rehabilitation assessment technology, and in particular to a brain function assessment method and system based on the fusion of fNIRS and eye movement features. Background Technology

[0002] Autism, also known as autism spectrum disorder, is a subtype of autism spectrum disorder (ASD). Its core impairment features include social interaction impairment, language communication impairment, and repetitive and stereotyped behaviors. It is one of the most representative diseases among pervasive developmental disorders in children.

[0003] The development of neuroimaging technology has provided objective technical means for the diagnosis and assessment of children with ASD. Among these, magnetic resonance imaging (MRI) and electroencephalography (EEG) are commonly used to assess brain function abnormalities in ASD. However, in clinical practice, due to the weak resistance to motion artifacts in these imaging techniques, most studies are limited to resting-state assessments of children with ASD and are not suitable for recording brain function activities under specific task paradigms. Furthermore, comorbidities are severe in children with ASD, especially involving functional impairments in speech, cognitive control, and social interaction; therefore, a single indicator is insufficient to reveal the abnormal brain function characteristics of children with autism. In addition, most studies only focus on children with ASD and healthy controls, without establishing a link with the severity of ASD functional impairment, making it difficult to describe the abnormal brain function characteristics of children with different degrees of ASD under task performance, and thus failing to provide effective guidance for clinical rehabilitation assessment and intervention. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a brain function evaluation method based on fNIRS and eye movement feature fusion to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present invention provides a brain function assessment method based on fNIRS and eye movement feature fusion, the method comprising the following steps:

[0006] Tests were conducted on test subjects within a pre-constructed social scenario task paradigm to obtain test data, which included functional near-infrared spectral data and eye-tracking feature data.

[0007] The temporal dynamic features of functional activation and the spatial organization pattern features of brain functional networks are extracted from the functional near-infrared spectral data, and the temporal dynamic features and the spatial organization pattern features of brain functional networks are constructed into a near-infrared brain functional feature set. The temporal dynamic features of functional activation include the average intensity and dynamic rate of change of functional activity, and the brain functional network organization pattern features include the modularity, small-world properties and hemispherical asymmetry features of the corresponding undirected weighted functional network.

[0008] For the near-infrared brain functional feature set corresponding to the functional near-infrared spectral data and the eye movement spatiotemporal feature set corresponding to the eye movement feature data, each feature in the near-infrared brain functional feature set is constructed with the feature in the eye movement spatiotemporal feature set as a feature pair, the correlation coefficient of each feature pair is calculated, and features are selected based on the correlation coefficient.

[0009] The features selected from the near-infrared brain function feature set and the eye movement spatiotemporal feature set are input into the classification and assessment model to obtain the severity of behavioral dysfunction and brain function analysis values.

[0010] Using the above approach, this method first collects initial functional near-infrared spectral data and eye-tracking feature data from test subjects under a pre-constructed social scenario task paradigm. By analyzing the two types of data, multiple features are obtained and constructed into near-infrared brain function feature sets and eye-tracking spatiotemporal feature sets. Then, feature pairs are constructed from the features in the two feature sets, and each pair is analyzed and screened to identify features that can effectively characterize brain function. Finally, a classification assessment model is used to give the severity of behavioral dysfunction and brain function analysis values. This method can provide an effective reference for clinical rehabilitation assessment and intervention through brain function analysis values.

[0011] In some embodiments of the present invention, in the step of testing the subject under a pre-constructed social scenario task paradigm and acquiring test data, a near-infrared brain functional imaging device equipped with a 40-channel fNIRS acquisition headgear is used to cover the bilateral prefrontal, temporal, parietal, and occipital lobes of the subject based on a 10-10 positioning system, and hemodynamic response information is acquired at a sampling frequency of 10Hz as functional near-infrared spectral data; a desktop eye tracker is used to track the movement of the subject's eyes through an infrared camera and record the subject's eye movement feature data during the test.

[0012] In some embodiments of the present invention, the eye movement feature data includes fixation point location, fixation time, and saccade path.

[0013] In some embodiments of the present invention, subsets of fNIRS brain function features and eye movement features that are significantly correlated with the functional state of the subject are extracted to achieve preliminary feature screening; in the step of constructing feature pairs by combining features from each near-infrared brain function feature set with features from the eye movement spatiotemporal feature set, and calculating the correlation coefficient of each feature pair:

[0014] The objective function value is calculated based on the features in the feature pair, and the optimal coefficient vector is determined by an alternating minimization method.

[0015] The correlation coefficient of feature pairs is calculated based on the optimal coefficient vector.

[0016] In some embodiments of the present invention, the steps of the near-infrared brain function feature set corresponding to the functional near-infrared spectral data and the eye movement spatiotemporal feature set corresponding to the eye movement feature data further include extracting functional near-infrared spectral data and eye movement feature data that are significantly correlated with the functional state of the subject, thereby achieving preliminary feature screening.

[0017] In some embodiments of the present invention, in the step of calculating the objective function value based on the features in the feature pair and determining the optimal coefficient vector using an alternating minimization method, the objective function value is calculated based on the following formula:

[0018]

[0019] in, Represents the objective function value. This represents the covariance matrix constructed from the X and Y features in the feature pair. and All of these are regularization parameters. and These are the optimal coefficient vectors. and optimal coefficient vector L1 norm, Represents the optimal coefficient vector The transpose of .

[0020] In some embodiments of the present invention, in the step of calculating the correlation coefficient of feature pairs based on the optimal coefficient vector, the correlation coefficient is calculated using the following formula:

[0021]

[0022] in, This represents the correlation coefficient.

[0023] In some embodiments of the present invention, in the step of filtering features based on correlation coefficient, for two features in a feature pair, after calculating the correlation coefficient, the dimension of the value in each feature is adjusted for a preset number of times, and the correlation coefficient is calculated again for a preset number of times. The correlation coefficient calculated again is compared with the correlation coefficient calculated for the first time to determine whether the feature in the feature pair is selected as a calculation feature.

[0024] In some embodiments of the present invention, in the step of comparing the recalculated correlation coefficient with the first calculated correlation coefficient to determine whether a feature in a feature pair is selected as a computational feature, the difference between the recalculated correlation coefficient and the first calculated correlation coefficient is calculated, and the absolute value is calculated. If the absolute value is greater than a preset threshold, then the feature in the feature pair is selected as a computational feature.

[0025] In some embodiments of the present invention, in the step of extracting the temporal dynamic characteristics of functional activation and the spatial organization pattern characteristics of brain functional networks from the functional near-infrared spectral data, the average amplitude of each fNIRS channel time series data in the functional near-infrared spectral data over the entire time series is calculated as the average intensity of functional activity; the rate of change of fNIRS signal amplitude at adjacent time points in the functional near-infrared spectral data is calculated as the dynamic rate of change; Pearson correlation analysis is used to calculate the values ​​between fNIRS channels and construct an undirected weighted functional network; for the undirected weighted functional network, graph theory algorithms are used to calculate the values ​​of modularity, small-world properties, and hemispherical asymmetry characteristics.

[0026] In some embodiments of the present invention, in the step of calculating the average amplitude over the entire time series of time series data for each fNIRS channel in functional near-infrared spectral data as the average intensity of functional activity, the average amplitude is calculated using the following formula: ,in, This represents the average amplitude, and T represents the number of time points. Indicates a point in time The amplitude; in the step of calculating the rate of change of the fNIRS signal amplitude at adjacent time points in the functional near-infrared spectral data as the dynamic rate of change, the dynamic rate of change is calculated using the following formula: in, Indicates a point in time The rate of dynamic change Indicates a point in time The amplitude.

[0027] A second aspect of the present invention also provides a brain function evaluation system based on fNIRS and eye movement feature fusion. The system includes a computer device, the computer device including a processor and a memory, the memory storing computer instructions, and the processor executing the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0028] A third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned brain function evaluation method based on fNIRS and eye-tracking feature fusion.

[0029] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the text, or may be learned by practice of the invention. The objects and other advantages of the invention will become apparent from the description and the accompanying drawings.

[0030] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0031] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0032] Figure 1 This is a schematic diagram of one embodiment of the brain function evaluation method based on fNIRS and eye movement feature fusion of the present invention;

[0033] Figure 2 This is a schematic diagram of the processing architecture of the brain function assessment method based on fNIRS and eye movement feature fusion of the present invention;

[0034] Figure 3 This is a schematic diagram of the overall architecture of the brain function assessment method based on fNIRS and eye movement feature fusion of the present invention;

[0035] Figure 4 This is a diagram illustrating the multidimensional spatiotemporal feature calculation framework of this scheme.

[0036] Figure 5 This is a framework diagram for feature analysis of this solution;

[0037] Figure 6 This is a schematic diagram of the processing flow of the classification and evaluation model in this scheme;

[0038] Figure 7 This is a schematic diagram of the final evaluation results. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0040] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0041] like Figure 1-3 As shown, this invention proposes a brain function assessment method based on the fusion of fNIRS and eye movement features. The steps of this method include:

[0042] Step S100: Test the test subject under the pre-constructed social scenario task paradigm and obtain test data, which includes functional near-infrared spectral data and eye movement feature data;

[0043] Specifically, the pre-constructed social scenario task paradigm adopts a social scenario task paradigm related to the functional impairment of the test subjects.

[0044] Step S200: Extract the temporal dynamic features of functional activation and the spatial organization pattern features of brain functional networks from the functional near-infrared spectral data, and construct the temporal dynamic features and the spatial organization pattern features of brain functional networks into a near-infrared brain functional feature set. The temporal dynamic features of functional activation include the average intensity and dynamic rate of change of functional activity, and the brain functional network organization pattern features include the modularity, small-world properties and hemispherical asymmetry features of the corresponding undirected weighted functional network.

[0045] Specifically, in undirected weighted functional networks, the edges of the network (i.e., connections between brain regions) are not directional, indicating that the functional connections between two nodes (such as fNIRS channels) are bidirectional and symmetrical. For example, the Pearson correlation coefficient between channel A and channel B is equivalent to the correlation coefficient between B and A, without needing to distinguish direction.

[0046] like Figure 4 As shown, specifically, before step S200, the process includes preprocessing the functional near-infrared spectral data to obtain a preprocessed dataset. This includes filtering, removing head movement artifacts, and removing physiological noise interference. Specifically, a bandpass filter of 0.01-0.1 Hz is used to filter and remove low-frequency noise related to heartbeat and respiration, as well as high-frequency measurement noise. A time derivative repair distribution algorithm is used to correct head movement artifacts and improve the signal-to-noise ratio. After preprocessing, a preprocessed dataset of functional near-infrared spectral data for each acquisition channel is obtained. In the preprocessing of eye-tracking feature data, the original data is segmented based on the period division of different stimulus content in the face recognition task, resulting in eye-tracking data segments under different stimulus windows.

[0047] Step S300: For the near-infrared brain functional feature set corresponding to the functional near-infrared spectral data and the eye movement spatiotemporal feature set corresponding to the eye movement feature data, construct feature pairs by combining the features in each near-infrared brain functional feature set with the features in the eye movement spatiotemporal feature set, calculate the correlation coefficient of each feature pair, and filter features based on the correlation coefficient.

[0048] Specifically, in the step of extracting functional near-infrared spectral data and eye-tracking feature data that are significantly correlated with the functional status of the test subjects, and in the preliminary feature screening step, features in the near-infrared brain function feature set and the eye-tracking spatiotemporal feature set are initially selected. For the near-infrared brain function feature set and the eye-tracking spatiotemporal feature set respectively, correlation analysis is used to extract subsets of fNIRS brain function features and eye-tracking features that are significantly correlated with the autism functional status (CARS scale score), and features with weak explanatory power for the target variable are initially removed, and preliminary dimensionality reduction of the indicator features is performed; feature fusion regression analysis, such as... Figure 4 As shown.

[0049] Specifically, the sparse canonical correlation analysis, based on the preliminary feature dimensionality reduction described above, employs the sparse canonical correlation analysis method to perform a fusion analysis between brain functional features and eye-movement behavior, determining the correlation between each brain functional feature and eye-movement behavior. The selected features are ranked according to their correlation magnitude, and significant eye-movement behavior parameters and brain functional features are selected to determine a subset of fNIRS brain functional features and eye-movement fusion features relevant to the functions of children with autism, completing feature screening and constructing a multi-feature subset.

[0050] Specifically, the preparation of brain function and eye-movement behavior data involves vectorizing the relevant feature matrices into feature vectors. For each subject, the spatiotemporal dimension of near-infrared brain function features is p, and the brain function feature matrix... The eye movement spatiotemporal feature dimension of each subject is q, then the eye movement behavior feature matrix is... , where n represents the number of subjects;

[0051] Specifically, the brain functional feature matrix and eye movement behavior feature matrix Standardization is performed so that the mean of each feature is 0 and the variance is 1, thus eliminating the influence of different feature scales.

[0052] Specifically, the preparation of brain function and eye-movement behavior data involves vectorizing the relevant feature matrices into feature vectors. For each subject, the spatiotemporal dimension of near-infrared brain function features is p, and the brain function feature matrix... The eye movement spatiotemporal feature dimension of each subject is q, then the eye movement behavior feature matrix is... , where n represents the number of subjects.

[0053] Step S400: Input the features from the near-infrared brain function feature set and the eye movement spatiotemporal feature set into the classification assessment model to obtain the severity of behavioral dysfunction and brain function analysis values.

[0054] The severity of the behavioral dysfunction is determined based on brain function analysis values. Specifically, the severity of the behavioral dysfunction is determined by comparing a threshold with the brain function analysis values.

[0055] In the specific implementation process, the classification assessment model adopts the KNN algorithm, using the selected features as a feature subset as input, the autism severity scale score as the category label, and the optimal K value is determined through ten-fold cross-validation to construct a classification assessment model.

[0056] Specifically, firstly, Z-score standardization is performed on each feature data x in the feature subset. ,in These are the original eigenvalues. It is the mean of the features. It is the standard deviation of the feature;

[0057] like Figure 6 As shown, specifically, in a multi-feature space, Euclidean distance is applied to measure the similarity between samples. For two... Dimensional Samples and Based on the calculated Euclidean distance, the K nearest neighbors (K-neighbor samples) are found by sorting the distances. A weighted average is used for classification, with the reciprocal of the distance as the weight. The classification decision is then made based on the frequency of each weighted category. The sample to be classified is assigned to the category with the highest frequency. Using a subset of features from the training set and their corresponding labels, the model is trained according to the KNN algorithm steps. Ten-fold cross-validation is used to continuously adjust the model parameters. Accuracy, precision, and recall are calculated to evaluate the model's classification performance, determine the optimal K value, train and optimize the model, and construct a classification evaluation model.

[0058] Specifically, the severity of autism is assessed using a scale as the category label. According to the Childhood Autism Rating Scale (CARS, total score of 60), a total score of less than 30 points is defined as no autism; a total score between 30 and 37 points is defined as mild to moderate autism; and a total score between 37 and 60 points is defined as severe autism.

[0059] Using the above approach, this method first collects initial functional near-infrared spectral data and eye-tracking feature data of the test subjects under a pre-constructed social scenario task paradigm. By analyzing the two types of data, multiple features are obtained and constructed into near-infrared brain function feature sets and eye-tracking spatiotemporal feature sets. Then, feature pairs are constructed from the features in the two feature sets, and each pair is analyzed and screened to select features that can effectively characterize brain function characteristics. Finally, brain function analysis values ​​are given through a classification assessment model. This method can provide an effective reference for clinical rehabilitation assessment and intervention through brain function analysis values.

[0060] In some embodiments of the present invention, in the step of testing the subject under a pre-constructed social scenario task paradigm and acquiring test data, a near-infrared brain functional imaging device equipped with a 40-channel fNIRS acquisition headgear is used to cover the bilateral prefrontal, temporal, parietal, and occipital lobes of the subject based on a 10-10 positioning system, and hemodynamic response information is acquired at a sampling frequency of 10Hz as functional near-infrared spectral data; a desktop eye tracker is used to track the movement of the subject's eyes through an infrared camera and record the subject's eye movement feature data during the test.

[0061] In its implementation, this scheme uses a face recognition task as the paradigm for social scenarios. The core functional impairments in children with autism manifest as social interaction difficulties, primarily in face recognition, emotion perception, and attention to faces. Therefore, by presenting different types of face images, the brain function characteristics and eye movement patterns of the children are observed simultaneously during this process. This task paradigm has good compatibility and is applicable to children with varying degrees of autism. Therefore, the face recognition task used in this scheme as a functional assessment paradigm has broader applicability.

[0062] Specifically, the child sits upright in a chair at an appropriate height and distance, with their hands placed naturally. After the "start" command, images of familiar faces (such as parents and caregivers), unfamiliar faces, and different types of faces (joy, anger, sorrow, happiness, etc.) are randomly displayed on the screen in sequence on the table. The images are presented one by one, with each presentation lasting 5 seconds, for a total of 120 random face images.

[0063] Specifically, the near-infrared brain functional imaging device is equipped with a multi-channel fNIRS acquisition cap, which covers the bilateral prefrontal, temporal, parietal, and occipital regions of interest based on a 10-10 system, and acquires hemodynamic response information at a frequency of 10Hz.

[0064] Specifically, the eye-tracking technology uses a desktop eye tracker that tracks eye movements through an infrared camera, recording eye movement data such as the child's gaze point position, gaze duration, and saccade path while viewing images of faces.

[0065] In some embodiments of the present invention, the eye movement feature data includes fixation point location, fixation time, and saccade path.

[0066] In some embodiments of the present invention, in the step of constructing feature pairs by combining features from each near-infrared brain functional feature set with features from the eye-tracking spatiotemporal feature set, and calculating the correlation coefficient of each feature pair:

[0067] The objective function value is calculated based on the features in the feature pair, and the optimal coefficient vector is determined by an alternating minimization method.

[0068] The correlation coefficient of feature pairs is calculated based on the optimal coefficient vector.

[0069] In some embodiments of the present invention, in the step of calculating the objective function value based on the features in the feature pair and determining the optimal coefficient vector using an alternating minimization method, the objective function value is calculated based on the following formula:

[0070]

[0071] in, Represents the objective function value. This represents the covariance matrix constructed from the X and Y features in the feature pair. and All of these are regularization parameters. and These are the optimal coefficient vectors. and optimal coefficient vector The L1 norm, Represents the optimal coefficient vector The transpose of .

[0072] Specifically, by using the alternating minimization method, we can find... and Substitute this into the objective function to maximize it.

[0073] In some embodiments of the present invention, in the step of calculating the correlation coefficient of feature pairs based on the optimal coefficient vector, the correlation coefficient is calculated using the following formula:

[0074]

[0075] in, This represents the correlation coefficient.

[0076] like Figure 5 As shown, in some embodiments of the present invention, in the step of filtering features based on correlation coefficient, for two features in a feature pair, after calculating the correlation coefficient, the dimension of the value in each feature is adjusted for a preset number of times, and the correlation coefficient is calculated again for a preset number of times. The correlation coefficient calculated again is compared with the correlation coefficient calculated for the first time to determine whether the feature in the feature pair is selected as a calculation feature.

[0077] Specifically, the above steps involve performing a statistical significance analysis on the actual calculated correlation coefficient and the correlation coefficient calculated after shuffling (for example, shuffling 1000 times will generate 1000 correlation coefficients). If the actual calculated value is significantly greater than the distribution of these 1000 values, it indicates that the correlation is statistically significant.

[0078] Using the above scheme, significantly correlated canonical variable features are identified through permutation tests to extract a subset of fusion features. First, the original data undergoes multiple random permutations to shuffle the correspondences, and the canonical correlation coefficient is recalculated to obtain a null distribution. The actual calculated canonical correlation coefficient is compared with the null distribution. If the actual value is significantly greater than the value in the null distribution, the corresponding canonical variable pair is considered to have a significant correlation. Indicators that clearly demonstrate a significant correlation between brain function and eye-movement behavior parameters can be used as the subset of fusion features, ensuring the relevance of feature selection.

[0079] In some embodiments of the present invention, in the step of comparing the recalculated correlation coefficient with the first calculated correlation coefficient to determine whether a feature in a feature pair is selected as a computational feature, the difference between the recalculated correlation coefficient and the first calculated correlation coefficient is calculated, and the absolute value is calculated. If the absolute value is greater than a preset threshold, then the feature in the feature pair is selected as a computational feature.

[0080] In some embodiments of the present invention, in the step of extracting the temporal dynamic characteristics of functional activation and the spatial organization pattern characteristics of brain functional networks from the functional near-infrared spectral data, the average amplitude of each fNIRS channel time series data in the functional near-infrared spectral data over the entire time series is calculated as the average intensity of functional activity; the rate of change of fNIRS signal amplitude at adjacent time points in the functional near-infrared spectral data is calculated as the dynamic rate of change; Pearson correlation analysis is used to calculate the values ​​between fNIRS channels and construct an undirected weighted functional network; for the undirected weighted functional network, graph theory algorithms are used to calculate the values ​​of modularity, small-world properties, and hemispherical asymmetry characteristics.

[0081] like Figure 4 As shown, in the specific implementation process, modularization utilizes the Louvain algorithm to divide the network into modules by optimizing the modularity (Q-value). A higher Q-value indicates a more significant degree of modularity in the functional network. Small-world property analysis describes the network by calculating the average path length (L) and clustering coefficient (C). The average path length refers to the average of the shortest path lengths between any two nodes in the network, reflecting the efficiency of information propagation in the network; the clustering coefficient measures the tightness of the connections between a node's neighbors, reflecting the local connectivity characteristics of the network. Hemispherical asymmetry is defined based on the difference between intra-hemispherical and inter-hemispherical connections in the functionally connected network, defining the hemispherical autonomy coefficient. , This represents the sum of the functional connection strengths between channel i and the same-side hemisphere; This represents the sum of functional connections between channel i and the opposite hemisphere; and These represent the sum of functional connections in the ipsilateral and contralateral hemispheres, respectively. The HA value can be used to reflect the imbalance of functional connections within and between hemispheres. The larger the HA, the stronger the connection between the channel and the intrahemispheric connection is compared to the interhemispheric connection. By comparing the HA values ​​of the left and right hemispheres, cortical lateralization can be indirectly reflected.

[0082] In some embodiments of the present invention, in the step of calculating the average amplitude over the entire time series of time series data for each fNIRS channel in functional near-infrared spectral data as the average intensity of functional activity, the average amplitude is calculated using the following formula: ,in, This represents the average amplitude, and T represents the number of time points. Indicates a point in time The amplitude; in the step of calculating the rate of change of the fNIRS signal amplitude at adjacent time points in the functional near-infrared spectral data as the dynamic rate of change, the dynamic rate of change is calculated using the following formula: in, Indicates a point in time The rate of dynamic change Indicates a point in time The amplitude.

[0083] Furthermore, the mean and standard deviation of the functional near-infrared spectral data were calculated to characterize the channel's response to changes in functional activity during the monitoring period.

[0084] Using the above scheme, the average amplitude reflects the overall activity level of the fNIRS channel during the monitoring period.

[0085] In the specific implementation process, the eye movement feature data is segmented to extract the gaze behavior features of eye movement information under different face tasks, and the behavior patterns of eye movement in time and space are analyzed.

[0086] The aforementioned eye-tracking spatial features include the distribution of fixation point locations. By dividing the face into facial and non-facial regions, and further subdividing the facial region into regions of interest (ROIs) for the eyes, nose, mouth, and ears, the number of fixation points falling within the ROIs is specifically analyzed to characterize the spatial distribution of eye-tracking fixation point locations.

[0087] The aforementioned eye movement time characteristics include fixation time, saccade speed, and blink frequency. Fixation time refers to the length of time the eyes remain on a specific fixation area; saccade speed refers to the time required to complete one facial scan, and changes in saccade time can reflect the speed and efficiency of an individual's information acquisition; blink frequency refers to the number of blinks per unit of time, which can be used to characterize the degree of attention concentration.

[0088] Based on the above analysis, we obtained the spatiotemporal feature set of eye movements under different facial stimuli. Specifically, the eye movement information features include the spatiotemporal feature set of multi-stimulus task under familiar faces, unfamiliar faces, and faces with different emotions.

[0089] Specifically, after the final brain function analysis values ​​are calculated, the child's functional status is determined by comparison, and multidimensional spatiotemporal characteristics are visualized, such as... Figure 5 , 6 As shown in Figure 7. The visualization is used to provide feedback on the spatiotemporal characteristics of brain function and eye movement information under the task paradigm of the test subject. The spatiotemporal characteristics of brain function are displayed in the form of functional activation and bar charts, while the eye movement behavior characteristics are displayed by drawing eye movement heatmaps, with the density of fixation points represented by the intensity of color, highlighting the key areas of individual attention, intuitively showing the eye movement characteristics of autistic children, and providing feedback to medical staff and guardians.

[0090] Specifically, in actual use, assessors can use the model output scores, as well as specific brain function characteristics and eye movement information characteristics, as references to conduct a comprehensive and objective assessment of the functional status of children with autism, and can track the effectiveness of rehabilitation training longitudinally and make adjustments in real time.

[0091] In summary, this invention uses face recognition tasks as a paradigm for assessing social function. Based on a multi-channel near-infrared brain function and non-contact eye-tracking device covering the bilateral prefrontal, temporal, occipital, and parietal lobes, it effectively integrates multi-level brain function and eye movement information by extracting multi-dimensional spatiotemporal features and using fusion regression analysis methods. This enables effective assessment of the functional status of children with autism and provides a method for assessors and guardians to develop personalized rehabilitation training programs.

[0092] This invention also provides a brain function evaluation system based on fNIRS and eye movement feature fusion. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0093] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned brain function evaluation method based on fNIRS and eye-tracking feature fusion. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0094] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0095] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0096] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0097] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A brain function assessment method based on fNIRS and eye-tracking feature fusion, characterized in that, The steps of this method include: Tests were conducted on test subjects within a pre-constructed social scenario task paradigm to obtain test data, which included functional near-infrared spectral data and eye-tracking feature data. The temporal dynamic features of functional activation and the spatial organization pattern features of brain functional networks are extracted from the functional near-infrared spectral data, and the temporal dynamic features and the spatial organization pattern features of brain functional networks are constructed into a near-infrared brain functional feature set. The temporal dynamic features of functional activation include the average intensity and dynamic rate of change of functional activity, and the brain functional network organization pattern features include the modularity, small-world properties and hemispherical asymmetry features of the corresponding undirected weighted functional network. For the near-infrared brain function feature set corresponding to the functional near-infrared spectral data and the eye movement spatiotemporal feature set corresponding to the eye movement feature data, each feature in the near-infrared brain function feature set is constructed with the feature in the eye movement spatiotemporal feature set to form a feature pair. The correlation coefficient of each feature pair is calculated. Based on the correlation coefficient, features are selected. For two features in a feature pair, after calculating the correlation coefficient, the dimension of the value in each feature is adjusted for a preset number of times, and the correlation coefficient is calculated again for a preset number of times. The correlation coefficient calculated again is compared with the correlation coefficient calculated for the first time to determine whether the feature in the feature pair is selected as a computational feature. The difference between the correlation coefficient calculated again and the correlation coefficient calculated for the first time is calculated, and the absolute value is calculated. If the absolute value is greater than a preset threshold, the feature in the feature pair is selected as a computational feature. The features selected from the near-infrared brain function feature set and the eye movement spatiotemporal feature set are input into the classification and assessment model to obtain the severity of behavioral dysfunction and brain function analysis values.

2. The brain function evaluation method based on fNIRS and eye-tracking feature fusion according to claim 1, characterized in that, In the pre-constructed social scenario task paradigm, the test subjects were tested. In the step of acquiring test data, a near-infrared brain functional imaging device equipped with a 40-channel fNIRS acquisition head cap was used to cover the bilateral prefrontal, temporal, parietal, and occipital lobes of the test subjects based on a 10-10 positioning system. Hemodynamic response information was acquired at a sampling frequency of 10Hz as functional near-infrared spectral data. A desktop eye tracker was used to track the movement of the test subjects' eyes through an infrared camera and record the eye movement feature data of the test subjects during the test.

3. The brain function evaluation method based on fNIRS and eye-tracking feature fusion according to claim 1, characterized in that, In the step of constructing feature pairs by comparing each feature in the near-infrared brain function feature set with features in the eye movement spatiotemporal feature set, and calculating the correlation coefficient of each feature pair: The objective function value is calculated based on the features in the feature pair, and the optimal coefficient vector is determined by an alternating minimization method. The correlation coefficient of feature pairs is calculated based on the optimal coefficient vector.

4. The brain function evaluation method based on fNIRS and eye-tracking feature fusion according to claim 3, characterized in that, The steps for the near-infrared brain function feature set corresponding to the functional near-infrared spectral data and the eye movement spatiotemporal feature set corresponding to the eye movement feature data also include extracting functional near-infrared spectral data and eye movement feature data that are significantly correlated with the functional state of the subject, thereby achieving preliminary feature screening.

5. The brain function evaluation method based on fNIRS and eye-tracking feature fusion according to claim 4, characterized in that, In the step of calculating the correlation coefficient of feature pairs based on the optimal coefficient vector, the correlation coefficient is calculated using the following formula: in, Represents the correlation coefficient. Represents the optimal coefficient vector transpose, This represents the covariance matrix constructed from the X and Y features in the feature pair. Represents the optimal coefficient vector .

6. The brain function evaluation method based on fNIRS and eye-tracking feature fusion according to claim 1, characterized in that, In the step of extracting the temporal dynamics of functional activation and the spatial organization pattern of brain functional networks from the functional near-infrared spectral data, for each fNIRS channel time series data in the functional near-infrared spectral data, the average amplitude of the fNIRS channel time series data over the entire time series is calculated as the average intensity of functional activity; the rate of change of the fNIRS signal amplitude at adjacent time points in the functional near-infrared spectral data is calculated as the dynamic rate of change; Pearson correlation analysis is used to calculate the values ​​between fNIRS channels and construct an undirected weighted functional network; for the undirected weighted functional network, graph theory algorithms are used to calculate the values ​​of modularity, small-world properties, and hemispherical asymmetry features.

7. The brain function evaluation method based on fNIRS and eye-tracking feature fusion according to claim 6, characterized in that, In the step of calculating the average amplitude of the fNIRS channel time series data over the entire time series as the average intensity of functional activity for each fNIRS channel in the functional near-infrared spectral data, the average amplitude is calculated using the following formula: ,in, This represents the average amplitude, and T represents the number of time points. Indicates a point in time The amplitude; in the step of calculating the rate of change of the fNIRS signal amplitude at adjacent time points in the functional near-infrared spectral data as the dynamic rate of change, the dynamic rate of change is calculated using the following formula: in, Indicates a point in time The rate of dynamic change Indicates a point in time The amplitude.

8. A brain function assessment system based on fNIRS and eye-tracking feature fusion, characterized in that, The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as described in any one of claims 1 to 7.

Images