Method for identifying brain function diseases by using neural network based on near-infrared data

Abstract

The invention provides a method for identifying brain function diseases by using a neural network based on near-infrared data and a processing device. The method comprises the following steps: on the basis of first near-infrared data of a plurality of acquisition channels, utilizing Grubrum angle field transformation to obtain Grubrum angle field image data; and performing convolution and pooling operation by using the neural network based on the Grubrum angle field image data so as to convert the Grubrum angle field image data into a first feature vector below two dimensions. And obtaining a fusion feature vector based on the first feature vector and a second feature vector with the same dimension, wherein the second feature vector is obtained based on features of other dimensions of the target brain. And based on the fusion feature vector, determining an identification result of the brain function disease of the subject. Therefore, the second feature vectors of different dimensions can be conveniently fused as supplementary information, the complex difference between the brain function disease patients, especially the patients with mild brain function diseases or early brain function diseases and healthy people is comprehensively captured, and the classification precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural networks, in particular to a method for identifying brain function diseases based on near-infrared data using a neural network and a processing device. BACKGROUND

[0002] With the increasing trend of global population aging, the incidence of brain function diseases is rising, which makes early identification of brain function diseases crucial. Early detection of brain function diseases and intervention treatment can effectively delay the progression of brain function diseases.

[0003] Near-infrared functional imaging technology (fNIRS) has become an important tool for studying brain activity. fNIRS has shown great application potential in the diagnosis, treatment and rehabilitation evaluation of brain function diseases due to its non-invasiveness, real-time and portability. Although attempts have been made to introduce neural networks into the automatic identification of brain function diseases based on fNIRS data, the brain is different from the lungs, blood vessels and other body parts, and the attribute parameters are more complex. Existing models often focus on distinguishing various brain function diseases from each other, but various situations occur, such as being well distinguished from other brain function diseases but not from healthy conditions, and the precision of distinguishing a certain brain function disease is not enough. Further, if mild brain function diseases (such as but not limited to mild cognitive impairment) can be identified early and treatment started as soon as possible, the treatment effect can be significantly improved. However, existing neural networks are usually based on a single feature, or there are difficulties in combining multiple features, or the fusion of features is improper, and further classification often fails to fully capture the complex differences between brain function disease patients and healthy control (HC) groups, resulting in poor classification accuracy. SUMMARY

[0004] To solve the above technical problems in the prior art, the present application is proposed. The present application aims to provide a method for identifying brain function diseases based on near-infrared data using a neural network and a processing device, which can conveniently fuse a second feature vector obtained based on other dimensions of the target brain of the subject as supplementary information on the basis of a first feature vector obtained based on first near-infrared data in the feature extraction stage. Based on the fused feature information, the complex differences between brain function disease patients, especially mild or early brain function disease patients, and healthy people can be fully captured, and the classification accuracy can be improved.

[0005] According to a first aspect of the present application, a method for identifying a brain function disease based on near-infrared data using a neural network is provided. The method comprises, by a processor: based on first near-infrared data of a plurality of acquisition channels of a target brain of a subject obtained, using Gram angular field transformation to obtain Gram angular field image data; based on the Gram angular field image data, using a neural network to perform the following processing. Convolution and pooling operations are performed to convert into a first feature vector. Based on the first feature vector and a second feature vector having the first dimension, a fusion feature vector is obtained, the second feature vector being obtained based on features of other dimensions of the target brain of the subject. And based on the fusion feature vector, an identification result of a brain function disease of the subject is determined.

[0006] According to a second aspect of the present application, a processing device for identifying a brain function disease based on near-infrared data using a neural network is provided. The processing device comprises a processor configured to: based on first near-infrared data of a plurality of acquisition channels of a target brain of a subject obtained, using Gram angular field transformation to obtain Gram angular field image data; based on the Gram angular field image data, using a neural network to perform the following processing. Convolution and pooling operations are performed to convert into a first feature vector. Based on the first feature vector and a second feature vector having the first dimension, a fusion feature vector is obtained, the second feature vector being obtained based on features of other dimensions of the target brain of the subject. And based on the fusion feature vector, an identification result of a brain function disease of the subject is determined.

[0007] According to a third aspect of the present application, a computer readable storage medium storing a computer program is provided. The computer program, when executed by a processor, causes the processor to perform a method for identifying a brain function disease based on near-infrared data using a neural network as described in various embodiments of the present application. The method comprises: based on first near-infrared data of a plurality of acquisition channels of a target brain of a subject obtained, using Gram angular field transformation to obtain Gram angular field image data; based on the Gram angular field image data, using a neural network to perform the following processing. Convolution and pooling operations are performed to convert into a first feature vector. Based on the first feature vector and a second feature vector having the first dimension, a fusion feature vector is obtained, the second feature vector being obtained based on features of other dimensions of the target brain of the subject. And based on the fusion feature vector, an identification result of a brain function disease of the subject is determined.

[0008] Compared with the prior art, the beneficial effects of the embodiments of the present application are that:

[0009] The method for identifying brain function diseases by using a neural network provided in the embodiments of the present application converts first near-infrared data into Gram angular field image data, so that the single-dimensional time series data is presented in a more abundant information in two-dimensional or three-dimensional space while the main features of the first near-infrared data are retained. A first feature vector is extracted based on the Gram angular field image data, and a second feature vector of the same low dimension is fused based on the first feature vector, the second feature vector being obtained based on other-dimensional features of a target brain of the subject. The dimensions of the first feature vector and the second feature vector are different, so that the fusion of multi-dimensional features is realized, and more comprehensive feature information is embodied. The same low dimension simplifies the fusion of the feature vectors, and enables the role weight of the first feature vector and the second feature vector in the fused vector to be accurately controlled. It is proved through subsequent experiments that the fusion of the second feature vector based on the first feature vector can improve the recognition ability and efficiency of the complex differences between brain function disease patients (even mild brain function disease patients or early brain function disease patients) and healthy people, and is beneficial to improve the recognition accuracy of brain function diseases.

[0010] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the contents of the specification can be implemented, and in order to make the above description and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0011] In the drawings, which are not necessarily drawn to scale, like numerals can describe similar components in different views. Like numerals having different letter suffixes can represent different instances of similar components. The drawings illustrate generally, by way of example, various embodiments discussed in the following detailed description.

[0012] Figure 1 A flowchart of a method for identifying brain function diseases by using a neural network according to an embodiment of the present application is shown.

[0013] Figure 2 A flowchart of a method for identifying brain function diseases by using a ResNet18 model according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0014] For those skilled in the art to better understand the technical solutions of the present application, the present application will be described in detail below in combination with the drawings and specific embodiments. The embodiments of the present application will be further described in detail below in combination with the drawings and specific embodiments, but not as a limitation to the present application.

[0015] The "first", "second" and similar words used in the present application do not represent any order, number or importance, but are only used to distinguish. The "including" or "containing" and similar words used in the present application mean that the elements before the word cover the elements listed after the word, and do not exclude the possibility of also covering other elements. In the present application, the arrows shown in the figure of each step are only an example of the execution order, not a limitation, and the technical solutions of the present application are not limited to the execution order described in the embodiments. Each step in the execution order can be combined, can be decomposed, can be exchanged in order, as long as it does not affect the logical relationship of the execution content.

[0016] All terms used in the present application (including technical terms or scientific terms) have the same meaning as understood by those skilled in the art to which the present application belongs, unless otherwise specifically defined. It should also be understood that terms defined in, such as general dictionaries, should be interpreted to have meanings consistent with their meanings in the context of the relevant art, and should not be interpreted in an idealized or excessively formalized sense, unless specifically defined here. The technology and equipment known to those skilled in the relevant art can not be discussed in detail, but in appropriate cases, the technology and equipment should be considered as part of the specification.

[0017] In the present application, the arrows shown in the figure of each step are only an example of the execution order, not a limitation, and the technical solutions of the present application are not limited to the execution order described in the embodiments. Each step in the execution order can be combined, can be decomposed, can be exchanged in order, as long as it does not affect the logical relationship of the execution content.

[0018] Figure 1A flow chart of a method for identifying a brain function disease using a neural network according to an embodiment of the present application is shown. Each of steps S101-S104 is executed by a processor. In step S101, a Gram angular field transform is used to obtain Gram angular field image data based on first near-infrared data of a plurality of acquisition channels of a target brain of a subject. The time series signal of the first near-infrared data is converted into an image, which can integrate time attention and spatial attention and improve the feature learning effect of the first near-infrared data. Specifically, the first near-infrared data of the subject when performing a task or in a resting state can be collected using a near-infrared data acquisition device. For example, the near-infrared data acquisition device has at least a head cap for wearing on the head of the subject. The head cap can have a plurality of probes for transmitting and / or receiving near-infrared light, wherein each of the plurality of probes can be configured as a transmitting probe or a receiving probe, and each pair of probes arranged in pairs can form a detection channel. In some embodiments, one transmitting probe can correspond to a plurality of receiving probes, or vice versa, one receiving probe can correspond to a plurality of transmitting probes, wherein the transmitting probe and the receiving probe are matched in pairs according to the specific requirements of the arrangement position of the probe, the brain function region to be detected, etc.

[0019] The acquired first near-infrared data is multi-channel time series data, and Gram angle field transformation can be performed on the time series data of each channel respectively, so as to convert the time series data of each channel into Gram angle field image data for better visualization and analysis. For example, the original time series data can be preprocessed as necessary, such as motion artifact removal, filtering, calculation of relative contents of oxyhemoglobin (HBO) and deoxyhemoglobin (HBR), standardization, and then Gram angle field transformation is performed. Specifically, the data at each time point is mapped to a unit circle to obtain a series of angle values, and then a certain function of the angle difference between any two time points is calculated to construct a Gram matrix, such as a weighted form composed of complex exponential function or sine and cosine function combination, and the Gram matrix is constructed according to these angle values, wherein the Gram matrix reflects the intensity relationship between each sample of the first near-infrared data. By performing a specific operation on the above Gram matrix, such as taking the sum of the elements on the diagonal to form GASF (Gram Angle Sum Field), or taking the difference between the elements on the diagonal to form GADF (Gram Angle Difference Field), a Gram angle field matrix is obtained. Finally, the Gram angle field matrix can be visualized in the form of a heat map, thereby showing the intensity relationship between each sample in the first near-infrared data. Specifically, the time series data of each channel after Gram angle field conversion becomes a two-dimensional image data, and the two-dimensional image data of each channel after conversion is spliced with the channel as the third dimension, and finally a three-dimensional data similar to an RGB image is obtained. The Gram angle field image data obtained by using Gram angle field transformation on the first near-infrared data of multiple acquisition channels is a three-dimensional data with the channel as the third dimension. This is only an exemplary illustration and does not constitute a limitation on the specific scheme.

[0020] In some embodiments, before the first near-infrared data of each acquisition channel is subjected to Gram angle field transformation, the first near-infrared data of multiple acquisition channels can be subjected to subsample reduction using a piecewise aggregate approximation method to obtain second near-infrared data of multiple acquisition channels, and then only the subsampled second near-infrared data is subjected to Gram angle field transformation, and the third Gram angle field image data of each acquisition channel obtained by transformation is spliced to obtain the Gram angle field image data.

[0021] The near-infrared data is continuously collected and has a large amount of data. The piecewise aggregation approximation method can effectively reduce the length of the time series data, reduce the storage and calculation overhead, retain the main characteristics of the time series, and reduce the possibility of overfitting in training. Through ablation experiments, it is confirmed that the piecewise aggregation approximation method for downsampling makes the number of sampling points after downsampling 45%-60% of the number of sampling points before downsampling, and the comprehensive performance is better. For example, the original fNIRS data of 49s, a total of 539 sampling points, is downsampled to 256, and the comprehensive performance is optimized.

[0022] The piecewise aggregation approximation method can be performed in various ways. For example, the first near-infrared data can be divided into several sub-time periods according to a certain time interval, the average value or median value of the data in each sub-time period is calculated to obtain a representative value representing the time period, and the calculated representative values are connected to form a downsampled time series, that is, the second near-infrared data.

[0023] The following steps S102-S104 are automatically performed using a neural network.

[0024] In step S102, based on the Gram angular field image data, the neural network is used to perform convolution and pooling operations to convert into a first feature vector with a first dimension number, and the first dimension number is below two dimensions. That is, the first dimension number can be two dimensions or one dimension, reducing the dimension number of the feature vector can improve the calculation efficiency.

[0025] The convolution and pooling operations can be performed using a neural network model, which can be a convolutional neural network (CNN), a deep convolutional generative adversarial network (DCGAN), a residual network (ResNet), an Inception network, a lightweight convolutional neural network (MobileNet), or an attention mechanism model (Attention Mechanism Models), etc. There is no limitation, as long as it can perform convolution and pooling operations based on the Gram angular field image data.

[0026] Specifically, the Gram angle field image data can be input into a convolution layer of a neural network model comprising the convolution layer and a pooling layer, the convolution layer extracts feature information in the image by performing a convolution operation on the Gram angle field image data using a convolution kernel to capture image features at different positions and scales. For example, an activation function can be used to introduce nonlinearity after the output of each convolution layer to increase the expression capacity of the network. After the convolution layer, a pooling layer is used to down-sample the feature map, where common pooling operations include Max Pooling and Average Pooling, the pooling layer can reduce the size of the feature map while retaining important feature information and expanding the receptive field for feature information. After the convolution and pooling operations, the feature map is flattened into a one-dimensional first feature vector.

[0027] For example only, without limiting the specific solutions, such as the convolution and pooling operations can be adaptively adjusted according to the different neural network models used, or adaptively modified to improve the accuracy of feature extraction.

[0028] In step S103, a fusion feature vector is obtained based on the first feature vector and a second feature vector having the first dimension, the second feature vector being obtained based on features of other dimensions of the target brain of the subject. In step S104, a recognition result of the brain function disease of the subject is determined based on the fusion feature vector. The dimensions of the first feature vector and the second feature vector are different, so that the fusion of multi-dimensional features is realized, which embodies more comprehensive feature information; the same low dimension simplifies the fusion of the feature vector, and enables the role weight of the first feature vector and the second feature vector in the fusion vector to be accurately controlled. Through subsequent experiments (described below), the fusion of the second feature vector on the basis of the first feature vector can improve the recognition ability and efficiency of the complex differences between brain function disease patients (even mild brain function disease patients or early brain function disease patients) and healthy people, which is beneficial to improve the recognition accuracy of brain function diseases, especially to recognize mild brain function diseases or early brain function diseases with mild differences (such as less difference or variable difference) compared to healthy people.

[0029] In some embodiments, the first dimension includes one dimension, and the brain function disease to be identified includes mild cognitive impairment.

[0030] MCI is more difficult to distinguish from healthy people than other brain function diseases. MCI is a state of early cognitive decline, with mild symptoms that may manifest as memory loss, difficulty concentrating, etc., and these symptoms are also common in healthy people. Secondly, the diagnostic criteria for MCI are relatively vague, and different definitions and evaluation methods may lead to uncertainty in the diagnosis, and some healthy people may also score slightly lower in cognitive tests, showing similar cognitive abilities to MCI. In addition, MCI may also coexist with other brain function diseases (such as depression, anxiety), further increasing the difficulty of identifying MCI.

[0031] However, accurate identification of MCI, early detection of possible brain function-related problems in patients, and early intervention with necessary interventions are important. For example, the early stage of Alzheimer's disease is usually manifested as MCI. MCI is a state between normal cognitive decline and dementia, and patients may have mild decline in memory, attention, language ability, etc., but can still continue their daily activities. Therefore, early identification and management of MCI is crucial for preventing the development of Alzheimer's disease.

[0032] In the following, MCI is taken as an example to be identified with the first dimension being one-dimensional, but it is understood that the present application is not limited thereto. Identifying MCI, which is easily confused with healthy conditions, with a one-dimensional feature vector is a high-difficulty task, and the method of the present application performs well in performing high-difficulty identification tasks, and can also maintain good performance when performing other lower-difficulty tasks (identifying other brain function diseases with more obvious functional differences).

[0033] In some embodiments, the second feature vector includes at least one feature vector, and the corresponding dimension of the at least one feature vector includes a detection modality other than near-infrared imaging, and the detection modality includes at least one of fMRI, a cognitive scale, and EEG. The second feature vector can be set by a doctor based on clinical experience, or can be compared by experiments to select the best feature vector or feature vector group in recognition effect as the second feature vector. For example, in the case of a brain function disease type of mild cognitive impairment, the second feature vector can be obtained based on the brain network correlation of the target brain of the subject. For example, in the case of a brain function disease type of depression, the second feature vector can be obtained based on scale data, which is only an exemplary description. If the feature information of at least one of fMRI, a cognitive scale, and EEG is one-dimensional itself, it can be directly used as the second feature vector. If it is not one-dimensional itself, it can be converted into a one-dimensional feature vector by using a similar method of steps S101-S104 as the second feature vector. The second feature vector can be a single feature vector, or at least one or more feature vectors, for example, the second feature vector includes a third feature vector and a fourth feature vector, the third feature vector is obtained based on the brain network correlation of the target brain of the subject, and the fourth feature vector is obtained based on the cognitive scale of the subject. Experiments show that the combination of the second feature vector performs particularly well in the recognition of mild cognitive impairment.

[0034] Based on the first feature vector and the second feature vector with the first dimension, various ways can be used to obtain the fusion feature vector, for example, the first feature vector and the second feature vector can be concatenated to obtain the fusion feature vector. For example, the first feature vector and the second feature vector can be linearly transformed respectively, and then added to obtain the fusion feature vector. Alternatively, the first feature vector and the second feature vector can also be multiplied element by element to obtain the fusion feature vector, which is only an exemplary description, and the specific fusion manner is not limited.

[0035] In some embodiments, the first feature vector and the second feature vector can be concatenated based on a preset length proportion relationship to obtain the fusion feature vector, and the preset length proportion relationship is associated with the brain function disease to be identified, the first feature vector, and the second feature vector. For example, the preset length proportion relationship makes the length proportion of the first feature vector in the fusion feature vector greater than 60% and less than 75%, so that in the independent recognition of a plurality of brain function diseases, the first feature vector is dominant, and the second feature vector can play a supplementary role to improve the recognition performance.

[0036] The first near-infrared data of healthy people and patients with mild cognitive impairment during the execution of the delayed matching task (DMST) were collected by using a near-infrared data acquisition device with 39 channels, and the following experiments were carried out.

[0037] The process of the delayed matching task (DMST) is as follows: rest for 15 s, provide observation pictures for 12 s, black screen memory for 12 s, provide picture option selection test for 10 s, and a total of 49 s for one trial. Each subject repeats 10 times. Specifically, one trial collects the first near-infrared data for 49 s.

[0038] The first near-infrared data (hereinafter referred to as fNIRS data) can be pre-processed, including motion artifact removal, filtering processing, and calculation of the relative contents of oxyhemoglobin (HBO) and deoxyhemoglobin (HBR).

[0039] The pre-processed fNIRS data is down-sampled by the piecewise aggregate approximation method, which can effectively reduce the length of time series data, reduce storage and computing overhead, retain the main features of time series, and reduce the possibility of overfitting in training. In addition, for neural network models, the training parameters of the model can also be reduced, and the training time can be reduced. Through ablation experiments, the original 49 s data of fNIRS data, a total of 539 sampling points, is down-sampled to 256, and the comprehensive performance is best.

[0040] The down-sampled data is subjected to Gram angle field transformation. Gram angle field is a method of converting time series into images. The original fNIRS data is a multi-channel time series data. The time series data of each channel is converted into a two-dimensional image data by Gram angle field. The image data of each channel after conversion is spliced according to the third dimension, i.e. the channel dimension, and finally a three-dimensional data similar to RGB image is obtained, and the data shape is 39*256*256, 39 is the channel number of fNIRS device.

[0041] Further, as Figure 2As shown, the ResNet18 model is used to identify mild cognitive impairment. In step S201, the Gram angle field image data obtained by Gram angle field transformation based on the first near-infrared data of multiple acquisition channels is input to the input layer. In step S202, the convolution layer and the pooling layer of the ResNet18 model are used to perform repeated convolution and pooling operations, extract features, and convert the extracted features into a one-dimensional first feature vector. Step S203, that is, obtaining the fusion feature vector based on the first feature vector and the second feature vector, is realized by the linear layer of ResNet18, and in this experiment, although different second feature vectors are used in each experimental group, the first feature vector and the second feature vector are flattened into one dimension.

[0042] In step S204, the length proportion of the first feature vector and the second feature vector is adjusted by using the linear layer to adjust the weight of the first feature vector and the second feature vector in the fusion feature vector. For example, the length proportion of the first feature vector and the second feature vector can be adjusted according to the importance of the first feature vector and the second feature vector, such as formula (1): y=ax1+bx2…, where y represents the fusion feature vector, x1 represents the first feature vector, x2 represents the second feature vector, and the ellipsis represents other supplementary feature vectors. Since there can be multiple supplementary feature vectors in addition to the first feature vector, the ellipsis is used to represent other supplementary feature vectors, a represents the length weight of the first feature vector, and b represents the length weight of the second feature vector. Of course, the so-called second feature vector and supplementary feature vector are only a way of expression, and the second feature vector can also be considered as one or more feature vectors in addition to the dimension of the first feature vector, that is, the second feature vector can have different feature combination modes. In this experiment, different second feature vectors with different feature combination modes are used to evaluate the recognition performance of mild cognitive impairment.

[0043] Specifically, for different brain function diseases to be identified, the optimized feature combination mode of the second feature vector can be different, and the optimized proportion range of a and b can also be different. For example, in the case of identifying brain function diseases as mild cognitive impairment, the feature vector obtained based on the brain network correlation of the target brain of the subject is taken as the main second feature vector in the second feature vector, and the length proportion of the first feature vector and the feature vector obtained based on the brain network correlation of the target brain of the subject in the second feature vector is 1.8 to 2.5, especially when the proportion value of a and b is close to 2:1, the recognition effect is better. The effect of taking the feature vector obtained based on the brain network correlation of the target brain of the subject as the main second feature vector will be described based on the comparative example below, and will not be described here.

[0044] It should be noted that in this experiment, the target brain includes frontal lobe, left and right temporal lobe, parietal lobe and occipital lobe. The 39 channels of the fNIRS device are arranged on the target brain of the subject, and the brain network correlation of the target brain of the subject can be obtained by calculating the Pearson correlation coefficient of the near-infrared data of each brain region, which will not be repeated here. In other embodiments, the brain network correlation of the target brain can also be calculated using EEG or fMRI data, which is not limited in the present application.

[0045] In this experiment, for the comparative examples in which the brain network correlation is included in the second feature vector, regardless of whether other feature vectors are included in the second feature vector, the length ratio of the first feature vector to the feature vector obtained based on the brain network correlation of the target brain of the subject is set to 2: 1 to obtain the fusion feature vector.

[0046] In step S205, the fusion feature vector is mapped to the category prediction result of the brain function disease by using the fully connected output layer, that is, in this experiment, it is mapped to the recognition result of mild cognitive impairment.

[0047] By using the network architecture of ResNet18, the feature extraction-fusion-mild cognitive impairment is efficiently embedded in the existing modules of the network architecture, that is, the input layer-convolutional layer-pooling layer-linear layer-fully connected output layer. Without modifying ResNet18, the user can use mature source code when writing algorithms, which is more efficient, has lower cost, and is more user-friendly.

[0048] In this experiment, the following comparative experiments were performed. For the first near-infrared data of the healthy person and the mild cognitive impairment patient during the execution of the above-mentioned same delayed matching task (Delayed Matching-to-Sample Task, DMST), preprocessing and down-sampling were performed to obtain the target fNIRS data.

[0049] The control example is to directly apply the trained ResNet18 to the target fNIRS data, the comparative example 1 only performs steps 201 and 202 to extract a one-dimensional first feature vector, and uses a fully connected output layer to map the first feature vector to the recognition result of mild cognitive impairment. Comparative example 2, comparative example 3 and comparative example 4 all perform steps 201 to 205, the difference lies in the different feature combination ways of the second feature vector, comparative example 2 only uses the cognitive scale-based feature vector as the second feature vector, comparative example 3 only uses the feature vector obtained based on the brain network correlation as the second feature vector, and comparative example 4 combines the cognitive scale-based feature vector obtained based on the brain network correlation as the second feature vector. Further, in Table 1, it is also noted that the performances of ResNet18 in mild cognitive impairment recognition in comparative examples 3 and 4 after being trained with cross-entropy loss function (without using FocalLoss loss function) and FocalLoss loss function, respectively. In Table 1, in comparative examples 1 and 2, ResNet18 is trained using cross-entropy loss function (without using FocalLoss loss function).

[0050] Table 1 shows the performances of different features and loss functions in mild cognitive impairment recognition

[0051]

[0052] The average classification accuracy and F1 score of mild cognitive impairment recognition in various cases are listed in Table 1 as recognition performance parameters. Among them, the F1 score is an index for measuring the performance of a classification model, which considers the precision and recall of the model, and the F1 score ranges from 0 to 1. The closer the F1 score is to 1, the higher the recognition accuracy and the better the classification effect.

[0053] As shown in Table 1, comparative example 1 only uses the first feature vector based on the Gram angle field image data without feature fusion, and can obtain an average classification accuracy of 71.36% and an F1 score of 69.72%. According to the embodiments of the present application, the second feature vector is fused on the basis of the first feature vector, and the obtained average classification accuracy and F1 score are better than those without feature fusion.

[0054] The different feature combination ways of the second feature vector also have different performances in mild cognitive impairment recognition.

[0055] As can be seen from Comparative Examples 2 and 3, when ResNet18 is trained using the cross-entropy loss function, the average classification accuracy (74.57%) and F1 score (72.91%) obtained by using only the feature vector based on brain network correlation as the second feature vector for feature fusion with the first feature vector are significantly higher than the average classification accuracy (72.47%) and F1 score (70.46%) obtained by using only the feature vector based on cognitive scales as the second feature vector for feature fusion with the first feature vector. That is, when the second feature vector is the feature vector based on brain network correlation, the recognition performance for mild cognitive impairment is more significantly improved.

[0056] Further, when the second feature vector is also the feature vector based on brain network correlation, and the weight ratio of the first feature vector to the feature vector based on brain network correlation is 2:1, further incorporating other dimensional feature vectors for fusion can further improve the recognition performance for mild cognitive impairment. As can be seen from Comparative Example 3 and Comparative Example 4 in Table 1, when the weight ratio of the first feature vector to the feature vector based on brain network correlation to the feature vector based on cognitive scales is 2:1:0.1, even if the FocalLoss loss function is not used for training, the recognition performance is better than that of the training using the FocalLoss loss function when the weight ratio of the first feature vector to the feature vector based on brain network correlation is 2:1, and the feature vector based on cognitive scales is not incorporated.

[0057] As can be seen from Table 1, by training ResNet18 with FocalLoss loss function, the performance in mild cognitive impairment recognition can be further optimized under the same conditions. In some embodiments of the present application, the training samples faced by the ResNet18 model are imbalanced in quantity. For example, taking mild cognitive impairment as an example, when collecting the first near-infrared data of healthy people (first category samples) and mild cognitive impairment patients (second category samples), the first near-infrared data of the first category samples is easy to obtain, but it is often difficult to collect a sufficient number of first near-infrared data of the second category samples, resulting in an imbalance in the number of samples between the first category samples and the second category samples. Imbalanced sample quantity can easily lead the ResNet18 model to learn to classify the recognition results more to the first category, rather than deeply learning the characteristics of the second category samples. To this end, a loss function such as FocalLoss can be set, and when calculating the loss and adjusting the parameters accordingly, the loss weight of the first category samples is reduced, and the loss weight of the second category samples is increased, so that the second category samples with fewer quantities have greater influence, thereby balancing the influence of different category samples on parameter adjustment during model training, making the ResNet18 model pay more attention to the second category samples with fewer quantities, and improving the classification effect. The introduction of a loss function such as FocalLoss can not only be applied to ResNet18, but also to other neural networks that can perform the processing steps of the embodiments of the present application, so that they can pay more attention to the second category samples with fewer quantities, for example, increasing the weight of samples with brain function diseases compared to samples with healthy brain during training, which can offset the adverse effects of sample type quantity imbalance, and thus improve the classification effect.

[0058] The above experiments are only examples, and the present application is not limited to the identification of mild cognitive impairment, but can also be applied to the identification of various mild brain function diseases or early brain function diseases. For example, various mental illness patients, such as mild depression patients, mild insomnia patients, mild anxiety patients, and other brain function disease patients who are difficult to distinguish from healthy people in terms of brain function performance or using other physiological signal devices, their near-infrared data can be used to identify the corresponding brain function disease state.

[0059] The above experiments use the DMST task, which is only an example. In some specific embodiments, the subject can perform a series of tasks according to a pre-set task paradigm, such as cognitive tasks, emotional tasks, or motor tasks, etc. These tasks are associated with the brain function disease category of the subject, so that a more comprehensive and accurate method can be provided for precise identification of brain function diseases and verification of identification performance.

[0060] In some embodiments of the present application, a processing device for identifying brain function diseases based on near-infrared data using a neural network is provided, and the processing device comprises a processor configured to execute a method for identifying brain function diseases using a neural network according to various embodiments of the present application. The method can comprise the following steps performed by the processor. Based on the acquired first near-infrared data of a plurality of acquisition channels of a target brain of a subject, a Gram angular field transform is used to obtain Gram angular field image data. Based on the Gram angular field image data, the neural network is used to perform the following processing: performing convolution and pooling operations to convert into a first feature vector with a first dimension number, the first dimension number being below two; based on the first feature vector and a second feature vector with the first dimension number, obtaining a fusion feature vector, the second feature vector being obtained based on features of other dimensions of the target brain of the subject; and based on the fusion feature vector, determining an identification result of a brain function disease of the subject. The details or examples in the methods and steps described in various embodiments of the present application can be independently or combinedly combined herein, and will not be described here. The processing device converts the first near-infrared data into the Gram angular field image data, so as to present more rich information in two-dimensional or three-dimensional space for the time series data of a single dimension while preserving the main features of the first near-infrared data. The first feature vector is extracted based on the Gram angular field image data, and the same low-dimensional second feature vector is fused based on the first feature vector, the second feature vector being obtained based on features of other dimensions of the target brain of the subject. The dimensions of the first feature vector and the second feature vector are different, so that the fusion of multi-dimensional features is realized, and more comprehensive feature information is embodied. The same low dimension simplifies the fusion of the feature vectors, and enables the role weight of the first feature vector and the second feature vector in the fusion vector to be accurately controlled. Through subsequent experiments, it is proved that the fusion of the second feature vector based on the first feature vector can improve the recognition ability and efficiency of the complex differences between brain function disease patients (even mild brain function disease patients or early brain function disease patients) and healthy people, and is beneficial to improve the recognition accuracy of brain function diseases.

[0061] The processor can be a processing device including one or more general- purpose processing devices, such as microprocessors, central processing units (CPUs), graphics processing units (GPUs), and the like. More particularly, the processor can be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, processor running other instruction sets, or processors running combination of instruction sets. The processor can also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), system on a chip (SoC), and the like. The processor can cooperate with the near-infrared brain function imaging device that acquires the first near-infrared data to perform the related data analysis and processing.

[0062] The present application describes various operations or functions, which can be implemented as software code or instructions or defined as software code or instructions. Such content can be source code or differential code ("delta" or "patch" code) that can be directly executable ("object" or "executable" form). The software code or instructions can be stored in a computer-readable storage medium and, when executed, can cause a machine to perform the described functions or operations, and include any mechanism that stores information in a form accessible by a machine (e.g., computing device, electronic system, etc.), such as recordable or non-recordable media (e.g., read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

[0063] The exemplary method described in the present application can be at least partially implemented by a machine or computer. In some embodiments, a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the method of identifying brain function diseases based on near-infrared data using a neural network described in various embodiments of the present application. The method comprises: based on the acquired first near-infrared data of a plurality of acquisition channels of a target brain of a subject, using Gram angle field transformation to obtain Gram angle field image data; based on the Gram angle field image data, using a neural network to perform the following processing. Convolution and pooling operations are performed to convert into a first feature vector. Based on the first feature vector and a second feature vector having the first dimension, a fusion feature vector is obtained, the second feature vector being obtained based on features of other dimensions of the target brain of the subject. And, based on the fusion feature vector, an identification result of a brain function disease of the subject is determined. The details or examples in the methods and steps described in various embodiments of the present application can be independently or combinedly combined herein, which will not be described here. The processing device converts the first near-infrared data into Gram angle field image data, thereby presenting more rich information in two-dimensional or three-dimensional space for the time series data of a single dimension while preserving the main features of the first near-infrared data. The first feature vector is extracted based on the Gram angle field image data, and the same low-dimensional second feature vector is fused based on the first feature vector, the second feature vector being obtained based on features of other dimensions of the target brain of the subject. The dimensions of the first feature vector and the second feature vector are different, so that the fusion of multi-dimensional features is realized, which embodies more rich and comprehensive feature information; the same low dimension simplifies the fusion of feature vectors, and enables accurate control of the role weight of the first feature vector and the second feature vector in the fusion vector. Through subsequent experiments, it is proved that such fusion of the second feature vector based on the first feature vector can improve the recognition ability and efficiency of the complex differences between brain function disease patients (even mild brain function disease patients or early brain function disease patients) and healthy people, which is beneficial to improve the recognition accuracy of brain function diseases.

[0064] Implementations of such methods can include software code, e.g., microcode, assembly language code, a higher-level languages code, etc. Various software programming techniques and languages can be used to create the various programs or program modules. For example, program portions or program modules can be designed in or with Java, Python, C, C++, assembly language, or any known programming language. One or more of such software portions or modules can be integrated into a computer system and / or computer readable medium. Such software code can include computer readable instructions for performing various methods. The software code can form a part of a computer program product or computer program modules. Further, in an example, the software code can be tangibly stored on one or more volatile, non-transitory, or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media can include, but are not limited to, hard disk, removable magnetic disk, removable optical disk, magnetic cassette, memory card or stick, random access memory (RAM), read only memory (ROM), and the like.

[0065] Further, although example embodiments have been described herein, the scope of coverage of this patent will include any and all embodiments having equivalent elements, modifications, omissions, combinations (e.g., of

[0066] The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) can be used in combination with each other. Other examples can use, as an example, those to one of ordinary skill in the art upon reading the above description. In addition, in the specific embodiments described previously, various features can be grouped together or divided into different features for the purpose of streamlining the disclosure. This should not be interpreted as a requirement that the claimed subject matter must claim all features in any one claim. Rather, the subject matter has many alternate combinations and permutations of the described features that are within the scope of the disclosed subject matter. As such, the claims are not to be construed as a requirement that the claimed subject matter must cover all features enumerated in any one claim. Each claim is to be interpreted independently and in the context of the entire disclosure, with each claim being capable of being combined with any other claim or combination of claims to form a new claim.

[0067] The above examples are only exemplary embodiments of the present application, and are not intended to limit the present application, and the protection scope of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements to the present application within the spirit and protection scope of the present application, and such modifications or equivalent replacements are also considered to fall within the protection scope of the present application.

Claims

1. A method for identifying a brain function disease using a neural network based on near-infrared data, the method comprising: obtaining near-infrared data of a subject; and identifying the brain function disease of the subject using the neural network based on the near-infrared data. The method comprises, by a processor: based on the acquired first near-infrared data of a plurality of acquisition channels of the target brain of the subject, using Gram angular field transformation to obtain Gram angular field image data; and based on the Gram angular field image data, using the neural network to perform the following processing: performing convolution and pooling operations to convert into a first feature vector with a first dimension number, the first dimension number being below two dimensions; based on the first feature vector and a second feature vector with the first dimension number, obtaining a fusion feature vector, the second feature vector being obtained based on other dimensional features of the target brain of the subject; and based on the fusion feature vector, determining the recognition result of the brain function disease of the subject.

2. The method according to claim 1, characterized in that The brain function disease to be recognized includes a mild brain function disease or an early brain function disease with a slight difference compared to a healthy person.

3. The method of claim 1, wherein, The first dimension number includes one dimension, and the brain function disease to be recognized includes mild cognitive impairment.

4. The method according to claim 1 or 3, characterized in that, Based on the first feature vector and the second feature vector with the first dimension number, obtaining a fusion feature vector specifically includes: based on a preset length ratio relationship associated with the brain function disease to be recognized, the first feature vector and the second feature vector, concatenating the first feature vector and the second feature vector to obtain a fusion feature vector.

5. The method of claim 4, wherein, The preset length ratio relationship makes the length proportion of the first feature vector in the fusion feature vector greater than 60% and less than 75%.

6. The method of claim 4, wherein, The second feature vector includes at least one feature vector, and the corresponding dimension includes a detection modality other than near-infrared imaging, and the detection modality includes at least one of fMRI, cognitive scale and EEG.

7. The method of claim 4, wherein, The second feature vector includes at least one feature vector, which is obtained based on the brain network correlation of the target brain of the subject.

8. The method of claim 1, wherein, Based on the acquired first near-infrared data of a plurality of acquisition channels, using Gram angular field transformation to obtain Gram angular field image data, specifically comprising: using a piecewise aggregate approximation method to downsample the first near-infrared data of a plurality of acquisition channels to obtain second near-infrared data of a plurality of acquisition channels; using Gram angular field transformation on the second near-infrared data of each acquisition channel to obtain third Gram angular field image data of each acquisition channel; splicing the third Gram angular field image data of each acquisition channel to obtain the Gram angular field image data.

9. The method of claim 8, wherein, Using the piecewise aggregate approximation method for downsampling makes the sampling points after downsampling 45%-60% of the number of sampling points before downsampling.

10. The method of claim 8, wherein, Further comprising: using the convolution and pooling layers of the ResNet18 model to perform the convolution and pooling operations to convert into a first feature vector; using the linear layer of the ResNet18 model to adjust the weights of the first feature vector and the second feature vector in the fusion feature vector; using the fully connected output layer of the ResNet18 model to map the fusion feature vector to the category prediction result of the brain function disease.

11. The method according to any one of claims 1-10, characterized in that, The neural network is trained by using a loss function for the unbalanced sample quantity, and the loss function increases the weight of the sample with brain function disease of the target brain compared with the sample of the target brain health during training.

12. The method of claim 7, wherein, The preset length ratio relationship is such that the length ratio of the feature vector based on the brain network correlation of the target brain of the subject in the first feature vector and the second feature vector is 1.8 to 2.

5.

13. The method of claim 7, wherein, The second feature vector includes a third feature vector and a fourth feature vector, the third feature vector is obtained based on the brain network correlation of the target brain of the subject, the fourth feature vector is obtained based on the cognitive scale of the subject, and the neural network is trained by using a Focalloss loss function.

14. A processing device for identifying brain functional diseases using a neural network based on near-infrared data, characterized in that: The processing device includes a processor configured to execute the method for identifying brain function disease based on near-infrared data by using a neural network according to any one of claims 1-13.

15. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program causes the processor to execute the method for identifying brain function disease based on near-infrared data by using a neural network according to any one of claims 1-13 when the computer program is executed by the processor.

Images