Functional near-infrared spectroscopy and infrared thermography identification method for attention deficit hyperactivity disorder

Abstract

The present invention discloses a method for identifying attention deficit hyperactivity disorder (ADHD) using functional near-infrared spectroscopy (fNIRS) and infrared thermography. The method includes separately collecting near-infrared blood oxygenation data of a subject in a resting state and a task state, as well as infrared thermograms of the head and neck regions. Then, the near-infrared blood oxygenation data is preprocessed and feature-extracted. After normalization and principal component analysis (PCA) for feature dimensionality reduction, it is input into a functional near-infrared blood oxygenation data anomaly detection model for detection, and a blood oxygen classification result is output. The infrared thermogram is input into an infrared thermogram anomaly detection model to output an infrared thermogram classification result. Finally, the blood oxygen classification result and the infrared thermogram classification result are summarized. The present invention utilizes the non-invasive, non-intrusive, and wide applicability characteristics of functional near-infrared spectroscopy and infrared thermography examinations to quickly and accurately achieve the purpose of classifying abnormal near-infrared blood oxygenation data and abnormal regions in infrared thermograms. Combining the two detection methods can make the detection results more comprehensive, reliable, and persuasive.

Description

Technical Field

[0001] The present invention relates to the technical fields of near-infrared spectroscopy recognition and image recognition, and specifically to a recognition method for functional near-infrared spectroscopy and infrared thermography of attention deficit hyperactivity disorder (ADHD). Background Art

[0002] Attention Deficit Hyperactivity Disorder (ADHD) is a syndrome of mild brain dysfunction, commonly known as ADHD, which is a common neurobehavioral disorder in childhood. The main manifestations are general inattention and excessive impulsive and hyperactive behaviors. It causes harm to functions in multiple fields, and this harm may persist into adulthood. There are research surveys showing that the worldwide prevalence of ADHD is approximately 5.9% to 7.1% in children and approximately 1.2% to 7.3% in adults. The total prevalence of ADHD in children in China is 5.5%, and the prevalence in boys is higher than that in girls. ADHD has two manifestations: attention disorder and impulsive hyperactivity, and they may also occur simultaneously. Currently, the detection of attention deficit hyperactivity disorder is mostly based on the patient's main complaints and medical history, questionnaire assessment and other methods, and the conclusion is drawn by a doctor's comprehensive diagnosis, which is highly subjective and has a low judgment accuracy.

[0003] Functional near-infrared spectroscopy technology is a new tool for studying cerebral hemodynamic responses, which can detect changes in local tissue blood oxygen parameters caused by brain neuron activities and reflect the cerebral cortical hemodynamic state in real time. Using functional near-infrared spectroscopy combined with infrared thermography, there are relevant physiological indicators that can be quantitatively evaluated, which can meet the requirements of clinical rapid detection, have a relatively low average detection cost, and can greatly improve the detection efficiency through machine learning and deep learning models for auxiliary judgment. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a recognition method for functional near-infrared spectroscopy and infrared thermography of ADHD, so as to quickly obtain the classification results in the functional near-infrared spectroscopy and infrared thermography images of patients with attention deficit hyperactivity disorder (ADHD).

[0005] To solve the above technical problem, the present invention provides a recognition method for functional near-infrared spectroscopy and infrared thermography of ADHD, and the specific process includes:

[0006] Step S1: Collect the near-infrared blood oxygen data of the subject in the resting state and the task state, as well as the infrared thermograms of the head and neck regions respectively;

[0007] Step S2: Blood oxygen abnormality detection

[0008] Step S2.1: Data preprocessing

[0009] In the host computer, the near-infrared blood oxygen data in the resting state and task state are respectively trimmed and removed for the first 5 seconds and the last 5 seconds at both ends, and then the last 30 seconds of the near-infrared blood oxygen data in the trimmed resting state and the last 30 seconds of the task state are respectively taken and spliced into a 60-second preprocessed near-infrared blood oxygen data;

[0010] Step S2.2, Feature extraction

[0011] Calculate the Pearson correlation coefficient and wavelet coherence coefficient of the preprocessed near-infrared blood oxygen data, and then splice the Pearson correlation coefficient and wavelet coherence coefficient into a 648-dimensional feature vector, and then perform normalization and principal component analysis PAC feature reduction;

[0012] Step S2.3, Input the features after normalization and principal component analysis PAC feature reduction into the functional near-infrared blood oxygen data anomaly detection model for detection, and output the blood oxygen classification result;

[0013] Step S3, Infrared thermal image anomaly detection

[0014] In the host computer, scale the infrared thermal image to an image with a resolution of 300*300, input it into the infrared thermal image anomaly detection model, and output the infrared thermal image classification result;

[0015] Step S4, Summarize the blood oxygen classification result and the infrared thermal image classification result, and present them in the host computer in four cases: both the blood oxygen classification result and the infrared thermal image classification result are 0; both the blood oxygen classification result and the infrared thermal image classification result are 1; the blood oxygen classification result is 1 and the infrared thermal image classification result is 0; the blood oxygen classification result is 0 and the infrared thermal image classification result is 1.

[0016] As an improvement to the method for identifying functional near-infrared spectroscopy and infrared thermal images of attention deficit hyperactivity disorder of the present invention:

[0017] The functional near-infrared blood oxygen data anomaly detection model includes a parallel random forest classifier and a support vector machine classifier. The probabilities output by the two classifiers are arithmetically averaged, and the binary classification result output after threshold judgment is used as the blood oxygen classification result.

[0018] As a further improvement to the method for identifying functional near-infrared spectroscopy and infrared thermal images of attention deficit hyperactivity disorder of the present invention:

[0019] The infrared thermal image anomaly detection model includes a target detection network module and an infrared thermal image classification network module connected in series. The binary classification result output after the target region feature map output by the target detection network module passes through the infrared thermal image classification network module is used as the infrared thermal image classification result;

[0020] The target detection network module is based on the SSD target detection network. The outputs of the convolutional layers conv4_3, conv7, conv8_2, and conv9_2 in the SSD target detection network are respectively passed through an attention mechanism model and then connected to the target detection layer. The convolutional layers conv10_2 and conv11_2 are directly connected to the target detection layer. Then, after passing through the target detection layer and non-maximum suppression processing, the output is the target region feature map;

[0021] The infrared thermal image classification network module uses the AlexNet network as the backbone network, including 5 consecutive convolutional layers, 3 fully connected layers, and 1 softmax layer. The convolutional layer Conv2 in the AlexNet network is replaced with an Inception module, and the number of neurons in the softmax layer is changed to 2.

[0022] As a further improvement to the method for identifying functional near-infrared spectroscopy and infrared thermal images of attention deficit hyperactivity disorder according to the present invention:

[0023] The parameters of the random forest classifier are as follows: the number of decision trees is 50, the Gini index gain value is used as the basis for the decision tree to select features, the maximum depth is 10, and the minimum number of samples for a branch is 2;

[0024] The parameters of the support vector machine classifier are as follows: the kernel function is the RBF function, the gamma value is the reciprocal of the number of features, and the penalty parameter c is 1.0.

[0025] As a further improvement to the method for identifying functional near-infrared spectroscopy and infrared thermal images of attention deficit hyperactivity disorder according to the present invention:

[0026] The near-infrared blood oxygen data includes four channels, and each channel contains 4 parameters, namely TOI blood oxygen saturation, THI hemoglobin concentration index, ΔCHb deoxyhemoglobin change value, and ΔCHbO2 oxyhemoglobin change value;

[0027] The Pearson correlation coefficient is calculated by pairwise combining the preprocessed near-infrared blood oxygen data according to the four channels according to formula (1), and then the same parameter is taken from the two combined channels to calculate the Pearson correlation coefficient, and a total of 24 Pearson correlation coefficients are obtained;

[0028]

[0029] Among them, cov is the covariance, σ is the standard deviation, E is the expected value, ρ is the Pearson correlation coefficient, and X and Y are the signal values of the same parameter taken on the two channels;

[0030] The wavelet coherence coefficient is calculated by formula (2) for the preprocessed near-infrared blood oxygen data in pairs among four channels respectively, and then 624 wavelet coherence coefficients in 26 frequency bands are calculated by taking the same parameter for both of the two combined channels:

[0031]

[0032] Among them, C XY is the wavelet coherence coefficient of signals X and X, P X is the cross-power spectral density of X and Y, P X is the power spectral density of X, and P Y is the power spectral density of Y.

[0033] As a further improvement to the method for identifying functional near-infrared spectroscopy and infrared thermography for attention deficit hyperactivity disorder of the present invention:

[0034] The normalization is performed using a standard normalizer with a mean of 0 and a variance of 1:

[0035]

[0036] Among them, x' is the value after normalization, x is the original value, is the mean, and s is the standard deviation;

[0037] The principal component analysis PAC dimensionality reduction is to sort and accumulate the variances of the principal components from large to small, and retain all features until the cumulative value of the proportion of the variances of the principal components exceeds 0.9.

[0038] As a further improvement to the method for identifying functional near-infrared spectroscopy and infrared thermography for attention deficit hyperactivity disorder of the present invention:

[0039] The training process of the functional near-infrared blood oxygen data anomaly detection model is as follows: collect the prefrontal near-infrared blood oxygen data of the resting state and task state of the patient group and normal control group evaluated by doctors in the top three hospitals, perform the data preprocessing described in step S2.1 and the feature extraction described in step S2.2 on the near-infrared blood oxygen data to obtain feature vectors, and then use the features after the normalization described in step S2.3 and the principal component analysis PAC feature dimensionality reduction as the blood oxygen data set E1 for training the functional near-infrared blood oxygen data anomaly detection model; divide the blood oxygen data set E1 into a training set and a test set at a ratio of 4:1, input the training set into the functional near-infrared blood oxygen data anomaly detection model, and set the maximum number of iterations of the random forest classifier and the support vector machine classifier to 300 for training respectively. After the training is completed, input the test set into the trained functional near-infrared blood oxygen data anomaly detection model for detection and verification to obtain the functional near-infrared blood oxygen data anomaly detection model for online use.

[0040] As a further improvement to the method for identifying attention deficit hyperactivity disorder (ADHD) using functional near-infrared spectroscopy (fNIRS) and infrared thermography in the present invention:

[0041] The training process of the infrared thermography anomaly detection model is as follows: Infrared thermograms of the head and neck regions of the patient group and the normal control group are collected. Each infrared thermogram is marked with the position and classification label of the prefrontal region by doctors from a top-three hospital and then used as the infrared thermogram set E2. Then, the infrared thermogram set E2 is used as the training sample of the target detection network module. 20% of the samples are randomly selected as the cross-validation set during network training. The loss function for training is the loss function of the SSD target detection network. Training is carried out for 1000 epochs, and the model with the smallest loss function value on the cross-validation set is saved as the trained target detection network module. The infrared thermograms in the infrared thermogram set E2 pass through the trained target detection network module to output the target region feature map. The target region feature map and the classification label of its original infrared thermogram are used as the training samples of the infrared thermogram classification network. 20% of the samples are randomly selected as the cross-validation set during the training of the infrared thermogram classification network. The objective function is the cross-entropy loss function. Training is carried out for 1000 epochs, and the model with the smallest loss function value on the cross-validation set is saved as the infrared thermogram classification network for online use.

[0042] As a further improvement to the method for identifying attention deficit hyperactivity disorder (ADHD) using functional near-infrared spectroscopy (fNIRS) and infrared thermography in the present invention:

[0043] The pairwise combination method between the four channels is channel 1 and channel 2, channel 1 and channel 3, channel 1 and channel 4, channel 2 and channel 3, channel 2 and channel 4, channel 3 and channel 4;

[0044] When calculating the wavelet coherence coefficient, the 0.02 Hz - 0.15 Hz range is evenly divided into the 26 frequency bands.

[0045] The beneficial effects of the present invention are mainly reflected in:

[0046] The present invention utilizes the non-invasive, non-intrusive, and wide applicability characteristics of functional near-infrared spectroscopy and infrared thermography examinations. It combines these two detection methods, constructs a feature extraction method for brain functional connectivity, combines innovative time series data processing techniques and deep learning image processing techniques and analysis methods, and through the training of machine learning models, can quickly and accurately achieve the purpose of classifying abnormal near-infrared blood oxygen data and abnormal parts in infrared thermograms. Combining the two detection methods can make the detection results more comprehensive, reliable, and persuasive. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The following further details the specific embodiments of the present invention with reference to the accompanying drawings.

[0048] Figure 1 Schematic flow diagram of the method for identifying attention deficit hyperactivity disorder (ADHD) using functional near-infrared spectroscopy (fNIRS) and infrared thermography;

[0049] Figure 2 is Figure 1 Flow and model structure diagrams for detecting abnormal near-infrared blood oxygenation in the middle part;

[0050] Figure 3 is Figure 1 Flow and model structure diagrams for detecting abnormal infrared thermography in the middle part;

[0051] Figure 4 is Figure 3 Network structure diagram of the target detection network module in the middle part;

[0052] Figure 5 is Figure 3 Network structure diagram of the infrared thermography classification network in the middle part;

[0053] Figure 6 Is an example of four near-infrared blood oxygenation data of test samples. Specific implementation manner

[0054] The present invention will be further described below in conjunction with specific embodiments, but the protection scope of the present invention is not limited thereto:

[0055] Embodiment 1. The method for identifying ADHD using fNIRS and infrared thermography, as Figures 1 - 6 shown, includes the following steps:

[0056] Step 1. Collect near-infrared blood oxygenation data of the prefrontal lobe based on fNIRS technology and perform blood oxygenation abnormality detection, as Figure 2 shown;

[0057] Step 1.1. Collect near-infrared blood oxygenation data;

[0058] Use a non-invasive near-infrared tissue blood oxygen parameter monitor to collect near-infrared blood oxygenation data of the prefrontal lobe of the subject in the resting state and the task state respectively. The non-invasive near-infrared tissue blood oxygen parameter monitor includes four collection probes, and each probe includes a light source and a detector to form a channel; the near-infrared blood oxygenation data includes 4 channels, and each channel includes 4 parameters, namely: TOI blood oxygen saturation, THI hemoglobin concentration index, ΔCHb deoxyhemoglobin change value, ΔCHbO2 oxyhemoglobin change value. Therefore, the collected near-infrared blood oxygenation data includes near-infrared blood oxygenation data of 4 parameters × 4 channels, a total of 16 signals in the resting state and near-infrared blood oxygenation data of 4 parameters × 4 channels, a total of 16 signals in the task state.

[0059] The resting state is when the subject sits still for 45 seconds. Immediately afterwards, the task state is a Go / NoGo paradigm evaluation task, which lasts for 1 minute and 30 seconds. The Go / NoGo task is a commonly used paradigm for studying the ability to stop responses and is often used in the detection of attention deficit hyperactivity disorder (ADHD). In this task, two different letters or patterns are randomly and alternately presented on the screen. The subject is required to respond to one of the stimuli (Go) and not respond to the other stimulus (NoGo). An incorrect response to the NoGo stimulus is usually considered an indicator of difficulty in response inhibition.

[0060] In the Go / NoGo paradigm evaluation task adopted in this embodiment, the Go stimulus is the stimulus of presenting the number "1" on the screen, and the NoGo stimulus is the stimulus of presenting the number "2" on the screen. Three seconds after the start of the task, a cross will be presented in the center of the screen as a sign to prompt the subject that the stimulus is about to start, lasting for 7 seconds. After that, a random stimulus will be presented every 1 second, and the stimulus lasts for 1 second, with a total of 40 groups of stimuli. The overall ratio of Go stimuli to NoGo stimuli is 3:1.

[0061] Collecting near-infrared blood oxygen data in the resting state and the task state includes 4 parameters, especially the TOI blood oxygen saturation and the THI hemoglobin concentration index, rather than only the ΔCHb deoxyhemoglobin change value and the ΔCHbO2 oxyhemoglobin change value, which can more comprehensively reflect the physiological activity state of the subject's brain.

[0062] Step 1.2, preprocessing of near-infrared blood oxygen data

[0063] The near-infrared blood oxygen data in the resting state (16 signals in total, 4 parameters × 4 channels) and the near-infrared blood oxygen data in the task state (16 signals in total, 4 parameters × 4 channels) (collected together, with continuous signals, and the original data is as attached Figure 6 shown) are respectively trimmed and removed for the first and last 5 seconds. Then, the near-infrared blood oxygen data of the last 30 seconds in the resting state and the last 30 seconds in the task state in the trimmed near-infrared blood oxygen data segments are spliced together to form a total of 60 seconds of near-infrared blood oxygen data as the preprocessed near-infrared blood oxygen data. Each sample still has 16 signals, 4 parameters × 4 channels, and corresponds to the original channels and parameters.

[0064] Step 1.3, extracting features from the preprocessed near-infrared blood oxygen data by constructing brain functional connectivity;

[0065] For the preprocessed near-infrared blood oxygen data, pairwise combinations are made between the four channels. The pairwise combination methods of the channels are channel 1 and 2, 1 and 3, 1 and 4, 2 and 3, 2 and 4, 3 and 4. Then, the same parameter is taken for the two combined channels to calculate the Pearson correlation coefficient. For example, calculate the Pearson correlation coefficient between the TOI of channel 1 and the TOI of channel 2, the THI of channel 2 and the THI of channel 3, the ΔCHb of channel 3 and the ΔCHb of channel 4, and so on. A total of Pearson correlation coefficients are obtained; the calculation formula for the Pearson correlation coefficient is:

[0066]

[0067] where cov is the covariance, σ is the standard deviation, E is the expected value, ρ is the Pearson correlation coefficient, and X and Y are the signal values of the same parameter taken on the two channels respectively.

[0068] Then, similarly, for the preprocessed near-infrared blood oxygen data, pairwise combinations are made between the four channels, and then the same parameter is taken for the two combined channels to calculate the wavelet coherence coefficients at 26 frequency bands (evenly divided into 26 frequency bands within 0.02 Hz - 0.15 Hz), and wavelet coherence coefficients are obtained; the calculation formula for the coherence coefficient is:

[0069]

[0070] where C XY is the wavelet coherence coefficient of signals X and X, P X is the cross-power spectral density of X and Y, P X is the power spectral density of X, and P Y is the power spectral density of Y.

[0071] The preprocessing and feature extraction of functional near-infrared blood oxygen data using the calculation of wavelet coherence coefficients between channels can make full use of the time-domain and frequency-domain information of the data and reflect the brain functional connectivity status.

[0072] Then, the 24 Pearson correlation coefficients and 624 wavelet coherence coefficients of each sample are concatenated into a 648-dimensional feature vector.

[0073] Step 1.4. Construct and train a functional near-infrared blood oxygen data anomaly detection model

[0074] Step 1.4.1. Construct a dataset for training and testing

[0075] 1), In the way of steps 1.1 - 1.3, actually collect the near-infrared blood oxygen data of the prefrontal lobe in the resting state and task state of 40 patients (classification label is 1, abnormal) and 10 normal controls (classification label is 0, normal) who have been evaluated by doctors in a tertiary hospital, and extract features to obtain a 648-dimensional feature vector. Each subject is collected 3 times to obtain 3 training samples for each subject, with a total of 150 training samples, that is, 150 648-dimensional feature vectors;

[0076] 2), Normalize the 150 648-dimensional feature vectors respectively, and perform principal component analysis PAC feature dimension reduction:

[0077] Normalize the input 648-dimensional feature vector using a standard normalizer with a mean of 0 and a variance of 1:

[0078]

[0079] where x′ is the value after normalization, x is the original value, is the mean, and s is the standard deviation;

[0080] Perform principal component analysis PAC dimension reduction on the normalized feature vectors, sort and accumulate the variances of the principal components from large to small, and retain all features until the cumulative value of the proportion of the variance sum of the principal components exceeds 0.9;

[0081] All the features after principal component analysis PAC dimension reduction are used as the blood oxygen data set E1 for training the functional near-infrared blood oxygen data anomaly detection model.

[0082] Step 1.4.2: Construct a functional near-infrared blood oxygen data anomaly detection model

[0083] The functional near-infrared blood oxygen data anomaly detection model includes a random forest classifier and a support vector machine classifier (Reference [1] Bebortta S, Panda M, Panda S. Classification of pathological disorders in children using random forest algorithm [C] / / 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE). IEEE, 2020.). The two classifiers are connected in parallel. The parameters of the random forest classifier are: the number of decision trees is 50, the Gini index gain value is used as the basis for the decision tree to select features, the maximum depth is 10, and the minimum number of samples for a branch is 2. The parameters of the support vector machine classifier are: the kernel function is the RBF function, the gamma value is the reciprocal of the number of features, and the penalty parameter c is 1.0.

[0084] After the blood oxygen dataset E1 is input into the random forest classifier and the support vector machine classifier respectively, the probability of abnormal samples (output range 0.0 - 1.0) is output by each of the two classifiers. The arithmetic mean of the two is calculated, and a threshold judgment is made to output the blood oxygen classification result: Let the threshold be 0.5. If it is greater than or equal to 0.5, the final output classification result is 1; if it is less than 0.5, the output classification result is 0.

[0085] The threshold judgment of the outputs of the two classifiers of the functional near-infrared blood oxygen data anomaly detection model, through the method of ensemble learning, combines the two classifiers to reduce randomness and ensure the stability of the model.

[0086] Step 1.4.3, Train the functional near-infrared blood oxygen data anomaly detection model

[0087] The blood oxygen dataset E1 obtained in Step 1.4.1 is divided into a training set and a test set for the functional near-infrared blood oxygen data anomaly detection model at a ratio of 4:1. The training set is input into the functional near-infrared blood oxygen data anomaly detection model for training. The maximum number of iterations of the random forest classifier and the support vector machine classifier are set to 300 respectively. The CPU of the training machine environment is Intel(R) Xeon(R) E5-2667@3.20GHz, the GPU is Nvidia Tesla P4, and the operating system is Centos7.3.1611. After the training is completed, the trained functional near-infrared blood oxygen data anomaly detection model is output.

[0088] Then, the test set is input into the trained functional near-infrared blood oxygen data anomaly detection model for detection and verification, so as to obtain a functional near-infrared blood oxygen data anomaly detection model that can be used online. Finally, the detection results on the test set are as follows in the table:

[0089] Accuracy Recall F1 Score Normal 90％ 100％ 95％ Abnormal 100％ 67％ 80％ Weighted Average 92％ 92％ 91％

[0090] Among them, the accuracy is the ratio of the number of correctly classified samples to the total number of samples.

[0091] The recall is the ratio of the number of correctly classified samples among all normal / anomalous samples.

[0092] The F1 score is an index used to measure the classification effect of the model, which can take into account both recall and precision. The calculation method is

[0093] Among them, the precision is the proportion of samples that are actually normal / anomalous among the samples identified as normal / anomalous.

[0094] Step 2: Collect the infrared thermogram data of the subject and perform infrared thermogram anomaly detection, as Figure 3 described;

[0095] Step 2.1: Construct a training data set

[0096] A total of 500 infrared thermograms including the head and neck regions are actually collected from 40 patients (the same patients as in Step 1.4.1, with a classification label of 1, abnormal) and 10 normal controls (the same as the normal group in Step 1.4.1, with a classification label of 0, normal). Each infrared thermogram is annotated with the position and classification label of the prefrontal region after being evaluated by doctors in a tertiary hospital; the annotated infrared thermograms and the corresponding classification labels are used as the infrared thermogram set E2 for the infrared thermogram anomaly detection model for training, and the resolution of the infrared thermograms is 256*324.

[0097] Step 2.2: Construct an infrared thermogram anomaly detection model

[0098] The infrared thermogram anomaly detection model includes a target detection network module and an infrared thermogram classification network module connected in sequence. The input of the target detection network module is the original infrared thermogram, and the output is the target region feature map of the prefrontal region. The target region feature map is used as the input of the infrared thermogram classification network module, and finally the infrared thermogram classification network outputs a binary classification result with a label of 0 or a label of 1.

[0099] Step 2.2.1: Construct a target detection network module

[0100] The target detection network module is constructed by adding an attention mechanism model to the SSD target detection network. The infrared thermal images in the infrared thermal image set E2 are scaled to a resolution of 300*300 and input into the target detection network module. The outputs of the convolutional layers conv4_3, conv7, conv8_2, and conv9_2 in the SSD target detection network are respectively passed through an attention mechanism model and then connected to the detection layer. The convolutional layers conv10_2 and conv11_2 are directly connected to the detection layer. Then, after passing through the detection layer and non-maximum suppression (NMS) processing, the output is the feature map of the prefrontal target area. The target detection network module is as Figure 4 shown. Among them, the output of the detection layer may have multiple target bounding boxes, and these candidate target bounding boxes may overlap with each other. At this time, the non-maximum suppression algorithm can be used to find the best target bounding box and eliminate redundant bounding boxes. The advantage of choosing the SSD target detection network is that the SSD target detection network uses 6 different feature maps to detect targets of different scales, with the lower layers predicting small targets and the higher layers predicting large targets, which can effectively adapt to the detection of patients of different ages and head sizes. Adding an attention mechanism module to the SSD target detection network can focus on the more critical information in the infrared thermal images of detecting patients with ADHD, reduce the attention to other information, filter out irrelevant information, and improve the efficiency and accuracy of model processing.

[0101] The input of the attention mechanism model is the feature map F output by the corresponding convolutional layer, and the output is the feature map F″ after attention mapping; the specific processing process is as follows:

[0102] Input the original feature map into the attention mechanism, and the attention mechanism performs max-pooling and average-pooling processing on the input feature map respectively, then inputs the processed results into the same multi-layer perceptron, adds the respective outputs, and after passing through the sigmoid activation function, obtains the channel attention mapping matrix (C is the number of channels, H is the image height, and W is the image width); pass the feature map through the channel attention mapping matrix to obtain the feature map F′ through the formula F′ = M c (F)*F; then, perform max-pooling and average-pooling processing on the feature map F′ at the same time, perform a convolution operation on the processed results to extract features, and after passing through the sigmoid activation function, obtain the spatial attention mapping matrix The feature map F′ is passed through the spatial attention mapping matrix The feature map F″ is obtained through the formula F″ = M c (F′)*F′, and the feature map F″ is the final output of the attention mechanism module.

[0103] Step 2.2.2, construct the infrared thermal image classification network

[0104] The infrared thermal image classification network uses the AlexNet network as the backbone network. The image size of the input target region feature map is 224*224*3. The original AlexNet network includes 5 consecutive convolutional layers, 3 fully connected layers, and 1 softmax layer. The parameters of the 5 convolutional layers are as follows in the table:

[0105] Convolutional Layer Number of Convolutional Kernels Size of Convolutional Kernels Stride Padding Conv1 48*2 11*11 4 0 Conv2 128*2 5*5 2 1 Conv3 192*2 3*3 1 1 Conv4 192*2 3*3 1 1 Conv5 128*2 3*3 1 1

[0106] After the 5 convolutional layers, there are 3 fully connected layers (FC6, FC7, FC8). The number of neurons in each fully connected layer is 4096. Finally, there is an output softmax layer with 1000 neurons after the fully connected layers.

[0107] In the original AlexNet network, the convolutional layer Conv2 in the original AlexNet network is replaced with the Inception module, which allows the model to learn multi-scale features and obtain better image representation information. At the same time, the number of neurons in the final softmax layer is changed to 2 (due to binary classification output), thus constructing the infrared thermal image classification network, as shown in the appendix Figure 5 as follows

[0108] Step 2.3, train the infrared thermal image anomaly detection model

[0109] The CPU of the training machine environment is Intel(R) Xeon(R) E5-2667@3.20GHz, the GPU is Nvidia Tesla P4, and the operating system is Centos 7.3.1611. The training process is as follows:

[0110] Step 2.3.1, train the target detection network module:

[0111] Use the infrared thermal images in the infrared thermal image set E2 obtained in Step 2.1 and the corresponding marked positions and classification labels of the prefrontal regions for training.

[0112] Randomly select 20% of the samples from the infrared thermal image set E2 as the cross-validation set during network training. The loss function for training the target detection network module is the loss function of the original SSD network (this loss function is described in detail in the original SSD paper [[2]Liu W, Anguelov D, Erhan D, et al. SSD: Single Shot MultiBox Detector[J]. Springer, Cham, 2016.]), which includes the log loss for classification and smooth L1 for localization. The learning rate is 2e-5, the optimizer is Adam, the batchsize is 16, and it is trained for 1000 epochs. Save the model with the minimum loss function value on the cross-validation set as the trained target detection network module.

[0113] Step 2.3.2, train the infrared thermal image classification network:

[0114] The infrared thermal images in the infrared thermal image set E2 pass through the trained target detection network module to output the target region feature map. The target region feature map and the classification label corresponding to its original infrared thermal image are used as the training samples of the infrared thermal image classification network.

[0115] Randomly select 20% of the samples from the training samples of the infrared thermal image classification network as the cross-validation set during training the infrared thermal image classification network. The objective function is the cross-entropy loss function, the learning rate is 1e-5, the optimizer is Adam, the batchsize is 16, and it is trained for 1000 epochs. Save the model with the minimum loss function value on the cross-validation set as the finally output trained infrared thermal image classification network.

[0116] Obtain the trained infrared thermal image anomaly detection model that can be used online through steps 2.3.1 and 2.3.2.

[0117] Step 3, online use

[0118] 1), Use a near-infrared tissue oxygenation parameter non-invasive monitor to collect the near-infrared blood oxygen data of the prefrontal lobe of the subject in the resting state and the task state respectively. The near-infrared blood oxygen data includes four channels, and each channel contains 4 parameters (TOI blood oxygen saturation, THI hemoglobin concentration index, ΔCHb deoxyhemoglobin change value, and ΔCHbO2 oxyhemoglobin change value);

[0119] 2), Data preprocessing

[0120] Preprocess the collected near-infrared blood oxygen data of the prefrontal lobe in the resting state and task state in the host computer according to Step 1.2: Crop and remove the first 5 seconds and the last 5 seconds of the near-infrared blood oxygen data in the resting state (16 signals in total, 4 parameters × 4 channels) and the near-infrared blood oxygen data in the task state (16 signals in total, 4 parameters × 4 channels) respectively. Then, splice the near-infrared blood oxygen data of the last 30 seconds in the resting state and the last 30 seconds in the task state in the cropped near-infrared blood oxygen data fragments into a total of 60 seconds of blood oxygen data as the preprocessed near-infrared blood oxygen data. Each sample still has 16 signals in total, 4 parameters × 4 channels, and corresponds to the original channels and parameters;

[0121] 3), Feature extraction and blood oxygen abnormality detection

[0122] According to Step 1.3, combine the preprocessed near-infrared blood oxygen data in pairs between the four channels. Then, for each pair of combined channels, calculate the Pearson correlation coefficient and the wavelet coherence coefficient at 26 frequency bands by taking the same parameter. Then, splice the Pearson correlation coefficient and the wavelet coherence coefficient of each sample into a 648-dimensional feature vector. After normalization and principal component analysis PAC feature reduction, input it into the functional near-infrared blood oxygen data abnormality detection model trained in Step 1, so as to output the blood oxygen classification result (label 0 or 1);

[0123] 4), Collect the infrared thermal images of the head and neck regions of the same subject, scale the infrared thermal images to an image with a resolution of 300*300 in the host computer, and input them into the infrared thermal image abnormality detection model trained in Step 2, so as to obtain the infrared thermal image classification result (label 0 or 1);

[0124] 5), Summarize the generated blood oxygen classification results and infrared thermal image classification results and present them in the host computer for the reference of the physician. There are four cases in total:

[0125] Both the blood oxygen classification result and the infrared thermal image classification result are 0;

[0126] Both the blood oxygen classification result and the infrared thermal image classification result are 1;

[0127] The blood oxygen classification result is 1 and the infrared thermal image classification result is 0;

[0128] The blood oxygen classification result is 0 and the infrared thermal image classification result is 1.

[0129] Experiment 1:

[0130] The machine environment of the experimental equipment is Intel(R) Xeon(R) E5-2667@3.20GHz CPU, Nvidia TeslaP4 GPU, and Centos 7.3.1611 operating system. The experimental data are the functional near-infrared blood oxygen data and infrared thermal images collected from 80 patients (abnormal) and 20 normal controls (normal) at rest and task states after evaluation by doctors from tertiary hospitals. Each subject collected 1 set of functional near-infrared blood oxygen data, totaling 100 sets; each subject took 1 infrared thermal image, totaling 100 images.

[0131] The functional near-infrared blood oxygen data of the subject is processed according to step 1.2 and step 1.3 in Example 1 and input into the functional near-infrared blood oxygen data anomaly detection model that can be used online in Example 1. The infrared thermal image of the subject is scaled to a resolution of 300*300 and input into the infrared thermal image anomaly detection model that can be used online in Example 1. The classification results output by the functional near-infrared blood oxygen data anomaly detection model and the infrared thermal image anomaly detection model and the classification results after the combination of the two are statistically analyzed respectively. The final detection result indicators on the experimental data set are as follows:

[0132] The accuracy of the functional near-infrared blood oxygen data anomaly detection model is 88%; the accuracy of the infrared thermal imaging anomaly detection model is 79%; combining the two classification results, the corresponding subject is judged to be abnormal only when both blood oxygen and infrared thermal imaging are abnormal, with an accuracy of 91%.

[0133]

[0134] The results show that the classification results of the present invention combining the above-mentioned functional near-infrared blood oxygen data anomaly detection model and the infrared thermal imaging anomaly detection model are more comprehensive, reliable, and convincing, and have significant clinical significance.

[0135] Finally, it should be noted that the above examples are only some specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments, and there are many variations. All variations that can be directly derived or associated with the content disclosed by a person skilled in the art should be considered as the protection scope of the present invention.

Claims

1. Functional near-infrared spectroscopy and infrared thermography identification method for attention deficit hyperactivity disorder, characterized in that The specific process is as follows: Step S1: Collect the near-infrared blood oxygen data of the subject in the resting state and the task state, as well as the infrared thermal images of the head and neck regions respectively; Step S2: Blood oxygen abnormality detection Step S2.1: Data preprocessing In the host computer, the near-infrared blood oxygen data in the resting state and the task state are respectively trimmed and removed for the first 5 seconds and the last 5 seconds at the head and the tail, and then the last 30 seconds of the near-infrared blood oxygen data in the trimmed resting state and the last 30 seconds of the near-infrared blood oxygen data in the task state are respectively taken and spliced into a 60-second preprocessed near-infrared blood oxygen data; Step S2.2: Feature extraction Calculate the Pearson correlation coefficient and the wavelet coherence coefficient of the preprocessed near-infrared blood oxygen data, and then splice the Pearson correlation coefficient and the wavelet coherence coefficient into a 648-dimensional feature vector and then perform normalization and principal component analysis PAC feature dimensionality reduction; Step S2.3: Input the features after normalization and principal component analysis PAC feature dimensionality reduction into the functional near-infrared blood oxygen data abnormality detection model for detection, and output the blood oxygen classification result; Step S3: Infrared thermal image abnormality detection In the host computer, scale the infrared thermal image to an image with a resolution of 300*300, input it into the infrared thermal image abnormality detection model, and output the infrared thermal image classification result; Step S4: Summarize the blood oxygen classification result and the infrared thermal image classification result, and present them in the host computer in four cases: both the blood oxygen classification result and the infrared thermal image classification result are 0; both the blood oxygen classification result and the infrared thermal image classification result are 1; the blood oxygen classification result is 1 and the infrared thermal image classification result is 0; the blood oxygen classification result is 0 and the infrared thermal image classification result is 1; The infrared thermal image abnormality detection model includes a target detection network module and an infrared thermal image classification network module connected in series, and the binary classification result output after the target region feature map output by the target detection network module passes through the infrared thermal image classification network module is used as the infrared thermal image classification result; The target detection network module is based on the SSD target detection network. The outputs of the convolutional layers conv4_3, conv7, conv8_2, and conv9_2 in the SSD target detection network are respectively connected to the target detection layer through an attention mechanism model, and the convolutional layers conv10_2 and conv11_2 are directly connected to the target detection layer, and then after passing through the target detection layer and non-maximum suppression processing, the output is the target region feature map; The infrared thermal image classification network module uses the AlexNet network as the backbone network, includes 5 convolutional layers, 3 fully connected layers, and 1 softmax layer connected in sequence, replaces the convolutional layer Conv2 in the AlexNet network with the Inception module, and changes the number of neurons in the softmax layer to 2.

2. The method for identifying functional near-infrared spectroscopy and infrared thermal images of attention deficit hyperactivity disorder according to claim 1, wherein: The functional near-infrared blood oxygen data anomaly detection model includes a parallel random forest classifier and a support vector machine classifier. The probabilities output by the two classifiers are arithmetically averaged, and the binary classification result output after threshold judgment is used as the blood oxygen classification result.

3. The method for identifying functional near-infrared spectroscopy and infrared thermography of attention deficit hyperactivity disorder according to claim 2, characterized in that: The parameters of the random forest classifier are: the number of decision trees is 50, the Gini index gain value is used as the basis for the decision tree to select features, the maximum depth is 10, and the minimum number of samples for splitting is 2; The parameters of the support vector machine classifier are: the kernel function is the RBF function, the gamma value is the reciprocal of the number of features, and the penalty parameter c is 1.

0.

4. The method for identifying functional near-infrared spectroscopy and infrared thermography of attention deficit hyperactivity disorder according to claim 3, characterized in that: The near-infrared blood oxygen data includes four channels, and each channel contains 4 parameters, namely TOI blood oxygen saturation, THI hemoglobin concentration index, ΔCHb deoxyhemoglobin change value, and ΔCHbO2 oxyhemoglobin change value; The Pearson correlation coefficient is calculated by pairwise combining the preprocessed near-infrared blood oxygen data according to four channels according to formula (1), and then the same parameter is taken from the two combined channels to calculate the Pearson correlation coefficient, and a total of 24 Pearson correlation coefficients are obtained; Among them, cov is the covariance, σ is the standard deviation, E is the expected value, ρ is the Pearson correlation coefficient, and X and Y are the signal values of the same parameter on the two channels; The wavelet coherence coefficient is calculated by pairwise combining the preprocessed near-infrared blood oxygen data according to four channels according to formula (2), and then the same parameter is taken from the two combined channels to calculate 624 wavelet coherence coefficients in 26 frequency bands: Among them, C XY is the wavelet coherence coefficient of signals X and X, P X is the cross-power spectral density of X and Y, P X is the power spectral density of X, P Y is the power spectral density of Y.

5. The method for identifying functional near-infrared spectroscopy and infrared thermography of attention deficit hyperactivity disorder according to claim 4, characterized in that: The normalization is performed using a standard normalizer with a mean of 0 and a variance of 1; where x' is the normalized value and x is the original value, is the mean value, and s is the standard deviation; The principal component analysis PAC feature dimension reduction is to sort and accumulate the variances of the principal components from large to small, and retain all features until the cumulative value of the variances of the principal components exceeds 0.

9.

6. The method for identifying functional near-infrared spectroscopy and infrared thermography of attention deficit hyperactivity disorder according to claim 5, characterized in that: The training process of the functional near-infrared blood oxygen data anomaly detection model is as follows: Collect the prefrontal near-infrared blood oxygen data of the patient group and the normal control group in the resting state and task state after being evaluated by doctors in the top three hospitals. After performing the data preprocessing described in step S2.1 and the feature extraction described in step S2.2 on the near-infrared blood oxygen data, obtain the feature vectors, and then use the features after the normalization described in step S2.3 and the principal component analysis PAC feature dimension reduction as the blood oxygen data set E1 for training the functional near-infrared blood oxygen data anomaly detection model; Divide the blood oxygen data set E1 into a training set and a test set at a ratio of 4:

1. Input the training set into the functional near-infrared blood oxygen data anomaly detection model, and set the maximum number of iterations of the random forest classifier and the support vector machine classifier to 300 for training. After the training is completed, input the test set into the trained functional near-infrared blood oxygen data anomaly detection model for detection and verification to obtain the functional near-infrared blood oxygen data anomaly detection model for online use.

7. The method for identifying functional near-infrared spectroscopy and infrared thermography for attention deficit hyperactivity disorder according to claim 6, characterized in that: The training process of the infrared thermography anomaly detection model is as follows: Collect the infrared thermograms of the head and neck regions of the patient group and the normal control group. Each infrared thermogram is marked with the position and classification label of the prefrontal region by doctors in the top three hospitals and used as the infrared thermogram set E2; Then use the infrared thermogram set E2 as the training sample of the target detection network module, and randomly select 20% of the samples as the cross-validation set during network training. The loss function for training is the loss function of the SSD target detection network. Train for 1000 epochs, and save the model with the smallest loss function value on the cross-validation set as the trained target detection network module; The infrared thermograms in the infrared thermogram set E2 pass through the trained target detection network module to output the target region feature map. Use the target region feature map and the classification label of its original infrared thermogram as the training sample of the infrared thermogram classification network, and randomly select 20% of the samples as the cross-validation set during the training of the infrared thermogram classification network. The objective function is the cross-entropy loss function. Train for 1000 epochs, and save the model with the smallest loss function value on the cross-validation set as the infrared thermogram classification network for online use.

8. The method for identifying functional near-infrared spectroscopy and infrared thermography for attention deficit hyperactivity disorder according to claim 7, characterized in that: The pairwise combination method between the four channels is channel 1 and channel 2, channel 1 and channel 3, channel 1 and channel 4, channel 2 and channel 3, channel 2 and channel 4, channel 3 and channel 4; When calculating the wavelet coherence coefficient, the 0.02Hz - 0.15Hz range is evenly divided into the 26 frequency bands.

Images