An autism spectrum disorder recognition system based on deep unfolding neural network

Abstract

The present invention belongs to the field of cognitive impairment, and specifically relates to an autism spectrum disorder recognition system based on a deep unfolding neural network. The construction process of the system includes: using a proximal gradient descent method to solve the differentiable part of a brain function network optimization problem with spatial information, defining a proximal operator for a non-differentiable part to convert the optimization problem into an iterative formula; using a deep unfolding algorithm to replace the proximal operator in the iterative formula with a double-layer convolutional layer, and then reconstructing the iterative formula into an iterative network to obtain an iterative network module; integrating a convolutional neural network classifier into prototype learning to obtain a convolutional neural network classifier based on prototype learning; combining the iterative network module and the convolutional neural network classifier based on prototype learning to obtain an obstacle recognition system; training the system and testing the system performance to obtain the optimal obstacle recognition system; the present invention improves the fitting ability and generalization ability of the network system, and further improves the accuracy of identifying diseases.

Description

Technical Field

[0001] The present invention belongs to the field of cognitive impairment, and in particular relates to an autism spectrum disorder recognition system based on a deep unfolding neural network. Background Art

[0002] The brain is a complex dynamic system, and its developmental changes are closely related to human higher-order cognitive abilities. Abnormal cerebral cortical morphology can be observed in many neurodevelopmental disorders and cortical malformations. The sulcus pattern not only reflects the various properties of the cerebral cortex, but also reflects the spatial, geometric and topological relationships between multiple sulci. Studying this pattern helps to understand the optimal organization and arrangement of cortical functional areas and underlying white matter fiber connections. Changes in the global sulcus pattern reflect changes in early brain development and manifest as individual differences in cognitive function, personality traits or mental disorders. Therefore, studying the sulcus pattern has important academic significance and clinical application value for in-depth understanding of the normal developmental mechanism of the brain or the pathogenesis of neurodevelopmental disorders.

[0003] Most of the technologies currently used are to solve brain function network optimization problems with biological prior knowledge to reveal the correlation between different brain regions, and use brain blood oxygen level dependence to construct and classify brain function networks. The traditional method based on node correlation can effectively show the degree of correlation between nodes, but there is a lot of redundant information. The improved sparse representation-based optimization algorithm can effectively reduce useless node connections, but it does not combine domain knowledge. In addition, as the research progresses, the algorithm based on the traditional model has poor fitting ability, and to achieve better experimental results, it relies more on a lot of manual feature extraction.

[0004] Problems with existing technologies: The accuracy of autism spectrum disorder identification is low, it is difficult to accurately extract brain functional network features, and the brain functional network construction and feature extraction process is complex. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes an autism spectrum disorder recognition system based on a deep unfolding neural network. The construction process of the system includes:

[0006] Acquire the cerebral blood oxygen level-dependent signal characterized by resting-state functional magnetic resonance imaging;

[0007] A configurable pipeline method for relationship analysis is used to preprocess the cerebral blood oxygen level-dependent signals characterized by resting-state functional magnetic resonance imaging technology, and the three-dimensional image signals of cerebral blood oxygen level-dependent signals are converted into two-dimensional time series signals according to the AAL partition template, and each two-dimensional time series signal is normalized, and then all normalized two-dimensional time series signals are divided into training set and test set in a ratio of 9:1;

[0008] The proximal gradient descent method is used to solve the differentiable part of the brain function network optimization problem with spatial information, and a proximal operator is defined for the non-differentiable part to convert the optimization problem into an iterative formula.

[0009] The proximal operator in the iterative formula is replaced with a double-layer convolutional layer using a deep expansion algorithm, and the iterative formula is reconstructed into an iterative network to obtain an iterative network module. The number of iterations is the number of iterative network modules.

[0010] Integrate the convolutional neural network classifier into prototype learning to obtain a convolutional neural network classifier based on prototype learning;

[0011] The iterative network module and the convolutional neural network classifier based on prototype learning are combined to obtain an obstacle recognition system. The system is trained using a training set and the system performance is tested using a test set to obtain the optimal obstacle recognition system.

[0012] Preferably, the proximal gradient descent method is used to solve the differentiable part of the brain function network optimization problem with spatial information, which is expressed as:

[0013]

[0014] in, represents the connection strength matrix of different brain partitions after the k+1th iteration in the differentiable part of the optimization problem, W k represents the connection strength matrix of different brain partitions after the kth iteration of the differentiable part, α represents the iteration step, λ1 represents the control parameter of the spatial information prior, X represents the time series signal of cerebral blood oxygen level dependence, D represents a diagonal matrix for auxiliary calculation, and S represents the similarity of the connection topology between different brain partitions.

[0015] Preferably, a proximal operator is defined for the non-differentiable part to convert the optimization problem into an iterative formula, which is expressed as:

[0016]

[0017] Among them, prox(·) represents the proximal operator, W k+1 represents the connection strength matrix of different brain partitions after the k+1th iteration of the non-differentiable part of the optimization problem, represents the connection strength matrix of different brain partitions after the k+1th iteration in the differentiable part of the optimization problem, W k represents the connection strength matrix of different brain partitions after the kth iteration of the differentiable part, α represents the iteration step, λ1 represents the control parameter of the spatial information prior, X represents the time series signal of cerebral blood oxygen level dependence, D represents a diagonal matrix for auxiliary calculation, and S represents the similarity of the connection topology between different brain partitions.

[0018] Preferably, the iterative network module comprises:

[0019] The unknown proximal operator prox(·) in the iterative formula is replaced by a double-layer convolutional layer. The two convolutional layers are connected by the ReLU activation function. The convolutional layers all use 3×3 convolution kernels and set edge padding to keep the dimension size unchanged. The first convolutional layer sets 32 convolution kernel features. The step size α and the control parameter λ1 are adaptively changed through network learning. The entire network module autonomously learns the representation method of the proximal operator through network training to obtain a feature representation that is more consistent with the category label.

[0020] Preferably, the convolutional neural network classifier is integrated into prototype learning to obtain a convolutional neural network classifier based on prototype learning, which specifically includes:

[0021] In the classifier, 4 layers of convolution are used to extract features, and the pooling layer is used for downsampling. The number of filters used in the convolution layer is 64, 64, 32, and 1 respectively. After each pooling operation, the RELU function is used for activation and regularization. After the convolution pooling layer, the graph is expanded into a one-dimensional input to the fully connected layer. There are four fully connected layers in total, including two hidden layers, and their number of neurons is 10, 64, 32, and 2 respectively. There is a ReLU activation function after each layer. In order to avoid overfitting, there is a Dropout operation in the fully connected layer. After the fully connected layer, prototype learning is connected, multiple prototypes are set for each category, and the network parameters are trained using the prototypes. The loss function of the classifier uses a distance-based cross entropy loss function, and the shortest Euclidean distance between the output and each prototype is used as the classification basis. The closer the distance to a certain type of prototype, the higher the possibility of belonging to its category. At the same time, the entire network uses ADAM as the optimizer, the learning rate is set to 0.0001, the probability of random discarding the Dropout item is 0.5, and the weight decay parameter is set to 0.0001.

[0022] Furthermore, multiple prototypes are set for each category, and the network parameters are trained using the prototypes, which can be expressed as:

[0023]

[0024] Where preclass represents the category predicted by the system, f(x;θ) represents the output of the feature extraction network in the classifier, x represents the input of the feature extraction network in the classifier, θ represents the network parameters of the feature extraction network in the classifier, and m ij Represents each prototype, C represents the number of categories, and K represents the number of prototypes in each category.

[0025] Furthermore, the distance-based cross entropy loss function is expressed as:

[0026]

[0027] Among them, loss(x,θ,m) represents the distance-based cross entropy loss function, d() represents the similarity between the system output and the prototype, γ represents the hyperparameter of the probability assignment difficulty, x represents the input of the feature extraction network in the classifier, θ represents the network parameters of the feature extraction network in the classifier, m represents the prototype of the category, and m ij represents the jth prototype of the i-th category, C represents the number of categories, K represents the number of prototypes in each category, k represents the category number, l represents the prototype number, e represents the natural constant, and m kl represents the lth prototype of the kth class.

[0028] Preferably, the batch size is set to 32, the data in the training set is input into the established obstacle recognition system and undergoes a shuffle process, a batch of data samples is input for each data iteration, the data passes through 8 iterative network modules, and a brain function network feature map of size N×N is output, and then extracted into a 5×5 feature map through four convolutional layers, the 5×5 feature map is expanded into a 1×10 vector and passed through three fully connected layers to obtain the final result, and while obtaining the result, two prototypes are trained for each label category, the loss is calculated using a distance-based cross entropy loss function, and the full connection parameters, convolution kernel parameters, including the step size parameter α and the control parameter λ1 set to be adaptively learnable are back-propagated and updated, and finally, after 50 epochs of training, the system is obtained when the loss drops to the lowest point; the data in the test set is input into the system when the loss drops to the lowest point, and the optimal obstacle recognition system under the test indicators of AUC, ACC, SPE, and SEN is obtained.

[0029] Beneficial effects of the present invention:

[0030] The present invention utilizes a deep expansion algorithm to combine the iterative formula with a deep neural network, and reconstructs the iterative formula into an iterative network; the self-learning network eliminates the arduous task of feature extraction, and the network's powerful fitting ability further improves the accuracy of disease identification; in addition, the prototype learning framework added to the classifier further improves the generalization ability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the overall structure of the autism spectrum disorder identification system of the present invention;

[0032] Figure 2 It is a schematic diagram of the iterative network module structure of the present invention;

[0033] Figure 3 It is a schematic diagram of the structure of a convolutional neural network classifier based on prototype learning of the present invention; DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] A system for identifying autism spectrum disorders based on deep unrolled neural networks, such as Figure 1 As shown in the figure, the construction process of the system includes:

[0036] Acquire the cerebral blood oxygen level-dependent signal characterized by resting-state functional magnetic resonance imaging;

[0037] A configurable pipeline method for relationship analysis is used to preprocess the cerebral blood oxygen level-dependent signals characterized by resting-state functional magnetic resonance imaging technology, and the three-dimensional image signals of cerebral blood oxygen level-dependent signals are converted into two-dimensional time series signals according to the AAL partition template, and each two-dimensional time series signal is normalized, and then all normalized two-dimensional time series signals are divided into training set and test set in a ratio of 9:1;

[0038] The proximal gradient descent method is used to solve the differentiable part of the brain function network optimization problem with spatial information, and a proximal operator is defined for the non-differentiable part to convert the optimization problem into an iterative formula.

[0039] The proximal operator in the iterative formula is replaced with a double-layer convolutional layer using a deep expansion algorithm, and the iterative formula is reconstructed into an iterative network to obtain an iterative network module. The number of iterations is the number of iterative network modules.

[0040] Integrate the convolutional neural network classifier into prototype learning to obtain a convolutional neural network classifier based on prototype learning;

[0041] The iterative network module and the convolutional neural network classifier based on prototype learning are combined to obtain an obstacle recognition system. The system is trained using a training set and the system performance is tested using a test set to obtain the optimal obstacle recognition system.

[0042] The data used in the experiment was selected from the ABIDE dataset published in 2013, which was obtained by researchers from multiple international sites. The selected dataset preprocessing process used the configurable pipeline for analyzing connectomes (CPAC) preprocessing method, and the brain was divided into 116 partitions using the AAL partition template, each of which had a string of time series data. The preprocessed dataset was then normalized and divided into a training set and a test set in a ratio of 9:1.

[0043] The brain function network optimization problem with spatial information is expressed as:

[0044]

[0045] Among them, w i represents the connection strength vector between the i-th partition and all other partitions, w ij represents the connection strength between the i-th partition and the j-th partition, N represents the number of brain partitions, and x i represents the time series characteristics of the i-th partition, λ1 and λ2 are prior control parameters, and s ij represents the similarity of the connection topology between the i-th partition and the j-th partition, where d ij represents the spatial distance between the i-th partition and the j-th partition, d ij =||R i -R j ||2, R represents the three-dimensional coordinates of each ROI R i Represents the three-dimensional coordinates of the i-th partition.

[0046] The variables in the brain function network optimization formula can be expressed in matrix form as follows:

[0047]

[0048] Where X represents the time series data matrix of all partitions W represents the connection strength matrix F means calculating the F norm, tr means calculating the trace of the matrix, and D is a diagonal matrix used for auxiliary calculations, whose diagonal elements are S represents the similarity of connection topology between different brain regions, and λ1 and λ2 are a priori control parameters.

[0049] The proximal gradient descent method is used to solve the differentiable part of the brain function network optimization problem with spatial information, which is expressed as:

[0050]

[0051] in, represents the connection strength matrix of different brain partitions after the k+1th iteration in the differentiable part of the optimization problem, W k represents the connection strength matrix of different brain partitions after the kth iteration of the differentiable part, α represents the iteration step, λ1 represents the control parameter of spatial information prior, X represents the time series signal of brain blood oxygen level dependence, and D represents a diagonal matrix for auxiliary calculation, whose diagonal elements S represents the similarity of the connection topology between different brain regions, and its elements

[0052] Defining a proximal operator for the non-differentiable part converts the optimization problem into an iterative formula, expressed as:

[0053]

[0054] Among them, prox(·) represents the proximal operator, W k+1 represents the connection strength matrix of different brain partitions after the k+1th iteration of the non-differentiable part of the optimization problem, represents the connection strength matrix of different brain partitions after the k+1th iteration in the differentiable part of the optimization problem, W k represents the connection strength matrix of different brain partitions after the kth iteration of the differentiable part, α represents the iteration step, λ1 represents the control parameter of the spatial information prior, X represents the time series signal of cerebral blood oxygen level dependence, D represents a diagonal matrix for auxiliary calculation, and S represents the similarity of the connection topology between different brain partitions.

[0055] Iterative network modules, such as Figure 2 As shown, including:

[0056] The unknown proximal operator prox(·) in the iterative formula is replaced by a double-layer convolutional layer. The two convolutional layers are connected by the ReLU activation function. The convolutional layers all use 3×3 convolution kernels and set edge padding to keep the dimension size unchanged. The first convolutional layer sets 32 convolution kernel features. The step size α and the control parameter λ1 are adaptively changed through network learning. The entire network module autonomously learns the representation method of the proximal operator through network training to obtain a feature representation that is more consistent with the category label.

[0057] Integrate the convolutional neural network classifier into prototype learning to obtain a convolutional neural network classifier based on prototype learning, such as Figure 3 As shown, specifically including:

[0058] In the classifier, 4 layers of convolution are used to extract features, and the pooling layer is used for downsampling. The number of filters used in the convolution layer is 64, 64, 32, and 1 respectively. After each pooling operation, the RELU function is used for activation and regularization. After the convolution pooling layer, the graph is expanded into a one-dimensional input to the fully connected layer. There are four fully connected layers in total, including two hidden layers, and their number of neurons is 10, 64, 32, and 2 respectively. There is a ReLU activation function after each layer. In order to avoid overfitting, there is a Dropout operation in the fully connected layer. After the fully connected layer, prototype learning is connected, multiple prototypes are set for each category, and the network parameters are trained using the prototypes. The loss function of the classifier uses a distance-based cross entropy loss function, and the shortest Euclidean distance between the output and each prototype is used as the classification basis. The closer the distance to a certain type of prototype, the higher the possibility of belonging to its category. At the same time, the entire network uses ADAM as the optimizer, the learning rate is set to 0.0001, the probability of random discarding the Dropout item is 0.5, and the weight decay parameter is set to 0.0001.

[0059] Set multiple prototypes for each category and use the prototypes to train network parameters, expressed as:

[0060]

[0061] Where preclass represents the category predicted by the system, f(x;θ) represents the output of the feature extraction network in the classifier, x represents the input of the feature extraction network in the classifier, θ represents the network parameters of the feature extraction network in the classifier, and m ij Represents each prototype, C represents the number of categories, and K represents the number of prototypes in each category, where i∈{1,2,...,C}, C represents the number of categories, and j∈{1,2,...,K}.

[0062] The distance-based cross entropy loss function is expressed as:

[0063]

[0064] Among them, loss(x,θ,m) represents the distance-based cross entropy loss function, It represents the similarity between the system output and the prototype, γ represents the hyperparameter of the probability assignment difficulty, that is, it controls the influence of the similarity on the probability; x represents the input of the feature extraction network in the classifier, θ represents the network parameters of the feature extraction network in the classifier, m represents the prototype of the category, mij represents the jth prototype of the i-th category, C represents the number of categories, K represents the number of prototypes of each category, k represents the category number, l represents the prototype number, e represents the natural constant, and mkl represents the lth prototype of the k-th category.

[0065] The batch size is set to 32, and the data in the training set is input into the established obstacle recognition system and goes through the shuffle process. A batch of data samples is input for each iteration of the data. After the data passes through 8 iterative network modules, a brain function network feature map of size N×N (N represents the number of different brain regions) is output, and then extracted into a 5×5 feature map through four convolutional layers. The 5×5 feature map is expanded into a 1×10 vector and passed through three fully connected layers to obtain the final result. While obtaining the results, two prototypes are trained for each label category. The distance-based cross entropy loss function is used to calculate the loss and back-propagate to update the fully connected parameters, convolution kernel parameters, including the step size parameter α and control parameter λ1 set to be adaptively learned. Finally, after 50 epochs of training, the system with the loss dropped to the lowest point is obtained; the data in the test set is input into the system with the loss dropped to the lowest point, and the test indicators of AUC (area under the ROC curve), ACC (accuracy), SPE (specificity), and SEN (sensitivity) are obtained. At the same time, comparative experiments are designed for other existing algorithms and classifiers.

[0066] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A system for identifying autism spectrum disorder based on a deep unfolding neural network, the system comprising an autism spectrum disorder identification model based on a deep unfolding neural network, characterized in that: The model construction process includes: Acquire the cerebral blood oxygen level-dependent signal characterized by resting-state functional magnetic resonance imaging; A configurable pipeline method for relationship analysis is used to preprocess the cerebral blood oxygen level-dependent signals characterized by resting-state functional magnetic resonance imaging technology, and the three-dimensional image signals of cerebral blood oxygen level-dependent signals are converted into two-dimensional time series signals according to the AAL partition template, and each two-dimensional time series signal is normalized, and then all normalized two-dimensional time series signals are divided into training set and test set in a ratio of 9:1; The proximal gradient descent method is used to solve the differentiable part of the brain function network optimization problem with spatial information, and a proximal operator is defined for the non-differentiable part to convert the optimization problem into an iterative formula. The proximal gradient descent method is used to solve the differentiable part of the brain function network optimization problem with spatial information, which is expressed as: in, represents the connection strength matrix of different brain partitions after the k+1th iteration in the differentiable part of the optimization problem, W k represents the connection strength matrix of different brain partitions after the kth iteration of the differentiable part, α represents the iteration step, λ1 represents the control parameter of the spatial information prior, X represents the time series signal of brain blood oxygen level dependence, D represents a diagonal matrix for auxiliary calculation, and S represents the similarity of the connection topology between different brain partitions; Defining a proximal operator for the non-differentiable part converts the optimization problem into an iterative formula, expressed as: Among them, prox(·) represents the proximal operator, W k+1 Represents the connection strength matrix of different brain partitions after the k+1th iteration of the non-differentiable part of the optimization problem; The proximal operator in the iterative formula is replaced with a double-layer convolutional layer using a deep expansion algorithm, and the iterative formula is reconstructed into an iterative network to obtain an iterative network module. The number of iterations is the number of iterative network modules. Integrate the convolutional neural network classifier into prototype learning to obtain a convolutional neural network classifier based on prototype learning; The iterative network module and the convolutional neural network classifier based on prototype learning are combined to obtain an obstacle recognition model. The model is trained using a training set and the model performance is tested using a test set to obtain the optimal obstacle recognition model.

2. The autism spectrum disorder recognition system based on deep unfolding neural network according to claim 1, characterized in that: Iterative network module, including: The unknown proximal operator prox(·) in the iterative formula is replaced by a double-layer convolutional layer. The two convolutional layers are connected by the ReLU activation function. The convolutional layers all use 3×3 convolution kernels and set edge padding to keep the dimension size unchanged. The first convolutional layer sets 32 convolution kernel features. The step size α and the control parameter λ1 are adaptively changed through network learning. The entire network module autonomously learns the representation method of the proximal operator through network training to obtain a feature representation that is more consistent with the category label.

3. The autism spectrum disorder recognition system based on deep unfolding neural network according to claim 1, characterized in that: The convolutional neural network classifier is integrated into prototype learning to obtain a convolutional neural network classifier based on prototype learning, which specifically includes: In the classifier, 4 layers of convolution are used to extract features, and the pooling layer is used for downsampling. The number of filters used in the convolution layer is 64, 64, 32, and 1 respectively. After each pooling operation, the RELU function is used for activation and regularization. After the convolution pooling layer, the graph is expanded into a one-dimensional input to the fully connected layer. There are four fully connected layers in total, including two hidden layers, and their number of neurons is 10, 64, 32, and 2 respectively. There is a ReLU activation function after each layer. In order to avoid overfitting, there is a Dropout operation in the fully connected layer. After the fully connected layer, prototype learning is connected. Multiple prototypes are set for each category. The network parameters are trained using the prototype. The loss function of the classifier uses a distance-based cross entropy loss function. The shortest Euclidean distance between the output and each prototype is used as the classification basis. The closer the distance to a certain type of prototype, the higher the possibility of belonging to its category. At the same time, the entire network uses ADAM as the optimizer, the learning rate is set to 0.0001, the probability of random discarding the Dropout item is 0.5, and the weight decay parameter is set to 0.0001.

4. The autism spectrum disorder recognition system based on deep unfolding neural network according to claim 3, characterized in that: Set multiple prototypes for each category and use the prototypes to train network parameters, expressed as: Where preclass represents the category predicted by the model, f(x;θ) represents the output of the feature extraction network in the classifier, x represents the input of the feature extraction network in the classifier, θ represents the network parameters of the feature extraction network in the classifier, and m ij Represents each prototype, C represents the number of categories, and K represents the number of prototypes in each category.

5. The autism spectrum disorder recognition system based on deep unfolding neural network according to claim 3, characterized in that: The distance-based cross entropy loss function is expressed as: Among them, loss(x,θ,m) represents the distance-based cross entropy loss function, d() represents the similarity between the model output and the prototype, γ represents the hyperparameter of the probability assignment difficulty, x represents the input of the feature extraction network in the classifier, θ represents the network parameters of the feature extraction network in the classifier, m represents the prototype of the category, and m ij represents the jth prototype of the i-th category, C represents the number of categories, K represents the number of prototypes in each category, k represents the category number, l represents the prototype number, e represents the natural constant, and m kl represents the lth prototype of the kth class.

6. The autism spectrum disorder recognition system based on deep unfolding neural network according to claim 1, characterized in that: Use the training set to train the model and use the test set to test the model performance, including: The batch size is set to 32, and the data in the training set are input into the established obstacle recognition model and go through the shuffle process. A batch of data samples are input for each data iteration. After the data passes through 8 iterative network modules, a brain function network feature map of size N×N is output, and then extracted into a 5×5 feature map through four convolutional layers. The 5×5 feature map is expanded into a 1×10 vector and passed through three fully connected layers to obtain the final result. While obtaining the results, two prototypes are trained for each label category. The distance-based cross entropy loss function is used to calculate the loss and back-propagate to update the full connection parameters, convolution kernel parameters, including the step size parameter α and control parameter λ1 set to be adaptively learned. Finally, after 50 epochs of training, the model when the loss drops to the lowest point is obtained; the data in the test set is input into the model when the loss drops to the lowest point, and the optimal obstacle recognition model under the test indicators of AUC, ACC, SPE, and SEN is obtained.

Images