Brain image classification method and storage medium
By constructing the graph characteristic matrix and using the feature extraction, clustering and classification subnet of the image classification network, the inaccuracy problem in traditional brain image classification methods is solved, and higher classification accuracy is achieved.
Patent Information
- Application Number
- CN202010662852.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-10
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2040-12-24
AI Technical Summary
Traditional brain image classification methods have the problem of inaccurate classification, and it is difficult to effectively utilize graph structure information, resulting in uncertainty in classification results.
A brain image classification method is adopted to obtain brain functional images, build a graph characteristic matrix, and input it into the feature extraction subnet, clustering subnet and classification subnet in the image classification network to extract feature information and clustering brain respectively. and optimize brain circuits, and finally obtain accurate classification results.
The accuracy of brain image classification is improved, and the brain regions are accurately clustered and optimized through clustering subnets, which reduces the characteristic time dimension of brain circuits and obtains more accurate brain function image classification results.
Smart Images
Figure CN111949812B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a brain image classification method and storage medium. Background Art
[0002] With the development of neural network technology, a special neural network with graph structure, namely Graph Convolutional Network (GCN), has also been proposed. GCN uses a deep learning network model to learn information in the graph structure and has good performance in classification.
[0003] In traditional technology, when classifying brain images, all brain areas in the brain image are separated to obtain multiple regions of interest. The functional signal correlation is obtained by calculating the correlation between the two regions of interest. The obtained functional signal correlation is used as the connection strength between the two regions of interest to form a whole-brain network diagram. GCN is used to analyze the whole-brain network diagram to obtain the classification result of the brain image.
[0004] However, traditional brain image classification methods have the problem of inaccurate brain image classification. Summary of the invention
[0005] Based on this, it is necessary to provide a brain image classification method and storage medium that can improve the accuracy of brain image classification in order to solve the above technical problems.
[0006] A brain image classification method, the method comprising:
[0007] Acquiring brain functional images to be classified;
[0008] According to the brain function image, obtaining a graph characteristic matrix corresponding to the brain function image;
[0009] Inputting the image characteristic matrix into a feature extraction subnetwork of a preset image classification network to obtain feature information between brain regions in the brain function image;
[0010] Inputting the characteristic information between the brain regions into the clustering subnetwork of the image classification network, clustering the brain regions, and clustering the brain regions into different brain circuits;
[0011] Each of the brain circuits is input into the classification subnetwork of the image classification network to obtain the classification result of the brain function image.
[0012] In one embodiment, the clustering subnetwork includes a clustering layer and an optimization layer; the step of inputting the feature information between the brain regions into the clustering subnetwork of the image classification network, clustering the brain regions, and clustering the brain regions into different brain circuits includes:
[0013] Inputting the characteristic information between the brain regions into the clustering layer, clustering the brain regions according to a preset clustering allocation matrix, and clustering the brain regions into different initial brain circuits;
[0014] Each of the initial brain circuits is input into the optimization layer, and each of the initial brain circuits is optimized to obtain each of the brain circuits.
[0015] In one embodiment, inputting each of the initial brain circuits into the optimization layer, optimizing each of the initial brain circuits, and obtaining each of the brain circuits includes:
[0016] Each of the initial brain circuits is input into the optimization layer, the features of each of the initial brain circuits are learned and updated, and each of the initial brain circuits is optimized according to the updated features and a preset optimization method to obtain each of the brain circuits.
[0017] In one embodiment, the classification subnetwork includes a graph convolution layer; and inputting each of the brain circuits into the classification subnetwork of the image classification network to obtain the classification result of the brain function image includes:
[0018] Inputting each of the brain circuits into the graph convolution layer, reducing the time dimension of the features of each of the brain circuits, and converting the features of each of the brain circuits after the time dimension is reduced into corresponding feature values;
[0019] The classification result of the brain function image is obtained according to the characteristic values corresponding to the characteristics of each brain circuit.
[0020] In one embodiment, the classification subnetwork further includes a classification layer; and obtaining the classification result of the brain function image according to the feature values corresponding to the features of each of the brain circuits includes:
[0021] The feature values corresponding to the features of each brain circuit are input into the classification layer, a corresponding weight value is assigned to each feature value, and the classification result of the brain function image is obtained according to each feature value and the weight value corresponding to each feature value.
[0022] In one embodiment, the method further comprises:
[0023] According to the weight values corresponding to the characteristic values, the target brain circuit related to the classification result is determined.
[0024] In one embodiment, the method further comprises:
[0025] According to the cluster allocation matrix corresponding to the target brain circuit, the brain regions included in the target brain circuit are determined.
[0026] In one of the embodiments, the feature extraction subnetwork includes a graph convolutional long short-term memory network.
[0027] In one embodiment, obtaining a graph characteristic matrix corresponding to the brain function image according to the brain function image includes:
[0028] Acquiring characteristic information of each brain region from the brain function image as a node feature;
[0029] According to the node characteristics of each brain region, obtaining connection information between each brain region, and using the connection information as the connection between the nodes;
[0030] The node features and the connections between the nodes are constructed into the graph property matrix.
[0031] A brain image classification device, comprising:
[0032] A first acquisition module is used to acquire brain function images to be classified;
[0033] A second acquisition module is used to obtain a graph characteristic matrix corresponding to the brain function image according to the brain function image;
[0034] A feature extraction module, used for inputting the image characteristic matrix into a feature extraction subnetwork of a preset image classification network to obtain feature information between various brain regions in the brain function image;
[0035] A clustering module, used for inputting the characteristic information between the brain regions into the clustering subnetwork of the image classification network, clustering the brain regions, and clustering the brain regions into different brain circuits;
[0036] The classification module is used to input each of the brain circuits into the classification subnetwork of the image classification network to obtain the classification result of the brain function image.
[0037] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0038] Acquiring brain functional images to be classified;
[0039] According to the brain function image, obtaining a graph characteristic matrix corresponding to the brain function image;
[0040] Inputting the image characteristic matrix into a feature extraction subnetwork of a preset image classification network to obtain feature information between brain regions in the brain function image;
[0041] Inputting the characteristic information between the brain regions into the clustering subnetwork of the image classification network, clustering the brain regions, and clustering the brain regions into different brain circuits;
[0042] Each of the brain circuits is input into the classification subnetwork of the image classification network to obtain the classification result of the brain function image.
[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:
[0044] Acquiring brain functional images to be classified;
[0045] According to the brain function image, obtaining a graph characteristic matrix corresponding to the brain function image;
[0046] Inputting the image characteristic matrix into a feature extraction subnetwork of a preset image classification network to obtain feature information between brain regions in the brain function image;
[0047] Inputting the characteristic information between the brain regions into the clustering subnetwork of the image classification network, clustering the brain regions, and clustering the brain regions into different brain circuits;
[0048] Each of the brain circuits is input into the classification subnetwork of the image classification network to obtain the classification result of the brain function image.
[0049] The above-mentioned brain image classification method, device, computer equipment and storage medium input the image characteristic matrix corresponding to the brain function image to be classified into the feature extraction subnetwork of the preset image classification network, and can accurately obtain the feature information between the brain regions in the brain function image to be classified through the feature extraction subnetwork. In this way, the feature information between the brain regions is input into the clustering subnetwork of the image classification network, and the brain regions can be accurately clustered through the clustering subnetwork, and the brain regions can be accurately clustered into different brain circuits, thereby improving the accuracy of the obtained brain circuits, and then each brain circuit is input into the classification subnetwork of the image classification network, and the classification results of the brain function image can be accurately obtained through the classification subnetwork, thereby improving the accuracy of the classification results of the obtained brain function image. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 A schematic diagram of the internal structure of a computer device provided in an embodiment;
[0051] Figure 2 is a schematic diagram of a flow chart of a brain image classification method in one embodiment;
[0052] Figure 2a A schematic diagram of the structure of a hierarchical graph convolutional neural network provided in one embodiment;
[0053] Figure 2b A schematic diagram of a cross-section of a brain circuit provided in one embodiment;
[0054] Figure 3 is a schematic flow chart of a brain image classification method in another embodiment;
[0055] Figure 4 is a schematic flow chart of a brain image classification method in another embodiment;
[0056] Figure 5 is a schematic flow chart of a brain image classification method in another embodiment;
[0057] Figure 6 A schematic diagram of the structure of a brain image classification device provided in one embodiment. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0059] The brain image classification method provided in the embodiment of the present application can be applied to Figure 1 The computer device shown. The computer device includes a processor and a memory connected by a system bus, and a computer program is stored in the memory. When the processor executes the computer program, the steps of the following method embodiment can be executed. Optionally, the computer device may also include a network interface, a display screen and an input device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory, and the non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. Optionally, the computer device can be a server, a personal computer, a personal digital assistant, or other terminal devices, such as a tablet computer, a mobile phone, etc., or a cloud or remote server. The embodiment of the present application does not limit the specific form of the computer device.
[0060] In one embodiment, Figure 2 As shown, a brain image classification method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:
[0061] S201, obtaining a brain function image to be classified.
[0062] Wherein, brain function image refers to the functional magnetic resonance image of the brain of the subject obtained by functional magnetic resonance imaging (FMRI). Specifically, the computer device obtains the brain function image to be classified. Optionally, the computer device can obtain the brain function image from the PACS server, or obtain the brain function image from the FMRI imaging device. Optionally, when using the FMRI imaging device to obtain the functional magnetic resonance image of the brain of the subject, there can be 137 scanning time points, and the interval between each time point is 2ms. Optionally, the computer device can also pre-process the acquired brain function image to reduce the error caused by data acquisition and physiological characteristics, wherein the pre-processing of the acquired brain function image can include at least one of time registration processing, head motion correction processing, standardization processing and real space filtering processing.
[0063] S202, obtaining a graph characteristic matrix corresponding to the brain function image according to the brain function image.
[0064] Specifically, the computer device can obtain a graph characteristic matrix corresponding to the acquired brain function image based on the acquired brain function image. Optionally, the computer device can perform the above-mentioned preprocessing on the acquired brain function image, and then partition the preprocessed brain function image according to a preset brain partition template, extract functional connection information of each brain region from the preprocessed brain function image, and obtain the graph characteristic matrix corresponding to the brain function image based on the extracted functional connection information of each brain region and each brain region.
[0065] S203, inputting the image feature matrix into a feature extraction subnetwork of a preset image classification network to obtain feature information between brain regions in the brain function image.
[0066] Specifically, the computer device inputs the obtained graph feature matrix into a feature extraction subnetwork of a preset image classification network to obtain feature information between brain regions in the above brain function image. Optionally, the preset image classification network can be a hierarchical graph convolutional neural network, such as Figure 2aAs shown, the hierarchical graph convolutional neural network can be composed of three parts: the information processing layer of the low-order graph is mainly used to extract the information between brain regions in the brain function image according to the graph feature matrix; the transformation from the low-order graph to the high-order graph is mainly through the graph structure pooling method, according to the connection between brain regions, the brain regions are divided into different circuits, and the connection between these brain circuits is calculated and learned; the information processing layer of the high-order graph is mainly used to extract high-order information in the brain circuit network, so as to complete the classification. Optionally, the feature extraction subnetwork includes a graph convolutional long short-term memory network, wherein the graph convolutional long short-term memory network is a network structure based on graph convolution, which can be used to process dynamic graph information. The graph convolutional long short-term memory network continuously extracts information from the vicinity of the node according to the closeness of the relationship between nodes to update the information of the current node, and uses the long short-term memory structure to process the time series signal. Optionally, the computer device can input the graph feature matrix into a feature extraction subnetwork, which continuously extracts information from the vicinity of each node of the graph feature matrix, updates the information of the current node, and uses a long short-term memory structure to process the time series signal to obtain feature information between the brain regions in the above-mentioned brain function image.
[0067] S204, inputting feature information between the brain regions into the clustering subnetwork of the image classification network, clustering the brain regions, and clustering the brain regions into different brain circuits.
[0068] Among them, brain circuits refer to several brain regions that are closely connected in function. For example, the language center that controls language is a brain circuit. For example, Figure 2b As shown, it is a schematic diagram of the transverse, sagittal and coronal positions of the two brain circuits. Specifically, the computer device inputs the characteristic information between the brain regions in the brain function image obtained above into the clustering subnetwork of the above image classification network, clusters the brain regions in the above brain function image, and clusters the brain regions into different brain circuits. Optionally, the computer device inputs the characteristic information between the brain regions into the clustering subnetwork of the image classification network, and the clustering subnetwork can cluster the brain regions in the above brain function image by using the allocation matrix, and cluster the brain regions in the brain function image into different brain circuits.
[0069] S205, input each brain circuit into the classification sub-network of the image classification network to obtain the classification result of the brain function image.
[0070] Specifically, the computer device inputs each brain circuit obtained above into the classification subnetwork of the image classification network to obtain the classification result of the brain function image. Optionally, the computer device inputs each brain circuit into the classification subnetwork of the image classification network, through which the characteristics of each brain circuit can be compressed into a specific value, and then, according to the corresponding value of each brain region, the classification result of the brain function image is obtained.
[0071] In the above-mentioned brain image classification method, the computer device inputs the image characteristic matrix corresponding to the brain function image to be classified into the feature extraction subnetwork of the preset image classification network, and can accurately obtain the feature information between the brain regions in the brain function image to be classified through the feature extraction subnetwork. In this way, the feature information between the brain regions is input into the clustering subnetwork of the image classification network, and the brain regions can be accurately clustered through the clustering subnetwork, and the brain regions can be accurately clustered into different brain circuits, thereby improving the accuracy of the obtained brain circuits, and then each brain circuit is input into the classification subnetwork of the image classification network, and the classification results of the brain function image can be accurately obtained through the classification subnetwork, thereby improving the accuracy of the classification results of the obtained brain function image.
[0072] In the above scenario where the characteristic information of each brain region is input into the clustering subnetwork of the image classification network, each brain region is clustered, and each brain region is clustered into different brain circuits, the clustering subnetwork includes a clustering layer and an optimization layer. Figure 3 As shown, the above S204 includes:
[0073] S301, inputting feature information between brain regions into a clustering layer, clustering each brain region according to a preset clustering allocation matrix, and clustering each brain region into different initial brain circuits.
[0074] Specifically, the computer device inputs the characteristic information between each brain region in the brain function image to be classified into the clustering layer of the clustering subnetwork, clusters each brain region according to a preset clustering allocation matrix, and clusters each brain region into different initial brain circuits. Exemplarily, assuming that the preset clustering allocation matrix is P, the clustering of each brain region can be performed by merging M brain regions into K clusters using the allocation matrix P. For example, the formula X (l+1) =P T X (l) , A l+1 =P T A (l) p clusters each brain region into different initial brain circuits, where the matrix X represents the feature matrix of each brain region, each element in the matrix A is the connection between the corresponding brain regions, and the distribution matrix P∈R M×KEach element in represents the contribution of each brain region to the brain circuit. The allocation matrix P can be parameterized as a learnable parameter and its value can be updated based on the brain regions of all subjects at all time points.
[0075] S302, input each initial brain circuit into the optimization layer, optimize each initial brain circuit, and obtain each brain circuit.
[0076] Specifically, the computer device inputs each initial brain circuit obtained by clustering into the optimization layer of the clustering subnetwork, optimizes each initial brain circuit, and obtains each brain circuit of the brain function image to be classified. Optionally, the computer device can input each initial brain circuit into the optimization layer of the clustering subnetwork, learn and update the characteristics of each initial brain circuit, and optimize each initial brain circuit according to the updated characteristics and the preset optimization method, to obtain each brain circuit of the brain function image to be classified. It should be noted that, as shown in the above formula X (l+1) =P T X (l) , A l+1 =P T A (l) As shown in Figure 1, when each brain region is clustered into brain circuits, the relationship between the circuits is initialized by averaging the connections between brain regions within the circuit. This initialization is the initialization of the high-order circuit-based graph structure. In order to further learn the graph structure of the high-order graph, the characteristics of each initial brain circuit can be learned and updated through the optimization layer of the clustering subnetwork. Each initial brain circuit is optimized according to the characteristics of each initial brain circuit to obtain the brain circuits of the above-mentioned brain function images to be classified. Optionally, the optimization layer of the clustering subnetwork can be calculated by formula a i,j =MLP(|x i -x j |) optimizes each initial brain circuit to obtain each brain circuit of the brain function image to be classified, where a ij is the (i, j)th element of the above matrix A. The above formula means that each initial brain circuit is optimized by nonlinear projection (MLP) of the absolute difference between each brain circuit feature to obtain each brain circuit of the brain function image to be classified.
[0077] In this embodiment, the computer device first inputs the feature information between the brain regions of the brain function image to be classified into the clustering layer of the clustering subnetwork, and can cluster the brain regions of the brain function image to be classified according to a preset clustering allocation matrix, cluster the brain regions into different initial brain circuits, and then input the obtained initial brain circuits into the optimization layer of the clustering subnetwork, so as to optimize the initial brain circuits and accurately obtain the brain circuits of the brain function image to be classified, thereby improving the accuracy of the brain circuits of the obtained brain function image to be classified.
[0078] In the above scenario where each brain circuit of the brain function image to be classified is input into the classification subnetwork of the image classification network to obtain the classification result of the brain function image to be classified, the classification subnetwork includes a graph convolution layer. Figure 4 As shown, the above S205 includes:
[0079] S401, input each brain circuit into a graph convolution layer, reduce the time dimension of the features of each brain circuit, and convert the features of each brain circuit after the time dimension is reduced into corresponding feature values.
[0080] Specifically, the computer device inputs each brain circuit of the brain function image to be classified into the pooling layer of the classification subnetwork, reduces the time dimension of the features of each brain circuit, and converts the features of each brain circuit after reducing the time dimension into corresponding feature values. Optionally, the graph convolution layer of the classification subnetwork can be a global average pooling layer. It should be noted that brain integration and brain isolation are determined as the main brain functional connection patterns. Brain integration refers to the effect of different brain regions on the same function, such as the language circuit in the brain, etc. Brain isolation means that each brain region also has its own function. In the related technology, it is mainly focused on one or two independent brain regions for research. The related technology can only explain the function of brain isolation. In this embodiment, the proposed method is based on the high-order graph generated by the brain region. Therefore, the most important brain circuits that contribute to the classifier can be highlighted, but because the time dimension of the high-order graph is high, the time dimension of the features of each brain circuit can be reduced by the graph convolution layer of the classification subnetwork, and the features of each brain circuit after reducing the time dimension are converted into corresponding feature values. For example, the feature value of brain circuit 1 can be 2.35, and the feature value of brain circuit 2 can be -3.13.
[0081] S402, obtaining a classification result of the brain function image according to the feature values corresponding to the features of each brain circuit.
[0082] Specifically, the computer device can obtain the classification result of the above-mentioned brain function image to be classified according to the characteristic values corresponding to the characteristics of each brain circuit. Optionally, the computer device can input the characteristic values corresponding to the characteristics of each brain circuit into a preset classifier to obtain the classification result of the above-mentioned brain function image to be classified.
[0083] In this embodiment, the computer device inputs each brain circuit into the graph convolution layer of the classification subnetwork, which can reduce the time dimension of the characteristics of each brain circuit. The characteristics of each brain circuit with reduced time dimension can accurately represent the characteristics of each brain circuit, and then the computer device can accurately convert the characteristics of each brain circuit after reducing the time dimension into corresponding feature values, so that the brain function images to be classified can be accurately classified according to the feature values corresponding to the characteristics of each brain circuit, and the classification results of the brain function images to be classified can be accurately obtained.
[0084] In the above scenario where the classification result of the brain function image is obtained according to the feature values corresponding to the features of each brain circuit of the brain function image to be classified, the classification subnetwork also includes a classification layer. In one embodiment, based on the above embodiment, the above S402 includes: inputting the feature values corresponding to the features of each brain circuit into the classification layer, assigning corresponding weight values to each feature value, and obtaining the classification result of the brain function image according to each feature value and the weight value corresponding to each feature value.
[0085] Specifically, the above-mentioned classification subnetwork also includes a classification layer, and the computer device can input the feature values corresponding to the features of each brain circuit into the classification layer, assign corresponding weight values to the feature values corresponding to the features of each brain circuit, and obtain the classification results of the above-mentioned brain function images to be classified according to the feature values and the weight values corresponding to each feature value. Optionally, the computer device can assign corresponding weight values to the feature values corresponding to the features of each brain circuit according to the values of each cluster when clustering each brain region. Further, the computer device can also determine the target brain circuit related to the above-mentioned classification result according to the weight value corresponding to the feature values corresponding to the features of each brain circuit, and then determine the brain region included in the target brain circuit according to the cluster allocation matrix corresponding to the target brain circuit, that is, the computer device determines each node in the target brain circuit according to the cluster allocation matrix corresponding to the target brain circuit, and determines each node in the target brain circuit as the brain region included in the target brain circuit. Optionally, the computer device can determine the brain circuit with the largest weight value corresponding to the feature values corresponding to the features of each brain circuit as the target brain circuit related to the above-mentioned classification result.
[0086] In this embodiment, the computer device inputs the feature values corresponding to the features of each brain circuit into the classification layer of the classification subnetwork, and can accurately assign corresponding weight values to the feature values corresponding to the features of each brain circuit, and then can accurately obtain the classification results of the brain function images to be classified according to the feature values of each brain circuit and the weight values corresponding to each feature value, thereby improving the accuracy of the classification results of the brain function images to be classified.
[0087] In the above scenario of obtaining a graph characteristic matrix corresponding to the brain function image to be classified based on the brain function image to be classified, the computer device can obtain characteristic information of each brain region from the brain function image to be classified, and obtain the graph characteristic matrix corresponding to the brain function image to be classified based on the characteristic information of each brain region. Figure 5 As shown, the above S202 includes:
[0088] S501, acquiring feature information of each brain region from the brain function image as a node feature.
[0089] Specifically, the computer device obtains the characteristic information of each brain region from the brain function image to be classified as the node feature. Optionally, the computer device may first partition the brain function image to be classified according to a preset brain partition template, and then extract the characteristic information of each brain region from the brain function image to be classified as the node feature. Optionally, the computer device may use the volume of each brain region as the node feature, or any one of the surface area of each brain region, the curvature of each brain region, and the grayscale of each brain region as the node feature. Optionally, the preset brain partition template may be an anatomical automatic labeling (AAL) template, or may be other brain partition templates, for example, an SRI24 template. Optionally, the computer device may divide the above-mentioned brain function image to be classified into 116 brain regions according to the preset brain partition template.
[0090] S502, acquiring connection information between each brain region according to the node characteristics of each brain region, and using the connection information as the connection between nodes.
[0091] Specifically, the computer device obtains connection information between each brain region according to the node characteristics of each brain region of the brain function image to be classified, and uses the connection information between each brain region as the connection between nodes. Optionally, the computer device can obtain functional connection information of each brain region according to the node characteristics of each brain region of the brain function image to be classified, and use the functional connection information of each brain region as the connection between the above nodes.
[0092] S503, constructing a graph feature matrix by combining node features and connections between nodes.
[0093] Specifically, the computer device constructs the node features and connections between nodes obtained above into a graph characteristic matrix. That is, the computer device can use the volume of each brain region as the node feature and the functional connection information between each brain region as the connection between nodes to construct the graph characteristic matrix. Exemplarily, taking the brain structure image and the brain function image as four brain regions as an example, the feature information of each of the four brain regions is extracted from the brain structure image as the node feature, and the functional connection information of each of the four brain regions is extracted from the brain function image as the connection between nodes, and the node features and the connection between nodes are constructed into a graph characteristic matrix X,
[0094]
[0095] Among them, v 11 ,v 22 ,v 33 ,v 44 They are the characteristic information of each of the four brain regions, v 12 ,v 21 is the functional connection information between brain area 1 and brain area 2, v 13 ,v 31 is the functional connection information between the first and third brain regions, v 14 ,v 41 is the functional connection information between one brain area and four brain areas, v 23 ,v 32 is the functional connection information between the second and third brain areas, v 24 ,v 42 is the functional connection information between the second and fourth brain areas, v 34 ,v 43 It is the functional connection information between the third and fourth brain areas.
[0096] In this embodiment, the computer device can obtain the characteristic information of each brain region from the brain function image to be classified, and then the characteristic information of each brain region can be used as the node feature. According to the node feature of each brain region, the connection information between each brain region can be obtained, and the connection information between each brain region can be used as the connection between nodes. Then the node features and the connection between nodes can be constructed into a graph feature matrix. In this way, the graph feature matrix contains the characteristic information of each brain region in the brain function image to be classified and the functional connection information between each brain region in the brain function image to be classified, and can more comprehensively and accurately reflect the information of the brain function image to be classified, and then the brain function image to be classified can be accurately classified according to the constructed graph feature matrix, thereby improving the accuracy of the classification result of the brain function image to be classified.
[0097] It should be understood that although Figure 2-5The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-5 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.
[0098] In one embodiment, Figure 6 As shown, a brain image classification device is provided, comprising: a first acquisition module, a second acquisition module, a third acquisition module, a clustering module and a classification module, wherein:
[0099] The first acquisition module is used to acquire brain function images to be classified.
[0100] The second acquisition module is used to obtain a graph characteristic matrix corresponding to the brain function image according to the brain function image.
[0101] The third acquisition module is used to input the image feature matrix into the feature extraction subnetwork of the preset image classification network to obtain the feature information between the brain regions in the brain function image.
[0102] The clustering module is used to input the feature information between each brain region into the clustering subnetwork of the image classification network, cluster each brain region, and cluster each brain region into different brain circuits.
[0103] The classification module is used to input each brain circuit into the classification subnetwork of the image classification network to obtain the classification results of brain function images.
[0104] Optionally, the feature extraction subnetwork includes a graph convolutional long short-term memory network.
[0105] The brain image classification device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0106] Based on the above embodiment, optionally, the above clustering subnetwork includes a clustering layer and an optimization layer; the above clustering module includes: a clustering unit and an optimization unit, wherein:
[0107] The clustering unit is used to input the characteristic information between each brain region into the clustering layer, cluster each brain region according to a preset clustering allocation matrix, and cluster each brain region into different initial brain circuits.
[0108] The optimization unit is used to input each initial brain circuit into the optimization layer, optimize each initial brain circuit, and obtain each brain circuit.
[0109] The brain image classification device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0110] On the basis of the above embodiment, optionally, the optimization unit is specifically used to input each initial brain circuit into the optimization layer, learn and update the characteristics of each initial brain circuit, and optimize each initial brain circuit according to the updated characteristics and the preset optimization method to obtain each brain circuit.
[0111] The brain image classification device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0112] Based on the above embodiment, optionally, the classification subnetwork includes a graph convolution layer; the above classification module includes: a conversion unit and a classification unit, wherein:
[0113] The conversion unit is used to input each brain circuit into the graph convolution layer, reduce the time dimension of the characteristics of each brain circuit, and convert the characteristics of each brain circuit after the time dimension is reduced into corresponding feature values.
[0114] The classification unit is used to obtain the classification result of the brain function image according to the characteristic values corresponding to the characteristics of each brain circuit.
[0115] The brain image classification device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0116] On the basis of the above embodiment, optionally, the classification subnetwork also includes a classification layer; a classification unit, which is specifically used to input the feature values corresponding to the features of each brain circuit into the classification layer, assign a corresponding weight value to each feature value, and obtain the classification result of the brain function image according to each feature value and the weight value corresponding to each feature value.
[0117] The brain image classification device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0118] Based on the above embodiment, optionally, the above device further includes: a first determining module, wherein:
[0119] The first determination module is used to determine the target brain circuit related to the classification result according to the weight value corresponding to each feature value.
[0120] The brain image classification device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0121] Based on the above embodiment, optionally, the above device further includes: a second determining module, wherein:
[0122] The second determination module is used to determine the brain regions included in the target brain circuit according to the cluster allocation matrix corresponding to the target brain circuit.
[0123] The brain image classification device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0124] On the basis of the above embodiment, optionally, the above second acquisition module includes: a first acquisition unit, a second acquisition unit and a construction unit, wherein:
[0125] The first acquisition unit is used to acquire feature information of each brain region from the brain function image as a node feature.
[0126] The second acquisition unit is used to acquire connection information between each brain region according to the node characteristics of each brain region, and use the connection information as the connection between nodes.
[0127] The construction unit is used to construct the node features and the connections between nodes into a graph feature matrix.
[0128] The brain image classification device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0129] For the specific definition of the brain image classification device, please refer to the definition of the brain image classification method above, which will not be repeated here. Each module in the above-mentioned brain image classification device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0130] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0131] Acquiring brain functional images to be classified;
[0132] According to the brain function image, a graph characteristic matrix corresponding to the brain function image is obtained;
[0133] Input the image feature matrix into the feature extraction subnetwork of the preset image classification network to obtain the feature information between the brain regions in the brain function image;
[0134] Input the characteristic information between each brain region into the clustering subnetwork of the image classification network, cluster each brain region, and cluster each brain region into different brain circuits;
[0135] Each brain circuit is input into the classification subnetwork of the image classification network to obtain the classification results of brain function images.
[0136] The implementation principle and technical effect of the computer device provided in the above embodiment are similar to those of the above method embodiment, and will not be repeated here.
[0137] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0138] Acquiring brain functional images to be classified;
[0139] According to the brain function image, a graph characteristic matrix corresponding to the brain function image is obtained;
[0140] Input the image feature matrix into the feature extraction subnetwork of the preset image classification network to obtain the feature information between the brain regions in the brain function image;
[0141] Input the characteristic information between each brain region into the clustering subnetwork of the image classification network, cluster each brain region, and cluster each brain region into different brain circuits;
[0142] Each brain circuit is input into the classification subnetwork of the image classification network to obtain the classification results of brain function images.
[0143] The computer-readable storage medium provided in the above embodiment has similar implementation principles and technical effects as those of the above method embodiment, and will not be described in detail here.
[0144] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0146] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A brain image classification method, characterized in that: The method comprises: Acquiring brain functional images to be classified; According to the brain function image, obtaining a graph characteristic matrix corresponding to the brain function image; Inputting the image characteristic matrix into a feature extraction subnetwork of a preset image classification network to obtain feature information between brain regions in the brain function image; Inputting the characteristic information between the brain regions into the clustering subnetwork of the image classification network, clustering the brain regions, and clustering the brain regions into different brain circuits; wherein, when clustering the brain regions into different brain circuits, the relationship between brain circuits is initialized by averaging the connections between brain regions within the circuit, and the initialization is based on the graph structure of the brain circuit; Each of the brain circuits is input into the classification subnetwork of the image classification network to obtain the classification result of the brain function image.
2. The method according to claim 1, characterized in that: The clustering subnetwork includes a clustering layer and an optimization layer; the feature information between the brain regions is input into the clustering subnetwork of the image classification network, the brain regions are clustered, and the brain regions are clustered into different brain circuits, including: Inputting the characteristic information between the brain regions into the clustering layer, clustering the brain regions according to a preset clustering allocation matrix, and clustering the brain regions into different initial brain circuits; Each of the initial brain circuits is input into the optimization layer, and each of the initial brain circuits is optimized to obtain each of the brain circuits.
3. The method according to claim 2, characterized in that The step of inputting each of the initial brain circuits into the optimization layer and optimizing each of the initial brain circuits to obtain each of the brain circuits includes: Each of the initial brain circuits is input into the optimization layer, the features of each of the initial brain circuits are learned and updated, and each of the initial brain circuits is optimized according to the updated features and a preset optimization method to obtain each of the brain circuits.
4. The method according to claim 1, characterized in that: The classification subnetwork includes a graph convolution layer; the step of inputting each of the brain circuits into the classification subnetwork of the image classification network to obtain the classification result of the brain function image includes: Inputting each of the brain circuits into the graph convolution layer, reducing the time dimension of the features of each of the brain circuits, and converting the features of each of the brain circuits after the time dimension reduction into corresponding feature values; The classification result of the brain function image is obtained according to the characteristic values corresponding to the characteristics of each brain circuit.
5. The method according to claim 4, characterized in that The classification subnetwork further includes a classification layer; the classification result of the brain function image is obtained according to the feature values corresponding to the features of each of the brain circuits, including: The feature values corresponding to the features of each brain circuit are input into the classification layer, a corresponding weight value is assigned to each feature value, and the classification result of the brain function image is obtained according to each feature value and the weight value corresponding to each feature value.
6. The method according to claim 5, characterized in that The method further comprises: According to the weight values corresponding to the characteristic values, the target brain circuit related to the classification result is determined.
7. The method according to claim 6, characterized in that The method further comprises: According to the cluster allocation matrix corresponding to the target brain circuit, the brain regions included in the target brain circuit are determined.
8. The method according to claim 1, characterized in that The feature extraction subnetwork includes a graph convolutional long short-term memory network.
9. The method according to claim 1, characterized in that: The step of obtaining a graph characteristic matrix corresponding to the brain function image according to the brain function image includes: Acquiring characteristic information of each brain region from the brain function image as a node feature; According to the node characteristics of each brain region, obtaining connection information between each brain region, and using the connection information as the connection between the nodes; The node features and the connections between the nodes are constructed into the graph property matrix.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.