Brain disease classification system, method, and device based on multi-modal feature similarity

Abstract

The application discloses a brain disease classification system and method based on multi-modal feature similarity, and relates to the technical field of magnetic resonance imaging, and comprises the following: a feature splicing module, which is used for screening target functional state data, inputting data features of structural state data and the target functional state data into a data splicing model, and outputting magnetic resonance brain image features; a similarity calculation module, which is used for calculating a first similarity between the brain image features and training sample image features; calculating a second similarity between sample image features of each training sample in a training sample set; obtaining a target similarity graph based on the second similarity and the first similarity; and a graph classification module, which is used for classifying the target similarity graph to perform disease classification. The similarity graph is classified, and brain disease classification is performed on a test set of people.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of magnetic resonance imaging, and particularly relates to a brain disease classification system and method based on multi-modal feature similarity. BACKGROUND

[0002] The development of magnetic resonance technology has played a positive role in the diagnosis and analysis of brain diseases. Existing research shows that using machine learning methods to analyze normal people and brain disease patients using sMRI (structural Magnetic Resonance Imaging) and fMRI (functional Magnetic Resonance Imaging) can effectively promote the understanding and diagnosis of brain diseases.

[0003] Machine learning has promoted people's ability to analyze brain diseases using magnetic resonance imaging, but different age and gender groups have great differences, and classic machine learning algorithms often cannot use different disease analysis methods according to different age and gender groups.

[0004] To sum up, how to calculate different similarities for different age and gender groups, enhance the ability of machine learning methods to analyze different age and gender groups, reduce the classification error of individual differences on magnetic resonance imaging, and improve the classification accuracy is a technical problem to be solved in the field. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a brain disease classification system and method based on multi-modal feature similarity, which can calculate different similarities for different age and gender groups, enhance the ability of machine learning methods to analyze different age and gender groups, reduce the classification error of individual differences on magnetic resonance imaging, and improve the classification accuracy. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a brain disease classification system based on multi-modal feature similarity, comprising:

[0007] The feature splicing module is configured to filter the functional state data based on the trained functional state data filtering model to obtain target functional state data, and input the structural state data features of the structural state data and the functional state data features of the target functional state data into a data splicing model, so that the data splicing model performs data feature splicing and outputs magnetic resonance brain image features.

[0008] The similarity calculation module is configured to perform first similarity calculation on the brain image feature and the training sample image feature to obtain a first similarity; perform second similarity calculation on sample image features between each training sample in the training sample set to obtain a second similarity; take the second similarity representing disease classification factors between each training sample as an edge of a constructed similarity graph, and take the first similarity as a node value in the similarity graph to obtain a target similarity graph.

[0009] The graph classification module is configured to input the target similarity graph into a disease classification prediction model, so that the disease classification prediction model outputs a label indicating whether it is a disease.

[0010] Optionally, the brain disease classification system based on multi-modal feature similarity further comprises:

[0011] The data conversion module is configured to filter the structural state data based on a statistical method to obtain target structural state data of different difference position voxels, and convert the target structural state data into point cloud data.

[0012] Optionally, the feature splicing module comprises:

[0013] The feature adjustment unit is configured to construct a data splicing model containing a regular term of a weight difference value of adjacent point cloud data, so that the data splicing model adjusts features of structural state data of the target structural state data and features of functional state data of the target functional state data by using different regularization parameters of the regular term.

[0014] The feature splicing unit is configured to splice the adjusted features of the structural state data and the features of the functional state data to output brain image features of the magnetic resonance image data.

[0015] Optionally, the brain disease classification system based on multi-modal feature similarity further comprises:

[0016] The regular term construction unit is configured to determine adjacent point pairs from the point cloud data and obtain weights of point cloud data in the adjacent point pairs; calculate an absolute value of a weight difference of the point cloud data in the adjacent point pairs, and construct a weight loss amount based on the absolute value of the weight difference and a regularization parameter, so as to construct a regular term based on the weight loss amount.

[0017] Optionally, the feature splicing module comprises:

[0018] The data screening unit is configured to pre-train a functional state data screening model by using target structure state data, to obtain a trained functional state data screening model containing target brain region weights for screening target functional state data corresponding to a target brain region of the target functional state data; and screen the functional state data based on the trained functional state data screening model, to obtain target functional state data of the target brain region.

[0019] Optionally, the similarity calculation module comprises:

[0020] The similarity calculation unit is configured to calculate similarities between the brain image feature and image features of each training sample in the training sample set, to obtain a plurality of first similarities; and calculate second similarities of age and gender factors between each training sample in the training sample set.

[0021] The similarity graph construction unit is configured to construct edges of a target similarity graph corresponding to the brain image feature based on the second similarities, and take the first similarities as node values of the target similarity graph.

[0022] Optionally, the graph classification module comprises:

[0023] The feature dimension reduction unit is configured to input the target similarity graph into the disease classification prediction model, so as to remove image features of disease classification factors containing age and gender from the target similarity graph based on a graph convolution strategy of the disease classification prediction model, to obtain a cleaned target similarity graph.

[0024] The graph classification unit is configured to fit and classify the cleaned target similarity graph by using a binary classification module in the disease classification prediction model, to output a classification result label corresponding to whether the cleaned target similarity graph is a disease.

[0025] In a second aspect, the present application discloses a brain disease classification method based on a similarity graph, comprising:

[0026] Screen the functional state data based on the trained functional state data screening model, to obtain target functional state data, and input structure state data features of the structure state data and functional state data features of the target functional state data into a data splicing model, so that the data splicing model splices data features, and outputs brain image features of the magnetic resonance image data.

[0027] The first similarity calculation is performed on the brain image feature and the training sample image feature to obtain a first similarity; the second similarity calculation is performed on the sample image features between each training sample in the training sample set to obtain a second similarity; the second similarity representing the disease classification factor between each training sample is taken as an edge of a constructed similarity graph, and the first similarity is taken as a node value in the similarity graph to obtain a target similarity graph;

[0028] The target similarity graph is input into a disease classification prediction model, so that the disease classification prediction model outputs a label of whether it is a disease.

[0029] In a third aspect, the present application discloses an electronic device, comprising:

[0030] A memory for saving a computer program;

[0031] A processor for executing the computer program to implement the steps of the brain disease classification method based on the similarity graph disclosed above.

[0032] It can be seen that the application discloses a brain disease classification system based on multi-modal feature similarity, which comprises a feature splicing module, a similarity calculation module and a graph classification module. The feature splicing module is used for screening functional state data based on a trained functional state data screening model to obtain target functional state data, and inputting structural state data features of structural state data and functional state data features of the target functional state data into a data splicing model, so that the data splicing model performs data feature splicing and outputs magnetic resonance brain image features. The similarity calculation module is used for performing first similarity calculation on the brain image features and training sample image features to obtain a first similarity, performing second similarity calculation on sample image features between each training sample in a training sample set to obtain a second similarity, taking the second similarity representing disease classification factors between each training sample as an edge of a similarity graph, and taking the first similarity as a node value in the similarity graph to obtain a target similarity graph. The graph classification module is used for inputting the target similarity graph into a disease classification prediction model, so that the disease classification prediction model outputs a label of whether it is a disease. It can be seen that the functional state data in the magnetic resonance image to be classified is screened to obtain target functional state data, and then the structural state data features of the structural state data and the functional state data features of the target functional state data are input into the data splicing model for data feature splicing to obtain brain image features of the magnetic resonance image to be classified. Then, similarity calculation is performed on the brain image features, and similarity calculation is also performed on the training samples in the training sample set. The similarities calculated according to the two are taken as the edges and node values of the similarity graph respectively. Through similarity analysis on the similarity graph, different similarities can be calculated for different age and gender groups, the ability of the machine learning method to analyze different age and gender groups is increased, the influence of individual differences on disease classification is reduced, and the classification accuracy for severe depressives is improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on the provided drawings.

[0034] Figure 1 A structural schematic diagram of a brain disease classification system based on multi-modal feature similarity disclosed by the present application;

[0035] Figure 2 A process schematic diagram of converting structural state data into point cloud data disclosed by the present application;

[0036] Figure 3A schematic diagram of a multi-feature splicing process of different intensity regularization disclosed in the present application;

[0037] Figure 4 A schematic diagram of a specific brain disease classification system structure based on multi-modal feature similarity disclosed in the present application;

[0038] Figure 5 A schematic diagram of a functional state data screening process disclosed in the present application;

[0039] Figure 6 A schematic diagram of a similarity calculation and similarity graph construction process disclosed in the present application;

[0040] Figure 7 A schematic diagram of a target similarity graph fitting and disease classification process disclosed in the present application;

[0041] Figure 8 A brain disease classification method based on a similarity graph disclosed in the present application;

[0042] Figure 9 A schematic diagram of an electronic device structure disclosed in the present application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0044] The development of magnetic resonance technology has played a positive role in the diagnosis and analysis of brain diseases. Existing research shows that using machine learning methods to analyze sMRI and fMRI of normal people and brain disease patients can effectively promote the understanding and diagnosis of brain diseases.

[0045] Machine learning has promoted people's ability to analyze brain diseases using magnetic resonance images, but different age and gender groups have great differences, and classic machine learning algorithms often cannot use different disease analysis methods according to different age and gender groups.

[0046] Therefore, the present application provides a brain disease classification scheme based on a similarity graph, which can calculate different similarities for different age and gender groups, enhance the ability of machine learning methods to analyze different age and gender groups, reduce the classification error of magnetic resonance images caused by individual differences, and improve the classification accuracy.

[0047] REFERENCE Figure 1As shown, the embodiment of the present application discloses a brain disease classification system based on multi-modal feature similarity, comprising:

[0048] The feature splicing module 11 is configured to filter the functional state data based on the trained functional state data filtering model to obtain target functional state data, and input the structural state data features of the structural state data and the functional state data features of the target functional state data into a data splicing model, so that the data splicing model performs data feature splicing and outputs the magnetic resonance brain image features.

[0049] The similarity calculation module 12 is configured to perform first similarity calculation on the brain image features and the training sample image features to obtain first similarity, perform second similarity calculation on the sample image features between each training sample in the training sample set to obtain second similarity, and take the second similarity representing the disease classification factors between each training sample as an edge of a similarity graph, and take the first similarity as a node value in the similarity graph to obtain a target similarity graph.

[0050] The graph classification module 13 is configured to input the target similarity graph into a disease classification prediction model, so that the disease classification prediction model outputs a label indicating whether it is a disease.

[0051] The brain disease classification system based on multi-modal feature similarity further comprises:

[0052] The data conversion module is configured to filter the structural state data based on a statistical method to obtain target structural state data of different difference position voxel states, and convert the target structural state data into point cloud data.

[0053] The brain disease classification system based on multi-modal feature similarity, the feature splicing module comprises:

[0054] The feature adjustment unit is configured to construct a data splicing model containing a regular term of a weight difference value of adjacent point cloud data, so that the data splicing model adjusts the structural state data features of the target structural state data and the functional state data features of the target functional state data by using different regularization parameters of the regular term.

[0055] The feature splicing unit is configured to perform data feature splicing on the adjusted structural state data features and functional state data features, and output the brain image features of the magnetic resonance image data.

[0056] The brain disease classification system based on multi-modal feature similarity further comprises:

[0057] The regular term construction unit is configured to determine a pair of adjacent points from the point cloud data, and obtain a weight of the point cloud data in the pair of adjacent points; calculate an absolute value of a weight difference of the point cloud data in the pair of adjacent points, and construct a weight loss amount based on the absolute value of the weight difference and a regularization parameter, so as to construct a regular term based on the weight loss amount.

[0058] It can be understood that the structural state and functional state of the brain to be classified are obtained by using the data acquisition module, wherein the structural state data is specifically sMRI data, which is used to detect whether the organ structure in the brain has structural changes such as growth, atrophy, deformation, etc. through sMRI, and the functional state data is fMRI data, which is used to detect the active state of the brain region, the signal rise of the brain region, etc. However, the obtained structural state data and functional state data need to be screened and processed. Specifically, the structural state data SMRI is screened and analyzed by using the data conversion module, the structural state data SMRI of the voxel state is converted into point cloud data by a statistical method, so as to facilitate the further solving of the similarity in the next step. It should be noted that only the structural state data of the position with difference is retained when the structural state data is screened, and then the structural state data of the position with difference is converted into point cloud data to construct a point cloud-like, as shown in Figure 2 .

[0059] It can be understood that the target functional state data obtained by screening the functional state data and the functional state data features of the screened structural state data are input into the data splicing model together, so that the data splicing model splices the data features to obtain the brain image features after dimensionality reduction. The data splicing model is obtained by using the feature adjustment unit to construct a data splicing model containing a regular term of the weight difference value of the adjacent point cloud data, and then using the data splicing model to extract the features of the structural state data features and the functional state data features under different regularization parameters to obtain the data features under different regularization strengths, and then using the feature splicing unit to splice the data features, and taking the spliced data features as the brain image features. In the construction process of the data splicing model, the regular term construction unit is also included. Specifically, the weight difference value of the adjacent point cloud is added to the neural network as a regularization term, so as to reduce the overfitting phenomenon of the neural network. For example, if point A and point B are adjacent points, the weight of point A is W a when the neural network is used for calculation, and the weight of point B is W b , a weight loss amount Loss=k×W a -W b is defined and added to the loss function calculation of the neural network as a regularization term, and k is a parameter for adjusting the regularization strength. By adjusting the value of k, regularization of different strengths can be realized.

[0060] It can be understood that after the brain image features are acquired, the similarity calculation module 13 is used to calculate the similarity of the features, wherein the first similarity and the second similarity need to be calculated. It should be noted that the first similarity is the similarity between the brain image features and the training sample image features, and the second similarity is the feature similarity between each training sample. The second similarity represents the disease classification factors between each training sample. Then, a graph structure is constructed with the second similarity as the edge, and the first similarity is taken as the value of the node of the graph structure to obtain the target similarity graph structure of the brain image features. Finally, the target similarity graph structure is input into the disease classification prediction model, the edges and values of the target similarity graph structure are classified and analyzed by the disease classification prediction model, and finally the disease classification result of the magnetic resonance image data is output.

[0061] It can be seen that the application discloses a brain disease classification system based on multi-modal feature similarity, which comprises: a feature splicing module, which is used to filter functional state data based on a trained functional state data filtering model to obtain target functional state data, and input structural state data features of structural state data and functional state data features of the target functional state data into a data splicing model, so that the data splicing model splices data features and outputs brain image features of magnetic resonance image data; a similarity calculation module, which is used to calculate the first similarity between the brain image features and the training sample image features, and obtain the first similarity; calculate the second similarity between the sample image features of each training sample in the training sample set, and obtain the second similarity; take the second similarity representing the disease classification factors between each training sample as the edge of the similarity graph, and take the first similarity as the node value in the similarity graph to obtain the target similarity graph; and a graph classification module, which is used to input the target similarity graph into a disease classification prediction model, so that the disease classification prediction model outputs a label indicating whether it is a disease. It can be seen that by filtering the functional state data in the magnetic resonance image to be classified to obtain target functional state data, and then inputting the structural state data features of the structural state data and the functional state data features of the target functional state data into the data splicing model to splice the data features, the brain image features of the magnetic resonance image to be classified are obtained, and then the similarity of the brain image features is calculated, and the similarity between the training samples in the training sample set is also calculated. The similarities calculated according to the similarities of the two parties are respectively taken as the edge and the node value of the similarity graph. By analyzing the similarity of the similarity graph, different similarities can be calculated for different age and gender groups, the ability of the machine learning method to analyze different age and gender groups is increased, the influence of individual differences on disease classification is reduced, and the classification accuracy for severe depressives is improved.

[0062] Reference Figure 4As shown, the embodiment of the present application discloses a specific brain disease classification system based on multi-modal feature similarity. Compared with the previous embodiment, the present embodiment further describes and optimizes the technical solution. Specifically,

[0063] The feature splicing module 11 comprises:

[0064] The data screening unit 111 is configured to pre-train a functional state data screening model by using target structure state data, to obtain a trained functional state data screening model containing target brain region weights for screening target functional state data corresponding to a target brain region of the target functional state data; and screen the functional state data based on the trained functional state data screening model, to obtain target functional state data of the target brain region.

[0065] The similarity calculation module 12 comprises:

[0066] The similarity calculation unit 121 is configured to calculate similarities between the brain image features and each training sample image feature in the training sample set, to obtain a plurality of first similarities; and calculate second similarities between each training sample in the training sample set, which contain age and gender disease classification factors.

[0067] The similarity graph construction unit 122 is configured to construct edges of a target similarity graph corresponding to the brain image features based on the second similarities, and take the first similarities as node values of the target similarity graph.

[0068] The graph classification module 13 comprises:

[0069] The feature dimension reduction unit 131 is configured to input the target similarity graph into the disease classification prediction model, so as to remove image features containing age and gender disease classification factors from the target similarity graph based on a graph convolution strategy of the disease classification prediction model, to obtain a cleaned target similarity graph.

[0070] The graph classification unit 132 is configured to fit and classify the cleaned target similarity graph by using a binary classification module in the disease classification prediction model, to output a classification result label corresponding to whether the cleaned target similarity graph is a disease.

[0071] It can be understood that the data screening unit 111 screens the functional state data, wherein the screening of the functional state data needs to use the trained neural network as a functional state data screening model, and the structural state data sMRI is also needed to pre-train the neural network in the functional state data screening model training process to preliminarily obtain the brain region weight, and then the trained neural network is used as the functional state data screening model to screen the functional state data, and the Pearson correlation matrix is calculated using the screened fMRI data to realize the construction of the functional brain network. The region to be processed is outlined in a box, a circle, an ellipse, an irregular polygon and the like from the processed magnetic resonance image data, which is called a region of interest ROI (Region Of Interest, region of interest), as shown in Figure 5 The 116 functional state data ROIs 1, 2, 3, …, 116 to be screened are uniformly input to the functional state data screening model trained using the structural state data, so that the functional state data screening model screens out the target functional state data, removes n invalid functional state data, and obtains 116-n target functional state data.

[0072] It can be understood that the similarity calculation unit 121 obtains a brain image feature after each magnetic resonance image data sample, and puts the corresponding brain image feature into a unified set as a test set, and calculates the first similarity between the brain image feature in the test set and each training sample image feature in the training set. The test set is a set of brain image features obtained by processing the magnetic resonance brain images of people of unknown brain disease types respectively. The second similarity between each sample in the training set is calculated. It should be noted that the calculated second similarity is used to represent the disease classification influencing factors such as age and gender. The training set is a set of brain image features obtained by processing the magnetic resonance brain images of people of known brain disease types respectively. It should be noted that the training set and the test set are features obtained by feature extraction and splicing of sMRI and fMRI data. Their feature extraction steps and feature splicing steps are the same. However, the populations used by the training set and the test set are different. In general, the brain disease types of all people in the training set population are known, but the brain disease types of all people in the test set population are unknown, so the brain image data of the training set population is used to train the disease classification prediction model, and then the disease classification prediction model is used to predict the brain disease types of the test set population.

[0073] As Figure 6As shown, the similarity graph construction unit 122 uses the second similarity between each sample in the training set as the edge between each sample to construct a graph structure, and uses the first similarity obtained by respectively calculating a test set sample with all samples in the training set as the value of the node of this graph structure, to obtain a similarity graph structure for each test set sample. The samples contained in the training set are a, b, c, d, and e, and then the edges are formed according to the second similarity of the samples in the training set, wherein the edge between each sample and sample represents the similarity between the two samples connected by the edge. The samples contained in the test set are 1 and 2. Then, the first similarity a1, b1, c1, d1, e1 between sample 1 and each a, b, c, d, e is calculated, and the first similarity a2, b2, c2, d2, e2 between sample 2 and each a, b, c, d, e is calculated. Then, a1, b1, c1, d1 are used to replace the samples a, b, c, d, e connected by the edges in the training set as node values to generate the target similarity graph of sample 1; and a2, b2, c2, d2, e2 are used to replace the samples a, b, c, d, e connected by the edges in the training set as node values to generate the target similarity graph of sample 2.

[0074] It can be understood that when the target similarity graph is obtained, as shown in Figure 7 The feature dimension reduction unit 131 inputs the target similarity graph into the disease classification prediction model, so as to reduce the dimension of the target similarity graph corresponding to each test set sample by using the graph convolutional neural network autoencoder in the model, wherein the graph convolution strategy is used to reduce the feature difference caused by factors other than age and gender, thereby reducing the influence of the age and gender factors on disease classification.

[0075] The graph classification unit 132 uses the SVM (Support Vector Machines, binary classification model) to fit and classify the features obtained by using the graph convolutional neural network for dimension reduction, thereby obtaining a label for each sample in the test set whether it is a disease.

[0076] As can be seen, by using the graph convolutional neural network autoencoder in the disease classification prediction model to reduce the dimension of the constructed target similarity graph, the influence of individual differences on disease classification can be greatly reduced, especially the influence of the age and gender factors represented in the second similarity on disease classification, thereby improving the classification accuracy of the existing machine learning algorithm on the magnetic resonance image by 2%-10%.

[0077] Referring to Figure 8 The application also discloses a brain disease classification method based on a similarity graph, which comprises the following steps:

[0078] Step S11: screening the functional state data based on the trained functional state data screening model to obtain target functional state data, and inputting the structural state data features of the structural state data and the functional state data features of the target functional state data into the data splicing model, so that the data splicing model performs data feature splicing and outputs the magnetic resonance brain image features.

[0079] In this embodiment, the trained neural network is used as a functional state data screening model for screening. In the training process of the functional state data screening model, the neural network needs to be pre-trained using the screened structural state data SMRI to preliminarily obtain brain region weights. Then, the trained neural network is used as a functional state data screening model for screening of functional state data, and the Pearson correlation matrix is calculated using the screened FMRI data to realize construction of a functional brain network. The target functional state data after screening is obtained, and the functional state data features and the structural state data features of the target functional state data and the structural state data are input into the data splicing model, so that the data splicing model performs data feature splicing on the two to obtain brain image features after splicing and dimensionality reduction. The data splicing model is obtained as follows: a data splicing model including a regular term of a weight difference value of adjacent point cloud data is constructed by using a feature adjustment unit, then feature extraction of the structural state data features and the functional state data features is performed under different regularization parameters by using the data splicing model to obtain data features under different regularization strengths, then the data features are spliced by using a feature splicing unit, and the spliced data features are used as brain image features. In the construction process of the data splicing model, a regular term construction unit is further included, specifically, the weight difference value of adjacent point clouds is added to the neural network as a regularization term to reduce the overfitting phenomenon of the neural network. For example, if point A and point B are adjacent points, the weight of point A is W a when the neural network is used for calculation, and the weight of point B is W b , a weight loss amount Loss=k×W a -W b is defined and added to the loss function calculation of the neural network as a regularization term, and k is a parameter for adjusting the regularization strength. By adjusting the value of k, regularization of different strengths can be realized.

[0080] In this embodiment, before the functional state data is screened based on the trained functional state data screening model, the method further includes: acquiring magnetic resonance image data containing the structural state and the functional state of the brain to be classified. Specifically, the structural state data and the functional state data of the brain to be classified are acquired, wherein the structural state data is specifically SMRI data, which is used to detect whether the structure of organs in the brain has structural changes such as growth, atrophy, deformation, etc. through SMRI, and the functional state data is FMRI data, which is used to detect the active state of the brain region, the signal rise of the brain region, etc. during different activities. Then, the structural state data SMRI is screened and analyzed. Specifically, the structural state data SMRI is screened by a statistical method, and the structural state data SMRI of the voxel state is converted into point cloud data, so as to facilitate further solving of the similarity in the next step. It should be noted that only the structural state data of the position with difference is retained when the structural state data is screened, and then the structural state data of the position with difference is converted into point cloud data to construct a point cloud-like structure.

[0081] Step S12: performing first similarity calculation on the brain image feature and the training sample image feature to obtain a first similarity; performing second similarity calculation on the sample image features between each training sample in the training sample set to obtain a second similarity; taking the second similarity representing the disease classification factors between each training sample as an edge of a constructed similarity graph, and taking the first similarity as a node value in the similarity graph to obtain a target similarity graph.

[0082] In this embodiment, the first similarity and the second similarity are calculated. It should be noted that the first similarity is the similarity between the brain image feature and the training sample image feature, and the second similarity is the feature similarity between each training sample. The second similarity represents the disease classification factors between each training sample. Then, the second similarity is taken as an edge to construct a graph structure, and the first similarity is taken as a node value of the graph structure to obtain a target similarity graph structure of the brain image feature.

[0083] Step S13: inputting the target similarity graph into a disease classification prediction model, so that the disease classification prediction model outputs a label of whether it is a disease.

[0084] In this embodiment, the target similarity graph structure is input into the disease classification prediction model, so that the graph convolutional neural network autoencoder in the model reduces the dimensionality of the target similarity graph corresponding to each test set sample. The graph convolution strategy is used to reduce the feature difference not caused by age and gender, thereby reducing the influence of the age and gender factors on disease classification. Then, the features obtained by using the graph convolutional neural network for dimensionality reduction are fitted and classified by using SVM, so as to obtain a label of whether it is a disease for each sample in the test set.

[0085] It can be seen that the application discloses a method for classifying brain diseases based on a similarity graph. The method comprises the following steps: obtaining functional state data of a magnetic resonance image to be classified; screening the functional state data based on a trained functional state data screening model to obtain target functional state data; inputting structural state data features of structural state data and functional state data features of the target functional state data into a data splicing model, so that the data splicing model performs data feature splicing and outputs brain image features of the magnetic resonance image data; performing first similarity calculation on the brain image features and training sample image features to obtain a first similarity; performing second similarity calculation on sample image features between each training sample in a training sample set to obtain a second similarity; taking the second similarity representing disease classification factors between each training sample as an edge of a similarity graph, and taking the first similarity as a node value in the similarity graph to obtain a target similarity graph; and inputting the target similarity graph into a disease classification prediction model, so that the disease classification prediction model outputs a label indicating whether the magnetic resonance image to be classified is a disease. It can be seen that, by screening the functional state data in the magnetic resonance image to be classified, obtaining the target functional state data, and then inputting the structural state data features of the structural state data and the functional state data features of the target functional state data into the data splicing model to perform data feature splicing on the two, the brain image features of the magnetic resonance image to be classified are obtained, and then similarity calculation is performed on the brain image features and the training sample set. The similarities calculated based on the two are taken as edges and node values of a similarity graph, respectively. By performing similarity analysis on the similarity graph, different similarities can be calculated for different age and gender groups, the ability of a machine learning method to analyze different age and gender groups is increased, the influence of individual differences on disease classification is reduced, and the classification accuracy for severe depressive patients is improved.

[0086] Further, the embodiment of the application further discloses an electronic device, Figure 9 FIG. 1 is a structural diagram of an electronic device 20 according to an example embodiment. The contents in the figure cannot be considered as any limitation on the use range of the application.

[0087] Figure 9 A structural diagram of an electronic device 20 is provided in the embodiment of the application. The electronic device 20 specifically can include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is used to store a computer program, the computer program is loaded and executed by the processor 21 to realize the related steps in the brain disease classification method based on the similarity graph disclosed in any of the preceding embodiments. In addition, the electronic device 20 in the embodiment specifically can be an electronic computer.

[0088] In this embodiment, the power supply 23 is configured to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 is configured to create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which will not be specifically limited herein; the input and output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application requirements, which will not be specifically limited herein.

[0089] The processor 21 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 21 can also include a main processor and a coprocessor. The main processor is a processor for processing data in a wake-up state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 can also include an AI (Artificial Intelligence) processor configured to process machine learning-related computing operations.

[0090] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0091] The operating system 221 is used to manage and control each hardware device on the electronic device 20 and the computer program 222, so as to realize the operation and processing of the processor 21 on the mass data 223 in the memory 22, which can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the similarity map-based brain disease classification method disclosed by the electronic device 20, the computer program 222 can further include a computer program capable of completing other specific work. In addition to the data received by the electronic device from the external device, the data 223 can also include the data collected by the self input and output interface 25, etc.

[0092] Finally, it should be noted that the relational terms herein such as first and second, and the like, are used solely to distinguish one from another entity or action, without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... " does not, without more limitations, preclude the existence of additional identical elements in the process, method, article, or apparatus that includes the recited element.

[0093] The above describes the brain disease classification system, method and device based on multi-modal feature similarity provided by the present application in detail. The principle and implementation of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. For those skilled in the art, according to the idea of the present application, the specific implementation and application range can be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A brain disease classification system based on multi-modal feature similarity, characterized in that, The method comprises the following steps: The feature splicing module is used for screening the functional state data based on the trained functional state data screening model to obtain target functional state data, and inputting the structural state data features of the structural state data and the functional state data features of the target functional state data into a data splicing model, so that the data splicing model performs data feature splicing and outputs magnetic resonance brain image features; The similarity calculation module is used for performing first similarity calculation on the brain image features and training sample image features to obtain first similarity, performing second similarity calculation on sample image features between each training sample in the training sample set to obtain second similarity, taking the second similarity representing disease classification factors between each training sample as an edge of a similarity graph, and taking the first similarity as a node value in the similarity graph to obtain a target similarity graph; The graph classification module is used for inputting the target similarity graph into a disease classification prediction model to output a label of whether it is a disease; The similarity calculation module comprises a similarity calculation unit, wherein After each magnetic resonance image data sample obtains a brain image feature, the similarity calculation unit is used for putting the corresponding brain image feature into a unified set as a test set, calculating first similarity between the brain image features in the test set and each training sample image feature in the training set, and the test set is a set of brain image features obtained by respectively processing magnetic resonance brain images of people with unknown brain disease types, and the training set and the test set are features obtained by performing feature extraction and splicing on functional state data features and structural state data features, and the feature extraction step and the feature splicing step are the same; The graph classification module is specifically used for performing dimension reduction on the target similarity graph corresponding to each test set sample by a graph convolutional neural network autoencoder in the disease classification prediction model. 2.The brain disease classification system based on multi-modal feature similarity according to claim 1, wherein, Further comprising: The data conversion module is used for screening the structural state data based on a statistical method to obtain target structural state data of different difference position voxel states, and converting the target structural state data into point cloud data. 3.The brain disease classification system based on multi-modal feature similarity according to claim 2, wherein, The feature splicing module comprises: The feature adjustment unit is used for constructing a data splicing model containing a regular term of a weight difference value of adjacent point cloud data, so that the data splicing model performs feature adjustment on the structural state data features of the target structural state data and the functional state data features of the target functional state data by using different regularization parameters of the regular term; The feature splicing unit is used for performing data feature splicing on the adjusted structural state data features and functional state data features to output brain image features of magnetic resonance image data; The feature adjustment unit is specifically used for constructing a data splicing model containing a regular term of a weight difference value of adjacent point cloud data, and then performing feature extraction on the structural state data features and the functional state data features by using the data splicing model under different regularization parameters to obtain data features under different regularization strengths. The feature splicing unit is specifically configured to splice the structural state data features and the functional state data features under different regularization strengths, and take the spliced data features as the brain image features. 4.The brain disease classification system based on multi-modal feature similarity according to claim 3, wherein, Further comprising: The regularization term construction unit is configured to determine adjacent point pairs from the point cloud data, and obtain weights of the point cloud data in the adjacent point pairs; Calculate the absolute value of the weight difference of the point cloud data in the adjacent point pairs, and construct a weight loss based on the absolute value of the weight difference and a regularization parameter, so as to construct a regularization term based on the weight loss. 5.The brain disease classification system based on multi-modal feature similarity according to claim 1, wherein, The feature splicing module comprises: The data screening unit is configured to pre-train a functional state data screening model using target structural state data to obtain a trained functional state data screening model containing weights of target brain regions, which is used to screen target functional state data corresponding to target brain regions of the target functional state data; and screen the functional state data based on the trained functional state data screening model to obtain target functional state data of the target brain region.

6. The brain disease classification system based on multi-modal feature similarity according to any one of claims 1 to 5, characterized in that, The similarity calculation module comprises: The similarity calculation unit is configured to calculate the similarity between the brain image features and each training sample image feature in the training sample set to obtain a plurality of first similarities; and calculate the second similarity of the age and gender factors between each training sample in the training sample set; The similarity graph construction unit is configured to construct edges of a target similarity graph corresponding to the brain image features based on the second similarity, and take the first similarity as a node value of the target similarity graph. 7.The brain disease classification system based on multi-modal feature similarity according to claim 1, wherein, The graph classification module comprises: The feature dimension reduction unit is configured to input the target similarity graph into the disease classification prediction model, so that the disease classification factors including age and gender in the target similarity graph are removed by using a graph convolution strategy based on the disease classification prediction model, to obtain a cleaned target similarity graph; The graph classification unit is configured to fit and classify the cleaned target similarity graph by using a binary classification module in the disease classification prediction model, to output a classification result label corresponding to whether the cleaned target similarity graph is a disease. 8.A method for classifying brain diseases based on a similarity map, characterized by, Based on the trained functional state data screening model, the functional state data is screened to obtain target functional state data, and the structural state data features of the structural state data and the functional state data features of the target functional state data are input into a data splicing model, so that the data splicing model splices data features and outputs magnetic resonance brain image features; The brain image features and the training sample image features are subjected to first similarity calculation to obtain first similarities, and the sample image features between each training sample in the training sample set are subjected to second similarity calculation to obtain second similarities; the second similarities representing disease classification factors between each training sample are taken as edges of a similarity graph, and the first similarities are taken as node values in the similarity graph, to obtain a target similarity graph; The target similarity graph is input into a disease classification prediction model, so that the disease classification prediction model outputs a label of whether it is a disease; ​ The first similarity calculation of the brain image feature and the training sample image feature includes: After obtaining a brain image feature for each magnetic resonance image data sample, the corresponding brain image feature is put into a unified set as a test set; the first similarity of the brain image feature in the test set and each training sample image feature in the training set is calculated; the test set is a set composed of brain image features obtained by respectively processing magnetic resonance brain images of people with unknown brain disease types; the training set and the test set are features obtained by using functional state data features and structural state data features for feature extraction and splicing, and the feature extraction step and the feature splicing step are the same; The target similarity graph is input into a disease classification prediction model, including: The target similarity graph corresponding to each test set sample is reduced in dimension by a graph convolutional neural network autoencoder in the disease classification prediction model.

9. An electronic device, comprising: Including: A memory for saving a computer program; A processor for executing the computer program to implement the steps of the brain disease classification method based on a similarity graph according to claim 8.

Images