Autism spectrum disorder classification method and system based on evidence decision fusion

Abstract

The invention discloses an autism spectrum disorder classification method and system based on evidence decision fusion. The method comprises the following steps: creating an autism multi-modal data set; an autism classification model based on evidence decision fusion is constructed and comprises a data preprocessing module, an evidence extraction module, a category credibility and classification result uncertainty estimation module and a decision fusion module. Training a model by using samples in the autism multi-modal data set; and performing autism classification on a newly input subject sample by using the trained model. According to the method, key evidences of three modes of T1 weighted imaging, diffusion tensor imaging and functional magnetic resonance imaging are comprehensively utilized, and a reliable and credible classification decision framework is constructed according to a Dempster combination rule, so that the accuracy and robustness of autism classification can be improved, overall classification uncertainty evaluation of a final decision can be given, and the accuracy and robustness of autism classification are improved. And better model interpretability and decision credibility are provided.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and particularly to an autism spectrum disorder classification method and system based on evidence decision fusion. Background Art

[0002] Autism spectrum disorder (ASD), also known as autism, is defined as a pervasive developmental disorder caused by developmental disorders of the nervous system. Its main clinical manifestations are social (communication) disorders, repetitive and stereotyped behaviors, and narrow interests and hobbies. It is more common in children. The earlier autism is detected and intervened, the better the prognosis. In particular, the nervous system of infants and young children has high plasticity. Timely and appropriate early intervention can improve the adaptability and cognitive ability of patients. Therefore, it is very necessary to carry out early screening for autism in infants and young children and enter the diagnosis process as early as possible after discovering risk cases.

[0003] Traditional early screening methods for autism mainly include two forms: caregiver reports or professional observations based on scales and observation checklists based on game tasks. The diagnostic results rely on the experience of medical staff and are affected by personal subjective factors, lacking effectiveness and objectivity. The main reason for the current difficulty in diagnosing autism and the low diagnostic accuracy is the heterogeneity of the pathological mechanism of autism itself. In recent years, the development of neuroimaging technology has greatly promoted the understanding of the pathological mechanism of autism. The combination of neuroimaging technology and artificial intelligence technology provides a new opportunity for the early and accurate diagnosis of autism.

[0004] Neuroimaging technology has great advantages in obtaining fine information on brain structure and function and then capturing specific features of different pathological subtypes. Therefore, the objective diagnosis research of autism based on neuroimaging has received great attention. Currently, the imaging technologies widely used in the refined diagnosis of autism mainly include structural magnetic resonance imaging (sfMRI) and functional magnetic resonance imaging (fMRI). Structural magnetic resonance imaging (sfMRI) includes T1-weighted imaging (T1WI) and diffusion-tensor imaging (DTI), which can capture subtle brain structure variations in autistic infants and young children, and thus have good performance in the early diagnosis of autism. Functional magnetic resonance imaging (fMRI) has both high temporal resolution and spatial resolution, and can provide extremely rich information on brain function activities and the static and dynamic characteristics of brain function networks, providing effective features for the objective diagnosis of autism. At the same time, with the rapid development of artificial intelligence technology and the improvement of computer hardware computing power, people have begun to study the use of computer technology to assist doctors in the diagnosis of ASD. Using deep learning technology to process neuroimaging for ASD classification has become an important research topic.

[0005] However, in traditional deep neural network classification models, the Softmax activation function is usually used in the last layer to output the prediction probabilities of each category, which may lead to overconfidence of the model and fail to effectively reflect the uncertainty of the classification results.

[0006] On the other hand, currently in the field of ASD screening, mainly single-modal images are used. However, due to the limitations of various imaging technologies in terms of specificity, reliability, and sensitivity, only single-modal image data cannot provide more comprehensive pathological information, resulting in difficulties in improving the classification accuracy of ASD; while multi-modal data can combine various different modal data to achieve information complementarity, which can effectively improve the classification accuracy. But how to effectively fuse multi-modal data is a challenging problem. In current research, multi-modal fusion methods are mainly divided into three categories: data-level fusion, feature-level fusion, and decision-level fusion. Data-level fusion can fuse information from multiple data sources, but it has great limitations when facing heterogeneous data. Feature-level fusion methods can combine abstract representations at different levels in a deep learning model to enhance the expressive ability of features and further improve the robustness of the model. However, different fusion methods are applicable to different features and tasks, and effective fusion methods for different modal features are a problem worthy of research. Decision-level fusion can fuse multiple decisions to reduce the risk and error of a single decision, thereby improving the reliability of the decision. However, the performance of decision-level fusion methods is not only related to the performance of a single modality but also affected by the conflicts between different modalities. How to balance the complex relationships between modalities is the key and difficult point in achieving decision-level fusion. Summary of the Invention

[0007] The present invention aims to solve at least one of the technical problems existing in the related art to a certain extent.

[0008] An object of the present invention is to provide an autism spectrum disorder classification method based on evidence decision fusion. By applying the Subjective Logic theory, the basic probability assignment function in the Dempster-Shafer evidence theory is formalized as a Dirichlet distribution in the classification framework. The category credibility and classification result uncertainty of each modality in the classification decision are evaluated through the parameterization of the Dirichlet distribution, and a reliable and credible classification decision framework is constructed according to the Dempster combination rule to achieve reliable multi-modal fusion, with the expectation of improving the sensitivity and accuracy of early autism screening.

[0009] Another object of the present invention is to provide an autism spectrum disorder classification system based on evidence decision fusion.

[0010] To achieve the above object, on the one hand, the present invention provides an autism spectrum disorder classification method based on evidence decision fusion, including:

[0011] S1. Obtain multi-modal imaging data and corresponding autism category labels, and construct a data set;

[0012] S2. Construct an autism classification model based on evidence decision fusion, and the autism classification model is constructed as:

[0013] Use the multi-modal imaging data as input, and extract evidence from each modal imaging data respectively;

[0014] Then, according to the evidence extracted from each modality, generate the category credibility and classification result uncertainty value corresponding to each modality;

[0015] Again, according to the category credibility and classification result uncertainty value of each modality, use the Dempster combination rule to calculate the total category credibility and total classification result uncertainty value after multi-modal fusion, and obtain the final classification result based on this, as the output of the autism classification model;

[0016] S3. Use the data set constructed in step S1 to train the autism classification model constructed in step S2, and adjust the parameters of the autism classification model to the optimal through the error backpropagation algorithm;

[0017] S4. Use the trained autism classification model to classify the newly input multi-modal imaging data to be tested for autism spectrum disorder.

[0018] A further preferred technical solution of the present invention is that the multi-modal imaging data in steps S1 and S4 includes fMRI images, T1WI images, and DTI images.

[0019] Preferably, when training the autism classification model or classifying the newly input multi-modal imaging data to be tested for autism spectrum disorder, first preprocess the fMRI images, T1WI images, and DTI images to generate fMRI modality samples, T1WI modality samples, and DTI modality samples respectively.

[0020] Preferably, the method for extracting evidence from each modal imaging data in step S2 is as follows:

[0021] S21. Construct a deep neural network for each of the fMRI modality, T1WI modality, and DTI modality, perform feature learning on the preprocessed fMRI modality samples, T1WI modality samples, and DTI modality samples respectively, and generate evidence supporting the k-th category through a fully connected layer containing K neurons and a Softplus activation function The superscript m represents the modal type, where m ∈ {M, T, D}, m = M represents the fMRI modality, m = T represents the T1WI modality, and m = D represents the DTI modality; the subscript k represents the index number of the autism category, k = 1, 2, …, K, and K is the number of autism categories; The larger the value of

[0022] Preferably, in step S2, according to the evidence of each extracted modality, the category credibility and the classification result uncertainty value of the corresponding modality are generated. The specific method is as follows:

[0023] S22. Convert the evidence of each modality supporting the k-th category extracted into the concentration parameter of the Dirichlet distribution of the corresponding modality The calculation formula is:

[0024]

[0025] S23. For each sample in the K-classification, assign the K autism category credibilities of each modality and the classification result uncertainty u m , and satisfy:

[0026]

[0027] where, u m ≥ 0;

[0028] S24. Calculate the category credibility provided by the corresponding modality and the classification result uncertainty u according to the evidence of each modality supporting the k-th category m . The calculation formulas are respectively:

[0029]

[0030] where, S m is the Dirichlet intensity of the corresponding modality,

[0031] Preferably, in step S2, according to the category credibility and the classification result uncertainty value of each modality, the total category credibility and the total classification result uncertainty value after multi-modal fusion are calculated by the Dempster combination rule, and the final classification result is obtained accordingly; the specific method is as follows:

[0032] S25. Use the category credibility provided by each modality and the classification result uncertainty u m to obtain the subjective opinions of the fMRI modality, the T1WI modality, and the DTI modality respectively and and calculate the total subjective opinion after fusing the three modalities according to Dempster's combination rule The total class credibility b after fusion k and the uncertainty u of the total classification result, and their calculation formulas are respectively:

[0033]

[0034] where is the class credibility provided by the fMRI modality, and u M is the uncertainty of the classification result provided by the fMRI modality; is the class credibility provided by the T1WI modality, and u T is the uncertainty of the classification result provided by the T1WI modality; is the class credibility provided by the DTI modality, and u D is the uncertainty of the classification result provided by the DTI modality;

[0035] C is the normalization factor, and its calculation formula is:

[0036]

[0037] S26. Calculate the total evidence e supporting the k-th class after fusing the three modalities k the total Dirichlet distribution concentration parameter α k and the total Dirichlet strength S, and their calculation formulas are respectively:

[0038]

[0039] S27. Calculate the predicted probability p that the sample belongs to the k-th class k , which is equal to the mean of the Dirichlet distribution, and its calculation formula is:

[0040]

[0041] The class corresponding to the maximum predicted probability is the final classification result.

[0042] Preferably, when using the dataset constructed in step S1 to train the autism classification model constructed in step S2 in step S3, the loss function is:

[0043]

[0044] where L(α M ), L(α T ) and L(α DThey respectively represent the independent loss functions of the fMRI modality, the T1WI modality, and the DTI modality; λ is a hyperparameter for adjusting the weights of the single-modal losses, where 0 < λ < 1; and L(α) represents the combined loss function after evidence fusion of the three modalities.

[0045] Preferably, the independent loss functions L(α M ), L(α T ), and L(α D ), as well as the calculation formulas for the combined loss function L(α) after fusion of the three modalities are respectively:

[0046]

[0047]

[0048] Among them, y k represents the true class label of the training sample assigned to the k-th class. When using one-hot encoding, if the training sample is assigned to the k-th class, then y k = 1, otherwise y k = 0; Ψ(·) is the Digamma function, which is monotonically increasing on (0, +∞); S M is the Dirichlet strength of the fMRI modality, is the Dirichlet distribution concentration parameter of the fMRI modality; S T is the Dirichlet strength of the T1WI modality, is the Dirichlet distribution concentration parameter of the T1WI modality; S D is the Dirichlet strength of the DTI modality, is the Dirichlet distribution concentration parameter of the DTI modality; S is the total Dirichlet strength after fusion of the three modalities, and α k is the total Dirichlet distribution concentration parameter after fusion of the three modalities.

[0049] On the other hand, the present invention provides an autism spectrum disorder classification system based on evidence decision fusion for implementing the above classification method, including:

[0050] A data acquisition module for acquiring multi-modal image data and corresponding autism category labels to construct a data set; and simultaneously for acquiring the image data of the multi-modal to be measured;

[0051] An autism classification model construction module for constructing an autism classification model based on evidence decision fusion;

[0052] A model training module for training the constructed autism classification model through the constructed data set, and adjusting the parameters of the autism classification model to the optimal through the error backpropagation algorithm;

[0053] An autism classification module for classifying autism spectrum disorder for the multi-modal image data to be measured obtained by the data acquisition module by using the autism classification model output by the model training module.

[0054] Preferably, the autism classification model construction module includes:

[0055] A data preprocessing sub-module for preprocessing the input multi-modal image data to obtain respective modal samples;

[0056] An evidence extraction sub-module for constructing neural networks corresponding to respective modalities and extracting evidence from the respective modal samples output by the data preprocessing sub-module;

[0057] A class credibility and classification uncertainty estimation sub-module for obtaining the evidence of respective modalities output by the evidence extraction sub-module and generating the class credibility and classification result uncertainty values corresponding to the respective modalities;

[0058] A decision fusion sub-module for calculating the total class credibility and total classification result uncertainty value after multi-modal fusion according to the output of the class credibility and classification uncertainty estimation sub-module by using the Dempster combination rule, and obtaining the final classification result based on this as the output of the autism classification model.

[0059] On the other hand, the present invention provides a non-transitory computer-readable storage medium, on which computer instructions are stored, and the computer instructions cause the computer to execute the above-mentioned autism spectrum disorder classification method based on evidence decision fusion.

[0060] On the other hand, the present invention provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus, and the processor calls the logic instructions in the memory to execute the above-mentioned autism spectrum disorder classification method based on evidence decision fusion.

[0061] On yet another aspect, the present invention provides a computer program product, the computer program product includes a computer program, the computer program is stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer executes the above-mentioned autism spectrum disorder classification method based on evidence decision fusion.

[0062] Advantages: (1) The autism spectrum disorder classification method and system based on evidence decision fusion of the present invention first adopts a dedicated neural network to learn modal features according to the unique characteristics of various modal data, and generates key evidence required for decision-making through the Softplus activation function. Then, the Dirichlet distribution is used to simulate the distribution of category probabilities, parameterize the evidence of various modalities, and then accurately model the category credibility and classification result uncertainty of various modalities in classification decision-making. Finally, considering the similarity and conflict between different modal evidences, multiple modal evidences are effectively fused according to the Dempster combination rule to construct a reliable and credible classification decision framework, which can not only improve the accuracy and robustness of autism classification, but also give an overall classification uncertainty assessment of the final decision, provide better model interpretability and decision credibility, and help clinicians formulate individualized diagnosis and treatment plans.

[0063] (2) The present invention innovatively proposes an evidence decision fusion method. Different from the existing decision-level multi-modal fusion methods, this method particularly focuses on the decision-making risks of each modality, comprehensively considers the similarity and conflict between different modal evidences, and effectively fuses multiple modal evidences based on the Dempster combination rule, thereby forming a credible and reliable classification decision. This fusion method can not only improve the accuracy and robustness of autism classification, but also give an overall classification uncertainty assessment of the final decision, provide better model interpretability and decision credibility.

[0064] (3) The evidence decision fusion method proposed by the present invention overcomes the instability that the fusion results are very different due to the slight change of the basic probability assignment function when using the decision rule based on the traditional Dempster-Shafer evidence theory for fusion, and the problem of producing results contrary to common sense when dealing with highly conflicting evidences. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a flowchart of the autism spectrum disorder classification method based on evidence decision fusion of the present invention;

[0066] Figure 2 is a structural schematic diagram of the autism classification model based on evidence decision fusion constructed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] To make the objectives, technical solutions and advantages of the present invention more clear, the following will, in conjunction with the accompanying drawings in the present invention, clearly and completely describe the technical solutions in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention, and they should not be construed as limitations on the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for the purpose of description and cannot be construed as indicating or implying relative importance.

[0068] The following will, in conjunction with Figure 1 - Figure 2 describe the autism spectrum disorder classification method and system based on evidence decision fusion provided by the present invention.

[0069] Embodiment 1: This embodiment provides an autism spectrum disorder classification method based on evidence decision fusion. As Figure 1 shown, the general steps of this method are as follows:

[0070] S1. Obtain multi-modal image data and corresponding autism category labels, and construct a data set;

[0071] S2. Construct an autism classification model based on evidence decision fusion;

[0072] S3. Use the data set constructed in step S1 to train the autism classification model constructed in step S2, and adjust the parameters of the autism classification model to the optimal through the error backpropagation algorithm;

[0073] S4. Use the trained autism classification model to classify the newly input multi-modal image data to be tested for autism spectrum disorder.

[0074] The following will, in conjunction with specific data and cases, elaborate on each step in detail.

[0075] Step S1 is mainly used to create an autism multi-modal data set.

[0076] The multi-modal data set created in this embodiment is sourced from 19 sites of the Autism Brain Imaging Data Exchange (ABIDE). These sites provided sample data from 521 ASD patients and 593 normal controls (NC). Each sample data includes T1-weighted imaging (T1WI) images, diffusion tensor imaging (DTI) images, resting-state functional magnetic resonance imaging (fMRI) images, and phenotypic data. Among them, the phenotypic data includes age, gender, site, diagnostic information, etc.

[0077] Step S2 is to construct an autism classification model based on evidence decision fusion. As Figure 2 shown, the model includes a data preprocessing sub-module, an evidence extraction sub-module, a class credibility and classification result uncertainty estimation sub-module, and a decision fusion sub-module.

[0078] The specific method for the model to process data is as follows:

[0079] S21. Data preprocessing: Preprocess the fMRI images, T1WI images, and DTI images to generate fMRI modality samples, T1WI modality samples, and DTI modality samples respectively; including:

[0080] Use the tools in the neuroimaging data analysis software package FSL (FfMRIB Software Library) to process the DTI data, and calculate the fractional anisotropy (FA) value of each voxel; then, perform affine transformation registration on the FA image and the T1WI image; then, non-linearly register the structural image to the standard space MNI152 template.

[0081] Use the brain imaging data processing and analysis tool DPABI (Data Processing & Analysis for Brain Imaging) to process the fMRI images. First, perform slice timing correction and head motion correction on the fMRI images; then register the T1-weighted image T1WI to the fMRI images, and normalize the fMRI images to the standard space MNI152, and resample them to 3×3×3mm 3 , and perform spatial smoothing processing using a 4mm full width at half maximum (FWHM) Gaussian kernel; use a band-pass filter with a frequency range of 0.01 - 0.1Hz to reduce the influence of high-frequency noise and low-frequency drift, and extract the time series of each subject from 116 brain regions.

[0082] S22. Extract evidence from the multi-modal imaging data;

[0083] Construct a deep neural network for each of the fMRI modality, T1WI modality, and DTI modality, perform feature learning on the preprocessed fMRI modality samples, T1WI modality samples, and DTI modality samples respectively, and generate evidence to support the k-th category through a fully connected layer with K neurons and a Softplus activation function where the superscript m represents the modality type, m ∈ {M, T, D}, m = M represents the fMRI modality, m = T represents the T1WI modality, and m = D represents the DTI modality.

[0084] In this embodiment, the subjects are divided into two categories: ASD patients and normal control group (NC), K = 2; for fMRI evidence extraction, a pre-trained MobileViT backbone is used to perform feature learning on the pre-processed fMRI samples, and an fMRI evidence supporting the k-th category is generated through a fully connected layer with 2 neurons and a Softplus activation function k = 1, 2, The larger the value, the higher the confidence of the fMRI modality in supporting the k-th category; for T1WI evidence extraction, a pre-trained MobileViT backbone is used to perform feature learning on the pre-processed T1WI samples, and a T1WI evidence supporting the k-th category is generated through a fully connected layer with 2 neurons and a Softplus activation function k = 1, 2, The larger the value, the higher the confidence of the T1WI modality in supporting the k-th category; for DTI evidence extraction, a pre-trained MobileViT backbone is used to perform feature learning on the pre-processed DTI samples, and a DTI evidence supporting the k-th category is generated through a fully connected layer with 2 neurons and a Softplus activation function k = 1, 2, The larger the value, the higher the confidence of the DTI modality in supporting the k-th category.

[0085] S23. Use the evidence of each modality supporting the k-th category extracted to generate the autism category credibility provided by the corresponding modality and the classification result uncertainty u m , and the specific steps are as follows:

[0086] (1) Convert the evidence of each modality supporting the k-th category extracted into the concentration parameter of the Dirichlet distribution of the corresponding modality The calculation formula is:

[0087]

[0088] where and the superscript m in represents the modality category, m ∈ {M, T, D}, m = M represents the fMRI modality, m = T represents the T1WI modality, m = D represents the DTI modality, k = 1, 2;

[0089] (2) For each sample in the binary classification, assign the 2 autism category credibilities of each modality and the classification result uncertainty u m , and satisfy:

[0090]

[0091] Among them, u m ≥0, and u m The superscript m in represents the modal category, m ∈ {M, T, D}, k = 1, 2;

[0092] Calculate the autism category credibility provided by each modality and the classification result uncertainty u The calculation formulas are respectively: m ,

[0093]

[0094]

[0095] Among them, S m is the Dirichlet strength, S m The superscript m in represents the modal category, m ∈ {M, T, D}, k = 1, 2.

[0096] S24. Using the autism category credibility provided by each modality and the classification result uncertainty u m , according to the Dempster combination rule, calculate the total category credibility b k and the total classification result uncertainty u after multi-modal fusion, and output the final classification result. The specific steps are as follows:

[0097] (1) Using the autism category credibility provided by each modality and the classification result uncertainty u m , respectively obtain the subjective opinions of the fMRI modality, T1WI modality, and DTI modality And according to the Dempster combination rule, calculate the total subjective opinion after fusing the three modalities The total category credibility b k and the total classification result uncertainty u after fusion. The calculation formulas are respectively:

[0098]

[0099] Among them, is the category credibility provided by the fMRI modality, u M is the classification result uncertainty provided by the fMRI modality; is the category credibility provided by the T1WI modality, u T is the classification result uncertainty provided by the T1WI modality; The class credibility provided for the DTI modality, u D The classification result uncertainty provided for the DTI modality;

[0100] C is the normalization factor, and its calculation formula is:

[0101]

[0102] (2) Calculate the total evidence e that supports the k-th category after fusing the three modalities k , the total Dirichlet distribution concentration parameter α k and the total Dirichlet strength S, and their calculation formulas are respectively:

[0103]

[0104]

[0105] (3) Calculate the predicted probability p that the sample belongs to the k-th category k , which is equal to the mean of the Dirichlet distribution, and its calculation formula is:

[0106]

[0107] where k = 1, 2, and the category corresponding to the maximum predicted probability is the final classification result.

[0108] Step S3 is to train the constructed autism classification model, and adjust the parameters of the autism classification model to the optimal through the error backpropagation algorithm. The loss function of each training sample during the model training process is defined as:

[0109]

[0110] where L(α M ), L(α T ) and L(α D ) respectively represent the independent loss functions of the fMRI modality, the T1WI modality and the DTI modality; λ is a hyperparameter that adjusts the single-modal loss weight, 0 < λ < 1; L(α) represents the comprehensive loss function after evidence fusion of the three modalities;

[0111] L(α M ), L(α T ), L(α D ) and L(α)'s calculation formulas are respectively:

[0112]

[0113] where y kIndicates that the training sample is assigned the true class label of the k-th class. When using one-hot encoding, if the training sample is assigned to the k-th class, then y k = 1, otherwise y k = 0; Ψ(·) is the Digamma function, which is monotonically increasing on (0, +∞); S M is the Dirichlet intensity of the fMRI modality, and is the concentration parameter of the Dirichlet distribution of the fMRI modality; S T is the Dirichlet intensity of the T1WI modality, and is the concentration parameter of the Dirichlet distribution of the T1WI modality; S D is the Dirichlet intensity of the DTI modality, and is the concentration parameter of the Dirichlet distribution of the DTI modality; S is the total Dirichlet intensity after fusing the three modalities, and α k is the total concentration parameter of the Dirichlet distribution after fusing the three modalities.

[0114] During the model training process of this embodiment, the batch size is set to 64, the Adam optimizer is used to update the model parameters, the initial learning rate is set to 0.001, and the learning rate is adjusted by means of sinusoidal oscillation.

[0115] Step S4 is to use the trained autism classification model to classify the newly input multi-modal image data to be tested for autism spectrum disorder.

[0116] This method comprehensively utilizes the key evidence of three modalities: T1-weighted imaging, diffusion tensor imaging, and functional magnetic resonance imaging, and constructs a reliable and credible classification decision framework according to the Dempster combination rule. It can not only improve the accuracy and robustness of autism classification, but also give an overall classification uncertainty assessment of the final decision, providing better model interpretability and decision credibility.

[0117] Embodiment 2: This embodiment provides an autism spectrum disorder classification system based on evidence decision fusion for implementing the classification method of Embodiment 1, including:

[0118] A data acquisition module, which is used to obtain multi-modal image data and corresponding autism class labels to construct a data set; and is also used to obtain multi-modal image data to be tested. The multi-modal image data includes functional magnetic resonance imaging (fMRI) images, T1-weighted imaging (T1WI) images, and diffusion tensor imaging (DTI) images.

[0119] An autism classification model construction module, which is used to construct an autism classification model based on evidence decision fusion; the model includes:

[0120] A data preprocessing sub-module for preprocessing the input fMRI images, T1WI images, and DTI images to obtain fMRI samples, T1WI samples, and DTI samples respectively.

[0121] An evidence extraction sub-module, including an fMRI evidence extraction sub-module, a T1WI evidence extraction sub-module, and a DTI evidence extraction sub-module;

[0122] The fMRI evidence extraction sub-module uses a pre-trained MobileViT backbone to perform feature learning on the preprocessed fMRI samples, and generates fMRI evidence supporting the k-th category through a fully connected layer with 2 neurons and a Softplus activation function k = 1, 2, The larger the value, the higher the confidence of the fMRI modality in supporting the k-th category;

[0123] The T1WI evidence extraction sub-module uses a pre-trained MobileViT backbone to perform feature learning on the preprocessed T1WI samples, and generates T1WI evidence supporting the k-th category through a fully connected layer with 2 neurons and a Softplus activation function k = 1, 2, The larger the value, the higher the confidence of the T1WI modality in supporting the k-th category;

[0124] The DTI evidence extraction sub-module uses a pre-trained MobileViT backbone to perform feature learning on the preprocessed DTI samples, and generates DTI evidence supporting the k-th category through a fully connected layer with 2 neurons and a Softplus activation function k = 1, 2, The larger the value, the higher the confidence of the DTI modality in supporting the k-th category.

[0125] A category confidence and classification uncertainty estimation sub-module for using the evidence supporting the k-th category extracted from each modality To generate the autism category confidence provided by the corresponding modality And the classification result uncertainty u m ;

[0126] A decision fusion sub-module for using the autism category confidence provided by each modality And the classification result uncertainty u m , according to the Dempster combination rule, calculate the autism category confidence b after multi-modal fusion k And the overall classification result uncertainty u, and output the final classification result.

[0127] A model training module, which is used to train the constructed autism classification model through the constructed data set, and adjust the parameters of the autism classification model to the optimal through the error backpropagation algorithm.

[0128] An autism classification module, which is used to use the autism classification model output by the model training module to classify the multi-modal image data to be measured obtained by the data acquisition module for autism spectrum disorder.

[0129] Embodiment 3: This embodiment provides a non-transitory computer-readable storage medium, on which computer instructions are stored. The computer instructions cause the computer to execute an autism spectrum disorder classification method based on evidence decision fusion. The method includes the following steps:

[0130] S1. Obtain multi-modal image data and corresponding autism category labels, and construct a data set;

[0131] S2. Construct an autism classification model based on evidence decision fusion. The autism classification model is constructed as follows:

[0132] Using the multi-modal image data as input, extract evidence from each modal image data respectively;

[0133] Then, according to the evidence extracted from each modality, generate the category credibility and classification result uncertainty value corresponding to each modality;

[0134] Again, according to the category credibility and classification result uncertainty value of each modality, use the Dempster combination rule to calculate the total category credibility and total classification result uncertainty value after multi-modal fusion, and obtain the final classification result accordingly, as the output of the autism classification model;

[0135] S3. Use the data set constructed in step S1 to train the autism classification model constructed in step S2, and adjust the parameters of the autism classification model to the optimal through the error backpropagation algorithm;

[0136] S4. Use the trained autism classification model to classify the newly input multi-modal image data to be measured for autism spectrum disorder.

[0137] Embodiment 4: This embodiment provides an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus. The processor can call the logical instructions in the memory to execute an autism spectrum disorder classification method based on evidence decision fusion. The method includes the following steps:

[0138] S1. Obtain multi-modal image data and corresponding autism category labels, and construct a data set;

[0139] S2. Construct an autism classification model based on evidence decision fusion, and the autism classification model is constructed as follows:

[0140] Take the multi-modal image data as input, and extract evidence from each modal image data respectively;

[0141] Then, according to the evidence of each modality extracted, generate the category credibility and classification result uncertainty value of the corresponding modality;

[0142] Furthermore, according to the category credibility and classification result uncertainty value of each modality, use the Dempster combination rule to calculate the total category credibility and total classification result uncertainty value after multi-modal fusion, and obtain the final classification result based on this, as the output of the autism classification model;

[0143] S3. Use the data set constructed in step S1 to train the autism classification model constructed in step S2, and adjust the parameters of the autism classification model to the optimal through the error backpropagation algorithm;

[0144] S4. Use the trained autism classification model to classify the newly input multi-modal image data to be tested for autism spectrum disorder.

[0145] In addition, when the logical instructions in the above-mentioned memory can be implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0146] Embodiment 5: What this embodiment provides is a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the autism spectrum disorder classification method based on evidence decision fusion. The method includes the following steps:

[0147] S1. Obtain multi-modal image data and corresponding autism category labels, and construct a data set;

[0148] S2. Construct an autism classification model based on evidence decision fusion, and this autism classification model is constructed as follows:

[0149] Use the multi-modal image data as input, and extract evidence from each modal image data respectively;

[0150] Then, according to the evidence of each modality extracted, generate the category credibility of the corresponding modality and the classification result uncertainty value;

[0151] Again, according to the category credibility of each modality and the classification result uncertainty value, use the Dempster combination rule to calculate the total category credibility and the total classification result uncertainty value after multi-modal fusion, and obtain the final classification result based on this, as the output of the autism classification model;

[0152] S3. Use the data set constructed in step S1 to train the autism classification model constructed in step S2, and adjust the parameters of the autism classification model to the optimal through the error backpropagation algorithm;

[0153] S4. Use the trained autism classification model to classify the newly input multi-modal image data to be tested for autism spectrum disorder.

[0154] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0155] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence, or the part that contributes to the prior art can be embodied in the form of a software product, and this computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for classifying autism spectrum disorders based on evidence-based decision fusion, characterized in that: include: S1. Obtain multimodal imaging data and corresponding autism category labels to construct a dataset; S2. Construct an autism classification model based on evidence-based decision fusion. The autism classification model is constructed as follows: Taking multi-modal image data as input, evidence is extracted from each modality of image data; Then, based on the evidence extracted from each modality, the category credibility and classification result uncertainty value of the corresponding modality are generated; Then, according to the category credibility and classification result uncertainty value of each modality, the total category credibility and total classification result uncertainty value after multimodal fusion are calculated using the Dempster combination rule, and the final classification result is obtained as the output of the autism classification model. S3, using the data set constructed in step S1, training the autism classification model constructed in step S2, and adjusting the parameters of the autism classification model to the optimum through an error back propagation algorithm; S4. Using the trained autism classification model, classify the newly input multimodal image data to be tested into autism spectrum disorders.

2. The autism spectrum disorder classification method based on evidence-based decision fusion according to claim 1, characterized in that: The multimodal image data in step S1 and step S4 include fMRI images, T1WI images and DTI images.

3. The autism spectrum disorder classification method based on evidence-based decision fusion according to claim 2, characterized in that: When training an autism classification model or classifying autism spectrum disorder on newly input multi-modal imaging data to be tested, the fMRI images, T1WI images and DTI images are first preprocessed to generate fMRI modality samples, T1WI modality samples and DTI modality samples, respectively.

4. The autism spectrum disorder classification method based on evidence-based decision fusion according to claim 3, characterized in that: In step S2, evidence is extracted from the image data of each modality. The specific method is as follows: S21. Build a deep neural network for each of the fMRI modality, T1WI modality, and DTI modality, perform feature learning on the preprocessed fMRI modality samples, T1WI modality samples, and DTI modality samples, and generate evidence supporting the kth category through a fully connected layer containing K neurons and a Softplus activation function. The superscript m indicates the modality type, m∈{M,T,D}, m=M indicates fMRI modality, m=T indicates T1WI modality, and m=D indicates DTI modality; the subscript k indicates the index number of the autism category, k=1,2,…,K, where K is the number of autism categories; The larger the value of , the higher the credibility of the modality in supporting the kth category.

5. The autism spectrum disorder classification method based on evidence-based decision fusion according to claim 4, characterized in that: In step S2, based on the extracted evidence of each modality, the category credibility and classification result uncertainty value of the corresponding modality are generated. The specific method is as follows: S22. Extract the evidence of each modality to support the kth category Converted to the Dirichlet distribution concentration parameter of the corresponding mode The calculation formula is: S23. For each sample in the K categories, assign K autism category credibility of each modality and the classification result uncertainty u m , and satisfy: in, S24. Evidence supporting the kth category based on each modality Calculate the category credibility provided by the corresponding modality and the classification result uncertainty u m , the calculation formulas are: Among them, S m is the Dirichlet intensity of the corresponding mode, 6. The autism spectrum disorder classification method based on evidence-based decision fusion according to claim 5, characterized in that: In step S2, according to the category credibility and classification result uncertainty value of each modality, the total category credibility and total classification result uncertainty value after multimodal fusion are calculated using the Dempster combination rule, and the final classification result is obtained accordingly; the specific method is: S25. Using the category credibility provided by each modality and the classification result uncertainty u m , respectively, to obtain subjective opinions of fMRI modality, T1WI modality and DTI modality and And according to Dempster's combination rule, the total subjective opinion after the fusion of the three modalities is calculated The total category credibility after fusion b k And the total classification result uncertainty u, the calculation formulas are: in, The class confidence provided for the fMRI modality, u M uncertainty in the classification results provided for the fMRI modality; The class confidence provided for the T1WI modality, u T Uncertainty of classification results provided for T1WI modality; The class confidence provided for the DTI modality, u D The uncertainty of the classification results provided for the DTI modality; C is the normalization factor, calculated as: S26. Calculate the total evidence e supporting the kth category after the three modal fusion k , total Dirichlet distribution concentration parameter α k And the total Dirichlet intensity S, calculated by: S27. Calculate the predicted probability p that the sample belongs to the kth category k , which is equal to the mean of the Dirichlet distribution, is calculated as: The category corresponding to the maximum predicted probability is the final classification result.

7. The autism spectrum disorder classification method based on evidence-based decision fusion according to claim 4, characterized in that: In step S3, when the data set constructed in step S1 is used to train the autism classification model constructed in step S2, the loss function is: Among them, L(α M )、L(α T ) and L(α D ) represent the independent loss functions of fMRI, T1WI and DTI respectively; λ is the hyperparameter for adjusting the single modality loss weight, 0<λ<1; L(α) represents the comprehensive loss function of the three modalities after evidence fusion.

8. The autism spectrum disorder classification method based on evidence-based decision fusion according to claim 7, characterized in that: Independent loss functions L(α M )、L(α T ) and L(α D ), and the calculation formulas of the comprehensive loss function L(α) after the three modes are fused are: Among them, y k Indicates that the training sample is assigned the true category label of the kth category. When one-hot encoding is used, if the training sample is assigned the kth category, then y k =1, otherwise y k =0; Ψ(·) is the Digamma function, which increases monotonically on (0, +∞); S M is the Dirichlet intensity of the fMRI modality, is the Dirichlet distribution concentration parameter of the fMRI modality; S T is the Dirichlet intensity of the T1WI mode, is the Dirichlet distribution concentration parameter of T1WI mode; S D is the Dirichlet intensity of the DTI mode, is the Dirichlet distribution concentration parameter of the DTI modality; S is the total Dirichlet intensity after the fusion of the three modalities, α k is the total Dirichlet distribution concentration parameter after the fusion of the three modes.

9. An autism spectrum disorder classification system based on evidence-decision fusion for implementing any one of the classification methods of claims 1-8, characterized in that: include: The data acquisition module is used to obtain multimodal image data and corresponding autism category labels to construct a data set; it is also used to obtain multimodal image data to be tested; Autism classification model building module, used to build an autism classification model based on evidence-based decision fusion; A model training module is used to train the constructed autism classification model through the constructed data set, and adjust the parameters of the autism classification model to the optimal value through the error back propagation algorithm; The autism classification module is used to classify autism spectrum disorders on the multimodal image data to be tested obtained by the data acquisition module using the autism classification model output by the model training module.

10. The autism spectrum disorder classification system based on evidence-based decision fusion according to claim 9, characterized in that: The autism classification model constructed based on evidence-based decision fusion includes: The data preprocessing submodule is used to preprocess the input multi-modal image data to obtain samples of each modality; The evidence extraction submodule is used to construct a neural network corresponding to each modality and extract evidence from each modality sample output by the data preprocessing submodule; The category credibility and classification uncertainty estimation submodule is used to obtain the evidence of each modality output by the evidence extraction submodule and generate the category credibility and classification result uncertainty value of the corresponding modality; The decision fusion submodule is used to calculate the total category credibility and total classification result uncertainty value after multimodal fusion according to the output of the category credibility and classification uncertainty estimation submodule using the Dempster combination rule, and obtain the final classification result as the output of the autism classification model.

Images