A classification method for multimodal image missing completion based on tensor network model

Abstract

The present invention proposes a method for missing multimodal image completion based on a tensor network model. First, features are extracted from the multimodal image to obtain an incomplete feature tensor, and then the incomplete high-order tensor is decomposed into a series of low-order kernel tensors through tensor ring decomposition. At the same time, low-rank constraints are added to the rank modulus and dimension modulus expansion of the kernel tensor and the nuclear norm regularization is performed on each modulus expansion. Then, the inverse process of tensor ring decomposition is used to restore the complete tensor, and finally it is put into the classifier for classification. The algorithm was verified on BraTS2018 and BraTS2019 data. The results show that the method proposed by the present invention is superior to the traditional method and has better completion and classification effects.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and technical analysis, and specifically relates to a classification method for multimodal image missing completion based on a tensor network model. Background Art

[0002] In the application of medical image processing, affected by factors such as the selection of imaging device parameters and the macroscopic movement of the subject, the acquired multimodal images often have artifacts in one or more modalities. A large degree of artifacts may reduce the image quality, cover up or even appear false lesions, and cause misdiagnosis. When extracting features from multimodal images containing artifacts and training model classification, the modality with artifacts needs to be removed, resulting in waste and loss of feature data, which is unacceptable for medical image data with small sample imbalance characteristics. Re-collecting data is too costly and not feasible. At present, the more ideal solution is to estimate missing values ​​through interpolation to complete the missing feature data. In this way, data waste is avoided and the time and economic cost required for re-collecting data are saved.

[0003] In the field of data completion, many techniques and methods for estimating missing values ​​have been proposed, including: zero filling method, expectation maximization algorithm, singular value decomposition and accurate low-rank tensor completion algorithm, etc. These methods are mainly designed for random missing in low-dimensional data, and the effect is not ideal when facing multimodal high-dimensional data. Summary of the invention

[0004] In order to avoid data waste and improve classification accuracy so as to better realize clinical applications, the present invention innovatively adopts a completion method based on a tensor network model to solve a series of problems caused by the missing multimodal images. Traditional completion methods often have problems such as general effect or excessive computation when facing multimodal high-dimensional data. The present invention can capture high-dimensional structural information that cannot be obtained by traditional completion methods by using tensor ring decomposition and low-rank constraints on the core tensor. At the same time, it also reduces the problem of excessive computation caused by nuclear norm regularization of tensor ring decomposition on the original tensor, making classification more accurate and efficient.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A classification method for missing multimodal image completion based on a tensor network model. This method uses statistical and texture features extracted from multimodal images to form a feature tensor, uses tensor ring low-rank factor decomposition to capture the potential structure of high-dimensional space and the low-rank structure of global data, completes the data with modal missing, and finally classifies it. The specific steps are as follows:

[0007] (I) Data preprocessing: First, multimodal data is collected, and then the first-order statistical and second-order texture features are extracted for each modality. The extracted features are concatenated in sequence to obtain the feature tensor Where N is the tensor order, I1,I2,...I N is the dimension size of the tensor;

[0008] (ii) Random missing data setting: samples are selected from each modality according to a certain ratio, and the corresponding modal features of the selected samples are processed as missing to obtain an incomplete tensor T;

[0009] (III) Completing the incomplete tensor T through low-rank factorization of the tensor ring:

[0010] First, input the incomplete tensor T and initialize it TR rank, and in, Obey the distribution N~(0,1), The TR rank is a 1×3 vector; the variables are iteratively updated in the following order using the ADMM algorithm: and The process is as follows:

[0011] pass renew

[0012] pass renew

[0013] pass renew

[0014] pass renew

[0015] in, is the i-th core tensor obtained by decomposition, is the TR factor, fold i represents the operator symbol for folding the tensor according to the i-th mode, μ and λ are penalty parameters, represents the expansion matrix of the second mode of the other core tensors except the nth core tensor, express The transpose of is the identity matrix, It is introduced to simplify the optimization process. Auxiliary variables, is a variable A collection of is the Lagrange multiplier, is the recovered complete tensor, X <n>< / n> Representation variables The expansion matrix of the nth mode, Ψ(*) represents the inverse process of tensor ring decomposition, Representation variables The expansion matrix of the second mode of Representation variables The expansion matrix of the second mode of is the set of all N core tensors, D β (·) is the singular value threshold (SVT) operation, P Ω (*) represents all known elements of the tensor, Ω represents the set of known element subscripts, represents a set of unknown element subscripts, For all A collection of;

[0016] Calculate the two iterations before and after The error between When the error is less than the threshold tol or the maximum number of iterations k is reached max , terminate the iteration and output the completed tensor and TR factor

[0017] (IV) The completed tensor The data are sent to the support vector machine for classification to obtain the classification result. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of a classification method for missing multimodal completion based on a tensor network model of the present invention. DETAILED DESCRIPTION

[0019] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and examples.

[0020] like Figure 1 As shown, the specific structure and method of the classification method for multimodal image missing completion based on the tensor network model are shown as follows:

[0021] (I) Extract first-order statistical and second-order texture features for each modality of each sample in the dataset. The first-order statistical features include volume, solidity, and eccentricity; the second-order texture features include 3 global texture features, 9 features based on gray-level co-occurrence matrix, 13 features based on gray-level run matrix, 5 features based on neighborhood gray-level difference matrix, and 13 features based on gray-level region size matrix. The features of the four modalities are horizontally spliced, and then all feature vectors are arranged vertically in order according to the patient order to obtain the final feature tensor. Where I1 is the number of samples and I2 is the number of features.

[0022] (ii) Data random missing setting: Samples are randomly selected at a ratio of 40% for each modality, and the corresponding modal features of the selected samples are missing to obtain an incomplete feature tensor T;

[0023] (III) Completing the incomplete tensor T through low-rank factorization of the tensor ring:

[0024] First, input the incomplete tensor T and initialize it TR rank, and in, Obey the distribution N~(0,1), The TR rank is a 1×3 vector; the variables are iteratively updated in the following order using the ADMM algorithm: and The process is as follows:

[0025] pass renew

[0026] pass renew

[0027] pass renew

[0028] pass renew

[0029] in, is the i-th core tensor obtained by decomposition, is the TR factor, fold i represents the operator symbol for folding the tensor according to the i-th mode, μ and λ are penalty parameters, represents the expansion matrix of the second mode of the other core tensors except the nth core tensor, express The transpose of is the identity matrix, It is introduced to simplify the optimization process. Auxiliary variables, is a variable A collection of is the Lagrange multiplier, is the recovered complete tensor, X ＜n> Representation variables The expansion matrix of the nth mode, Ψ(*) represents the inverse process of tensor ring decomposition, Representation variables The expansion matrix of the second mode of Representation variables The expansion matrix of the second mode of is the set of all N core tensors, D β (·) is the singular value threshold (SVT) operation, P Ω (*) represents all known elements of the tensor, Ω represents the set of known element subscripts, represents a set of unknown element subscripts, For all A collection of;

[0030] Calculate the two iterations before and after The error between When the error is less than the threshold tol or the maximum number of iterations k is reached max , terminate the iteration and output the completed tensor and TR factor

[0031] (IV) The completed tensor The data are sent to the support vector machine for classification to obtain the classification result.

[0032] The effect of the present invention can be further illustrated by experimental results:

[0033] To verify the performance of the present invention, BraTS2018 and BraTS2019 data sets were used. Among them, BraTS2018 and BraTS2019 data sets contain multimodal data of 285 and 335 patients, respectively. Each data set is divided into a training set and a test set, and the support vector machine SVM is used for classification. The average accuracy of 5 times is taken as the result, and compared with the traditional multimodal image missing completion classification method. The experimental results are shown in Table 1. Among them, "complete data" means the result of classification using complete feature data without missing processing, the zero filling method, the expectation maximization algorithm, the singular value decomposition algorithm, and the accurate low-rank tensor completion algorithm are all the results of classification after completion using traditional methods, and the tensor ring low-rank factor completion algorithm is the result of classification of the completion method used in the present invention. The experimental results show that the classification method based on the tensor ring low-rank factor completion algorithm has better completion and classification effects than the traditional method, can better restore missing data, and classify, and when the data sample is reduced, the classification method based on the tensor ring low-rank factor completion algorithm has better stability.

[0034] Table 1

[0035]

[0036]

Claims

1. A classification method for multimodal image missing completion based on a tensor network model. This method uses statistical and texture features extracted from multimodal images to form a feature tensor. Use tensor ring low-rank factor decomposition to capture the potential structure of high-dimensional space and the low-rank structure of global data, complete the data with missing modes, and finally classify them. The specific steps are as follows: (I) Data preprocessing: First, multimodal data is collected, and then the first-order statistical and second-order texture features are extracted for each modality. The extracted features are concatenated in sequence to obtain the feature tensor Where N is the tensor order, I1,I2,...I N is the dimension size of the tensor; (ii) Random missing data setting: samples are selected from each modality according to a certain ratio, and the corresponding modal features of the selected samples are processed as missing to obtain an incomplete tensor T; (III) Completing the incomplete tensor T through low-rank factorization of the tensor ring: First, input the incomplete tensor T and initialize it TR rank, and in, Obey the distribution N~(0,1), The TR rank is a 1×3 vector; the variables are iteratively updated in the following order using the ADMM algorithm: and The process is as follows: pass renew pass renew pass renew pass renew in, is the i-th core tensor obtained by decomposition, is the TR factor, fold i represents the operator symbol for folding the tensor according to the i-th mode, μ and λ are penalty parameters, represents the expansion matrix of the second mode of the other core tensors except the nth core tensor, express The transpose of is the identity matrix, It is introduced to simplify the optimization process. Auxiliary variables, is a variable A collection of is the Lagrange multiplier, is the recovered complete tensor, X ＜n＞ Representation variables The expansion matrix of the nth mode, Ψ(*) represents the inverse process of tensor ring decomposition, Representation variables The expansion matrix of the second mode of Representation variables The expansion matrix of the second mode of is the set of all N core tensors, D β (·) is the singular value threshold (SVT) operation, P Ω (*) represents all known elements of the tensor, Ω represents the set of known element subscripts, represents a set of unknown element subscripts, For all A collection of; Calculate the two iterations before and after The error between When the error is less than the threshold tol or the maximum number of iterations k is reached max , terminate the iteration and output the completed tensor and TR factor (IV) The completed tensor The data are sent to the support vector machine for classification to obtain the classification result.

Images