Method for constructing nerve disease positioning, qualification and evaluation by applying multi-mode MRI image data

Abstract

The invention relates to the technical field of medical image processing, and discloses a method for constructing nerve disease positioning, qualification and evaluation by applying multi-modal magnetic resonance imaging (MRI) image data, which comprises the following steps of: standardizing the multi-modal MRI image data to enable each modal image to have the same spatial resolution; unifying the MRI images of all the modes into a high-order tensor, inputting the high-order tensor into a Tucker decomposition model, outputting a low-dimensional factor matrix and a core tensor, and extracting interaction features between the modes; the extracted interaction features are used for constructing an energy function, a difference term and a regular term are input, the energy function is minimized through a variational method, and an optimized spatial transformation function is output; the feature points are mapped to a unified space through a transformation function, a feature graph structure is constructed, the similarity between nodes is input, optimized graph structure information is output, and image fusion is assisted; by adopting the technical scheme of multi-modal image standardization and tensor decomposition, interaction information between modals is effectively extracted, and the problems of information loss and local feature neglect are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly to a method for constructing a method for localizing, qualitatively diagnosing, and evaluating neurological diseases using multi-modal MRI image data. Background Art

[0002] In existing multi-modal MRI image registration techniques, most methods rely on the processing of single-modal images and lack in-depth exploration of the interaction information between different modalities. Although these methods can achieve registration in some cases, due to the lack of full consideration of the mutual influence between modalities, they usually have difficulty dealing with complex medical image data, resulting in insufficient accuracy of image registration, especially errors often occur when dealing with complex structures or details. This limitation makes it difficult for the existing technology to meet the requirements of high-precision, multi-modal image analysis.

[0003] In addition, common transformation function optimization methods in the existing technology mostly rely on simple image comparison, such as minimizing the difference term, while ignoring the local deformation or non-linear changes that may occur during the image registration process. Traditional transformation methods are mostly based on affine transformation, and such methods are difficult to capture complex non-linear deformations in images, resulting in the quality of the registration results not being fully guaranteed. Therefore, when dealing with multi-modal images with large deformations or complex shapes, the existing technology is prone to registration errors, affecting the accuracy of the final results.

[0004] In the extraction and registration of feature points in the existing technology, there is often a lack of effective modeling of the relationship between local feature points. Although some methods attempt to use key point detection algorithms, these methods mainly focus on the local features of a single modality and fail to effectively utilize the deep-level associations between different modality images, resulting in inaccurate feature point matching and thus affecting the overall effect of the registration results. Therefore, there are certain blind spots in the extraction and fusion of feature points of different modality images in the existing technology, and the local details between images cannot be fully reflected.

[0005] In addition, in the evaluation of image registration results, traditional quality evaluation methods mostly rely on a single index or qualitative analysis. Although common evaluation methods such as mutual information (MI) or structural similarity (SSIM) can measure the image quality to a certain extent, they usually have difficulty fully considering the diversity of different modality images and their impact on the final fusion effect. Therefore, there are deficiencies in the comprehensive evaluation of image quality in the existing technology, and the reliability and practicality of the final fused image in actual applications cannot be fully ensured.

[0006] Therefore, the present invention proposes a method for constructing a method for localizing, qualitatively diagnosing, and evaluating neurological diseases using multi-modal MRI image data to solve the deficiencies of the existing technology. Summary of the Invention

[0007] In view of the deficiencies of the prior art, the present invention provides a method for constructing the localization, qualitative analysis, and evaluation of neurological diseases using multi-modal MRI image data, which solves the problems of insufficient information fusion, low registration accuracy, and insufficient image quality evaluation in the multi-modal MRI image registration process of the prior art.

[0008] To achieve the above objectives, the present invention is realized through the following technical solutions: A method for constructing the localization, qualitative analysis, and evaluation of neurological diseases using multi-modal MRI image data, comprising the following steps:

[0009] S1. Standardize the multi-modal MRI image data so that each modal image has the same spatial resolution;

[0010] S2. Unify each modal MRI image into a high-order tensor, input it into the Tucker decomposition model, output the low-dimensional factor matrix and the core tensor, and extract the interaction features between modalities;

[0011] S3. Use the extracted interaction features to construct an energy function, input the difference term and the regularization term, and minimize the energy function through the variational method to output the optimized spatial transformation function;

[0012] S4. Use the transformation function to map the feature points to a unified space, construct a feature map structure, input the similarity between nodes, and output the optimized graph structure information to assist image fusion;

[0013] S5. Register the images according to the optimized transformation function to generate the final fused image;

[0014] S6. Evaluate the registration and fusion results.

[0015] Preferably, the tensor decomposition adopts the Tucker decomposition model, and the core tensor and factor matrix obtained by decomposition are used to represent the interaction information between multi-modal images and realize the dimensionality reduction processing of modal data.

[0016] Preferably, in the variational method optimization step, the registration result is optimized by constructing an energy function and minimizing its value. The energy function includes the difference term between the source image and the target image and the regularization term of the gradient of the transformation function.

[0017] Preferably, in the graph theory optimization step, the similarity between the feature points of the modal images is represented by constructing the adjacency matrix of the graph, and the structure of the graph is calculated using the Laplacian matrix to optimize the registration path of the modal images.

[0018] Preferably, the registration step includes using the transformation function to transform the source image, generating the aligned image and performing subsequent processing.

[0019] Preferably, the step of fusing images includes fusing the registered multimodal images using the weighted average method, where the image weight of each modality is determined by the importance of the modality and the image quality.

[0020] Preferably, the evaluation step includes evaluating the registration accuracy using indicators such as the Dice coefficient and the Hausdorff distance, and verifying the quality of the fused image.

[0021] The present invention also provides a system for localizing, qualitatively analyzing, and evaluating neurological diseases using multimodal MRI image data, including the following modules:

[0022] Data processing module: used to perform standardization, resampling, and noise removal processing on the input multimodal MRI image data;

[0023] Tensor decomposition module: used to perform high-order tensor decomposition on the standardized multimodal images, extract the interaction information between modalities, and perform dimensionality reduction processing;

[0024] Optimization module: used to optimize the registration result by variational method, minimize the registration error, and calculate the optimal transformation function;

[0025] Graph theory module: used to construct a graph structure to represent the relationship between the feature points of the modality images, optimize the connectivity of the graph using the Laplacian matrix, and calculate the shortest registration path between the modality images;

[0026] Fusion module: used to perform weighted average fusion on the registered multimodal images to generate the final image;

[0027] Evaluation module: used to evaluate the quality of the registration and fusion results, and calculate indicators such as the Dice coefficient and the Hausdorff distance.

[0028] Preferably, the data processing module uses the B-spline interpolation method for image resampling and uses the variational denoising algorithm to remove noise from the images.

[0029] Preferably, the optimization module includes a variational method optimization algorithm for minimizing the registration error and optimizing the transformation function by the gradient descent method.

[0030] The present invention provides a method for localizing, qualitatively analyzing, and evaluating neurological diseases using multimodal MRI image data. It has the following beneficial effects:

[0031] 1. The present invention adopts the technical solutions of multimodal image standardization and tensor decomposition, achieving the purpose of effectively extracting the interaction information between modalities. Compared with the single-modal processing solution in the prior art, this method solves the problems of information loss and local feature neglect, enabling more comprehensive analysis and understanding of different modal images.

[0032] 2. The technical solution of the present invention uses the variational method to optimize the energy function, achieving the purpose of improving the registration accuracy by minimizing the registration error. Compared with traditional registration methods, it can reduce the registration error between images, avoid the problem of local optimal solutions, and ensure that the registration process is more accurate and reliable.

[0033] 3. The present invention constructs a graph structure of the feature point relationship, achieving the purpose of comprehensively capturing the mutual relationship of feature points between modalities. This innovative solution solves the deficiency of the lack of combination of local features in the prior art and improves the overall registration accuracy of multi-modal images by optimizing the relationship between nodes.

[0034] 4. The present invention introduces a technical solution that combines quantitative and qualitative evaluation means, achieving the purpose of comprehensively evaluating the quality of the fused image. This method is more objective and detailed compared with the traditional single evaluation criterion, ensuring that the final fused image meets the clinical standards and improving the accuracy and practicality of medical data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flowchart of the method steps of the present invention;

[0036] Figure 2 is a schematic diagram of the system module framework of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0038] Please refer to the attached Figure 1 , the embodiments of the present invention provide a method for constructing neuro-disease localization, characterization, and evaluation using multi-modal MRI image data, including the following steps:

[0039] S1. Standardize the multi-modal MRI image data so that each modal image has the same spatial resolution;

[0040] In this embodiment, first, the multi-modal MRI image data needs to be standardized so that the images of different modalities are consistent in spatial resolution. The standardization step is crucial because MRI images of different modalities usually have large differences in spatial resolution due to factors such as scanning protocols, hardware differences, and scanning parameters. Without standardization, there will be a large registration error between modalities, which will affect the subsequent registration and analysis results.

[0041] In general, MRI images of different modalities may vary in spatial size and resolution, which makes it impossible to perform effective registration in the same coordinate system when conducting comparative analysis on different-modal images. Therefore, the key to the standardization step is to resample these images so that they have the same spatial resolution under the same spatial grid.

[0042] As an option, the MRI images of each modality can be resampled by the B-spline interpolation method. The B-spline interpolation method is a commonly used interpolation technique that can effectively perform spatial transformation on images while ensuring image smoothness. Through the B-spline interpolation method, we can convert images of different modalities into images with the same resolution and ensure their alignment on the same spatial grid. Specifically, the B-spline interpolation method can generate new pixel values of the image based on the weighted average of control points, ensuring a smooth transition of the image in space.

[0043] In the present invention, the specific implementation manner of B-spline interpolation is as follows:

[0044]

[0045] Where: I interp (x, y) is the pixel value of the interpolated image, C m,n is the control point coefficient, B m (x) and B n (y) are the values of the B-spline basis functions in the xx and yy directions respectively, and M and N are the orders of the B-spline basis functions.

[0046] Specifically, the role of the B-spline basis function is to perform interpolation based on the neighborhood of pixel points, so that each pixel value of the image can reflect the weighted average of surrounding pixels. During the interpolation process, the interpolation points calculate their values through the B-spline basis function, thus ensuring the smoothness and continuity during the image interpolation process.

[0047] In a possible implementation manner, B-spline interpolation is not only applied to two-dimensional images but also to three-dimensional image data to adapt to the processing of volume data. For three-dimensional data, the interpolation process involves three-dimensional B-spline basis functions and takes into account the influence of interpolation and resampling in three-dimensional space.

[0048] In this embodiment, the specific steps of data standardization are as follows:

[0049] Image resampling: For the image data of each modality, it is resampled into images with a unified resolution and spatial size through the B-spline interpolation method. The pixel value of each pixel in each image is calculated by the weighted average of the surrounding adjacent pixels, ensuring that the interpolated image is smooth and continuous.

[0050] Multi-modal image alignment: After all modality images are resampled, they have the same spatial grid and resolution. This process ensures that different modality images can be compared and analyzed in the same coordinate system, laying the foundation for subsequent registration operations.

[0051] Applied to different modality MRI data: In multi-modal MRI images, the image data of different modalities may be inconsistent due to differences in acquisition methods, scanning protocols, and even patient positions. Therefore, the standardization process is very necessary. Especially when performing tasks such as modality registration and fusion, the standardization of image data can greatly reduce the differences between modalities and improve the registration accuracy.

[0052] Through this standardization step, we can analyze different modality MRI images at the same spatial resolution, eliminating the influence of resolution differences on the registration accuracy. In this way, subsequent processing steps such as tensor decomposition, variational optimization, and graph theory optimization can better align the images between modalities in space, further improving the accuracy and effect of multi-modal image registration.

[0053] In some embodiments, if there is partial region information loss in the image due to noise or distortion, the interpolation method can be combined during standardization to process these missing regions, avoiding the impact of image loss on subsequent steps.

[0054] Other technical extensions

[0055] Generally, the B-spline interpolation method is a flexible and common interpolation method. However, in some special cases, other interpolation methods can be used according to requirements, such as the cubic convolution interpolation method, which can also ensure high interpolation accuracy in image resampling and provide better results when dealing with some non-linear transformations.

[0056] In addition, if high-performance parallel processing of multi-modal MRI data is required, the B-spline interpolation can also be optimized through GPU acceleration technology to improve the computational efficiency of the resampling process. By optimizing the computational complexity in the interpolation process, efficient data standardization and preprocessing operations can be achieved on large-scale datasets.

[0057] S2. Unify the MRI images of each modality into a high-order tensor, input it into the Tucker decomposition model, output the low-dimensional factor matrix and the core tensor, and extract the interaction features between modalities;

[0058] In this embodiment, the goal of step S2 is to represent multi-modal MRI image data as a high-order tensor, and use tensor decomposition technology to reduce the dimension of the data and extract the high-dimensional interaction information between modalities. The core of this process is to reduce the dimension of the data through tensor decomposition, reduce the computational complexity, and at the same time retain the important interaction information between different modalities. The image normalization and resampling operations in step S1 have ensured that the spatial resolutions of different-modal images are consistent. The next task is to convert these normalized image data into the form of a high-order tensor for further analysis.

[0059] Specifically, in this embodiment, first, the MRI images of each modality are represented as a tensor. As a multi-dimensional array, a tensor can better represent the multi-level feature information of multi-modal data. In the context of MRI data, each image data can be regarded as a three-dimensional tensor, representing a volume data in space. The MRI data of each modality will form a high-order tensor, represented as a data structure with multiple dimensions.

[0060] In this implementation process, the tensor decomposition technology used is Tucker decomposition. Tucker decomposition is an efficient tensor decomposition method. By decomposing a high-order tensor into multiple factor matrices and a core tensor, the dimension reduction and feature extraction of multi-dimensional data are realized. Specifically, the tensor

[0061] can be decomposed into:

[0062]

[0063] where, is the original high-order tensor, containing image data from different modalities, is the core tensor, representing the low-dimensional representation after tensor decomposition, are factor matrices, corresponding to the features of each dimension respectively. R1, R2, and R3 are the ranks of each dimension during decomposition.

[0064] In this decomposition process, the core tensor G and the factor matrix U together constitute the low-dimensional representation of the data, capturing the important interaction information between different modalities. Through this method, the dimension of the data is effectively compressed, and at the same time, the correlation between modalities is retained, which is especially suitable for processing data with complex structures such as multi-modal MRI images.

[0065] In this embodiment, first, the standardized multi-modal image data is tensorized, and the data of each modality is represented as a three-dimensional tensor. Specifically, assuming that for an MRI image I(x,y) of a certain modality, its data can be represented as a three-dimensional tensor, where the first dimension represents the width of the image, the second dimension represents the height of the image, and the third dimension represents the depth or slice of the image. In this way, the images of each modality can be represented as a three-dimensional array, enabling images of different modalities to be processed through the same tensor structure.

[0066] For example, for multi-modal brain MRI data, assume there are three modalities: T1-weighted images, T2-weighted images, and fMRI images. The MRI data of each modality can be transformed into a three-dimensional tensor, and assume they are respectively (where represents the tensor of the i-th modality). Then, the data of all modalities will be decomposed in a unified tensor space. In this way, we can extract the high-dimensional interaction information between modalities within a shared tensor space.

[0067] Next, Tucker decomposition is used to reduce the dimension of these high-order tensors. Tucker decomposition compresses the data and extracts features by decomposing the tensor of each modality into a core tensor and factor matrices. During this process, the core tensor G represents the global information among all modalities, while the factor matrices correspond to the local features of the data of each modality respectively. Therefore, the process of tensor decomposition can effectively extract the high-dimensional interaction information between modalities, compress the data dimension, and improve the computational efficiency.

[0068] As an option, in actual implementation, the tensor decomposition can be optimized by optimizing the objective function. Specifically, the optimization objective is to minimize the reconstruction error between the original data and the decomposed data. This objective function can be expressed as:

[0069]

[0070] where, is the original tensor, is the tensor reconstructed by Tucker decomposition, represents the Frobenius norm. During the optimization process, by adjusting the core tensor G and the factor matrices the reconstruction error is minimized, thus ensuring the effectiveness of the dimension reduction process.

[0071] Through this way of tensor decomposition, not only can the data dimension be effectively reduced and the computing efficiency be improved, but also the complex relationships among multi-modal data can be captured, especially the high-order interaction information among different modalities. This is crucial for subsequent tasks such as registration and fusion. Through these processes, the finally obtained low-dimensional representation can better support subsequent image registration and disease localization.

[0072] In some embodiments, in order to further improve the decomposition accuracy, in addition to Tucker decomposition, other types of tensor decomposition methods can also be considered, such as CANDECOMP / PARAFAC decomposition (CP decomposition). CP decomposition represents a tensor as the sum of multiple rank-one tensors, which may be more effective for certain types of data, especially when the interaction between some modalities is relatively simple.

[0073] In addition, in order to handle large-scale datasets or real-time application requirements, GPU acceleration technology can be combined to optimize the tensor decomposition process. In this case, the operation of tensor decomposition can be accelerated through parallel computing to improve the processing efficiency and meet the real-time requirements in clinical applications.

[0074] S3. Use the extracted interaction features to construct an energy function, input the difference term and the regularization term, and minimize the energy function through the variational method to output the optimized spatial transformation function;

[0075] In this embodiment, step S3 aims to minimize the registration error by constructing an energy function optimized by the variational method, thereby optimizing the transformation function and achieving accurate registration of multi-modal MRI images. In the aforementioned steps S1 and S2, the standardization and tensor decomposition of multi-modal images have been completed, and the high-dimensional interaction information between modalities has been extracted. Next, with the optimization technology of the variational method, by setting an appropriate energy function, the registration between images is further optimized, making different modal images more aligned in space and reducing errors.

[0076] Specifically, the variational method is a mathematical method commonly used in optimization problems, which finds the optimal solution to the problem by minimizing a certain energy function. In the context of image registration, the optimization goal of the variational method is to minimize the registration error between the source image and the target image, and by calculating the transformation function, the source image can be spatially aligned with the target image. The optimization of the transformation function can be carried out through numerical optimization methods such as the gradient descent method to obtain the optimal registration effect.

[0077] In this embodiment, first, according to the processing steps in S1 and S2, we obtain the multi-modal MRI image data after standardization and tensoring. On this basis, the core task of the variational method optimization is to define and minimize an energy function, which quantifies the difference between the source image and the target image as a numerical value, so that it can be effectively minimized through the optimization algorithm.

[0078] In general, the energy function of the variational method usually consists of two parts: one is the difference term between the source image and the target image, and the other is the regularization term of the transformation function. The difference term is used to measure the matching degree between the source image and the target image, and the regularization term is used to control the smoothness of the transformation function, prevent overfitting, and ensure the geometric rationality in the registration process.

[0079] In the present invention, the general form of the energy function E(T) is:

[0080]

[0081] where I1(x, y) and I2(x, y) are the pixel values of the source image and the target image at the coordinate (x, y) respectively, and T(x, y) is the transformation function for image registration, which represents how each point in the source image is mapped to the corresponding point in the target image, ∥I1(x, y) - I2(T(x, y))∥ 2 is the difference term, which is used to measure the difference between the pixel values of the source image I1 and the target image I2 after transformation, is the regularization term of the transformation function, which is used to control the smoothness of the transformation function and avoid excessive deformation. α is the weight of the regularization term, usually a small constant, which is used to balance the influence of the difference term and the regularization term.

[0082] In this embodiment, the transformation function T(x, y) is usually an affine transformation or a non - linear transformation, depending on the image registration requirements. For simple image registration tasks, the affine transformation can meet the requirements, but in some complex neuroimage registration tasks, a more complex non - linear transformation model is usually adopted, which can capture the complex geometric transformation between images more precisely.

[0083] As an option, the transformation function T(x, y) can be defined by an elastic transformation or a B - spline - based transformation model. These transformation models can ensure the smoothness of the transformation while flexibly adapting to the complex structural changes in the image.

[0084] To optimize the energy function E(T), the gradient descent method or other numerical optimization methods can be used. Specifically, the optimization process calculates the gradient of the energy function with respect to the transformation function and adjusts the transformation function according to the gradient information to gradually approach the optimal solution.

[0085] The update rule of the gradient descent method is:

[0086]

[0087] where T (t(x, y) is the transformation function at the t-th iteration, and η is the learning rate that controls the step size of each update. is the gradient of the energy function E(T) with respect to the transformation function T.

[0088] Through multiple iterations of optimization, the transformation function T(x, y) will gradually converge to an optimal solution, ensuring the minimum registration error between the source image and the target image, thereby achieving precise image alignment.

[0089] In some embodiments, for image registration tasks of different modalities, a multi-scale optimization strategy can be combined to accelerate the convergence process. Under this strategy, registration is first performed at a low resolution to obtain a preliminary transformation function, and then gradually transferred to a high resolution for detailed optimization. In this way, the registration accuracy can be improved and the optimization process can be accelerated.

[0090] In addition, to further improve the quality of the registration results, a global optimization method, such as the simulated annealing algorithm, can be combined to avoid the gradient descent method falling into a local optimal solution. By combining multiple optimization methods, a more accurate image registration result can be obtained while ensuring computational efficiency.

[0091] S4. Use the transformation function to map the feature points to a unified space, construct a feature map structure, with the input being the similarity between nodes and the output being the optimized graph structure information to assist image fusion.

[0092] In this embodiment, the goal of step S4 is to further optimize image registration by constructing a graph structure to represent the relationship between feature points in multi-modal MRI images. In the aforementioned steps S1 and S2, the standardization and quantization of the images have been completed, and the registration error between the images has been optimized by variational methods to obtain a relatively accurate transformation function. Next, the core of step S4 is to construct a graph structure that can effectively represent the relationship between feature points in the images, thereby providing more detailed local information for subsequent image registration and fusion.

[0093] The graph structure is a powerful data representation method that can capture local feature points and their mutual relationships in the image through the relationship between nodes and edges. In the application of multi-modal MRI image registration, the graph structure can intuitively map the feature points and their spatial relationships between different modalities as nodes and edges in the graph, thereby providing effective information for image alignment and optimization. Through this graph structure, the registration operations in subsequent steps can more precisely process the details of local regions, thereby improving the overall registration effect.

[0094] In this embodiment, after optimizing the transformation function in step S3, we obtained a pair of registered images and achieved global alignment between the images. To further refine the registration process, we introduced a graph structure to capture the detailed information between different modality images at the feature point level. Specifically, first, a number of feature points need to be extracted from each modality image, and then these feature points and their mutual relationships are represented as nodes and edges in the graph.

[0095] Generally, the extraction of feature points can be achieved through classical image processing techniques, such as corner detection (Harris corner detection, Shi-Tomasi corner detection) or key-point based algorithms (such as SIFT, SURF). These algorithms can effectively identify regions with significant differences in the image as feature points for subsequent processing. The extracted feature points will serve as nodes in the graph, and the edges between the nodes represent the spatial relationships between these feature points.

[0096] In a possible implementation, the graph structure can be represented using an adjacency matrix to represent the relationship between nodes. The adjacency matrix A is an N×N matrix, where each element a ij represents the weight of the edge between node i and node j. The magnitude of the weight reflects the similarity or correlation degree between feature points. If the spatial distance between feature points is relatively close, or they exhibit similar features in a local region of the image, the weight of the edge between these two feature points will be larger.

[0097] The graph structure can be represented in the following way:

[0098]

[0099] where, a ij is the weight between node i and node j in the graph, reflecting the spatial relationship between feature points;

[0100] N is the total number of nodes in the graph, that is, the number of extracted feature points.

[0101] As an option, the weight of the adjacency matrix can be defined according to the similarity of feature points or the local consistency of the image. For example, if two feature points are close in spatial position in the image and their gray or color values are similar, the weight between them will be larger. These weights can be obtained by calculating the Euclidean distance, gray difference or other similarity metrics between feature points.

[0102] After constructing the graph structure, graph theory optimization algorithms can be applied next to further optimize the registration relationship between feature points. Generally, the goal of the optimization process is to calculate the shortest distance between nodes through shortest path algorithms (such as Dijkstra's algorithm, Bellman-Ford algorithm), so as to achieve the best registration between feature points. The optimized graph structure can provide more refined local registration results and effectively improve the accuracy of image registration.

[0103] In some embodiments, to improve the accuracy and stability of feature point matching, a multi-scale image pyramid can be considered. By extracting feature points at different scales and constructing corresponding graph structures, multi-level information in the image can be effectively captured. Especially for images with large deformations or complex structures, using a multi-scale strategy can better perform registration.

[0104] In addition, the representation and optimization of the graph structure can also be improved by combining deep learning methods such as graph convolutional networks (GCN). By using graph convolutional networks to process graph structure data, local feature learning can be performed in each layer of the graph, further enhancing the effect of image registration. Especially for relatively complex neuroimaging data, deep learning methods can play a greater advantage in feature extraction and graph optimization.

[0105] S5. Register the image according to the optimized transformation function to generate the final fused image;

[0106] In this embodiment, the main goal of step S5 is to register the multi-modal MRI images according to the optimized transformation function in step S3 to generate the final fused image. In the foregoing steps, the graph structure constructed in step S4 enables us to better understand the relationship between feature points of different modalities, laying a foundation for the final registration. By combining this information, the registration process will not only be limited to global alignment but also take into account the local structure and features between images, thereby improving the accuracy and effect of registration.

[0107] The registration process is to transform the source image into the coordinate system of the target image so that MRI images with different modalities can be jointly analyzed based on the same spatial information. By applying the optimal transformation function obtained previously, accurate registration between the source image and the target image can be achieved, and finally, fused image information is generated.

[0108] In this embodiment, the transformation function T(x, y) obtained through the foregoing steps will be used for the registration of the source image. The transformation function can be an affine transformation or a non-linear transformation, specifically depending on the requirements and characteristics of the imaging data. Each pixel of the source image obtains a new position in the coordinate system of the target image through the transformation function, thereby achieving registration.

[0109] Specifically, let the source image be I1(x,y) and the target image be I2(x,y). Through the formula:

[0110]

[0111] where, is the pixel value of the source image after transformation, and T(x,y) is the transformation function, which describes how each point in the source image is mapped to the corresponding point in the target image.

[0112] In this process, each pixel I1 in the source image will obtain a new pixel value according to its transformation under T This process can be implemented by interpolation methods (e.g., B-spline interpolation or nearest neighbor interpolation) to ensure the smoothness and continuity of the transformed image.

[0113] Generally, the interpolation result can reflect the reasonable value of the corresponding position of the source image on the target image after transformation. Especially in the case of non-integer pixel positions, the interpolation method is particularly important.

[0114] To further integrate the modal information and obtain the final fusion It is necessary to combine the features of different modal images on this basis. In some embodiments, the fusion strategy can be carried out by the method of weighted average, and the values of each modality at the corresponding target position are weighted and averaged. The fusion formula is expressed as:

[0115]

[0116] where, I k is the k-th modal image, T k (x,y) is the transformation function of the corresponding modality, w k is the weight coefficient, which is allocated by combining the reliability and importance of different modalities, and N is the total number of modalities.

[0117] Through the above weighted fusion, the image information of different modalities is integrated to generate the final fusion image. This process ensures that the complementarity of each modality can be fully utilized, thereby improving the overall quality and information content of the image.

[0118] In some embodiments, in order to improve the quality of the fusion result, advanced image fusion techniques such as wavelet transform or deep learning models can be considered. The wavelet transform has good time-frequency characteristics and can perform image fusion in different frequency bands, which helps to preserve the detailed information. Deep learning methods, such as convolutional neural network (CNN) models, can adaptively learn the fusion strategy and are especially suitable for complex multi-modal data scenarios.

[0119] In addition, the registered images can serve as the basis for subsequent analysis, enabling further quantitative analysis and processing, such as lesion detection and segmentation, feature extraction, etc. By combining advanced image analysis techniques, the potential of the fused images can be fully exploited to enable more efficient auxiliary diagnostic tools.

[0120] S6. Evaluate the registration and fusion results

[0121] In this embodiment, the main purpose of step S6 is to evaluate the effectiveness of the registered and fused images to ensure that they meet the quality standards in medical applications. In the aforementioned step S5, the registration of multi-modal MRI images has been completed, and fused images have been generated. Now, through a series of evaluation metrics and methods, the registration effectiveness, information retention degree, and overall quality of the images are evaluated to ensure high-quality data support for clinical analysis.

[0122] The evaluation process not only focuses on the visual effect of the registration results but also measures the quality of the images through quantitative metrics. Usually, such evaluations can include metrics such as Mutual Information, Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), etc. These evaluation methods can effectively reflect the degree of information retention in the fused images and their similarity to the authoritative standard images.

[0123] In this embodiment, first, a quantitative evaluation of the fused image I fused is required. Usually, mutual information is a commonly used metric for evaluating the registration effectiveness of images and is defined as:

[0124] MI(I1, I2) = H(I1) + H(I2) - H(I1, I2)

[0125] where H(I1) and H(I2) are the entropies of the source image I1 and the target image I2, respectively, representing the amount of information in each image, and H(I1, I2) is the joint entropy of the source image and the target image, representing the common information between the two.

[0126] The larger the mutual information, the higher the degree of information overlap between the fused image and the reference image, which is often proportional to the registration quality.

[0127] Furthermore, the Structural Similarity Index (SSIM) is another widely used quality evaluation criterion. SSIM evaluates the similarity between two images by comparing their brightness, contrast, and structure. The calculation formula of SSIM is:

[0128]

[0129] where μ1 and μ2 are the average values of images I1 and I2, respectively, and is the variance of the image, σ 12 is the covariance of two images, and C1 and C2 are constants used to stabilize the calculation.

[0130] The value range of SSIM is between -1 and 1, and the closer the value is to 1, the higher the image similarity.

[0131] After obtaining these quantitative evaluation metrics, a visual effect evaluation can be further carried out. This process includes subjective analysis of the fused image, usually inviting medical experts to review the image quality and evaluate its practicality and effectiveness in clinical applications.

[0132] As an option, the evaluation may include practical analysis related to the application field of the image. For example, in brain MRI, it may be necessary to evaluate the visualization effect and recognition accuracy of key structures (such as brain tumors or blood vessels). This link combines quantitative metrics and expert reviews to form a comprehensive quality assessment.

[0133] In some embodiments, to enhance the accuracy and objectivity of the evaluation, machine learning methods can be considered to automatically evaluate the image quality. For example, a convolutional neural network (CNN) can be trained for specific types of image analysis, and by comparing a large amount of labeled data, more detailed evaluation results can be provided.

[0134] In addition, the strategy of image quality evaluation can also be combined with computer vision techniques, using feature extraction and pattern recognition methods to detect subtle differences in the image, whether in terms of image contrast, clarity or details, to ensure that the finally generated fused image has practical value for clinical applications.

[0135] In summary, the present invention relates to a method for registering and fusing multi-modal MRI images, which realizes effective image registration and information integration through a series of steps. The method first standardizes and tensor decomposes the multi-modal images, extracts the interaction information between modalities, and then constructs an optimized energy function through variational methods to minimize the registration error, thereby obtaining an accurate transformation function. On this basis, a graph structure is constructed to represent the relationship between feature points, and the registration between nodes is optimized through the shortest path algorithm. Finally, the image registration is performed by applying the transformation function, and the fused image is generated according to the weighted average strategy. This method combines quantitative and qualitative evaluation means to evaluate the quality of the generated fused image to ensure that it meets the high standards of clinical applications. This innovative method not only improves the analysis ability of multi-modal MRI images, but also provides reliable support for the scientific interpretation of imaging data and medical decision-making.

[0136] Please refer to Figure 2 , the present invention also provides a system for constructing a neural disease localization, qualitative and evaluation system using multi-modal MRI image data, including the following modules:

[0137] Data processing module: used to perform normalization, resampling, and noise removal on the input multi-modal MRI image data;

[0138] Tensor decomposition module: used to perform high-order tensor decomposition on the normalized multi-modal images, extract the interaction information between modalities, and perform dimensionality reduction;

[0139] Optimization module: used to optimize the registration result by variational method, minimize the registration error, and calculate the optimal transformation function;

[0140] Graph theory module: used to construct a graph structure to represent the relationship between the feature points of the modal images, optimize the connectivity of the graph using the Laplacian matrix, and calculate the shortest registration path between the modal images;

[0141] The technical solution of this system is the same as the content obtained by the above method, and will not be elaborated here.

[0142] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a method for localizing, qualitatively analyzing, and evaluating neurological diseases using multi-modal MRI image data, characterized in that, It includes the following steps: Normalize the multi-modal MRI image data so that each modal image has the same spatial resolution; Unify each modal MRI image into a high-order tensor, input it into the Tucker decomposition model, output the low-dimensional factor matrix and the core tensor, and extract the interaction features between modalities; Use the extracted interaction features to construct an energy function, input the difference term and the regularization term, and minimize the energy function through variational method to output the optimized spatial transformation function; Use the transformation function to map the feature points to a unified space, construct a feature map structure, input the similarity between nodes, and output the optimized graph structure information to assist image fusion; Register the image according to the optimized transformation function to generate the final fused image; Evaluate the registration and fusion results.

2. A method for constructing the localization, qualitative analysis and evaluation of neurological diseases by using multimodal MRI image data according to claim 1, characterized in that The tensor decomposition adopts the Tucker decomposition model, and the core tensor and factor matrix obtained by decomposition are used to represent the interaction information between multi-modal images and realize the dimensionality reduction processing of modal data.

3. A method for constructing localization, qualitative analysis, and evaluation of neurological diseases using multi-modal MRI image data according to claim 1, characterized in that, In the variational method optimization step, the registration result is optimized by constructing an energy function and minimizing its value. The energy function includes the difference term between the source image and the target image and the regularization term of the gradient of the transformation function.

4. A method for constructing the localization, qualitative analysis, and evaluation of neurological diseases using multi-modal MRI image data according to claim 1, characterized in that The graph theory optimization step represents the similarity between the feature points of the modal images by constructing the adjacency matrix of the graph, and uses the Laplacian matrix to calculate the structure of the graph, so as to optimize the registration path of the modal images.

5. A method for constructing neuro-disease localization, characterization and evaluation using multi-modal MRI image data according to claim 1, characterized in that, The registration step includes using the transformation function to transform the source image, generating the aligned image and performing subsequent processing.

6. A method for constructing neuro-disease localization, qualitative analysis, and evaluation using multi-modal MRI image data according to claim 1, characterized in that, The fused image step includes fusing the registered multi-modal images using the weighted average method, where the image weight of each modality is determined by the importance of the modality and the image quality.

7. A method for constructing a method for localizing, qualitatively analyzing, and evaluating neurological diseases using multimodal MRI image data according to claim 1, wherein The evaluation step includes evaluating the registration accuracy using indicators such as the Dice coefficient and the Hausdorff distance, and verifying the quality of the fused image.

8. A method for constructing the localization, qualitative analysis, and evaluation of neurological diseases using multimodal MRI image data according to claim 1, characterized in that, It includes the following modules: Data processing module: used to perform standardization, resampling and noise removal processing on the input multi-modal MRI image data; Tensor decomposition module: used to perform high-order tensor decomposition on the standardized multi-modal images, extract the interaction information between modalities and perform dimensionality reduction processing; Optimization module: used to optimize the registration result through variational method, minimize the registration error and calculate the optimal transformation function; Graph theory module: used to construct a graph structure to represent the relationship between the feature points of the modal images, and use the Laplacian matrix to optimize the connectivity of the graph and calculate the shortest registration path between the modal images; Fusion module: used to perform weighted average fusion on the registered multi-modal images to generate the final image; Evaluation module: used to evaluate the quality of the registration and fusion results, and calculate indicators such as the Dice coefficient and the Hausdorff distance.

9. A system for localizing, qualitatively analyzing, and evaluating neurological diseases using multimodal MRI image data according to claim 8, characterized in that, The data processing module uses the B-spline interpolation method for image resampling and uses the variational denoising algorithm to remove noise from the image.

10. A system for localizing, qualitatively analyzing, and evaluating neurological diseases using multi-modal MRI image data according to claim 8, characterized in that, The optimization module includes a variational method optimization algorithm, which is used to minimize the registration error and optimize the transformation function through the gradient descent method.

Images