Full-automatic human cerebrovascular reconstruction method based on multi-modal image fusion technology

Abstract

The invention discloses a human cerebrovascular full-automatic reconstruction method based on a multi-modal image fusion technology, and relates to the technical field of human cerebrovascular full-automatic reconstruction. The problem of incomplete reconstruction caused by low contrast, local deformation and vascular rupture in a tumor compression area in traditional human cerebrovascular reconstruction is solved; after CT and MRI images are preprocessed, each branch adopts an encoder-decoder structure to extract multi-scale features, then a cross attention module is introduced to realize feature normalization and adaptive fusion, and signal performance of weak blood vessels in a tumor compression area is enhanced; and on the basis of global registration, local deformation areas generated by tumor compression in the image are automatically detected by utilizing area mutual information and local structure constraints, and the areas are corrected by adopting a refined registration algorithm, so that the local blood vessel trend is effectively recovered.

Description

Technical Field

[0001] The present invention relates to the technical field of fully automatic reconstruction of human cerebral blood vessels, and in particular to a fully automatic reconstruction method of human cerebral blood vessels based on multimodal image fusion technology. Background Art

[0002] At present, the fully automatic reconstruction technology of human brain blood vessels mainly uses multimodal imaging data to automatically generate three-dimensional vascular structures through edge extraction, image fusion and segmentation algorithms.

[0003] In actual applications, for some complex cases, due to brain pathological changes or local anatomical abnormalities, the edge information of blood vessels in the image will be relatively blurred, and the signal contrast will be reduced. The effect of adjusting with fixed weight fusion and preset threshold segmentation will not be good, which will lead to segmentation errors or omissions in some subtle vascular areas, resulting in breaks or incomplete connections in the 3D reconstructed model. When dealing with local deformation, traditional methods mostly introduce regional growing, active contour and other technologies for local compensation, but they require a lot of manual intervention and the effect is unstable, especially when there is tissue compression or other pathological changes near the blood vessels, the direction of the blood vessels will be obviously bent or twisted, affecting the overall vascular network. Some traditional solutions use noise filtering and bone peeling technology in the preprocessing stage to reduce external interference, and use gradient information and mathematical morphology methods to enhance the signal in the segmentation stage, but this method has little effect on local signal attenuation and structural variation.

[0004] It can be seen that traditional human brain vascular reconstruction technology faces problems such as incomplete reconstruction and vascular rupture in scenarios with weak local signals and abnormal structures, such as tissue deformation or local compression caused by pathological factors. This has prompted subsequent improvements to require a more flexible fully automatic human brain vascular reconstruction method based on multimodal image fusion technology to more accurately restore the real vascular structure and provide imaging support for clinical applications. Summary of the invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] The present invention provides a fully automatic reconstruction method of human brain blood vessels based on multimodal image fusion technology to solve the problem that when the tumor is located in the central area, the edges of the surrounding blood vessels are blurred due to compression and tissue deformation, and there is a significant difference in local contrast between CT and MRI; the traditional fusion method based on edge feature extraction and fixed weight addition is difficult to capture the tiny and curved blood vessels around the tumor, resulting in incomplete reconstruction of key blood supply pathways in surgical planning.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] An embodiment of the present invention provides a fully automatic human cerebral blood vessel reconstruction method based on multi-modal image fusion technology, which includes:

[0009] Step S1: Obtain a sequence of human brain CT images and a sequence of magnetic resonance imaging (MRI) images, and preprocess the images.

[0010] Step S2: Construct a dual-branch deep neural network, where the first branch is used to process the preprocessed CT images in Step S1, and the second branch is used to process the preprocessed MRI images.

[0011] Step S3: At the intermediate layer of the dual-branch deep neural network, use the multi-scale features from each branch as inputs, and introduce a cross-attention module for feature fusion.

[0012] Step S4: Implement a local adaptive registration mechanism for local deformations within the tumor compression area.

[0013] Step S5: Use the fused feature map output by Step S3 and the local registration adjustment results obtained in Step S4 to generate a multi-modal fusion image through fusion processing; perform threshold adaptive segmentation on the multi-modal fusion image using an improved watershed algorithm to extract the human cerebral blood vessel area.

[0014] Step S6: Perform graph theory post-processing on the blood vessel segmentation results obtained in Step S5.

[0015] Step S7: Combine the processed blood vessel segmentation results in Step S6 with the original image data preprocessed in Step S1 and the local registration information obtained in Step S4.

[0016] As a preferred solution of the fully automatic human cerebral blood vessel reconstruction method based on multi-modal image fusion technology of the present invention: The CT image sequence is a computed tomography angiography (CTA) or computed tomography venography (CTV) image, and the MRI image sequence is a magnetic resonance angiography (MRA) or magnetic resonance venography (MRV) image.

[0017] The preprocessing includes removing the scalp and skull, noise filtering, and normalization processing of the image gray values.

[0018] In the dual-branch deep neural network in Step S2:

[0019] Each branch uses an encoder-decoder structure to extract multi-scale features, where the multi-scale features refer to local and global image representations extracted at different levels.

[0020] The encoder part uses multi-layer convolution and pooling operations, and the decoder part uses upsampling and skip connections to restore details.

[0021] As a preferred solution of the fully automatic human cerebral blood vessel reconstruction method based on multi-modal image fusion technology of the present invention, wherein: the cross-attention module in step S3 normalizes the input CT branch features and MRI branch features, calculates the similarity matrix between the two by dot multiplication, and generates local fusion weights using the Softmax function,

[0022] and accordingly weights and fuses each branch feature to output a set of fused multi-modal feature maps;

[0023] The fused feature maps are used for subsequent steps and play a role in enhancing the low-contrast signals in the tumor compression area.

[0024] As a preferred solution of the fully automatic human cerebral blood vessel reconstruction method based on multi-modal image fusion technology of the present invention, wherein: in step S3, the step of normalizing the input CT branch features and MRI branch features, calculating the similarity matrix between the two by dot multiplication, and generating local fusion weights is as follows,

[0025] In the dual-branch deep network, let the multi-scale features extracted from the CT branch be:

[0026] X C ∈R n×d where R n×d represents a matrix composed of n rows and d columns,

[0027] and the multi-scale features extracted from the MRI branch are:

[0028] X M ∈R n×d ;

[0029] Normalize each feature matrix to obtain:

[0030]

[0031] and

[0032] where X C represents the feature matrix output by the CT branch, X M represents the feature matrix output by the MRI branch, n is the number of features, d is the dimension of each feature, |X C |2 and |X M |2 respectively represent the Euclidean norms of the corresponding matrices;

[0033] Calculate the similarity matrix between the normalized features of the two branches using dot multiplication, and the formula is:

[0034]

[0035] where \(S\in R\) n×n , \(S\) ij represents the similarity between the \(i\)-th CT feature and the \(j\)-th MRI feature, and \(T\) represents the transpose operation;

[0036] Apply the Softmax function to the similarity matrix to generate a local fusion weight matrix:

[0037]

[0038] where \(W\) ij represents the weight of the \(j\)-th element in the \(i\)-th row, and \(k\) is the summation index, ranging from 1 to \(n\);

[0039] Perform feature fusion through weighted summation, and output the fused multi-modal feature map \(F\), expressed as:

[0040] \(F = W\cdot X\) M +(I - W)\cdot X C ,

[0041] where \(W\) refers to the local fusion weight matrix, \(F\in R\) n×d is the fused feature map, and \(I\) is the \(n\times n\) identity matrix.

[0042] As a preferred solution of the fully automatic human cerebral blood vessel reconstruction method based on multi-modal image fusion technology according to the present invention, wherein: the local adaptive registration mechanism in step S4 includes:

[0043] a) Perform global registration on the entire preprocessed image to align the CT image and the MRI image in the overall space;

[0044] b) Automatically detect the region with local deformation caused by tumor compression by using regional mutual information and local structure constraints,

[0045] c) Apply a refined registration algorithm to the detected region to correct the local deformation and match the blood vessel directions before and after the compression of the compressed region, and output the local registration adjustment result.

[0046] As a preferred solution of the fully automatic human cerebral blood vessel reconstruction method based on multi-modal image fusion technology according to the present invention, wherein: in step S4, the method of local adaptive registration is,

[0047] For the local deformation within the tumor compression region, adopt region detection and refined registration. Let the local region be denoted as \(R\), and the joint probability distribution corresponding to the CT and MRI images within the local region be denoted as \(p\) XY (x,y), and their respective marginal probabilities are \(p\) X (x) and \(p\) Y (y), and the local mutual information is defined as \(I(R)\):

[0048]

[0049] Among them, x and y are pixel values in the local area, I(R) reflects the local statistical correlation. When it is detected that I(R) < τ, it is determined that the region R has deformation, and τ is a preset threshold;

[0050] Within the detection area, let the local registration transformation function be T d (·), where d is the transformation parameter vector, and for the MRI image feature X M (R) and the CT image feature X C (R), find the optimal registration parameters to solve the minimum mean square error problem:

[0051]

[0052] Among them, d * is the optimal transformation parameter, and T d (X M (R)) represents the feature of the MRI image after being transformed by T in the region R d , and the output local registration adjustment result is denoted as F reg .

[0053] As a preferred solution of the method for automatically reconstructing human cerebral blood vessels based on multi-modal image fusion technology described in the present invention, among them: the multi-modal fusion image in step S5 is an image obtained by adaptively weighted fusion and local deformation correction of the CT image and the MRI image.

[0054] As a preferred solution of the method for automatically reconstructing human cerebral blood vessels based on multi-modal image fusion technology described in the present invention, among them: the graph theory post-processing in step S6 includes:

[0055] d) Construct a vascular network graph model, where the nodes represent the endpoints or bifurcation points of blood vessels, and the edges represent blood vessel segments,

[0056] e) Detect the non-connected nodes or broken regions existing in the vascular network,

[0057] f) Automatically calculate and complete the broken blood vessel part using the shortest path algorithm, and verify the connectivity of the completion result.

[0058] As a preferred solution of the method for automatically reconstructing human cerebral blood vessels based on multi-modal image fusion technology described in the present invention, among them: the graph theory post-processing in step S6 includes:

[0059] Let the binary vascular segmentation map obtained in step S5 be denoted as B ∈ {0, 1} m×n×p, based on this, a vascular network graph G=(V, E) is constructed, where V represents the set of vascular endpoints or bifurcation points, and E represents the set of vascular segments;

[0060] For any edge e∈E, an edge weight function is assigned:

[0061] w(e)=α·f(e),

[0062] where w(e) reflects the weight of edge e, α is a scaling factor, and f(e) is a function that reflects the feature discontinuity in the region corresponding to the edge;

[0063] For the unconnected nodes or broken regions detected in the network, let the two nodes to be completed be v i and v j , and solve the shortest path problem:

[0064]

[0065] where, represents the set of all paths connecting node v i and v j , P is an arbitrary candidate path connecting v i , v j , and P * is the path with the minimum weight sum. After completing the broken blood vessel part, the connectivity of the entire vascular network is verified, and finally the complete network structure is output.

[0066] As a preferred solution of the automatic reconstruction method of human cerebral blood vessels based on multi-modal image fusion technology described in the present invention, wherein: in step S7, the combination method is:

[0067] Use standard tools (such as the image processing software packages Nibabel and VTK) to perform three-dimensional reconstruction on the combined data, and output a three-dimensional model of human cerebral blood vessels for clinical surgical planning;

[0068] The processed blood vessel segmentation result refers to the binary blood vessel image obtained after improved segmentation and post-processing in steps S5 and S6.

[0069] The beneficial effects of the present invention are as follows: The present invention adopts a dual-branch deep neural network combined with local adaptive registration and graph theory post-processing, which solves the problem of incomplete reconstruction in the tumor compression area due to low contrast, local deformation, and blood vessel breakage in traditional human cerebral blood vessel reconstruction; after preprocessing CT and MRI images, each branch uses an encoder-decoder structure to extract multi-scale features, and then introduces a cross-attention module to achieve feature normalization and adaptive fusion, so as to make full use of the advantages of the two modalities and enhance the signal performance of weak blood vessels in the tumor compression area.

[0070] Based on global registration, the present invention automatically detects local deformation regions caused by tumor compression in images by using regional mutual information and local structure constraints, and corrects these regions by using a refined registration algorithm, effectively restoring the local blood vessel orientation; it has a high degree of automation, can reduce manual intervention, and improve the matching degree of two-modal images in the deformed regions.

[0071] In addition, by constructing a blood vessel network graph, the present invention automatically detects and complements blood vessel breaks caused by segmentation errors by using the shortest path algorithm, repairs the disconnected parts in the network, and enables the constructed three-dimensional blood vessel model to have a continuous and complete network structure.

[0072] The present invention effectively overcomes the disadvantages of traditional methods such as low contrast, local deformation, and blood vessel breaks in the tumor compression region, significantly improves the detail degree and overall connectivity of blood vessel segmentation, provides a more accurate and complete three-dimensional blood vessel model for neurosurgical operation planning, and has good clinical application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0074] Figure 1 It is a schematic flowchart of the method for fully automatic reconstruction of human cerebral blood vessels based on multi-modal image fusion technology of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0075] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific embodiments of the present invention in conjunction with the drawings in the specification.

[0076] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0077] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that can be included in at least one implementation manner of the present invention. The appearances of "in one embodiment" in different places in this specification do not all refer to the same embodiment, nor are they separate or alternative embodiments that exclude other embodiments.

[0078] Example 1, refer to Figure 1, this embodiment provides a fully automatic reconstruction method for human cerebral blood vessels based on multi-modal image fusion technology, including the following steps

[0079] Step S1, obtain a sequence of human brain CT images and a sequence of magnetic resonance imaging (MRI) images, and preprocess the images;

[0080] Step S2, construct a dual-branch deep neural network, where the first branch is used to process the preprocessed CT images in Step S1, and the second branch is used to process the preprocessed MRI images;

[0081] The CT image sequence is a computed tomography angiography (CTA) or computed tomography venography (CTV) image, and the MRI image sequence is a magnetic resonance angiography (MRA) or magnetic resonance venography (MRV) image;

[0082] The preprocessing includes removing the scalp and skull, noise filtering, and normalization of the image gray value;

[0083] In the dual-branch deep neural network in Step S2:

[0084] Each branch uses an encoder-decoder structure to extract multi-scale features, where the multi-scale features refer to local and global image representations extracted at different levels;

[0085] The encoder part uses multi-layer convolution and pooling operations, and the decoder part uses upsampling and skip connections to restore details;

[0086] Step S3, at the intermediate layer of the dual-branch deep neural network, use the multi-scale features from each branch as inputs, and introduce a cross-attention module for feature fusion;

[0087] The cross-attention module in Step S3 normalizes the input CT branch features and MRI branch features, calculates the similarity matrix between the two using dot product operation, and generates local fusion weights using the Softmax function,

[0088] Based on this, weighted fusion is performed on the features of each branch, and a set of fused multi-modal feature maps is output;

[0089] The fused feature maps are used in subsequent steps and play a role in enhancing the low-contrast signals in the tumor compression area;

[0090] In Step S3, the steps of normalizing the input CT branch features and MRI branch features, calculating the similarity matrix between the two using dot product operation, and generating local fusion weights using the Softmax function are as follows

[0091] In the dual-branch deep network, let the multi-scale features extracted from the CT branch be:

[0092] XC ∈R n×d ,R n×d represents a matrix composed of n rows and d columns,

[0093] The multi-scale features extracted from the MRI branch are:

[0094] X M ∈R n×d ;

[0095] Normalize each feature matrix to obtain:

[0096]

[0097] and

[0098] where X C represents the feature matrix output by the CT branch, X M represents the feature matrix output by the MRI branch, n is the number of features, d is the dimension of each feature, |X C |2 and |X M |2 represent the Euclidean norms of the corresponding matrices respectively;

[0099] Calculate the similarity matrix between the normalized features of the two branches using the dot product operation. The formula is:

[0100]

[0101] where S ∈ R n×n ,S ij represents the similarity between the i-th CT feature and the j-th MRI feature, and T represents the transpose operation;

[0102] Apply the Softmax function to the similarity matrix to generate the local fusion weight matrix:

[0103]

[0104] where W ij represents the weight of the j-th element in the i-th row, and k is the summation index, ranging from 1 to n;

[0105] Perform feature fusion through weighted summation, and output the fused multi-modal feature map F, expressed as:

[0106] F = W · X M +(I - W) · X C ,

[0107] where W refers to the local fusion weight matrix, F ∈ R n×d is the fused feature map, and I is the n × n identity matrix;

[0108] Specifically, here, the multi-scale features of the CT and MRI branches are first normalized to eliminate the scale difference between the two, and then the local feature similarity is calculated by dot product. Through the Softmax function, the local fusion weights can adaptively reflect the importance of the two modalities at each feature position, and weighted fusion is performed accordingly. The finally output fused feature map enhances the low-contrast signals in the tumor compression area to a certain extent, fully utilizes the information advantages of the two modalities respectively during the feature fusion process, automatically adjusts the weight distribution, so that more accurate vascular structure information can be obtained in the subsequent segmentation stage. The whole process is realized through end-to-end training without manual weight setting;

[0109] Step S4: Implement a local adaptive registration mechanism for the local deformation in the tumor compression area;

[0110] The local adaptive registration mechanism in step S4 includes:

[0111] a) Perform global registration on the entire preprocessed image to align the CT image and the MRI image in the overall space;

[0112] b) Automatically detect the area with local deformation caused by tumor compression by using regional mutual information and local structure constraints,

[0113] c) Apply a refined registration algorithm to the detected area to correct the local deformation and match the blood vessel directions before and after the deformation in the compression area, and output the local registration adjustment result;

[0114] In step S4, the local adaptive registration method is,

[0115] For the local deformation in the tumor compression area, use regional detection and refined registration. Let the local area be denoted as R, and the joint probability distribution of the CT and MRI images in the local area be denoted as p XY (x, y), and their respective marginal probabilities are p X (x) and p Y (y). The local mutual information is defined as I(R):

[0116]

[0117] Among them, x and y are pixel values in the local area, I(R) reflects the local statistical correlation. When it is detected that I(R) < τ, it is determined that the area R has deformation, and τ is a preset threshold;

[0118] In the detected area, let the local registration transformation function be T d (·), where d is the transformation parameter vector, and for the MRI image feature X M (R) and the CT image feature X C(R) Find the optimal registration parameters to solve the minimum mean square error problem:

[0119]

[0120] where d * is the optimal transformation parameter, and T d (X M (R)) represents the features of the MRI image in region R after transformation T d . The output of the local registration adjustment result is denoted as F reg ;

[0121] Specifically, the local adaptive registration mechanism calculates the local mutual information, automatically detects the local deformation area caused by tumor compression using the statistical features within the region. When the detection index is lower than the preset threshold, it indicates a significant mismatch between the two-modal images in this region. The refined registration algorithm is used to obtain the local optimal transformation parameters by solving the minimum mean square error problem, thereby locally correcting the MRI image;

[0122] Step S5: Use the fused feature map output in step S3 and the local registration adjustment result obtained in step S4 to generate a multi-modal fused image through fusion processing; perform threshold adaptive segmentation on the multi-modal fused image using an improved watershed algorithm to extract the human cerebrovascular region;

[0123] The multi-modal fused image in step S5 is the image after adaptive weighted fusion and local deformation correction of the CT image and the MRI image;

[0124] Step S6: Perform graph theory post-processing on the vascular segmentation result obtained in step S5;

[0125] The graph theory post-processing in step S6 includes:

[0126] d) Construct a vascular network graph model, where the nodes represent the endpoints or bifurcation points of blood vessels, and the edges represent blood vessel segments,

[0127] e) Detect the non-connected nodes or broken regions existing in the vascular network,

[0128] f) Automatically calculate and complete the broken blood vessel part using the shortest path algorithm, and verify the connectivity of the completion result;

[0129] The graph theory post-processing in step S6 includes:

[0130] Let the binary vascular segmentation map obtained in step S5 be denoted as B ∈ {0, 1} m×n×p , and based on this, construct a vascular network graph G = (V, E), where V represents the set of endpoints or bifurcation points of blood vessels, and E represents the set of blood vessel segments;

[0131] For any edge \(e\in E\), assign an edge weight function:

[0132] \(w(e)=\alpha\cdot f(e)\),

[0133] where \(w(e)\) reflects the weight of edge \(e\), \(\alpha\) is a scaling factor, and \(f(e)\) is a function that reflects the feature discontinuity within the region corresponding to the edge;

[0134] For the unconnected nodes or broken regions detected in the network, let the two nodes to be completed be \(v\) i and \(v\) j , and solve the shortest path problem:

[0135]

[0136] where, denotes the set of all paths connecting node \(v\) i and \(v\) j , \(P\) is any candidate path connecting \(v\) i , \(v\) j , and \(P\) * is the path with the minimum weight sum. After completing the broken blood vessel part, perform a connectivity verification on the entire blood vessel network, and finally output the complete network structure;

[0137] Specifically, the graph theory post-processing constructs a blood vessel network graph based on the blood vessel segmentation result, takes the endpoints and bifurcation points of the blood vessels as the nodes of the graph, the blood vessel segments as the edges, and reflects the local structure continuity by setting the edge weights. The shortest path algorithm is used to automatically complete the detected broken regions, which helps to restore the connectivity of the broken blood vessels. The connectivity verification step ensures that the overall blood vessel network after completion conforms to the actual distribution of the anatomical structure, providing an effective automated solution for dealing with local breaks caused by segmentation errors, making the finally constructed three-dimensional blood vessel model accurately reflect the continuous structure of the blood vessel network, with high robustness and practicality;

[0138] Step S7: Combine the processed blood vessel segmentation result in step S6 with the preprocessed original image data in step S1 and the local registration information obtained in step S4;

[0139] In step S7, the combination method is:

[0140] Use standard tools (such as the image processing software packages Nibabel and VTK) to perform three-dimensional reconstruction on the combined data, and output a three-dimensional model of the human cerebral blood vessels for clinical surgical planning;

[0141] The processed blood vessel segmentation result refers to the binary blood vessel image obtained after improved segmentation and post-processing in steps S5 and S6.

[0142] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for fully automatic reconstruction of human cerebral blood vessels based on multimodal image fusion technology, characterized in that: include, Step S1, obtaining a human brain CT image sequence and a magnetic resonance imaging MRI image sequence, and preprocessing the images; Step S2, constructing a dual-branch deep neural network, wherein the first branch is used to process the CT image preprocessed in step S1, and the second branch is used to process the preprocessed MRI image; Step S3, in the middle layer of the dual-branch deep neural network, taking the multi-scale features from each branch as input, and introducing a cross attention module to perform feature fusion; Step S4, implementing a local adaptive registration mechanism for local deformation in the tumor compression area; Step S5, using the fusion feature map outputted by step S3 and the local registration adjustment result obtained by step S4, a multimodal fusion image is generated by fusion processing; an improved watershed algorithm is used to perform threshold adaptive segmentation on the multimodal fusion image to extract the human brain vascular area; Step S6, performing graph theory post-processing on the blood vessel segmentation result obtained in step S5; Step S7, combining the blood vessel segmentation result processed in step S6 with the original image data preprocessed in step S1 and the local registration information obtained in step S4.

2. The method for fully automatic reconstruction of human cerebral blood vessels based on multimodal image fusion technology as claimed in claim 1, characterized in that: The CT image sequence is angiography CTA or venography CTV image, and the MRI image sequence is magnetic resonance angiography MRA or magnetic resonance venography MRV image; The preprocessing includes removing the scalp and skull, filtering out noise, and normalizing the image grayscale value; In the dual-branch deep neural network in step S2: Each branch uses an encoder-decoder structure to extract multi-scale features, where the multi-scale features refer to local and global image representations extracted at different levels; The encoder part adopts multi-layer convolution and pooling operations, and the decoder part adopts upsampling and skip connections to restore details.

3. The method for fully automatic reconstruction of human cerebral blood vessels based on multimodal image fusion technology as claimed in claim 2, characterized in that: The cross attention module in step S3 normalizes the input CT branch features and MRI branch features, calculates the similarity matrix between the two using a dot multiplication operation, and generates local fusion weights using a Softmax function. Based on this, the features of each branch are weighted and fused to output a set of fused multimodal feature maps.

4. The method for fully automatic reconstruction of human cerebral blood vessels based on multimodal image fusion technology as claimed in claim 3, characterized in that: In step S3, the steps of normalizing the input CT branch features and MRI branch features, calculating the similarity matrix between the two using a dot multiplication operation, and generating local fusion weights using a Softmax function are as follows: In the dual-branch deep network, the multi-scale features extracted from the CT branch are assumed to be: X C ∈R n×d , R n×d represents a matrix consisting of n rows and d columns, The multi-scale features extracted from the MRI branch are: X M ∈R n×d ; Normalize each feature matrix and get: and Among them, X C represents the feature matrix of the CT branch output, X M represents the feature matrix output by the MRI branch, n is the number of features, d is the dimension of each feature, |X C |2 and |X M |2 represents the Euclidean norm of the corresponding matrix; The similarity matrix between the normalized features of the two branches is calculated using the dot multiplication operation. The formula is: Among them, S∈R n×n , S ij represents the similarity between the i-th CT feature and the j-th MRI feature, and T represents the transposition operation; Apply the Softmax function to the similarity matrix to generate the local fusion weight matrix: Among them, W ij represents the weight of the jth element in the i-th row, k is the sum index, and its value range is 1 to n; The feature fusion is performed by weighted summation, and the fused multimodal feature map F is output, which is expressed as: F=W·X M +(I-W)·X C , Among them, W refers to the local fusion weight matrix, F∈R n×d is the fused feature map, and I is the n×n unit matrix.

5. The method for fully automatic reconstruction of human cerebral blood vessels based on multimodal image fusion technology as claimed in claim 4, characterized in that: The local adaptive registration mechanism in step S4 includes: a) Perform global registration on the entire preprocessed image to align the CT image with the MRI image in the overall space; b) Automatically detect the area where local deformation occurs due to tumor compression by using regional mutual information and local structural constraints, c) A refined registration algorithm is used for the detection area to correct the local deformation and match the blood vessel direction before and after the compression area is deformed, and the local registration adjustment result is output.

6. The method for fully automatic reconstruction of human cerebral blood vessels based on multimodal image fusion technology as claimed in claim 5, characterized in that: In step S4, the local adaptive registration method is: In view of the local deformation in the tumor compression area, regional detection and refined registration are used. The local area is denoted as R, and the joint probability distribution of CT and MRI images in the local area is denoted as p XY (x,y), the marginal probabilities of each are p X (x) and p Y (y), the local mutual information is defined as I(R): Where x, y are pixel values ​​in the local area, I(R) reflects the local statistical correlation, and when I(R) < τ is detected, it is determined that the area R is deformed, and τ is the preset threshold; In the detection area, let the local registration transformation function be T d (·), where d is the transformation parameter vector, which is used to transform the MRI image feature X in the local area. M (R) and CT image feature X C (R) Find the optimal registration parameters and solve the minimum mean square error problem: Among them, d * is the optimal transformation parameter, T d (X M (R)) represents the MRI image in region R after transformation T d The output local registration adjustment result is recorded as F reg .

7. The method for fully automatic reconstruction of human cerebral blood vessels based on multimodal image fusion technology as claimed in claim 6, characterized in that: The multimodal fusion image in step S5 is an image obtained by adaptively weighting and fusion of the CT image and the MRI image and correcting local deformation.

8. The method for fully automatic reconstruction of human cerebral blood vessels based on multimodal image fusion technology as claimed in claim 7, characterized in that: The graph theory post-processing in step S6 includes: d) construct a vascular network graph model, where nodes represent vascular endpoints or bifurcation points, and edges represent vascular segments. e) Detecting disconnected nodes or broken areas in the vascular network, f) The shortest path algorithm is used to automatically calculate and complete the broken blood vessel part, and the connectivity of the completion result is verified.

9. The method for fully automatic reconstruction of human cerebral blood vessels based on multimodal image fusion technology as claimed in claim 8, characterized in that: The graph theory post-processing of step S6 includes: Suppose the binary blood vessel segmentation map obtained in step S5 is denoted as B∈{0,1} m×n×p On this basis, a vascular network graph G = (V, E) is constructed, where V represents the set of vascular endpoints or bifurcation points, and E represents the set of vascular segments; For any edge e∈E, assign the edge weight function: w(e)=α·f(e), Among them, w(e) reflects the weight of edge e, α is the scaling factor, and f(e) is a function that reflects the feature discontinuity in the area corresponding to the edge; For the disconnected nodes or broken areas detected in the network, let the two nodes to be completed be v i and v j , solve the shortest path problem: in, Represents all connected nodes v i With v j The path set of P is the path connecting v i ,v j Any candidate path, P * After completing the broken blood vessel part, the connectivity of the entire vascular network is verified, and finally the complete network structure is output.

10. The method for fully automatic reconstruction of human cerebral blood vessels based on multimodal image fusion technology as claimed in claim 9, characterized in that: In step S7, the combination method is: The combined data are reconstructed in three dimensions using standard tools to output a three-dimensional model of human cerebral vascular vessels for clinical surgical planning; The processed blood vessel segmentation result refers to the binary blood vessel image obtained in steps S5 and S6 after improved segmentation and post-processing.

Images