A full-automatic reconstruction method for human brain blood vessels based on multi-modal image fusion technology

Abstract

The application discloses a kind of based on multimodal image fusion technology's human cerebral vessel full automatic reconstruction method, it is related to human cerebral vessel full automatic reconstruction technical field, the present application, using double-branch deep neural network is combined with local adaptive registration and graph theory post-processing, it has solved the problem of incomplete reconstruction caused by low contrast, local deformation and blood vessel fracture in traditional human cerebral vessel reconstruction in tumor compression area;After pre-processing CT and MRI image, each branch uses encoder-decoder structure to extract multi-scale features, then introduces cross attention module to realize feature normalization and adaptive fusion, enhance the signal performance of weak blood vessels in tumor compression area;Also on the basis of global registration, using regional mutual information and local structure constraint to automatically detect local deformation area in image due to tumor compression, correct these areas using thinning registration algorithm, effectively restore local blood vessel direction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human brain blood vessel full-automatic reconstruction, and particularly relates to a human brain blood vessel full-automatic reconstruction method based on multi-modal image fusion technology. BACKGROUND

[0002] At present, the human brain blood vessel full-automatic reconstruction technology mainly utilizes multi-modal image data to automatically generate a three-dimensional blood vessel structure through edge extraction, image fusion and segmentation algorithm.

[0003] In actual application, for some complex cases, due to pathological changes of the brain or abnormal local anatomical structure, the edge information of the blood vessels in the image is relatively blurred, and the signal contrast is reduced. Therefore, the adjustment effect is not good when the fixed weight fusion and the preset threshold segmentation are used again, which can cause segmentation errors or missed detection of some subtle blood vessel regions, and can cause the three-dimensional reconstruction model to be broken or not fully connected. In the traditional method, when dealing with local deformation, region growing and active contour technology are usually introduced for local compensation, but a large amount of manual intervention is required, and the effect is unstable. Especially in the case where there is tissue compression or other pathological changes near the blood vessels, the blood vessel direction can be obviously bent or twisted, which affects the overall blood vessel network. In addition, some traditional schemes use noise filtering and bone skin stripping technology in the preprocessing stage to reduce external interference, and use gradient information and mathematical morphology method to enhance the signal in the segmentation stage, but for local signal attenuation and structure variation, the effect of this method is very small.

[0004] It can be seen that the traditional human brain blood vessel reconstruction technology has problems such as incomplete reconstruction and blood vessel rupture in the scene of local weak signal and abnormal structure, such as tissue deformation or local compression caused by pathological factors, which promotes the subsequent improvement to need a more flexible human brain blood vessel full-automatic reconstruction method based on multi-modal image fusion technology, which can more finely restore the real blood vessel structure to provide image support for clinical application. SUMMARY

[0005] In view of the above existing problems, the present application is proposed.

[0006] The present application provides a human brain blood vessel full-automatic reconstruction method based on multi-modal image fusion technology, which solves the problem that when the tumor is located in the central region, the edge of the surrounding blood vessels is blurred due to compression and tissue deformation, and there is an obvious difference in local contrast between CT and MRI. The traditional fusion method based on edge feature extraction and fixed weight addition is difficult to capture the small and curved blood vessels around the tumor, which leads to incomplete reconstruction of the key blood supply path in the surgical planning.

[0007] To solve the above technical problems, the present application provides the following technical scheme:

[0008] The embodiment of the present application provides a kind of based on multimodal image fusion technology's human cerebral vessel full automatic reconstruction method, it includes,

[0009] Step S1, obtain human brain CT image sequence and magnetic resonance imaging MRI image sequence, and pre-process image;

[0010] Step S2, construct double-branch deep neural network, first branch is used to process the CT image after pre-processing in step S1, second branch is used to process the MRI image after pre-processing;

[0011] Step S3, in the middle layer of the double-branch deep neural network, multi-scale features from each branch are input as input, cross attention module is introduced to carry out feature fusion;

[0012] Step S4, for local deformation in tumor compression area, implement local adaptive registration mechanism;

[0013] Step S5, using the fusion feature map output by step S3 and the local registration adjustment result obtained in step S4, generate multimodal fusion image by fusion processing;The multimodal fusion image is segmented by improved watershed algorithm to adaptively, and brain blood vessel region is extracted;

[0014] Step S6, the blood vessel segmentation result obtained in step S5 is processed by graph theory;

[0015] Step S7, the blood vessel segmentation result after processing in step S6 is combined with the original image data after pre-processing in step S1 and the local registration information obtained in step S4.

[0016] As a preferred scheme of the present application, wherein: the CT image sequence is angiography CTA or venography CTV image, and the MRI image sequence is magnetic resonance angiography MRA or magnetic resonance venography MRV image;

[0017] The pre-processing includes removing scalp and skull, noise filtering and normalizing the image gray value;

[0018] In the double-branch deep neural network in step S2:

[0019] Each branch uses encoder-decoder structure to extract multi-scale features, wherein the multi-scale features refer to local and global image representation extracted at different levels;

[0020] The encoder part uses multi-layer convolution and pooling operation, and the decoder part uses upsampling and jump connection to restore details.

[0021] As a preferred scheme of the brain blood vessel full-automatic reconstruction method based on the multi-modal image fusion technology, in the step S3, the cross attention module is used to normalize the input CT branch feature and the MRI branch feature, point multiplication operation is used to calculate the similarity matrix between the two, and a local fusion weight is generated by using a Softmax function,

[0022] According to the above, the branch features are weighted and fused, and a set of fused multi-modal feature maps are output.

[0023] The fused feature maps are used in the subsequent steps and play a role in enhancing the low contrast signal of the tumor compression area.

[0024] As a preferred scheme of the brain blood vessel full-automatic reconstruction method based on the multi-modal image fusion technology, in the step S3, the cross attention module is used to normalize the input CT branch feature and the MRI branch feature, point multiplication operation is used to calculate the similarity matrix between the two, and a local fusion weight is generated by using a Softmax function,

[0025] In the double-branch deep network, the multi-scale features extracted from the CT branch are:

[0026] X C ∈R n×d , R n×d represents a matrix composed of n rows and d columns,

[0027] The multi-scale features extracted from the MRI branch are:

[0028] X M ∈R n×d ;

[0029] The normalized features are obtained by normalizing each feature matrix:

[0030]

[0031] and

[0032] wherein, X C represents the feature matrix output by the CT branch, X M represents the feature matrix output by the MRI branch, n is the number of features, d is the dimension of each feature, |X C |2 and |X M |2 respectively represent the Euclidean norm of the corresponding matrix;

[0033] The similarity matrix between the normalized features of the two branches is calculated by using point multiplication operation, and the formula is:

[0034]

[0035] wherein S∈R n×n , S ij represents the similarity between the i-th CT feature and the j-th MRI feature, and T represents the transpose operation;

[0036] A Softmax function is applied to the similarity matrix to generate a local fusion weight matrix:

[0037]

[0038] wherein W ij represents the weight of the j-th element in the i-th row, k is the summation index, and n is the range of values;

[0039] Feature fusion is performed by weighted summation, and the fused multi-modal feature map F is output, represented as:

[0040] F=W·X M +(I-W)·X C ,

[0041] wherein W represents the local fusion weight matrix, F∈R n×d is the fused feature map, and I is an n×n unit matrix.

[0042] As a preferred scheme of the brain blood vessel full-automatic reconstruction method based on multi-modal image fusion technology, the local adaptive registration mechanism in step S4 comprises:

[0043] a) Global registration is performed on the entire pre-processed image, so that the CT image and the MRI image are aligned in the overall space;

[0044] b) The region mutual information and the local structure constraint are used to automatically detect the region with local deformation caused by tumor compression,

[0045] c) A thinning registration algorithm is used on the detected region to correct the local deformation and match the blood vessel direction before and after the compression region deformation, and a local registration adjustment result is output.

[0046] As a preferred scheme of the brain blood vessel full-automatic reconstruction method based on multi-modal image fusion technology, in step S4, the local adaptive registration is performed in the following manner:

[0047] For the local deformation in the tumor compression region, region detection and thinning registration are used, the local region is denoted as R, the joint probability distribution of the CT and MRI images corresponding to the local region is denoted as p XY (x,y), the respective edge probabilities are p X (x) and p Y (y), and the local mutual information is defined as I(R):

[0048]

[0049] where x, y are pixel values in the local region, I(R) reflects the local statistical correlation, and it is detected that the region R exists deformation when I(R) < τ, τ is a preset threshold value;

[0050] In the detection region, suppose that the local registration transformation function is T d (·), wherein d is a transformation parameter vector, and the MRI image feature X M (R) and the CT image feature X C (R) in the local region are solved to obtain optimal registration parameters, and a least mean square error problem is solved:

[0051]

[0052] where d * is the optimal transformation parameter, T d (X M (R)) represents the feature of the MRI image in the region R after being transformed by T d , and the output local registration adjustment result is recorded as F reg .

[0053] As a preferred scheme of the brain blood vessel full-automatic reconstruction method based on the multi-modal image fusion technology, in the step S5, the multi-modal fusion image is an image obtained by adaptively weighting and fusing the CT image and the MRI image and correcting local deformation.

[0054] As a preferred scheme of the brain blood vessel full-automatic reconstruction method based on the multi-modal image fusion technology, in the step S6, the graph theory post-processing includes:

[0055] d) constructing a blood vessel network graph model, wherein a node represents a blood vessel end point or a bifurcation point, and an edge represents a blood vessel segment,

[0056] e) detecting non-connected nodes or broken regions existing in the blood vessel network,

[0057] f) automatically calculating and completing the broken blood vessel part by using a shortest path algorithm, and verifying the connectivity of the completion result.

[0058] As a preferred scheme of the brain blood vessel full-automatic reconstruction method based on the multi-modal image fusion technology, in the step S6, the graph theory post-processing includes:

[0059] Suppose that the binary blood vessel segmentation image obtained in the step S5 is recorded as B ∈ {0, 1} m×n×pBased on this, a vascular network diagram G = (V, E) is constructed, where V represents the set of vascular endpoints or bifurcation points and E represents the set of vascular segments.

[0060] For any edge e∈E, assign an edge weight function:

[0061] w(e)=α·f(e),

[0062] Where w(e) reflects the weight of edge e, α is the scaling factor, and f(e) is a function that reflects the discontinuity of features within the region corresponding to the edge;

[0063] For disconnected nodes or broken regions detected in the network, let v be the two nodes to be completed. i and v j Solve the shortest path problem:

[0064]

[0065] in, Indicates all connected nodes v i With v j The set of paths, P is the connection v i ,v j Any candidate path, P * To find the path with the minimum weight, after completing the broken blood vessel section, the connectivity of the entire blood vessel network is verified, and finally the complete network structure is output.

[0066] As a preferred embodiment of the fully automated human brain vascular reconstruction method based on multimodal image fusion technology described in this invention, in step S7, the combination method is as follows:

[0067] Standard tools (such as the image processing software packages Nibabel and VTK) are used to perform three-dimensional reconstruction of the combined data, and output a three-dimensional model of the human brain blood vessels for clinical surgical planning.

[0068] The processed blood vessel segmentation result refers to the binarized blood vessel image obtained in steps S5 and S6 after improved segmentation and post-processing.

[0069] The beneficial effects of this invention are as follows: This invention uses a dual-branch deep neural network combined with local adaptive registration and graph theory post-processing to solve the problem of incomplete reconstruction caused by low contrast, local deformation and vascular rupture in the tumor compression area in traditional human cerebral vascular reconstruction. After preprocessing CT and MRI images, each branch uses an encoder-decoder structure to extract multi-scale features, and then introduces a cross-attention module to achieve feature normalization and adaptive fusion, thereby making full use of the advantages of the two modalities and enhancing the signal performance of weak blood vessels in the tumor compression area.

[0070] The application also automatically detects local deformation regions caused by tumor compression in the image on the basis of global registration, uses regional mutual information and local structure constraint, corrects the regions by using a refinement registration algorithm, and effectively restores the local blood vessel direction.

[0071] In addition, the application automatically detects and completes the blood vessel fracture caused by segmentation error by constructing a blood vessel network graph and using a shortest path algorithm, repairs the disconnected part in the network, and makes the constructed three-dimensional blood vessel model have a continuous and complete network structure.

[0072] The application effectively overcomes the disadvantages of low contrast, local deformation and blood vessel fracture in the tumor compression area in the traditional method, significantly improves the meticulousness and overall connectivity of the blood vessel segmentation, provides a more accurate and complete three-dimensional blood vessel model for neurosurgery planning, and has a good clinical application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0073] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0074] Figure 1 The figure is a flowchart of the method for automatically reconstructing human brain blood vessels based on multi-modal image fusion technology. DETAILED DESCRIPTION

[0075] In order to make the above-mentioned purposes, features and advantages of the application more apparent and easy to understand, the specific embodiments of the application will be described in detail below with reference to the drawings.

[0076] In the following description, many specific details are set forth in order to provide a thorough understanding of the application, but the application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the application, therefore the application is not limited to the specific embodiments disclosed below.

[0077] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the application. In this specification, "in one embodiment" appearing in different places does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0078] Embodiment 1, refer to Figure 1The embodiment provides a human brain blood vessel full-automatic reconstruction method based on a multi-modal image fusion technology, and comprises the following steps

[0079] In step S1, a CT image sequence and a magnetic resonance imaging (MRI) image sequence of a human brain are acquired, and the images are preprocessed;

[0080] In step S2, a double-branch deep neural network is constructed, a first branch of the double-branch deep neural network is used for processing the CT image preprocessed in step S1, and a second branch of the double-branch deep neural network is used for processing the MRI image preprocessed;

[0081] The CT image sequence is a CT angiography (CTA) or a CT venography (CTV) image, and the MRI image sequence is a magnetic resonance angiography (MRA) or a magnetic resonance venography (MRV) image;

[0082] The preprocessing comprises removing a scalp and a skull, noise filtering and normalization processing of image gray values;

[0083] In the double-branch deep neural network in step S2:

[0084] Each branch adopts an encoder-decoder structure to extract multi-scale features, wherein the multi-scale features refer to local and global image representations extracted at different levels;

[0085] The encoder part adopts multi-layer convolution and pooling operations, and the decoder part adopts up-sampling and skip connection to restore details;

[0086] In step S3, the multi-scale features from each branch are taken as input in the middle layer of the double-branch deep neural network, and a cross-attention module is introduced to perform feature fusion;

[0087] In step S3, the cross-attention module performs standardization processing on the input CT branch features and MRI branch features, adopts dot multiplication operation to calculate a similarity matrix between the CT branch features and the MRI branch features, and generates local fusion weights by using a Softmax function,

[0088] Accordingly, the branch features are weighted and fused, and a set of fused multi-modal feature maps are output;

[0089] The fused feature maps are used in subsequent steps and play a role in enhancing low-contrast signals of tumor compression areas;

[0090] In step S3, the step of performing standardization processing on the input CT branch features and MRI branch features, adopting dot multiplication operation to calculate a similarity matrix between the CT branch features and the MRI branch features, and generating local fusion weights by using a Softmax function is,

[0091] In the double-branch deep network, let the multi-scale features extracted from the CT branch be:

[0092] XC ∈R n×d , R n×d denotes a matrix composed of n rows and d columns,

[0093] The multi-scale features extracted from the MRI branch are:

[0094] X M ∈R n×d ;

[0095] The normalization processing is performed on each feature matrix to obtain:

[0096]

[0097] and

[0098] wherein, X C denotes the feature matrix output by the CT branch, X M denotes the feature matrix output by the MRI branch, n is the number of features, d is the dimension of each feature, |X C |2 and |X M |2 respectively represent the Euclidean norm of the corresponding matrix;

[0099] The dot product operation is used to calculate the similarity matrix between the normalized features of the two branches, and the formula is:

[0100]

[0101] wherein, S ∈ R n×n , S ij denotes the similarity between the i-th CT feature and the j-th MRI feature, and T denotes the transpose operation;

[0102] The Softmax function is applied to the similarity matrix to generate a local fusion weight matrix:

[0103]

[0104] wherein, W ij denotes the weight of the j-th element in the i-th row, k is the summation index, and the value range is 1 to n;

[0105] The feature fusion is performed through weighted summation, and the fused multi-modal feature map F is output, which is represented as:

[0106] F = W·X M +(I-W)·X C ,

[0107] wherein, W refers to the local fusion weight matrix, F ∈ R n×d is the fusion feature map, and I is an n×n unit matrix;

[0108] Specifically, the multi-scale features of CT and MRI branches are first normalized to eliminate the scale difference between them, and then the point multiplication is used to calculate the local feature similarity. The local fusion weight can adaptively reflect the importance of two modalities at each feature position through the Softmax function, and the weighted fusion is performed accordingly. The final output of the fusion feature map enhances the low contrast signal in the tumor compression area to some extent. In the feature fusion process, the information advantages of the two modalities are fully utilized, and the weight distribution is automatically adjusted, so that more accurate vascular structure information can be obtained in the subsequent segmentation stage. The whole process is realized through end-to-end training without manual weight setting;

[0109] Step S4, for local deformation in the tumor compression area, a local adaptive registration mechanism is implemented;

[0110] The local adaptive registration mechanism in step S4 includes:

[0111] a) Global registration is performed on the entire preprocessed image to align the CT image and the MRI image in the overall space;

[0112] b) The regional mutual information and local structure constraint are used to automatically detect the region of local deformation caused by tumor compression,

[0113] c) A thinning registration algorithm is used for the detected region to correct the local deformation and match the blood vessel direction before and after the compression area deformation, and the local registration adjustment result is output;

[0114] In step S4, the local adaptive registration is performed in the following manner:

[0115] For local deformation in the tumor compression area, region detection and thinning registration are used. Let the local region be R, and the joint probability distribution of the CT and MRI images corresponding to the local region be p XY (x,y), the respective edge probabilities be p X (x) and p Y (y), and the local mutual information be I(R):

[0116]

[0117] where x and y are pixel values in the local region, I(R) reflects the local statistical correlation, and when I(R)<τ, it is determined that the region R has deformation, and τ is a preset threshold;

[0118] In the detected region, let the local registration transformation function be T d (·), where d is the transformation parameter vector, and the MRI image feature X M (R) and the CT image feature X C(R) to solve the least mean square error problem:

[0119]

[0120] where d * is the optimal transformation parameter, T d (X M (R)) represents the features of the MRI image in region R after transformation T d , and the output of the local registration adjustment result is denoted as F reg .

[0121] Specifically, the local adaptive registration mechanism automatically detects the local deformation region caused by tumor compression by calculating the local mutual information using the statistical features in the region. When the detection index is lower than the preset threshold, it indicates that there is obvious mismatch between the two modal images in the region. A refinement registration algorithm is used to obtain the locally optimal transformation parameter by solving the least mean square error problem, so as to correct the MRI image locally.

[0122] Step S5, using the fusion feature map output by step S3 and the local registration adjustment result obtained in step S4, a multi-modal fusion image is generated through fusion processing; the multi-modal fusion image is subjected to threshold adaptive segmentation using an improved watershed algorithm to extract the cerebral vascular region;

[0123] The multi-modal fusion image in step S5 is the image after adaptive weighted fusion and local deformation correction of the CT image and the MRI image;

[0124] Step S6, the blood vessel segmentation result obtained in step S5 is subjected to graph theory post-processing;

[0125] The graph theory post-processing in step S6 includes:

[0126] d) constructing a vascular network graph model, wherein the nodes represent the vascular end points or bifurcation points, and the edges represent the vascular segments,

[0127] e) detecting the non-connected nodes or broken regions existing in the vascular network,

[0128] f) automatically calculating and completing the broken vascular parts using the shortest path algorithm, and verifying the connectivity of the completed result;

[0129] The graph theory post-processing of step S6 includes:

[0130] Let the binary blood vessel segmentation graph obtained in step S5 be denoted as B∈{0,1} m×n×p , and on this basis, a vascular network graph G=(V,E) is constructed, wherein V represents the set of vascular end points or bifurcation points, and E represents the set of vascular segments;

[0131] For any edge e∈E, assign an edge weight function:

[0132] w(e)=α·f(e),

[0133] Where w(e) reflects the weight of edge e, α is the scaling factor, and f(e) is a function that reflects the discontinuity of features within the region corresponding to the edge;

[0134] For disconnected nodes or broken regions detected in the network, let v be the two nodes to be completed. i and v j Solve the shortest path problem:

[0135]

[0136] in, Indicates all connected nodes v i With v j The set of paths, P is the connection v i ,v j Any candidate path, P * To find the path with the minimum weight, after completing the broken blood vessel section, the connectivity of the entire blood vessel network is verified, and the complete network structure is finally output.

[0137] Specifically, graph theory post-processing constructs a vascular network graph based on the vascular segmentation results, using the endpoints and bifurcation points of the vessels as nodes and vascular segments as edges. By setting edge weights, the continuity of the local structure is reflected. The shortest path algorithm is used to automatically complete the detected broken areas, which helps to restore the connectivity of broken vessels. The connectivity verification step ensures that the completed vascular network conforms to the actual distribution of the anatomical structure. This provides an effective automated solution for handling local breaks caused by segmentation errors, enabling the final 3D vascular model to accurately reflect the continuous structure of the vascular network, and has high robustness and practicality.

[0138] Step S7: Combine the blood vessel segmentation results processed in step S6 with the preprocessed original image data in step S1 and the local registration information obtained in step S4.

[0139] In step S7, the combination method is as follows:

[0140] Standard tools (such as the image processing software packages Nibabel and VTK) are used to perform three-dimensional reconstruction of the combined data, and output a three-dimensional model of the human brain blood vessels for clinical surgical planning.

[0141] The processed blood vessel segmentation result refers to the binarized blood vessel image obtained in steps S5 and S6 after improved segmentation and post-processing.

[0142] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A method for full-automatic reconstruction of human brain blood vessels based on a multi-modal image fusion technique, characterized in that: The application relates to a method for extracting brain blood vessels from multi-modal images. The method comprises the following steps: S1, acquiring a CT image sequence and a magnetic resonance imaging (MRI) image sequence of a human brain, and preprocessing the images; S2, constructing a double-branch deep neural network, wherein a first branch is used for processing the CT images preprocessed in step S1, and a second branch is used for processing the MRI images preprocessed in step S1; S3, in the middle layer of the double-branch deep neural network, inputting multi-scale features from each branch as input, introducing a cross-attention module to perform feature fusion; S4, performing a local adaptive registration mechanism for local deformation in a tumor compression area; S5, generating a multi-modal fusion image through fusion processing by using the fusion feature map output in step S3 and the local registration adjustment result obtained in step S4; performing threshold adaptive segmentation on the multi-modal fusion image by using an improved watershed algorithm to extract a brain blood vessel region; S6, performing graph theory post-processing on the blood vessel segmentation result obtained in step S5; S7, combining the blood vessel segmentation result processed in step S6 with the original image data preprocessed in step S1 and the local registration information obtained in step S4; The local adaptive registration mechanism in step S4 comprises the following steps: a) performing global registration on the whole preprocessed image to align the CT image and the MRI image in the overall space; b) detecting a region with local deformation caused by tumor compression by using regional mutual information and local structure constraint, c) correcting the blood vessel direction before and after deformation of the compression area by using a thinning registration algorithm on the detected region to output a local registration adjustment result; For local deformation in tumor compression area, region detection and thinning registration are used. The local region is denoted as R, and the joint probability distribution of CT and MRI images in the local region is denoted as p XY (x,y), and the edge probability of each is p X (x) and p Y (y), and the local mutual information is defined as I(R): In step S4, the local adaptive registration is performed in the following manner: In the detection region, let the local registration transformation function be T d (·), where d is the transformation parameter vector, and find the optimal registration parameters for the MRI image features X M (R) and CT image features X C (R) in the local region by solving the least mean square error problem: where d * is the optimal transformation parameter, T d (X M (R)) represents the features of the MRI image in region R after transformation T d , and the output local registration adjustment result is denoted as F reg .

2. The method of claim 1, wherein the method is a full-automatic reconstruction method of human cerebral vessels based on a multi-modal image fusion technique. Wherein, x and y are pixel values in a local region, I(R) reflects local statistical correlation, and when I(R) < tau, it is determined that the region R has deformation, and tau is a preset threshold value; The CT image sequence is a CT angiography (CTA) or CT venography (CTV) image, and the MRI image sequence is a magnetic resonance angiography (MRA) or magnetic resonance venography (MRV) image; The preprocessing comprises removing a scalp and a skull, noise filtering and normalization processing on image gray values; In the double-branch deep neural network in step S2: Each branch adopts an encoder-decoder structure to extract multi-scale features, wherein the multi-scale features refer to local and global image representations extracted at different levels; 3. The method of claim 2, wherein the method is a full-automatic reconstruction method of human cerebral vessels based on a multi-modal image fusion technique. The encoder part adopts multi-layer convolution and pooling operations, and the decoder part adopts up-sampling and jump connection to restore details. In step S3, the cross-attention module performs standardization processing on the input CT branch features and MRI branch features, calculates a similarity matrix between the two by using point multiplication operation, and generates local fusion weights by using a Softmax function, 4. The method of claim 3, wherein the method is characterized in that: Accordingly, the branch features are weighted and fused to output a set of fused multi-modal feature maps. In step S3, the step of performing standardization processing on the input CT branch features and MRI branch features, calculating a similarity matrix between the two by using point multiplication operation, and generating local fusion weights by using a Softmax function is as follows: In the dual-branch deep network, let the multi-scale features extracted from the CT branch be: X C ∈R n×d , R n×d denotes a matrix consisting of n rows and d columns, The multi-scale features extracted from the MRI branch are: X M ∈R n×d ; The normalization processing is performed on each feature matrix to obtain: and where X C represents the feature matrix of CT branch output, X M represents the feature matrix of MRI branch output, n is the number of features, d is the dimension of each feature, |X C |2and |X M |2respectively represent the Euclidean norm of the corresponding matrix; The similarity matrix between the normalized features of the two branches is calculated using the dot product operation, and the formula is: where S ∈ R n×n , S ij denotes the similarity between the i-th CT feature and the j-th MRI feature, and T denotes the transpose operation. The Softmax function is applied to the similarity matrix to generate a local fusion weight matrix: wherein W ij represents the weight of the jth element in the ith row, k is a summation index, and takes a value ranging from 1 to n; Feature fusion is performed by weighted summation, and the fused multi-modal feature map F is output, represented as: F = W X M + (I - W) X C , where W denotes a local fusion weight matrix, F e R n×d is the fused feature map, and I is an n x n identity matrix.

5. The method of claim 4, wherein the method is a full-automatic reconstruction method of human cerebral vessels based on a multi-modal image fusion technique. The multi-modal fused image in step S5 is an image obtained after adaptive weighted fusion and local deformation correction of the CT image and the MRI image.

6. The method of claim 5, wherein the method is a full-automatic reconstruction method of human cerebral vessels based on a multi-modal image fusion technique. The graph theory post-processing in step S6 includes: d) Constructing a vascular network graph model, wherein the nodes represent the vascular endpoints or bifurcation points, and the edges represent the vascular segments, e) Detecting non-connected nodes or broken areas existing in the vascular network, f) Automatically calculating and completing the broken vascular parts using the shortest path algorithm, and verifying the connectivity of the completion results.

7. The method of claim 6, wherein the method is a full-automatic reconstruction method of human cerebral vessels based on a multi-modal image fusion technique. The graph theory post-processing in step S6 includes: Let the binary vessel segmentation map obtained in step S5 be denoted as B ∈ {0, 1} m×n×p On this basis, a vessel network graph G = (V, E) is constructed, where V represents a set of vessel endpoints or bifurcation points, and E represents a set of vessel segments. For any edge e∈E, a edge weight function is given: w(e)=α·f(e), where w(e) represents the weight of edge e, α is a scaling factor, and f(e) is a function reflecting the feature discontinuity in the region corresponding to the edge; For the non-connected nodes or broken areas detected in the network, suppose the two nodes to be completed are v i and v j , solve the shortest path problem: wherein, represents all connected nodes v i and v j , P is the path set of v i , v j is an arbitrary candidate path P * with the minimum weight, and the entire vascular network is verified for connectivity after the broken vascular portion is completed, and the final output is the complete network structure.

8. The method of claim 7, wherein the method is a full-automatic reconstruction method of human cerebral vessels based on a multi-modal image fusion technique. In step S7, the combination mode is: Using standard tools to perform three-dimensional reconstruction on the combined data, and outputting a three-dimensional model of the cerebral blood vessels for clinical surgical planning; The processed vascular segmentation result refers to the binary vascular image obtained after improved segmentation and post-processing in steps S5 and S6.

Images