Intracranial MRA blood vessel reconstruction system, electronic equipment and storage medium

Abstract

The invention discloses an intracranial MRA blood vessel reconstruction system, electronic equipment and a storage medium, the intracranial MRA blood vessel reconstruction system comprises a data collection module, a data preprocessing module, a segmentation model mixed training module and a model reasoning module, the data collection module is used for screening patient data and labels suitable for mixed training; the data preprocessing module is used for processing the patient data; the segmentation model mixed training module is used for carrying out mixed training on the segmentation model by utilizing the existing segmentation data; and the model reasoning module is used for segmenting the brain blood vessel in the input MRA image by using the segmentation model after mixed training to obtain a reconstruction result. By the adoption of the technology, based on mixed training, the requirement for MRA training data can be lowered, the precision of MRA blood vessel reconstruction is improved, the diagnosis efficiency and the diagnosis accuracy are improved, a doctor can formulate an operation scheme according to the three-dimensional structure of the brain blood vessel, and the success rate of an operation is increased.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to an intracranial MRA blood vessel reconstruction system, electronic equipment and storage medium. Background Art

[0003] With the development of medical imaging technology, cerebrovascular imaging technology has been widely used in clinical applications, greatly promoting the prevention and treatment of cerebrovascular diseases. Timely detection of abnormal changes in cerebrovascular morphology through imaging technology can avoid sudden diseases to the greatest extent, which is of great significance for the prevention and early treatment of cerebrovascular diseases.

[0004] Magnetic Resonance Imaging (MRI) is a technology that uses the nuclear magnetic resonance phenomenon of hydrogen protons in the body under specific radio frequency pulses to perform imaging. It is divided into conventional scanning and functional scanning. Magnetic Resonance Angiography (MRA) is a conventional scan that is mainly used to reflect the anatomical morphology of blood vessels. Because of its advantages such as simple imaging and no radiation, it is more common in cerebrovascular examinations, especially in the screening of cerebrovascular diseases.

[0005] At present, clinical analysis of cerebrovascular structure is mainly based on doctors observing the information in the images. This method is highly dependent on the doctor's anatomical foundation and film reading experience, and requires clinical training and experience accumulation of a large number of cases. It also requires the evaluation of multiple professional skills such as symptomatology to effectively diagnose and implement treatment plans. In addition, due to the complex shape and structure of cerebrovascular vessels, variable size and curvature, multiple branches in a small range, and fine terminal branches, when the blood flow is slow and the blood vessels are slender, the imaging quality is often reduced due to noise, bias fields, etc., which also increases the difficulty of image observation.

[0006] An intuitive and clear means of observation is to segment and reconstruct the three-dimensional structure of cerebral blood vessels. The three-dimensional reconstruction of cerebral vascular structure allows doctors to intuitively understand the vascular information of the patient's lesion area in a non-invasive way, including the identification of arterial stenosis, detection and analysis of aneurysms, which is critical for accurate clinical diagnosis and successful implementation of surgery. Accurate cerebral vascular segmentation is an important prerequisite for three-dimensional reconstruction. However, due to the complex geometric structure of cerebral blood vessels, manual cerebral vascular segmentation is a voxel-based task, which consumes a lot of manpower and economic costs. Therefore, fully automatic high-precision vascular reconstruction is an urgent function to be solved and realized in the clinical application of intracranial MRA.

[0007] With the continuous improvement of computer technology and cerebrovascular imaging databases, deep learning segmentation models have developed rapidly since the advent of Fully Convolutional Networks (FCN) in 2015, and cerebrovascular segmentation methods based on deep learning have also been proposed one after another. Compared with traditional theoretical methods, deep learning drives models by mining the semantic information of cerebrovascular images, and has the advantages of high accuracy and strong generalization. However, due to the huge number of parameters of deep neural networks, the data required for training and adjusting network parameters is also huge. Compared with natural images, the data scale of medical images is usually several orders of magnitude smaller, and the amount of intracranial MRA data is particularly small, making it difficult to collect and form a large-scale training data set, which is a huge challenge in achieving high-precision automatic vascular reconstruction. Summary of the invention

[0008] The purpose of the present invention is to provide a cerebrovascular reconstruction system based on deep neural network hybrid training, which uses a relatively large number of intracranial CTA data and MRA data that are relatively easy to obtain to simultaneously train the network, reduce the demand for MRA data, improve the accuracy of network reconstruction of blood vessels, and provide doctors with more accurate information for detecting and diagnosing cerebrovascular diseases, so as to solve the problems raised in the above-mentioned background technology.

[0009] To achieve the above-mentioned purpose, one aspect of the present invention provides an intracranial MRA vascular reconstruction system, including a data collection module, a data preprocessing module, a segmentation model hybrid training module, and a model reasoning module, wherein:

[0010] The data collection module is used to screen patient data and labels suitable for hybrid training;

[0011] The data preprocessing module is used to process the patient data;

[0012] The segmentation model hybrid training module uses the existing segmentation data to perform hybrid training on the segmentation model;

[0013] The model inference module uses the segmentation model after hybrid training to segment the cerebral blood vessels in the input MRA image to obtain a reconstruction result.

[0014] Furthermore, the data suitable for hybrid training in the data collection module includes human brain MRA modality images and other modality images including intracranial blood vessels, and are assigned corresponding cerebral vascular masks and modality type labels.

[0015] Furthermore, the data preprocessing module includes processing methods such as resampling, threshold clipping, maximum and minimum value normalization, block processing and data enhancement.

[0016] Furthermore, during mixed training, MRA modality images and other modality images will simultaneously participate in the training of the segmentation model. The segmentation model consists of an input part, a downsampling part, an upsampling part, and an output part. The segmentation model includes UNet, VNet, and their variants. The output part contains two branches. The first branch is used to output the vascular reconstruction results, and the second branch is used to output the predicted results of the modality type.

[0017] Furthermore, during hybrid training, the second branch of the segmentation model output part will adapt the model features to different types of modal data, and its implementation method is a domain adaptation method, including adding a gradient reversal layer method and a generative adversarial network method.

[0018] Furthermore, during hybrid training, the segmentation model receives preprocessed MRA images and other model images as input, and outputs vascular reconstruction results and modality type prediction results at the output end. The error distance between them and the cerebrovascular mask and the modality type label is calculated to obtain the segmentation loss and domain loss. The loss functions used include dice, cross entropy and their variants to train the segmentation model.

[0019] Furthermore, in the model inference module, the output part of the segmentation model only retains the first branch and outputs the vascular reconstruction result. If the hardware performance cannot input the entire image at one time, different areas of the image are input multiple times in a sliding window manner, and the results of different areas are merged to obtain the overall result.

[0020] Furthermore, the gradient reversal layer uses an adversarial training mechanism to adapt the model features to data of different modalities: in forward propagation, the gradient reversal layer is an identity transformation; in backward propagation, it multiplies the gradient by a negative number, so that the training objectives before and after the layer are opposite, achieving an adversarial effect; the gradient reversal layer is represented by a pseudo function R δ (x), its forward propagation and back propagation formulas are as follows:

[0021] R δ (x) = x

[0022] dR δ / dx=-δI

[0023] Among them, δ=2 / [1+exp(-γ*p)]-1, I is a unit matrix, p represents the ratio of the current number of iterations to the total number of iterations, and γ is a constant, usually 10.

[0024] Another aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an intracranial MRA vascular reconstruction system as described in any one of claims 1 to 8 is implemented.

[0025] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an intracranial MRA vascular reconstruction system according to any one of claims 1 to 8.

[0026] Compared with the prior art, the present system and method have the following advantages:

[0027] 1. The intracranial MAR reconstruction system based on hybrid training proposed in this patent, compared with the method based only on MRA image training, can utilize CTA images and MRA images together to participate in the training of the neural network model, thereby reducing the demand for MRA images, reducing the difficulty of data collection, and improving the vascular reconstruction capability of the model.

[0028] 2. The intracranial MRA reconstruction system proposed in this patent realizes hybrid training based on the gradient inversion layer, uses the feature extraction capability of the CTA image training model, and at the same time guarantees and improves the accuracy of the model's vascular reconstruction on the MRA image based on the MRA data. Neurologists can use the method of this patent to automatically extract intracranial blood vessels, better observe and diagnose patients, and improve diagnostic efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a structural diagram of an intracranial MRA vascular reconstruction system.

[0030] Figure 2 Schematic diagram of hybrid training of segmentation model.

[0031] Figure 3 This is a curve chart of the changes in different indicators during the hybrid training process.

[0032] Figure 4 Figure 2 shows the test cases and comparison results. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] like Figure 1 The figure shows the system structure diagram of an embodiment of the present invention. Our cranial MRA vascular reconstruction system mainly includes the following modules: data collection module, data preprocessing module, segmentation model hybrid training module, and model reasoning module.

[0035] The data collection module is used to filter out human brain MRA image sequences and CTA brain images from the patient's multi-sequence data, store them in NIfTI format, and assign cerebrovascular masks and modality type labels to these images respectively. The image data of the two modalities may not be paired data.

[0036] The data preprocessing module uses SimpleITK to read images of MRA or CTA modalities, resample the voxels of different modality images to the same spacing, and perform voxel threshold clipping and maximum and minimum value normalization on the images to suppress the influence of outliers. After that, images of different sizes are further clipped or padded to the specified size to meet the input requirements of the model. For the training data, the cerebrovascular mask corresponding to the image will also undergo the same preprocessing operation to achieve matching between the image and the mask. After that, the training image will also choose whether to perform Gaussian noise enhancement, flip, scale, and rotate the image according to a certain probability, providing more perturbation samples for the training process and enhancing the generalization ability of the model. Among them, the upper and lower limits of the image voxel threshold clipping are the values ​​at 99.9% and 0.1% of the voxel value size arrangement, the mean of Gaussian noise enhancement is 0, the standard deviation is 0.1 times the standard deviation of the image voxel, the image scaling ratio is controlled within 15%, and the image rotation angle is controlled within 20 degrees.

[0037] In addition, since MRA and CTA images usually have higher resolution and larger spatial dimensions, it is difficult to use complete images directly for model training due to the limitation of video memory size. Therefore, the images are cut into blocks (for example, into 192*192*192 3D image blocks), and a random cutting strategy is implemented for the images during training.

[0038] Segmentation model hybrid training module, in one training epoch, the images in the training set are first divided into multiple batches according to a certain batch size, and are input into the vascular reconstruction model respectively. After the forward propagation of the network model, the output result is obtained, and the loss value is obtained by calculating the error distance between the output result and the label. Finally, the parameters of the model are optimized and updated through back propagation. The above process will iterate multiple epochs on the training set, and AdamW will be selected as the optimizer with a learning rate of 0.001 until the model loss converges.

[0039] The specific training process is as follows: Figure 2As shown, the input image is an MRA image or a CTA image, both of which are single-channel 3D images with a dimension of 1×d×h×w. The overall vascular segmentation model is an encoder-decoder architecture, which receives MRA or CTA images as input and outputs different prediction results through two branches at the output end. One branch outputs the prediction result of vascular segmentation, and calculates the error distance between it and the cerebral vascular mask to obtain the segmentation loss (segmentation loss, Ls). The other branch can predict the modality type of the image after passing through the gradient reversal layer (gradient reversal layer), and calculates the error distance between it and the modality type label to obtain the domain loss (domain loss, Ld). The segmentation loss can be measured by dice or cross entropy. The purpose of optimizing this loss is to improve the accuracy of vascular segmentation. The domain loss is usually calculated by cross entropy. It can narrow the data distribution differences between images of different modalities, thereby realizing the knowledge transfer from CTA images to MRA images and enhancing the model's vascular reconstruction performance on MRA images. During training, the above two types of losses are added together with a weight of 0.5 each. Figure 3 The changes of the two losses during training are shown.

[0040] Vessel segmentation models include but are not limited to UNet, VNet and other segmentation models and their variants. Figure 2 A typical model structure is shown in , which consists of an input part, a downsampling part, an upsampling part, and an output part. The input part includes a 3D convolution layer, a BatchNorm layer, and an activation function layer, which can transform a single-channel input image into a multi-channel image feature and improve the expressiveness of the image feature. The downsampling part includes 4 stages, each of which consists of a 3D convolution layer, a BatchNorm layer, an activation function layer, and a Dropout layer. The step size of the 3D convolution layer is 2. After each stage of downsampling, the number of channels of the image feature will double, and the remaining dimensions will become half of the original. The upsampling part also includes 4 stages, each of which consists of a 3D deconvolution layer, a BatchNorm layer, an activation function layer, and a Dropout layer. After each stage of the upsampling layer, the dimensional change of the image feature is exactly the opposite of that of the downsampling layer. In the corresponding stages of the downsampling part and the upsampling part, the image feature will also be transferred from the downsampling part to the upsampling part for feature merging. The output part contains two branches, one of which is used to output the vascular reconstruction results of the input image. It consists of a 3D convolution layer and a BatchNorm layer. The output dimension of this branch is consistent with the input image dimension, both of which are 1×d×h×w. The other branch is used to predict the modality type of the input image after passing through the gradient reversal layer. It consists of a pooling layer, a flatten layer, and a fully connected layer. The output length of this branch is the number of modality types.

[0041] The gradient reversal layer is the key to realize the mixed training of MRA images and CTA images. It uses the adversarial training mechanism to make the model features adapt to data of different modalities. In the forward propagation, the gradient reversal layer is an identity transformation. In the backward propagation, it multiplies the gradient by a negative number, so that the training objectives before and after the layer are opposite, achieving the adversarial effect. Formally, the gradient reversal layer can be regarded as a pseudo function R δ (x), its forward propagation and back propagation formulas are as follows:

[0042] R δ (x) = x

[0043] dR δ / dx=-δI

[0044] Among them, δ=2 / [1+exp(-γ*p)]-1, I is a unit matrix, p represents the ratio of the current number of iterations to the total number of iterations, and γ is a constant, usually 10.

[0045] In the model inference module, after the vascular segmentation model training is completed, the gradient reversal layer branch in the output module will be closed so that the model only outputs the vascular reconstruction results. During inference, the MRA image to be tested will first be processed according to the same resampling and threshold clipping operations in the data preprocessing module, and then directly input into the reconstruction model to obtain the segmentation results of the intracranial arteries. After clipping or filling and the inverse operation of voxel resampling, the cerebrovascular structure of the original image is reconstructed. When the hardware performance does not meet the requirements of inputting the entire image at one time, different regions of the image can be input multiple times in a sliding window manner to obtain the segmentation results of different regions. These segmentation results can be merged to obtain the cerebrovascular reconstruction results. In our experiment, after using 126 MRA images and 300 CTA images to train the vascular reconstruction model according to the above steps, the average accuracy of the 11 MRA images tested was 0.863. Figure 4 The vascular reconstruction result of an MRA image is further demonstrated, and the difference in results between hybrid training and MRA single modality training is compared. From the results in the figure, it can be seen that the hybrid training method we proposed can enhance the model's vascular reconstruction ability for MRA images.

[0046] Neurologists can use this method to automatically extract cerebral blood vessels, so as to better observe and diagnose patients' cerebral blood vessels, improve diagnostic efficiency and accuracy, and doctors can formulate surgical plans based on the three-dimensional structure of cerebral blood vessels to improve the success rate of surgery.

[0047] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intracranial MRA vascular reconstruction system, characterized in that: It includes data collection module, data preprocessing module, segmentation model hybrid training module, and model reasoning module, among which, The data collection module is used to screen patient data and labels suitable for hybrid training; The data preprocessing module is used to process the patient data; The segmentation model hybrid training module uses the existing segmentation data to perform hybrid training on the segmentation model; The model inference module uses the segmentation model after hybrid training to segment the cerebral blood vessels in the input MRA image to obtain a reconstruction result.

2. The intracranial MRA vascular reconstruction system according to claim 1, characterized in that: The data suitable for mixed training in the data collection module include human brain MRA modality images and other modality images containing intracranial blood vessels, and are assigned corresponding cerebral blood vessel masks and modality type labels.

3. An intracranial MRA vascular reconstruction system according to claim 1, characterized in that: The data preprocessing module includes resampling, threshold reduction, maximum and minimum value normalization, block processing and data enhancement.

4. The intracranial MRA vascular reconstruction system according to claim 1, characterized in that: During mixed training, MRA modality images and other modality images will participate in the training of the segmentation model at the same time. The segmentation model consists of an input part, a downsampling part, an upsampling part and an output part. The segmentation model includes UNet, VNet and its variants. The output part contains two branches. The first branch is used to output the vascular reconstruction results, and the second branch is used to output the prediction results of the modality type.

5. The intracranial MRA vascular reconstruction system according to claim 4, characterized in that: During hybrid training, the second branch of the segmentation model output part will adapt the model features to different types of modal data. The implementation method is the domain adaptation method, including adding the gradient reversal layer method and the generative adversarial network method.

6. The intracranial MRA vascular reconstruction system according to claim 2, characterized in that: During hybrid training, the segmentation model receives preprocessed MRA images and other modality images as input, and outputs vascular reconstruction results and modality type prediction results at the output end. The error distance between them and the cerebrovascular mask and the modality type label is calculated to obtain the segmentation loss and domain loss. The loss functions used include dice, cross entropy and their variants to train the segmentation model.

7. The intracranial MRA vascular reconstruction system according to claim 4, characterized in that: In the model inference module, the output part of the segmentation model only retains the first branch and outputs the vascular reconstruction result. If the hardware performance cannot input the entire image at one time, different areas of the image are input multiple times in a sliding window manner, and the results of different areas are merged to obtain the overall result.

8. The intracranial MRA vascular reconstruction system according to claim 4, characterized in that: The gradient reversal layer uses an adversarial training mechanism to adapt the model features to data of different modalities: in forward propagation, the gradient reversal layer is an identity transformation; in backward propagation, it multiplies the gradient by a negative number, making the training objectives before and after the layer opposite, achieving an adversarial effect; the gradient reversal layer is represented by a pseudo function R δ (x), its forward propagation and back propagation formulas are as follows: R δ (x)=x dR δ / dx=-δI Among them, δ=2 / [1+exp(-γ*p)]-1, I is a unit matrix, p represents the ratio of the current number of iterations to the total number of iterations, and γ is a constant, usually 10.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, an intracranial MRA vascular reconstruction system according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, an intracranial MRA vascular reconstruction system according to any one of claims 1 to 8 is implemented.

Images