Brain target positioning method based on dual-mode MRI (Magnetic Resonance Imaging) and cyclic convolutional neural network

Abstract

The invention discloses a brain target positioning method based on dual-mode MRI (Magnetic Resonance Imaging) and a cyclic convolutional neural network. According to the method, one mode can be processed independently, and magnetic resonance imaging data of the T1 mode and the T2 mode can be processed at the same time; the overall structure of the invention adopts a cyclic convolutional neural network, and can directly process an image sequence; according to the method, a single neural network is used, the classification step and the positioning step are integrated, and the calculation efficiency is further improved; according to the method, a high-resolution network structure is used, so that the calculation precision is improved; according to the method, a more perfect data preprocessing technology is used, and the robustness of the method is improved. Through the innovations, the precision, the efficiency and the flexibility of the method are further improved on the basis of the method based on the convolutional neural network.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and relates to a method for locating a target point for deep brain electrical stimulation, and specifically to an improved method for locating a target point for deep brain electrical stimulation based on dual-mode magnetic resonance imaging (MRI) and a recurrent convolutional neural network. Background Art

[0002] Essential tremor (ET) is one of the most common movement disorders. Its global incidence is about 0.4%-6% [5]. The first choice for ET treatment is medication, but only about half of patients can significantly improve their symptoms after taking medication. For ET patients who are not responsive to medication, deep brain stimulation (DBS) is another possible treatment. The basic idea of ​​DBS is to use electrodes to stimulate specific locations in the brain (see Figure 1 ). These locations are called targets. The choice of targets plays an important role in the treatment effect. The classic target area for DBS treatment of ET is the ventral intermediate nucleus (Vim), but the clinical effects vary, with tremor improvement rates ranging from 48% to 57% [7]. Some scholars also recommend the use of the posterior subthalamic area (PSA) as the DBS target area for the treatment of ET. Its symptom improvement rate can reach up to 81% [6]. A recent randomized double-blind clinical trial comparing the treatment effects of Vim and PSA targets found that PSA-DBS is at least as effective as Vim-DBS and may be more effective than Vim-DBS [1]. However, in different studies, the specific stimulation sites in the PSA area vary greatly, mainly because the target is not visible on magnetic resonance imaging (MRI) and is usually within an artificially defined range. Several teams have proposed different target calculation methods, but the optimal target locations obtained are significantly different, and the clinical symptom improvement rates range from 55% to 81% [6,2,8,7]. Therefore, the optimal stimulation target based on PSA is still unknown. In clinical practice, most neurosurgeons still use visual inspection and experience based on MRI to determine the target location, but the accuracy varies.

[0003] How to determine the location of the optimal target within the PSA region is a cutting-edge issue in today's medical field and is also a basic issue faced by the present invention. There are currently several solutions to this problem.

[0004] 1. Visual inspection;

[0005] Experienced doctors can determine the approximate location of the target by visual inspection based on MRI images.

[0006] 2.DRTT analysis;

[0007] Based on diffusion tensor imaging (DTI) studies, the dentato-rubro-thalamic tract (DRTT) in PSA is considered to be the anatomical basis for PSA-DBS to reduce tremor[3,4]. Different research teams have used MRI to locate the best stimulation targets based on anatomical structures and found that they are all located in the center or near the edge of the DRTT fibers[7]. Therefore, a new high-precision target location method has emerged, called DRTT analysis[9], which first draws the DRTT structure and then determines the target location based on the DRTT.

[0008] 3. Target estimation based on convolutional neural network;

[0009] Another technology has emerged based on DRTT analysis, which is to use convolutional neural networks to estimate the target position based on magnetic resonance imaging data. This method uses machine learning to summarize the results of DRTT analysis to obtain the regularity of target position. For details, please refer to Chinese patent CN202111386383.7 "Deep brain electrical stimulation target positioning method based on magnetic resonance imaging and neural network".

[0010] Disadvantages of background technology:

[0011] 1. Visual inspection;

[0012] Advantages: No need to rely too much on computer analysis, and the results are interpretable.

[0013] Disadvantages: Depends on the doctor's experience and condition, lacks accuracy guarantee, and cannot be implemented automatically.

[0014] 2.DRTT analysis;

[0015] Advantages: high accuracy, the results tend to be optimal.

[0016] shortcoming:

[0017] 1) Technical requirements: In addition to MRI, DTI technology is also required to generate DRTT structures. However, DTI technology is different from general MRI and is not a conventional medical examination method;

[0018] 2) Personnel requirements: Generating DRTT structures and determining target locations involve several complex data processing steps, which require manual operation by professionals with medical imaging knowledge;

[0019] 3) Time requirement: The whole process is slow, and it takes several hours or even more to process a case.

[0020] 3. Target estimation based on convolutional neural network;

[0021] advantage:

[0022] 1) Positioning accuracy is higher than visual inspection;

[0023] 2) The speed is higher than DRTT analysis;

[0024] 3) Compared with DRTT analysis, it is more convenient to use.

[0025] shortcoming:

[0026] 1) The positioning accuracy is lower than that of DRTT analysis;

[0027] 2) can only process one MRI mode (e.g., T1 or T2) and cannot utilize data from multiple modes;

[0028] 2) The overall structure uses a convolutional neural network, which can only process single images in sequence and cannot directly process image sequences;

[0029] 3) A two-step calculation structure is used, which requires the use of two neural networks (classification network and localization network).

[0030] In summary, the evaluation of target localization mainly considers two aspects: accuracy and efficiency. Visual inspection lacks both accuracy and efficiency. DRTT analysis technology has high accuracy but low efficiency. It is still in the academic stage and is still far from widespread clinical application. On the other hand, the method based on convolutional neural network has achieved a compromise between accuracy and efficiency, but it is still not perfect and there is a lot of room for improvement.

[0031] References:

[0032] [1]MTBarbe, P.Reker, S.Hamacher, J.Franklin, D.Kraus, TADembek, J.Becker, JKSteffen, N.Allert, J.Wirths, HSDafsari, J.Voges, GRFink, V.Visser-Vandewalle, and L.Timmermann. DBS of the PSA and the VIM in essential tremor:Arandomized,double-blind,crossover trial. Neurology, 91(6):e543–e550,082018.

[0033] [2]P.Blomstedt,U.Sandvik,and S.Tisch.Deep brain stimulation in theposterior subthalamic area in the treatment of essential tremor.Mov Disord,25(10):1350–1356,Jul 2010.

[0034] [3]V.A.Coenen,N.Allert,and B.Madler.A role of diffusion tensorimaging fiber tracking in deep brain stimulation surgery:DBS of the dentato-rubro-thalamic tract(drt)for the treatment of therapy-refractory tremor.ActaNeurochir(Wien),153(8):1579–1585,Aug 2011.

[0035] [4]V.A.Coenen,N.Allert,S.Paus,M.Kronenburger,H.Urbach,andB.Madler.Modulation of the cerebello-thalamo-cortical network in thalamicdeep brain stimulation for tremor:a diffusion tensor imagingstudy.Neurosurgery,75(6):657–669,Dec 2014.

[0036] [5]E D Louis.The roles of age and aging in essential tremor:Anepidemiological perspective.Neuroepidemiology,52(1-2):111–118,2019.

[0037] [6]J.Murata,M.Kitagawa,H.Uesugi,H.Saito,Y.Iwasaki,S.Kikuchi,K.Tashiro,and Y.Sawamura.Electrical stimulation of the posterior subthalamicarea for the treatment of intractable proximal tremor.J Neurosurg,99(4):708–715,Oct 2003.

[0038] [7]A.Nowacki,I.Debove,F.Rossi,J.A.Schlaeppi,K.Petermann,R.Wiest,M.Schupbach,and C.Pollo.Targeting the posterior subthalamic area foressential tremor:proposal for MRI-based anatomical landmarks.J Neurosurg,131(3):820–827,102018.

[0039] [8]P.Plaha,S.Javed,D.Agombar,G.O’Farrell,S.Khan,A.Whone,andS.Gill.Bilateral caudal zona incerta nucleus stimulation for essentialtremor:outcome and quality of life.J NeurolNeurosurg Psychiatry,82(8):899–904,Aug 2011.

[0040] [9]Juergen R.Schlaier, Anton L.Beer, Rupert Faltermeier, Claudia Fellner, Kathrin Steib, Max Lange, Mark W. Greenlee, Alexander T. Brawanski, and Judith M. Anthofer. Probabilistic vs. deterministic fiber tracking and the influence of different seed regions to delineate cerebellar-thalamic fibers in deep brain stimulation. European Journal of Neuroscience, 45(12):1623–1633, 2017. Summary of the invention

[0041] In view of the shortcomings of the prior art, the present invention provides a brain target localization method based on dual-mode MRI and recurrent convolutional neural network. The purpose of the present invention is to design a machine learning algorithm based on neural network and deep learning, and estimate the optimal target position for deep brain stimulation (DBS) therapy based on magnetic resonance imaging (MRI) data of two modes (T1 and T2), and surpass the accuracy of the background technology (mainly targeting methods based on convolutional neural networks). Figure 2 Demonstration diagram for target positioning.

[0042] The method of the present invention can process one mode alone or simultaneously process magnetic resonance imaging data of both T1 and T2 modes; the overall structure of the present invention adopts a recurrent convolutional neural network, which can directly process image sequences; the present invention uses a single neural network to combine the classification and positioning steps into one, further improving the computational efficiency; the present invention uses a high-resolution network structure to improve the computational accuracy; the present invention uses a more complete data preprocessing technology to improve the robustness of the method. Through these innovations, the method of the present invention further improves the accuracy, efficiency and flexibility over the aforementioned convolutional neural network-based methods.

[0043] Complete technical solution:

[0044] The method of the present invention estimates the optimal DBS target position based on the MRI data of the patient's head. The data of an MRI scan is composed of many image layers, but the target may only exist in a few layers. In addition, conventional MRI scans can usually be divided into two modes, T1 and T2, which show different emphases. The basic idea of ​​the present invention is: using two-mode MRI image sequences as input, while judging whether each image layer may have a target, estimate the target for each image layer, and then select the target estimation on the image layer that is most likely to contain the target. This dual-mode processing method can further improve the accuracy, which is very different from the single-mode processing method in the background technology. This idea can be specifically expanded as follows: Figure 3 Flowchart in .

[0045] The core of the present invention is to use a recurrent neural network for image sequences, called a "classification and positioning network". The input of this network is two (one when running in single mode) MRI image sequences of length N: one is T1 mode and the other is T2 mode, corresponding to two scans of the same patient, and it is assumed that the brain positions corresponding to any i-th layer of mode 1 and mode 2 are consistent. The function of the classification and positioning network is to determine whether each image layer in the input image sequence contains the target, and at the same time estimate the position of the optimal target on each image layer in the input image sequence. Its output is an array sequence of length N, in which each element is an array (A, B) containing two parts: the first part (i.e. A) is called the classification score, which can be understood as the probability that the corresponding image layer contains the target, and the larger it is, the more likely it is to contain the target; the second part (i.e. B) is a two-dimensional coordinate pair, corresponding to the two-dimensional target coordinates of the left and right brains respectively. The two parts are represented separately, and the output sequence can be split into a score sequence and a target coordinate pair sequence. The index of the image layer that is most likely to contain the target is combined with the two-dimensional coordinate output for the layer to uniquely determine the three-dimensional coordinates of the target. The method of using one neural network to simultaneously complete the classification and positioning tasks improves the implementation efficiency and is very different from the method of using two neural networks separately in the background technology.

[0046] The basic process of the present invention is as follows:

[0047] Step 1: Data preprocessing;

[0048] Step 2: Image sequence classification and target location;

[0049] For the input dual-mode MRI image layer sequences T1 and T2 of a single patient, a classification and localization network is used to simultaneously calculate the possibility that each image layer contains the target, as well as the target coordinate estimation of each image layer; the output of the classification and localization network is saved and split into a sequence of component values ​​and a sequence of target coordinate pairs.

[0050] Step 3: Select the top K layers that are most likely to contain the target and output the corresponding target coordinates.

[0051] Calculate the final score of each image layer according to the score sequence, and set the score below T to 0; select the K with the largest final score in the image layer, and output the corresponding target coordinates; if no layer is selected, terminate the algorithm and switch to manual analysis;

[0052] Step 4: Verification and output;

[0053] Verify whether the target is standardized according to the pre-set rules; if all the targets are not standardized, terminate the algorithm and switch to manual analysis; output the verified target coordinates and the corresponding layer index;

[0054] Furthermore, the specific method of step 1 is as follows:

[0055] The preprocessing of MRI images includes the following steps:

[0056] a) Image resampling: resample all images to a uniform voxel size.

[0057] b) Denoising: Remove noise from MRI data.

[0058] c) Bias Field Correction: Corrects the uneven distribution of image brightness.

[0059] d) Image registration: Register the image to a standard space (such as MNI space) for comparison with other images.

[0060] e) Normalization: Normalize the pixel intensities of the image so that they are on the same scale. Divide all pixel values ​​by x% (e.g. 95%) of the maximum pixel value, so that all pixel values ​​are limited to between 0 and 1 (values ​​exceeding 1 are taken as 1).

[0061] f) Skull Stripping: Remove non-brain tissue from the image.

[0062] Since the size range of the target is relatively small (several millimeters), it only exists in a few layers of MRI data, and most of the image layers of head MRI are actually irrelevant to the target location. Therefore, the layer indexes with target are counted, and a range is set artificially, and the image layers outside this range are ignored.

[0063] All retained MRI image layers are cropped to a set standard size.

[0064] Furthermore, the classification and positioning network structure of step 2 is as follows:

[0065] The classification and positioning network generally adopts the "circular convolution" neural network structure. The network consists of two parts: a recurrent neural network part and a dual-mode convolutional neural network. Among them, the dual-mode convolutional neural network part is responsible for processing the dual-mode image sequence input for feature extraction and outputting feature sequences; the recurrent neural network part is responsible for comprehensively processing the output sequence of the dual-mode convolutional neural network and calculating the target position. The dual-mode convolutional neural network part includes two branches with the same structure (corresponding to T1 and T2 modes respectively) and a fusion block (concatenation). A single branch is based on the known high-resolution network (HRNet) structure, mainly consisting of a convolution layer and two high-resolution blocks (HR blocks). Each high-resolution block is based on the known residual network (ResNet) structure, consisting of several residual blocks, and multi-scale information exchange is realized through the residual block (repeated n times); the output data of the two branches are fused through a fusion block (concatenation) to obtain a dual-mode feature vector. The recurrent neural network is mainly composed of multiple fully connected layers.

[0066] The last layer of the classification and positioning network is a fully connected layer, which is used to process the output of the recurrent neural network. The output result is a 5-dimensional array, which can be split into a 1-dimensional array and a 4-dimensional array.

[0067] 1) A 1-dimensional array is defined as the classification score. Due to the sigmoid function, the classification score is between 0 and 1, which is the probability that the input image contains the target. Different MRI image layers can be sorted according to the score.

[0068] 2) The 4-dimensional array can be converted into a 2x2-dimensional form, that is, two 2D coordinates, corresponding to the positions of the left and right targets. Due to the effect of the sigmoid function, the coordinate values ​​are between 0 and 1, which are the coordinates normalized according to the length and width of the image. Assuming that the normalized coordinates are (x, y), the resolution of the image is H×W, and the length and width of the pixel are p and q respectively, the actual coordinates are (xHp, yWq).

[0069] Furthermore, supervised learning is used to train the classification and localization network, which requires a certain number of MRI images with known target locations for training and verification. A stochastic gradient descent algorithm based on mini batches is used to optimize the network parameters using back-propagation. The known target locations of the MRI images are the optimal targets obtained using DRTT analysis technology and manually annotated;

[0070] Furthermore, the DRTT analysis technique requires three types of magnetic resonance data, including T1-MRI, T2-MRI and diffusion tensor imaging (DTI) data. The specific process is as follows.

[0071] T1-MRI and T2-MRI preprocessing process: The default MRI image original layer thickness is 0.7–1.0 mm. MRI is corrected according to the AC-PC line. The midpoint of the AC-PC line is set as the origin of the coordinate axis and re-layered.

[0072] Diffusion tensor imaging preprocessing process: First, deterministic fiber tracking was performed. The generalized q-sampling imaging method was used to reconstruct the magnetic resonance diffusion tensor imaging data to obtain the tensor within each voxel. Rigid transformation was used to align the MRI and diffusion tensor imaging data. Three regions of interest were defined on the MRI: 1) manually segmented ipsilateral dentate nucleus; 2) manually segmented ipsilateral PSA area; 3) ipsilateral precentral gyrus automatically extracted using FreeSurfer DKT atlas. The seed point was set to the ipsilateral dentate nucleus. The minimum and maximum fiber lengths of deterministic fiber tracking were set to 20 and 200 mm, respectively. The angle threshold was set to 55, the step size was set to 0.5, and the seed tracking upper limit was set to 50,000. The reconstructed DRT fibers were manually adjusted to remove anatomically unreasonable fiber bundles.

[0073] Optimal target annotation: MRI and reconstructed fiber bundles were fused and exported as a new MRI_label image. The MRI_label image was opened in FSLview software, and the midpoints of bilateral DRT fibers were identified at the level with the largest diameter of the red nucleus and 5 levels above and below (a total of 11 levels), and the coordinates were recorded, saved, and exported.

[0074] Beneficial effects brought about by the technical solution of the present invention:

[0075] There are three main existing target localization methods, namely the traditional method (clinicians analyze MRI images with the naked eye and based on experience), the DRTT analysis method (using DTI for fiber bundle angiography), and the method of using convolutional neural networks to locate targets. Compared with the background technology, the present invention can achieve a better balance between accuracy and efficiency. In terms of accuracy, the target localization accuracy of the present invention can exceed that of the traditional method, and is closer to the target localization accuracy of DRTT analysis than the new method based on convolutional neural networks. In terms of efficiency, the analysis speed of the scheme of the present invention far exceeds that of the traditional method, and is better than or close to the latest convolutional neural network method; since the analysis of the entire magnetic resonance image sequence can be completed at one time, the convenience has been greatly improved, and it is currently the most efficient method. The present invention is mainly aimed at essential tremor (ET), but is also applicable to other similar diseases, such as Parkinson's disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 is a schematic diagram of DBS;

[0077] Figure 2 This is a demonstration of target positioning. The white mark indicated by the arrow is the real target, and the red mark is the estimated target. The numbers are distance errors. The classification score (confidence) of the MRI image layer is shown in the lower right corner;

[0078] Figure 3 This is a flowchart of the technical solution of an embodiment of the present invention;

[0079] Figure 4 is a schematic diagram of MRI data representation;

[0080] Figure 5 This is an example of the structure of a recurrent convolutional neural network according to an embodiment of the present invention;

[0081] Figure 6 This is an example of a dual-mode convolutional neural network structure according to an embodiment of the present invention;

[0082] Figure 7 This is an example of a recurrent neural network structure according to an embodiment of the present invention;

[0083] Figure 8 Schematic diagram of the high-resolution block and residual block structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0084] The following is a detailed description and supplementary description of the details of the solution not mentioned in the technical solution of the present invention in combination with the drawings and embodiments.

[0085] Representation of MRI data and target locations

[0086] The basic working mode of MRI is to scan the human body on continuous sections while the human body is moving. Therefore, a complete set of MRI images is composed of many stacked grayscale images. The interval between two adjacent layers depends on the equipment and is fixed and known, such as 0.7 mm. Human head MRI usually contains hundreds of image layers. They can be viewed as a sequence of two-dimensional arrays or as a three-dimensional array. Relatively speaking, the position of the target can be expressed as a combination of a two-dimensional coordinate and a layer index, or as a three-dimensional coordinate. The solution of the present invention adopts the former to represent the target position, which estimates the two-dimensional coordinates of the target in the layer on the premise that the layer is selected. Figure 4 Schematic diagram of MRI data representation. The target position can be determined by coordinates (x, y) and layer index z.

[0087] The brain target localization method based on dual-mode MRI and recurrent convolutional neural network includes the following steps:

[0088] Step 1: Image preprocessing;

[0089] The preprocessing of MRI images includes the following steps:

[0090] a) Image resampling: resample all images to a uniform voxel size, such as 0.7 mm.

[0091] b) Denoising: Use Gaussian smoothing, non-local mean filtering (NLM) and other methods to remove noise from MRI data.

[0092] c) Bias Field Correction: Use methods such as N4ITK and FASTMRI to correct the uneven distribution of image brightness.

[0093] d) Image registration: Use linear registration and nonlinear registration algorithms to register images to a standard space (such as MNI space) for comparison with other images.

[0094] e) Normalization: Normalize the pixel intensities of the image so that they are on the same scale. Divide all pixel values ​​by x% (e.g. 95%) of the maximum pixel value, so that all pixel values ​​are limited to between 0 and 1 (values ​​exceeding 1 are taken as 1). On this basis, in order to match the input of the neural network, the data can be further standardized, for example, to make the mean close to zero and the variance close to one.

[0095] f) Removal of non-brain tissue (Skull Stripping): Use tools such as BET (FSL) and 3dSkullStrip (AFNI) to remove non-brain tissue such as skull and skin from the image.

[0096] These preprocessing steps can help improve the accuracy and stability of subsequent image analysis, making the features extracted from MRI data more reliable.

[0097] Since the size range of the target is relatively small (several millimeters), it only exists in a few layers of MRI data. Most of the image layers of head MRI are actually irrelevant to the target location. Therefore, we can count the layer indexes where the target exists, set a range artificially, and ignore the image layers beyond this range. This can reduce unnecessary calculations.

[0098] On the other hand, an MRI image may contain a lot of useless black blank areas. In order to reduce data redundancy and match subsequent machine learning, a standard size needs to be specified. All MRI image layers need to be cropped to reach the standard size. For example, the standard image layer size adopted in an embodiment of the present invention is about 220x280 (pixel spacing is 0.7 to 1 mm). A relatively simple cropping method is fixed position cropping. Since the patient's head is not necessarily in the middle of the image, this method may cut off the edges of a small part of the useful image (i.e., the non-blank part in the image). Another possible but more complicated cropping method is to center the non-blank part in the image before cropping.

[0099] Step 2: Neural network structure;

[0100] The classification and positioning network in the solution of the present invention adopts a "circular convolution" neural network structure. This type of network is mainly composed of a cyclic neural network part and a dual-mode convolutional neural network part. Figure 5 This is the overall schematic diagram of this type of network. The dual-mode convolutional neural network part is responsible for processing the dual-mode image sequence input for feature extraction and outputting the feature sequence; the recurrent neural network part is responsible for comprehensively processing the output sequence of the dual-mode convolutional neural network and calculating the target position. Figure 6 is a specific example of a convolutional neural network. The dual-mode convolutional neural network consists of two branches with the same structure (corresponding to the T1 and T2 modes respectively) and a fusion block (concatenation). A single branch is based on the known high-resolution network (HRNet) structure, mainly consisting of a convolution layer and two high-resolution blocks (HR blocks). Each high-resolution block is based on the known residual network (ResNet) structure, consisting of several residual blocks (residual blocks), and its structure is Figure 8As shown in the figure (left), the residual block realizes multi-scale information exchange (repeated n times); the output data of the two branches are fused through a fusion block (concatenation) to obtain a dual-mode feature vector. The residual block is also mainly composed of convolutional layers, and its specific structure is as follows: Figure 8 (right side). Figure 7 It is a specific example of a recurrent neural network, which is mainly composed of multiple fully connected layers.

[0101] In addition to convolutional layers and fully connected layers, the example diagram also uses some other common neural network operations, including max pooling, ReLU, sigmoid, batch normalization, and concatenation. Their descriptions and some basic parameter settings are listed in Table 1. Since these are standard operations in the field of neural networks and deep learning, their specific mathematical definitions are explained in relevant literature (such as "Dive into Deep Learning" https: / / d2l.ai), and will not be described in detail in this document.

[0102] In practice, classification and localization networks can also use different backbone structures. For example, a relatively simple way is to change the network structure by changing the number of high-resolution blocks, residual blocks, or fully connected layers in the network.

[0103] Table 1 Description of the components of the neural network in the embodiment.

[0104]

[0105] The last layer of the classification and positioning network is a fully connected layer, which is used to process the output of the recurrent neural network. The output result is a 5-dimensional array, which can be split into a 1-dimensional array and a 4-dimensional array.

[0106] 1) A 1-dimensional array is defined as the classification score. Due to the sigmoid function, the classification score is between 0 and 1, which can be understood as the probability that the input image contains the target. Different MRI image layers can be sorted according to the score.

[0107] 2) The 4-dimensional array can be converted into a 2x2-dimensional form, that is, two 2D coordinates, corresponding to the positions of the left and right targets. Due to the effect of the sigmoid function, the coordinate values ​​are between 0 and 1, which can be understood as the coordinates normalized according to the length and width of the image. Assuming that the normalized coordinates are (x, y), the resolution of the image is H×W, and the length and width of the pixel are p and q respectively, the actual coordinates are (xHp, yWq).

[0108] Step 3: Training of classification and positioning network;

[0109] Neural network algorithms need to be trained before they can be used. The purpose of training is to determine the parameters of the network. This scheme adopts supervised learning, and requires a certain number of MRI images with known target positions for training and verification. Generally, the more training data, the better the verification effect. The embodiment in this scheme used about 150 groups of head MRI scan images in the experiment, of which 4 / 5 were randomly selected for training and the rest were used for verification, and good results were obtained. This scheme adopts a stochastic gradient descent algorithm based on minibatch, for example, 5 to 10 groups of training data are used each time, and back-propagation is used to optimize the network parameters. These methods are standard operations in neural network calculations, and their specific mathematical definitions are explained in relevant literature and will not be described in detail in this document. The known target position data of the MRI image is the optimal target obtained and manually annotated using DRTT analysis technology;

[0110] Step 4: DRTT analysis;

[0111] One of the key points of this solution is to use DRTT analysis technology to obtain the optimal target and perform manual annotation. This technology requires three types of magnetic resonance data, including T1-MRI, T2-MRI and diffusion tensor imaging (DTI) data. In the case of missing data from one MRI mode, the analysis can still be completed using another MRI mode. The specific process is as follows.

[0112] T1-MRI and T2-MRI preprocessing process: The default original layer thickness of MRI images is 0.7–1.0 mm. MRI is corrected according to the AC-PC line using dedicated software (e.g., SPM 12.0 based on MATLAB). The midpoint of the AC-PC line is set as the origin of the coordinate axis and re-layered.

[0113] Diffusion tensor imaging preprocessing process: Deterministic fiber tracking was performed using dedicated software (e.g., DSI studio). The MRI diffusion tensor imaging data were reconstructed using the generalized q-sampling imaging method to obtain the tensor within each voxel. The MRI and diffusion tensor imaging data were registered using a rigid transformation. Three regions of interest were defined on the MRI: 1) the manually segmented ipsilateral dentate nucleus; 2) the manually segmented ipsilateral PSA region; and 3) the ipsilateral precentral gyrus automatically extracted using the FreeSurfer DKT atlas. The seed point was set to the ipsilateral dentate nucleus. The minimum and maximum fiber lengths for deterministic fiber tracking were set to 20 and 200 mm, respectively. The angle threshold was set to 55, the step size was set to 0.5, and the seed tracking upper limit was set to 50,000. The reconstructed DRT fibers were manually adjusted to remove anatomically unreasonable fiber bundles.

[0114] Optimal target annotation: MRI and reconstructed fiber bundles were fused and exported as a new MRI_label image. The MRI_label image was opened in FSLview software, and the midpoints of bilateral DRT fibers were identified at the level with the largest diameter of the red nucleus and 5 levels above and below (a total of 11 levels), and the coordinates were recorded, saved, and exported.

[0115] Step 5: Target positioning is performed through the trained classification and positioning network;

[0116] For the input dual-mode MRI image layer sequences T1 and T2 of a single patient, a classification and localization network is used to simultaneously calculate the possibility that each image layer contains the target, as well as the target coordinate estimation of each image layer; the output of the classification and localization network is saved, and different MRI image layers are sorted according to the classification scores to obtain a score sequence; the actual coordinates of the target are determined based on the obtained positions of the left and right targets, to obtain a sequence of target coordinate pairs.

[0117] Calculate the final score of each image layer according to the score sequence, and set the score below T to 0; select the K with the largest final score in the image layer, and output the corresponding target coordinates; K is a parameter (integer), usually less than or equal to the maximum number of layers that may contain targets, such as 11. If no layer is selected, the algorithm is terminated and manual analysis is performed.

[0118] Step 6: Verification and output;

[0119] Verify whether the target is standardized according to the pre-set rules; if all the targets are not standardized, terminate the algorithm and switch to manual analysis; output the verified target coordinates and the corresponding layer index.

[0120] Pre-set rules: In order to reduce the generation of unreasonable target location results, some simple target verification rules need to be defined. These rules represent situations where the target cannot appear. For example, a target in the left brain cannot appear in the right brain. The target verification rules are determined according to the corresponding relevant specifications in the medical field.

[0121] Outside the basic process, some low-probability events may also occur. For example, the classification score of the selected layer is too low (i.e., less than T). T is a normalized threshold parameter (floating point number), usually between 0 and 1. Another case is that the target location obtained by the positioning network does not conform to common sense (i.e., some basic assumptions) and cannot pass subsequent verification. When these low-probability events occur, the results obtained by the algorithm will be considered invalid; if all results are invalid, manual analysis is required.

[0122] The method of the present invention can be used for fully automatic target location, or it can be used for semi-automatic target location in cooperation with doctors. There are three specific usage modes:

[0123] 1. The classification network selects the appropriate layer, and the doctor locates the target on the selected layer.

[0124] 2. The doctor selects the appropriate layer, and the positioning network locates the target on the selected layer.

[0125] 3. The classification network selects the appropriate layer, and the positioning network locates the target on the selected layer.

[0126] It can be seen that the first two methods are part of the last method, and the last method is the complete solution described above. In addition, the method of the present invention can be used for both dual-mode data and single-mode data, specifically in the following three ways:

[0127] 1. Input data is in T1 mode (T2 mode input is empty).

[0128] 2. Input data is in T2 mode (T1 mode input is empty).

[0129] 3. The input data is in T1 and T2 dual mode.

[0130] The terms involved in the above analysis process are explained in the relevant neurological medicine and medical imaging literature and will not be described in detail in this document.

[0131] The above contents are further detailed descriptions of the present invention in combination with specific / preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, they can also make several substitutions or modifications to these described embodiments without departing from the concept of the present invention, and these substitutions or modifications should be regarded as belonging to the protection scope of the present invention.

[0132] Parts of the present invention that are not described in detail belong to the well-known technologies of those skilled in the art.

Claims

1. A brain target localization method based on dual-mode MRI and recurrent convolutional neural network, characterized in that: The steps include: Step 1: Data preprocessing; Step 2: Image sequence classification and target location; For the input dual-mode MRI image layer sequences T1 and T2 of a single patient, a classification and localization network is used to simultaneously calculate the possibility that each image layer contains the target, as well as the target coordinate estimation of each image layer; the output of the classification and localization network is saved and split into a sequence of component values ​​and a sequence of target coordinate pairs; Step 3: Select the top K layers that are most likely to contain the target and output the corresponding target coordinates; Calculate the final score of each image layer according to the score sequence, and set the score below T to 0; select the K with the largest final score in the image layer, and output the corresponding target coordinates; if no layer is selected, terminate the algorithm and switch to manual analysis; Step 4: Verification and output; Verify whether the target is standardized according to the pre-set rules; if all the targets are not standardized, terminate the algorithm and switch to manual analysis; output the verified target coordinates and the corresponding layer index.

2. The method for brain target localization based on dual-mode MRI and recurrent convolutional neural network according to claim 1, characterized in that: Step 1: The preprocessing of MRI images includes the following steps: a) Image resampling: resample all images to a uniform voxel size; b) Denoising: removing noise from MRI data; c) Bias field correction: Corrects the uneven distribution of image brightness; d) Image registration: registering an image to a standard space for comparison with other images; e) Normalization: Normalize the pixel intensity of the image to make it on the same scale; Divide all pixel values ​​by x% of the maximum pixel value, so that all pixel values ​​are limited to between 0 and 1, and values ​​exceeding 1 are set to 1; f) Remove non-brain tissue: remove non-brain tissue from the image; The layer indexes with target points are counted and a range is set artificially, and the image layers beyond this range are ignored; all the retained MRI image layers are cropped to the set standard size.

3. The method for brain target localization based on dual-mode MRI and recurrent convolutional neural network according to claim 1, characterized in that: The classification and positioning network structure of step 2 is as follows: The classification and positioning network generally adopts the "circular convolution" neural network structure; the network consists of a recurrent neural network part and a dual-mode convolutional neural network. Among them, the dual-mode convolutional neural network part is responsible for processing the dual-mode image sequence input for feature extraction and outputting the feature sequence; the recurrent neural network part is responsible for comprehensively processing the output sequence of the dual-mode convolutional neural network and calculating the target position; the dual-mode convolutional neural network part includes two branches with the same structure and a fusion block; a single branch is based on a known high-resolution network structure, mainly consisting of a convolution layer and two high-resolution blocks; each high-resolution block is based on a known residual network structure, consisting of several residual blocks, and multi-scale information exchange is achieved through the residual block; the output data of the two branches are fused through a fusion block to obtain a dual-mode feature vector; the recurrent neural network is mainly composed of multiple fully connected layers; The last layer of the classification and positioning network is a fully connected layer, which is used to process the output of the recurrent neural network. The output result is a 5-dimensional array, which can be split into a 1-dimensional array and a 4-dimensional array; among them: 1) A 1-dimensional array is defined as the classification score; due to the sigmoid function, the classification score is between 0 and 1, which is the probability that the input image contains the target; different MRI image layers can be sorted according to the score; 2) The 4-dimensional array can be converted into a 2x2-dimensional form, that is, two 2D coordinates, corresponding to the positions of the left and right targets respectively; due to the effect of the sigmoid function, the coordinate values ​​are between 0 and 1, which are the coordinates normalized according to the length and width of the image; assuming that the normalized coordinates are (x, y), the resolution of the image is H×W, and the length and width of the pixel are p and q respectively, then the actual coordinates are (xHp, yWq).

4. The method for brain target localization based on dual-mode MRI and recurrent convolutional neural network according to claim 3, characterized in that: Supervised learning is used to train the classification and localization network, and a certain number of MRI images with known target locations are required for training and verification. A small-batch-based stochastic gradient descent algorithm is used to optimize the network parameters using back propagation. The known target locations of the MRI images are the optimal targets obtained using DRTT analysis technology and manually annotated.

5. The method for brain target localization based on dual-mode MRI and recurrent convolutional neural network according to claim 4, characterized in that: The DRTT analysis technology requires three types of magnetic resonance data, including T1-MRI, T2-MRI and diffusion tensor imaging data; the specific process is as follows; T1-MRI and T2-MRI preprocessing process: The default original layer thickness of MRI images is 0.7–1.0 mm; MRI is corrected according to the AC-PC line; the midpoint of the AC-PC line is set as the origin of the coordinate axis and re-layered; Diffusion tensor imaging preprocessing process: first, deterministic fiber tracking is performed; Generalized q-sampling imaging was used to reconstruct MRI diffusion tensor imaging data to obtain the tensor within each voxel; rigid transformation was used to align MRI and diffusion tensor imaging data; three regions of interest were defined on MRI: 1) manually segmented ipsilateral dentate nucleus; 2) manually segmented ipsilateral PSA region; 3) ipsilateral precentral gyrus automatically extracted using FreeSurfer DKT atlas; the seed point was set to the ipsilateral dentate nucleus; the minimum and maximum fiber lengths of deterministic fiber tracking were set to 20 and 200 mm, respectively; the angle threshold was set to 55, the step size was set to 0.5, and the seed tracking upper limit was set to 50,000; the reconstructed DRT fibers were manually adjusted to remove anatomically unreasonable fiber bundles; Optimal target annotation: MRI and reconstructed fiber bundles were fused and exported as a new MRI_label image; the MRI_label image was opened in the FSLview software, and the midpoints of the bilateral DRT fibers were identified at the level with the largest diameter of the red nucleus and the five levels above and below, and the coordinates were recorded, saved, and exported.

Images