An end-to-end automatic reorientation method for magnetic resonance brain images

Abstract

The application discloses an end-to-end magnetic resonance brain image automatic reorientation method, which comprises the following steps: reading in a brain image to be processed, adjusting the brain image to a standard shape, normalizing an image array of the standard shape brain image, inputting the normalized image array into a deep learning model to obtain a pre-result, combining the pre-result and an affine matrix equivalent to the process of adjusting the brain image to the standard shape and normalizing the image array of the standard shape brain image, constructing a transformation matrix contained by the brain image and the standard shape brain image after being reoriented, and finally packaging and outputting the reoriented brain image and the standard shape brain image. The application utilizes the deep learning technology, predicts the transformation matrix contained by the brain image after being reoriented through the image array of the brain image, and directly reorients the brain image, so that the process is simple and efficient, and the problems of complex process or excessive dependence on templates in the existing brain image automatic reorientation method are effectively solved.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of image processing, artificial intelligence and brain science, and more specifically, to an end-to-end automatic redirection method for magnetic resonance brain images. Background Technology

[0002] Magnetic Resonance Imaging (MRI) is a scanning technique that uses magnetic fields to visualize the structure and function of the human body in detail. Brain MRI images (hereinafter referred to as brain images) are digital measurements of the structure or function of the human brain obtained using MRI technology. Brain images generally include a voxel array, an image head, and a transformation matrix from the voxel coordinate system to the world coordinate system (hereinafter referred to as the transformation matrix). The voxel array is the image array. The image head contains metadata describing the brain image, including voxel dimensions and image array dimensions. The voxel coordinate system describes the storage location of voxel values ​​within the image array; the storage index is the voxel coordinate. The world coordinate system describes the spatial location of brain structures and is determined by the voxel coordinate system and the transformation matrix. A unified world coordinate system is fundamental for comparing and analyzing different brain images; it is often achieved by registering different brain images to a template brain image (hereinafter referred to as a template). The establishment of a template world coordinate system often relies on some landmark brain structures, such as the anterior commissure (AC) and the posterior commissure (PC).

[0003] For newly acquired brain images, the world coordinate system is usually the scanner coordinate system by default. Manually adjusting the brain image world coordinate system is an optional operation during brain image acquisition, but due to omissions, inconsistent operating standards, or non-standard operations, a significant deviation in position can occur between the brain image world coordinate system and the template world coordinate system, meaning that the brain structures they represent are very different. Preprocessing is a necessary step in brain image analysis, which includes using optimization algorithms to register the brain image to the template. If the brain image world coordinate system deviates significantly from the template world coordinate system, the optimization algorithm may get stuck in a local optimum, leading to registration failure and severely impacting the quality of the preprocessed brain image. Therefore, before preprocessing, it is usually necessary to adjust the brain image world coordinate system to be as close as possible to the template world coordinate system; this process is called reorientation.

[0004] The most popular brain image redirection method currently is manual adjustment, which involves manually adjusting the position of the brain image in the world coordinate system based on the brain structure. Manual adjustment standards are difficult to standardize, requires operators to have a background in brain anatomy, and is extremely time-consuming when dealing with a large number of brain images. Therefore, it is necessary to develop an automatic brain image redirection method. Existing automatic brain image redirection methods mainly fall into two categories. The first is based on template matching, which is mainly inaccurate, time-consuming, and dependent on templates. The second is based on brain structure segmentation and registration, such as CN109829922A, where segmentation uses deep learning methods. It is more accurate than the first method, but after segmenting the brain structure using deep learning methods, it still involves registration, regression, and other steps, making the process complex and dependent on templates. Summary of the Invention

[0005] To address the issues of automation in brain image redirection in existing technologies and the complexity or over-reliance on templates in current automatic brain image redirection methods, this invention provides an end-to-end automatic magnetic resonance brain image redirection method. This method utilizes deep learning technology to predict the transformation matrix of the redirected brain image, thereby directly redirecting the brain image and providing a simple and efficient new method for brain image redirection.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] An end-to-end automated retargeting method for magnetic resonance brain images includes the following steps:

[0008] S10. Read in the brain image N to be processed, which includes at least an image array A, a transformation matrix T and an image head H.

[0009] S20. Adjust brain image N to a standard shape: Adjust image array A to a standard-dimensional image array A1. Modify the dimensional information of image array A in image head H to the dimensional information of image array A1, obtain image head H1, and obtain the affine matrix T equivalent to the process of adjusting image array A to image array A1. c Obtain the transformation matrix Then, the standard-shaped brain image N1 is composed of the image array A1, the image head H1, and the transformation matrix T1;

[0010] S30, Normalized Image Array: Image array A1 is normalized into image array A2 through dimension unification and numerical normalization operations;

[0011] S40. Prediction: Input the image array A2 into the fully trained deep learning model to obtain the preliminary result P;

[0012] S50. Construct the transformation matrix: Combine the pre-result P with the affine matrix T, which is equivalent to the image array dimension unification process in step S30.s Construct the transformation matrix T2; combine the transformation matrix T2 with the affine matrix T in step S20. c Construct the transformation matrix T3;

[0013] S60. Output the redirected brain image: The encapsulated image array A1, image head H1 and transformation matrix T2 form brain image N2, which is the result of redirecting the standard-shaped brain image N1. The encapsulated image array A, image head H and transformation matrix T3 form brain image N3, which is the result of redirecting brain image N.

[0014] Specifically, the process of adjusting the brain image N to a standard shape in step S20 includes:

[0015] S21. Calculate the standard dimension: The resampled template with a voxel size of 1mm*1mm*1mm is resampled to the same voxel size as the brain image N. The dimension of the image array of the resampled template is the standard dimension of A.

[0016] S22. Based on the standard dimensions calculated in step S21, adjust the Z dimension, Y dimension and X dimension of image array A to the standard Z dimension, standard Y dimension and standard X dimension in sequence to obtain image array A1.

[0017] S23. Modify the dimension information of image array A in image head H to the dimension information of image array A1 to obtain image head H1;

[0018] S24. Find the affine matrix T equivalent to the process of adjusting image array A to image array A1. c ;

[0019] S25. According to the formula Calculate the transformation matrix T1. At this point, the image array A1, the image head H1, and the transformation matrix T1 form a standard-shaped brain image N1. The world coordinate system positions of brain image N1 and brain image N are the same.

[0020] Specifically, the process of sequentially adjusting the Z, Y, and X dimensions of image array A to standard Z, Y, and X dimensions in step S22 is as follows:

[0021] Adjust the Z-dimensionality of image array A to the standard Z-dimensionality, and set the Z-slice queue equivalent to image array A to Q. z ;

[0022] First calculate Q z Find the mean of each Z-slice in Q, then count the number m of Z-slices with a mean greater than zero. If m is less than or equal to the standard Z-dimension, remove the Q-slice. z The Z-slice with a mean less than or equal to zero yields the queue Q. z1 Then in Qz1 Fill the beginning and end with approximately the same number of Z-slices, each with a value of zero, so that Q... z1 The length of m is equal to the standard Z dimension, resulting in the Z-dimensional adjusted image array A; if m is greater than the standard Z dimension, Q is removed first. z The Z-slice with a mean less than or equal to zero yields the queue Q. z2 Then, following the order of Z-slice mean from smallest to largest, remove Q slices one by one. z2 The Z-slice with a smaller mean makes Q z2 The length is equal to the standard Z dimension, resulting in the Z-dimensional adjusted image array A;

[0023] Then, in the same way, adjust the Y dimension to the standard Y dimension on the image array A after the Z dimension has been adjusted, to obtain the image array A after the Z and Y dimensions have been adjusted. Finally, adjust the X dimension to the standard X dimension on the image array A after the Z and Y dimensions have been adjusted, to obtain the image array A1.

[0024] Specifically, in step S24, the affine matrix T equivalent to the process of adjusting image array A to image array A1 is obtained. c The process is as follows:

[0025] Set up X-slice queues, Y-slice queues, and Z-slice queues equivalent to image array A. During the adjustment of image array A to image array A1, the change in the head slice of each queue is c. X c Y and c Z A negative value indicates that a slice at the head of the queue has been deleted, corresponding to image array cropping; a positive value indicates that a new slice has been added to the head of the queue, corresponding to image array expansion. This affine matrix T c Represented as

[0026]

[0027] Specifically, step S30 includes the following process:

[0028] S31. Calculate the scaling ratio required for dimensional unification based on the voxel size of brain image N1 and the set target voxel size;

[0029] S32. Dimensional Unification: Scale image array A1 according to the scaling ratio to obtain image array A′1. Then, fill the tail of the equivalent X-slice queue of image array A′1 with slices containing all zero values ​​up to the set length to obtain image array A′. 1X In image array A′ 1X The equivalent Y-slice queue is padded with slices of all zero values ​​at the end to a set length to obtain image array A′. 1XY In image array A′ 1XYThe equivalent Z-slice queue is padded with slices of all zero values ​​at the end until a set length is reached, resulting in image array A″1;

[0030] S33, Numerical Normalization: Through the formula Normalize the image array A″1 to obtain the image array A2, where Q1, Q2, and Q3 represent the first quartile, median, and third quartile, respectively. Then, set the negative values ​​in the image array A2 to zero and cover the original image array A2.

[0031] Specifically, the process of obtaining the fully trained deep learning model in step S40 includes:

[0032] S41. Collect a sufficient number of brain images as training data;

[0033] S42. Adjust all collected brain images to standard shapes and construct dataset D from all standard-shaped brain images;

[0034] S43. Split the dataset D into a training dataset D1 and a pre-training dataset D2 according to a set ratio. The training dataset D1 is labeled with quantities that are directly related to the redirection task, and the pre-training dataset D2 is labeled with quantities that are not directly related to or unrelated to the redirection task but can be directly obtained.

[0035] S44. Create original labels for training dataset D1 based on expertise in brain anatomy;

[0036] S45. While ensuring the integrity of brain regions in the image array, rotate or translate the image array of brain images in training dataset D1 to obtain the brain images in enhanced training dataset D1′. Then, combine the original labels of the brain images in training dataset D1 with the affine matrix T equivalent to the rotation and translation operations. rt The original labels of the corresponding brain images in the augmented training dataset D1′ are obtained.

[0037] S46. Construct a deep learning model B based on the dimensions of the image array after unification in step S30.

[0038] S47. Pre-trained model: By adjusting the output dimension of deep learning model B to adapt to the label dimension of pre-training dataset D2, deep learning model B1 is obtained; the image array of brain images in pre-training dataset D2 is normalized, and then the normalized image array is used as input, combined with the labels of pre-training dataset D2, to train deep learning model B1.

[0039] S48. Training Model: Image arrays of brain images in normalized training dataset D1 and augmented training dataset D1′, and recording the affine matrix T. sThe original labels of the training dataset D1 and the augmented training dataset D1′ are transformed into new labels. Then, the output dimension of the deep learning model B1 is adjusted to adapt to the new label dimension of the training dataset D1 and the augmented training dataset D1′ to obtain the deep learning model B2. Then, the normalized image array is used as input, combined with the new labels of the training dataset D1 and the augmented training dataset D1′, and the deep learning model B2 is trained with the help of the transfer learning strategy.

[0040] S49. Repeatedly validate and test the deep learning model B2 to obtain a fully trained deep learning model.

[0041] As one approach, the process of creating original labels for the training dataset D1 based on expertise in brain anatomy in step S44 is as follows:

[0042] With the help of professionals with a background in brain anatomy, the AC and PC points of the brain images in D1 and any point M that is closer to the top of the head than the AC and PC points on the midsagittal plane of the brain are labeled. The original labels of the brain images are formed by the coordinates of the AC, PC and M points. After labeling all the brain images, the original labels of the training dataset D1 are obtained.

[0043] The new label in step S48 contains a normalized vector representing the R-axis direction, calculated from the original label. By T s The dot product of the original labels yields nine new coordinates for AC and PC.

[0044] Furthermore, the process of constructing the transformation matrix in step S50 is as follows:

[0045] The prediction results include 9 quantities, which will be processed first. Multiply the coordinates of AC and PC in the prediction result to obtain the coordinates of AC and PC in image array A1;

[0046] In image array A1, the coordinates of AC point, PC point coordinates, and the normalized vector in the prediction results are used. The position L1 of the redirected world coordinate system of brain image N1 is determined, and the position L2 of the original world coordinate system is determined by transformation matrix T1.

[0047] The affine matrix T is used to translate and rotate from position L1 to position L2. rt1 The transformation matrix T2 = T1·T is determined. rt1 And the transformation matrix T3 = T2·T c .

[0048] As an alternative approach, the process of creating original labels for the training dataset D1 based on expertise in brain anatomy in step S44 is as follows:

[0049] With the help of professionals with a background in brain anatomy, the brain images in D1 are manually redirected according to the set world coordinate system position of the brain images. The transformation matrix contained in the brain image after redirection is the original label of the brain image. After all brain images are redirected, the original label of the training dataset D1 is obtained.

[0050] The new label in step S48 contains the dot product of the original label transformation matrix. The resulting transformation matrix contains 12 quantities in the first three rows.

[0051] Furthermore, the process of constructing the transformation matrix in step S50 is as follows:

[0052] The prediction result P consists of 12 quantities. Arrange the prediction results into a 3*4 matrix in order, and add a row [0,0,0,1] to the end of the matrix to obtain matrix T. p Construct the transformation matrix T2 = T p ·T s And the transformation matrix T3 = T2·T c .

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] This invention utilizes deep learning technology to predict the transformation matrix of a brain image after redirection using an image array, thereby directly redirecting the brain image. This effectively solves the problems of automating brain image redirection and addressing the issues of complex processes or excessive reliance on templates in existing automatic brain image redirection methods. Furthermore, this invention is simple, efficient, and largely template-free, while also exhibiting good scalability and transferability, making it suitable for application in MRI brain image redirection. Attached Figure Description

[0055] Figure 1 This is a schematic diagram showing the positions of the world coordinate system and voxel coordinate system on the MNI template Y slice.

[0056] Figure 2 This is a schematic diagram showing the positions of the world coordinate system and voxel coordinate system on the MNI template X slice.

[0057] Figure 3 This is a flowchart illustrating an embodiment of the present invention.

[0058] Figure 4 This is a schematic diagram showing the positions of the world coordinate system and voxel coordinate system on the Y-slice of the brain image to be processed in an embodiment of the present invention.

[0059] Figure 5 This is a schematic diagram of the world coordinate system and voxel coordinate system positions on the X-slice of the brain image to be processed in an embodiment of the present invention.

[0060] Figure 6 This is a schematic diagram illustrating the adjustment of an image array to standard dimensions using a cropping method in an embodiment of the present invention.

[0061] Figure 7 This is a schematic diagram of the process of adjusting brain images to a standard shape in an embodiment of the present invention.

[0062] Figure 8 This is a schematic diagram of the location of point M on the Y-slice of the brain image to be processed in an embodiment of the present invention.

[0063] Figure 9 This is a schematic diagram of the location of point M on the X-slice of the brain image to be processed in an embodiment of the present invention. Detailed Implementation

[0064] The present invention will be further described below with reference to the accompanying drawings and embodiments. The embodiments of the present invention include, but are not limited to, the following embodiments.

[0065] Example

[0066] like Figures 1 to 9 As shown, this end-to-end automatic retargeting method for magnetic resonance imaging (MRI) brain images is based on deep learning technology, combined with brain anatomy and image processing techniques, to achieve automatic retargeting of brain images end-to-end. The brain image retargeting operation only changes the world coordinate system of the brain image, without altering the image array, i.e., it does not change the voxel coordinate system. In brain imaging, the world coordinate system is determined by the voxel coordinate system and the transformation matrix. Adjusting the world coordinate system while keeping the voxel coordinate system unchanged is equivalent to adjusting the transformation matrix. Therefore, brain image retargeting is essentially a process of adjusting the transformation matrix, while the image array remains unchanged. This means that the transformation matrix contained in the retargeted brain image can be predicted from the image array, thus directly achieving brain image retargeting.

[0067] The brain imaging described in this invention primarily refers to high-resolution three-dimensional structural magnetic resonance imaging (sMRI), but can also be extended to other modalities, such as functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI), through non-creative modifications as needed. The deep learning technology described in this invention primarily refers to 3D convolutional neural networks (3D CNN), but other similar deep learning technologies, such as Transformer, can also be used. The image processing techniques applied cover image cropping, normalization, and affine transformations.

[0068] For ease of explanation and understanding, the following concepts and expressions are used here first in this embodiment:

[0069] 1. The voxel coordinate system axes are defined as X, Y, and Z axes, respectively; the world coordinate system axes are defined as R, A, and S axes, respectively; and the world coordinate system basis vectors are defined as follows: and

[0070] 2. Coordinates specifically refer to voxel coordinates, and transformation matrix specifically refers to the transformation matrix from the voxel coordinate system to the world coordinate system.

[0071] 3. It is agreed that the world coordinate system before brain image redirection is the original world coordinate system, and the world coordinate system after redirection is the redirected world coordinate system, and the redirected world coordinate system satisfies the RAS+ coordinate system.

[0072] 4. By convention, the origin of the redirected world coordinate system is located at point AC, the A-axis is along the PC-AC line, the S-axis is along the midline of the brain, and the R-axis is parallel to... and Figure 1 and Figure 2 The MNI template is approximately positioned in the world coordinate system. Figure 1 and Figure 2 The positions of the MNI template in the world coordinate system and voxel coordinate system are shown on the Y and X slices of the MINI template. v O is the origin of the voxel coordinate system, and the edges of the image array are the coordinate axes of the voxel coordinate system. w The origin of the world coordinate system is located near point AC. The second axis (A-axis) of the world coordinate system passes through point PC. The third axis (S-axis) of the world coordinate system is along the midline of the brain. The first axis (R-axis) of the world coordinate system is parallel to... The world coordinate system of the MNI template uses the RAS+ coordinate system, and the X-axis of the voxel coordinate system and the R-axis of the world coordinate system are opposite in direction.

[0073] 5. Define the two-dimensional matrix formed by all voxel values ​​with the same X coordinate in the image array as the X slice. Obtain the Y slice and Z slice in the same way. Figure 1 What is shown is a Y-slice. Figure 2 The image shown is an X-slice. For the MNI template, the Y-slice is parallel to the coronal plane, and the X-slice is parallel to the sagittal plane.

[0074] 6. All X slices in the image array are arranged in their original order to form an X slice queue, with the head of the queue closest to the origin of the voxel coordinates. The Y slice queue and Z slice queue are obtained in the same way. It is easy to see that the X slice queue, Y slice queue and Z slice queue are all equivalent to the image array.

[0075] 7. Let the length of the X slice queue be the X dimension. In the same way, we can obtain the Y and Z dimensions. It is easy to see that the image array dimension is X dimension * Y dimension * Z dimension.

[0076] The basic steps of the end-to-end automatic redirection method for magnetic resonance brain images in this embodiment are as follows:

[0077] S10: Read in any brain image, denoted by N. The brain image N contains at least an image array A, a transformation matrix T, and an image head H.

[0078] S20: Adjust the brain image to a standard shape. Adjust A to standard dimensions to obtain A1. Modify the dimensional information of A in H to the dimensional information of A1 to obtain H1. Let T be the affine matrix equivalent to the process of adjusting A to A1. c ,but At this point, A1, H1, and T1 together form a standard-shaped brain image N1.

[0079] S30: Normalized image array. Normalizing A1 yields A2; the operations include dimension unification and numerical normalization. Let T be the affine matrix equivalent to the dimension unification process. s .

[0080] S40: Prediction. Input A2 into the fully trained deep learning model to obtain the preliminary result P.

[0081] S50: Construct the transformation matrix. Combine the prediction result P with T from step S3. s The transformation matrix T2 can be constructed. Combining the transformation matrix T2 with the T from step S2... c The transformation matrix T3 can be constructed.

[0082] S60: Output the redirected brain image. Encapsulate A1, H1, and T2 into brain image N2, and encapsulate A, H, and T3 into brain image N3. Therefore, N2 is the redirected N1, and N3 is the redirected N.

[0083] exist Figure 3 In the diagram, node ① represents the operation in step S20, adjusting A to the standard dimensions; node ② represents the operation in step S20, modifying the dimension information of A in H; and node ③ represents the operation in step S20... Operations: Node ④ represents step S30; Node ⑤ represents the brain image N1 operation in step S20; Node ⑥ represents the transformation matrix T2 operation in step S50; Node ⑦ represents the brain image N2 operation in step S60; Node ⑧ represents the transformation matrix T3 operation in S50 and the brain image N3 operation in step S60. All transformation matrices are 4*4 matrices, and the last row is [0,0,0,1].

[0084] The specific process of this embodiment is as follows:

[0085] Any brain image read in step S10 should ideally contain a complete brain region. The positions of the voxel coordinate system and the world coordinate system on its Y and X slices are as follows: Figure 4 and Figure 5As shown. By comparison Figure 1 , Figure 2 and Figure 4 , Figure 5 It can be seen that the brain image and the MNI template differ in their positions in the voxel coordinate system and the world coordinate system. The voxel coordinate system of the brain image is consistent with that of the MNI template, but the original world coordinate system deviates significantly from the world coordinate system of the MNI template. It should be noted that the difference in the voxel coordinate system position depends on the difference in the image array storage structure; the difference in the world coordinate system position depends on the difference in the brain structure where the coordinate origin and coordinate axes are located.

[0086] In step S20, adjusting the brain images to a standard shape essentially involves adjusting the image array to a standard dimension and modifying the image head and transformation matrix. The aim is to ensure that the image arrays of different dimensions have the same dimension after scaling in step S30, and to reduce the proportion of non-brain regions in the image array. The specific steps are as follows:

[0087] Step S21: Calculate the standard dimension. Resample the template with a voxel size of 1mm*1mm*1mm to the same voxel size as the brain image N. The dimension of the template image array after resampling is the standard dimension of A. Taking the MNI template as an example, the MNI template with a voxel size of 1mm*1mm*1mm has an image array dimension of 182*218*182. Let the voxel size of N be a mm*b mm*c mm, then the standard dimension of A is (182 / a)*(218 / b)*(182 / c). The corresponding standard X dimension, standard Y dimension, and standard Z dimension of A are (182 / a), (218 / b), and (182 / c). If the division result is a decimal, round up.

[0088] Step S22: Based on the standard dimensions calculated in S21, adjust the Z, Y, and X dimensions of A sequentially to the standard Z, Y, and X dimensions to obtain A1. Specifically, taking the adjustment of A's Z dimension to the standard Z dimension as an example, let Q be the Z-slice queue equivalent to A. z First, calculate Q. z The mean of each Z-slice is calculated, and then the number of Z-slices with a mean greater than zero is counted, denoted as m. If m is less than or equal to the standard Z-dimension, then the Q-slice is removed. z The Q-slice is obtained from a Z-slice with a mean less than or equal to zero. z1 Then in Q z1 Fill the beginning and end with approximately the same number (equal or differing by only one) of Z-slices whose values ​​are all zero, up to Q. z1 The length is equal to the standard Z dimension, resulting in the Z-dimensional adjusted image array A. If m is greater than the standard Z dimension, then remove Q. z The Q-slice is obtained from a Z-slice with a mean less than or equal to zero. z1Then, following the order of Z-slice mean from smallest to largest, remove the Q slices one by one. z1 Z-slices with smaller mean, to Q z1 The length is equal to the standard Z dimension, resulting in the image array A after Z-dimensional adjustment.

[0089] Then, in the same way, adjust the Y dimension to the standard Y dimension on the image array A that has been adjusted in the Z dimension to obtain the image array A with adjusted Z and Y dimensions. Finally, adjust the X dimension to the standard X dimension on the image array A that has been adjusted in the Z and Y dimensions to obtain A1.

[0090] Step S23: Modify the dimension information of image array A in image head H to the dimension information of image array A1 to obtain image head H1.

[0091] Step S24: Find the affine matrix T equivalent to the process of adjusting A to A1. c Let X slice queue, Y slice queue, and Z slice queue be equivalent to A. During the adjustment of A to A1, the change in the slice at the head of the queue is c. X c Y and c Z A negative value indicates that a slice at the head of the queue has been deleted, corresponding to image array cropping; a positive value indicates that a new slice has been added to the head of the queue, corresponding to image array expansion. In most cases, brain imaging arrays are larger than standard dimensions; therefore, image array cropping is used as an example. Figure 6 As shown in the figure, A is the original image array, A1 is the image array remaining after cropping around A, and O v Let N be the origin of the voxel coordinate system, |c X |、|c Y | and |c Z | This characterizes the amount of clipping in the three coordinate axes during the adjustment from A to A1.

[0092] Then T c for:

[0093] Step S25: According to the formula Calculate the transformation matrix T1. At this point, A1, H1, and T1 together form a standard-shaped brain image N1. N1 maintains the same position as the N world coordinate system because the transformation matrix has been modified accordingly.

[0094] like Figure 7 The diagram illustrates the process of adjusting a brain image to a standard shape. (a) shows the original brain image N, with the voxel coordinate system axes X, Y, and Z marked at the edges. The white crosshairs in the image represent its original world coordinate system axes R and S. (b) shows Q. z Mean distribution plot of the Z-slice, with the two axes being Q. z(c) is a schematic diagram of the brain image formed after adjusting the Z dimension of A, with a transformation matrix of T. (d) is a schematic diagram of the brain image formed after adjusting the Z, Y, and X dimensions of A, with an image array of A1 and a transformation matrix of T. (e) is a standard-shaped brain image N1, with an image array of A1 and a transformation matrix of T1. Comparing (a), (c), (d), and (e), the changes in the brain structures located on the white cross-shaped coordinate axes in the images can be observed.

[0095] In step S30, the image array is normalized. This operation includes dimensionality unification and numerical normalization, with the aim of dimensionality reduction and ensuring that the data dimension and numerical range input to the deep learning model in step S40 are consistent. The specific steps are as follows:

[0096] Step S31: Calculate the scaling ratio required for dimensional unification. Scaling is performed based on the voxel size. For example, if the target voxel size is 2.5mm*2.5mm*2.5mm, and the voxel size of N1 is a mm*b mm*c mm, then the required scaling ratios for the X, Y, and Z dimensions of A1 can be calculated as a / 2.5, b / 2.5, and c / 2.5, respectively. The scaling process is equivalent to the affine matrix T. s for:

[0097]

[0098] Step S32: Dimensional unification. Scale A1 according to the scaling ratio to obtain A′1. Then, fill the tail of the X-slice queue equivalent to A′1 with slices containing all zero values ​​up to a specified length, such as 87, to obtain the image array A′. 1X , in A′ 1X The equivalent Y-slice queue is padded with slices containing all zero values ​​to a set length, such as 87, to obtain image array A′. 1XY , in A′ 1XY The equivalent Z-slice queue is filled with zero values ​​at the end of the slices until the set length, such as 87, to obtain image array A″1; the length reference voxel size is 2.5mm*2.5mm*2.5mm MNI template, the image array dimension is 73*87*73, and the maximum length is equal to 87.

[0099] Step S33: Normalize the values. Use the formula Normalize A″1 to obtain A2, then set the negative values ​​in A2 to zero, overwriting the original A2. Here, Q1, Q2, and Q3 represent the first quartile, median, and third quartile, respectively.

[0100] In step S40, a fully trained deep learning model is prepared. The specific process for obtaining this model is as follows:

[0101] Step S41: Collect brain images. Collect as many brain images as possible, preferably no less than 5,000, and the image quality should be good.

[0102] Step S42: Adjust brain images to a standard shape. Following the method in step S20, adjust all collected brain images to a standard shape and save the standard-shaped brain images. Let D be the dataset composed of all standard-shaped brain images.

[0103] Step S43: Split the dataset. Split D into training dataset D1 and pre-training dataset D2 according to a set ratio (e.g., 1:4). D1 and D2 correspond to different types of labels. D1 is labeled with quantities directly related to the redirection task. D2 is labeled with quantities that are not directly related to the redirection task but are readily available, such as age and gender.

[0104] Step S44: Create the original label for D1. This embodiment provides two options, either of which can be selected:

[0105] Option 1: Have professionals with a background in brain anatomy mark the AC points, PC points, and any point M on the midsagittal plane of the brain that is closer to the top of the head than the AC and PC points. The coordinates of these three points will be the original labels for the brain image. It is recommended to preferably use the intersection of the septum pellucidum and the posterior surface of the genu of the corpus callosum on the cross-sectional image as M. Figure 8 and Figure 9 As shown. It is easy to see that the plane defined by points AC, PC, and M basically coincides with the midsagittal plane of the brain.

[0106] Option 2 involves having professionals with a background in brain anatomy manually redirect brain images to the agreed-upon world coordinate system location. The transformation matrix contained in the redirected brain image then becomes the original label of that brain image.

[0107] Step S45: Enhance D1 to obtain D1′. While ensuring the integrity of the brain regions in the image array, rotating or translating the image array of brain images in D1 will yield the brain images in D1′. Let T be the affine matrix equivalent to the rotation and translation operations. rt For option one in step S44, let T... rt Multiplying the original labels of the brain images in D1 yields the original labels of the corresponding brain images in D1′. For Scheme 2 in step S44, the original labels of the brain images in D1 are multiplied by the original labels. This will give you the original labels for the corresponding brain images in D1′.

[0108] Step S46: Construct the deep learning model backbone B. The input dimension of B must be consistent with the dimension after unifying the image array dimension in step S32. Various beneficial existing methods and techniques can be introduced during this construction process to improve model performance, such as network structure and loss function, which will not be elaborated in this implementation.

[0109] Step S47: Pre-train the model. Modify the output dimension of B to fit the label dimension of D2 to obtain B1. Normalize the brain image array in D2 using the method in step S30, and then pre-train B1 using the normalized image array as input and labels such as age or gender.

[0110] Step S48: Train the model. Normalize the brain image arrays of D1 and D1′ using the method in step S30 and record T. s The original labels of the training dataset D1 and the augmented dataset D1′ are transformed into new labels. The output dimension of B1 is then modified to adapt to the new label dimensions of D1 and D1′, resulting in B2. Then, using the normalized image array as input, combined with the new labels of D1 and D1′, B2 is trained with the help of relevant transfer learning strategies.

[0111] To simplify the label creation process as much as possible, step S44 only yields the original labels. Therefore, before training the model, it is necessary to perform some transformations on the original labels to obtain new labels.

[0112] For Option 1 in step S44, first use T recorded in step S48. s Multiplying the coordinates in the original brain image labels by the dot product yields new coordinates for points AC, PC, and M. Calculations are then performed using these new coordinates. Obtain vector Represents the direction of the R-axis. A normalized vector. The new coordinates of AC point and PC point, a total of 9 quantities, are the new labels for this brain image.

[0113] For Scheme 2 in step S44, multiply the transformation matrix in the original brain image labels by the matrix recorded in step S47. The new transformation matrix is ​​obtained. The first three rows of the new transformation matrix, totaling 12 values, are used as the new labels for the brain image.

[0114] S49: Repeat the validation and testing of B2 to obtain a fully trained deep learning model.

[0115] To further improve model performance, a self-supervised method can be used to optimize step S47.

[0116] In step S50, a transformation matrix is ​​constructed. For the model trained with two different types of labels in step S48, step S40 will also output two different prediction results P.

[0117] If the prediction result P consists of 12 quantities, arrange the prediction results into a 3*4 matrix in order, and add a row [0,0,0,1] at the end of the matrix to obtain T. p Then T2 = T p ·T sT3 = T2·T c .

[0118] If the prediction result consists of 9 quantities, then they represent the coordinates of points AC and PC in A2 and the normalized vector. First use Multiplying the coordinates of two points by their dot product yields the coordinates of points AC and PC in coordinate system A1. In A1, the coordinates of point AC, the coordinates of point PC, and the normalized vector... The position L1 of the redirected world coordinate system of N1 can be determined (the position of a coordinate system can be determined by an origin and the directions of the two coordinate axes), and the position L2 of the original world coordinate system of N1 can be determined by T1. Let T be the affine matrix that translates and rotates L1 to L2. rt1 Then T2 = T1·T rt1 T3 = T2·T c .

[0119] Through the above process, this invention achieves end-to-end brain image redirection with a simple and efficient workflow. Furthermore, the process is largely template-free and can even customize the position of the redirection world coordinate system. This invention has high scalability; by multiplying the redirection world coordinate system by an affine matrix, it can be extended to any template's world coordinate system. Moreover, this invention has good transferability, applicable not only to brain images but also to any other similar images requiring redirection.

[0120] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any changes made based on the design principles of the present invention, or any non-creative modifications made thereon, shall fall within the scope of protection of the present invention.

Claims

1. An end-to-end automatic redirection method for magnetic resonance brain images, characterized in that, Includes the following steps: S10. Read in the brain image N to be processed, which includes at least an image array A, a transformation matrix T and an image head H. S20. Adjust brain image N to a standard shape: Adjust image array A to a standard-dimensional image array A1. Modify the dimensional information of image array A in image head H to the dimensional information of image array A1, obtain image head H1, and obtain the affine matrix T equivalent to the process of adjusting image array A to image array A1. c Obtain the transformation matrix Then, the standard-shaped brain image N1 is composed of the image array A1, the image head H1, and the transformation matrix T1; S30, Normalized Image Array: Image array A1 is normalized into image array A2 through dimension unification and numerical normalization operations; S40. Prediction: Input the image array A2 into the fully trained deep learning model to obtain the preliminary result P; S50. Construct the transformation matrix: Combine the pre-result P with the affine matrix T, which is equivalent to the image array dimension unification process in step S30. s Construct the transformation matrix T2; combine the transformation matrix T2 with the affine matrix T in step S20. c Construct the transformation matrix T3; S60. Output the redirected brain image: The encapsulated image array A1, image head H1 and transformation matrix T2 form brain image N2, which is the result of redirecting the standard-shaped brain image N1. The encapsulated image array A, image head H and transformation matrix T3 form brain image N3, which is the result of redirecting brain image N. The process of adjusting the brain image N to a standard shape in step S20 includes: S21. Calculate the standard dimension: The resampled template with a voxel size of 1mm*1mm*1mm is resampled to the same voxel size as the brain image N. The dimension of the image array of the resampled template is the standard dimension of A. S22. Based on the standard dimensions calculated in step S21, adjust the Z dimension, Y dimension and X dimension of image array A to the standard Z dimension, standard Y dimension and standard X dimension in sequence to obtain image array A1. S23. Modify the dimension information of image array A in image head H to the dimension information of image array A1 to obtain image head H1; S24. Find the affine matrix T equivalent to the process of adjusting image array A to image array A1. c ; S25. According to the formula Calculate the transformation matrix T1. At this point, the image array A1, the image head H1, and the transformation matrix T1 form a standard-shaped brain image N1. The world coordinate system positions of brain image N1 and brain image N are the same.

2. The end-to-end automatic redirection method for magnetic resonance brain images according to claim 1, characterized in that, The process of sequentially adjusting the Z, Y, and X dimensions of image array A to standard Z, Y, and X dimensions in step S22 is as follows: Adjust the Z-dimensionality of image array A to the standard Z-dimensionality, and set the Z-slice queue equivalent to image array A to Q. z ; First calculate Q z Find the mean of each Z-slice in Q, then count the number m of Z-slices with a mean greater than zero. If m is less than or equal to the standard Z-dimension, remove the Q-slice. z The Z-slice with a mean less than or equal to zero yields the queue Q. z1 Then in Q z1 Fill the beginning and end with approximately the same number of Z-slices, each with a value of zero, so that Q... z1 The length of m is equal to the standard Z dimension, resulting in the Z-dimensional adjusted image array A; if m is greater than the standard Z dimension, Q is removed first. z The Z-slice with a mean less than or equal to zero yields the queue Q. z2 Then, following the order of Z-slice mean from smallest to largest, remove Q slices one by one. z2 The Z-slice with a smaller mean makes Q z2 The length is equal to the standard Z dimension, resulting in the Z-dimensional adjusted image array A; Then, in the same way, adjust the Y dimension to the standard Y dimension on the image array A after the Z dimension has been adjusted, to obtain the image array A after the Z and Y dimensions have been adjusted. Finally, adjust the X dimension to the standard X dimension on the image array A after the Z and Y dimensions have been adjusted, to obtain the image array A1.

3. The end-to-end automatic redirection method for magnetic resonance brain images according to claim 2, characterized in that, In step S24, the affine matrix T equivalent to the process of adjusting image array A to image array A1 is obtained. c The process is as follows: Set up X-slice queues, Y-slice queues, and Z-slice queues equivalent to image array A. During the adjustment of image array A to image array A1, the change in the head slice of each queue is c. X c Y and c Z A negative value indicates that a slice at the head of the queue has been deleted, corresponding to image array cropping; a positive value indicates that a new slice has been added to the head of the queue, corresponding to image array expansion. This affine matrix T c Represented as .

4. The end-to-end automatic redirection method for magnetic resonance brain images according to claim 1, characterized in that, The process of step S30 includes: S31. Calculate the scaling ratio required for dimensional unification based on the voxel size of brain image N1 and the set target voxel size; S32, Dimensional Unification: Scale the image array A1 according to the scaling ratio to obtain the image array. Then in the image array The equivalent X-slice queue is padded with zero values ​​at the end of the slices until a set length is reached, resulting in an image array. In the image array The equivalent Y-slice queue is padded with slices of all zero values ​​to a set length to obtain an image array. In the image array The equivalent Z-slice queue is padded with zero values ​​at the end of the slices until a set length is reached, resulting in an image array. ; S33, Numerical Normalization: Through the formula Normalized image array The image array A2 is obtained, where Q1, Q2, and Q3 represent the first quartile, median, and third quartile, respectively. Then, the negative values ​​in the image array A2 are set to zero, covering the original image array A2.

5. The end-to-end automatic redirection method for magnetic resonance brain images according to any one of claims 1 to 4, characterized in that, The process of obtaining the fully trained deep learning model in step S40 includes: S41. Collect a sufficient number of brain images as training data; S42. Adjust all collected brain images to standard shapes and construct dataset D from all standard-shaped brain images; S43. Split the dataset D into a training dataset D1 and a pre-training dataset D2 according to a set ratio. The training dataset D1 is labeled with quantities that are directly related to the redirection task, and the pre-training dataset D2 is labeled with quantities that are not directly related to or unrelated to the redirection task but can be directly obtained. S44. Create original labels for training dataset D1 based on expertise in brain anatomy; S45. While ensuring the integrity of brain regions in the image array, rotate or translate the image array of brain images in training dataset D1 to obtain the enhanced training dataset. The brain images in the dataset, combined with the original labels of the brain images in the training dataset D1 and the affine matrix T equivalent to rotation and translation operations, are used. rt To obtain an enhanced training dataset The original labels corresponding to the brain images; S46. Construct a deep learning model B based on the dimensions of the image array after unification in step S30. S47. Pre-trained model: By adjusting the output dimension of deep learning model B to adapt to the label dimension of pre-training dataset D2, deep learning model B1 is obtained; the image array of brain images in pre-training dataset D2 is normalized, and then the normalized image array is used as input, combined with the labels of pre-training dataset D2, to train deep learning model B1. S48. Training the model: Normalized training dataset D1 and augmented training dataset Midbrain imaging array and recording of affine matrix T s Transform the training dataset D1 and the augmented training dataset The original labels are used as new labels, and then the output dimension of the deep learning model B1 is adjusted to adapt to the training dataset D1 and the augmented training dataset. The new label dimension is used to obtain the deep learning model B2; then, the normalized image array is used as input, combined with the training dataset D1 and the augmented training dataset. The new label, using a transfer learning strategy, trains the deep learning model B2; S49. Repeatedly validate and test the deep learning model B2 to obtain a fully trained deep learning model.

6. The end-to-end automatic redirection method for magnetic resonance brain images according to claim 5, characterized in that, The process of creating original labels for training dataset D1 based on expertise in brain anatomy in step S44 is as follows: With the help of professionals with a background in brain anatomy, the AC and PC points of the brain images in D1 and any point M that is closer to the top of the head than the AC and PC points on the midsagittal plane of the brain are labeled. The original labels of the brain images are formed by the coordinates of the AC, PC and M points. After labeling all the brain images, the original labels of the training dataset D1 are obtained. The new label in step S48 contains a normalized vector representing the R-axis direction, calculated from the original label. By T s The product consists of nine quantities, including the new coordinates of AC and PC obtained by multiplying the original labels of AC and PC.

7. The end-to-end automatic redirection method for magnetic resonance brain images according to claim 6, characterized in that, The process of constructing the transformation matrix in step S50 is as follows: The prediction results include 9 quantities, which will be processed first. Multiply the coordinates of AC and PC points in the prediction result to obtain the coordinates of AC and PC points in image array A1; In image array A1, the coordinates of AC point, PC point coordinates, and the normalized vector in the prediction results are used. The position L1 of the redirected world coordinate system of brain image N1 is determined, and the position L2 of the original world coordinate system is determined by transformation matrix T1. The affine matrix T is used to translate and rotate from position L1 to position L2. rt1 Determine the transformation matrix and transformation matrix .

8. The end-to-end automatic redirection method for magnetic resonance brain images according to claim 5, characterized in that, The process of creating original labels for training dataset D1 based on expertise in brain anatomy in step S44 is as follows: With the help of professionals with a background in brain anatomy, the brain images in D1 are manually redirected according to the set world coordinate system position of the brain images. The transformation matrix contained in the brain image after redirection is the original label of the brain image. After all brain images are redirected, the original label of the training dataset D1 is obtained. The new label in step S48 contains the dot product of the original label transformation matrix. The resulting transformation matrix contains 12 quantities in the first three rows.

9. The end-to-end automatic redirection method for magnetic resonance brain images according to claim 8, characterized in that, The process of constructing the transformation matrix in step S50 is as follows: The prediction result P consists of 12 quantities. Arrange the prediction results into a 3x4 matrix in order, and add a row [0, 0, 0, 1] to the end of the matrix to obtain matrix T. p Construct the transformation matrix and transformation matrix .

Images