A medical image registration method and device based on deep learning neural network

Abstract

This invention discloses a medical image registration method and apparatus based on deep learning neural networks. The medical image pairs to be registered are preprocessed. A registration model combining a U-Net convolutional neural network and an attention mechanism is used to predict the deformation field between the fixed and moving images in the medical image pair. Based on this deformation field, a registration image is obtained to align the registered image with the fixed image. The convolutional neural network extracts local features, and an attention mechanism is used to learn the weights of these local features. Finally, global features are extracted, resulting in sufficient feature extraction and high registration accuracy.

Description

Technical Field

[0001] The present invention relates to the field of image registration technology, and in particular to a medical image registration method and device based on deep learning neural network. Background Art

[0002] When analyzing medical images, it is often necessary to analyze several images of the same patient together to obtain comprehensive information about the patient and improve the level of medical diagnosis and treatment. Quantitative analysis of several different images first requires solving the problem of strict alignment of these images, which is the process of image registration. Medical image registration involves finding a spatial transformation (or a series of spatial transformations) for one medical image to achieve spatial consistency with corresponding points in another medical image. This consistency means that the same anatomical point on the human body has the same spatial position in the two matching images. The registration result should ensure that all anatomical points in the two images, or at least all points of diagnostic significance and surgical interest, are matched. Medical image registration provides an important basis for doctors to analyze and diagnose patient conditions and is the most critical preprocessing step in medical image analysis.

[0003] Current registration methods based on deep learning neural networks primarily use supervised or unsupervised learning to generate a registration image for alignment with a reference image. Supervised learning, however, is often insufficient due to the difficulty in providing realistic deformation field data to train the network model. Unsupervised learning, on the other hand, is prone to mismatching, resulting in inaccurate registration images and low registration accuracy.

[0004] Therefore, the existing technology needs to be improved and enhanced. Summary of the Invention

[0005] The main purpose of the present invention is to provide a medical image registration method, device, intelligent terminal and storage medium based on deep learning neural network, which can solve the problem that the currently generated registration images are not accurate enough and the registration accuracy is not high.

[0006] In order to achieve the above objectives, the first aspect of the present invention provides a medical image registration device based on a deep learning neural network, wherein the device comprises:

[0007] a data preprocessing module, configured to preprocess the medical image pair to be registered to obtain a preprocessed medical image pair, wherein the medical image pair to be registered includes a fixed image and a moving image;

[0008] The registration model is constructed based on the U-Net convolutional neural network. The jump connection of the registration model is a multi-layer network model based on the attention mechanism. The registration model is used to predict the deformation field between the fixed image and the moving image and obtain the registered image based on the deformation field.

[0009] Optionally, the last layer of the registration model is provided with three convolution kernels, each of which is used to obtain a displacement field of one dimension.

[0010] Optionally, each layer of the multi-layer network model includes a convolutional patch embedding module and several Transformer coding blocks, and the number of the Transformer coding blocks is the same as the layer number of the layer where the Transformer coding block is located.

[0011] Optionally, starting from the second layer of the multi-layer network model, a feedforward layer is also included, and the output of the previous layer is added to the current layer through a residual connection, and the output of the feedforward layer is added to the convolutional patch embedding module in the form of a patch.

[0012] Optionally, the feedforward layer includes a multi-layer perceptron and two hidden layers, and a deep convolution layer is provided between the hidden layers.

[0013] A second aspect of the present invention provides a medical image registration method based on a deep learning neural network, the method comprising:

[0014] Preprocessing the medical image pair to be registered to obtain a preprocessed medical image pair, wherein the medical image pair to be registered includes a fixed image and a moving image;

[0015] The preprocessed medical image pair is input into a trained registration model, which is constructed based on a U-Net convolutional neural network. The jump connection of the registration model is a multi-layer network model based on an attention mechanism. The registration model is used to predict the deformation field between the fixed image and the moving image and obtain a registered image based on the deformation field.

[0016] Optionally, preprocessing the medical image pair to be registered includes:

[0017] respectively calculating the centroid coordinates of each medical image in the pair of medical images to be registered;

[0018] Each medical image is cropped to a target size with the centroid coordinates as the center.

[0019] Optionally, training the registration model includes:

[0020] Preprocessing a medical image pair sample with a mask label, wherein the medical image pair sample includes a fixed image and a moving image;

[0021] Extracting features of the fixed image and the moving image to obtain a deformation field composed of spatial transformation parameters;

[0022] spatially transforming the moving image according to the deformation field to obtain a registered image;

[0023] A Dice loss value between the mask label of the registered image and the mask label of the fixed image is calculated, and the registration model is optimized according to the Dice loss value.

[0024] The third aspect of the present invention provides an intelligent terminal, which includes a memory, a processor, and a medical image registration program based on deep learning neural network stored in the memory and runnable on the processor. When the medical image registration program based on deep learning neural network is executed by the processor, the step of any one of the medical image registration methods based on deep learning neural network is implemented.

[0025] A fourth aspect of the present invention provides a computer-readable storage medium, on which a medical image registration program based on a deep learning neural network is stored. When the medical image registration program based on a deep learning neural network is executed by a processor, the medical image registration program based on a deep learning neural network implements any step of the medical image registration method based on a deep learning neural network.

[0026] As can be seen from the above, compared with existing technologies, this method first preprocesses the medical image pairs to be registered. It then uses a registration model that combines a U-Net convolutional neural network and an attention mechanism to predict the deformation field between the fixed and moving images in the medical image pair and obtain a registered image based on this deformation field. The convolutional neural network extracts local features, uses the attention mechanism to learn the weights of these local features, and extracts global features. This results in comprehensive feature extraction, resulting in a precise registered image with high registration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 Schematic diagram of the structure of a medical image registration device based on deep learning neural network provided by an embodiment of the present invention;

[0029] Figure 2 yes Figure 1 Schematic diagram of the structure of the improved U-Net convolutional neural network in the embodiment;

[0030] Figure 3 yes Figure 1 A schematic diagram of the structure of a multi-layer network model in an embodiment;

[0031] Figure 4 yes Figure 1 Schematic diagram of the training framework of the registration model in the embodiment;

[0032] Figure 5 This is a schematic diagram of a specific process of a medical image registration method based on a deep learning neural network provided by an embodiment of the present invention;

[0033] Figure 6 Schematic diagram of a specific process of training a registration model provided by an embodiment of the present invention;

[0034] Figure 7 This is a block diagram of the internal structure principle of a smart terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0036] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0037] It should also be understood that the terms used in the present specification are only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0038] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0039] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0040] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0042] Medical image registration provides an important basis for doctors to analyze and diagnose patients' conditions and is the most critical preprocessing step in medical image analysis. Its goal is to find a spatial transformation that establishes anatomical correspondence between two or more medical images.

[0043] Current registration methods based on deep learning neural networks primarily use supervised or unsupervised learning to generate a registration image for alignment with the reference image. Supervised learning, however, is often insufficient due to the difficulty in providing realistic deformation field data to train the network model. Unsupervised learning, on the other hand, is prone to mismatching, particularly for target organs that experience large displacements. This results in inaccurate registration images and low registration accuracy.

[0044] To address the above problems, the present invention proposes a medical image registration device and method based on deep learning neural networks. By improving the U-Net convolutional neural network to extract local features of the image, and combining it with the transformer attention mechanism to extract global features of the image, it can produce good registration effects even for target organs with large displacements, generate accurate registration images, and improve registration accuracy.

[0045] Exemplary devices

[0046] like Figure 1As shown, an embodiment of the present invention provides a medical image registration device based on a deep learning neural network. The device runs on electronic devices such as smart terminals, PCs, and hospital backend servers, and is used to register three-dimensional CT images of the lungs and kidneys. The device primarily comprises a data preprocessing module 600 and a registration model 610. The data preprocessing module 600 is used to segment the medical image pair to be registered into a uniform size and perform operations such as grayscale conversion to obtain a preprocessed medical image pair. The medical image pair to be registered includes a fixed image and a moving image. The fixed image serves as the registration reference, while the moving image is deformed to generate a registered image, which is then registered with the fixed image. Taking kidney CT as an example, since unenhanced plain CT images only vaguely reveal the kidney outline and obscure other anatomical structures, making labeling difficult, comparison with enhanced CT is required. This embodiment includes three phases of CT images: plain, excretory, and cortical. The plain CT image is a normal CT image, serving as the comparison reference; the excretory and cortical phases serve as the enhanced CT images, which are then registered with the plain CT images.

[0047] After the medical image pair is input into the registration model 610, the registration model 610 predicts the deformation field between the fixed image and the moving image and obtains a registered image based on the deformation field. Specifically, after the registration model 610 predicts the deformation field between the fixed image and the moving image, it applies the deformation field to the moving image to obtain a registered image, thereby achieving registration with the fixed image.

[0048] Because three-dimensional medical images have high resolution, a single image takes up a large amount of memory. Therefore, to improve computational efficiency, the medical images in the medical image pair are first preprocessed in the data preprocessing module 600. For example, the medical images are first resampled so that the pixel units of each medical image are uniform. In one example, all CT images are resampled to 1mm×1mm×1mm images. The center of mass coordinates of the medical images are then calculated using the center of mass formula, and each medical image is cropped to a uniform size based on the center of mass coordinates. The formula for calculating the center of mass coordinates (x0, y0, z0) is:

[0049]

[0050]

[0051]

[0052] The center of mass of the lung or kidney image is calculated using the centroid coordinate formula. Using the centroid coordinates as the image center, the lung or kidney image is cropped to a size of 256×256×128, ensuring that the cropped image encompasses the entire target organ. Grayscale normalization is then performed on the cropped image. Cropping the medical images reduces the computational complexity of the registration model 610 and improves computational efficiency.

[0053] It should be noted that there is no limitation on the method for cropping medical images. Cutting according to centroid coordinates is only one example. Other image processing methods, such as edge segmentation methods, can also be used for cutting.

[0054] A convolutional neural network based on the U-Net structure does not require relevant supervisory information such as the actual deformation field during training, and directly predicts the deformation field of the entire spatial transformation. The present invention improves upon the U-Net convolutional neural network by removing network components that do not contribute to registration and performing corresponding processing on three-dimensional medical images. This ensures that the image features are fully preserved and utilized after network processing, better constraining the regression network to learn parameters and obtain registration model 610.

[0055] The registration model 610 includes an encoder and a decoder, referring to Figure 2 As shown, the encoder is on the left and the decoder is on the right. The encoder consists of convolutional layers with a stride of 2 to achieve upsampling and extract high-dimensional feature vectors. The decoder consists of upsampling and convolutional layers with a stride of 1 to reconstruct low-dimensional feature vectors into high-dimensional feature vectors. The activation function connected after each convolutional layer in the registration model 610 is LeakyReLU. Figure 2 The input image pair consists of a moving image and a fixed image concatenated together to form a two-channel image. The solid line represents a convolution operation with a stride of n, and each convolution layer is followed by an activation function. The dashed line represents the upsampling operation. Unlike commonly used convolutional neural networks, the pooling layer is discarded after the convolution layer and a convolution kernel with a stride of 2 is used instead for downsampling to avoid losing the relative position information between image regions. Because medical images are three-dimensional, the network's output image is also three-dimensional. Each dimension requires a displacement field. Three convolution kernels are designed in the final layer of the network, each for obtaining a displacement field in one dimension.

[0056] Although the improved convolutional neural network performs well in the registration task, the accuracy of the generated registration image is still not high enough when the position or shape of the organ changes significantly. To address this problem, the registration model 610 is further improved, and the transformer attention module is structurally improved to obtain a multi-layer network model based on the attention mechanism. The multi-layer network model is then used as a jump connection for the U-Net convolutional neural network, ultimately ensuring that global features are extracted during the registration process. After the above improvements, the registration model 610 can learn the low-level features extracted from the convolutional network, combine the low-level and high-level features through the attention gate, and assign weights to each high-level feature. By first performing a rough global registration and then using the attention mechanism to perform refined registration, the generated registration image is more accurate and the accuracy of the registration is improved.

[0057] like Figure 3 As shown in the figure, the multi-layer network model consists of a three-layer structure. The model structure of all layers is the same, including a convolutional patch embedding module and Ni (i represents the layer, and there are several blocks for each layer) Transformer encoding blocks. Each Transformer encoding block contains an alternating multi-layer self-attention module and a convolutional feed-forward layer. Because it is not helpful for alignment, the normalization operation in the attention mechanism is deleted; and starting from the second layer of the multi-layer network model, a feed-forward layer (MPL Head in the figure) is also provided. The output of the previous layer is added to the embedding patch layer of this layer through a residual connection, and the output of the feed-forward layer is added to the convolutional patch embedding module in the form of a patch.

[0058] The convolutional feedforward layer consists of a multi-layer perceptron (MLP) and two hidden layers, which are applied to the convolutional patch embedding module in a patch-like manner. To preserve local features, a 3×3×3 deep convolutional layer is added between the two hidden layers. This modification not only removes network structural layers that are not helpful for registration, reducing computational complexity, but also ensures that global features such as large displacements are preserved.

[0059] By employing an attention mechanism for 3D medical image registration, the global connectivity and locality of convolutional visual transformers, along with their multi-resolution strategies, can be leveraged to learn global registration. This allows the convolutional neural network to extract local features, while the attention module learns to assign local feature weights to the convolutional neural network, assigning weights to global features and ultimately extracting global features. Combining the Transformer with a U-Net convolutional neural network, thanks to the self-attention mechanism, overcomes the limitations of convolution kernel size, making it more suitable for registration tasks involving large deformations and displacements of target organs.

[0060] When training the registration model 610, if Figure 4 As shown, it is driven by label data. Clinical experts accurately annotated the contours of the three-phase CT data of the lungs and kidneys, and selected 600 cases of each of the three-phase CT images as experimental data. The image pairs to be registered, including fixed images and moving images, are input into the registration model 610. After network training, the parameters of the spatial transformation are obtained. The parameters are back-propagated and updated using the loss function. The final spatial transformation is applied to the corresponding labels of the moving images to obtain the deformed labels, which are compared with the corresponding labels of the fixed images to obtain the registration effect. Specifically, the CT data of the plain scan period is used as the fixed image, and the CT data of the excretory and cortical phases are used as the moving images. The overall registration process is to register the excretory and cortical phase images with the plain scan period CT respectively. The registration results are quantitatively compared and analyzed by calculating the Dice value between the labels.

[0061] The training process uses an epoch as the loop unit. For each patient, three CT images of the kidneys and lungs—plain, cortical, and excretory phases—are registered separately, with the plain CT image serving as the fixed image and the excretory and cortical CT images serving as the moving images. 600 images from each phase were used, with 500 selected as the training set and 100 as the validation set. The registration model's initial learning rate was set to 0.001, and the Adam optimizer was selected. When the error in the validation set did not significantly decrease, the best-performing model was saved as the registration model result.

[0062] This embodiment uses weakly supervised learning to train the registration model, using the masks corresponding to the medical image data as training labels. The network is trained using a similarity loss and a regularization loss for label pairs. This reduces the reliance on high-quality medical image datasets, eliminates the need for a true deformation field for the image pairs to be registered, and uses label similarity instead of image similarity, addressing the difficulty of calculating multimodal image similarity and achieving effective training results.

[0063] Exemplary Methods

[0064] The embodiment of the present invention also provides a medical image registration method based on deep learning neural network. Figure 5 As shown, this embodiment includes the following steps:

[0065] Step S100: pre-processing the medical image pair to be registered to obtain a pre-processed medical image pair, wherein the medical image pair to be registered includes a fixed image and a moving image.

[0066] Specifically, the centroid coordinates of each medical image in the medical image pair to be registered are calculated respectively, and each medical image is cropped to the target size, such as 256×256×128, with the centroid coordinates as the center, and then grayscale normalization is performed to obtain the preprocessed medical image pair.

[0067] Step S200: The preprocessed medical image pair is input into a trained registration model. The registration model is constructed based on the U-Net convolutional neural network. The jump connection of the registration model is a multi-layer network model based on the attention mechanism. The registration model is used to predict the deformation field between the fixed image and the moving image and obtain the registered image based on the deformation field.

[0068] Specifically, improvements and innovations were made to the U-Net convolutional neural network, removing network components that do not contribute to registration. Furthermore, the network was processed specifically for 3D medical images, ensuring that image features are fully preserved and utilized after processing, effectively constraining the regression network's parameter learning. The attention mechanism module was also modified to be suitable for use in the registration network, serving as a skip connection within the convolutional neural network. The global connectivity and locality of the convolutional visual transformer, along with a multi-resolution strategy, were leveraged to learn global registration, achieving the goal of simultaneously attending to large global displacements and deformations while also extracting important local features for registration.

[0069] like Figure 6 As shown, the steps for training the registration model include:

[0070] Step S300: pre-processing a sample of medical image pairs with mask labels, where the sample of medical image pairs includes a fixed image and a moving image.

[0071] Step S400: extracting features of the fixed image and the moving image to obtain a deformation field composed of spatial transformation parameters.

[0072] Step S500: spatially transforming the moving image according to the deformation field to obtain a registered image.

[0073] Step S600: Calculate the Dice loss between the mask labels of the registered image and the mask labels of the fixed image, and optimize the registration model according to the Dice loss.

[0074] Specifically, each patient has three phases of CT images corresponding to the kidneys and lungs: plain scan, cortical scan, and excretory scan. The plain scan CT is used as a fixed image, and the excretory or cortical CT is used as a moving image. Each medical image is manually annotated with a mask and used as a medical image pair sample to train the registration model.

[0075] First, the image to be registered is resampled and cropped to locate the center of mass of the target organ. The input image to the registration network is resized to remove image noise and reduce computational complexity. Feature extraction is performed on the input image, and spatial transformation parameters are automatically extracted to obtain a deformation field composed of these parameters. The moving image is then transformed based on this deformation field to obtain the registered image. The Dice value of the mask labels of the registered image and the fixed image is calculated to quantitatively determine the registration accuracy. The registration model is then iteratively optimized based on the Dice value.

[0076] Specifically, in this embodiment, the specific functions of each step of the above-mentioned medical image registration method based on deep learning neural network can refer to the corresponding description in the above-mentioned medical image registration device based on deep learning neural network, and will not be repeated here.

[0077] In summary, the medical image registration method based on deep learning neural network provided in this embodiment adopts a weakly supervised learning strategy, uses labeled data for deformation and registration, learns the local feature extraction method of convolutional neural network through the transformer module, and applies it to global feature extraction, while assigning weights, which can improve the registration accuracy.

[0078] Based on the above embodiment, the present invention also provides an intelligent terminal, whose principle block diagram can be shown as follows: Figure 7 As shown. The above-mentioned intelligent terminal includes a processor, a memory, a network interface and a display screen connected through a system bus. Among them, the processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a medical image registration program based on deep learning neural network. The internal memory provides an environment for the operation of the operating system and the medical image registration program based on deep learning neural network in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal through a network connection. When the medical image registration program based on deep learning neural network is executed by the processor, the steps of any one of the above-mentioned medical image registration methods based on deep learning neural network are implemented. The display screen of the intelligent terminal can be a liquid crystal display or an electronic ink display.

[0079] Those skilled in the art will understand that Figure 7The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention and does not constitute a limitation on the smart terminal to which the solution of the present invention is applied. The specific smart terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0080] In one embodiment, a smart terminal is provided. The smart terminal includes a memory, a processor, and a deep learning neural network-based medical image registration program stored in the memory and executable on the processor. When executed by the processor, the deep learning neural network-based medical image registration program performs the following operation instructions:

[0081] Preprocessing the medical image pair to be registered to obtain a preprocessed medical image pair, wherein the medical image pair to be registered includes a fixed image and a moving image;

[0082] The preprocessed medical image pair is input into a trained registration model, which is constructed based on a U-Net convolutional neural network. The jump connection of the registration model is a multi-layer network model based on an attention mechanism. The registration model is used to predict the deformation field between the fixed image and the moving image and obtain a registered image based on the deformation field.

[0083] Optionally, preprocessing the medical image pair to be registered includes:

[0084] respectively calculating the centroid coordinates of each medical image in the pair of medical images to be registered;

[0085] Each medical image is cropped to a target size with the centroid coordinates as the center.

[0086] Optionally, training the registration model includes:

[0087] Preprocessing a medical image pair sample with a mask label, wherein the medical image pair sample includes a fixed image and a moving image;

[0088] Extracting features of the fixed image and the moving image to obtain a deformation field composed of spatial transformation parameters;

[0089] spatially transforming the moving image according to the deformation field to obtain a registered image;

[0090] A Dice loss value between the mask label of the registered image and the mask label of the fixed image is calculated, and the registration model is optimized according to the Dice loss value.

[0091] An embodiment of the present invention also provides a computer-readable storage medium, on which a medical image registration program based on a deep learning neural network is stored. When the medical image registration program based on a deep learning neural network is executed by a processor, the steps of any one of the medical image registration methods based on a deep learning neural network provided in an embodiment of the present invention are implemented.

[0092] It should be understood that the sequence numbers of the steps in the above embodiments do not imply a specific order of execution; the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0093] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0094] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0095] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0096] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units described above is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented.

[0097] If the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0098] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A medical image registration device based on deep learning neural network, characterized in that: The device comprises: a data preprocessing module, configured to preprocess the medical image pair to be registered to obtain a preprocessed medical image pair, wherein the medical image pair to be registered includes a fixed image and a moving image; A registration model constructed based on a U-Net convolutional neural network, wherein the skip connections of the registration model are a multi-layer network model based on an attention mechanism, and the registration model is used to predict the deformation field between the fixed image and the moving image and obtain a registered image based on the deformation field; The last layer of the registration model is provided with three convolution kernels, each of which is used to obtain a displacement field of one dimension; Each layer of the multi-layer network model includes a convolutional patch embedding module and a plurality of Transformer coding blocks, and the number of the Transformer coding blocks is the same as the layer number of the layer where the Transformer coding block is located; Starting from the second layer of the multi-layer network model, a feed-forward layer is also included, and the output of the previous layer is added to the current layer through a residual connection, and the output of the feed-forward layer is added to the convolutional patch embedding module in a patch manner; The feedforward layer includes a multi-layer perceptron and two hidden layers, and a deep convolution layer is provided between the hidden layers.

2. A medical image registration method based on a deep learning neural network based on the medical image registration device based on a deep learning neural network according to claim 1, characterized in that: The method comprises: Preprocessing the medical image pair to be registered to obtain a preprocessed medical image pair, wherein the medical image pair to be registered includes a fixed image and a moving image; The preprocessed medical image pair is input into a trained registration model, which is constructed based on a U-Net convolutional neural network. The jump connection of the registration model is a multi-layer network model based on an attention mechanism. The registration model is used to predict the deformation field between the fixed image and the moving image and obtain a registered image based on the deformation field.

3. The medical image registration method based on deep learning neural network according to claim 2, characterized in that: The preprocessing of the medical image pair to be registered includes: respectively calculating the centroid coordinates of each medical image in the pair of medical images to be registered; Each medical image is cropped to a target size with the centroid coordinates as the center.

4. The medical image registration method based on deep learning neural network according to claim 2, characterized in that: Training the registration model, comprising: Preprocessing a medical image pair sample with a mask label, wherein the medical image pair sample includes a fixed image and a moving image; Extracting features of the fixed image and the moving image to obtain a deformation field composed of spatial transformation parameters; spatially transforming the moving image according to the deformation field to obtain a registered image; A Dice loss value between the mask label of the registered image and the mask label of the fixed image is calculated, and the registration model is optimized according to the Dice loss value.

5. An intelligent terminal, characterized in that: The intelligent terminal includes a memory, a processor, and a medical image registration program based on a deep learning neural network stored in the memory and runnable on the processor. When the medical image registration program based on a deep learning neural network is executed by the processor, the steps of the medical image registration method based on a deep learning neural network as described in any one of claims 2 to 4 are implemented.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a medical image registration program based on a deep learning neural network. When the medical image registration program based on a deep learning neural network is executed by a processor, the steps of the medical image registration method based on a deep learning neural network as described in any one of claims 2 to 4 are implemented.

Images