Unsupervised magnetic resonance image registration method, equipment, medium and product

Abstract

The invention discloses an unsupervised magnetic resonance image registration method and device, a medium and a product, and relates to the field of image processing, and the method comprises the steps: obtaining a target MRI image and a to-be-registered MRI image; according to the target MRI image and the MRI image to be registered, using an image registration model to determine a registered MRI image; wherein the image registration model is obtained by training a PyraMLP-Net model by using a training data set; the PyraMLP-Net model comprises a double-flow feature extraction module, a pyramid feature decoding module and a space conversion network module which are connected in sequence. According to the invention, high-precision and high-efficiency registration of the brain MRI image is realized.

Description

An unsupervised magnetic resonance image registration method, device, medium and product Technical Field

[0001] This application relates to the field of image processing, and in particular to an unsupervised magnetic resonance image registration method, device, medium, and product. Background Technology

[0002] In the field of magnetic resonance imaging (MRI) analysis, images often exhibit significant spatial and anatomical differences due to variations in data acquisition parameters, imaging equipment, and individual physiological characteristics, posing challenges to clinical diagnosis and disease assessment. Image registration technology is a key method for addressing this issue, spatially aligning two or more images acquired at different times, from different viewpoints, or in different imaging modes to provide a basis for disease diagnosis, interventional treatment, and disease progression monitoring.

[0003] Traditional image registration methods primarily rely on iterative optimization strategies, resulting in high computational complexity. Currently, the mainstream deep learning methods fall into two categories: those based on Convolutional Neural Networks (CNNs) and those based on Transformers. Both have limitations: CNNs' inherent local receptive field limits their ability to model long-range dependencies, leading to limited registration accuracy; Transformers possess global modeling capabilities, establishing connections between arbitrary locations in the image through self-attention mechanisms, significantly improving registration accuracy, but suffer from high computational complexity, typically limiting their application to downsampled feature resolutions and making it difficult to capture fine pixel-level correspondences at full image resolution. Developing novel algorithms that balance registration accuracy and computational efficiency is crucial for improving image registration accuracy and, consequently, enhancing the reliability of medical image-assisted diagnosis. Summary of the Invention

[0004] The purpose of this application is to provide an unsupervised magnetic resonance image registration method, device, medium, and product to improve the accuracy and efficiency of image registration.

[0005] To achieve the above objectives, this application provides the following solution: Firstly, this application provides an unsupervised magnetic resonance image registration method, comprising: acquiring a target MRI image and an MRI image to be registered; determining a registered MRI image using an image registration model based on the target MRI image and the MRI image to be registered; wherein the image registration model is obtained by training a PyraMLP-Net model using a training dataset; the PyraMLP-Net model includes a dual-stream feature extraction module, a pyramid feature decoding module, and a spatial transformation network module connected in sequence; the dual-stream feature extraction module is used to extract features from the target MRI image and the MRI image to be registered to obtain target MRI image features and MRI image features to be registered; the pyramid feature decoding module is used to fuse the target MRI image features and the MRI image features to be registered to generate a deformation field; the spatial transformation network module is used to generate a registered MRI image based on the deformation field and the MRI image to be registered.

[0006] In one embodiment, the dual-stream feature extraction module is a Siamese network, which includes two sets of sequentially connected first convolutional blocks, first max pooling layers, second convolutional blocks, second max pooling layers, third convolutional blocks, third max pooling layers, and first convolutional blocks.

[0007] In one implementation, each convolutional block includes two convolutional layers connected in sequence; each convolutional layer is followed by an instance normalization operation and a Leaky ReLU activation function.

[0008] In one embodiment, the pyramid feature decoding module includes: a first decoding layer, a second decoding layer, a third decoding layer, and a fourth decoding layer connected in sequence; the first decoding layer includes a first CMW-MLP module, a first convolutional layer, and a first upsampling layer connected in sequence; the second decoding layer includes a second upsampling layer, a second CMW-MLP module, a second convolutional layer, and a first fusion layer connected in sequence; the third decoding layer includes a fourth upsampling layer and a third upsampling layer, a third CMW-MLP module, a third convolutional layer, and a second fusion layer connected in sequence; the fourth decoding layer includes a sixth upsampling layer and a fifth upsampling layer, a fourth CMW-MLP module, a fourth convolutional layer, and a second fusion layer connected in sequence. The third fusion layer; the first CMW-MLP module is connected to two of the fourth convolutional blocks and the second upsampling layer respectively; the first upsampling layer is connected to the first fusion layer; the second CMW-MLP module is connected to two of the third convolutional blocks and the third upsampling layer respectively; the first fusion layer is connected to the fourth upsampling layer; the third CMW-MLP module is connected to two of the second convolutional blocks and the fifth upsampling layer respectively; the second fusion layer is connected to the sixth upsampling layer; the fourth CMW-MLP module is connected to two of the first convolutional blocks respectively; the third fusion layer is connected to the spatial transformation network module.

[0009] In one implementation, each Fusion layer consists of a spatial transformation network and residual connections.

[0010] In one embodiment, each CMW-MLP module includes a local correlation calculation unit, a multi-range dependency capture unit, and a feature optimization and output unit. The local correlation calculation unit includes a correlation layer, a fifth convolutional layer, and a first normalization layer. The multi-range dependency capture unit includes a 3×3×3 window, a first gated multilayer perceptron and a first region fusion subunit connected in sequence, a 5×5×5 window, a second gated multilayer perceptron and a second region fusion subunit connected in sequence, and a 7×7×7 window, a third gated multilayer perceptron and a third region fusion subunit connected in sequence. The 3×3×3 window, the 5×5×5 window, and the 7×7×7 window are all connected to the first normalization layer. The feature optimization and output unit includes a first global average pooling layer, two fully connected layers, a Softmax function layer, a linear projection layer, and a residual channel attention module connected in sequence.

[0011] In one embodiment, the residual channel attention module includes: a second normalization layer, a sixth convolutional layer, a seventh convolutional layer, and a channel attention layer based on a squeeze-excitation structure connected in sequence; the channel attention layer based on the squeeze-excitation structure includes a second global average pooling layer, an eighth convolutional layer, a ninth convolutional layer, and a sigmoid function layer connected in sequence.

[0012] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described unsupervised magnetic resonance image registration method.

[0013] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described unsupervised magnetic resonance image registration method.

[0014] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described unsupervised magnetic resonance image registration method.

[0015] According to the specific embodiments provided in this application, this application achieves the following technical effects: This application provides an unsupervised magnetic resonance image registration method, device, medium, and product, which acquires a target MRI image and an MRI image to be registered; based on the target MRI image and the MRI image to be registered, an image registration model is used to determine the registered MRI image; wherein, the image registration model is obtained by training a PyraMLP-Net model using a training dataset; the PyraMLP-Net model includes a dual-stream feature extraction module, a pyramid feature decoding module, and a spatial transformation network module connected in sequence. This application achieves high-precision and high-efficiency registration of brain MRI images by using the PyraMLP-Net model and combining it with a coarse-to-fine optimization strategy for the full-resolution deformation field. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 is a flowchart illustrating an unsupervised magnetic resonance image registration method according to an embodiment of this application; Figure 2 is a structural diagram of PyraMLP-Net according to an embodiment of this application.

[0018] Figure 3 is a structural diagram of a dual-stream feature extraction module provided in an embodiment of this application.

[0019] Figure 4 is a structural diagram of the CMW-MLP module provided in an embodiment of this application.

[0020] Figure 5 is a structural diagram of a gMLP provided in an embodiment of this application.

[0021] Figure 6 is a flowchart of training PyraMLP-Net according to an embodiment of this application.

[0022] Figure 7 is an execution flowchart of PyraMLP-Net provided in an embodiment of this application.

[0023] Figure 8 is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] This application proposes a feature pyramid network, PyraMLP-Net, based on a multi-layer perceptron (MLP). By using a correlation-aware multi-window MLP (CMW-MLP) module without self-attention mechanism and a coarse-to-fine optimization strategy for the full-resolution deformation field, it achieves high-precision and high-efficiency registration of brain MRI images.

[0027] In an exemplary embodiment, as shown in FIG1, an unsupervised magnetic resonance image registration method is provided, including the following steps: S1: acquiring a target MRI image and an MRI image to be registered.

[0028] S2: Based on the target MRI image and the MRI image to be registered, the registered MRI image is determined using an image registration model; wherein, the image registration model is obtained by training a PyraMLP-Net model using a training dataset; the PyraMLP-Net model includes a dual-stream feature extraction module, a pyramid feature decoding module, and a spatial transformation network module connected in sequence.

[0029] The dual-stream feature extraction module is used to extract features from the target MRI image and the MRI image to be registered, to obtain the target MRI image features and the MRI image to be registered features.

[0030] In this embodiment, the dual-stream feature extraction module is a Siamese network, which includes two sets of sequentially connected first convolutional blocks, first max-pooling layers, second convolutional blocks, second max-pooling layers, third convolutional blocks, third max-pooling layers, and first convolutional blocks. Each convolutional block includes two sequentially connected convolutional layers; each convolutional layer is followed by an instance normalization operation and a Leaky ReLU activation function.

[0031] The pyramid feature decoding module is used to fuse the target MRI image features and the MRI image features to be registered to generate a deformation field.

[0032] In this embodiment, the pyramid feature decoding module includes a first decoding layer, a second decoding layer, a third decoding layer, and a fourth decoding layer connected in sequence.

[0033] The first decoding layer includes a first CMW-MLP module, a first convolutional layer, and a first upsampling layer connected in sequence.

[0034] The second decoding layer includes a second upsampling layer, a second CMW-MLP module, a second convolutional layer, and a first Fusion layer connected in sequence.

[0035] The third decoding layer includes a fourth upsampling layer and a third upsampling layer, a third CMW-MLP module, a third convolutional layer, and a second Fusion layer connected in sequence.

[0036] The fourth decoding layer includes a sixth upsampling layer and a fifth upsampling layer, a fourth CMW-MLP module, a fourth convolutional layer, and a third Fusion layer connected in sequence.

[0037] The first CMW-MLP module is connected to the two fourth convolutional blocks and the second upsampling layer respectively; the first upsampling layer is connected to the first Fusion layer.

[0038] The second CMW-MLP module is connected to the two third convolutional blocks and the third upsampling layer respectively; the first Fusion layer is connected to the fourth upsampling layer.

[0039] The third CMW-MLP module is connected to the two second convolutional blocks and the fifth upsampling layer respectively; the second Fusion layer is connected to the sixth upsampling layer.

[0040] The fourth CMW-MLP module is connected to two of the first convolutional blocks respectively; the third Fusion layer is connected to the spatial transformation network module.

[0041] Each Fusion layer consists of a spatial transformation network and residual connections.

[0042] Each CMW-MLP module includes a local correlation calculation unit, a multi-range dependency capture unit, and a feature optimization and output unit.

[0043] The local correlation calculation unit includes a correlation layer, a fifth convolutional layer, and a first normalization layer.

[0044] The multi-range dependency capture unit includes a 3×3×3 window, a first gated multilayer perceptron and a first region fusion subunit connected in sequence, a 5×5×5 window, a second gated multilayer perceptron and a second region fusion subunit connected in sequence, a 7×7×7 window, a third gated multilayer perceptron and a third region fusion subunit connected in sequence; the 3×3×3 window, the 5×5×5 window and the 7×7×7 window are all connected to the first normalization layer.

[0045] The feature optimization and output unit includes a first global average pooling layer, two fully connected layers, a Softmax function layer, a linear projection layer, and a residual channel attention module connected in sequence.

[0046] In this embodiment, the residual channel attention module includes: a second normalization layer, a sixth convolutional layer, a seventh convolutional layer, and a channel attention layer based on a squeeze-excitation structure, connected in sequence.

[0047] The channel attention layer based on the squeeze-excitation structure includes a second global average pooling layer, an eighth convolutional layer, a ninth convolutional layer, and a sigmoid function layer connected in sequence.

[0048] The spatial transformation network module is used to generate a registered MRI image based on the deformation field and the MRI image to be registered.

[0049] The PyraMLP-Net model takes the target MRI image and the MRI image to be registered as input. First, it extracts multi-scale features from the input image through downsampling. Then, it upsamples the input image through a feature pyramid decoding module. During this process, the CWM-MLP module is used to calculate the local correlations between the input features, capture multi-range dependencies, and output a full-resolution deformation field. Finally, the MRI image to be registered and the deformation field are input into the Spatial Transformer Networks (STN) module to obtain the registered MRI image.

[0050] As shown in Figure 2, the PyraMLP-Net model consists of three modules: a two-stream feature extraction module, a pyramid feature decoding module, and an STN module.

[0051] (1) Dual-stream feature extraction module.

[0052] The dual-stream feature extraction module employs a Siamese network structure. Its inputs are a target MRI image and a MRI image to be registered, both with dimensions H×W×D (height×width×depth) and C channels. The outputs are two sets of four-level hierarchical feature pyramids. and (l=1,2,3,4), where, and These represent the target MRI image features and the MRI image features to be registered, respectively, output by the l-th convolutional module.

[0053] As shown in Figure 3, each subnetwork in the Siamese network consists of four consecutive convolutional blocks and three max-pooling layers. Each convolutional block contains two 3×3×3 convolutional layers (stride of 1, padding of the same value). Each convolutional layer is followed by a Leaky ReLU activation function with a parameter of 0.2 and instance normalization (IN). Downsampling is achieved between adjacent convolutional blocks through 2×2×2 max-pooling layers (stride of 2, padding of the same value), which halves the feature map size and doubles the number of channels module by module.

[0054] (2) Pyramid feature decoding module.

[0055] The feature pyramid decoding module employs a multi-level recursive fusion strategy, combining the traditional pyramid structure with CMW-MLP to progressively optimize the deformation field between the target MRI image and the MRI image to be registered. As shown in Figure 2, this module contains four decoding layers, corresponding to the four levels of the dual-stream feature extraction module, forming a symmetrical network structure.

[0056] In the first decoding layer, the module receives the target MRI image features output from the fourth level of the dual-stream feature extraction module. and features of the MRI images to be registered By calculating feature relevance using CMW-MLP, relevant perceptual fusion features F1 are generated. (1) Subsequently, F1 is processed by a 3×3×3 convolution (3 channels, 1 stride, same padding) to generate an initial displacement field of size H / 8×W / 8×D / 8×3. : (2) The number of channels is 3, representing the pixel displacement in the x, y, and z spatial dimensions. To make... Matching the displacement field size obtained from the next level operation, we perform an upsampling operation on it (3×3×3 kernel size, stride of 1, and same padding) to obtain: (3) The size is H / 4×W / 4×D / 4×3. Simultaneously, an upsampling operation is performed on F1, resulting in: (4) In the subsequent decoding layers (layers 2-4), a dual-path processing mechanism of feature fusion and deformation field refinement is adopted: First, the target MRI image features of the current layer are fused. Features of MRI images to be registered Upsampled fusion features output from the previous level CMW-MLP module ( After concatenation, the data is input into the CMW-MLP module of the current level to obtain the relevant perceptual fusion features F of the current level. i : (5) Subsequently, the network is divided into two processing branches: one branch generates a further refined deformation field through 3×3×3 convolution (3 channels, 1 stride, same padding). : (6) Constructing the Fusion layer will and the upsampled deformation field of the previous level By fusing, we obtain: (7) Among them, This represents the fusion deformation field at the current level. Subsequently, for... Upsampling yields: (8) Among them, This is the upsampled deformation field for the current level. Another branch is for F. i Upsampling yields: (9) Among them, This refers to the upsampling-related perceptual fusion features at the current level. Input the next decoding layer for splicing.

[0057] The fusion layer consists of STN and residual connections, where the STN converts the upsampled deformation field obtained from the previous stage into a fusion layer. Compared with the deformation field predicted by the current level Precise alignment ensures spatial consistency of the deformation field; residual connections will connect the aligned components. Compared with the deformation field predicted by the current level Residual fusion is achieved by adding elements one by one. (10) Finally, the fourth decoding layer outputs the final deformed field. : (11) With the MRI image to be registered Registered MRI images are generated using the STN module. w : In equation (12), "°" represents the spatial transformation operation performed by STN.

[0058] (3) CMW-MLP module.

[0059] As shown in Figure 4, the CMW-MLP module employs a three-stage processing flow: local correlation calculation, multi-range dependency capture, and feature optimization and output. The working principle is explained below using the CMW-MLP module in the 2nd-4th decoding layers as an example.

[0060] ① Local correlation calculation (i.e., stage 1 in Figure 4).

[0061] First, the three feature maps to be matched are... and (N is the batch size) Input the relevant layer and calculate the results in a 3×3×3 local neighborhood. , and The correlation between them is used to generate the corresponding correlation feature map. , and .

[0062] Then, the feature map to be matched , , Correlation feature maps respectively , , The data is concatenated along the channel dimension and then compressed through a 3×3×3 convolutional layer (stride 1, padding same) to obtain the relevant perceptual feature map. .

[0063] Finally, to ensure consistency in feature scale, the relevant perceptual feature maps are... Perform layer normalization to obtain initial normalized features. .

[0064] ② Multi-range dependency capture (i.e., stage 2 in Figure 4).

[0065] In the multi-scale dependency capture stage, feature dependencies at different scales are captured through parallel window branches of a multi-window MLP. Specifically, the process is as follows: First, the initial normalized features are processed according to window sizes of 3×3×3 (fine-grained), 5×5×5 (medium-scale), and 7×7×7 (large-scale), respectively. Perform non-overlapping partitioning to generate corresponding sub-block features. (k=1, 2, 3). For feature map boundary regions that are not divisible, zero-padding is used to ensure the integrity of the window division.

[0066] Then, a gated multilayer perceptron (gMLP) based on a gated mechanism is used to analyze the sub-block features. (k=1, 2, 3) are processed to mine long-range spatial dependencies within the sub-block regions. As shown in Figure 5, the specific implementation process is as follows: Step 1: Process sub-block features Perform layer normalization to eliminate differences in the distribution of feature values.

[0067] Step 2: Transform the channel dimension from... using a linear projection layer. Expand to Furthermore, the Gaussian Error Linear Unit (GeLU) activation function is applied to enhance the nonlinear expressive power, thereby obtaining intermediate features. .

[0068] Step 3: Transfer intermediate features Divide into content branches along the channel dimension. and gated branches ,in, Preserve the semantic content features of sub-blocks. Used to generate space-gated signals and filter Key registration locations (such as structural edge pixels) in the data.

[0069] Step 4: After layer normalization and learning the spatial dependencies within sub-blocks through a linear transformation layer, spatial gating signals are generated. : In equation (13), It is a learnable linear transformation matrix, P is the total number of elements in the sub-block, and Linear() represents the linear transformation.

[0070] Step 5: and Element-wise multiplication is performed to obtain the gated fusion features. This feature, through a linear projection layer, reduces the channel dimension from... Restore to and sub-block features The residuals are concatenated and added together to form the final output: In formula (14), It is the weight matrix of the linear projection layer, and its function is to restore the channel number to its original value. , This represents element-wise multiplication. Indicates to The element-wise increment operation is used to gate the signal during the initial training phase. When it is weak (close to 0), avoid The original semantic information is excessively suppressed.

[0071] Finally, the sub-block features processed by gMLP are... The features were re-merged into a complete feature map, resulting in: (15) In the formula, k = 1, 2, 3, This is the complete feature map after merging the k-th window branches, and `unsplit_images()` is the sub-block merging function. S is the sub-block mesh size in the k-th window branch, i.e., the number of sub-blocks in the feature map in the depth, height, and width directions. k Window size: (16) ③ Feature optimization and output.

[0072] In the feature optimization and output stage, the complete feature map after merging the three window branches is first processed. , and Element-wise summation is performed to obtain preliminary fused features. Then, Global Average Pooling (GAP) is used to compress the spatial dimension and extract feature vectors with global information, resulting in feature vector a: (17) The feature vector a undergoes a nonlinear transformation through two fully connected layers to learn the mapping relationship between channel information and the weights of each window. Then, the initial weight scores of the output are normalized using the Softmax function to generate the dynamic weights of each window branch, resulting in: In equation (18), FC(·) represents the fully connected layer operation. The weight of the k-th window branch satisfies .

[0073] The learned dynamic weights and the output feature maps of the three window branches are adaptively weighted and fused, and the result is then input into a linear projection layer to adjust the feature dimensions. Finally, it is combined with the initial normalized feature F. LN Perform residual connections to obtain fused features: In equation (19), It is a feature of fusion. It is a learnable linear projective weight matrix.

[0074] To further enhance feature representation capabilities, a Residual Channel Attention Block (RCAB) is introduced. This module consists of layer normalization, two 3D convolutional layers, and a squeeze-and-excitation channel attention layer (SE-CAL) based on a squeeze-and-excitation structure. The specific processing is as follows: First, layer normalization is applied to the FMLP. Then, feature transformation is performed through two 3×3×3 3D convolutional layers (stride 1, padding same). The first 3D convolutional layer is used for spatial feature extraction, and the second 3D convolutional layer refines the features. A Leaky ReLU activation function is used in between to enhance non-linear representation capabilities, resulting in intermediate features after convolutional transformation. .

[0075] The channel attention mechanism employs an improved squeeze-excitation structure, which uses global average pooling to optimize F... Conv The spatial information of each channel is aggregated into a global statistic, thereby avoiding local noise interference and highlighting the global contribution of each channel. Then, two 1×1×1 3D convolutional layers (with a stride of 1, same padding, and Leaky ReLU activation in between) are used to learn the dependencies between channels. Finally, a sigmoid function is used to generate channel attention weights z. z close to 1 indicates that the channel is crucial for registration, while z close to 0 indicates that the channel is redundant.

[0076] Finally, z and F Conv Channel-by-channel multiplication and fusion feature F MLP Perform residual connections to obtain relevant perceptual fusion features. : (20) In this embodiment, the process of training the PyraMLP-Net model is as follows: (1) Determine the loss function.

[0077] To achieve a good balance between registration accuracy and deformation smoothness, the total network loss consists of two parts: image similarity loss and spatial regularization term. In equation (21), It is image similarity loss. It is a space regularization term. It is the target MRI image. It is a hyperparameter that adjusts the weights of the two items, and is determined through cross-validation.

[0078] Image similarity loss is used to evaluate the similarity between the target MRI image and the registered MRI image, and is defined as: In equation (22), N represents the image voxel space. pThe local neighborhood centered on voxel p (this application uses a 9×9×9 cube neighborhood), p i It is N p The i-th pixel within, p in the deformation field i The value at that location, p in the target MRI image i The intensity value at that location, Voxels in the registered MRI image The intensity value at that location, It is the average pixel intensity value within a local neighborhood Np of the target MRI image. In the registered MRI images, in voxel The average pixel intensity value within the local neighborhood centered on the pixel.

[0079] The space regularization term is used to constrain the geometric properties of the deformation field to ensure its smoothness and topological rationality, and is defined as follows: In equation (23), It is a deformation field In voxels The spatial gradient at a given point is calculated using the central difference: In equation (24), , and These are deformation fields In voxels The gradient along the x, y, and z directions.

[0080] (2) Training steps.

[0081] The training strategy and hyperparameter settings for the PyraMLP-Net model are shown in Table 1, and the training process is shown in Figure 6. Checkpoints are set after each training round to verify model performance, and finally, the model parameters with the highest performance are retained.

[0082] Table 1 Network Training Strategy and Hyperparameter Settings

[0083] The training process is as follows: Step 1: Initially, the number of times the training set is traversed is epoch=0.

[0084] Step 2: Initialize the training set index i=1.

[0085] Step 3: The i-th mini-batch training set in training set A The N samples are input into the network for forward propagation.

[0086] Step 4: Set the loss function .

[0087] Step 5: Set the gradient of each layer parameter in the network to 0.

[0088] Step 6: Calculate the gradient of each layer of the network using the backpropagation algorithm.

[0089] Step 7: Optimize the network parameters using the Adam gradient descent algorithm, setting the learning rate to 0.0001.

[0090] Step 8: Let i ← i+1. If i ≤ N, return to step 3; otherwise, let epoch ← epoch+1 and execute step 9.

[0091] Step 9: If epoch < 500, return to step 2; otherwise, stop traversing and the training process ends.

[0092] After training, the network execution steps are shown in Figure 7, and are as follows: Step 1: Initialize network parameters: number of feature channels channels=8, number of convolutions conv=0, number of poolings pool=0.

[0093] Step 2: Input the target MRI image and the MRI image to be registered into the dual-stream feature extraction module, and perform one convolutional block operation through the `channels` feature channels: each convolutional block contains two 3×3×3 convolutions (stride of 1, padding of 1), two instance normalizations, and two Leaky ReLU activation functions to obtain the target MRI image feature layer map. t,c and the feature layer map of the MRI image to be registered m,c .

[0094] Step 3: Let conv←conv+2.

[0095] Step 4: If conv < 2, execute step 2; if conv = 2, execute step 5.

[0096] Step 5: Map the feature layers of the target MRI image t,c and the feature layer map of the MRI image to be registered m,c Downsampling is performed after a max-pooling layer (kernel size 2×2×2, stride 2, padding 2), followed by one convolutional block operation to generate two channels of linear activation responses. The Leaky ReLU function value of each linear activation response is calculated to obtain the target MRI feature layer map after one downsampling. t,c+1 and the MRI feature layer to be registered after one downsampling (map) m,c+1 .

[0097] Step 6: Let conv←conv+2, channels←channels×2, pool←pool+1.

[0098] Step 7: If conv < 4, proceed to step 5; if conv = 4, proceed to step 8.

[0099] Step 8: Map the target MRI feature layer after one downsampling. t,c+1 and the MRI feature layer to be registered after one downsampling (map) m,c+1 Downsampling is performed after a max-pooling layer (kernel size 2×2×2, stride 2, padding 2), followed by one convolutional block operation to generate two channels of linear activation responses. The LeakyReLU function value of each linear activation response is calculated to obtain the target MRI feature layer map after secondary downsampling. t,c+2 and the MRI feature layer to be registered after secondary downsampling (map) m,c+2 .

[0100] Step 9: Let conv←conv+2, channels←channels×2, pool←pool+1.

[0101] Step 10: If conv < 6, proceed to step 8; if conv = 6, proceed to step 11.

[0102] Step 11: Map the target MRI feature layer after secondary downsampling. t,c+2 and the MRI feature layer to be registered after secondary downsampling (map) m,c+2 Downsampling is performed after a max-pooling layer (kernel size 2×2×2, stride 2, padding 2), followed by one convolutional block operation to generate two channels of linear activation responses. The LeakyReLU function value of each linear activation response is calculated to obtain the target MRI feature layer map after three downsampling operations. t,c+3 and the MRI feature layer to be registered after three downsampling steps (map) m,c+3 .

[0103] Step 12: Let conv←conv+2, channels←channels×2, pool←pool+1.

[0104] Step 13: If conv < 8, proceed to step 11; if conv = 8, proceed to step 14.

[0105] Step 14: Map the target MRI feature layer after three downsampling steps. t,c+3 and the MRI feature layer to be registered after three downsampling steps (map) m,c+3 Input into the CMW-MLP module.

[0106] Step 15: Map the target MRI feature layer after three downsampling steps.t,c+3 and the MRI feature layer to be registered after three downsampling steps (map) m,c+3 Pairwise correlation calculations are performed within a 3×3×3 local neighborhood using a 3D correlation layer, followed by channel compression using a 3×3×3 convolution (stride 1, padding 1) to obtain the relevant perceptual feature map. 1,c Then perform layer normalization on it to obtain the initial feature. 2,c .

[0107] Step 16: Let conv←conv+1.

[0108] Step 17: Then, the initial features are... 2,c Non-overlapping segmentation was performed using different window sizes (3×3×3, 5×5×5, 7×7×7) to obtain sub-block features. gMLP was then used to model feature relationships within each sub-block. Subsequently, the `unsplit_images()` sub-block merging operation was used to fuse the gMLP-processed features, resulting in the fused features corresponding to the three windows. 3,c feature 4,c and feature 5,c .

[0109] Step 18: Combine the fused features corresponding to the three windows. 3,c feature 4,c and feature 5,c Element-wise summation and global average pooling are performed, followed by a non-linear transformation through two fully connected layers, and then normalization is achieved using the Softmax function to obtain the dynamic weights of each window branch. 1,c .

[0110] Step 19: Let pool ← pool + 1.

[0111] Step 20: Merge the features corresponding to the three windows. 3,c feature 4,c and feature 5,c Dynamic weights of each window branch 1,c Perform adaptive weighted fusion, then input the linear projection layer to adjust the feature dimensions, and the output result is the same as the feature... 1,c Add them together to obtain the fused feature. 6,c .

[0112] Step 21: Fuse features 6,cAfter layer normalization, two 3×3×3 convolution operations are performed (stride 1, padding 1). A Leaky ReLU activation function is applied between the two convolution operations to obtain intermediate features. 7,c .

[0113] Step 22: Let conv←conv+2.

[0114] Step 23: Transfer intermediate features 7,c Global average pooling is performed, followed by two 1×1×1 convolutions (stride 1, padding 1). A Leaky ReLU activation function is applied between the two convolutions. Finally, the sigmoid function is used to generate the channel attention weights. 2,c .

[0115] Step 24: Let conv←conv+2, pool←pool+1.

[0116] Step 25: Transfer intermediate features 7,c and channel attention weight 2,c Multiply by channel, then combine with the fused feature 6,c Perform addition to output the relevant perceptual fusion feature. 8,c .

[0117] Step 26: Relevant perceptual fusion features 8,c Perform a 3×3×3 convolution (stride 2, padding 1) to obtain a deformed field field1 with a channel dimension of 3. Then perform an upsampling operation to obtain an upsampled deformed field field. 1,up .

[0118] Step 27: Let conv←conv+1.

[0119] Step 28: Relevant perceptual fusion features 8,c After performing the upsampling operation, and with map t,c+2 and map m,c+2 The data is input into the CMW-MLP module, and steps 14 to 25 are executed to output the upsampled fused feature. 8,c+1 .

[0120] Step 29: Fuse features from a single upsampling operation 8,c+1 Performing a 3×3×3 convolution (stride 2, padding 1) yields a deformable field field2 with a channel dimension of 3. Then, the field... 1,upThe field is fused with the input field2 from the Fusion module, and then upsampled to output a secondary upsampled deformed field. 2,up .

[0121] Step 30: Let conv←conv+1.

[0122] Step 31: Fuse features from a single upsampling operation 8,c+1 After performing the upsampling operation, and with map t,c+1 and map m,c+1 The data is input into the CMW-MLP module, and steps 14 to 25 are executed to output the secondary upsampling fused feature. 8,c+2 .

[0123] Step 32: Fusion of features by secondary upsampling 8,c+2 Performing a 3×3×3 convolution (stride 2, padding 1) yields a deformable field field3 with a channel dimension of 3. Then, the field... 2,up The field is fused with the field3 input to the Fusion module, and then upsampled to output a cubic upsampled deformed field. 3,up .

[0124] Step 33: Let conv←conv+1.

[0125] Step 34: Fusion of features by secondary upsampling 8,c+2 After performing the upsampling operation, and with map t,c and map m,c The data is input into the CMW-MLP module, and steps 14 to 25 are executed to output the triple upsampling fused feature. 8,c+3 .

[0126] Step 35: Fuse features from three upsampling operations 8,c+3 Performing a 3×3×3 convolution (stride 2, padding 1) yields a deformable field field4 with a channel dimension of 3. Then, the field... 3,up After fusing with the field4 input Fusion module, the final deformed field field is output.

[0127] Step 36: Let conv←conv+1.

[0128] Step 37: Input the MRI image to be registered and the deformation field into the STN module to perform spatial transformation operation to obtain the registered MRI image.

[0129] This application proposes a feature pyramid network, PyraMLP-Net, based on a multi-layer perceptron (MLP). Through the CMW-MLP module and a coarse-to-fine optimization strategy for the full-resolution deformation field, it achieves high-precision and high-efficiency registration of brain MRI images.

[0130] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described unsupervised magnetic resonance image registration method.

[0131] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described unsupervised magnetic resonance image registration method.

[0132] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described unsupervised magnetic resonance image registration method.

[0133] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram is shown in Figure 8. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The I / O interfaces of the computer device are used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an unsupervised magnetic resonance image registration method.

[0134] Those skilled in the art will understand that the structure shown in Figure 8 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or may combine certain components, or may have different component arrangements.

[0135] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0136] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0137] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An unsupervised magnetic resonance image registration method, characterized in that, include: Acquire the target MRI image and the MRI image to be registered; Based on the target MRI image and the MRI image to be registered, an image registration model is used to determine the registered MRI image. The image registration model is trained using a training dataset on a PyraMLP-Net model. The PyraMLP-Net model includes a two-stream feature extraction module, a pyramid feature decoding module, and a spatial transformation network module connected in sequence. The two-stream feature extraction module extracts features from the target MRI image and the MRI image to be registered, obtaining target MRI image features and MRI image features to be registered. The pyramid feature decoding module fuses the target MRI image features and MRI image features to be registered to generate a deformation field. The spatial transformation network module generates the registered MRI image based on the deformation field and the MRI image to be registered.

2. The unsupervised magnetic resonance image registration method according to claim 1, characterized in that, The dual-stream feature extraction module is a Siamese network, which includes two sets of sequentially connected first convolutional blocks, first max pooling layers, second convolutional blocks, second max pooling layers, third convolutional blocks, third max pooling layers, and first convolutional blocks.

3. The unsupervised magnetic resonance image registration method according to claim 2, characterized in that, Each convolutional block consists of two convolutional layers connected in sequence; each convolutional layer is followed by an instance normalization operation and a Leaky ReLU activation function.

4. The unsupervised magnetic resonance image registration method according to claim 2, characterized in that, The pyramid feature decoding module includes: a first decoding layer, a second decoding layer, a third decoding layer, and a fourth decoding layer connected in sequence; the first decoding layer includes a first CMW-MLP module, a first convolutional layer, and a first upsampling layer connected in sequence; the second decoding layer includes a second upsampling layer, a second CMW-MLP module, a second convolutional layer, and a first fusion layer connected in sequence; the third decoding layer includes a fourth upsampling layer and a third upsampling layer, a third CMW-MLP module, a third convolutional layer, and a second fusion layer connected in sequence; the fourth decoding layer includes a sixth upsampling layer and a fifth upsampling layer, a fourth CMW-MLP module, a fourth convolutional layer, and a third fusion layer connected in sequence. The first CMW-MLP module is connected to two of the fourth convolutional blocks and the second upsampling layer, respectively; the first upsampling layer is connected to the first Fusion layer; the second CMW-MLP module is connected to two of the third convolutional blocks and the third upsampling layer, respectively; the first Fusion layer is connected to the fourth upsampling layer; the third CMW-MLP module is connected to two of the second convolutional blocks and the fifth upsampling layer, respectively; the second Fusion layer is connected to the sixth upsampling layer; the fourth CMW-MLP module is connected to two of the first convolutional blocks, respectively; the third Fusion layer is connected to the spatial transformation network module.

5. The unsupervised magnetic resonance image registration method according to claim 4, characterized in that, Each Fusion layer consists of a spatial transformation network and residual connections.

6. The unsupervised magnetic resonance image registration method according to claim 4, characterized in that, Each CMW-MLP module includes a local correlation calculation unit, a multi-range dependency capture unit, and a feature optimization and output unit. The local correlation calculation unit includes a correlation layer, a fifth convolutional layer, and a first normalization layer. The multi-range dependency capture unit includes a 3×3×3 window, a first gated multilayer perceptron and a first region fusion subunit connected in sequence, a 5×5×5 window, a second gated multilayer perceptron and a second region fusion subunit connected in sequence, a 7×7×7 window, a third gated multilayer perceptron and a third region fusion subunit connected in sequence. The 3×3×3 window, the 5×5×5 window, and the 7×7×7 window are all connected to the first normalization layer. The feature optimization and output unit includes a first global average pooling layer, two fully connected layers, a Softmax function layer, a linear projection layer, and a residual channel attention module connected in sequence.

7. The unsupervised magnetic resonance image registration method according to claim 6, characterized in that, The residual channel attention module includes: a second normalization layer, a sixth convolutional layer, a seventh convolutional layer, and a channel attention layer based on a squeeze-excitation structure, connected in sequence; the channel attention layer based on the squeeze-excitation structure includes a second global average pooling layer, an eighth convolutional layer, a ninth convolutional layer, and a sigmoid function layer connected in sequence.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the unsupervised magnetic resonance image registration method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the unsupervised magnetic resonance image registration method according to any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the unsupervised magnetic resonance image registration method according to any one of claims 1-7.

Images