Artifact removal method and device, electronic equipment and storage medium

Abstract

The application provides an artifact removal method and device, electronic equipment and a storage medium. The method comprises: acquiring multi-channel data, and performing Fourier transform on the multi-channel data to obtain corresponding k-space data; generating a plurality of masks based on a plurality of preset acceleration multiples; multiplying the plurality of masks with the k-space data respectively, and performing inverse Fourier transform on the multiplied results to obtain a plurality of multi-channel images; generating a plurality of corresponding single-channel images based on the plurality of multi-channel images; and inputting the plurality of single-channel images into an image fusion model to obtain a fused image. The application can remove artifacts of different degrees while better preserving image details and structures, and the image quality is high.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to methods, apparatuses, electronic devices and storage media for artifact removal. Background Technology

[0002] Magnetic resonance imaging (MRI) is a radiation-free imaging technique that provides high-contrast, high-resolution MR images. It has become an important clinical medical examination tool, providing valuable information for clinical imaging diagnosis.

[0003] Although magnetic resonance imaging (MRI) technology is now very mature, image quality can still be affected by factors such as artifacts. Artifacts are various forms of images that appear in the image but do not actually exist. The presence of artifacts reduces image quality and affects the diagnosis and analysis of diseases.

[0004] Therefore, there is an urgent need to provide methods, apparatus, electronic devices and storage media for artifact removal to improve the shortcomings of existing technologies. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, electronic device and storage medium for artifact removal, which removes artifacts of varying degrees while better preserving image details and structure, resulting in high image quality.

[0006] In a first aspect, this application provides a method for artifact removal, the method comprising:

[0007] Acquire multi-channel data and perform Fourier transform on the multi-channel data to obtain the corresponding k-space data;

[0008] Multiple masks are generated based on multiple preset acceleration factors, with each acceleration factor corresponding to one mask;

[0009] The multiple masks are multiplied with the k-space data respectively, and the results of the multiplication are subjected to inverse Fourier transform to obtain multiple multi-channel images.

[0010] Based on the multiple multi-channel images, multiple corresponding single-channel images are generated, with each multi-channel image corresponding to one single-channel image.

[0011] The multiple single-channel images are input into the image fusion model to obtain a fused image.

[0012] In some optional embodiments, generating multiple masks based on preset multiple acceleration factors includes:

[0013] For each acceleration factor, based on the size of the k-space data and the acceleration factor, calculate the total number of actual sampling lines of the k-space data in the phase coding direction;

[0014] Based on the total number of actual sampling lines, determine the number of actual sampling lines in the central region of the k-space data;

[0015] Random sampling is performed based on a preset probability distribution function to generate a mask corresponding to the acceleration factor.

[0016] In some optional embodiments, generating multiple corresponding single-channel images based on the multiple multi-channel images includes:

[0017] For each multi-channel image, channel compression is performed on the pixels of all channels at the same position in the multi-channel image to obtain the corresponding single-channel image.

[0018] In some optional embodiments, the plurality of single-channel images include the first single-channel image to the nth single-channel image, where n is an integer greater than 1;

[0019] The image fusion model includes convolutional modules from the first to the nth convolutional module;

[0020] The input of the first convolutional module includes the first single-channel image, and the output includes the corresponding first output image and the first feature map.

[0021] The input to the i-th convolutional module includes the i-th single-channel image and the (i-1)-th output image, and the output includes the corresponding i-th output image and the i-th feature map. <i≤n。

[0022] In some optional embodiments, the image fusion model further includes an attention module, and the process of obtaining the fused image includes:

[0023] Channel compression is performed on the first to nth feature maps respectively to obtain n single-channel feature maps;

[0024] The n single-channel feature maps are concatenated, and the concatenated feature map is input into the attention module to obtain a probability map;

[0025] The first to the nth output images are stitched together by channels, and the stitched channel image is fused with the probability map to obtain the fused image.

[0026] In some optional embodiments, the image fusion model is trained using a pre-defined recurrent neural network, and the training process of the image fusion model includes:

[0027] Multiple single-channel images of samples are acquired, and the multiple single-channel images of samples are respectively input into the recurrent neural network to obtain a sample fusion image;

[0028] The recurrent neural network is trained using the fused sample image and the standard fused image corresponding to the multiple single-channel sample images to obtain an image fusion model.

[0029] In some optional embodiments, training the recurrent neural network using the fused sample image and the standard fused image corresponding to the plurality of single-channel sample images includes:

[0030] Using a preset loss function, the sample fused image, and the standard fused image, calculate the preset loss corresponding to the recurrent neural network;

[0031] The adversarial loss corresponding to the recurrent neural network is calculated using the adversarial loss function, the sample fused image, and the standard fused image;

[0032] The parameters of the recurrent neural network are updated based on the preset loss and the adversarial loss.

[0033] Secondly, this application provides an artifact removal apparatus, the apparatus comprising:

[0034] The image transformation module is used to acquire multi-channel data and perform Fourier transform on the multi-channel data to obtain the corresponding k-space data.

[0035] The mask generation module is used to generate multiple masks based on multiple preset acceleration factors, with each acceleration factor corresponding to one mask;

[0036] The inverse transform module is used to multiply the multiple masks with the k-space data respectively, and perform an inverse Fourier transform on the multiplication result to obtain multiple multi-channel images;

[0037] The channel compression module is used to generate multiple corresponding single-channel images based on the multiple multi-channel images, with each multi-channel image corresponding to one single-channel image;

[0038] The image fusion module is used to input the multiple single-channel images into the image fusion model to obtain a fused image.

[0039] In some optional embodiments, the mask generation module includes:

[0040] For each acceleration factor, based on the size of the k-space data and the acceleration factor, calculate the total number of actual sampling lines of the k-space data in the phase coding direction;

[0041] Based on the total number of actual sampling lines, determine the number of actual sampling lines in the central region of the k-space data;

[0042] Random sampling is performed based on a preset probability distribution function to generate a mask corresponding to the acceleration factor.

[0043] In some optional embodiments, the channel compression module is used for:

[0044] For each multi-channel image, channel compression is performed on the pixels of all channels at the same position in the multi-channel image to obtain the corresponding single-channel image.

[0045] In some optional embodiments, the plurality of single-channel images include the first single-channel image to the nth single-channel image, where n is an integer greater than 1;

[0046] The image fusion model includes convolutional modules from the first to the nth convolutional module;

[0047] The input of the first convolutional module includes the first single-channel image, and the output includes the corresponding first output image and the first feature map.

[0048] The input to the i-th convolutional module includes the i-th single-channel image and the (i-1)-th output image, and the output includes the corresponding i-th output image and the i-th feature map. <i≤n。

[0049] In some optional embodiments, the image fusion model further includes an attention module, and the process of obtaining the fused image includes:

[0050] Channel compression is performed on the first to nth feature maps respectively to obtain n single-channel feature maps;

[0051] The n single-channel feature maps are concatenated, and the concatenated feature map is input into the attention module to obtain a probability map;

[0052] The first to the nth output images are stitched together by channels, and the stitched channel image is fused with the probability map to obtain the fused image.

[0053] In some optional embodiments, the image fusion model is trained using a pre-defined recurrent neural network, and the training process of the image fusion model is implemented using the following modules:

[0054] The sample fusion module is used to acquire multiple single-channel images of samples and input the multiple single-channel images of samples into the recurrent neural network to obtain a sample fused image.

[0055] The fusion training module is used to train the recurrent neural network using the fused sample image and the standard fused image corresponding to the multiple single-channel sample images to obtain an image fusion model.

[0056] In some optional embodiments, the fusion training module includes:

[0057] The first loss unit is used to calculate the preset loss corresponding to the recurrent neural network using a preset loss function, the sample fusion image, and the standard fusion image;

[0058] The second loss unit is used to calculate the adversarial loss corresponding to the recurrent neural network using the adversarial loss function, the sample fused image, and the standard fused image.

[0059] The parameter update unit is used to update the parameters of the recurrent neural network based on the preset loss and the adversarial loss.

[0060] Thirdly, this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above-described artifact removal methods.

[0061] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described artifact removal methods.

[0062] The aforementioned artifact removal method, apparatus, electronic device, and storage medium first perform a Fourier transform on the multi-channel data to obtain the corresponding k-space data. Then, multiple masks are generated according to different speedup factors. Masks obtained by sampling under different speedup conditions can filter out artifacts in the k-space data to varying degrees. Next, the multiple masks are multiplied with the k-space data respectively, and then an inverse Fourier transform is performed to obtain multiple multi-channel images, which are images with artifacts removed to varying degrees. These multiple multi-channel images are then compressed to generate multiple corresponding single-channel images. Finally, the multiple single-channel images are input into an image fusion model for image fusion. The resulting fused image, while removing artifacts to varying degrees, retains image details and structures relatively well, resulting in high image quality and providing significant assistance to doctors in diagnostic analysis. Attached Figure Description

[0063] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0064] Figure 1 This is a flowchart illustrating an artifact removal method provided in an embodiment of this application.

[0065] Figure 2 This is a schematic diagram of the mask generation process provided in the embodiments of this application.

[0066] Figure 3This is a schematic diagram of the structure of an image fusion network provided in an embodiment of this application.

[0067] Figure 4 This is a schematic diagram of a process for obtaining a fused image provided in an embodiment of this application.

[0068] Figure 5 This is a schematic diagram of the process of training an image fusion model provided in an embodiment of this application.

[0069] Figure 6 This is a schematic diagram of another training image fusion model provided in an embodiment of this application.

[0070] Figure 7 This is a structural block diagram of an artifact removal device provided in an embodiment of this application.

[0071] Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0072] Figure 9 This is a schematic diagram of the structure of a program product provided in an embodiment of this application. Detailed Implementation

[0073] The present application will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0074] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0075] It should also be noted that, in the embodiments of this application, the words "exemplary" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any implementation or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other implementations or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0076] Magnetic resonance imaging (MRI) is a radiation-free imaging modality that provides high-contrast, high-resolution images, and has become an important clinical medical examination tool, providing invaluable information for clinical imaging diagnosis. Although MRI has many advantages compared to other imaging modalities, its long scan time can cause discomfort for patients. Patients' involuntary movements within the device can also lead to motion artifacts in the MRI images. Furthermore, organ movements such as breathing, swallowing, and natural heartbeat can cause image blurring and contrast distortion, hindering the further promotion and development of MRI.

[0077] The root cause of artifacts in magnetic resonance imaging (MRI) is often due to anomalous data in the k-space phase encoding direction. The stronger the artifact, the more anomalous data exists in the k-space phase encoding direction.

[0078] In recent years, deep learning has gradually been applied to the field of artifact removal in magnetic resonance imaging, but the actual application results have not been good. There are three main approaches: First, training convolutional neural networks based on simulated motion artifact data; this approach performs poorly in real-world scenarios. Second, training the network using real data from limited scenarios; while the trained model can achieve artifact removal in similar scenes, it is difficult to generalize to large-scale applications, limiting its usability. Third, treating artifact removal as a style transfer problem without paired training data, using GAN networks to perform image transformation; this approach also exhibits poor performance.

[0079] The following section will introduce the methods, devices, electronic equipment, and storage media for artifact removal. While removing artifacts of varying degrees, the image details and structure are well preserved, resulting in high image quality and providing significant assistance to doctors in diagnostic analysis.

[0080] See Figure 1 , Figure 1 This is a flowchart illustrating an artifact removal method provided in an embodiment of this application.

[0081] The method includes steps S101 to S105.

[0082] Step S101: Acquire multi-channel data and perform Fourier transform on the multi-channel data to obtain the corresponding k-space data.

[0083] k-space refers to the Fourier dual space, that is, the frequency space of the Fourier transform, also known as Fourier space.

[0084] In this embodiment, the multi-channel data can be multi-channel (image) data obtained from magnetic resonance scanning. Correspondingly, k-space is the space carrying magnetic resonance image data with frequency space as the coordinate system. Through subsequent image processing (e.g., inverse Fourier transform), the k-space data can be converted into the corresponding magnetic resonance image (i.e., the multi-channel image below).

[0085] Step S102: Generate multiple masks based on multiple preset acceleration factors, with each acceleration factor corresponding to one mask.

[0086] The acceleration factor, also known as the speedup factor, in this embodiment refers to the acceleration factor in the phase encoding direction, which is the acceleration factor for parallel acquisition.

[0087] For example, with a speedup factor of 3, randomly sampling one line from every three k-space lines in k-space can reduce the sampling time to one-third of the original. The maximum speedup factor that can be enabled depends on the number of coil units in the phase encoding direction and the number of RF channels for the received signal.

[0088] In a specific application, the acceleration factor can be gradually reduced, and the specific acceleration factor can be set according to the actual situation of multi-channel data.

[0089] For example, the first acceleration factor is set to 6, the second acceleration factor is set to 4, and the third acceleration factor is set to 2. The smaller the acceleration factor, the more k-space lines are actually collected, and the longer the sampling time.

[0090] See Figure 2 , Figure 2 This is a schematic diagram of the mask generation process provided in the embodiments of this application.

[0091] In some embodiments, step S102 may include steps S201 to S203.

[0092] Step S201: For each acceleration factor, based on the size of the k-space data and the acceleration factor, calculate the total number of actual sampling lines of the k-space data in the phase encoding direction.

[0093] In a specific application: the size of the k-space data is used to indicate the total number of k-space lines. The total number of actual k-space lines in the phase encoding direction is calculated as follows:

[0094] Total number of actual mining lines = Total number of k-space lines ÷ Acceleration factor.

[0095] For example, if the total number of k-space lines is 50 and the acceleration factor is 2, then the total number of actual mining lines is 25.

[0096] Step S202: Based on the total number of actual sampling lines, determine the number of actual sampling lines in the central region of the k-space data.

[0097] The central region of k-space data refers to the area within a certain distance from the center point of k-space, where the number of actual mining lines in the central region is less than or equal to the total number of actual mining lines.

[0098] In a specific application, the number of actual sampling lines in the central area is determined as follows:

[0099] Using the middle mining line as a reference, take the same number of mining lines above and below as the mining lines for the central area.

[0100] For example, if there are a total of 25 mining lines, and the middle mining line is the 13th mining line, and taking the 13th k-space line as the reference, take 3 mining lines above and below it, then there are 7 mining lines in the central area (including the 13th mining line), namely the 10th to the 16th mining lines.

[0101] Step S203: Random sampling is performed based on a preset probability distribution function to generate a mask corresponding to the acceleration factor.

[0102] In this embodiment of the application, the preset probability distribution function can be a Gaussian distribution function.

[0103] Step S203 may include:

[0104] Based on the total number of actual mining lines and the number of actual mining lines in the central area, calculate the number of actual mining lines in the peripheral area (non-central area) (number of actual mining lines in the peripheral area = total number of actual mining lines - number of actual mining lines in the central area);

[0105] Based on the number of actual sampling lines in the central region and the number of actual sampling lines in the edge region, random sampling is performed using a Gaussian distribution function to generate a mask corresponding to the acceleration factor.

[0106] This application takes into account that the cause of artifacts is the presence of abnormal data in the phase encoding direction of k-space data. By using this random sampling method, some abnormal data in the phase encoding direction can be randomly filtered out. Furthermore, different speedup settings result in different degrees of abnormal data filtering in the resulting mask, thereby achieving different degrees of artifact removal.

[0107] Step S103: Multiply the multiple masks with the k-space data respectively, and perform inverse Fourier transform on the multiplication results to obtain multiple multi-channel images.

[0108] In the embodiments of this application, the multi-channel image, as well as the single-channel image and fused image mentioned below, can all be magnetic resonance images.

[0109] It should be noted that multiplying the mask with k-space data refers to multiplication at the pixel level.

[0110] The process of acquiring a multi-channel image is as follows:

[0111] mc_input i =ifft(fft(target)*mask i )

[0112] Among them, mc_input i This refers to the i-th multi-channel image, target refers to the multi-channel data obtained from the magnetic resonance scan, fft refers to Fourier transform, ifft refers to inverse Fourier transform, and mask refers to the mask. i It refers to the i-th mask.

[0113] Step S104: Based on the multiple multi-channel images, generate multiple corresponding single-channel images, with each multi-channel image corresponding to one single-channel image.

[0114] In some implementations, step S104 may include:

[0115] For each multi-channel image, channel compression is performed on the pixels of all channels at the same position in the multi-channel image to obtain the corresponding single-channel image.

[0116] In a specific application, channel compression can be performed by summing the squares of the pixels at the same position in the multi-channel image, then taking the square root to obtain the corresponding single-channel image.

[0117] The specific channel compression process is as follows:

[0118]

[0119] Among them, input i This represents the i-th single-channel image. This represents the pixel in the c-th channel of the i-th multi-channel image.

[0120] By performing channel compression on multi-channel images, they can be converted into corresponding single-channel images, which facilitates subsequent image fusion of single-channel images and improves the efficiency of image processing.

[0121] In some implementations, before inputting multiple single-channel images into the image fusion model, each single-channel image can be normalized to uniformly scale the pixel values ​​of the image to a preset range (e.g., 0 to 1), thus simplifying the image processing process.

[0122] Step S105: Input the multiple single-channel images into the image fusion model to obtain a fused image.

[0123] The aforementioned artifact removal method first performs a Fourier transform on the multi-channel data to obtain the corresponding k-space data. Then, it generates multiple masks based on different speedup factors. Masks obtained under different speedup conditions can filter out artifacts in the k-space data to varying degrees. Next, the multiple masks are multiplied with the k-space data respectively, and then an inverse Fourier transform is performed. The resulting multi-channel images are images with artifacts removed to varying degrees. These multi-channel images are then compressed to generate multiple single-channel images. Finally, the multiple single-channel images are input into an image fusion model for image fusion. The resulting fused image removes artifacts to varying degrees while preserving image details and structure well, resulting in high image quality and providing significant assistance to doctors in diagnostic analysis.

[0124] In some implementations, the plurality of single-channel images include the first single-channel image to the nth single-channel image, where n is an integer greater than 1;

[0125] The image fusion model includes convolutional modules from the first to the nth convolutional module;

[0126] The input of the first convolutional module includes the first single-channel image, and the output includes the corresponding first output image and the first feature map.

[0127] The input to the i-th convolutional module includes the i-th single-channel image and the (i-1)-th output image, and the output includes the corresponding i-th output image and the i-th feature map. <i≤n。

[0128] In this embodiment, the structure of each convolutional module can be the same, and the convolutional module can adopt common convolutional network structures, including but not limited to Unet, ResNet, etc.

[0129] See Figure 3 , Figure 3 This is a schematic diagram of the structure of an image fusion network provided in an embodiment of this application.

[0130] exist Figure 3 In the diagram, the first convolutional module is denoted by CNN1, the second convolutional module by CNN2, and the nth convolutional module by CNNn.

[0131] The input of the first convolutional module is input1 (the first single-channel image), and the output includes fea1 (the first feature map) and output1 (the first output image); the input of the second convolutional module includes input2 (the second single-channel image) and output1 (the first output image), and the output includes fea2 (the second feature map) and output2 (the second output image); the input of the nth convolutional module includes inputn (the nth single-channel image) and outputn-1 (the (n-1)th output image), and the output includes fean (the nth feature map) and outputn (the nth output image).

[0132] In this embodiment, a recurrent neural network can be used to construct an image fusion model. The recurrent neural network contains multiple sub-networks (convolutional modules). The input of each sub-network consists of two parts: one part is the output of the previous sub-network, and the other part is obtained by sampling the original data (k-space data) and the mask with the corresponding speedup factor. This data processing method of concatenating the inputs and outputs of different sub-networks enables the image fusion model to have strong anti-interference ability and fault tolerance.

[0133] See Figure 4 , Figure 4 This is a schematic diagram of a process for obtaining a fused image provided in an embodiment of this application.

[0134] In some implementations, the image fusion model further includes an attention module, and the process of acquiring the fused image may include steps S301 to S303.

[0135] Step S301: Perform channel compression on the first to the nth feature maps respectively to obtain n single-channel feature maps.

[0136] In some implementations, the channel compression method of the feature map can be:

[0137] The i-th feature map is input into the i-th convolutional layer with a kernel size of 1x1 for channel compression to obtain the i-th single-channel feature map.

[0138] like Figure 3 As shown, the first convolutional layer is denoted by Conv1, the second convolutional layer by Conv2, and the nth convolutional layer by Convn. fea1 (the first feature map) is input to Conv1 to obtain the first single-channel feature map; fea2 (the second feature map) is input to Conv2 to obtain the second single-channel feature map; and fean (the nth feature map) is input to Convn to obtain the nth single-channel feature map.

[0139] Step S302: Concatenate the n single-channel feature maps and input the concatenated feature map into the attention module to obtain a probability map.

[0140] The attention module can consist of two convolutional layers with a kernel size of 3x3. The activation layer corresponding to the first convolutional layer includes, but is not limited to, ReLU and its variations, while the activation layer corresponding to the second convolutional layer includes, but is not limited to, sigmoid, softmax, and other types.

[0141] In this embodiment, the output of the attention module is a probability map with n channels of the same size as the input image (stitched feature map). The probability map is used to indicate the probability distribution of pixels at each position in the stitched feature map across different channels.

[0142] Step S303: Perform channel stitching on the first output image to the nth output image, and fuse the stitched channel image with the probability map to obtain the fused image.

[0143] By fusing and learning the feature maps of different sub-networks, the weight of each pixel in the feature map of each sub-network is adaptively obtained using an attention mechanism. The weights are then combined to sum the output images of each sub-network to obtain the final fused image. This results in the fused image removing artifacts of varying degrees while preserving the detailed structure of the image.

[0144] See Figure 5 , Figure 5 This is a schematic diagram of the process of training an image fusion model provided in an embodiment of this application.

[0145] In some optional embodiments, the image fusion model is trained using a preset recurrent neural network, and the training process of the image fusion model may include steps S401 to S402.

[0146] Step S401: Acquire multiple single-channel images of samples, and input the multiple single-channel images of samples into the recurrent neural network respectively to obtain a sample fusion image.

[0147] In some implementations, the acquisition of multiple sample single-channel images can be performed using the methods described in steps S101 to S104 above. That is, in both the training and usage phases of the image fusion model, multi-channel data obtained from magnetic resonance scanning can be used, and data processing methods such as Fourier transform and inverse Fourier transform can be applied to it, with the resulting single-channel images used as input to the model.

[0148] The specific process for obtaining multiple single-channel images of samples is as follows:

[0149] 1) First, acquire multi-channel sample data, then generate n sample masks corresponding to different acceleration factors, and the acceleration factors gradually decrease. The specific acceleration factor can be set according to the actual situation.

[0150] 2) Perform Fourier transform on the multi-channel sample data to obtain the corresponding k-space sample data. Multiply the sample masks with different speed-up factors and the k-space sample data at the pixel level, and then perform inverse Fourier transform to obtain multiple multi-channel sample images.

[0151] 3) Sum the squares of the pixels at the same position in the multi-channel image of each sample, and then take the square root to obtain the corresponding single-channel image of the sample.

[0152] In some implementations, data augmentation can be performed on each single-channel image before inputting multiple sample images into the recurrent neural network. Data augmentation can help prevent model overfitting to some extent.

[0153] Data augmentation operations include, but are not limited to, horizontal flipping, vertical flipping, transposition, and data normalization.

[0154] Step S402: Train the recurrent neural network using the fused sample image and the standard fused image corresponding to the multiple single-channel sample images to obtain an image fusion model.

[0155] During the training process of the above model, a recurrent neural network is used to process multiple sample single-channel images to obtain corresponding sample fusion images. Then, the model is updated and iterated using the standard fusion image and the sample fusion image, thereby continuously improving the accuracy and performance of the image fusion model.

[0156] See Figure 6 , Figure 6 This is a schematic diagram of another training image fusion model provided in an embodiment of this application.

[0157] In some embodiments, step S402 may include steps S501 to S503.

[0158] Step S501: Calculate the preset loss corresponding to the recurrent neural network using the preset loss function, the sample fusion image, and the standard fusion image.

[0159] In some implementations, the preset loss function may include, but is not limited to, the L1 loss function, the L2 loss function, the SSIM (structural similarity) loss function, and the MS-SSIM (multi-scale structural similarity) loss function.

[0160] The L1 loss function, also known as the absolute value loss function, simply means minimizing the sum of the absolute differences between the target value and the estimated value.

[0161] Step S502: Calculate the adversarial loss corresponding to the recurrent neural network using the adversarial loss function, the sample fused image, and the standard fused image.

[0162] In some implementations, the discriminator required to calculate the adversarial loss can be selected from commonly used network architectures, including but not limited to VGG, ResNet, etc.

[0163] Step S503: Update the parameters of the recurrent neural network based on the preset loss and the adversarial loss.

[0164] In some implementations, the preset loss function is an L1 loss function, and the preset loss is an L1 loss. Step S503 may include:

[0165] Obtain the first weight corresponding to the L1 loss and the second weight corresponding to the adversarial loss;

[0166] The total loss of the recurrent neural network is calculated based on the L1 loss, the adversarial loss, the first weight, and the second weight.

[0167] The parameters of the recurrent neural network are updated based on the total loss.

[0168] The total loss of a recurrent neural network is calculated as follows:

[0169]

[0170] in, I represents the total loss. output I represents the sample fusion image. gt The standard fused image is represented by α, β, and L1. gan It signifies resistance to loss.

[0171] The embodiments of this application do not limit the values ​​of α and β. In a specific application, α = 1 and β = 100000.

[0172] In some implementations, the Adam optimizer is used to compute the gradients of the network parameters, and stochastic gradient descent is combined to update the network parameters. The initial learning rate of the Adam optimizer can be set to 0.0001 and can be adjusted appropriately according to the training process.

[0173] After several iterations, until the total loss of the recurrent neural network converges, the recurrent neural network at this point is used as the image fusion model.

[0174] During model training, two different loss functions (preset loss function and adversarial loss function) can be used to calculate the preset loss and adversarial loss of the recurrent neural network respectively. The parameters of the recurrent neural network can be updated by combining the preset loss and adversarial loss, thereby further improving the accuracy and performance of the image fusion model.

[0175] See Figure 7 , Figure 7 This is a structural block diagram of an artifact removal device provided in an embodiment of this application.

[0176] The specific implementation of the artifact removal device is consistent with the implementation method and the technical effect achieved as described in the embodiments of the above artifact removal method, and some details will not be repeated here.

[0177] The device includes:

[0178] Image transformation module 101 is used to acquire multi-channel data and perform Fourier transform on the multi-channel data to obtain corresponding k-space data;

[0179] The mask generation module 102 is used to generate multiple masks based on multiple preset acceleration factors, with each acceleration factor corresponding to one mask;

[0180] The inverse transform module 103 is used to multiply the multiple masks with the k-space data respectively, and perform an inverse Fourier transform on the result of the multiplication to obtain multiple multi-channel images.

[0181] The channel compression module 104 is used to generate multiple corresponding single-channel images based on the multiple multi-channel images, with each multi-channel image corresponding to a single-channel image;

[0182] The image fusion module 105 is used to input the multiple single-channel images into the image fusion model to obtain a fused image.

[0183] In some optional embodiments, the mask generation module 102 includes:

[0184] For each acceleration factor, based on the size of the k-space data and the acceleration factor, calculate the total number of actual sampling lines of the k-space data in the phase coding direction;

[0185] Based on the total number of actual sampling lines, determine the number of actual sampling lines in the central region of the k-space data;

[0186] Random sampling is performed based on a preset probability distribution function to generate a mask corresponding to the acceleration factor.

[0187] In some optional embodiments, the channel compression module 104 is used for:

[0188] For each multi-channel image, channel compression is performed on the pixels of all channels at the same position in the multi-channel image to obtain the corresponding single-channel image.

[0189] In some optional embodiments, the plurality of single-channel images include the first single-channel image to the nth single-channel image, where n is an integer greater than 1;

[0190] The image fusion model includes convolutional modules from the first to the nth convolutional module;

[0191] The input of the first convolutional module includes the first single-channel image, and the output includes the corresponding first output image and the first feature map.

[0192] The input to the i-th convolutional module includes the i-th single-channel image and the (i-1)-th output image, and the output includes the corresponding i-th output image and the i-th feature map. <i≤n。

[0193] In some optional embodiments, the image fusion model further includes an attention module, and the process of obtaining the fused image includes:

[0194] Channel compression is performed on the first to nth feature maps respectively to obtain n single-channel feature maps;

[0195] The n single-channel feature maps are concatenated, and the concatenated feature map is input into the attention module to obtain a probability map;

[0196] The first to the nth output images are stitched together by channels, and the stitched channel image is fused with the probability map to obtain the fused image.

[0197] In some optional embodiments, the image fusion model is trained using a pre-defined recurrent neural network, and the training process of the image fusion model is implemented using the following modules:

[0198] The sample fusion module is used to acquire multiple single-channel images of samples and input the multiple single-channel images of samples into the recurrent neural network to obtain a sample fused image.

[0199] The fusion training module is used to train the recurrent neural network using the fused sample image and the standard fused image corresponding to the multiple single-channel sample images to obtain an image fusion model.

[0200] In some optional embodiments, the fusion training module includes:

[0201] The first loss unit is used to calculate the preset loss corresponding to the recurrent neural network using a preset loss function, the sample fusion image, and the standard fusion image;

[0202] The second loss unit is used to calculate the adversarial loss corresponding to the recurrent neural network using the adversarial loss function, the sample fused image, and the standard fused image.

[0203] The parameter update unit is used to update the parameters of the recurrent neural network based on the preset loss and the adversarial loss.

[0204] See Figure 8 This application also provides an electronic device, which includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0205] The memory 210 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 211 and / or cache memory 212, and may further include read-only memory (ROM) 213.

[0206] The memory 210 also stores a computer program, which can be executed by the processor 220, causing the processor 220 to perform the steps of the method in the embodiments of this application. The specific implementation method is consistent with the implementation method and the technical effect achieved in the above method embodiments, and some contents will not be repeated.

[0207] The memory 210 may also include a utility 214 having at least one program module 215, such program module 215 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0208] Accordingly, processor 220 can execute the aforementioned computer program, and can also execute utility 214.

[0209] Bus 230 can represent one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any of the various bus structures.

[0210] The electronic device can also communicate with one or more external devices 240, such as a keyboard, pointing device, Bluetooth device, etc., and with one or more devices capable of interacting with the electronic device, and / or with any device that enables the electronic device to communicate with one or more other computing devices (e.g., a router, modem, etc.). This communication can be performed via input / output interface 250. Furthermore, the electronic device can communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 260. Network adapter 260 can communicate with other modules of the electronic device via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0211] This application also provides a computer-readable storage medium for storing a computer program. When the computer program is executed, it implements the steps of the artifact removal method in this application. The specific implementation method is consistent with the implementation method and the technical effect achieved in the above-described artifact removal method embodiments, and some contents will not be repeated.

[0212] Figure 9 This embodiment illustrates a program product for implementing the artifact removal method described above. It can employ a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of this invention is not limited thereto. In this application, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The program product can employ any combination of one or more readable media. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0213] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, or any suitable combination thereof. Program code for performing operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to user computing devices via any type of network, including local area networks (LANs) or wide area networks (WANs), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0214] This application describes the invention from the perspectives of purpose, performance, progress, and novelty, and it meets the functional enhancement and use requirements emphasized by the Patent Law. The above description and drawings are merely preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc., that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

Claims

1. A method for artifact removal, characterized in that, The method includes: Acquire multi-channel data and perform Fourier transform on the multi-channel data to obtain the corresponding k-space data; Multiple masks are generated based on multiple preset acceleration factors, with each acceleration factor corresponding to one mask; The multiple masks are multiplied with the k-space data respectively, and the results of the multiplication are subjected to inverse Fourier transform to obtain multiple multi-channel images. Based on the multiple multi-channel images, multiple corresponding single-channel images are generated, with each multi-channel image corresponding to one single-channel image. The multiple single-channel images are input into the image fusion model to obtain a fused image; The step of generating multiple corresponding single-channel images based on the multiple multi-channel images includes: For each multi-channel image, channel compression is performed on the pixels of all channels at the same position in the multi-channel image to obtain the corresponding single-channel image.

2. The artifact removal method according to claim 1, characterized in that, The generation of multiple masks based on preset multiple acceleration factors includes: For each acceleration factor, based on the size of the k-space data and the acceleration factor, calculate the total number of actual sampling lines of the k-space data in the phase coding direction; Based on the total number of actual sampling lines, determine the number of actual sampling lines in the central region of the k-space data; Random sampling is performed based on a preset probability distribution function to generate a mask corresponding to the acceleration factor.

3. The artifact removal method according to claim 1, characterized in that, The plurality of single-channel images include the first single-channel image to the nth single-channel image, where n is an integer greater than 1; The image fusion model includes convolutional modules from the first to the nth convolutional module; The input of the first convolutional module includes the first single-channel image, and the output includes the corresponding first output image and the first feature map. The input to the i-th convolutional module includes the i-th single-channel image and the (i-1)-th output image, and the output includes the corresponding i-th output image and the i-th feature map. <i≤n。 4. The artifact removal method according to claim 3, characterized in that, The image fusion model also includes an attention module, and the process of obtaining the fused image includes: Channel compression is performed on the first to nth feature maps respectively to obtain n single-channel feature maps; The n single-channel feature maps are concatenated, and the concatenated feature map is input into the attention module to obtain a probability map; The first to the nth output images are stitched together by channels, and the stitched channel image is fused with the probability map to obtain the fused image.

5. The artifact removal method according to claim 3, characterized in that, The image fusion model is trained using a pre-defined recurrent neural network. The training process of the image fusion model includes: Multiple single-channel images of samples are acquired, and the multiple single-channel images of samples are respectively input into the recurrent neural network to obtain a sample fusion image; The recurrent neural network is trained using the fused sample image and the standard fused image corresponding to the multiple single-channel sample images to obtain an image fusion model.

6. The artifact removal method according to claim 5, characterized in that, The step of training the recurrent neural network using the fused sample image and the standard fused image corresponding to the multiple single-channel sample images includes: Using a preset loss function, the sample fused image, and the standard fused image, calculate the preset loss corresponding to the recurrent neural network; The adversarial loss corresponding to the recurrent neural network is calculated using the adversarial loss function, the sample fused image, and the standard fused image; The parameters of the recurrent neural network are updated based on the preset loss and the adversarial loss.

7. An artifact removal device, characterized in that, The device includes: The image transformation module is used to acquire multi-channel data and perform Fourier transform on the multi-channel data to obtain the corresponding k-space data. The mask generation module is used to generate multiple masks based on multiple preset acceleration factors, with each acceleration factor corresponding to one mask; The inverse transform module is used to multiply the multiple masks with the k-space data respectively, and perform an inverse Fourier transform on the multiplication result to obtain multiple multi-channel images; The channel compression module is used to generate multiple corresponding single-channel images based on the multiple multi-channel images, with each multi-channel image corresponding to one single-channel image; The image fusion module is used to input the multiple single-channel images into the image fusion model to obtain a fused image; The step of generating multiple corresponding single-channel images based on the multiple multi-channel images includes: For each multi-channel image, channel compression is performed on the pixels of all channels at the same position in the multi-channel image to obtain the corresponding single-channel image.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images