Method and device for eliminating artifacts in medical images

Abstract

The invention discloses a method for eliminating artifacts in a medical image, and the method comprises the steps: carrying out the first registration processing of to-be-registered images and artifact segmentation results corresponding to the to-be-registered images based on a reference image, and obtaining a first registration image and a first registration artifact segmentation result; the reference image and the first registration image are input into a registration model, a prediction transformation field is obtained, and the registration model is a diffusion model; respectively transforming the first registration image and the first registration artifact segmentation result based on the prediction transformation field to obtain a second registration image and a second registration artifact segmentation result; and performing artifact elimination on an artifact part in the reference image based on the reference image, the second registration image and the second registration artifact segmentation result. According to the scheme provided by the embodiment of the invention, the advantages of the diffusion model in image processing are fully utilized, and the artifact part in the medical image can be accurately fused, so that the medical image without artifacts is generated, and the quality of the medical image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical imaging image quality management, in particular to a method for eliminating artifacts in medical imaging images and a device for eliminating artifacts in medical imaging images. Background Art

[0002] Artifacts in medical images refer to abnormal images generated during the imaging process that are unrelated to the scanned tissue structure. Common artifacts in medical images include motion artifacts, metal artifacts, truncation artifacts, chemical shift artifacts, convolution artifacts, partial volume artifacts, ray hardening artifacts, insufficient scan range artifacts, reconstruction algorithm artifacts, and electromagnetic interference artifacts, etc. These abnormal images reduce the quality of medical imaging images and even affect the analysis and diagnosis of lesions. Therefore, reducing or even eliminating the artifact areas in medical imaging images is crucial for ensuring high medical quality and accuracy.

[0003] Currently, a common practice for reducing or eliminating artifacts in medical imaging images is to repeat the scan multiple times to achieve the goal of obtaining high-quality medical imaging images as much as possible. However, since the artifacts in medical images are mainly divided into two categories according to their sources: hardware system failures or errors and human factors, it is very difficult to eliminate motion artifacts caused by human factors through multiple repeated scans. For example, in some scenarios, patients are not particularly stable due to injuries or critical conditions, so it is often difficult for them to control themselves during the scan. In this case, even if multiple scans are performed, relatively serious motion artifacts will still appear in each scan. Thus, it is still very difficult to obtain an image completely without motion artifacts through multiple scans. Moreover, too many repeated scans will bring new problems. For example, multiple repeated scans bring ionizing injuries with excessive X-ray doses to patients; another example is that the images of multiple repeated scans pushed to the PACS result in waste of storage; also, multiple repeated scans waste inspection time and increase the precious waiting time of other emergency patients; furthermore, the diagnostic doctor needs to make a comprehensive judgment from the images of multiple repeated scans, increasing the diagnostic time of the diagnostic doctor.

[0004] There is also a prior art solution that weakens the influence of artifacts in medical imaging images through simple registration and image fusion after registration. For example, the invention patent with the publication number CN117092570A, but this technical solution requires strict control of imaging parameters during magnetic resonance imaging and is only applicable to the scenario of cardiac imaging, with limited application scenarios. In addition, it only eliminates artifacts through simple rough registration, with low accuracy and easy to make mistakes. Summary of the Invention

[0005] Based on the above background, one of the objectives of the present invention is to provide a solution for eliminating artifacts in medical imaging images. It makes full use of the advantages of diffusion models in image processing, can accurately fuse the artifact parts in medical imaging images, thereby generating medical imaging images without artifacts and improving the quality of medical imaging images.

[0006] According to the first aspect of the present invention, there is provided a method for eliminating artifacts in medical imaging images, which includes:

[0007] Determining a reference image and a to-be-registered image in a medical imaging image sequence according to the artifact segmentation result of the medical imaging image;

[0008] Performing first registration processing on the to-be-registered image and the artifact segmentation results corresponding to each to-be-registered image respectively based on the reference image to obtain a first registered image and the first registered artifact segmentation results corresponding to each first registered image;

[0009] Inputting the reference image and the first registered image into a pre-trained registration model to obtain a predicted transformation field output by the registration model, wherein the pre-trained registration model is a diffusion model that takes the reference image and the first registered image as inputs and the predicted transformation field as the output;

[0010] Performing transformation on the first registered image and the first registered artifact segmentation results corresponding to each first registered image respectively based on the predicted transformation field to obtain a second registered image and the second registered artifact segmentation results corresponding to each second registered image;

[0011] Eliminating the artifacts in the reference image based on the reference image, the second registered image and the second registered artifact segmentation results corresponding to each second registered image to generate a medical imaging image with artifacts eliminated.

[0012] According to the second aspect of the present invention, there is provided a device for eliminating artifacts in medical imaging images, which includes:

[0013] A preprocessing module for determining a reference image and a to-be-registered image in a medical imaging image sequence according to the artifact segmentation result of the medical imaging image;

[0014] A rough registration module for performing first registration processing on the to-be-registered image and the artifact segmentation results corresponding to each to-be-registered image respectively based on the reference image to obtain a first registered image and the first registered artifact segmentation results corresponding to each first registered image;

[0015] A parameter prediction module, configured to input a reference image and a first registered image into a pre-trained registration model, and obtain a predicted transformation field output by the registration model, wherein the pre-trained registration model is a diffusion model that takes the reference image and the first registered image as inputs and the predicted transformation field as the output;

[0016] A fine registration module, configured to respectively perform transformation on the first registered image and the first registered artifact segmentation result corresponding to each first registered image based on the predicted transformation field to obtain a second registered image and the second registered artifact segmentation result corresponding to each second registered image;

[0017] An artifact replacement module, configured to perform artifact elimination on the artifact part in the reference image based on the reference image, the second registered image, and the second registered artifact segmentation result corresponding to each second registered image, and generate a medical image with artifacts eliminated.

[0018] According to a third aspect of the present invention, there is provided an electronic device, which includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of the method according to the first aspect of the present invention.

[0019] According to a fourth aspect of the present invention, there is provided a storage medium, on which a computer program is stored, characterized in that when the program is executed by a processor, it implements the steps of the method according to any one of the first aspects of the present invention.

[0020] The solution provided by the present invention is that the method of the embodiment of the present invention enables the reference image, the image to be registered, and their artifact segmentation results to be accurately aligned through one initial registration and one fine registration based on the predicted transformation field generated by the diffusion model. Then, the present invention replaces the artifact areas based on the accurately aligned reference image and the artifact segmentation results of the image to be registered, thereby realizing the accurate matching and fusion of artifact-free areas of two or more medical images, enabling the generation of complete artifact-free medical images using two or more scanned medical images with artifact defects, improving the quality and reliability of medical images, and helping to reduce the frequency of repeated scans and improve medical efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the processing process of a method for eliminating artifacts in a medical image according to an embodiment of the present invention;

[0022] Figure 2 Schematically shows an embodiment of Figure 1The flowchart of the specific implementation method of step S11 in

[0023] Figure 3 Schematically shows the flowchart of the method for generating a prediction transformation field according to an embodiment;

[0024] Figure 4 Schematically shows the network architecture diagram of the registration model according to an embodiment of the present invention;

[0025] Figure 5 Schematically shows the flowchart of the training method of the registration model according to an embodiment of the present invention;

[0026] Figure 6 Schematically shows the flowchart of the specific implementation method of step S15 according to an embodiment;

[0027] Figure 7 Schematically shows the principle block diagram of the device for eliminating artifacts in medical imaging images according to an embodiment of the present invention;

[0028] Figure 8 Schematically shows the schematic diagram of an electronic device according to an embodiment of the present invention. Specific embodiments

[0029] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0030] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.

[0031] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0032] In the present invention, "module", "device", "system", etc. refer to relevant entities applied to a computer, such as hardware, a combination of hardware and software, software, or software in execution, etc. Specifically, for example, an element can be, but is not limited to, a process running on a processor, a processor, an object, an executable element, an execution thread, a program, and / or a computer. Also, an application program or a script program running on a server, and the server can both be elements. One or more elements can be in an execution process and / or thread, and the elements can be localized on one computer and / or distributed between two or more computers, and can be run by various computer-readable media. The elements can also communicate through local and / or remote processes according to a signal having one or more data packets, for example, a signal from data that interacts with another element in a local system, a distributed system, and / or interacts with other systems through a signal on a network of the Internet.

[0033] Finally, it should also be noted that in this text, relative terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise" and "include", not only include those elements, but also include other elements not explicitly listed, or also include elements inherent in such a process, method, article, or device. Without more limitations, the elements defined by the statement "comprising..." do not exclude the existence of additional identical elements in the process, method, article, or device including the said elements.

[0034] The method for eliminating artifacts in medical imaging images according to the embodiments of the present invention can be used to eliminate artifacts in images of any type of medical imaging device in any hospital, medical consortium, or regional medical institution, for example, images of medical imaging devices such as MRI (magnetic resonance) and CT (X-ray computed tomography). The method according to the embodiments of the present invention can be implemented by an independent terminal device, or can also be implemented by combining the terminal device with a cloud server, and the present invention does not limit this. Through the solution according to the embodiments of the present invention, accurate registration transformation is performed on the image sequence through a diffusion model to accurately recognize the registration relationship between the reference image and the image to be registered, so that the artifact area in the reference image can be artifact-replaced and eliminated using the image to be registered, effectively reducing the adverse effects brought by artifacts to medical efficiency (such as misdiagnosis caused by artifacts, the need for patients to repeatedly make appointments for imaging examinations afterwards, etc.), thereby helping the hospital improve its medical quality, medical efficiency, and equipment utilization efficiency, etc.

[0035] The present invention will be further described in detail below with reference to the accompanying drawings.

[0036] Figure 1Schematically shows a method process for eliminating artifacts in medical imaging images according to an embodiment of the present invention. The execution subject of this method can be a processor such as a PC, computer, tablet, smartphone, server, etc. As Figure 1 shown, the method includes:

[0037] Step S11, determining a reference image and a to-be-registered image in a medical imaging image sequence according to the artifact segmentation result of the medical imaging image;

[0038] Step S12, respectively performing a first registration process on the to-be-registered image and the artifact segmentation result corresponding to each to-be-registered image based on the reference image to obtain a first registered image and a first registered artifact segmentation result corresponding to each first registered image;

[0039] Step S13, inputting the reference image and the first registered image into a pre-trained registration model to obtain a predicted transformation field output by the registration model. Among them, the pre-trained registration model is a diffusion model that takes the reference image and the first registered image as inputs and the predicted transformation field as the output;

[0040] Step S14, respectively performing transformation on the first registered image and the first registered artifact segmentation result corresponding to each first registered image based on the predicted transformation field to obtain a second registered image and a second registered artifact segmentation result corresponding to each second registered image;

[0041] Step S15, eliminating the artifacts in the reference image based on the reference image, the second registered image, and the second registered artifact segmentation result corresponding to each second registered image to generate a medical imaging image with artifacts eliminated.

[0042] In step S11, the artifact segmentation result of the medical image is obtained by inputting the medical image sequence into a pre-trained artifact segmentation model. Among them, the pre-trained artifact segmentation model is a semantic segmentation model that takes the medical image sequence as the input and the artifact segmentation sequence as the output. The artifact segmentation model aims to perform precise artifact segmentation processing on the input image sequence to ensure the accurate identification and segmentation of each pixel point in the artifact area. Among them, in order to train the aforementioned desired artifact segmentation model, the artifact segmentation model preferably adopts advanced 3D segmentation models, including but not limited to UNet3D, UNETR, and Swin-UNETR, etc. Taking the Swin-UNETR semantic segmentation model as an example of the artifact segmentation model, when training the model, the loss function can adopt Dice, the optimization algorithm can select AdamW, and the training dataset can be generated based on the per-pixel artifact area annotation (i.e., semantic segmentation annotation) of each sample image. Then, using the formed training dataset, loss function, and optimization algorithm to train the semantic segmentation model, a semantic segmentation model that takes the medical image sequence as the input and the artifact segmentation sequence as the output can be generated. Among them, the artifact segmentation sequence includes the artifact segmentation results corresponding to each medical image in the medical image sequence. Exemplarily, when the medical image sequence [image_1, image_2,..., image_n] is ready, in step S11, these images will be sequentially input into the trained Swin-UNETR model for artifact segmentation prediction. Through the processing of the artifact segmentation model, the corresponding artifact segmentation sequence [mask_1, mask_2,..., mask_n] can be obtained as the output result of the artifact segmentation model, and each mask in the artifact segmentation sequence [mask_1, mask_2,..., mask_n] is the artifact segmentation result corresponding to the corresponding image in the image sequence. Preferably, the artifact segmentation result corresponding to the medical image is the coordinate set of the artifact pixels in the medical image. It should be noted that in the method of the embodiment of the present invention, when it is detected by the artifact segmentation model that there is no artifact phenomenon in a certain image in the image sequence, the image will be directly returned as the final artifact-free medical image output, so there is no need to further execute the subsequent image fusion process from step S12 to step S15. On the contrary, if it is detected by the artifact segmentation model that all the images in the image sequence have artifacts, the subsequent processing steps from step S12 to step S15 will be performed to fuse the artifact-containing image sequence to generate an artifact-free medical image, so as to realize the efficient utilization of the artifact-containing image sequence, improve the quality of the medical image, and avoid wasting medical resources, etc.In addition, it should be noted that in the embodiments of the present invention, in practical applications, the medical image sequence is the CT image of a real patient, rather than a virtual image generated by a model. That is, the method in the embodiments of the present invention can realize the fusion and repair of medical images with artifact defects obtained by real scanning, so as to eliminate artifacts and generate qualified high-quality medical images, reduce the repeated scanning of patients, and improve medical efficiency.

[0043] As a possible implementation manner, in step S11, the reference image and the image to be registered can be determined based on the artifact volume in the medical image, where Figure 2 FIG. schematically shows the method flow for determining the reference image and the image to be registered in a medical image sequence according to an embodiment of the present invention, as Figure 2 shown, which can be implemented to include:

[0044] Step S111, determining the artifact volume in each medical image according to the artifact segmentation result of each medical image in the medical image sequence;

[0045] Step S112, selecting the medical image with the smallest artifact volume in the medical image sequence as the reference image, and using the other medical images in the medical image sequence as the images to be registered.

[0046] In step S111, when determining the artifact volume in each medical image, based on the artifact segmentation result, the pixel value of the pixel points belonging to the artifact region in the medical image can be set to 1, and the other regions are set to 0. Then, according to the set pixel values, the number of voxels occupied by the artifact region is counted. Finally, based on the volume of a single voxel and the number of voxels occupied by the calculated artifact, the artifact volume in the corresponding medical image is determined. Exemplarily, taking the number of voxels occupied by the artifact region in a certain medical image as n as an example, the artifact volume can be determined based on the spacing attribute information in the medical image, where the spacing attribute information in the medical image is used to describe the actual physical size represented by each voxel in the medical image, that is, there is an actual volume v = sx * sy * sz corresponding to a voxel, where sx, sy, and sz are the spacing attribute information of a voxel in the three directions of xyz. Therefore, multiplying the counted number of voxels by the actual volume of a single voxel can obtain the total volume of the artifact in a certain medical image. After determining the artifact volume in each medical image, select the image with the smallest artifact volume as the reference image, and the remaining images as the images to be registered.

[0047] Exemplarily, the implementation process may be as follows: first, set the pseudo-artifact pixel values of each medical image in the image sequence according to each pseudo-artifact segmentation result in the set of pseudo-artifact segmentation sequences [mask_1, mask_2,..., mask_n], set the pixel values of the artifact area to 1, and set the other areas to 0, and calculate the artifact volume in each medical image based on the number of voxels occupied by the artifact area. Then, based on the calculation results, determine the image with the smallest artifact volume as the reference image, and use the pseudo-artifact segmentation result corresponding to the reference image as the reference pseudo-artifact segmentation result. The remaining images will be regarded as the image sequence to be registered [moving_image_1, moving_image_2,..., moving_image_n-1], and the corresponding pseudo-artifact segmentation results will form the pseudo-artifact segmentation results to be registered [moving_mask_1, moving_mask_2,..., moving_mask_n-1], where the pseudo-artifact segmentation results to be registered are the pseudo-artifact segmentation results corresponding to each image to be registered.

[0048] In step S12, preferably, a rigid registration algorithm is applied to perform the first registration process on the image to be registered and the pseudo-artifact segmentation result to be registered. This algorithm aims to map each image to be registered and its corresponding pseudo-artifact segmentation result to the spatial coordinate system where the reference image is located. Specifically, by traversing the image sequence to be registered [moving_image_1, moving_image_2,..., moving_image_n-1] and its corresponding pseudo-artifact segmentation results to be registered [moving_mask_1, moving_mask_2,..., moving_mask_n-1], register them to the image space coordinate system of the reference image. Exemplarily, the first registration process is implemented using a deep learning-based rigid registration algorithm, and this algorithm preferably uses mutual information as the objective function and LBFGS2 as the optimization algorithm, and the algorithm framework adopts a multi-resolution framework. The method of the embodiment of the present invention ensures that the subsequent image fusion task can rely on an initial registration result as a cornerstone by performing a round of rigid registration process on the image to be registered and its corresponding pseudo-artifact segmentation result first, reducing the difficulty of the subsequent process.

[0049] After performing rigid registration on the to-be-registered image and the segmentation result of the to-be-registered artifact, in the embodiments of the present invention, fine registration will be further performed on the to-be-registered image and the segmentation result of the to-be-registered artifact. Preferably, the fine registration in the embodiments of the present invention refers to the pixel-to-pixel matching of the reference image and the to-be-registered image based on the diffusion model. As an advanced generative model, the diffusion model has become a key advancement in the field of machine learning in recent years. The powerful capabilities of the diffusion model, especially in the field of image synthesis, have surpassed traditional generative adversarial networks, attracting extensive attention in various application fields. However, no one has thought of applying it to the field of medical image fusion. In this application, the inventor takes the lead in the industry and first thinks of using a combination of deep learning and machine learning to eliminate medical image artifacts. Moreover, when performing image fusion, the technical solution of the embodiments of the present invention uses the CT images of real patients, rather than virtual images generated by the model, for image fusion. Therefore, it is more real, and the available value and accuracy of the generated images are higher. Based on this technical concept, in step S13, the embodiments of the present invention input the reference image and the first registered image obtained by the rough registration in step S12 into a pre-trained registration model to obtain the predicted transformation field, where the registration model used is the diffusion model. Specifically, Figure 3 Schematically shows the method process of generating the predicted transformation field based on the reference image and the first registered image using the diffusion model, as Figure 3 shown, the registration model of the embodiments of the present invention includes a shared encoder, a registration decoder, and a diffusion decoder. The features at all levels of the encoder, the registration decoder, and the diffusion decoder will be fused. The input of the model is two identical reference images and a set of first registered images corresponding to the reference image. The output of the model is the predicted transformation field. Among them, the input set of first registered images and the two reference images are stacked along the channel and then continuously input into the subsequent encoder, registration encoder, and diffusion encoder, and the predicted transformation field is finally obtained through the fusion of features at all levels. After obtaining the predicted transformation field, in step S14, the first registered image is transformed using the predicted transformation field to obtain the predicted second registered image. Among them, in step S14, the same transformation field is also used to transform the segmentation result of the first registered artifact corresponding to the first registered image to obtain the predicted segmentation result of the second registered artifact.

[0050] As a preferred embodiment, the registration model adopts a denoising diffusion probability model, which includes two Unet networks. One Unet network is composed of an encoder and a registration encoder, and the other Unet network is composed of an encoder and a diffusion encoder. The two Unet networks are designed to share the parameters of the encoder. The Unet network is used to generate a predicted transformation field based on the input reference image and the first registered image. For the network architecture of the registration model with this structure, refer to Figure 4 as shown.

[0051] The following will combine Figure 5 to illustrate the training process of the registration model in the embodiments of the present invention. As Figure 5 shown, the process of the training method of this model includes:

[0052] Step S51: Obtain a sample reference image and a sample image to be registered, and add noise to the sample reference image based on the principle of the denoising diffusion probability model to form at least one set of training data. Each set of training data includes a sample reference image, a sample reference image with added noise, and a set of sample images to be registered;

[0053] Step S52: Input each set of training data into a pre-constructed diffusion model respectively, and optimize the model parameters of the diffusion model based on a loss function and an optimizer, and generate the registration model according to the optimization result. The loss function includes a diffusion loss function and a registration loss function.

[0054] Among them, the sample reference image and the sample image to be registered in step S51 can be obtained by performing the above step S11 on a sample image sequence. The sample image sequence can be a historical scan image sequence or an image sequence obtained by sampling the historical scan image sequence, etc. The embodiments of the present invention do not limit this. The specific process of adding noise to the sample reference image may include: First, randomly generate an integer t, such as randomly generating an integer t from 1 to 2000, and then calculate α t , then generate a random noise ∈ from a normal distribution, and finally add noise to the sample reference image (x 0 ) through formula (2) to obtain the sample reference image with added noise (x t ).

[0055]

[0056] Among them, t is the generated random integer, ∈ is the random noise, x 0 represents the sample reference image before adding noise, and x t represents the sample reference image after adding noise.

[0057] After obtaining a set of training data, the sample reference image, the sample reference image with added noise, and the sample image to be registered in the same set of training data are stacked along the channel, and input into the network structure composed of an encoder, a registration decoder, and a diffusion decoder as shown in Figure 4 . The network structure is two UNet networks, where the encoder part shares parameters, and the decoder part fuses the features of the encoder and the two decoders (i.e., the registration decoder and the diffusion decoder). The specific feature fusion method is as shown in Figure 6 . Let the encoder feature be f1_encoder, the registration decoder feature be f1_decoder_r, and the diffusion decoder feature be f1_decoder_g. After stacking the three along the channel, they are input into the convolutional layer conv to obtain the registration decoder feature f2_decoder_r of the next layer, thereby realizing the fusion of the three features. The final output of the registration decoder can be used as the predicted transformation field for the second registration process. Among them, during the process of inputting the training data into the network structure shown in Figure 4 for training, the training method of the embodiment of the present invention uses two loss functions and the AdamW optimizer to optimize and adjust the model parameters to train a registration model that meets the expectations. Among them, the two loss functions are the diffusion loss function and the registration loss function. Let the output of the diffusion decoder be G, the image obtained after the second registration process using the predicted transformation field is the second registered image registered image, and the random noise generated from the normal distribution when adding noise to the reference image is ∈. Then the diffusion loss function L diffusion and the registration loss function L registration can be respectively represented by the following formulas (3) and (4):

[0058]

[0059] Among them, NCC represents the normalized cross-correlation loss, γ is a hyperparameter used to control the weight, and its value can be 1. The total loss can be expressed as:

[0060] L = L diffusion + λL registration

[0061] Preferably, during the specific training process, λ can be set to 20, the learning rate of the AdamW optimizer is set to 3e-4, and the batch size is set to 1. Thus, the model parameters of the diffusion model can be trained and optimized based on the expected target of the total loss, thereby training the desired registration model.

[0062] It should be noted that during the training process, the model network structure of the embodiments of the present invention adopts the superposition of three channels. The training inputs of the three channels are respectively the sample reference image, the sample reference image with added noise, and the sample registration image. In actual applications, the input of the channel of the sample reference image with added noise is replaced by the reference image, that is, two identical reference images and a set of first registration images corresponding to the reference image are input to the pre-trained registration model at the same time. Thus, the embodiments of the present invention apply the diffusion model as a deformable registration algorithm to finely adjust the local area of the image, making full use of the advantages of machine learning models, especially the diffusion model, in the field of image processing. As a result, pixel-to-pixel matching between the reference image and the image to be registered can be achieved, enabling the embodiments of the present invention to perform accurate artifact replacement processing based on pixel points, improving the accuracy of artifact elimination and the accuracy of the generated image quality.

[0063] After obtaining the reference image and its corresponding artifact segmentation result (i.e., the reference artifact segmentation result) and the second registration image registered to the reference image and its corresponding second registration artifact segmentation result through the processing of steps S11 to S14, as a possible implementation manner, the technical concept of the embodiments of the present invention is to replace the pixels containing artifacts in the reference image with the artifact-free pixels at the corresponding positions in the second registration image, so as to achieve image fusion and artifact removal, and generate a medical image without artifacts. Based on this, Figure 6 Schematically shows a specific implementation method of step S15 of an implementation manner, as Figure 6 shown, which is implemented to include:

[0064] Step S151, identifying and separating all artifact instances in the reference image according to the artifact segmentation result corresponding to the reference image, where each artifact instance corresponds to an independent connected set of artifact pixels;

[0065] Step S152, screening out the replaceable instances of each artifact instance in the second registration image based on the artifact instances and the second registration artifact segmentation results corresponding to each second registration image. Each replaceable instance also corresponds to a set of pixels. The position of the set of pixels of the replaceable instance in the corresponding second registration image is the same as the position of its corresponding artifact instance in the reference image, and the set of pixels in the replaceable instance does not contain artifact pixels;

[0066] Step S153, replacing the pixel values of the corresponding regions in the reference image based on the replaceable instances corresponding to each artifact instance to generate a medical image with artifacts removed.

[0067] In step S151, an artifact instance is specifically defined by an independent set of connected pixel points, and the independent set of connected pixel points defines an independent and connected artifact region. As a possible implementation, all artifact instances in the artifact segmentation result corresponding to the reference image can be identified and separated by performing connected component analysis on the artifact segmentation result. The connected component analysis can be implemented using existing connected component analysis algorithms, such as the Two-Pass algorithm, the Seed-Filling algorithm, etc. The specific execution process thereof is not elaborated in the embodiments of the present invention.

[0068] In step S152, the process of screening replaceable instances can specifically be, for each artifact instance, traversing the set of artifact pixel point coordinates of each second registered artifact segmentation result. If the artifact instance does not overlap with the set of artifact pixel point coordinates in a certain second registered artifact segmentation result, it means that the region corresponding to the artifact instance in the second registered image is free of artifacts. Therefore, the set of pixel points in the region corresponding to the artifact instance in the second registered image can be used as the replaceable instance of the artifact instance. Thus, by separately performing this process on all artifact instances, the replaceable instances corresponding to all artifact instances of the reference image can be screened out. It should be noted that the set of pixel points corresponding to the replaceable instance is the set of pixel points in the second registered image, and the region positions covered by these pixel points are the same as those of the artifact instance. It should be noted that since the second registered image is registered pixel by pixel based on the reference image, and both are registered to the spatial coordinate system of the reference image, the position of the set of pixel points of the replaceable instance in the corresponding second registered image being the same as the position of its corresponding artifact instance in the reference image can mean that the coordinate regions covered by the two are the same. It should be noted that when there are multiple replaceable instances screened out for a certain artifact instance, such as when there are no artifact pixel points at the positions corresponding to the artifact instance in multiple second registered images, any one of the replaceable instances can be selected as the replaceable instance for subsequent artifact replacement. In a preferred implementation, the first traversed replaceable instance can be selected for subsequent artifact replacement.

[0069] In step S153, by using the pixel values at the same positions in the second registered image, that is, using the pixel values corresponding to the pixel points in the replaceable instances corresponding to each artifact instance determined in step S152 to replace the pixel values of the corresponding artifact instances in the reference image, a medical image with artifacts removed can be obtained. Through this replacement process, not only can the artifact instances in the reference image be removed, but also as much original image detail and structural information as possible can be retained, and the quality of the formed medical image is higher.

[0070] The method of the embodiment of the present invention performs coordinate system and pixel-to-pixel registration on a medical image sequence with artifacts through an algorithm that combines deep learning and machine learning, so as to realize the fusion of the image sequences obtained by multiple scans into an artifact-free image by replacing the artifact pixels in the reference image, solving the problem that motion artifacts are difficult to eliminate through multiple scans. Moreover, the embodiment of the present invention uses a diffusion model for image fusion, and the images are fused based on real medical image scans obtained, so the resulting images are more realistic, that is, they are not virtual results generated by algorithms or models. Therefore, the generated images can retain as much original image detail and structural information as possible, and the image quality is higher and more reliable.

[0071] Figure 7 Schematically shows a device 7 for eliminating artifacts in medical images, as Figure 7 shown, which includes:

[0072] A preprocessing module 71, configured to determine a reference image and a to-be-registered image in a medical image sequence according to the artifact segmentation result of the medical image;

[0073] A rough registration module 72, configured to perform first registration processing on the to-be-registered image and the artifact segmentation results corresponding to each to-be-registered image respectively based on the reference image, to obtain a reference image, a first registered image, and the first registered artifact segmentation results corresponding to each first registered image;

[0074] A parameter prediction module 73, configured to input the reference image and the first registered image into a pre-trained registration model to obtain a predicted transformation field output by the registration model, wherein the pre-trained registration model is a diffusion model that takes the reference image and the first registered image as inputs and the predicted transformation field as the output;

[0075] A fine registration module 74, configured to perform transformation on the first registered image and the first registered artifact segmentation results corresponding to each first registered image respectively based on the predicted transformation field to obtain a second registered image and the second registered artifact segmentation results corresponding to each second registered image;

[0076] An artifact replacement module 75, configured to eliminate the artifacts in the reference image based on the reference image, the second registered image, and the second registered artifact segmentation results corresponding to each second registered image, and generate a medical image with artifacts eliminated.

[0077] Among them, the device for eliminating artifacts in medical imaging images can be a medical imaging device or a device such as a computer that can communicate with the medical imaging device for data. Thus, artifacts in real-time acquired medical imaging images can be eliminated, so that during the process of acquiring medical imaging images, artifact-free medical imaging images with higher image quality can be generated through artifact elimination, assisting in improving medical efficiency.

[0078] It should be noted that the model structure and training process of the registration model in the embodiments of the present invention are as described above, so they will not be elaborated here. It should also be noted that in specific practice, the training of the registration model can be carried out on the device 3 for eliminating artifacts in medical imaging images, or not on this device. When the training is carried out on this device, the device 3 may further include a model training module, which is used to train and generate the above-mentioned registration model, and its specific implementation process can refer to the previous description.

[0079] In addition, it should be noted that the implementation process and implementation principle of each module of the device for eliminating artifacts in medical imaging images in the embodiments of the present invention can be specifically referred to the corresponding description of the above method embodiments, so they will not be elaborated here.

[0080] In some embodiments, the embodiments of the present invention provide a non-volatile computer-readable storage medium, in which one or more programs including execution instructions are stored. The execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute the method for eliminating artifacts in medical imaging images in any one of the above embodiments of the present invention.

[0081] In some embodiments, the embodiments of the present invention further provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is made to execute the method for eliminating artifacts in medical imaging images in any one of the above embodiments.

[0082] In some embodiments, the embodiments of the present invention further provide an electronic device, which includes: at least one processor, and a memory communicatively connected to the at least one processor. Among them, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method for eliminating artifacts in medical imaging images in any one of the above embodiments.

[0083] In some embodiments, the embodiments of the present invention further provide a storage medium, on which a computer program is stored, characterized in that when the program is executed by a processor, it implements the method for eliminating artifacts in medical imaging images in any one of the above embodiments.

[0084] Figure 8 FIG. is a schematic hardware structure diagram of an electronic device for executing the method for eliminating artifacts in medical imaging images provided by another embodiment of the present invention, as Figure 8 shown, the device includes:

[0085] One or more processors 710 and a memory 720, Figure 8 Taking one processor 710 as an example.

[0086] The device for executing the method for eliminating artifacts in medical imaging images may further include: an input device 730 and an output device 740.

[0087] The processor 710, the memory 720, the input device 730 and the output device 740 may be connected through a bus or other means, Figure 8 Taking connection through a bus as an example.

[0088] The memory 720, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs and modules, such as program instructions / modules corresponding to the method for eliminating artifacts in medical imaging images in the embodiments of the present invention. The processor 710 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 720, that is, implements the method for eliminating artifacts in medical imaging images in the above method embodiments.

[0089] The memory 720 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the method for eliminating artifacts in medical imaging images, etc. In addition, the memory 720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 720 may optionally include a memory remotely set relative to the processor 710, and these remote memories may be connected to the electronic device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network and their combinations.

[0090] The input device 730 can receive input digital or character information, and generate signals related to user settings and function control of the image processing device. The output device 740 may include a display device such as a display screen.

[0091] The one or more modules are stored in the memory 720 and, when executed by the one or more processors 710, perform the method for eliminating artifacts in medical image images in any of the above method embodiments.

[0092] The above product can execute the method provided by the embodiments of the present invention, and has functional modules and beneficial effects corresponding to the execution of the method. For technical details not described in detail in this embodiment, reference may be made to the method provided by the embodiments of the present invention.

[0093] The electronic device according to the embodiment of the present invention exists in various forms, including but not limited to:

[0094] (1) Mobile communication devices: These devices are characterized by having mobile communication functions and mainly aim to provide voice and data communication. Such terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones, etc.

[0095] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have the characteristic of mobile Internet access. Such terminals include: PDA, MID, and UMPC devices, etc., such as iPad.

[0096] (3) Portable entertainment devices: These devices can display and play multimedia content. Such devices include: audio and video players (such as iPod), handheld game consoles, e-books, and smart toys and portable in-vehicle navigation devices.

[0097] (4) Servers: Devices that provide computing services. The composition of a server includes a processor, hard disk, memory, system bus, etc. A server is similar to a general computer architecture, but due to the need to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, manageability, etc.

[0098] (5) Other electronic devices with data interaction functions.

[0099] The device embodiments described above are merely illustrative, where the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0101] The above are only some embodiments of the present invention. For those of ordinary skill in the art, without departing from the inventive concept of the present invention, several deformations and improvements can still be made, and these all belong to the protection scope of the present invention.

Claims

1. A method for eliminating artifacts in medical imaging images, characterized in that: The method comprises: Determine a reference image and an image to be registered in a sequence of medical image images according to an artifact segmentation result of the medical image images; Based on the reference image, first registration processing is performed on the image to be registered and the artifact segmentation results corresponding to each image to be registered, respectively, to obtain a first registered image and first registration artifact segmentation results corresponding to each first registered image; Inputting the reference image and the first registered image into a pre-trained registration model to obtain a predicted transformation field output by the registration model, wherein the pre-trained registration model is a diffusion model that takes the reference image and the first registered image as input and takes the predicted transformation field as output; Based on the predicted transformation field, the first registered image and the first registration artifact segmentation results corresponding to each first registered image are respectively transformed to obtain a second registered image and the second registration artifact segmentation results corresponding to each second registered image; Based on the reference image, the second registered image and the second registration artifact segmentation results corresponding to each second registered image, the artifact part in the reference image is eliminated to generate a medical image with artifacts eliminated.

2. The method according to claim 1, characterized in that The artifact segmentation result of the medical image is obtained by inputting the medical image sequence into a pre-trained artifact segmentation model, wherein the pre-trained artifact segmentation model is a semantic segmentation model with the medical image sequence as input and the artifact segmentation sequence as output, and the artifact segmentation sequence includes artifact segmentation results corresponding to each medical image in the medical image sequence.

3. The method according to claim 2, characterized in that According to the artifact segmentation result of the medical image, the reference image and the image to be registered in the medical image sequence are determined to include: Determine the artifact volume in each medical image according to the artifact segmentation result of each medical image in the medical image sequence; A medical image with the smallest artifact volume in the medical image sequence is selected as a reference image, and other medical image images in the medical image sequence are used as images to be registered.

4. The method according to claim 1, characterized in that The registration model adopts a denoising diffusion probability model, which includes two Unet networks, one of which is composed of an encoder and a registration encoder, and the other is composed of an encoder and a diffusion encoder. The two Unet networks are designed to share the parameters of the encoder. The Unet network is used to generate a predicted transformation field based on the input reference image and the first registration image.

5. The method according to claim 4, characterized in that The registration model is trained by the following method: Acquire a sample reference image and a sample image to be registered, and add noise to the sample reference image based on the principle of a denoising diffusion probability model to form at least one set of training data, wherein each set of training data includes a sample reference image, a sample reference image with added noise, and a set of sample images to be registered; Each group of training data is input into a pre-built diffusion model, and the model parameters of the diffusion model are optimized based on a loss function and an optimizer, and the registration model is generated according to the optimization result, wherein the loss function includes a diffusion loss function and a registration loss function.

6. The method according to claim 5, characterized in that When the reference image and the first registration image are input into the pre-trained registration model, the method is to simultaneously input two identical reference images and a set of first registration images corresponding to the reference images into the pre-trained registration model.

7. The method according to claim 5, characterized in that Performing artifact elimination on the artifact part of the reference image based on the reference image, the second registered image, and the second registered artifact segmentation results corresponding to each second registered image to generate a medical image with artifacts eliminated includes: According to the artifact segmentation result corresponding to the reference image, all artifact instances in the reference image are identified and separated, wherein each artifact instance corresponds to an independent connected artifact pixel point set; Based on the artifact instance and the second registration artifact segmentation result corresponding to each second registered image, a replaceable instance of each artifact instance in the second registered image is screened out, wherein each replaceable instance also corresponds to a pixel point set, and the position of the pixel point set of the replaceable instance in the corresponding second registered image is the same as the position of the corresponding artifact instance in the reference image, and the pixel point set in the replaceable instance does not contain artifact pixels; The pixel values ​​of the corresponding areas in the reference image are replaced based on the replaceable instances corresponding to the artifact instances to generate a medical imaging image with artifacts eliminated.

8. A device for eliminating artifacts in medical imaging images, characterized in that: The device comprises: A preprocessing module, used for determining a reference image and an image to be registered in a sequence of medical image images according to an artifact segmentation result of the medical image images; A coarse registration module, configured to perform first registration processing on the image to be registered and the artifact segmentation results corresponding to each of the images to be registered based on the reference image, to obtain a first registered image and first registration artifact segmentation results corresponding to each of the first registered images; a parameter prediction module, used for inputting the reference image and the first registered image into a pre-trained registration model to obtain a predicted transformation field output by the registration model, wherein the pre-trained registration model is a diffusion model that takes the reference image and the first registered image as input and takes the predicted transformation field as output; A fine registration module, configured to transform the first registered image and the first registration artifact segmentation results corresponding to each first registered image based on the predicted transformation field to obtain a second registered image and the second registration artifact segmentation results corresponding to each second registered image; The artifact replacement module is used to perform artifact elimination on the artifact part of the reference image based on the reference image, the second registered image and the second registered artifact segmentation results corresponding to each second registered image, so as to generate a medical image with artifacts eliminated.

9. Electronic equipment, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the method described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images