Deep learning based artifact removal method, apparatus, device, and storage medium

Abstract

The disclosure provides a deep learning-based artifact removal method, device, equipment and storage medium, relating to the field of artificial intelligence, wherein the method comprises: acquiring a target projection image corresponding to a target image containing artifacts; determining an artifact-free projection area in the target projection image; performing image reconstruction on the target projection image based on the artifact-free projection area to obtain a first reconstructed projection image; inputting the first reconstructed projection image into a denoising image generation model to perform denoising processing on a back projection image corresponding to the first reconstructed projection image to obtain a first denoised image of the target image; wherein the denoising image generation model is a model obtained by constraint training of a denoising image generation model based on artifact-free sample images and sample images corresponding to sample noisy projection images. The technical solution provided by the disclosure can effectively remove artifacts in the image, improve the image quality and ensure the accuracy of the image information.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, and in particular, to a deep learning based artifact removal method and device, equipment and storage medium. BACKGROUND

[0002] At present, a lot of important information can be obtained through images, such as identity recognition information, lesion information, building structure information and location information, etc., therefore, it is necessary to ensure the quality of the image to ensure the accuracy of the image information. However, for various reasons, many images carry artifacts, which seriously reduce the image quality and clarity, and thus effective information cannot be obtained from the images with artifacts, for example, medical images often have metal artifacts or magnetic sensitive artifacts, etc., which leads to the inability to diagnose diseases according to the medical images with artifacts, bringing great challenges to the subsequent related treatment.

[0003] Therefore, it is necessary to provide a reliable and effective scheme to solve the problem of existing artifacts in images. SUMMARY

[0004] The present disclosure provides a deep learning based artifact removal method, device, equipment and storage medium, which can effectively remove artifacts in images, improve image quality and ensure the accuracy of image information.

[0005] In one aspect, the present disclosure provides a deep learning based artifact removal method, which comprises:

[0006] obtaining a target projection image corresponding to a target image containing artifacts;

[0007] determining a non-artifact projection area in the target projection image;

[0008] performing image reconstruction on the target projection image based on the non-artifact projection area to obtain a first reconstructed projection image;

[0009] inputting the first reconstructed projection image into a denoising image generation model to perform denoising processing on a back projection image corresponding to the first reconstructed projection image to obtain a first denoising image of the target image;

[0010] wherein the denoising image generation model is a model obtained by constraint training of a denoising image generation model based on a non-artifact sample image and a sample noisy projection image corresponding to the sample image.

[0011] In another aspect, a deep learning based artifact removal device is provided, which comprises:

[0012] an image acquisition module for acquiring a target projection image corresponding to a target image containing artifacts;

[0013] Image region determination module: used to determine the artifact-free projection region in the target projection image;

[0014] First image reconstruction module: used to reconstruct the target projection image based on the artifact-free projection region to obtain a first reconstructed projection image;

[0015] First image generation module: used to input the first reconstructed projection image into the denoising image generation model, perform denoising processing on the back projection image corresponding to the first reconstructed projection image, and obtain the first denoised image of the target image;

[0016] The denoising image generation model is a model obtained by constraining the generation model to generate denoised images based on artifact-free sample images and the denoised projection images of the corresponding sample images.

[0017] On the other hand, a deep learning-based artifact removal device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program segment, the at least one instruction or the at least one program segment being loaded and executed by the processor to implement the deep learning-based artifact removal method as described above.

[0018] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the deep learning-based artifact removal method as described above.

[0019] The deep learning-based artifact removal method, apparatus, device, and storage medium disclosed herein have the following technical advantages:

[0020] This disclosure enables the acquisition of a target projection image corresponding to a target image containing artifacts, the determination of artifact-free projection regions in the target projection image, and the reconstruction of the target projection image based on the artifact-free projection regions to obtain a first reconstructed projection image. This process removes interference from artifact-containing regions in the image, transforming the artifact removal problem into an image denoising problem. Then, the first reconstructed projection image is input into a denoising image generation model to denoise the back-projection image corresponding to the first reconstructed projection image, resulting in a first denoised image of the target image. This process preserves the details of the original image while obtaining a noise-free artifact-free image, effectively improving image quality and ensuring the accuracy of image information. Attached Figure Description

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, and the advantages thereof, the following will briefly introduce the drawings required by the embodiments or the prior art description. Obviously, the drawings described below are only some of the embodiments of the present disclosure, and for those of ordinary skill in the art, other drawings can be obtained from these drawings without creative labor.

[0022] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present disclosure;

[0023] Figure 2 is a flowchart of a training method of a denoising image generation model provided by an embodiment of the present disclosure;

[0024] Figure 3 is a process diagram of obtaining a CT sample image corresponding to a CT sample noise-added projection image provided by an embodiment of the present disclosure;

[0025] Figure 4 is a process diagram of using a generation model to perform denoising processing on a CT sample noise-added projection image to obtain a noise-free CT sample image provided by an embodiment of the present disclosure;

[0026] Figure 5 is a flowchart of a deep learning-based artifact removal method provided by an embodiment of the present disclosure;

[0027] Figure 6 is a deep learning-based artifact removal process diagram provided by an embodiment of the present disclosure;

[0028] Figure 7 is a deep learning-based artifact removal experimental result diagram provided by an embodiment of the present disclosure;

[0029] Figure 8 is a structural diagram of a deep learning-based artifact removal device provided by an embodiment of the present disclosure;

[0030] Figure 9 is a hardware structure block diagram of a server of a deep learning-based artifact removal method provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present disclosure.

[0032] It is to be understood that the terms "first", "second", and the like, used in the description and the claims of the present disclosure as well as the foregoing drawings of the related art, are used to differentiate between similar objects, and are not necessarily used to describe a particular sequential or chronological order. It is to be understood that the terms so used in the description are interchangeable under appropriate circumstances and embodiments of the present disclosure are capable of operating in other sequences than the one explicitly described or illustrated herein. Further, the terms "comprise" and "comprising" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, product, or device that comprises a list of steps or units is not necessarily limited to those steps or units that are expressly listed, but can include additional steps or units not expressly listed or inherent to such process, method, product, or device.

[0033] Before the embodiments of the present disclosure are further described, terms and names involved in the embodiments of the present disclosure are explained, and the terms and names involved in the embodiments of the present disclosure are applicable to the following explanations.

[0034] Artifact refers to various forms of images that do not exist in the scanned object but appear on the image.

[0035] CT (Computed Tomography, electronic computed tomography) is a kind of medical imaging device, which uses precisely collimated X-ray beams, gamma rays, ultrasonic waves, etc., together with a highly sensitive detector to make one after another cross-sectional scans around a certain part of the human body, and obtains the cross-sectional information of the object by measuring the projection of the object at different angles. Imaging can be used for the examination of various diseases.

[0036] HU (Hounsfield Unit, Hounsfield Unit) is used to measure CT value, which is a unit of measurement for determining the density of a local tissue or organ in the human body. Air is -1000, and dense bone is +1000.

[0037] CNN (Convolutional Neural Network, Convolutional Neural Network) is a kind of feedforward neural network (Feedforward Neural Networks) containing convolution calculation and having deep structure, which is one of the representative algorithms of deep learning (deep learning) and is widely used in image classification tasks.

[0038] Unsupervised learning is a kind of machine learning, which refers to automatically classifying or enhancing the input data without giving prior labeled training examples.

[0039] Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other directions.

[0040] In recent years, with the research and progress of artificial intelligence technology, artificial intelligence technology has been widely applied in many fields. The scheme provided by the embodiments of the present disclosure relates to machine learning / deep learning technology of artificial intelligence, which is specifically explained as follows:

[0041] Please refer to Figure 1 , Figure 1 is a schematic diagram of an application environment provided by the embodiments of the present disclosure, as Figure 1 shown, the application environment can at least include a server 01 and a terminal 02.

[0042] In the embodiments of the present disclosure, the server 01 can include a standalone server, or a distributed server, or a server cluster composed of multiple servers. The server 01 can include a network communication unit, a processor, a memory and the like. Specifically, the server 01 can be used for training and learning of a denoising image generation model. In the embodiments of the present disclosure, the denoising image generation model can be used for denoising processing of a reconstructed projection image to generate a denoising image without artifacts.

[0043] In the embodiments of the present disclosure, the terminal 02 can include an intelligent mobile phone, a desktop computer, a tablet computer, a notebook computer, a digital assistant, an augmented reality (AR) / virtual reality (VR) device, an intelligent wearable device, a medical imaging device (such as a CT computed scanning instrument and a nuclear magnetic resonance instrument), an AI disease diagnosis device and the like. It can also include software running in the physical device, such as an application program and the like. The operating system running on the terminal 02 in the embodiments of the present disclosure can include but is not limited to an Android system, an IOS system, linux, windows and the like. In the embodiments of the present disclosure, the terminal 02 can be used to provide image reconstruction services, acquire a reconstructed projection image of a target image, and provide image denoising processing services based on the denoising image generation model trained by the server 01, so that the image of the target image after image reconstruction and denoising processing retains the detail features of the original image, and the artifacts of the target image can be removed.

[0044] In the embodiments of the present disclosure, the artifacts can include, but are not limited to, metal artifacts, motion artifacts, confusing artifacts or wrapped artifacts, chemical shift artifacts, chemical misregistration artifacts, truncation artifacts, magnetic susceptibility artifacts, zipper artifacts, cross-excitation and corduroy artifacts, and the like. Accordingly, the target image can be an image containing artifacts, such as a CT image containing artifacts.

[0045] In addition, it should be noted that, Figure 1 The application environment shown is only an application environment of the deep learning-based artifact removal processing, and in actual application, the training and learning of the denoised image generation model can also be processed on a device providing image denoising processing services.

[0046] The following describes the deep learning-based artifact removal modeling method of the present disclosure. In the embodiments of the present disclosure, modeling can be performed based on image reconstruction and a deep learning model, and specifically can include:

[0047] S101: An initial model is constructed based on artifact removal iterative reconstruction, and the initial model includes a projection image reconstruction term and an image prior constraint term; wherein the projection image reconstruction term is used to estimate a reconstructed projection image without artifacts from a to-be-processed image containing artifacts, and the image prior constraint term is used to constrain the estimated reconstructed projection image without artifacts.

[0048] In the embodiments of the present disclosure, the artifact removal iterative reconstruction is to model the artifact removal problem as an image filling problem to construct the projection image reconstruction term; and the image prior knowledge is introduced to construct the image prior constraint term to constrain the reconstructed projection image; and the artifact removal image is obtained by solving the optimization artifact removal problem.

[0049] In some embodiments, step S101 can include:

[0050] S1011: A mask corresponding to an artifact region in the image containing artifacts is acquired.

[0051] S1012: A projection image reconstruction term is constructed based on the mask and a projection function.

[0052] S1013: An initial model is constructed based on the projection image reconstruction term and the weighted image prior constraint term.

[0053] In one embodiment, the expression of the initial model is:

[0054]

[0055] wherein, Y represents a target artifact removal image, Y is a projection image corresponding to the image containing artifacts; X is a variable to be solved, representing an artifact removal image obtained by artifact removal processing; M represents a mask corresponding to an artifact region in the image containing artifacts. tis a binary mask corresponding to the artifact region in the image containing artifacts, 1 in the mask represents that there is an artifact at the position, and 0 represents that there is no artifact at the position; A is a projection function; the first term in the above formula one is a projection image reconstruction term; R(·) represents an image prior constraint, and the second term R(X) in the above formula one is an image prior constraint term; λ is a first weight coefficient, used for weighting R(X) to balance the first term and the second term in the formula one.

[0056] In one embodiment, the image containing artifacts can be a CT image containing artifacts, and correspondingly, The image containing artifacts is represented as Y, and the de-artifact CT image obtained through de-artifact processing is represented as X.

[0057] In S102, a decoupling constraint term is added to the initial model to obtain a de-artifact model, and the decoupling constraint term is used to decouple the artifact-free projection image from the initial model.

[0058] In some embodiments, the decoupling constraint term is constructed by introducing a substitution variable.

[0059] In one embodiment, based on the above formula one, a substitution variable Z is introduced, and another Z is equal to AX. It can be known that Z is a projection image obtained by projecting the de-artifact image X, and is an artifact-free projection image. In the following, it is represented as an artifact-free reconstructed projection image, and then the decoupling constraint condition that Z is equal to AX is obtained.

[0060] Further, in order to facilitate subsequent calculation, a scaling factor The formula one of the initial model is transformed into the following formula two:

[0061]

[0062] Wherein, s.t. Z = AX is a decoupling constraint condition.

[0063] It should be noted that because the scaling factor is a scalar, it does not change the solution of the optimization problem in the initial model, so the formula one and the formula two are equivalent. The formula two can effectively decouple the artifact-free projection image Z (artifact-free reconstructed projection image) from the optimization problem in the formula one, and provide convenience for subsequent model solving;

[0064] Further, the Lagrange multiplier method is adopted, the decoupling constraint term is obtained based on the above decoupling constraint condition, and the decoupling constraint term is added to the above formula two to obtain the following formula three, and the formula three is an expression of the de-artifact model:

[0065] ​

[0066] wherein μ is a second weight coefficient for balancing the first two terms and the third term in Equation Three, the third term in Equation Three for decoupling the constraint term.

[0067] In practical applications, based on the above steps is S102, the preliminary modeling of the image de-artifact problem has been completed.

[0068] Based on the above embodiment, in the embodiment of the disclosure, the modeling method further includes a step of solving the above de-artifact model, which can specifically include:

[0069] S103: decompose the de-artifact model according to the variable separation method or the alternative direction multiplier algorithm to obtain a first sub-model and a second sub-model; wherein the first sub-model includes a projection image reconstruction term and a decoupling constraint term, and is used to reconstruct a corresponding artifact-free reconstructed projection image from an image containing artifacts; the second sub-model includes an image prior constraint term and a decoupling constraint term, and is used to generate a denoised artifact-free denoised image based on the artifact-free reconstructed projection image;

[0070] In one embodiment, the de-artifact model is decomposed by the variable separation method to obtain the first sub-model and the second sub-model.

[0071] Specifically, the optimization problem of Equation Three is decomposed into two sub-problems by the variable separation method, the first sub-problem is to reconstruct a corresponding artifact-free reconstructed projection image from an image containing artifacts, and the second sub-problem is to generate a denoised artifact-free denoised image based on the artifact-free reconstructed projection image.

[0072] Further, the essence of the first sub-problem is to calculate Z, and only the first term and the third term in Equation Three are related to Z, so the expression of the first sub-model corresponding to the first sub-problem is obtained based on the first term and the third term (please refer to the first row of Equation Four); the essence of the second sub-problem is to calculate X, and only the second term and the third term in Equation Three are related to X, so the expression of the second sub-model corresponding to the second sub-problem is obtained based on the second term and the third term (please refer to the second row of Equation Four).

[0073]

[0074] S104: obtaining an analytical solution of the first sub-model;

[0075] In practical applications, the first sub-model corresponding to the first sub-problem has an analytical solution.

[0076] In one embodiment, based on the above formula, the derivative of the expression of the first sub-model is taken, and the derivative is set to 0 to obtain the expression of the analytical solution of the first sub-model, as shown in the following Formula V, k represents the number of iterations. As can be seen from Formula V, the reconstruction projection image Z without artifacts is a linear combination of the projection image Y corresponding to the artifact image containing artifacts and AX, where AX is a projection image obtained by projecting the artifact-removed image. In this way, the details in the artifact image can be fully preserved, and the image obtained after artifact removal is more realistic and clear.

[0077]

[0078] S105: Construct a deep learning-based denoised image generation model according to the second sub-model;

[0079] In actual application, the second sub-problem corresponding to the second sub-model is essentially an image denoising problem, and therefore, a deep learning-based denoised image generation model can be constructed based on the second model and the second sub-problem.

[0080] In one embodiment, based on the above Formula IV and Formula V, the expression of the denoised image generation model can be as shown in the following Formula VI, and the denoised image generation model can be a model obtained by constraint training of a sample image pair generation model based on a sample image without artifacts and a sample noisy projection image corresponding to the sample image.

[0081]

[0082] S106: Determine the first sub-model and the denoised image generation model as the target artifact-removed model.

[0083] In this way, the training of the deep learning model such as the generation model based on the paired sample artifact image and the sample artifact-removed image is not required, the difficulty of obtaining the training data and the difficulty of training the denoised image generation model are reduced, the accuracy of the denoised image generation model is improved, the image artifact-removed problem is converted into an unsupervised learning problem, and the overall accuracy and universality of the target artifact-removed model are improved.

[0084] In actual application, in the artifact-removed application process of the target artifact-removed model, the first sub-model and the denoised image generation model are solved, and the denoised image output by the denoised image generation model is taken as the final target artifact-removed image.

[0085] In some embodiments, in the artifact-removed application process of the target artifact-removed model, the first sub-model and the denoised image generation model are alternately solved until the denoised image output by the denoised image generation model satisfies a preset convergence condition, and the denoised image satisfying the preset convergence condition is taken as the final target artifact-removed image.

[0086] Thus, by solving the second sub-model through the denoising image generation model, the prior knowledge learned by the denoising image generation model and having stronger generalization is used to constrain the artifact-free reconstructed projection image obtained by the first sub-model, so that the artifacts in the artifact image can be effectively removed, and the image authenticity and clarity can be improved.

[0087] The following introduces an embodiment of a training process of the denoising image generation model based on deep learning of the present disclosure. Specifically, the training process can include:

[0088] In the embodiment of the present disclosure, the denoising image generation model is a model obtained by constraint training of a denoising image generation model based on a sample image without artifacts and a sample noisy projection image corresponding to the sample image.

[0089] The following introduces an embodiment of a training method of the denoising image generation model based on deep learning. Figure 2 The following introduces an embodiment of a training method of the denoising image generation model based on deep learning.

[0090] S201: Obtain a sample image without artifacts and a sample noisy projection image corresponding to the sample image.

[0091] In the embodiment of the present disclosure, the sample image and the sample noisy projection image can form a sample training pair as training data of the generation model. Specifically, the sample image can be a large number of images.

[0092] In one embodiment, assuming that the category of the sample image is a medical image, the sample image can be a large number of CT images without artifacts. Correspondingly, a large number of CT images without artifacts can be obtained from a public CT dataset, and then the obtained CT images are preprocessed according to the requirements of the model for input images in subsequent training, such as adjusting the size of the image by scaling, so as to obtain the sample image.

[0093] Further, the CT images can be divided into a training set and a test set according to the size of the HU value in the CT image. Specifically, CT images containing artifacts, such as CT images containing metal artifacts, can be obtained from an internationally public CT dataset-SpineWeb to serve as the test set. In one specific embodiment, the training set includes more than 20,000 CT images without metal artifacts, and the test set includes more than 200 CT images containing metal artifacts.

[0094] In another embodiment, the sample image can also be a human image, such as a face image. Correspondingly, a large number of face images without artifacts of users can be collected from an Internet website, and then the collected face images are preprocessed according to the requirements of the model for input images in subsequent training, such as adjusting the size of the image by scaling, to serve as the sample object image.

[0095] It should be noted that the training method of the deep learning-based denoised image generation model provided in the present disclosure can be based on the target artifact removal model constructed in the foregoing artifact removal modeling method embodiment.

[0096] In this way, the training of the deep learning model such as the generation model does not need to be based on the paired sample artifact image and sample denoised image, the difficulty of obtaining the training data and the difficulty of training the generation model are reduced, the accuracy of the denoised image generation model is improved, and the artifact removal problem of the image is converted into an unsupervised learning problem, and the overall accuracy and universality of the target artifact removal model are improved.

[0097] In actual applications, the obtaining of the sample noisy projection image corresponding to the sample image in the foregoing step S201 can include:

[0098] S2011: performing projection transformation on the sample image to obtain a sample projection image;

[0099] S2012: performing noise addition processing on the sample projection image to obtain a sample noisy projection image.

[0100] In specific embodiments, the projection transformation is to establish a spatial mapping relationship between pixels of the original image and the projection image according to geometric constraints to obtain image transformation of the projection image. The projection transformation includes but is not limited to fan-shaped projection transformation and parallel projection transformation, etc. In an embodiment, the geometric constraint of the projection transformation can be the projection function (for example, the projection function A) in the foregoing artifact removal modeling method embodiment.

[0101] In some embodiments, the step S2012 can specifically be: performing Gaussian noise addition processing on the sample projection image to obtain a sample noisy projection image containing Gaussian noise.

[0102] In an embodiment, the sample image can be a CT sample image without artifacts. Please refer to Figure 3 , Figure 3 FIG. 1 is a process schematic diagram of obtaining a CT sample noisy projection image corresponding to a CT sample image provided in an embodiment, wherein Figure 3 a is the CT sample image, Figure 3 b is the CT sample projection image, Figure 3 c is the CT sample noisy projection image.

[0103] It should be noted that the noise addition processing on the sample projection image is not limited to the Gaussian noise addition processing described above, but can also include other noise addition processing capable of generating a noisy projection image.

[0104] Thus, by collecting the artifact-free images and batch-noising the artifact-free images, the training sample set is obtained, without collecting paired sample pairs, reducing the difficulty of obtaining training data and the difficulty of training the model, and improving the accuracy of the denoised image generation model.

[0105] S202: Based on the sample image and the sample noised projection image, the denoised image generation model is trained until the training denoised image output by the model meets the training convergence condition.

[0106] In the embodiments of the present disclosure, the generation model can include but is not limited to a deep learning model such as a convolutional neural network, a recurrent neural network or a recursive neural network.

[0107] In specific embodiments, the first convolutional layer of the generation model can be used to perform back-projection transformation processing on the input image, which can back-project the sample noised projection image to obtain the back-projection image of the sample noised projection image. Further, the other structure layers of the generation model perform denoising processing on the back-projection image of the sample noised projection image to output the denoised image.

[0108] In one embodiment, the geometric constraint of the back-projection transformation of the first convolutional layer can be the back-projection function corresponding to the projection function in the foregoing modeling method embodiment, for example, the back-projection function A -1 .

[0109] In actual applications, step S202 can include:

[0110] S2021: inputting the sample noised projection image into the generation model to perform denoising processing on the back-projection image corresponding to the sample noised projection image to obtain the training denoised image;

[0111] S2022: obtaining the image error between the training denoised image and the sample image;

[0112] S2023: adjusting the model parameters of the generation model based on the image error until the obtained image error meets the model convergence condition;

[0113] S2024: determining that the training denoised image corresponding to the image error meeting the model convergence condition meets the training convergence condition.

[0114] In specific embodiments, the sample noised projection image can be used as the input of the generation model to perform back-projection transformation processing on the sample noised projection image to obtain the back-projection image of the sample noised projection image; based on the back-projection image of the sample noised projection image and the sample image, the generation model is trained to learn the image prior knowledge of image denoising processing, and the training denoised image is obtained; then, the image error between the training denoised image and the sample image is calculated based on the loss function.

[0115] It should be noted that the model structure of the generation model can be set according to the requirements of the denoising processing, and the present disclosure does not limit this.

[0116] In actual application, step S2023 can include:

[0117] S20231: determining whether the image error meets the model convergence condition;

[0118] S20232: when the result of the determination is no, adjusting the model parameters in the generation model based on the gradient descent method, and repeating the training and learning steps of steps S2021, S2022 and S20231.

[0119] S20233: when the result of the determination is yes, performing step S2024.

[0120] In some embodiments, the image error meeting the model convergence condition can specifically be that the image error is less than or equal to a preset error threshold; specifically, the image error can represent the similarity between the training denoised image and the sample image.

[0121] In specific embodiments, the preset error threshold can be set in combination with the clarity requirement of the target denoised image obtained after the target image containing artifacts is subjected to the artifact removal processing in actual application. Generally, the smaller the preset error threshold, the higher the image clarity output by the trained denoised image generation model, but the longer the training time; on the contrary, the larger the preset error threshold, the lower the image clarity output by the trained denoised image generation model, but the shorter the training time.

[0122] S203: taking the generation model meeting the training convergence condition as a denoised image generation model.

[0123] In one embodiment, the sample image can be a CT sample image without artifacts, please refer to Figure 4 , Figure 4 is a process diagram provided by an embodiment for denoising a CT sample noisy projection image by using a generation model to obtain a CT sample image without noise, wherein, Figure 4 a is a CT sample noisy projection image, Figure 4 b is a CT sample image, and the dashed box represents a generation model.

[0124] It should be noted that the denoised image generation model in the training method embodiment of the denoised image generation model based on deep learning and the denoised image generation model in the denoising modeling method embodiment based on deep learning can be the same. In one embodiment, its expression can be the same as formula six.

[0125] In the embodiments of the present disclosure, the training and learning of the image denoising generation model in combination with the sample image and the sample noise-added projection image can improve the similarity between the image generated by the generation model and the sample image, and ensure that the denoising image generation model can process any projection image containing noise into an image without noise, clear and complete information.

[0126] The following describes a deep learning-based artifact removal method of the present disclosure based on a denoising image generation model, Figure 5 is a flowchart of a deep learning-based artifact removal method provided by the embodiments of the present disclosure. The present disclosure provides method operation steps as in the embodiments or flowcharts, but more or fewer operation steps can be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one of the many execution orders, and does not represent the only execution order. In actual system or server product execution, the method order shown in the embodiments or the drawings can be executed in sequence or in parallel (for example, in a parallel processor or multi-thread processing environment). Specifically as shown in Figure 5 The method can include:

[0127] S301: Obtain a target projection image corresponding to a target image containing artifacts.

[0128] In the embodiments of the present disclosure, artifacts can include but are not limited to metal artifacts, motion artifacts, confusion artifacts or wrapping artifacts, chemical shift artifacts, chemical misregistration artifacts, truncation artifacts, magnetic susceptibility artifacts, zipper artifacts, cross-excitation and corduroy artifacts, etc. Accordingly, the target image can be an image containing artifacts, such as a CT image containing artifacts.

[0129] In actual application, the target projection image corresponding to the target image is a projection image obtained by projection transformation of the target image. Specifically, the projection transformation here can be the same as the projection transformation in the foregoing artifact removal modeling method embodiments and the training method embodiments of the generation model, or it can be other types of projection transformation in the prior art, or it can be the projection transformation corresponding to the back projection transformation adopted in the generation process of the target image. For example, the target image is a CT image, and the process of obtaining the CT image by CT scanning reconstruction includes a back projection transformation process, and a corresponding back projection transformation geometric constraint is adopted. The projection transformation in step S301 can be the projection transformation corresponding to the back projection transformation geometric constraint.

[0130] S303: Determine a non-artifact projection region in the target projection image.

[0131] In the embodiments of the present disclosure, the non-artifact projection region is a region in the target projection image that is not disturbed by artifacts.

[0132] In actual application, before step S303, it can also include:

[0133] S302: Identify the artifact region and / or the non-artifact region in the target image.

[0134] Correspondingly, in an embodiment, step S303 can include:

[0135] S3031: Obtain the mapping relationship between the target image and the target projection image.

[0136] S3032: Determine the artifact projection region in the target projection image corresponding to the artifact region in the target image according to the mapping relationship between the target image and the target projection image.

[0137] S3033: Determine the non-artifact projection region in the target projection image as the region outside the artifact projection region in the target projection image.

[0138] Specifically, the non-artifact projection region in the target projection image is determined based on the artifact region in the target image, and the non-artifact projection region is the image region in the target projection image corresponding to the region outside the artifact region in the target image.

[0139] In another embodiment, step S303 can include:

[0140] S3031: Obtain the mapping relationship between the target image and the target projection image.

[0141] S3035: Determine the non-artifact projection region in the target projection image corresponding to the non-artifact region in the target image according to the mapping relationship between the target image and the target projection image.

[0142] In specific embodiments, the artifact region or the non-artifact region can be represented by a mask, or the artifact projection region or the non-artifact projection region can be represented by a mask. When the target image is a two-dimensional image, the mask can be a binary mask.

[0143] It should be noted that the artifact region and / or the artifact projection region in the target projection image can be identified first, and then the non-artifact region in the target image can be determined according to the mapping relationship between the target image and the target projection image.

[0144] S305: Perform image reconstruction on the target projection image based on the non-artifact projection region to obtain a first reconstructed projection image.

[0145] In the embodiments of the present disclosure, step S305 can include:

[0146] S3051: Perform image interpolation on the target image based on the non-artifact region in the target image corresponding to the non-artifact projection region to obtain an initial reconstructed image.

[0147] In actual application, the artifact region in the target image can be image interpolation reconstructed based on the characteristics (such as pixel characteristics, etc.) of the artifact-free region in the target image to obtain an initial reconstructed image. Specifically, the interpolation reconstruction can include but is not limited to linear interpolation reconstruction.

[0148] S3052: generating a first reconstructed projection image according to the projection image of the initial reconstructed image and the target sub-projection image corresponding to the artifact-free projection region in the target projection image.

[0149] In actual application, the projection image of the initial reconstructed image is an image obtained by projecting the initial reconstructed image, and the target sub-projection image is a projection image of the artifact-free projection region in the target projection image.

[0150] In some embodiments, step S3052 can be specifically: linearly combining the projection image of the initial reconstructed image and the target sub-projection image to obtain the first reconstructed projection image. Specifically, the linear combination here is weighted linear combination.

[0151] S307: inputting the first reconstructed projection image into the denoising image generation model to perform denoising processing on the back-projection image corresponding to the first reconstructed projection image to obtain a first denoised image of the target image.

[0152] The denoising image generation model is a model obtained by constraint training of a sample image generation model based on an artifact-free sample image and a sample noisy projection image corresponding to the sample image.

[0153] In the embodiments of the present disclosure, step S307 can include:

[0154] S3071: inputting the first reconstructed projection image into the denoising image generation model to perform back-projection transformation to obtain a back-projection image corresponding to the first reconstructed projection image;

[0155] S3072: performing denoising processing on the back-projection image corresponding to the first reconstructed projection image based on the denoising image generation model to obtain a first denoised image of the target image.

[0156] In actual application, the first convolutional layer of the denoising image generation model can be used to perform back-projection transformation processing on the input projection image to obtain the back-projection image of the input projection image. Further, the other structure layers of the denoising image generation model perform denoising processing on the back-projection image of the input projection image.

[0157] Based on all or part of the above embodiments, in some embodiments, the first denoised image is taken as a target artifact-removed image, and the artifact-removed processing of the target image is completed.

[0158] In the embodiments of the present disclosure, the target projection image corresponding to the target image containing the artifact can be obtained, the artifact-free projection region in the target projection image is determined, the target projection image is reconstructed based on the artifact-free projection region to obtain a first reconstructed projection image, the interference of the artifact-containing region in the image can be removed, the de-artifact problem of the image is converted into the de-noising problem of the image, then the first reconstructed projection image is input into the de-noising image generation model, the de-noising processing is performed on the back projection image corresponding to the first reconstructed projection image to obtain the first de-noising image of the target image, the de-artifact image without noise can be obtained on the basis of retaining the details of the original image, the image quality is effectively improved, and the accuracy of the image information is ensured.

[0159] Based on all or part of the above-mentioned embodiments, in some other embodiments, after the step S307, the de-artifact method can further include:

[0160] S309: performing projection transformation on the first de-noising image to obtain a first de-noising projection image.

[0161] In the embodiments of the present disclosure, the projection transformation processing on the first de-noising image can be similar to the projection transformation processing involved in the step S301.

[0162] S311: generating a second reconstructed projection image according to the first de-noising projection image and the target sub-projection image corresponding to the artifact-free region in the target projection image.

[0163] S313: inputting the second reconstructed projection image into the de-noising image generation model to perform de-noising processing on the back projection image corresponding to the second reconstructed projection image to obtain a second de-noising image.

[0164] In the embodiments of the present disclosure, the generation process of the second reconstructed projection image can be similar to the step S3052, and the generation process of the second de-noising image can be similar to the step S307, which will not be described herein.

[0165] S315: determining whether the second de-noising image satisfies a preset convergence condition.

[0166] In some embodiments, the preset convergence condition can include that the similarity between the currently generated de-noising image and the previously generated de-noising image is greater than or equal to a preset similarity. Correspondingly, the step S315 can include:

[0167] 1) obtaining the similarity between the second de-noising image and the first de-noising image;

[0168] 2) determining whether the similarity between the second de-noising image and the first de-noising image is greater than or equal to the preset similarity;

[0169] 3) if the determination result is yes, it is determined that the second de-noising image satisfies the preset convergence condition.

[0170] In specific embodiments, the preset similarity can be set according to the clarity requirement of the target de-artifact image obtained after the de-artifact processing of the target image containing artifacts. Generally, the greater the preset similarity, the higher the clarity of the output image, but the longer the de-artifact processing time; on the contrary, the smaller the preset similarity, the lower the clarity of the output image, but the shorter the de-artifact processing time.

[0171] In other embodiments, the preset convergence condition can include that the iteration number corresponding to the current generated de-noised image is a preset iteration number. Correspondingly, step S315 can include:

[0172] 1) obtaining the iteration number corresponding to the second de-noised image;

[0173] 2) determining whether the iteration number corresponding to the second de-noised image is consistent with the preset iteration number;

[0174] 3) if the determination result is yes, determining that the second de-noised image satisfies the preset convergence condition.

[0175] In specific embodiments, the preset iteration number can be set according to the clarity requirement of the target de-artifact image obtained after the de-artifact processing of the target image containing artifacts. Generally, the greater the preset iteration number, the higher the clarity of the output image, but the longer the de-artifact processing time; on the contrary, the smaller the preset iteration number, the lower the clarity of the output image, but the shorter the de-artifact processing time.

[0176] S317: if the second de-noised image satisfies the preset convergence condition, taking the second de-noised image as the target de-noised image of the target image.

[0177] S319: if the second de-noised image does not satisfy the preset convergence condition, repeating the above steps S309 to S315 until the obtained de-noised image satisfies the preset convergence condition, and taking the de-noised image satisfying the preset convergence condition as the target de-noised image of the target image.

[0178] In some embodiments, the linear combination in step S3052 is a weighted linear combination corresponding to a linear weighting coefficient. Correspondingly, when the preset convergence condition is that the iteration number corresponding to the current generated de-noised image is a preset iteration number, a preset linear weighting coefficient corresponding to each iteration in the de-artifact processing process can be set in advance, and the preset linear weighting coefficient gradually decreases with the increase of the iteration number.

[0179] In actual applications, the target de-noised image is determined as the target de-artifact image, and the de-artifact processing of the target image is completed.

[0180] In one embodiment, the generation process of the reconstructed projection image (including the first reconstructed projection image and the second reconstructed projection image, etc.) is based on the formula five in the foregoing artifact modeling method embodiment, and the expression of the denoised image generation model is the foregoing formula six, as shown below.

[0181]

[0182]

[0183] wherein M t is a binary mask corresponding to the artifact region in the target image, 1 in the mask represents that there is an artifact at the position, and 0 represents that there is no artifact at the position, and (1-M t ) is a NOT operation, representing the artifact-free region; Y is a target projection image corresponding to the target image, and correspondingly, (1-M t ) o Y represents a target sub-projection image; k is the number of iterations, A is a projection function, and μσ 2 may represent a linear weighting coefficient.

[0184] When k is equal to 0, in the formula five: X0 represents an initial reconstructed image, AX0 represents a projection image of the initial reconstructed image, and Z1(Z k+1 ) represents a first reconstructed projection image obtained by substituting X0 into the formula five, and X1(X k+1 ) in the formula six represents a first denoised image obtained by inputting the first reconstructed projection image into the formula six.

[0185] When k is greater than or equal to 1, X k in the formula five represents a denoised image obtained in the previous iteration based on the formula six, Y k is a back-projection image corresponding to X k , and AX k is a projection image obtained by projecting X k ; Z k+1 in the formula six is a reconstructed projection image obtained in the current iteration based on the formula five.

[0186] Further, if the foregoing first denoised image is taken as a target artifact-removed image, and k is equal to 1, the artifact removal of the target image is completed.

[0187] Further, if the foregoing target denoised image is determined as a target artifact-removed image, and k is equal to n (n≥1) and satisfies a preset convergence condition, the artifact removal of the target image is completed.

[0188] Further, if the preset convergence condition is that the number of iterations corresponding to the currently generated denoised image is a preset number of iterations (for example, 5 times), a preset linear weighting coefficient (for example, μσ2 the preset μσ value gradually decreases with the increase of the number of iterations. 2 the preset μσ value gradually decreases with the increase of the number of iterations.

[0189] It should be noted that in this case, step S3052 can obtain the first reconstructed projection image based on the above formula five, and Y is the initial reconstructed projection image, A is the projection function, X is the back projection image corresponding to Y, and AX is the projection image obtained by projecting the back projection image corresponding to Y.

[0190] In actual application, when the target image is processed, the target projection image corresponding to the target image containing artifacts is obtained, and the artifact-free projection region or the artifact projection region in the target projection image is determined, and / or the artifact region or the artifact-free region in the target image is determined; then, the target image is subjected to image interpolation to obtain an initial reconstructed image.

[0191] Further, the initial reconstructed image is substituted into the above formula five, and the formula five is solved according to the projection image of the initial reconstructed image and the target sub-projection image corresponding to the artifact-free projection region in the target projection image, to obtain a first initial reconstructed image; then, the first initial reconstructed image is input into formula six (denoising image generation model) for solving, and the back projection image corresponding to the first reconstructed projection image is subjected to denoising processing in the solving process, to obtain the first denoising image of the target image.

[0192] In the case where iterative calculation is not required, the first denoising image is taken as the target artifact-removed image.

[0193] Or, in the case where iterative calculation is required, the first denoising image is further subjected to projection transformation to obtain a first denoising projection image; then, the first denoising projection image is substituted into formula five, and formula five is solved according to the first denoising projection image and the target sub-projection image corresponding to the artifact-free projection region in the target projection image, to obtain a second reconstructed projection image; the second reconstructed projection image is input into formula six for solving, and the back projection image corresponding to the second reconstructed projection image is subjected to denoising processing in the solving process, to obtain a second denoising image, and the formula five and formula six are alternately solved until the denoising image output by the denoising image generation model satisfies the preset convergence condition, and the denoising image satisfying the preset convergence condition is taken as the target denoising image, i.e., the target artifact-removed image. Please refer to Figure 6 , Figure 6 is a deep learning-based artifact removal process diagram provided by the embodiment, wherein Figure 6 a is a target image, Figure 6 b is a target artifact-removed image.

[0194] Further, please refer to Figure 7 , Figure 7The deep learning-based de-artifact experimental result figure provided by the embodiment of the present disclosure is shown. The first row in the figure is a CT image containing metal artifacts, and the second row is a de-artifact CT image processed based on the de-artifact method of the present disclosure. It can be seen that the present method effectively removes the metal artifacts in the CT image, improves the quality of the CT image, and brings great convenience to subsequent disease diagnosis and treatment.

[0195] It should be noted that when the de-artifact method of the present disclosure is applied to medical image processing, it can be used as an image preprocessing method and executed in AI disease diagnosis equipment and the like, thereby improving the robustness of the AI disease diagnosis equipment to artifacts. Alternatively, it can also be used as an imaging algorithm and executed in a medical imaging instrument, thereby improving the imaging quality of the medical imaging instrument.

[0196] It should be noted that the deep learning-based de-artifact method of the present disclosure can include the aforementioned de-artifact modeling method and the training method of the de-noised image generation model.

[0197] In the embodiment of the present disclosure, a target projection image corresponding to a target image containing artifacts can be obtained, an artifact-free projection region in the target projection image is determined, and the target projection image is reconstructed based on the artifact-free projection region to obtain a first reconstructed projection image. The interference of the artifact-containing region in the image can be removed, the de-artifact problem of the image is converted into a de-noising problem of the image, then the first reconstructed projection image is input into a de-noised image generation model, and a back-projection image corresponding to the first reconstructed projection image is de-noised to obtain a first de-noised image of the target image. On the basis of preserving the details of the original image, a de-artifact image without noise can be obtained, the image quality is effectively improved, and the accuracy of the image information is ensured.

[0198] The embodiment of the present disclosure also provides a deep learning-based de-artifact device, as shown in Figure 8 The device comprises:

[0199] The image acquisition module 10 can be used to obtain a target projection image corresponding to a target image containing artifacts.

[0200] The image region determination module 20 can be used to determine an artifact-free projection region in the target projection image.

[0201] The first image reconstruction module 30 can be used to reconstruct the target projection image based on the artifact-free projection region to obtain a first reconstructed projection image.

[0202] The first image generation module 40 can be used to input the first reconstructed projection image into a de-noised image generation model, de-noise a back-projection image corresponding to the first reconstructed projection image, and obtain a first de-noised image of the target image.

[0203] The denoised image generation model is a model obtained by constraint training of a denoised image generation model based on a sample image without artifacts and a sample noise-added projection image corresponding to the sample image.

[0204] In some embodiments, the device of the present disclosure can further include:

[0205] The projection transformation module can be configured to, after inputting the first reconstructed projection image into the denoised image generation model to obtain a first denoised image of the target image by performing denoising processing on a back projection image corresponding to the first reconstructed projection image, perform projection transformation on the first denoised image to obtain a first denoised projection image;

[0206] The second image reconstruction module can be configured to generate a second reconstructed projection image according to the first denoised projection image and a target sub-projection image corresponding to the artifact-free region in the target projection image;

[0207] The second image generation module can be configured to input the second reconstructed projection image into the denoised image generation model to obtain a second denoised image by performing denoising processing on a back projection image corresponding to the second reconstructed projection image;

[0208] The target denoised image determination module can be configured to, if the second denoised image meets a preset convergence condition, determine the second denoised image as a target denoised image of the target image.

[0209] In some embodiments, the first image reconstruction module 30 can include:

[0210] The initial reconstruction unit can be configured to perform image interpolation on the target image based on an artifact-free region in the target image corresponding to the artifact-free projection region to obtain an initial reconstructed image;

[0211] The first reconstruction unit can be configured to generate a first reconstructed projection image according to a projection image of the initial reconstructed image and a target sub-projection image corresponding to the artifact-free projection region in the target projection image.

[0212] In some embodiments, the preset convergence condition can include: a similarity between a currently generated denoised image and a previously generated denoised image is greater than or equal to a preset similarity; or, an iteration number corresponding to the currently generated denoised image is a preset iteration number.

[0213] In some embodiments, the first image generation module 40 can include:

[0214] The back projection transformation unit can be configured to input the first reconstructed projection image into the denoised image generation model to obtain a back projection image corresponding to the first reconstructed projection image by performing back projection transformation;

[0215] The denoising processing unit can be configured to perform denoising processing on the back projection image corresponding to the first reconstructed projection image based on the denoising image generation model to obtain the first denoising image of the target image.

[0216] In some embodiments, the device of the present disclosure can further include:

[0217] The training sample data acquisition module can be configured to acquire the sample image without artifacts and the sample noisy projection image corresponding to the sample image.

[0218] The constraint training learning module can be configured to perform denoising image generation training on the generation model based on the sample image and the sample noisy projection image, so that the training denoising image output by the generation model satisfies the training convergence condition.

[0219] The generation model determination module can be configured to determine the generation model satisfying the training convergence condition as the denoising image generation model.

[0220] In some embodiments, the training sample data acquisition module can include:

[0221] The sample image back projection unit can be configured to perform projection transformation on the sample image to obtain the sample projection image.

[0222] The image noise adding unit can be configured to perform noise adding processing on the sample projection image to obtain the sample noisy projection image.

[0223] In some embodiments, the constraint training learning module can include:

[0224] The training denoising image generation unit can be configured to input the sample noisy projection image into the generation model, perform denoising processing on the back projection image corresponding to the sample noisy projection image, and obtain the training denoising image.

[0225] The image error acquisition unit can be configured to acquire the image error between the training denoising image and the sample image.

[0226] The model parameter adjustment unit can be configured to adjust the model parameter of the generation model based on the image error, so that the acquired image error satisfies the model convergence condition.

[0227] The training stop determination unit can be configured to determine that the training denoising image corresponding to the image error satisfying the model convergence condition satisfies the training convergence condition.

[0228] The device in the above device embodiment and the method embodiment are based on the same application concept.

[0229] The deep learning based artifact removing device provided in the embodiments of the present disclosure includes a processor and a memory, and the memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the deep learning based artifact removing method provided in the method embodiments.

[0230] The memory can be used to store software programs and modules, and the processor can execute various function applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by functions, etc.; and the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0231] The method embodiments provided in the embodiments of the present disclosure can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking the case of running on a server as an example, Figure 9 is a hardware structure block diagram of a server providing a deep learning based artifact removing method according to the embodiments of the present disclosure. As Figure 9 shown, the server 800 can have a large difference due to different configurations or performances, and can include one or more central processing units (CPU) 810 (the processor 810 can include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 830 for storing data, one or more storage media 820 (such as one or more mass storage devices) for storing application programs 823 or data 822. Among them, the memory 830 and the storage medium 820 can be temporary storage or persistent storage. The programs stored in the storage medium 820 can include one or more modules, and each module can include a series of instruction operations in the server. Further, the central processing unit 810 can be configured to communicate with the storage medium 820 to execute a series of instruction operations in the storage medium 820 on the server 800. The server 800 can also include one or more power supplies 860, one or more wired or wireless network interfaces 850, one or more input and output interfaces 840, and / or one or more operating systems 821, such as Windows Server TM , Mac OS X TM , Unix TMLinux™, FreeBSD™, and the like.

[0232] The input / output interface 840 can be configured to receive or transmit data via a network. The network can include a wireless network provided by a communication provider of the server 800. In an example, the input / output interface 840 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In an example, the input / output interface 840 can be a radio frequency (RF) module configured to communicate with the Internet through a wireless manner.

[0233] Those skilled in the art can understand that the server 800 can include more or less components than those shown, or have a different configuration of components than those shown. Figure 9 The structure shown is merely schematic, and does not limit the structure of the electronic device. For example, the server 800 can further include more or less components than those shown, or have a different configuration of components than those shown. Figure 9 Figure 9 The structure shown is merely schematic, and does not limit the structure of the electronic device. For example, the server 800 can further include more or less components than those shown, or have a different configuration of components than those shown.

[0234] The embodiments of the present disclosure further provide a storage medium, which can be arranged in a server to save at least one instruction or at least one program for implementing an image noise adding processing method in the method embodiments. The at least one instruction or the at least one program is loaded and executed by the processor to implement the image noise adding processing method provided by the above method embodiments.

[0235] Optionally, in the present embodiment, the storage medium can be located in at least one of a plurality of network servers in a computer network. Optionally, in the present embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0236] According to an aspect of the present disclosure, a computer program product or a computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the various optional implementation manners.

[0237] ​According to the embodiments of the deep learning-based artifact removal method, device, equipment, server or storage medium provided by the present disclosure, the target projection image corresponding to the target image containing artifacts is acquired, the artifact-free projection area in the target projection image is determined, the target projection image is reconstructed based on the artifact-free projection area to obtain a first reconstructed projection image, the interference of the artifact-containing area in the image can be removed, the artifact removal problem of the image is converted into a denoising problem of the image, then, the first reconstructed projection image is input into a denoising image generation model, the de-noising processing is performed on the back projection image corresponding to the first reconstructed projection image to obtain a first denoising image of the target image, the artifact-free image without noise can be obtained on the basis of retaining the details of the original image, the image quality is effectively improved, and the accuracy of the image information is ensured.

[0238] It should be noted that the above-mentioned embodiments of the present disclosure are only for description, not representing the advantages and disadvantages of the embodiments. The above-mentioned embodiments of the present disclosure are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multi-task processing and parallel processing are possible or can be advantageous.

[0239] Each of the embodiments of the present disclosure is described in a progressive manner, and the same or similar parts of each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the device, equipment and storage medium embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts are described in the method embodiment.

[0240] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing related hardware to complete, and the program can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk.

[0241] The above-mentioned is only the preferred embodiment of the present disclosure, and does not limit the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A deep learning based de-artifact method characterized in that, The method comprises: obtaining a target projection image corresponding to a target image containing an artifact; determining an artifact-free projection region in the target projection image; reconstructing the target projection image based on the artifact-free projection region to obtain a first reconstructed projection image; inputting the first reconstructed projection image into a denoising image generation model for back projection transformation to obtain a back projection image corresponding to the first reconstructed projection image; performing denoising processing on the back projection image corresponding to the first reconstructed projection image based on the denoising image generation model to obtain a first denoised image of the target image; wherein the denoising image generation model is a model obtained by constraint training of a generation model based on artifact-free sample images and sample noisy projection images corresponding to the sample images.

2. The method of claim 1, wherein, After the first reconstructed projection image is input into the denoising image generation model, the back projection image corresponding to the first reconstructed projection image is denoised to obtain the first denoised image of the target image, the method further comprises: performing projection transformation on the first denoised image to obtain a first denoised projection image; generating a second reconstructed projection image according to the first denoised projection image and a target sub-projection image corresponding to the artifact-free region in the target projection image; inputting the second reconstructed projection image into the denoising image generation model to perform denoising processing on the back projection image corresponding to the second reconstructed projection image to obtain a second denoised image; if the second denoised image satisfies a preset convergence condition, regarding the second denoised image as a target denoised image of the target image.

3. The method of claim 1, wherein, The method further comprises: obtaining a target projection image corresponding to a target image containing an artifact; determining an artifact-free projection region in the target projection image; 4. The method according to any one of claims 1 to 3, characterized in that, reconstructing the target projection image based on the artifact-free projection region to obtain a first reconstructed projection image; inputting the first reconstructed projection image into a denoising image generation model for back projection transformation to obtain a back projection image corresponding to the first reconstructed projection image; performing denoising processing on the back projection image corresponding to the first reconstructed projection image based on the denoising image generation model to obtain a first denoised image of the target image; wherein the denoising image generation model is a model obtained by constraint training of a generation model based on artifact-free sample images and sample noisy projection images corresponding to the sample images.

5. The method of claim 4, wherein, After the first reconstructed projection image is input into the denoising image generation model, the back projection image corresponding to the first reconstructed projection image is denoised to obtain the first denoised image of the target image, the method further comprises: performing projection transformation on the first denoised image to obtain a first denoised projection image; generating a second reconstructed projection image according to the first denoised projection image and a target sub-projection image corresponding to the artifact-free region in the target projection image; 6. The method of claim 4, wherein, inputting the second reconstructed projection image into the denoising image generation model to perform denoising processing on the back projection image corresponding to the second reconstructed projection image to obtain a second denoised image; if the second denoised image satisfies a preset convergence condition, regarding the second denoised image as a target denoised image of the target image. The method further comprises: obtaining a target projection image corresponding to a target image containing an artifact; determining an artifact-free projection region in the target projection image; reconstructing the target projection image based on the artifact-free projection region to obtain a first reconstructed projection image; inputting the first reconstructed projection image into a denoising image generation model for back projection transformation to obtain a back projection image corresponding to the first reconstructed projection image; performing denoising processing on the back projection image corresponding to the first reconstructed projection image based on the denoising image generation model to obtain a first denoised image of the target image; wherein the denoising image generation model is a model obtained by constraint training of a generation model based on artifact-free sample images and sample noisy projection images corresponding to the sample images. After the first reconstructed projection image is input into the denoising image generation model, the back projection image corresponding to the first reconstructed projection image is denoised to obtain the first denoised image of the target image, the method further comprises: performing projection transformation on the first denoised image to obtain a first denoised projection image; generating a second reconstructed projection image according to the first denoised projection image and a target sub-projection image corresponding to the artifact-free region in the target projection image; inputting the second reconstructed projection image into the denoising image generation model to perform denoising processing on the back projection image corresponding to the second reconstructed projection image to obtain a second denoised image; if the second denoised image satisfies a preset convergence condition, regarding the second denoised image as a target denoised image of the target image. The method further comprises: obtaining a target projection image corresponding to a target image containing an artifact; determining an artifact-free projection region in the target projection image; reconstructing the target projection image based on the artifact-free projection region to obtain a first reconstructed projection image; inputting the first reconstructed projection image into a denoising image generation model for back projection transformation to obtain a back projection image corresponding to the first reconstructed projection image; performing denoising processing on the back projection image corresponding to the first reconstructed projection image based on the denoising image generation model to obtain a first denoised image of the target image; wherein the denoising image generation model is a model obtained by constraint training of a generation model based on artifact-free sample images and sample noisy projection images corresponding to the sample images. After the first reconstructed projection image is input into the denoising image generation model, the back projection image corresponding to the first reconstructed projection image is denoised to obtain the first denoised image of the target image, the method further comprises: performing projection transformation on the first denoised image to obtain a first denoised projection image; generating a second reconstructed projection image according to the first denoised projection image and a target sub-projection image corresponding to the artifact-free region in the target projection image; inputting the second reconstructed projection image into the denoising image generation model to perform denoising processing on the back projection image corresponding to the second reconstructed projection image to obtain a second denoised image; if the second denoised image satisfies a preset convergence condition, regarding the second denoised image as a target denoised image of the target image. acquire an image error between the training denoised image and the sample image; adjust model parameters of the generation model based on the image error until the acquired image error satisfies a model convergence condition; determine that the training denoised image corresponding to the image error satisfying the model convergence condition satisfies the training convergence condition.

7. A deep learning based de-artifacting device, characterized by, The device comprises: an image acquisition module configured to acquire a target projection image corresponding to a target image containing an artifact; an image region determination module configured to determine an artifact-free projection region in the target projection image; a first image reconstruction module configured to perform image reconstruction on the target projection image based on the artifact-free projection region to obtain a first reconstructed projection image; a first image generation module configured to input the first reconstructed projection image into a denoised image generation model to perform back-projection transformation on the first reconstructed projection image to obtain a back-projection image corresponding to the first reconstructed projection image, and perform denoising processing on the back-projection image corresponding to the first reconstructed projection image based on the denoised image generation model to obtain a first denoised image of the target image; wherein the denoised image generation model is a model obtained by constraint training of a generation model based on an artifact-free sample image and a sample noisy projection image corresponding to the sample image.

8. The apparatus of claim 7, wherein, The device further comprises: a projection transformation module configured to perform projection transformation on the first denoised image to obtain a first denoised projection image after the first denoised image is obtained by inputting the first reconstructed projection image into the denoised image generation model and performing denoising processing on the back-projection image corresponding to the first reconstructed projection image to obtain the first denoised image of the target image; a second image reconstruction module configured to generate a second reconstructed projection image according to the first denoised projection image and a target sub-projection image corresponding to the artifact-free region in the target projection image; a second image generation module configured to input the second reconstructed projection image into the denoised image generation model to perform denoising processing on a back-projection image corresponding to the second reconstructed projection image to obtain a second denoised image; a target denoised image determination module configured to take the second denoised image as a target denoised image of the target image if the second denoised image satisfies a preset convergence condition.

9. The apparatus of claim 7, wherein, The first image reconstruction module comprises: an initial reconstruction unit configured to perform image interpolation on the target image based on an artifact-free region in the target image corresponding to the artifact-free projection region to obtain an initial reconstructed image; a first reconstruction unit configured to generate the first reconstructed projection image according to a projection image of the initial reconstructed image and a target sub-projection image corresponding to the artifact-free projection region in the target projection image.

10. The apparatus of any one of claims 7-9, wherein, The device further comprises: a training sample data acquisition module configured to acquire an artifact-free sample image and a sample noisy projection image corresponding to the sample image; a constraint training learning module configured to perform denoised image generation training on the generation model based on the sample image and the sample noisy projection image until a training denoised image output by the generation model satisfies a training convergence condition; a generation model determination module configured to take a generation model satisfying the training convergence condition as the denoised image generation model.

11. The apparatus of claim 10, wherein, The training sample data acquisition module comprises: a sample image back projection unit configured to perform projection transformation on the sample image to obtain a sample projection image; an image noise adding unit configured to perform noise adding processing on the sample projection image to obtain the sample noise-added projection image.

12. The apparatus of claim 10, wherein, The constraint training learning module comprises: a training denoised image generation unit configured to input the sample noise-added projection image into the generative model, perform denoising processing on a back projection image corresponding to the sample noise-added projection image, and obtain a training denoised image; an image error acquisition unit configured to acquire an image error between the training denoised image and the sample image; a model parameter adjustment unit configured to adjust model parameters of the generative model based on the image error until the acquired image error satisfies a model convergence condition; a training stop determination unit configured to determine that the training denoised image corresponding to the image error satisfying the model convergence condition satisfies the training convergence condition.

13. A computer readable storage medium, characterized in that, The storage medium has at least one instruction or at least one program stored therein, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the deep learning-based artifact removal method according to any one of claims 1 to 6.

14. A deep learning based de-artifacting device, characterized in that, The device comprises a processor and a memory, and the memory has at least one instruction or at least one program stored therein, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the deep learning-based artifact removal method according to any one of claims 1 to 6.

15. A computer program product, characterised in that, The computer program product comprises computer instructions stored in a computer readable storage medium, and a processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to implement the deep learning-based artifact removal method according to any one of claims 1 to 6.

Images