Image artifact removal method and device, equipment and storage medium

Abstract

The invention relates to an image artifact removal method and device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed target image, inputting the target image into a generative adversarial network model, removing the artifacts of the target image through a generator of the generative adversarial network model, and obtaining an artifact-free image; the generative adversarial network model is trained and generated through training data and a comprehensive loss function, the comprehensive loss function comprises an adversarial loss function determined through a window width and a window level, and the window width and the window level are combined with the generative adversarial network model, so that the contrast of the image and the definition of the structure can be enhanced; the artifact removing effect is more in line with the feeling of subjective vision, and the artifacts are effectively removed on the basis that hardware equipment and prior information are not introduced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for removing image artifacts. Background Art

[0002] Cone-Bone Computed Tomography (CBCT) is widely used in clinical examinations of the gums, chest, lower limbs, etc., due to its advantages of low radiation dose, fast scanning speed, simple equipment, and high X-ray utilization. However, compared with the CT (Computed Tomography) system, the detector cone angle used in the CBCT system is larger, and there is a lack of collimators to absorb scattering, which makes it more prone to scattering, motion, or streaks and other related artifacts, resulting in a decrease in image quality, thus affecting doctors' use and diagnosis.

[0003] At present, there are three main types of artifact removal methods: hardware correction method, software correction method, and hardware and software combined correction method. Hardware correction method and hardware and software combined correction method require hardware adjustments, which wastes the advantages of low cost, compactness and convenience of CBCT system. Software correction methods include traditional methods and deep learning methods. Traditional methods rely on prior information and are time-consuming, while deep learning methods can remove artifacts from data. However, conventional convolutional neural networks have problems with image transition smoothing, resulting in blurred images after processing, and generative adversarial networks also have problems such as blurring or loss of details.

[0004] Therefore, how to remove artifacts and improve the clarity of the de-artifacted image without introducing hardware equipment and prior information is a problem that those skilled in the art need to solve. Summary of the invention

[0005] The present application provides a method, device, equipment and storage medium for removing artifacts from an image, so as to remove artifacts without introducing hardware equipment and prior information, thereby improving the clarity of the de-artifacted image.

[0006] In a first aspect, the present application provides a method for removing artifacts from an image, comprising:

[0007] Obtaining a target image to be processed;

[0008] Inputting the target image into a generative adversarial network model, and removing artifacts of the target image through a generator of the generative adversarial network model to obtain an artifact-free image;

[0009] Among them, the generative adversarial network model is generated through training data and comprehensive loss function training, the comprehensive loss function includes an adversarial loss function, and the adversarial loss function is determined by window width and window position.

[0010] Optionally, before acquiring the target image to be processed, the method further includes:

[0011] Acquire training data; the training data includes training data with artifacts and training data without artifacts corresponding to the training data with artifacts;

[0012] Preprocessing the training data to generate a first training image with artifacts and a first training image without artifacts corresponding to the first training image with artifacts;

[0013] The initial generative adversarial network model is trained by using the first artifact-containing training image, the first artifact-free training image and a comprehensive loss function to obtain the generative adversarial network model; wherein the comprehensive loss function includes: an adversarial loss function, an L1 loss function and a perceptual loss function.

[0014] Optionally, training an initial generative adversarial network model by using the first artifact-containing training image, the first artifact-free training image and a comprehensive loss function to obtain the generative adversarial network model includes:

[0015] Inputting the first artifact-containing training image into an initial generator to obtain a first artifact-free output image; the initial generative adversarial network model includes an initial generator and an initial discriminator;

[0016] Using the window width and the window level, respectively process the first artifact-containing training image, the first artifact-free training image, and the first artifact-free output image to obtain processing results;

[0017] Generate a discrimination result through the first artifact-containing training image, the first artifact-free training image, the first artifact-free output image, the processing result and the initial discriminator;

[0018] Determine a comprehensive loss result using the first artifact-free training image, the first artifact-free output image, the discrimination result, and the comprehensive loss function;

[0019] The initial generative adversarial network model is trained according to the comprehensive loss result to obtain the generative adversarial network model.

[0020] Optionally, determining a comprehensive loss result using the first artifact-free training image, the first artifact-free output image, the discrimination result, and the comprehensive loss function includes:

[0021] Determine a first loss result using the discrimination result and the adversarial loss function;

[0022] Determine a second loss result using the first artifact-free training image, the first artifact-free output image, and an L1 loss function;

[0023] Determine a third loss result using the first artifact-free training image, the first artifact-free output image, and the perceptual loss function;

[0024] A comprehensive loss result is determined based on the first loss result, the second loss result and the third loss result.

[0025] Optionally, the first artifact-containing training image, the first artifact-free training image and the first artifact-free output image are processed respectively by using the window width and the window level, and the processing results obtained include:

[0026] Using the window width and the window level, the pixel value range of each pixel point in the first training image with artifacts is adjusted to obtain a second training image with artifacts;

[0027] Using the window width and the window level, adjusting the pixel value range of each pixel point in the first artifact-free training image to obtain a second artifact-free training image;

[0028] The pixel value range of each pixel point in the first artifact-free output image is adjusted by using the window width and the window level to obtain a second artifact-free output image.

[0029] Optionally, generating a discrimination result through the first artifact-containing training image, the first artifact-free training image, the first artifact-free output image, the processing result and the initial discriminator includes:

[0030] Inputting the first artifact-free output image and the first artifact-containing training image into the initial discriminator to obtain a first discrimination result;

[0031] Inputting the second artifact-free output image and the second artifact-containing training image into the initial discriminator to obtain a second discrimination result;

[0032] Inputting the first artifact-free training image and the first artifact-containing training image into the initial discriminator to obtain a third discrimination result;

[0033] The second artifact-free training image and the second artifact-containing training image are input into the initial discriminator to obtain a fourth discrimination result.

[0034] Optionally, preprocessing the training data to generate a first training image with artifacts and a first training image without artifacts corresponding to the first training image with artifacts includes:

[0035] Registering the artifact-free training data to obtain registered artifact-free training data;

[0036] The artifact-bearing training data and the registered artifact-free training data are subjected to effective layer conversion to obtain a first artifact-bearing training image and a first artifact-free training image.

[0037] In a second aspect, the present application provides an image artifact removal device, comprising:

[0038] An image acquisition module, used for acquiring a target image to be processed;

[0039] The artifact removal module is used to input the target image into a generative adversarial network model, and remove the artifacts of the target image through the generator of the generative adversarial network model to obtain an artifact-free image; wherein the generative adversarial network model is generated by training data and a comprehensive loss function, the comprehensive loss function includes an adversarial loss function, and the adversarial loss function is determined by a window width and a window position.

[0040] In a third aspect, the present application provides an electronic device, including:

[0041] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the above-mentioned artifact removal method of the present application through the computer program.

[0042] In a fourth aspect, the present application further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are used to execute the steps of the above-mentioned artifact removal method of the present application.

[0043] The above-mentioned technical scheme provided by the embodiment of the present application has the following advantages over the prior art: the present application provides a scheme for removing artifacts from an image. After obtaining a target image to be processed, the scheme inputs the target image into a generative adversarial network model, and removes artifacts from the target image through the generator of the generative adversarial network model to obtain an artifact-free image; wherein the generative adversarial network model is generated through training data and a comprehensive loss function, and the comprehensive loss function includes an adversarial loss function determined by a window width and a window position. It can be seen that the present application combines the window width and the window position with the generative adversarial network model to enhance the contrast of the image and the clarity of the structure, so that the artifact removal effect is more in line with the subjective visual perception, so that the present scheme can achieve effective removal of artifacts without introducing hardware equipment and prior information. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0046] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0047] Figure 1 A schematic diagram of a method for removing artifacts from an image provided in an embodiment of the present application;

[0048] Figure 2 A schematic flow chart of another method for removing artifacts from an image provided in an embodiment of the present application;

[0049] Figure 3 A schematic flow chart of another method for removing artifacts from an image provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of an initial generative adversarial network model structure provided in an embodiment of the present application;

[0051] Figure 5a An artifact image of the phantom provided in an embodiment of the present application;

[0052] Figure 5b An artifact-free image of the phantom provided in an embodiment of the present application;

[0053] Figure 5c An artifact-free image generated by the generative adversarial network model provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of the structure of an image artifact removal device provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] Artifacts refer to images that do not match the actual tissue morphology, which have a certain impact on subsequent segmentation, registration and other algorithms. Therefore, artifact removal is very critical, and an artifact removal algorithm with accurate effects and good generalization is needed.

[0057] At present, there are three main types of artifact removal methods: hardware correction method, software correction method, and hardware and software combined correction method. Among them, hardware correction methods include air gap method, collimator method, etc. These methods make adjustments on the hardware, wasting the advantages of the CBCT system of low cost, compactness and convenience. Software correction methods are divided into two categories: traditional methods and deep learning methods. Traditional methods include Monte Carlo simulation, but it relies on prior information and is very time-consuming. Non-local mean filtering has a certain effect on noise, but is not suitable for artifacts related to motion, strips, etc., and the parameters are not universal. Deep learning-based methods are currently popular and remove artifacts from the data, no longer relying on device information. Among them, conventional convolutional neural networks have a certain effect on artifact removal, but the generated images have the problem of over-smoothing. Generative adversarial networks can optimize image quality through the game between generators and discriminators, but are still subject to problems such as blur or loss of details. Subjectively, they do not fully meet visual perception and image quality needs to be further improved. Diffusion models are too time-consuming and not conducive to equipment deployment. There are methods such as scatter correction plate method and ray modulation method for hardware and software correction, but hardware equipment also needs to be introduced.

[0058] It can be seen that the existing methods have the problems of needing to introduce hardware equipment, relying on other prior information, being inefficient, having certain limitations in the details of the algorithm effect, and the effect not being consistent with the subjective visual experience when removing artifacts. Therefore, in the embodiments of the present application, a method, device, equipment and storage medium for removing artifacts from an image are disclosed. This solution realizes effective removal of artifacts by combining window width, window position and generative adversarial network. Moreover, this solution does not require the introduction of hardware equipment, nor does it rely on other prior knowledge. This solution is based only on image domain data, and while removing artifacts, it also strengthens important structural information in the image, better retains the details of the image, and makes the artifact removal effect more consistent with the subjective visual experience.

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0060] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0061] See also Figure 1 , is a flow chart of a method for removing artifacts from an image provided by an embodiment of the present application, and the method for removing artifacts specifically comprises the following steps:

[0062] S101, obtaining a target image to be processed;

[0063] S102. Input the target image into a generative adversarial network model, and remove the artifacts of the target image through the generator of the generative adversarial network model to obtain an artifact-free image; wherein the generative adversarial network model is generated through training data and a comprehensive loss function, and the comprehensive loss function includes an adversarial loss function, and the adversarial loss function is determined by a window width and a window position.

[0064] In the present application, the target image is an image whose artifacts are to be removed, and the image may be a CBCT image, or a CT image, an MR (Magnetic Resonance Imaging) image, etc., which is not specifically limited here.

[0065] After determining the target image to be processed, the present application inputs the target image into a generative adversarial network model, removes the artifacts of the target image through the generator of the generative adversarial network model, and obtains an artifact-free image. The generative adversarial network model includes a generator G and a discriminator D. The generator is a U-shaped structure, comprising an encoding layer (Encoder) and a decoding layer (Decoder), the encoding layer is used to implement the downsampling process, and the input image is subjected to different degrees of feature extraction, and the decoding layer is used to implement the upsampling process, so that the low-resolution feature map is restored to the original resolution. Among them, the encoding layer performs downsampling processing on the original resolution image through multiple convolutions, batch normalization, activation functions, and pooling processes to obtain feature maps of different low resolutions; the decoding layer performs feature fusion through multiple convolutions, batch normalization, activation functions, and upsampling, and performs jump connections with the encoding layer to restore the low-resolution feature map to the original resolution, so that the network is end-to-end, that is, the input image and the output image resolution are consistent.

[0066] Among them, the discriminator is the encoding layer (Encoder) of the CNN (Convolutional Neural Networks) structure. The encoding layer of the discriminator divides the original resolution image into n*n "patches" (blocks) through a sliding window scan through multiple convolutions, batch normalization, activation functions and other processes. After passing through the encoding layer, an n*n probability map is finally obtained to determine whether each small block is a "real" or "generated" image, and obtain the discrimination result.

[0067] In the present application, the initial generator and the initial discriminator in the initial generative adversarial network model are trained by training data and a comprehensive loss function, and the generative adversarial network model is generated after training. The trained generative adversarial network model includes a generator and a discriminator. In the training process, the training process is completed by the initial generator, the initial discriminator, the training data and the comprehensive loss function in the initial generative adversarial network model. However, in practical applications, only the generator in the trained generative adversarial network model is needed to obtain an artifact-free image.

[0068] Among them, the training data in this application is artifact training data, and artifact-free training data corresponding to the artifact training data. The comprehensive loss function can include multiple loss functions, such as: L1 loss function, adversarial loss function, etc., which are not specifically limited here. In order to improve the effect of removing artifacts by the generative adversarial network model, the present application adds transformation constraints of window width and window position in the discriminator structure, that is: the discriminator not only generates the discrimination result based on the result output by the generator, the artifact training data and the artifact-free training data, but also needs to generate the discrimination result using the data processed by the window width and window position. Correspondingly, in the adversarial loss function, the discrimination result generated by the window width and window position needs to be used to train the model. The present application combines the window width and window position adjustment with the generative adversarial network to enhance the image contrast and structural clarity, so that the artifact removal effect is more in line with the subjective visual experience.

[0069] When training the model, the present application adopts the gradient descent method to train the initial generative adversarial network model until convergence. Specifically, the network is trained end-to-end using artifact images and their corresponding artifact-free images until the initial generative adversarial network model converges, thereby obtaining a generative adversarial network model that combines window width and window position with a generative adversarial network for artifact removal.

[0070] In summary, the present application does not require the introduction of additional hardware equipment and is only based on data in the image domain, which simplifies data organization work; the generative adversarial network model in the present application can achieve end-to-end artifact removal, avoiding parameter adjustment or reliance on prior information of traditional algorithms; and, the present application combines the window width and window position with the generative adversarial network model, and adds the window width and window position to the adversarial loss function of the generative adversarial network model, which can enhance the contrast of the image and the clarity of the structure, making the artifact removal effect more in line with the subjective visual perception.

[0071] See also Figure 2 , is a flow chart of another method for removing artifacts from an image provided by an embodiment of the present application, the method for removing artifacts specifically comprises the following steps:

[0072] S201, acquiring training data; the training data includes training data with artifacts and training data without artifacts corresponding to the training data with artifacts;

[0073] S202, preprocessing the training data to generate a first training image with artifacts and a first training image without artifacts corresponding to the first training image with artifacts;

[0074] S203, training the initial generative adversarial network model through the first artifact training image, the first artifact-free training image and the comprehensive loss function to obtain the generative adversarial network model; wherein the comprehensive loss function includes: an adversarial loss function, an L1 loss function and a perceptual loss function; the adversarial loss function is determined by the window width and the window position;

[0075] S204, obtaining a target image to be processed;

[0076] S205, inputting the target image into a generative adversarial network model, and removing artifacts of the target image through a generator of the generative adversarial network model to obtain an artifact-free image; wherein the generative adversarial network model is generated through training data and a comprehensive loss function.

[0077] In this application, the target image to be processed is a CBCT image. In order to train the initial generative adversarial network model, the application first needs to obtain training data, which is a CBCT dataset. The dataset may include training data with artifacts such as phantoms, animals, and real objects, as well as corresponding training data without artifacts. Among them, the training data with artifacts may include artifact type data such as scattering, motion, and stripes.

[0078] After the present application obtains the training data, in order to more effectively train the initial generative adversarial network model, the training data needs to be preprocessed. The preprocessing operations performed by the present application on the training data may include: data 3D registration correction operations and data effective layer conversion operations, etc. Here, only the above two operations are used as examples to illustrate the preprocessing process, that is: in some embodiments of the present application, the training data is preprocessed to generate a first artifact training image and a first artifact-free training image corresponding to the first artifact training image, including: registering the artifact-free training data to obtain the registered artifact-free training data; performing effective layer conversion on the artifact training data and the registered artifact-free training data to obtain the first artifact-free training image and the first artifact-free training image.

[0079] In this application, when executing the data 3D registration and correction process, it is necessary to register the artifact-free training data. This application uses the artifact training data as a reference to implement the registration and correction process. For example: This application uses the image in the artifact training data as the reference image A, and the corresponding image in the artifact-free training data as the image to be registered B. This application needs to adjust the transformation parameters of the image B to be registered so that the NMI (Normalized Mutual Information) of the reference image A and the image B to be registered reaches the maximum value, and obtains the registered image B′, indicating that the statistical dependence of the two images is the strongest. The calculation formula of NMI is:

[0080]

[0081] Where H(A) is the entropy of image A, which is the amount of information of a single image. H(B) is the entropy of image B, which is the amount of information of a single image; H(A,B) is the joint entropy of images A and B, which is the joint information.

[0082] In the present application, after obtaining the aligned artifact-free training data, it is necessary to convert the artifact training data and the aligned artifact-free training data into a first artifact training image and a first artifact-free training image through a data valid layer conversion operation; wherein, the data valid layer conversion operation refers to selecting valid layers on both sides of the three-dimensional image A and the three-dimensional image B′, and converting the valid layers into a series of two-dimensional images to obtain a first artifact training image and a first artifact-free training image, and the first artifact training image and the first artifact-free training image are one-to-one corresponding and equal in number. For example, if the three-dimensional image A has 0-100 layers and the three-dimensional image B′ has 0-100 layers, but the data of layer 0 of the three-dimensional image B′ is blank data, then layer 0 of the three-dimensional image B′ is an invalid layer. After removing the invalid layer of the three-dimensional image B′, the data of layers 1-100 are obtained. At this time, in order to ensure the one-to-one correspondence between the first training image with artifacts and the first training image without artifacts, the layer 0 of the three-dimensional image A needs to be removed as well. That is, the converted three-dimensional image A includes the first training image with artifacts corresponding to layers 1-100, and the converted three-dimensional image B′ includes the first training image without artifacts corresponding to layers 1-100.

[0083] After obtaining the first training image with artifacts and the first training image without artifacts, the application trains the initial generative adversarial network model through the first training image with artifacts, the first training image without artifacts and the comprehensive loss function to obtain the generative adversarial network model; the comprehensive loss function in the application may include: adversarial loss function, L1 loss function and perceptual loss function. Among them, the adversarial loss function introduces window width and window position, which can enhance the contrast of the image and the clarity of the structure, while the L1 loss function and the perceptual loss function can better retain the structure and details and improve the quality of the generated image.

[0084] In summary, after obtaining the training data, the present application can perform preprocessing operations on the training data to more effectively train the initial generative adversarial network model; and, when training the initial generative adversarial network model, the present application uses the L1 loss function and the perceptual loss function in the generator structure. The combination of the L1 loss function and the perceptual loss function can better retain the structure and details and improve the quality of the generated image; the adversarial loss function is used in the discriminator structure. The adversarial loss function can enhance the contrast of the image and the clarity of the structure by introducing window width and window position, so that the artifact removal effect is more in line with the subjective visual perception.

[0085] See also Figure 3 , is a flow chart of another method for removing artifacts from an image provided by an embodiment of the present application, the method for removing artifacts specifically comprises the following steps:

[0086] S301, acquiring training data; the training data includes training data with artifacts and training data without artifacts corresponding to the training data with artifacts;

[0087] S302, preprocessing the training data to generate a first training image with artifacts and a first training image without artifacts corresponding to the first training image with artifacts;

[0088] S303, inputting the first artifact-containing training image into an initial generator to obtain a first artifact-free output image; the initial generative adversarial network model includes an initial generator and an initial discriminator;

[0089] S304, using the window width and the window level, respectively process the first artifact-containing training image, the first artifact-free training image, and the first artifact-free output image to obtain processing results;

[0090] S305, generating a discrimination result through the first artifact-containing training image, the first artifact-free training image, the first artifact-free output image, the processing result and the initial discriminator;

[0091] S306, using the first artifact-free training image, the first artifact-free output image, the discrimination result, and the comprehensive loss function to determine a comprehensive loss result; wherein the comprehensive loss function includes: an adversarial loss function, an L1 loss function, and a perceptual loss function; the adversarial loss function is determined by a window width and a window position;

[0092] S307, training the initial generative adversarial network model according to the comprehensive loss result to obtain a generative adversarial network model;

[0093] S308, obtaining a target image to be processed;

[0094] S309: Input the target image into the generative adversarial network model, and remove the artifacts of the target image through the generator of the generative adversarial network model to obtain an artifact-free image.

[0095] See also Figure 4 , is a schematic diagram of the structure of an initial generative adversarial network model disclosed in an embodiment of the present application, the initial generative adversarial network model includes an initial generator G and an initial discriminator D, wherein x is a first training image with artifacts, z is Gaussian noise, the first training image with artifacts x and Gaussian noise z are input into the initial generator G, and a first output image without artifacts G(x,z) is obtained. Then, the first training image with artifacts, the first training image without artifacts, and the first output image without artifacts are processed respectively using the window width and window level to obtain the processing results.

[0096] In one embodiment of the present application, the first artifact-containing training image, the first artifact-free training image and the first artifact-free output image are processed respectively using window width and window position, and the process of obtaining the processing result specifically includes: using the window width and window position to adjust the pixel value range of each pixel in the first artifact-containing training image to obtain the second artifact-free training image; using the window width and window position to adjust the pixel value range of each pixel in the first artifact-free training image to obtain the second artifact-free training image; using the window width and window position to adjust the pixel value range of each pixel in the first artifact-free output image to obtain the second artifact-free output image.

[0097] In this application, the window width is WW (window width), the window level is WL (window level), the window width and window level are parameters in the window technology, the window width refers to the gray value range selected in the image display, and the window level refers to the center point of the gray value range. Since the pixel range of the image is relatively large, in this application, the loss calculation after the window width and window level are additionally added to the loss function can be optimized in this way. The display and contrast of the image can be optimized, so that the important information in the image is more prominent. This method can change the visual presentation of the image, so that the result is more in line with the subjective visual effect of the human eye. This method can make the area of ​​interest easier to analyze, and can highlight the display of artifacts or tissues of interest in a targeted manner. This application adjusts the pixel value range of the pixel points in the image through the window width and window level function W (·), and then obtains each processed image, wherein the calculation formula of W (·) is:

[0098]

[0099] Among them, WW represents the window width, WL represents the window position, p represents the original pixel value of a pixel point in the original image, and p′ represents the pixel value of a pixel point after the window width and window position are adjusted. It can be seen from the window width and window position function that in the present application, if the pixel value of a pixel point in the image is not within the predetermined range, it needs to be modified to a value re-determined by the window width and window position. If the pixel value is within the predetermined range, the pixel value does not need to be modified. In this way, the pixel value within the predetermined range can be displayed more prominently, so that the artifact removal result is more in line with the subjective visual effect of the human eye; wherein the original image is any one of the first artifact training image x, the first artifact-free training image y and the first artifact-free output image G(x,z). After the original image is processed using the window width and window position, the processing results obtained include: a second artifact training image W(x), a second artifact-free training image W(y), and a second artifact-free output image W(G(x,z)).

[0100] Furthermore, after obtaining the processing result, the present application needs to combine the first artifact training image x, the first artifact-free training image y, and the first artifact-free output image G(x,z), input them into the initial discriminator, and generate a discrimination result. In one embodiment of the present application, the process of generating the discrimination result through the first artifact training image, the first artifact-free training image, the first artifact-free output image, the processing result, and the initial discriminator specifically includes: inputting the first artifact-free output image and the first artifact training image into the initial discriminator to obtain the first discrimination result; inputting the second artifact-free output image and the second artifact training image into the initial discriminator to obtain the second discrimination result; inputting the first artifact-free training image and the first artifact training image into the initial discriminator to obtain the third discrimination result; inputting the second artifact-free training image and the second artifact training image into the initial discriminator to obtain the fourth discrimination result.

[0101] See also Figure 4 , the present application obtains four discrimination results through the initial discriminator D, wherein the first artifact-free output image G(x,z) and the first artifact-containing training image x are input into the initial discriminator D to obtain a first discrimination result D(x,G(x,z)), which is used to represent: the discrimination result of the initial discriminator D on the real artifact-containing image and the generated artifact-free image; the second artifact-free output image W(G(x,z)) and the second artifact-containing training image W(x) are input into the initial discriminator D to obtain a second discrimination result D(W(x),W(G(x,z))), which is used to represent: the discrimination result of the initial discriminator D on the real artifact-containing image and the generated artifact-free image The first artifact-free training image x and the first artifact-containing training image y are input into the initial discriminator D to obtain a third discrimination result D(x, y), which is used to represent: the discrimination result of the initial discriminator D on the real artifact-containing image and the real artifact-free image; the second artifact-free training image W(y) and the second artifact-containing training image W(x) are input into the initial discriminator D to obtain a fourth discrimination result D(W(x), W(y)), which is used to represent: the discrimination result of the initial discriminator D on the real artifact-containing image window width and window position and the real artifact-free image window width and window position.

[0102] After obtaining the above four discrimination results, the present application will determine the comprehensive loss result in combination with the first artifact-free training image, the first artifact-free output image and the comprehensive loss function. In one embodiment of the present application, the process of determining the comprehensive loss result includes: using the discrimination result and the adversarial loss function to determine the first loss result; using the first artifact-free training image, the first artifact-free output image and the L1 loss function to determine the second loss result; using the first artifact-free training image, the first artifact-free output image and the perceptual loss function to determine the third loss result; determining the comprehensive loss result based on the first loss result, the second loss result and the third loss result.

[0103] In this application, the comprehensive loss function is:

[0104]

[0105] Among them, G represents the generator, D represents the discriminator, and L cGAN is the adversarial loss function, L1 is the L1 loss function, L perceptual is the perceptual loss function, α, β, and γ represent the weight coefficients of the adversarial loss function, L1 loss function, and perceptual loss function, respectively.

[0106] In this application, the adversarial loss function is:

[0107] L cGAN (G,D)=E x,y [logD(x,y)]+E x,z [log(1-D(x,G(x,z)))]+E x,y [logD(W(x),W(y))]+E x,z [log(1-D(W(x),W(G(x,z))))]

[0108] Among them, L cGAN (G, D) represents the first loss result of the calculation, E represents the expected value, and log is the logarithmic operation symbol.

[0109] In this application, the L1 loss function is:

[0110] L1(G)=E x,y,z ||yG(x,z)||1

[0111] Wherein, L1(G) is the calculated second loss result.

[0112] In this application, the perceptual loss function is

[0113]

[0114] Among them, L perceptual (G) is the calculated third loss result, Fi It represents the i-th layer feature mapping result based on the VGG16 or VGG19 network, N represents the set network layer number parameter, VGG (Visual Geometry Group) is a convolutional neural network architecture, and this application inputs the first artifact-free training image y and the first artifact-free output image G(x,z) into the VGG network, and then determines the third loss result according to the output result.

[0115] During training, the present application uses the initial generative adversarial network model to perform forward propagation calculations on the first artifact-containing training image to obtain the first artifact-free output image, and uses the gradient descent method, using the comprehensive loss function, the discrimination results generated based on the window width and window position, and other data to perform end-to-end training on the network until the initial generative adversarial network model converges, such as: the calculated comprehensive loss result is less than a predetermined threshold, or does not change. After the present application obtains the trained generative adversarial network model, the target image to be processed can be input into the generative adversarial network model, and the forward propagation calculation is performed through the generative adversarial network model to obtain an artifact-free image.

[0116] See also Figure 5a , is an artifact image of the phantom disclosed in the embodiment of the present application, see Figure 5b , is an artifact-free image of the phantom disclosed in the embodiment of the present application, see Figure 5c , which is an artifact-free image generated by the generative adversarial network model disclosed in the embodiment of the present application. It can be seen that Figure 5c The artifact-free image generated by the generative adversarial network model is Figure 5b The artifact-free images in are very similar, so the generative adversarial network model can effectively remove artifacts and improve the clarity of the image.

[0117] In summary, this application uses the L1 loss function and the perceptual loss function in the generator structure when training the initial generative adversarial network model. The combination of the L1 loss function and the perceptual loss function can better retain the structure and details and improve the quality of the generated image; the adversarial loss function is used in the discriminator structure. The adversarial loss function introduces window width and window position. This method can enhance the contrast of the image and the clarity of the structure, making the artifact removal effect more in line with the subjective visual perception.

[0118] See also Figure 6 , Figure 6 A schematic diagram of the structure of an image artifact removal device provided in an embodiment of the present application, the device specifically comprises:

[0119] An image acquisition module 11 is used to acquire a target image to be processed;

[0120] The artifact removal module 12 is used to input the target image into the generative adversarial network model, and remove the artifacts of the target image through the generator of the generative adversarial network model to obtain an artifact-free image; wherein the generative adversarial network model is generated by training data and a comprehensive loss function, and the comprehensive loss function includes an adversarial loss function, and the adversarial loss function is determined by window width and window position.

[0121] As an optional embodiment, the artifact removal device further includes:

[0122] A training data acquisition module, used to acquire training data; the training data includes training data with artifacts and training data without artifacts corresponding to the training data with artifacts;

[0123] A preprocessing module, used for preprocessing the training data to generate a first training image with artifacts and a first training image without artifacts corresponding to the first training image with artifacts;

[0124] A training module is used to train the initial generative adversarial network model through the first artifact training image, the first artifact-free training image and a comprehensive loss function to obtain the generative adversarial network model; wherein the comprehensive loss function includes: an adversarial loss function, an L1 loss function and a perceptual loss function.

[0125] As an optional embodiment, the training module includes:

[0126] An acquisition unit, configured to input the first artifact-containing training image into an initial generator to obtain a first artifact-free output image; the initial generative adversarial network model includes an initial generator and an initial discriminator;

[0127] a processing unit, configured to process the first artifact-bearing training image, the first artifact-free training image, and the first artifact-free output image respectively by using a window width and a window level to obtain a processing result;

[0128] A discriminant unit, configured to generate a discriminant result through the first artifact-containing training image, the first artifact-free training image, the first artifact-free output image, the processing result and the initial discriminator;

[0129] a determining unit, configured to determine a comprehensive loss result by using the first artifact-free training image, the first artifact-free output image, the discrimination result, and the comprehensive loss function;

[0130] A training unit is used to train the initial generative adversarial network model according to the comprehensive loss result to obtain the generative adversarial network model.

[0131] As an optional embodiment, the determining unit includes:

[0132] A first determination subunit, configured to determine a first loss result by using the discrimination result and the adversarial loss function;

[0133] A second determination subunit is used to determine a second loss result by using the first artifact-free training image, the first artifact-free output image and the L1 loss function;

[0134] A third determination subunit is used to determine a third loss result by using the first artifact-free training image, the first artifact-free output image and the perceptual loss function;

[0135] The fourth determining subunit is used to determine a comprehensive loss result according to the first loss result, the second loss result and the third loss result.

[0136] As an optional embodiment, the processing unit includes:

[0137] A first processing subunit is used to adjust the pixel value range of each pixel point in the first training image with artifacts by using a window width and a window level to obtain a second training image with artifacts;

[0138] A second processing subunit is used to adjust the pixel value range of each pixel point in the first artifact-free training image by using the window width and the window level to obtain a second artifact-free training image;

[0139] The third processing subunit is used to adjust the pixel value range of each pixel point in the first artifact-free output image by using the window width and the window level to obtain the second artifact-free output image.

[0140] As an optional embodiment, the determination unit includes:

[0141] A first discriminant subunit, configured to input the first artifact-free output image and the first artifact-containing training image into the initial discriminator to obtain a first discriminant result;

[0142] A second discriminant subunit, configured to input the second artifact-free output image and the second artifact-containing training image into the initial discriminator to obtain a second discriminant result;

[0143] A third discriminant subunit, configured to input the first artifact-free training image and the first artifact-containing training image into the initial discriminator to obtain a third discriminant result;

[0144] The fourth discriminant subunit is used to input the second artifact-free training image and the second artifact-containing training image into the initial discriminator to obtain a fourth discriminant result.

[0145] As an optional embodiment, the preprocessing module includes:

[0146] A registration unit, used for registering the artifact-free training data to obtain registered artifact-free training data;

[0147] The conversion unit is used to perform effective layer conversion on the artifact-containing training data and the registered artifact-free training data to obtain a first artifact-containing training image and a first artifact-free training image.

[0148] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0149] See also Figure 7 , Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, the electronic device specifically includes:

[0150] A processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. The processor 21 executes the steps of the artifact removal method described in any of the above method embodiments through the computer program.

[0151] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0152] The memory 22 may include one or more computer-readable storage media, which may be non-transitory. The memory 22 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 22 is at least used to store the following computer program 221, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps in the artifact removal method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 22 may also include an operating system 222 and data 223, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 222 may include Windows, Unix, Linux, etc.

[0153] In some embodiments, the electronic device may further include a display screen 23 , an input / output interface 24 , a communication interface 25 , a sensor 26 , a power source 27 , and a communication bus 28 .

[0154] certainly, Figure 7 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of the present application. In actual applications, the electronic device may include Figure 7 More or fewer components than shown, or combinations of certain components.

[0155] In another exemplary embodiment, a computer storage medium is also provided, and when the program instructions are executed by a processor, the steps of the artifact removal method described in any of the above method embodiments are implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0156] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0157] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0158] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for removing image artifacts, characterized in that: include: Obtaining a target image to be processed; Inputting the target image into a generative adversarial network model, and removing artifacts of the target image through a generator of the generative adversarial network model to obtain an artifact-free image; Among them, the generative adversarial network model is generated through training data and comprehensive loss function training, the comprehensive loss function includes an adversarial loss function, and the adversarial loss function is determined by window width and window position.

2. The artifact removal method according to claim 1, characterized in that: Before obtaining the target image to be processed, the method further includes: Acquire training data; the training data includes training data with artifacts and training data without artifacts corresponding to the training data with artifacts; Preprocessing the training data to generate a first training image with artifacts and a first training image without artifacts corresponding to the first training image with artifacts; The initial generative adversarial network model is trained by using the first artifact-containing training image, the first artifact-free training image and a comprehensive loss function to obtain the generative adversarial network model; wherein the comprehensive loss function includes: an adversarial loss function, an L1 loss function and a perceptual loss function.

3. The artifact removal method according to claim 2, characterized in that: The initial generative adversarial network model is trained by using the first artifact-containing training image, the first artifact-free training image and a comprehensive loss function to obtain the generative adversarial network model, including: Inputting the first artifact-containing training image into an initial generator to obtain a first artifact-free output image; the initial generative adversarial network model includes an initial generator and an initial discriminator; Using the window width and the window level, respectively process the first artifact-containing training image, the first artifact-free training image, and the first artifact-free output image to obtain processing results; Generate a discrimination result through the first artifact-containing training image, the first artifact-free training image, the first artifact-free output image, the processing result and the initial discriminator; Determine a comprehensive loss result using the first artifact-free training image, the first artifact-free output image, the discrimination result, and the comprehensive loss function; The initial generative adversarial network model is trained according to the comprehensive loss result to obtain the generative adversarial network model.

4. The artifact removal method according to claim 3, characterized in that: Determining a comprehensive loss result using the first artifact-free training image, the first artifact-free output image, the discrimination result, and the comprehensive loss function includes: Determine a first loss result using the discrimination result and the adversarial loss function; Determine a second loss result using the first artifact-free training image, the first artifact-free output image, and an L1 loss function; Determine a third loss result using the first artifact-free training image, the first artifact-free output image, and the perceptual loss function; A comprehensive loss result is determined based on the first loss result, the second loss result and the third loss result.

5. The artifact removal method according to claim 3, characterized in that: The first artifact-containing training image, the first artifact-free training image, and the first artifact-free output image are processed respectively by using the window width and the window level, and the processing results obtained include: Using the window width and the window level, the pixel value range of each pixel point in the first training image with artifacts is adjusted to obtain a second training image with artifacts; Using the window width and the window level, adjusting the pixel value range of each pixel point in the first artifact-free training image to obtain a second artifact-free training image; The pixel value range of each pixel point in the first artifact-free output image is adjusted by using the window width and the window level to obtain a second artifact-free output image.

6. The artifact removal method according to claim 5, characterized in that: Generating a discrimination result by using the first artifact-containing training image, the first artifact-free training image, the first artifact-free output image, the processing result and the initial discriminator, including: Inputting the first artifact-free output image and the first artifact-containing training image into the initial discriminator to obtain a first discrimination result; Inputting the second artifact-free output image and the second artifact-containing training image into the initial discriminator to obtain a second discrimination result; Inputting the first artifact-free training image and the first artifact-containing training image into the initial discriminator to obtain a third discrimination result; The second artifact-free training image and the second artifact-containing training image are input into the initial discriminator to obtain a fourth discrimination result.

7. The artifact removal method according to claim 2, characterized in that: Preprocessing the training data to generate a first training image with artifacts and a first training image without artifacts corresponding to the first training image with artifacts includes: Registering the artifact-free training data to obtain registered artifact-free training data; The artifact-bearing training data and the registered artifact-free training data are subjected to effective layer conversion to obtain a first artifact-bearing training image and a first artifact-free training image.

8. An image artifact removal device, characterized in that: include: An image acquisition module, used for acquiring a target image to be processed; The artifact removal module is used to input the target image into a generative adversarial network model, and remove the artifacts of the target image through the generator of the generative adversarial network model to obtain an artifact-free image; wherein the generative adversarial network model is generated by training data and a comprehensive loss function, the comprehensive loss function includes an adversarial loss function, and the adversarial loss function is determined by a window width and a window position.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the artifact removal method described in any one of claims 1 to 7 of the present application through the computer program.

10. A computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, and the computer executable instructions are used to execute the steps of the artifact removal method described in any one of claims 1 to 7 of the present application.

Images