A weight-sharing dual-region generative adversarial network and its image generation method

Abstract

The present invention discloses a dual-region generative adversarial network with shared weights and an image generation method thereof. The generative adversarial network includes an artifact-free region processing module and an artifact-containing region processing module, wherein the artifact-free region processing module includes a first feature generator, a first feature parser and a first discriminator; the artifact-containing region processing module includes a second feature generator, an artifact generator, a reconstruction parser, a second feature parser and a second discriminator. The present invention can improve the sharing of image features of the generator and the parser by repeatedly using the same generator and parser for training, and the use of two regions to train the model can be both supervised and unsupervised. The present invention can eliminate motion artifacts of medical images, and while generating image features, improve the image peak signal-to-noise ratio and structural similarity, thereby obtaining clearer medical images that meet diagnostic requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more specifically, to a weight-sharing dual-region generative adversarial network and an image generation method thereof. Background Art

[0002] Artifact removal in medical images has always been a difficult problem that researchers have been committed to solving. Taking medical image coronary computed tomography angiography (CCTA) as an example, it refers to a non-invasive imaging detection method that uses computers and X-rays to obtain cross-sectional images of the patient's heart after the subject is injected with appropriate contrast agents intravenously. CCTA has the advantages of short scanning time, extensive component information, and non-invasive visualization of vascular walls. It is suitable for the diagnosis of suspected coronary heart disease, follow-up of coronary artery bypass grafting, evaluation of valvular heart disease, and evaluation of heart quality. However, images obtained by CCTA may have motion artifacts and need to be re-examined. A large amount of X-ray exposure will have a cumulative effect of radiation dose, increase the possibility of various diseases, and then affect the physiological functions of the human body, damage human tissues and organs, and even endanger the life safety of patients. Therefore, research and development of motion artifact removal under artifact-producing images has important scientific significance and broad application prospects for the current medical diagnosis field.

[0003] The formation of motion artifacts during coronary CT imaging is due to the displacement of image pixels when CT acquires projection data at different angles. The degree of motion artifacts depends on the displacement rate and the correction result of the image reconstruction algorithm. Motion artifacts are usually eliminated from two aspects: the first is to control the heart rate, reduce the heart rate of the subject, prolong the cardiac cycle, slow down the movement of the coronary arteries, and prolong the time of low-speed movement of the coronary arteries, thereby reducing the need for temporal resolution during imaging; the second is to improve the temporal resolution. Based on the basic principles of MDCT coronary artery imaging and the causes of motion artifacts, to achieve CCTA imaging and avoid motion artifacts, it is necessary to match the highest temporal resolution obtained under a specific scanning method with the phase with the smallest coronary artery movement amplitude.

[0004] In the prior art, the solution to improve the temporal resolution is usually carried out from the hardware or software aspects. Specifically, in terms of hardware, the temporal resolution is improved by increasing the tube-ball rotation speed, using wide-body detectors, and using dual-detector technology. In terms of software, the multi-sector reconstruction technology, image reconstruction technology based on compressed sensing (Prior Image Constrained Compressed Sensing, PICCS), motion estimation and compensation algorithms, and motion correction technology (SnapShot Freeze, SSF) can effectively improve the temporal resolution.

[0005] At present, for the scheme of improving time resolution from the hardware aspect, increasing the tube-ball rotation speed is limited by physical characteristics; using multi-detector technology is limited by space; and using wide-body detector technology is limited by economic costs. For the scheme of improving time resolution from the software aspect, the use of multi-sector reconstruction technology requires maintaining the patient's heart rate stable and is limited by the tube-ball rotation time and scanning pitch; image reconstruction technology based on compressed sensing has not been verified; motion estimation and compensation algorithms rely on a lot of calculations and evaluations; motion correction technology requires collecting the phase with relatively small motion artifacts and good image quality in the image, and motion artifacts can only be eliminated after complex calculations. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a novel generative adversarial network and image generation method thereof, which utilizes a generative adversarial network with two-region multi-generators based on shared weights to eliminate motion artifacts in medical images, and while removing artifacts and generating image features, improves the image peak signal-to-noise ratio and structural similarity, thereby obtaining clearer medical images that meet diagnostic needs.

[0007] According to a first aspect of the present invention, a dual-region generative adversarial network with shared weights is provided, which comprises an artifact-free region processing module and an artifact-containing region processing module, wherein:

[0008] The artifact-free region processing module includes a first generator and a first discriminator, the first generator includes a first feature generator and a first feature parser, the first feature generator is used to extract image features of the artifact-free image, the first feature parser is used to reparse the feature image generated by the second feature generator with the artifact region into an artifact-free image, and the first discriminator is used to calculate the adversarial loss of the artifact-free region;

[0009] The artifact area processing module includes a second generator and a second discriminator, wherein the second generator includes a second feature generator, an artifact generator, a reconstruction parser and a second feature parser, the second feature generator is used to extract image features of the artifact image; the artifact generator is used to extract artifact features of the artifact image; the reconstruction parser is used to add the feature images generated by the second feature generator and the artifact generator, and the feature image obtained after parsing is constrained with the artifact image; the second feature parser is used to add the feature image of the artifact-free area and the artifact feature image of the artifact area to generate an artifact image with the artifact-free image as the base, and the second discriminator is used to calculate the adversarial loss of the artifact area.

[0010] According to a second aspect of the present invention, there is provided an image generating method, comprising:

[0011] Training the weight-sharing dual-region generative adversarial network provided by the present invention with a set joint loss function as a target;

[0012] The medical image to be processed is divided into image blocks containing artifacts and image blocks without artifacts, which are used as inputs of the artifact area processing module and the artifact-free area processing module respectively to obtain a generated image.

[0013] Compared with the prior art, the advantages of the present invention are that, by designing a two-region multi-generator generative adversarial network, motion artifacts in medical images can be effectively eliminated, and in terms of hardware, it is not limited by physical properties, space and economic costs; in terms of software, it does not require complex calculations and is not affected by changes in the patient's heart rate to obtain an image with artifacts eliminated.

[0014] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0016] Figure 1 is a structural diagram of a feature generator according to an embodiment of the present invention;

[0017] Figure 2 is a structural diagram of a feature analyzer according to an embodiment of the present invention;

[0018] Figure 3 is a structural diagram of a residual module according to an embodiment of the present invention;

[0019] Figure 4 is a structural diagram of a discriminator according to an embodiment of the present invention;

[0020] Figure 5 is a structural diagram of an artifact generator according to an embodiment of the present invention;

[0021] Figure 6 is a structural diagram of a content identifier according to an embodiment of the present invention;

[0022] Figure 7 is a schematic diagram of a training process of a content identifier according to an embodiment of the present invention;

[0023] Figure 8 is a schematic diagram of a discriminator training process for an artifact area and an artifact-free area according to an embodiment of the present invention;

[0024] Fig. 9is a schematic diagram of a training process of an artifact region and an artifact-free region generator according to an embodiment of the present invention;

[0025] Fig.10 is a comparison chart of experimental results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0027] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0028] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0029] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0030] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0031] In order to solve the problem of motion artifacts in medical images, the present invention designs a generative adversarial network with shared weight training. By repeatedly using the same generator and parser for training, the sharing of image features of the generator and the parser can be significantly improved, thereby improving the learning efficiency of the generator and the parser for the same features, and cross-generating image features from artifact areas and artifact-free areas, thereby improving the detail expression of the generated image, thereby effectively eliminating the motion artifacts of the image during the image generation process.

[0032] For the sake of clarity, CCTA images are used as an example for illustration herein, but it should be understood that, in addition to being applied to CCTA image artifact removal, the present invention can also be applied to other types of medical image denoising and artifact removal.

[0033] The shared weight dual-region generative adversarial network provided by the present invention as a whole includes an artifact-free region processing module and an artifact-containing region processing module, and the specific design process and operation steps are as follows.

[0034] Step S110: Designing an artifact-free region processing module.

[0035] In one embodiment, the artifact-free region processing module includes a generator Gen b and the discriminator Dis b The generator further contains the feature generator Encoder cont_b and feature parser Decoder cont_b The feature generator is used to extract the features of the artifact-free image, and the feature parser reparses the feature image generated by the first feature generator in the artifact area into an artifact-free image. The discriminator is used to calculate the adversarial loss of the artifact-free area.

[0036] Feature generators such as Figure 1 As shown in the figure, an artifact-free image of size 1×H×W is input, and the generator is trained to obtain the feature image of the artifact-free image. In order to obtain the feature image of the artifact-free image, a convolution operation is first performed on the input image, and a 7×7 convolution operation is used to obtain a l×H×W feature image. Multiple channels are added to the feature map, and multiple channels learn the features of the image from different angles. Then, the feature image is downsampled through two 4×4 convolutions to obtain a reduced feature image. Next, after a 4-layer residual block (RES), the low-dimensional and high-dimensional feature information of the feature image is learned to obtain The feature image is the final output of the feature generator. Figure 1 In , each convolutional layer is followed by a nonlinear activation function called leaky ReLU.

[0037] Feature parser such as Figure 2 As shown, the input size is The feature image is taken from the residual module, and the deep and shallow layers of information are learned. The feature image is restored to the size of the original image l×H×W through two upsampling (Up sample) and a convolution layer with a convolution kernel of 5×5. Then, after a convolution layer with a convolution kernel of 7×7, the feature image is restored to an artifact-free image with the same size as the original (1×H×W). Except for the last convolution layer, the activation function leakyReLU is included after the remaining convolution layers. The process of the feature parser is equivalent to the inverse process of the feature generator.

[0038] Figure 3 yes Figure 1 and Figure 2 The structure of the residual module in Figure 1 shows that, taking the input l×H×W image as an example, the feature image is obtained through two layers of 3×3 convolution, where the first layer of convolution is followed by a nonlinear activation function, leaky ReLU, and the feature image obtained by the second layer of convolution is added to the input image to obtain the final output of the residual module. This method of using multiple layers of residual modules can learn both deep and shallow information of the feature image.

[0039] Figure 4This is an example of the structure of the discriminator. The input image is 1×H×W, and it passes through 4 convolution layers with a convolution kernel of 3×3 and a stride of 2, and then passes through a convolution layer with a convolution kernel of 1×1 and a stride of 1. The feature image of the target image. During the training process, the least squares loss function can be used as a constraint. During the update process of the generator, the least squares loss function of the generated image itself is used. During the update process of the discriminator, the least squares loss function of the generated image and the target image is used.

[0040] Step S120: designing an artifact area processing module.

[0041] In one embodiment, the artifact region processing module includes a generator Gen a and the discriminator Dis a The generator further contains the feature generator Encoder cont_a , Artifact Generator Encoder art_a , Reconstruct the parser Decoder recons_a and feature parser Decoder cont_a The feature generator is used to extract image features of the artifact image; the artifact generator is used to extract artifact features of the artifact image; the reconstruction parser is used to add the feature images generated by the feature generator and the artifact generator, and the feature image obtained after parsing and the artifact image are used as constraints; the feature parser is used to add the feature image of the artifact-free area and the artifact feature image of the artifact area to generate an artifact image with the artifact-free image as the base. The discriminator is used to calculate the adversarial loss of the artifact area.

[0042] The structures of the feature generator, reconstruction parser and feature parser included in the artifact area processing module can also be respectively adopted Figure 1 , Figure 2 and Figure 2 .

[0043] Figure 5 This is an example of the structure of the artifact generator. After a convolution layer with a convolution kernel of 7×7, it passes through two convolution layers with a convolution kernel of 4×4 and a stride of 2, which is also downsampling. The size of the feature image obtained by downsampling should be consistent with that of the feature generator and the feature generator of the artifact-free area. Finally, it passes through a convolution layer with a convolution kernel of 1×1 to obtain the feature image of the artifact.

[0044] In addition, the size of the feature images obtained by upsampling the reconstruction parser and feature parser in the artifact area and the feature parser in the artifact-free area should be consistent.

[0045] Step S130, designing a content identifier.

[0046] Preferably, the designed dual-region generative adversarial network may further include a content discriminator or a style discriminator.

[0047] Content Identifier Dis cont The features used to extract image style can be implemented through traditional convolutional neural networks, such as Figure 6 As shown in the figure, it includes 5 convolution layers with a convolution kernel of 3×3 and a stride of 2, an average pooling layer (Avg pool), and finally a convolution layer with a convolution kernel of 1×1 to obtain the feature image of the image style. The content discriminator is used to calculate the adversarial loss for the feature generators of the artifact-free area and the artifact area generator. Adding the content discriminator plays an important role in maintaining the training stability of the feature generators of the two areas, which is conducive to improving the universality of the dual-area generative adversarial network for images with different contents and styles.

[0048] Step S140, training a dual-region generative adversarial network with the set joint loss function as a target.

[0049] In short, the overall training process includes: step S141, training the content discriminator; step S142, training the discriminator of the artifact and artifact-free areas; step S143, training the generator of the artifact and artifact-free areas.

[0050] All the Encoders, Decoders and Dis below correspond to the feature images processed by the corresponding subscripted generators, parsers and discriminators to obtain the processed feature images.

[0051] Specifically, in step S141, for the content identifier, in the training process of updating its own weight, as Figure 7 , the loss value is calculated using the least squares loss function, where the feature image label generated by the input image is 0 and the feature image label generated by the target image is 1. The formula is as follows:

[0052]

[0053] Where x represents the input image, y represents the target image, and 0 and 1 represent the labels of the artifact area and the artifact-free area, respectively.

[0054] In step S142, for the discriminator of the artifact area and the artifact-free area, the training process of updating its own weight is as follows: Figure 8 As shown, for the artifact-free area processing module, the target image is input and passed through the Encoder cont_b and Decoder cont_b , get the output image b, input the target image and image b into the discriminator Dis b , get the loss value lossdis_b ; For the artifact area processing module, the input image is input and passed through the feature generator Encoder cont_a and artifact generator Encoder art_a , get two feature images, add the two feature images, and then reconstruct the parser Decoder recons_a , get the output image a, input the input image and image a to the discriminator Dis a In the calculation, the loss value loss is obtained dis_a ; At the same time, the feature generator Encoder of the artifact-free area cont_b And the feature generator Encoder of the artifact area cont_a The obtained feature image is input to the content discriminator Dis cont , the feature images out_a and out_b are calculated.

[0055] In one embodiment, the loss functions of each discriminator in the training process are:

[0056] The loss calculation formula of the content discriminator is expressed as:

[0057]

[0058] It can be seen from formula (2) that in the process of training the discriminators of the two regions, in order to make the content discriminator play the technical effect of stabilizing the feature generators of the two regions so that the features generated by it are all based on image features, there is no distinction between real images and generated images during the training process, so the label is set to 0.5.

[0059] The loss calculation formula of the artifact-free region discriminator is expressed as:

[0060]

[0061] The loss calculation formula of the artifact area discriminator is expressed as:

[0062]

[0063] Combined with the hyperparameter calculation, the overall loss value calculation formula is expressed as:

[0064] loss total =GAN_weight×(loss dis_cont +loss dis_b +loss dis_a ) (5)

[0065] Among them, GAN_weight represents the set hyperparameters.

[0066] In step S143, for the generators with artifacts and without artifacts, the training process of updating their own weights is as follows: Fig. 9 As shown, for the input image (marked as x) and the target image (marked as y), the generator and discriminator of the entire framework are updated to complete the overall training process.

[0067] Specifically, the weight training of the entire framework is Fig. 9 As shown, it also includes the contents of step S141 and step S142. Therefore, the joint loss function includes the following contents:

[0068] Input image x passes through the feature generator Encoder in the artifact area cont_a and artifact generator Encoder art_a , and obtain the feature images x c and x a , x c and x a Add, by reconstructing the parser Decoder recons_a Obtain the reconstructed image and calculate the reconstruction loss of the reconstructed image and the input image. The formula is expressed as:

[0069]

[0070] x c Decoder through feature parser of artifact-free regions cont_b , get the feature image x cc , x cc Then pass the feature generator Decoder of the artifact-free area cont_b , thus obtaining the feature image x ccc ; The target image y passes through the feature parser Decoder of the artifact-free area cont_b Get the feature image y c ;y c and x a Add to get y ca ,y ca Through the feature parser Decoder of the artifact area cont_a , and the generated artifact image y is obtained cr ;y cr Through the feature generator Encoder of the artifact area cont_a With the artifact generator Encoder art_a Get the feature images y crc and cra , feature image y cra With x ccc Add them together and then pass them through the feature parser Decoder in the artifact area cont_a, and obtain a generated image that is theoretically consistent with the input image It is called the cycle consistency loss function in the artifact area:

[0071]

[0072] The feature images generated in the artifact area are used to calculate the kernel loss of the first generated content feature image, the first generated artifact feature image, the second generated content feature image, and the second generated artifact feature image in the same training. The formula is expressed as:

[0073]

[0074]

[0075]

[0076]

[0077] Feature image y generated by the artifact area cr The adversarial loss of the image of this feature is calculated using the least squares loss, and the formula is expressed as:

[0078]

[0079] Using the pre-trained VGG19 as the perception network, v(y cr ) and v(y) represent y cr The characteristic image obtained by the perception network is α = in(x), and α = in(x) represents the normalization of x. The calculation formula of the perceptual loss in the artifact area is expressed as:

[0080]

[0081] The target image is input into the feature generator and feature parser of the artifact-free area, and the reconstruction loss function is calculated between the feature image obtained from the feature parser and the target image. The formula is expressed as:

[0082]

[0083] y crc Input into the feature generator of the artifact-free area to obtain the feature image right The cycle consistency loss is calculated with the target image, which is called the cycle consistency loss of the artifact-free region:

[0084]

[0085] The feature images generated in the artifact-free area are used to calculate the kernel loss of the first generated content feature image and the second generated content feature image in the same training respectively. The formula is expressed as:

[0086]

[0087]

[0088] Feature image x generated in artifact-free area cc The adversarial loss of the image of this feature is calculated using the least squares loss, and the formula is expressed as:

[0089]

[0090] The perceptual loss calculation formula for the artifact-free area is expressed as:

[0091]

[0092] The content loss of the content discriminator is expressed as:

[0093]

[0094] The joint loss function of the dual-region training generative adversarial network with shared weights is expressed as:

[0095]

[0096] Among them, GAN w is the calculation weight of the adversarial loss of the artifact area, the adversarial loss of the artifact-free area, and the content loss of the style discriminator; KL w is the calculation weight of the content core loss of the area with artifacts, the artifact core loss of the area with artifacts, and the content core loss of the area without artifacts; is the calculation weight of the cyclic content kernel loss of the artifact area, the cyclic artifact kernel loss of the artifact area, and the cyclic content kernel loss of the artifact-free area; VGG w is the calculation weight of the perceptual loss of the two regions; REC w The weights for reconstruction loss calculation are respectively for the area with artifacts and the area without artifacts; REC_cyc_w is the calculation weight for the cycle consistency loss of the area with artifacts and the area without artifacts.

[0097] In summary, the present invention adopts multiple generators and discriminators and designs a joint multiple loss function, which can effectively ensure the stability of the model and the quality of the output image. In addition, compared with the traditional hardware-based method of collecting additional data for de-artifacting operation, the present invention can complete the de-artifacting operation without additional information. Using two regions to train the model can be both supervised and unsupervised.

[0098] It should be understood that in the entire network training optimization process, the existing Adam optimizer can be used for optimization, and the training network gradually reaches a convergence state. During training, CCTA images without artifacts and with artifacts can be converted into multiple image blocks of the same size, and the image blocks are used as inputs of the artifact-free area and the artifact-containing area, respectively, and the CCTA image without artifacts is used as a reference.

[0099] Furthermore, artifact removal of medical images can be achieved using a trained dual-region generative adversarial network. For example, the medical image to be processed is divided into image blocks containing artifacts and image blocks without artifacts, which are used as inputs of the artifact area processing module and the artifact-free area processing module, respectively, to obtain a generated image. For another example, in order to improve the perception of local information, the artifact-free and artifact-containing areas of the image to be processed are respectively converted into multiple image blocks of the same size, and the image blocks are used as inputs of the artifact-free area and the artifact-containing area, respectively. The actual processing process of the medical image is similar to the training process and will not be repeated here.

[0100] In summary, the present invention uses a dual-region generative adversarial network with shared weights to improve the generalization ability of the model and its application in the field of medical images. A deep learning network is used to directly upgrade the artifact image to a high-quality image without the need for other information. In addition to being applied to artifact removal and denoising, the present invention can also be applied to the field of image super-resolution after appropriate modifications. In addition, the present invention extracts image features in two regions, generates image features from the cross-regions with artifacts and the regions without artifacts, and the patch method can be further used between the training models to improve the model's perception of local information, thereby significantly improving the detail expression of the generated medical images.

[0101] In order to further verify the effect of the present invention, experiments were conducted. Fig.10 As shown, from left to right, they are the input image, the generated image and the target image. It can be seen that the method of the present invention can effectively improve the peak signal-to-noise ratio and structural similarity of the image, and at the same time, can restore the image detail information to a certain extent.

[0102] It should be understood that, without departing from the spirit and scope of the present invention, those skilled in the art may make appropriate changes or modifications to the above embodiments, such as setting more or fewer convolution layers, setting different convolution kernels, convolution steps, etc. for feature generators, feature parsers, artifact generators, reconstruction parsers, content identifiers, etc. For another example, the same or different structures may be used for the generators of artifact areas and the generators of artifact-free areas.

[0103] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0104] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0105] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0106] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0107] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0108] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0109] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0110] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0111] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A method for generating an image, comprising: A dual-region generative adversarial network with shared weights is trained with a set joint loss function as a target. The dual-region generative adversarial network with shared weights includes an artifact-free region processing module for processing artifact-free regions and an artifact-containing region processing module for processing artifact-containing regions, wherein: the artifact-free region processing module includes a first generator and a first discriminator, the first generator includes a first feature generator and a first feature parser, the first feature generator is used to extract the feature image of the artifact-free region, and the first discriminator is used to calculate the adversarial loss of the artifact-free region; the artifact-containing region processing module includes a second generator and a second discriminator, the second generator includes a second feature generator, an artifact generator, a reconstruction parser and a second feature parser The first feature analyzer is used to extract the feature image of the artifact area, the second feature generator is used to re-parse the feature image extracted by the second feature generator into a second artifact-free image; the artifact generator is used to extract the artifact features of the image with artifacts; the reconstruction analyzer is used to add the feature image extracted by the second feature generator and the artifact features of the image with artifacts extracted by the artifact generator, and the feature image obtained after parsing is used as the constraint of the artifact area; the second feature analyzer is used to add the feature image of the artifact-free area and the artifact features of the image with artifacts to generate an artifact image with the artifact-free area as the base, and the second discriminator is used to calculate the adversarial loss of the artifact area; Dividing the medical image to be processed into image blocks containing artifacts and image blocks without artifacts, respectively serving as inputs of an artifact area processing module and an artifact-free area processing module, to obtain a generated image; The weight-shared dual-region generative adversarial network further includes a content discriminator, which is used to calculate adversarial losses for the first feature generator and the second feature generator.

2. The image generation method according to claim 1, wherein: The joint loss function of the discriminator in the shared-weight dual-region generative adversarial network is expressed as: loss total =GAN_weight×(loss dis_cont +loss dis_b +loss dis_a ) Among them, loss dis_cont is the loss of the content discriminator, loss dis_b is the loss of the first discriminator of the artifact-free region processing module, loss dis_a is the loss of the second discriminator with the artifact area processing module, and GAN_weight is a set hyperparameter.

3. The image generation method according to claim 1, wherein: The first feature generator includes multiple layers of convolutional layers and multiple layers of residual modules in sequence, wherein the multiple layers of convolutional layers are configured to use multiple channels to learn feature images from different angles and downsample the feature images, and the multiple layers of residual modules are configured to learn low-dimensional and high-dimensional feature information of the feature images.

4. The image generation method according to claim 1, wherein: The artifact generator extracts artifact features of the artifact image by setting multiple convolution layers and performs down-sampling. The size of the feature image obtained by down-sampling is consistent with that of the second feature generator and the first feature generator of the artifact-free area.

5. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the steps of the image generation method according to any one of claims 1 to 4 are implemented.

6. A computer device comprising a memory and a processor, wherein a computer program capable of being run on the processor is stored in the memory, characterized in that: When the processor executes the program, the steps of the image generation method according to any one of claims 1 to 4 are implemented.

Images