Method, apparatus, storage medium, and processor for processing medical images

Abstract

The present application discloses a method, apparatus, storage medium, and processor for processing medical images. The method includes: obtaining a medical image of a first modality; converting the medical image of the first modality through a registration generative adversarial network to obtain a medical image of a second modality, where the registration generative adversarial network is trained by a generative adversarial network, and a generative model in the generative adversarial network and a target registration network are jointly trained to adaptively fit the unaligned noise distribution in the medical image of the first modality, and the first modality and the second modality are different. Through the present application, the problem of low accuracy in converting medical images in the related art is solved.

Description

Technical Field

[0001] This application relates to the field of image processing technology. Specifically, it relates to a method, device, storage medium, and processor for processing medical images. Background Art

[0002] Generative Adversarial Network (GAN) has shown great potential in the field of image-to-image translation. It has been successfully applied to medical image analysis, such as segmentation, registration, and dose calculation. However, existing models have their limitations.

[0003] Specifically, Generative Adversarial Network (GAN) realizes the process of image generation and transformation through adversarial training of a training generator G and a discriminator D. The generator is used to transform the distribution of the source domain image X into the distribution of the target domain image Y. The discriminator is used to determine whether the target domain image is likely to come from the generator or real data.

[0004]

[0005] Supervised Pix2Pix and unsupervised Cycle-consistency are two common modes in GAN. Pix2Pix updates the generator (G: X→Y) by minimizing the pixel-level L1 loss between the source image x and the target image Y. Therefore, it requires well-aligned paired images, where each pixel has a corresponding label.

[0006]

[0007] However, well-aligned paired images are not always available in real-world scenarios. To address the challenges brought by unaligned images, the Cycle-consistency mode has been developed in current technology. This consistency is based on the following assumption: The generator G from the source domain X to the target domain Y (G: X→Y) and the generator F from Y to X (F: Y→X) are inverses of each other. Compared with the Pix2Pix mode, the cycle-consistency mode performs better on unaligned or unpaired images.

[0008]

[0009] However, the Cycle-consistency mode has its limitations. In medical image-to-image generation, not only style conversion between image domains is required, but also conversion between specific image pairs. The optimal solution should be unique. For example, the generated image should preserve the anatomical features of the original image as much as possible. The Cycle-consistency mode may produce multiple solutions, which means that the training process may be relatively unstable and the results may be inaccurate. That is, the Cycle-consistency mode is very sensitive to perturbations and it is difficult to meet the high-precision requirements of medical image-to-image conversion tasks. The pix2pix mode is also not ideal. Even if it has a unique solution, it is difficult to meet the requirements for well-aligned paired images. For unaligned images, errors propagate through the Pix2Pix mode, which may lead to unreasonable displacements in the finally generated images.

[0010] In view of the problem of low accuracy in medical image conversion in the related art, no effective solution has been proposed yet. Summary of the Invention

[0011] The main purpose of this application is to provide a method, device, storage medium and processor for processing medical images, so as to solve the problem of low accuracy in medical image conversion in the related art.

[0012] To achieve the above object, according to one aspect of this application, a method for processing medical images is provided. The method includes: obtaining a medical image of a first modality; converting the medical image of the first modality through a registration generative adversarial network to obtain a medical image of a second modality, where the registration generative adversarial network is obtained by training a generative adversarial network, and the generative model in the generative adversarial network and the target registration network are jointly trained to adaptively fit the unaligned noise distribution in the medical image of the first modality, and the first modality and the second modality are different.

[0013] Further, before converting the medical image of the first modality through the registration generative adversarial network, the method further includes: obtaining a pair of medical sample images, where the pair of medical sample images includes: a medical sample image of the first modality and a medical sample image of the second modality; inputting the medical sample image of the first modality into the generative model to obtain a converted medical sample image of the second modality; registering the converted medical sample image of the second modality with the target registration network by using the medical sample image of the second modality to obtain a registered medical sample image of the second modality; adjusting the parameters of the generative model according to the registered medical sample image of the second modality to obtain the registration generative adversarial network.

[0014] Further, adjust the parameters of the generation model according to the registered medical sample image of the second modality to obtain a registration generative adversarial network, including: calculating the similarity between the registered medical sample image of the second modality and the medical sample image of the second modality; adjusting the parameters of the generation model based on the similarity to obtain a registration generative adversarial network.

[0015] Further, register the transformed medical sample image of the second modality by using the medical sample image of the second modality through a target registration network to obtain the registered medical sample image of the second modality, including: in the target registration network, using a smooth loss function to constrain the deformation field, where the deformation field acts on the transformed medical sample image of the second modality to register the transformed medical sample image of the second modality with the medical sample image of the second modality to obtain the registered medical sample image of the second modality.

[0016] Further, after obtaining the transformed medical sample image of the second modality, the method further includes: inputting the transformed medical sample image of the second modality and the medical sample image of the second modality into a discriminant model to obtain a discriminant result; adjusting the parameters of the generation model based on the discriminant result and the similarity to obtain a registration generative adversarial network.

[0017] Further, the target registration network is a U-Net.

[0018] To achieve the above object, according to another aspect of the present application, there is provided a medical image processing device. The device includes: a first acquisition unit for acquiring a medical image of a first modality; a first conversion unit for converting the medical image of the first modality through a registration generative adversarial network to obtain a medical image of a second modality, where the registration generative adversarial network is obtained by training a generative adversarial network, and the generation model in the generative adversarial network is jointly trained with a target registration network to adaptively fit the unaligned noise distribution in the medical image of the first modality, and the first modality and the second modality are different.

[0019] Further, the device further includes: a second acquisition unit for acquiring a pair of medical sample images before converting the medical image of the first modality through the registration generative adversarial network, where the pair of medical sample images includes: a medical sample image of the first modality and a medical sample image of the second modality; a first input unit for inputting the medical sample image of the first modality into the generation model to obtain a transformed medical sample image of the second modality; a first registration unit for registering the transformed medical sample image of the second modality by using the medical sample image of the second modality through a target registration network to obtain a registered medical sample image of the second modality; a first adjustment unit for adjusting the parameters of the generation model according to the registered medical sample image of the second modality to obtain a registration generative adversarial network.

[0020] Further, the first adjustment unit includes: a first calculation module, configured to calculate the similarity between the registered medical sample image of the second modality and the medical sample image of the second modality; a first adjustment module, configured to adjust the parameters of the generation model based on the similarity to obtain a registration generative adversarial network.

[0021] Further, the first registration unit includes: a first processing module, configured to use a smoothing loss function to constrain the deformation field in the target registration network, where the deformation field acts on the transformed medical sample image of the second modality, so that the transformed medical sample image of the second modality is registered with the medical sample image of the second modality to obtain the registered medical sample image of the second modality.

[0022] Further, the apparatus further includes: a second input unit, configured to input the transformed medical sample image of the second modality and the medical sample image of the second modality into a discriminant model after obtaining the transformed medical sample image of the second modality to obtain a discriminant result; a second adjustment unit, configured to adjust the parameters of the generation model based on the discriminant result and the similarity to obtain a registration generative adversarial network.

[0023] Further, the target registration network is a U-Net.

[0024] According to another aspect of the embodiments of the present application, there is also provided a processor, where the processor is used to run a program, and when the program runs, it executes the method of any one of the above.

[0025] According to another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium, on which a computer program / instructions are stored, and when the computer program / instructions are executed by a processor, the method of any one of the above is executed.

[0026] Through the present application, the following steps are adopted: obtaining a medical image of a first modality; converting the medical image of the first modality through a registration generative adversarial network to obtain a medical image of a second modality, where the registration generative adversarial network is obtained by training a generative adversarial network, and the generation model in the generative adversarial network is jointly trained with a target registration network to adaptively fit the unaligned noise distribution in the medical image of the first modality, and the first modality and the second modality are different, which solves the problem of low accuracy of medical image conversion in the related art. By converting the medical image of the first modality through the registration generative adversarial network, the effect of improving the accuracy of medical image conversion is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0028] Figure 1 is a flowchart of a method for processing medical images provided by an embodiment of the present application;

[0029] Figure 2 is a schematic diagram of three modes in the method for processing medical images provided by an embodiment of the present application; and

[0030] Figure 3 is a schematic diagram of a device for processing medical images provided by an embodiment of the present application. Detailed implementation manners

[0031] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data may be interchanged under appropriate circumstances for the embodiments of the present application described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0034] The present invention will be described below in conjunction with preferred implementation steps. Figure 1 is a flowchart of a method for processing medical images provided by an embodiment of the present application, as Figure 1 shown, the method includes the following steps:

[0035] Step S101, obtaining a medical image of a first modality;

[0036] For example, the medical image of the first modality is an MR medical image.

[0037] Step S102: Transform the medical image of the first modality through a registration generative adversarial network to obtain a medical image of the second modality. The registration generative adversarial network is obtained by training a generative adversarial network. In the generative adversarial network, the generative model and the target registration network are jointly trained to adaptively fit the unaligned noise distribution in the medical image of the first modality. The first modality and the second modality are different.

[0038] The above-mentioned generative adversarial network is a deep learning model. This model generates outputs through the mutual game learning of two models in the framework: the generative model and the discriminative model. On this basis, in this application, the generative model and the target registration network (such as U-Net) are jointly trained to adaptively fit the unaligned noise distribution, so as to find the common optimal target solution for image transformation and registration tasks. Thus, the generative model in this registration generative adversarial network transforms the medical image of the first modality to obtain the transformed medical image of the second modality, and then the transformed medical image of the second modality is registered through the target registration network to finally obtain the medical image of the second modality. Therefore, the accuracy of transforming the medical image of the first modality into the medical image of the second modality (such as CT medical images) is improved.

[0039] Optionally, in the method for processing medical images provided in the embodiments of this application, before transforming the medical image of the first modality through the registration generative adversarial network, the method further includes: obtaining a pair of medical sample images, where the pair of medical sample images includes: a medical sample image of the first modality and a medical sample image of the second modality; inputting the medical sample image of the first modality into the generative model to obtain the transformed medical sample image of the second modality; registering the transformed medical sample image of the second modality through the target registration network using the medical sample image of the second modality to obtain the registered medical sample image of the second modality; adjusting the parameters of the generative model according to the registered medical sample image of the second modality to obtain the registration generative adversarial network.

[0040] The above-mentioned medical sample image of the first modality can be an MR medical image obtained by photographing a target object, and the above-mentioned medical sample image of the second modality can be a CT medical image obtained by photographing a target object. The MR medical image and the CT medical image are used as a pair of medical sample images. Input the MR medical image into the generative model to obtain the transformed CT medical image. Use the CT medical image obtained by photographing the target object to register the transformed CT medical image through the target registration network, adaptively fit the unaligned noise distribution of the transformed CT medical image to avoid noise interference, obtain the registered CT medical image, and adjust the parameters of the generative model according to the registered CT medical image to obtain the registration generative adversarial network.

[0041] Optionally, in the medical image processing method provided in the embodiments of the present application, adjusting the parameters of the generation model according to the registered medical sample image of the second modality to obtain a registration generative adversarial network includes: calculating the similarity between the registered medical sample image of the second modality and the medical sample image of the second modality; adjusting the parameters of the generation model based on the similarity to obtain a registration generative adversarial network.

[0042] The above solution further defines how to adjust the parameters of the generation model according to the registered medical sample image of the second modality to obtain a registration generative adversarial network, ensuring the accuracy of obtaining the registration generative adversarial network.

[0043] Optionally, in the medical image processing method provided in the embodiments of the present application, registering the transformed medical sample image of the second modality by using the medical sample image of the second modality through a target registration network to obtain the registered medical sample image of the second modality includes: in the target registration network, using a smooth loss function to constrain the deformation field, where the deformation field acts on the transformed medical sample image of the second modality to register the transformed medical sample image of the second modality with the medical sample image of the second modality to obtain the registered medical sample image of the second modality.

[0044] In the above solution, by using a smooth loss function to constrain the deformation field in the target registration network, it is ensured that after the deformation field acts on the transformed CT medical image, the transformed CT medical image is registered with the CT medical image obtained by photographing the target object, thereby realizing the coordinate mapping between the transformed CT medical image and the CT medical image obtained by photographing the target object, and overall ensuring the accuracy of the registered CT medical image.

[0045] Optionally, in the medical image processing method provided in the embodiments of the present application, after obtaining the transformed medical sample image of the second modality, the method further includes: inputting the transformed medical sample image of the second modality and the medical sample image of the second modality into a discriminant model to obtain a discriminant result; adjusting the parameters of the generation model based on the discriminant result and the similarity to obtain a registration generative adversarial network.

[0046] The above discriminator is used to determine whether the input image may come from the generator or real data. By inputting the transformed medical sample image of the second modality and the medical sample image of the second modality into the discriminant model to obtain a discriminant result, and adjusting the parameters of the generation model based on the discriminant result and the similarity, the accuracy of the obtained registration generative adversarial network is ensured.

[0047] In summary, the method for processing medical images provided by the embodiments of the present application obtains a medical image of a first modality; converts the medical image of the first modality through a registration generative adversarial network to obtain a medical image of a second modality, where the registration generative adversarial network is trained by a generative adversarial network, and the generative model in the generative adversarial network and the target registration network are jointly trained to adaptively fit the unaligned noise distribution in the medical image of the first modality. The first modality and the second modality are different, which solves the problem of low accuracy in converting medical images in the related art. By converting the medical image of the first modality through the registration generative adversarial network, the effect of improving the accuracy of converting medical images is achieved.

[0048] As Figure 2 shown, (a) is the supervised Pix2Pix in the prior art, (b) is the cycle-consistency mode in the prior art, and (c) is the RegGAN (registration generative adversarial network) provided by the present application. Specifically, the mode method of RegGAN is as follows: In the present application, image misalignment is used as a noise label, and image-to-image generation training becomes a supervised learning process with a noise label. Given a training dataset where x n , are two images of different modalities, assuming y n is the correct image corresponding to x n , but y n is unknown. The goal of the present application is to train a generator G on the noisy dataset and require its performance to be as equivalent as possible to the performance trained on the noiseless dataset . If we choose to directly optimize using Pix2Pix in (a) with formula (4), it will lead to poor results because the generator cannot exclude the influence of noise.

[0049]

[0050] If there exists a noise transfer φ that can well adapt to the noise distribution, then the generator G is equivalent to being trained on the noiseless dataset , which also means that the model has a unique solution.

[0051]

[0052] The key to the problem then becomes how to obtain φ, which can map the correct image y nConsider it as a potential random variable and explicitly model the label noise as part of the network structure, denoted by R. Then, Equation (5) can be written in the form of the log-likelihood function:

[0053]

[0054] In this application, the log-likelihood function (6) can be directly used as the objective function to train the neural network. The training objective is to obtain a noise model R and a generation model G. For the type of noise distribution, it is clear and can be expressed as the displacement error: where T represents a random deformation field that causes each pixel of y in the image to have a random displacement, Then, a registration network is connected after the generator G as the noise model R to correct the result G(x) of the generator. Thus, the correction loss is obtained:

[0055]

[0056] where the registration network can adopt a common U-Net, denotes the deformation field for registering G(x) with to fit T. In this application, a smooth constraint, smooth loss, can also be imposed on the deformation field. By minimizing the gradient of the deformation field, the deformation is made as smooth as possible:

[0057]

[0058] Finally, in this application, the adversarial loss (Equation 1) between the generator and the discriminator is added, and the complete loss function is formed:

[0059]

[0060] The registration generative adversarial network can be obtained through Equation (9). In this application, it can also be mathematically proven the feasibility of the above method. Specifically, assume that the random noise distribution T is smooth enough and satisfies: ToT -1 = I, then the minimum value of the correction loss under the noise distribution is the same as the minimum value of the original loss under the clean distribution:

[0061]

[0062] Proof: Substitute into the left side of Equation (10) to get:

[0063]

[0064] It should be noted that the assumption condition in the proof process is not a very strong assumption. Because in the mode of RegGAN, there is a smooth constraint on registration, as shown in formula (3). At the same time, applying the same deformation T to the input x and the label will not change the minimization solution. Therefore, the feasibility of RegGAN is demonstrated from the theoretical perspective of "loss correction" as described above. This solution uses an additional registration network to train the generator to adaptively fit the unaligned noise distribution. The goal is to find a common optimal solution for image-to-image generation and registration tasks, thereby realizing an end-to-end denoising image generation mode. In addition, RegGAN eliminates the requirement for well-aligned paired images and searches for a unique solution during the training process. According to the conversion results of the above solution, RegGAN is superior to Pix2Pix on aligned data and superior to Cycle-consistency on unaligned or misaligned data. RegGAN can be integrated into other methods without changing the original network architecture. Compared with Cycle-consistency that uses two generators and discriminators, RegGAN can provide better performance with fewer network parameters. RegGAN achieves fewer parameters and a smaller network structure based on data that requires perfect alignment, greatly saving computing resources. -1

[0065] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0066] The embodiment of the present application also provides a medical image processing device. It should be noted that the medical image processing device of the embodiment of the present application can be used to execute the medical image processing method provided by the embodiment of the present application. The following introduces the medical image processing device provided by the embodiment of the present application.

[0067] Figure 3 Figure 3

[0068]

[0069] As shown, the device includes: a first acquisition unit 301, a first conversion unit 302. Specifically, the first acquisition unit 301 is configured to acquire a medical image of the first modality;

[0069] The first conversion unit 302 is configured to convert a medical image in a first modality through a registration generative adversarial network to obtain a medical image in a second modality. The registration generative adversarial network is trained by a generative adversarial network. In the generative adversarial network, the generative model and the target registration network are jointly trained to adaptively fit the misaligned noise distribution in the medical image in the first modality. The first modality and the second modality are different.

[0070] The medical image processing device provided by the embodiments of the present application obtains a medical image in a first modality through the first acquisition unit 301; the first conversion unit 302 converts the medical image in the first modality through a registration generative adversarial network to obtain a medical image in a second modality. The registration generative adversarial network is trained by a generative adversarial network. In the generative adversarial network, the generative model and the target registration network are jointly trained to adaptively fit the misaligned noise distribution in the medical image in the first modality. The first modality and the second modality are different. This solves the problem of low accuracy in medical image conversion in the related art, and thus achieves the effect of D.

[0071] Optionally, in the medical image processing device provided by the embodiments of the present application, the device further includes: a second acquisition unit, configured to acquire a pair of medical sample images before converting the medical image in the first modality through the registration generative adversarial network, where the pair of medical sample images includes: a medical sample image in the first modality and a medical sample image in the second modality; a first input unit, configured to input the medical sample image in the first modality into the generative model to obtain a converted medical sample image in the second modality; a first registration unit, configured to register the converted medical sample image in the second modality with the medical sample image in the second modality through the target registration network to obtain a registered medical sample image in the second modality; a first adjustment unit, configured to adjust the parameters of the generative model according to the registered medical sample image in the second modality to obtain a registration generative adversarial network.

[0072] Optionally, in the medical image processing device provided by the embodiments of the present application, the first adjustment unit includes: a first calculation module, configured to calculate the similarity between the registered medical sample image in the second modality and the medical sample image in the second modality; a first adjustment module, configured to adjust the parameters of the generative model based on the similarity to obtain a registration generative adversarial network.

[0073] Optionally, in the medical image processing device provided by the embodiments of the present application, the first registration unit includes: a first processing module, configured to constrain the deformation field by using a smoothing loss function in the target registration network, where the deformation field acts on the converted medical sample image in the second modality to register the converted medical sample image in the second modality with the medical sample image in the second modality to obtain a registered medical sample image in the second modality.

[0074] Optionally, in the medical image processing device provided in the embodiments of the present application, the device further includes: a second input unit, configured to input the converted medical sample image of the second modality and the medical sample image of the second modality into a discrimination model after obtaining the converted medical sample image of the second modality, so as to obtain a discrimination result; a second adjustment unit, configured to adjust parameters of the generation model based on the discrimination result and similarity adjustment to obtain a registration generative adversarial network.

[0075] Optionally, in the medical image processing device provided in the embodiments of the present application, the target registration network is a U-Net.

[0076] The medical image processing device includes a processor and a memory. The above-mentioned first acquisition unit 301, first conversion unit 302, etc. are all stored in the memory as program units, and the processor executes the above program units stored in the memory to implement corresponding functions.

[0077] The processor contains a kernel, and the kernel retrieves corresponding program units from the memory. One or more kernels can be set, and the parameters of the kernels are adjusted to process medical images.

[0078] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM). The memory includes at least one storage chip.

[0079] The embodiments of the present invention provide a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, it implements a medical image processing method.

[0080] The embodiments of the present invention provide a processor, and the processor is used to run a program. When the program runs, it executes a medical image processing method.

[0081] The embodiments of the present invention provide a device, the device includes a processor, a memory, and a program stored on the memory and executable on the processor. When the processor executes the program, the following steps are implemented: obtaining a medical image of a first modality; converting the medical image of the first modality through a registration generative adversarial network to obtain a medical image of a second modality, where the registration generative adversarial network is obtained by training a generative adversarial network, and the generation model in the generative adversarial network is jointly trained with a target registration network to adaptively fit the unaligned noise distribution in the medical image of the first modality, and the first modality and the second modality are different.

[0082] When the processor executes the program, the following steps are also implemented: Before converting the medical image in the first modality through the registration generative adversarial network, obtain a pair of medical sample images, where the pair of medical sample images includes: a medical sample image in the first modality and a medical sample image in the second modality; input the medical sample image in the first modality into the generative model to obtain the converted medical sample image in the second modality; use the medical sample image in the second modality to register the converted medical sample image in the second modality through the target registration network to obtain the registered medical sample image in the second modality; adjust the parameters of the generative model according to the registered medical sample image in the second modality to obtain the registration generative adversarial network.

[0083] When the processor executes the program, the following steps are also implemented: Calculate the similarity between the registered medical sample image in the second modality and the medical sample image in the second modality; adjust the parameters of the generative model based on the similarity to obtain the registration generative adversarial network.

[0084] When the processor executes the program, the following steps are also implemented: In the target registration network, use a smoothing loss function to constrain the deformation field, where the deformation field acts on the converted medical sample image in the second modality to register the converted medical sample image in the second modality with the medical sample image in the second modality to obtain the registered medical sample image in the second modality.

[0085] When the processor executes the program, the following steps are also implemented: After obtaining the converted medical sample image in the second modality, input the converted medical sample image in the second modality and the medical sample image in the second modality into the discriminative model to obtain a discriminative result; adjust the parameters of the generative model based on the discriminative result and the similarity to obtain the registration generative adversarial network.

[0086] When the processor executes the program, the following steps are also implemented: The target registration network is a U-Net.

[0087] The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0088] This application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program initialized with the following method steps: Obtain a medical image in the first modality; convert the medical image in the first modality through a registration generative adversarial network to obtain a medical image in the second modality, where the registration generative adversarial network is trained by a generative adversarial network, and the generative model in the generative adversarial network is jointly trained with the target registration network to adaptively fit the unaligned noise distribution in the medical image in the first modality, and the first modality and the second modality are different.

[0089] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: before converting a medical image in a first modality through a registration generative adversarial network, obtain a pair of medical sample images, where the pair of medical sample images includes: a medical sample image in the first modality and a medical sample image in a second modality; input the medical sample image in the first modality into a generative model to obtain a converted medical sample image in the second modality; use the medical sample image in the second modality to register the converted medical sample image in the second modality through a target registration network to obtain a registered medical sample image in the second modality; adjust the parameters of the generative model according to the registered medical sample image in the second modality to obtain a registration generative adversarial network.

[0090] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: calculate the similarity between the registered medical sample image in the second modality and the medical sample image in the second modality; adjust the parameters of the generative model based on the similarity to obtain a registration generative adversarial network.

[0091] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: in the target registration network, use a smoothing loss function to constrain the deformation field, where the deformation field acts on the converted medical sample image in the second modality to register the converted medical sample image in the second modality with the medical sample image in the second modality to obtain a registered medical sample image in the second modality.

[0092] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: after obtaining the converted medical sample image in the second modality, input the converted medical sample image in the second modality and the medical sample image in the second modality into a discriminative model to obtain a discriminative result; adjust the parameters of the generative model based on the discriminative result and the similarity to obtain a registration generative adversarial network.

[0093] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: the target registration network is a U-Net.

[0094] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one or more flows and / or blocks. Figure 1 in one or more flows and / or blocks Figure 1 or more blocks.

[0096] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in one or more flows and / or blocks. Figure 1 in one or more flows and / or blocks Figure 1 or more blocks.

[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows and / or blocks. Figure 1 in one or more flows and / or blocks Figure 1 or more blocks.

[0098] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0099] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0100] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0101] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0103] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for processing medical images, characterized in that, Comprising: Obtaining a medical image of a first modality; Converting the medical image of the first modality through a registration generative adversarial network to obtain a medical image of a second modality, wherein the registration generative adversarial network is trained by a generative adversarial network, and a generative model in the generative adversarial network is jointly trained with a target registration network to adaptively fit an unaligned noise distribution in the medical image of the first modality, and the first modality is different from the second modality; Wherein, before converting the medical image of the first modality through the registration generative adversarial network, the method further comprises: obtaining a pair of medical sample images, wherein the pair of medical sample images includes: a medical sample image of the first modality and a medical sample image of the second modality; inputting the medical sample image of the first modality into the generative model to obtain a converted medical sample image of the second modality; registering the converted medical sample image of the second modality by using the medical sample image of the second modality through the target registration network to obtain a registered medical sample image of the second modality; and adjusting parameters of the generative model according to the registered medical sample image of the second modality to obtain the registration generative adversarial network; Wherein, the method further comprises: converting the medical image of the first modality through the generative model in the registration generative adversarial network to obtain a converted medical image of the second modality, and then registering the converted medical image of the second modality through the target registration network to finally obtain the medical image of the second modality.

2. The method according to claim 1, characterized in that Adjusting parameters of the generative model according to the registered medical sample image of the second modality to obtain the registration generative adversarial network includes: Calculating a similarity between the registered medical sample image of the second modality and the medical sample image of the second modality; Adjusting parameters of the generative model based on the similarity to obtain the registration generative adversarial network.

3. The method according to claim 1, wherein Registering the converted medical sample image of the second modality by using the medical sample image of the second modality through the target registration network to obtain a registered medical sample image of the second modality includes: In the target registration network, using a smoothing loss function to constrain a deformation field, wherein the deformation field acts on the converted medical sample image of the second modality to register the converted medical sample image of the second modality with the medical sample image of the second modality to obtain the registered medical sample image of the second modality.

4. The method according to claim 2, wherein After obtaining the converted medical sample image of the second modality, the method further comprises: Inputting the converted medical sample image of the second modality and the medical sample image of the second modality into a discriminant model to obtain a discriminant result; Adjusting parameters of the generative model based on the discriminant result and the similarity to obtain the registration generative adversarial network.

5. The method according to claim 1, wherein The target registration network is a U-Net.

6. A processing device for medical images, characterized in that, Comprising: A first obtaining unit, configured to obtain a medical image of a first modality; A first conversion unit, configured to convert the medical image of the first modality through a registration generative adversarial network to obtain a medical image of the second modality, wherein the registration generative adversarial network is obtained by training a generative adversarial network, and a generative model in the generative adversarial network is jointly trained with a target registration network to adaptively fit the unaligned noise distribution in the medical image of the first modality, and the first modality is different from the second modality; Wherein, the apparatus further includes: a second acquisition unit, configured to acquire a pair of medical sample images before converting the medical image of the first modality through the registration generative adversarial network, wherein the pair of medical sample images includes: a medical sample image of the first modality and a medical sample image of the second modality; a first input unit, configured to input the medical sample image of the first modality into the generative model to obtain a converted medical sample image of the second modality; a first registration unit, configured to register the converted medical sample image of the second modality by using the medical sample image of the second modality through the target registration network to obtain a registered medical sample image of the second modality; a first adjustment unit, configured to adjust parameters of the generative model according to the registered medical sample image of the second modality to obtain the registration generative adversarial network; convert the medical image of the first modality through the generative model in the registration generative adversarial network to obtain a converted medical image of the second modality, and then register the converted medical image of the second modality through the target registration network to finally obtain the medical image of the second modality.

7. A computer-readable storage medium, characterized in that, The storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 5.

8. A processor, characterized in that, The processor is configured to run a program, wherein the program, when running, executes the method according to any one of claims 1 to 5.

Images