Image registration method, model training method, equipment, medium and product

Abstract

The invention discloses an image registration method, a model training method, equipment, a medium and a product, and belongs to the technical field of medical image processing. The method comprises the following steps: acquiring a fixed image and a floating image to be registered; the fixed image and the floating image are input into a pre-trained registration model, a target deformation result of the floating image is obtained, and the target deformation result is a registration image of the fixed image; wherein the pre-trained registration model comprises a pre-trained image registration module and a pre-trained image post-processing module; the pre-trained image registration module is used for performing image registration processing on the fixed image and the floating image to obtain grid sampling parameters; the pre-trained image post-processing module is used for determining an image deformation result, corresponding to the grid sampling parameters, of the floating image and performing image post-processing on the image deformation result to obtain a target deformation result, and the image post-processing comprises denoising and / or image restoration.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to an image registration method, a model training method, equipment, a medium and a product. Background Art

[0002] CT image registration is of great significance in the field of medical imaging. Since the imaging information of different modalities can be complementary, the accurate registration results between CT images and other modal images (such as MRI, PET, etc.) can provide a comprehensive diagnostic basis and help improve the accuracy of lesion localization and diagnosis.

[0003] However, existing image registration methods usually cannot guarantee the accuracy of image registration results. Therefore, it is necessary to provide an image registration method to improve the accuracy of image registration results. Summary of the invention

[0004] The present invention provides an image registration method, a model training method, a device, a medium and a product to solve the problem of low accuracy in existing image registration methods.

[0005] According to one aspect of the present invention, there is provided an image registration method, comprising:

[0006] Acquire a fixed image and a floating image to be registered;

[0007] Inputting the fixed image and the floating image into a pre-trained registration model to obtain a target deformation result of the floating image, wherein the target deformation result is a registration image of the fixed image;

[0008] Among them, the pre-trained registration model includes a pre-trained image registration module and a pre-trained image post-processing module; the pre-trained image registration module is used to perform image registration processing on the fixed image and the floating image to obtain grid sampling parameters; the pre-trained image post-processing module is used to determine the image deformation result corresponding to the floating image and the grid sampling parameters, and perform image post-processing on the image deformation result to obtain the target deformation result, and the image post-processing includes denoising and / or image restoration.

[0009] According to another aspect of the present invention, a model training method is provided, for training the registration model described in any embodiment, comprising:

[0010] Complete the training of the image registration module in the registration model to obtain a pre-trained image registration module;

[0011] Complete the training of the image post-processing module in the registration model to obtain a pre-trained image post-processing module;

[0012] Complete the joint training between the pre-trained image registration module and the pre-trained image post-processing module to update the pre-trained image registration module and the pre-trained image post-processing module, and use the registration model including the updated pre-trained image registration module and the pre-trained image post-processing module as the pre-trained registration model.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image registration method described in any embodiment of the present invention or the model training method described in any embodiment.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image registration method described in any embodiment of the present invention or the model training method described in any embodiment when executed.

[0018] According to another aspect of the present invention, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, it implements the image registration method described in any embodiment or the model training method described in any embodiment.

[0019] The technical solution provided in the embodiment of the present invention uses a pre-trained registration model to sequentially complete the registration operation of the floating image and the fixed image and the image post-processing operation. Compared with the prior art which only determines the registration result, or performs denoising before image registration, it can improve the accuracy of the image registration result and avoid the existence of noise or spatial misalignment in the registration result.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a flow chart of an image registration method provided according to an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of a model structure provided according to an embodiment of the present invention;

[0024] Figure 3 is a flowchart of an image training method provided according to an embodiment of the present invention;

[0025] Figure 4 is a structural schematic diagram of an image registration device provided according to an embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of the structure of a model training device provided according to an embodiment of the present invention;

[0027] Figure 6 It is a schematic diagram of the structure of an electronic device provided to implement an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Figure 1 This is a flow chart of an image registration method provided by an embodiment of the present invention. This embodiment is applicable to the case where a pre-trained registration model is used to determine a target deformation result of a floating image, and the target deformation result is a registration image of a fixed image. The method can be executed by an image registration device, which can be implemented in the form of hardware and / or software, and can be configured in a processor of an electronic device. Figure 1 As shown, the method includes:

[0031] S110: Acquire a fixed image and a floating image to be registered.

[0032] The fixed image can be understood as a reference image, which will not be deformed during the image registration process. The floating image can be understood as an image to be registered, which is mapped onto the reference image during the image registration process and will be deformed during the process.

[0033] In one embodiment, the fixed image and the floating image to be registered may be images of the same modality acquired at different times, such as CT images acquired at different times; or may be images of different modalities, such as a CT image and an MRI image.

[0034] S120: Input the fixed image and the floating image into a pre-trained registration model to obtain a target deformation result of the floating image, where the target deformation result is a registration image of the fixed image.

[0035] Among them, the pre-trained registration model includes a pre-trained image registration module and a pre-trained image post-processing module; the pre-trained image registration module is used to perform image registration processing on the fixed image and the floating image to obtain grid sampling parameters; the pre-trained image post-processing module is used to determine the image deformation result corresponding to the floating image and the grid sampling parameters, and perform image post-processing on the image deformation result to obtain the target deformation result, and the image post-processing includes denoising and / or image restoration.

[0036] The grid sampling parameters can be understood as a combination of pixel offsets. After determining the grid sampling parameters of the floating image relative to the fixed image, the pre-trained image registration module deforms the floating image based on the grid sampling parameters to obtain an image deformation result. The image deformation result is the initial registration image of the fixed image.

[0037] After the image deformation result is determined, it is input into the pre-trained image post-processing module, and the pre-trained image post-processing module performs denoising and / or image restoration processing on it to obtain the target deformation result. It is understandable that if the image deformation result includes noise, the pre-trained image post-processing module will perform denoising processing on it to remove the noise it includes; if the image deformation result includes spatial misalignment, etc., the pre-trained image post-processing module will perform image restoration processing on it. Since denoising and / or image restoration processing can significantly improve the accuracy of the image deformation result, the target deformation result in this embodiment has higher accuracy than the image registration result determined by the prior art.

[0038] Ideally, the image registration module can learn the transformation between the fixed image and the moving image, and apply the inverse transformation to register the floating image to the fixed image to obtain the registered image. However, the actual grid sampling parameters often lead to spatial errors, resulting in some information that should not appear in the registered image, such as misalignment or spatial displacement. Therefore, this solution adds an image post-processing module after the image registration module, and makes the two a whole, realizing image registration and performing image post-processing operations on the registered image at the same time to obtain the target deformation result.

[0039] In one embodiment, the image registration module and the image post-processing module both include: L layers of residual convolutional networks for compression and L-1 layers of residual convolutional networks for image fusion; the output of the previous residual convolutional network for compression is the input of the next residual convolutional network for compression; the decompression result of the output result of the L layers of residual convolutional networks for compression and the output result of the L-1 layers of residual convolutional networks for compression are both used as the input of the L-1 layers of residual convolutional networks for image fusion; the decompression result of the output result of the previous residual convolutional network for image fusion and the output result of the residual convolutional network for compression at the target level are both used as the input of the next residual convolutional network for image fusion, and the output result of the residual convolutional network for compression at the target level has the same resolution as the decompression result of the output result of the previous residual convolutional network for image fusion.

[0040] Simply put, Figure 2 The image registration module and the image post-processing module in the compression path include a compression path and a decompression path. For each residual convolution network in the compression path, the up-sampled result of its output result and the output result of the corresponding residual convolution network in the decompression path are input into the next residual convolution network in the decompression path by splicing the input channels, wherein the up-sampled result has the same image size as the output result of the corresponding reference convolution network in the decompression path. The number of channels corresponding to the level identifier of each residual convolution network is the same as the value corresponding to each level identifier.

[0041] The embodiment of the present invention uses a pre-trained registration model to complete the registration operation and image post-processing operation of the floating image and the fixed image. Compared with the prior art that only determines the registration result, or performs denoising before image registration, it can improve the accuracy of the image registration result and avoid noise or spatial misalignment in the registration result.

[0042] Figure 3 Flow chart of the model training method provided by the embodiment of the present invention. Figure 3 As shown, the method includes:

[0043] S210, completing the training of the image registration module in the registration model to obtain a pre-trained image registration module.

[0044] In one embodiment, the training of the image registration module is achieved by the following steps:

[0045] Step a1, obtaining a first sample set, wherein the first sample set includes a first set number of first samples, wherein the first samples include first fixed images and first floating images corresponding to each of the first fixed images, wherein the first floating images are spatial transformation results of the first fixed images, and the first floating images have the same size as the first fixed images.

[0046] The spatial transformation result may be a rotation result, a translation result, or an elastic transformation result.

[0047] In one embodiment, both the first fixed image and the first floating image are three-dimensional images, and both have the same size, i.e., h×w×d.

[0048] Step a2: using an image registration module to perform registration processing on the first fixed image and the first floating image in each of the first samples to obtain first grid sampling parameters corresponding to each of the first floating images.

[0049] For each first sample, the first fixed image and the first floating image in the current first sample are spliced ​​in the channel dimension to obtain 2×h×w×d input data, which is input into the image registration module so that the image registration module performs deformation processing on the first floating image according to the first fixed image to obtain the first grid sampling parameters corresponding to the first floating image.

[0050] When the sizes of the first fixed image and the first floating image are both h×w×d, the size of the first grid sampling parameter is h×w×d×3.

[0051] Step a3: according to the difference between the image deformation result corresponding to the first floating image and the first grid sampling parameter and the first fixed image, the network parameter optimization of the image registration module is completed to obtain a pre-trained image registration module.

[0052] According to the difference between the image deformation result corresponding to the first floating image and the first grid sampling parameter and the first fixed image, the network parameters of the image registration module are adjusted. If the current first sample is the last sample in the first sample set, the current image registration module is used as a pre-trained image registration module. It can be understood that by spatially sampling the first floating image based on the first grid sampling parameter, an image deformation result of the first floating image can be obtained, and the image deformation result is the registration image predicted by the image registration module. Ideally, the image deformation result completely matches the first fixed image. Therefore, based on the mean square error, the L1 loss function, and the difference between the image deformation result corresponding to the first floating image and the first grid sampling parameter and the first fixed image, the network parameters of the image registration module are adjusted.

[0053] This step aims to complete the preliminary training of the image registration module through the first sample set, so that it has better image registration ability, thereby reducing the difficulty of joint training of the image registration module and the image post-processing module, and improving the joint training speed of the image registration module and the image post-processing module.

[0054] In one embodiment, the image registration module is a UNet network, which is a deep learning network architecture for image segmentation.

[0055] In one embodiment, multiple image registration modules are pre-trained, and then based on a validation set, an optimal model is selected from the multiple pre-trained image registration modules as the final pre-trained image registration module.

[0056] S220, completing the training of the image post-processing module in the registration model to obtain a pre-trained image post-processing module.

[0057] In one embodiment, the training of the image post-processing module is completed by the following steps:

[0058] Step b1, obtaining a second sample set, the second sample set including a second set number of second samples, the second samples including a first image and a second image corresponding to the first image, the second image having at least one of artifacts, spatial dislocation, and noise compared to the first image.

[0059] Exemplarily, the size of the first image is h×w×d, and data enhancement is performed on the first image, for example, adding artifacts, spatial dislocation or Gaussian noise, etc., to obtain the second image. The size of the second image is also h×w×d.

[0060] Step b2: input the second image in each of the second samples into the image post-processing module to obtain a first post-processing result.

[0061] It can be understood that the first post-processing result is the image post-processing result of the second image predicted by the image post-processing module. Ideally, the first post-processing result is the same as the first image in the current second sample.

[0062] Specifically, the second image is input into the image post-processing module to obtain a first post-processing result, and the size of the first post-processing result is h×w×d.

[0063] Step b3: According to the difference between each of the first post-processing results and the corresponding first image, the network parameter optimization of the image post-processing module is completed to obtain a pre-trained image post-processing module.

[0064] This step aims to adjust the network parameters of the image post-processing module according to the difference between the first post-processing result and the first image, so as to improve the accuracy of the prediction result of the post-processing module. Specifically, the loss data corresponding to the first post-processing result and the first image is calculated using the mean square error and the L1 loss function, and the network parameters of the image post-processing module are adjusted based on the loss data.

[0065] Step b1-step b4, by using a second sample set different from the first sample set, completes the separate training of the image post-processing module, reduces the workload of the joint training of the image registration module and the image post-processing module, and improves the joint training speed of the image registration module and the image post-processing module.

[0066] In one embodiment, the image post-processing module is a UNet network, which is a deep learning network architecture for image segmentation.

[0067] In one embodiment, multiple image post-processing modules are pre-trained, and then based on a validation set, an optimal model is selected from the multiple pre-trained image post-processing modules as the final pre-trained image post-processing module.

[0068] S230, completing the joint training between the pre-trained image registration module and the pre-trained image post-processing module to update the pre-trained image registration module and the pre-trained image post-processing module, and using the registration model including the updated pre-trained image registration module and the pre-trained image post-processing module as the pre-trained registration model.

[0069] In one embodiment, the joint training between the pre-trained image registration module and the pre-trained image post-processing module is completed by the following steps:

[0070] Step c1, obtaining a third sample set, wherein the third sample set includes a second fixed image and a second floating image corresponding to each of the second fixed images, wherein the second floating image is a disturbance-added result of a spatial deformation result of the second fixed image, and the disturbance includes one or more of an artifact, a spatial misalignment, and a noise.

[0071] Compared with the second fixed image, the second floating image is deformed. The second floating image also includes at least one of an artifact, an artifact, and a spatial misalignment.

[0072] Step c2: input the second fixed image and the second floating image in each of the third samples into a pre-trained image registration module to obtain second grid sampling parameters corresponding to the second floating image.

[0073] The second grid sampling parameter can be understood as the motion data of each pixel of the second floating image compared with the corresponding pixel of the second fixed image. The second grid sampling parameter is the prediction data of the pre-trained image registration module, which can be combined with the second floating image to determine an image deformation result, and the image deformation result can be understood as the registration image of the second fixed image predicted by the pre-trained image registration module.

[0074] Step c3: determining corresponding first loss data according to the difference between the image deformation result corresponding to the second floating image and the second grid sampling parameter and the second fixed image.

[0075] Step c4, inputting the second floating image and the second grid sampling parameters into the pre-trained image post-processing module to obtain a second post-processing result, and determining the difference between the second post-processing result and the second fixed image to determine the corresponding second loss data.

[0076] Step c5: According to the first loss data and the second loss data corresponding to each of the third samples, the network parameters of the pre-trained image registration module and the network parameters of the pre-trained image post-processing module are optimized to obtain a pre-trained registration model.

[0077] The first loss data can reflect the current prediction ability of the pre-trained image registration module, and the second loss data can reflect the current prediction ability of the pre-trained image post-processing module; adjusting the network parameters of the pre-trained image registration module and the network parameters of the pre-trained image post-processing module according to the first loss data and the second loss data can improve the accuracy of the second post-processing result of the two as a whole prediction.

[0078] In one embodiment, the network parameter optimization of the pre-trained image registration module and the pre-trained image post-processing module is completed according to the mean of the first loss data and the second loss data. This embodiment can simply and quickly complete the network parameter adjustment of the pre-trained image registration module and the network parameter adjustment of the pre-trained image post-processing module.

[0079] In one embodiment, the network parameters of the pre-trained image registration module and the network parameters of the pre-trained image post-processing module are optimized according to the sum of the first loss data and the second loss data. This embodiment can simply and quickly complete the adjustment of the network parameters of the pre-trained image registration module and the network parameters of the pre-trained image post-processing module.

[0080] The technical solution provided by the embodiment of the present invention completes the training of the registration model by combining the individual training of each module with the joint training of the two modules, thereby improving the training speed of the registration model and the accuracy of the prediction results of the registration model.

[0081] Figure 4 Schematic diagram of the structure of the image registration device provided by the embodiment of the present invention. Figure 4 As shown, the device comprises:

[0082] An acquisition module 31 is used to acquire a fixed image and a floating image to be registered;

[0083] A prediction module 32, used for inputting the fixed image and the floating image into a pre-trained registration model to obtain a target deformation result of the floating image, wherein the target deformation result is a registration image of the fixed image;

[0084] Among them, the pre-trained registration model includes a pre-trained image registration module and a pre-trained image post-processing module; the pre-trained image registration module is used to perform image registration processing on the fixed image and the floating image to obtain grid sampling parameters; the pre-trained image post-processing module is used to determine the image deformation result corresponding to the floating image and the grid sampling parameters, and perform image post-processing on the image deformation result to obtain the target deformation result, and the image post-processing includes denoising and / or image restoration.

[0085] In one embodiment, the grid sampling parameters include an offset of each pixel in the floating image relative to a corresponding pixel in the fixed image.

[0086] The technical solution of the image registration device provided in the embodiment of the present invention uses a pre-trained registration model to complete the registration operation and image post-processing operation of the floating image and the fixed image. Compared with the prior art that only determines the registration result, or performs denoising before image registration, it can improve the accuracy of the image registration result and avoid noise or spatial misalignment in the registration result.

[0087] The image registration device provided in the embodiment of the present invention can execute the image registration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0088] Figure 5 Schematic diagram of the structure of the image training device provided by the embodiment of the present invention. Figure 5 As shown, the device comprises:

[0089] A first training module 41 is used to complete the training of the image registration module in the registration model to obtain a pre-trained image registration module;

[0090] A second training module 42, used to complete the training of the image post-processing module in the registration model to obtain a pre-trained image post-processing module;

[0091] The third training module 43 is used to complete the joint training between the pre-trained image registration module and the pre-trained image post-processing module to update the pre-trained image registration module and the pre-trained image post-processing module, and use the registration model including the updated pre-trained image registration module and the pre-trained image post-processing module as the pre-trained registration model.

[0092] In one embodiment, the first training module 41 is specifically used for:

[0093] Acquire a first sample set, the first sample set including a first set number of first samples, the first samples including first fixed images and first floating images corresponding to each of the first fixed images, the first floating images being spatial transformation results of the first fixed images, and the first floating images having the same size as the first fixed images;

[0094] Using an image registration module to perform registration processing on the first fixed image and the first floating image in each of the first samples to obtain first grid sampling parameters corresponding to each of the first floating images;

[0095] According to the difference between the image deformation result corresponding to the first floating image and the first grid sampling parameter and the first fixed image, the network parameter optimization of the image registration module is completed to obtain a pre-trained image registration module.

[0096] In one embodiment, the second training module 42 is specifically used for:

[0097] Acquire a second sample set, the second sample set including a second set number of second samples, the second samples including a first image and a second image corresponding to the first image, the second image having at least one of an artifact, a spatial misalignment, and noise compared to the first image;

[0098] Inputting the second image in each of the second samples into the image post-processing module to obtain a first post-processing result;

[0099] According to the difference between each of the first post-processing results and the corresponding first image, the network parameter optimization of the image post-processing module is completed to obtain a pre-trained image post-processing module.

[0100] In one embodiment, the third training module 43 includes:

[0101] an acquisition unit, configured to acquire a third sample set, wherein the third sample set includes second fixed images and second floating images corresponding to each of the second fixed images, wherein the second floating images are disturbance-added results of spatial deformation results of the second fixed images, and the disturbance includes one or more of artifacts, spatial dislocation, and noise;

[0102] A first prediction unit, configured to input the second fixed image and the second floating image in each of the third samples into a pre-trained image registration module to obtain a second grid sampling parameter corresponding to the second floating image;

[0103] A first loss data unit, configured to determine corresponding first loss data according to a difference between an image deformation result corresponding to the second floating image and the second grid sampling parameter and the second fixed image;

[0104] A second loss data unit, configured to input the second floating image and the second grid sampling parameter into the pre-trained image post-processing module to obtain a second post-processing result, and determine a difference between the second post-processing result and the second fixed image to determine a corresponding second loss data;

[0105] A network parameter adjustment unit is used to complete the network parameter optimization of the pre-trained image registration module and the pre-trained image post-processing module according to the first loss data and the second loss data corresponding to each of the third samples, so as to obtain a pre-trained registration model.

[0106] In one embodiment, the network parameter adjustment unit is specifically used to:

[0107] According to the mean of the first loss data and the second loss data, network parameter optimization of the pre-trained image registration module and the pre-trained image post-processing module is completed.

[0108] The technical solution provided by the embodiment of the present invention completes the training of the registration model by combining the individual training of each module with the joint training of the two modules, thereby improving the training speed of the registration model and the accuracy of the prediction results of the pre-trained registration model.

[0109] Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0110] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0111] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0112] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 performs the various methods and processes described above, such as an image registration method or a model training method.

[0113] In some embodiments, the image registration method or the model training method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the image registration method or the model training method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the image registration method or the model training method in any other appropriate manner (e.g., by means of firmware).

[0114] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0116] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0117] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0119] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0120] An embodiment of the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the image registration method or model training method provided in any embodiment of the present application.

[0121] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0122] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0123] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image registration method, characterized in that: include: Acquire a fixed image and a floating image to be registered; Inputting the fixed image and the floating image into a pre-trained registration model to obtain a target deformation result of the floating image, wherein the target deformation result is a registration image of the fixed image; Among them, the pre-trained registration model includes a pre-trained image registration module and a pre-trained image post-processing module; the pre-trained image registration module is used to perform image registration processing on the fixed image and the floating image to obtain grid sampling parameters; the pre-trained image post-processing module is used to determine the image deformation result corresponding to the floating image and the grid sampling parameters, and perform image post-processing on the image deformation result to obtain the target deformation result, and the image post-processing includes denoising and / or image restoration.

2. The method according to claim 1, characterized in that The grid sampling parameters include an offset of each pixel in the floating image relative to a corresponding pixel in the fixed image.

3. A model training method, characterized in that: Used to train the registration model described in claim 1 or 2, comprising: Complete the training of the image registration module in the registration model to obtain a pre-trained image registration module; Complete the training of the image post-processing module in the registration model to obtain a pre-trained image post-processing module; Complete the joint training between the pre-trained image registration module and the pre-trained image post-processing module to update the pre-trained image registration module and the pre-trained image post-processing module, and use the registration model including the updated pre-trained image registration module and the pre-trained image post-processing module as the pre-trained registration model.

4. The method according to claim 3, characterized in that The training of the image registration module in the registration model is completed to obtain a pre-trained image registration module, including: Acquire a first sample set, the first sample set including a first set number of first samples, the first samples including first fixed images and first floating images corresponding to each of the first fixed images, the first floating images being spatial transformation results of the first fixed images, and the first floating images having the same size as the first fixed images; Using an image registration module to perform registration processing on the first fixed image and the first floating image in each of the first samples to obtain first grid sampling parameters corresponding to each of the first floating images; According to the difference between the image deformation result corresponding to the first floating image and the first grid sampling parameter and the first fixed image, the network parameter optimization of the image registration module is completed to obtain a pre-trained image registration module.

5. The method according to claim 3, characterized in that: The step of completing the training of the image post-processing module in the registration model to obtain a pre-trained image post-processing module includes: Acquire a second sample set, the second sample set including a second set number of second samples, the second samples including a first image and a second image corresponding to the first image, the second image having at least one of an artifact, a spatial misalignment, and noise compared to the first image; Inputting the second image in each of the second samples into the image post-processing module to obtain a first post-processing result; According to the difference between each of the first post-processing results and the corresponding first image, the network parameter optimization of the image post-processing module is completed to obtain a pre-trained image post-processing module.

6. The method according to claim 4, characterized in that The joint training between the pre-trained image registration module and the pre-trained image post-processing module is completed, including: Acquire a third sample set, the third sample set including second fixed images and second floating images corresponding to each of the second fixed images, the second floating images being disturbance-added results of spatial deformation results of the second fixed images, the disturbance including one or more of artifacts, spatial dislocation, and noise; Inputting the second fixed image and the second floating image in each of the third samples into a pre-trained image registration module to obtain second grid sampling parameters corresponding to the second floating image; Determine corresponding first loss data according to a difference between an image deformation result corresponding to the second floating image and the second grid sampling parameter and the second fixed image; Inputting the second floating image and the second grid sampling parameter into the pre-trained image post-processing module to obtain a second post-processing result, and determining the difference between the second post-processing result and the second fixed image to determine the corresponding second loss data; According to the first loss data and the second loss data corresponding to each of the third samples, network parameter optimization of the pre-trained image registration module and the pre-trained image post-processing module is completed to obtain a pre-trained registration model.

7. The method according to claim 6, characterized in that The step of optimizing the network parameters of the pre-trained image registration module and the pre-trained image post-processing module according to the first loss data and the second loss data includes: According to the mean of the first loss data and the second loss data, network parameter optimization of the pre-trained image registration module and the pre-trained image post-processing module is completed.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image registration method described in claim 1 or 2, or the model training method described in any one of claims 3-7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image registration method described in claim 1 or 2 or the model training method described in any one of claims 3-7 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the image registration method described in claim 1 or 2 or the model training method described in any one of claims 3-7.

Images