Cross-modal image registration method and device, equipment, storage medium and program product
Through iterative training of alternating methods of deformation networks and feature translation networks, the problem of insufficient registration accuracy of cross-modal medical images in image translation technology is solved, and higher registration accuracy and detailed information retention are achieved.
Patent Information
- Application Number
- CN202410997787.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-22
AI Technical Summary
In the prior art, when cross-modal medical image registration is achieved through image translation technology, it is difficult to retain detailed information, resulting in the registration accuracy being unable to meet the requirements.
A cross-modal image registration method including deformation networks and feature translation networks is adopted. Iterative training is used to alternate training when the parameters of each network are frozen, and the feature translation network is used to align the features of different mode images to avoid loss of image detail information.
A higher cross-modal image registration accuracy is achieved, and fewer model parameters and computing power are used to avoid loss of image detail information.
Smart Images

Figure CN119006540A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical image technology, and in particular to a cross-modality image registration method, device, equipment, storage medium and program product. Background Art
[0002] With the development of technology, cross-modal medical image registration has received more and more attention. Specifically, given a pair of images of different modalities, the model predicts the registration field between the two images. The registered image is then fused with the target image, thus providing valuable supplementary information for actual clinical applications.
[0003] In the existing technology, cross-modal medical image registration can be achieved through image translation technology. The main idea of this technology is to use the image translation framework to transform the cross-modal registration problem into a simple single-modal registration problem. Specifically, at the image level, the image translation-based method uses a generative adversarial network or its variants to translate the source modality image into the corresponding image of the target modality, and then applies a mature single-modality registration algorithm in the target modality.
[0004] However, although image translation technology can simplify the cross-modal registration problem, a major drawback of image translation is that it is difficult to retain all the detailed information, including the detailed information that is beneficial to the registration task. The accuracy of registration is often limited by the quality of the translated image, so the current translation effect cannot meet the accuracy requirements of registration. Summary of the invention
[0005] The present invention provides a cross-modal image registration method, apparatus, device, storage medium and program product, which are used to solve the problem in the prior art that cross-modal registration achieved by image translation technology cannot meet the registration accuracy requirement.
[0006] The present invention provides a cross-modal image registration method, comprising: acquiring first training data; iteratively training a registration network based on the first training data until the network converges; wherein the registration network is used to achieve cross-modal registration of images, the registration network comprises a deformation network and a feature translation network, and the feature translation network is arranged in an encoder of the deformation network; the deformation network is used to achieve modal deformation of the image, and the feature translation network is used to align image features of images of different modalities; the iterative training comprises training the feature translation network while freezing the network parameters of the deformation network; and training the deformation network while freezing the network parameters of the feature translation network.
[0007] According to a cross-modal image registration method provided by the present invention, the encoder of the deformation network includes multiple feature layers, the feature translation network is arranged after the target feature layer of the deformation network, and the target feature layer is one of the multiple feature layers.
[0008] According to a cross-modal image registration method provided by the present invention, the multiple feature layers include a first feature layer, a second feature layer, a third feature layer and a fourth feature layer; the registration network includes a first registration network, a second registration network, a third registration network and a fourth registration network; the feature translation network of the first registration network is arranged after the first feature layer, the feature translation network of the second registration network is arranged after the second feature layer, the feature translation network of the third registration network is arranged after the third feature layer, and the feature translation network of the fourth registration network is arranged after the fourth feature layer.
[0009] According to a cross-modal image registration method provided by the present invention, the method also includes: acquiring second training data; iteratively training a dynamic routing network based on the second training data until the network converges; wherein the dynamic routing network includes the first registration network, the second registration network, the third registration network and the fourth registration network, and the dynamic routing network is used to determine the registration effect index of the input data and each registration network.
[0010] According to a cross-modal image registration method provided by the present invention, the method also includes: acquiring an image to be registered and a target image, wherein the image to be registered is a first modality image, and the target image is a second modality image; inputting the image to be registered and the target image into the dynamic routing network to determine a target registration network from the registration network, wherein the target registration network is a registration network with the highest registration effect index; inputting the image to be registered and the target image into the target registration network to obtain a registered image, wherein the registered image is the first modality image after registration.
[0011] According to a cross-modal image registration method provided by the present invention, the iterative training of the registration network based on the first training data until the network converges includes: randomly initializing the network parameters of the feature translation network; and pre-training the deformation network while freezing the network parameters of the feature translation network to determine the initialization parameters of the deformation network.
[0012] The present invention also provides a cross-modal image registration device, comprising the following modules: an acquisition module and a training module; the acquisition module is used to acquire first training data; the training module is used to iteratively train a registration network based on the first training data until the network converges; wherein the registration network is used to achieve cross-modal registration of images, the registration network includes a deformation network and a feature translation network, and the feature translation network is arranged in an encoder of the deformation network; the deformation network is used to achieve modal deformation of the image, and the feature translation network is used to align image features of images of different modalities; the iterative training includes training the feature translation network while freezing the network parameters of the deformation network; and training the deformation network while freezing the network parameters of the feature translation network.
[0013] According to a cross-modal image registration device provided by the present invention, the encoder of the deformation network includes multiple feature layers, the feature translation network is arranged after the target feature layer of the deformation network, and the target feature layer is one of the multiple feature layers.
[0014] According to a cross-modal image registration device provided by the present invention, the multiple feature layers include a first feature layer, a second feature layer, a third feature layer and a fourth feature layer; the registration network includes a first registration network, a second registration network, a third registration network and a fourth registration network; the feature translation network of the first registration network is arranged after the first feature layer, the feature translation network of the second registration network is arranged after the second feature layer, the feature translation network of the third registration network is arranged after the third feature layer, and the feature translation network of the fourth registration network is arranged after the fourth feature layer.
[0015] According to a cross-modal image registration device provided by the present invention, the acquisition module is used to acquire second training data; the training module is used to iteratively train the dynamic routing network based on the second training data until the network converges; wherein the dynamic routing network includes the first registration network, the second registration network, the third registration network and the fourth registration network, and the dynamic routing network is used to determine the registration effect index of the input data and each registration network.
[0016] A cross-modal image registration device provided by the present invention also includes a prediction module; the acquisition module is used to acquire the image to be registered and the target image, the image to be registered is the first modality image, and the target image is the second modality image; the prediction module is used to input the image to be registered and the target image into the dynamic routing network to determine the target registration network from the registration network, and the target registration network is the registration network with the highest registration effect index; the image to be registered and the target image are input into the target registration network to obtain a registered image, and the registered image is the first modality image after registration.
[0017] According to a cross-modal image registration device provided by the present invention, the training module is used to randomly initialize the network parameters of the feature translation network; while freezing the network parameters of the feature translation network, the deformable network is pre-trained to determine the initialization parameters of the deformable network.
[0018] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the cross-modal image registration method as described above is implemented.
[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the cross-modal image registration method as described in any one of the above is implemented.
[0020] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned cross-modal image registration methods.
[0021] The cross-modal image registration method, device, equipment, storage medium and program product provided by the present invention can obtain first training data; iteratively train the registration network based on the first training data until the network converges; wherein the registration network is used to realize cross-modal registration of images, the registration network includes a deformation network and a feature translation network, and the feature translation network is set in the encoder of the deformation network; the deformation network is used to realize the modal deformation of the image, and the feature translation network is used to align the image features of different modal images; the iterative training includes training the feature translation network while freezing the network parameters of the deformation network; training the deformation network while freezing the network parameters of the feature translation network. Through this scheme, image registration of different modalities can be realized based on feature translation. Compared with image translation, it can not only realize modal alignment with fewer model parameters and less computing power, but also avoid the problem of loss of image detail information, thereby achieving higher registration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0023] Figure 1 This is one of the flow charts of the cross-modal image registration method provided by the present invention; Figure 2 It is a schematic diagram of the training process of the registration network provided by the present invention; Figure 3 It is a schematic diagram of the setting position of the feature translation network provided by the present invention; Figure 4 It is a schematic diagram of the training process of the dynamic routing network provided by the present invention; Figure 5 This is the second flow chart of the cross-modal image registration method provided by the present invention; Figure 6 is a schematic structural diagram of a cross-modal image registration device provided by the present invention; Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0025] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0026] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0027] In order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and order of execution.
[0028] The embodiments of the present application describe some exemplary embodiments for the purpose of explanation. It should be understood that the present application can be implemented in other ways that are not specifically shown in the drawings.
[0029] like Figure 1 As shown, an embodiment of the present application provides a cross-modal image registration method, which can be applied to a cross-modal image registration device. The cross-modal image registration method can include S101-S102: S101. A cross-modal image registration apparatus obtains first training data.
[0030] The first training data may include a first modality image and a second modality image, the first modality image may be a magnetic resonance imaging (MRI), and the second modality image may be a computed tomography (CT) image.
[0031] It should be noted that the above-mentioned first modality image and second modality image are both two-dimensional images.
[0032] S102: The cross-modal image registration apparatus iteratively trains a registration network based on the first training data until the network converges.
[0033] Among them, the above-mentioned registration network is used to realize cross-modal registration of images, and the registration network includes a deformation network and a feature translation network, and the feature translation network is set in the encoder of the deformation network; the deformation network is used to realize the modal deformation of the image, and the feature translation network is used to align the image features of images of different modalities; the iterative training includes training the feature translation network while freezing the network parameters of the deformation network; and training the deformation network while freezing the network parameters of the feature translation network.
[0034] Specifically, Figure 2 As shown, the cross-modal image registration device can train the deformation network and the feature translation network based on the iterative framework. In the nth training, the cross-modal image registration device can train the sub-data in the first training data Input the registration network for network training. At this time, the network parameters of the feature translation network in the registration network are frozen. After the nth training, the network parameters of the deformation network are updated. In the n+1th training, the sub-data in the first training data can be The registration network is input for network training. At this time, the network parameters of the deformation network in the registration network are frozen. After the n+1th training, the network parameters of the feature translation network are updated, where m represents the image of the first modality of the target object, and f represents the image of the second modality of the target object. The deformation network and the feature translation network can be iteratively trained in the above manner until they are fully converged.
[0035] Optionally, taking the first modality image as an MRI image and the second modality image as a CT image as an example, the loss function of the feature translation network can be expressed as: ; Among them, the confrontation Promotes two feature domains and Alignment, translator and the discriminator Aims to transform features from Domain translation to Domain, Translator and the discriminator The effect is opposite. Used to ensure that the translation process maintains the characteristic content, regular term Used to reduce the risk of multiple solutions. and are hyperparameters and are set to 10 and 5 by default.
[0036] Optionally, the cross-modal image registration device may randomly initialize the network parameters of the feature translation network; and pre-train the deformable network while freezing the network parameters of the feature translation network to determine the initialization parameters of the deformable network.
[0037] Specifically, the cross-modal image registration device can first pre-train the deformable network to determine the initialization parameters of the deformable network. That is, given an image pair of two modalities: a source image m and a target image f, the network parameters of the feature translation network are first randomly initialized, and then the deformable network is pre-trained based on the source image m and the target image f while freezing the network parameters of the feature translation network, thereby determining the initialization parameters of the deformable network.
[0038] Optionally, the loss function of the pre-trained deformable network can be defined as: ; Among them, the measurement items Used to metrically register images The similarity between the target image f and the regularization term For enhanced registration field smoothness and spatial consistency. is a hyperparameter and is set to 0.01 by default.
[0039] Optionally, the encoder of the deformable network includes multiple feature layers, the feature translation network is arranged after the target feature layer of the deformable network, and the target feature layer is one of the multiple feature layers.
[0040] For example, Figure 3 As shown, the encoder of the deformation network may include a first feature layer , the second feature layer , the third characteristic layer , the fourth characteristic layer , the feature translation network can be set behind any feature layer of the deformation network. When the feature translation network is set at the second feature layer After that, the target feature layer is the second feature layer .
[0041] Optionally, the multiple feature layers include a first feature layer, a second feature layer, a third feature layer and a fourth feature layer; the registration network includes a first registration network, a second registration network, a third registration network and a fourth registration network; the feature translation network of the first registration network is set after the first feature layer, the feature translation network of the second registration network is set after the second feature layer, the feature translation network of the third registration network is set after the third feature layer, and the feature translation network of the fourth registration network is set after the fourth feature layer.
[0042] Specifically, the feature translation network is set after different feature layers to achieve different registration effects. In order to achieve the optimal registration effect, the cross-modal image registration device can set the feature translation network after different feature layers and train them separately to obtain the first registration network, the second registration network, the third registration network and the fourth registration network.
[0043] Optionally, the cross-modal image registration device can obtain second training data; iteratively train the dynamic routing network based on the second training data until the network converges; wherein the dynamic routing network includes the first registration network, the second registration network, the third registration network and the fourth registration network, and the dynamic routing network is used to determine the registration effect index of the input data and each registration network.
[0044] Specifically, after obtaining the first registration network, the second registration network, the third registration network, and the fourth registration network, the cross-modal image registration device can select the registration network that can achieve the best registration effect for the input data by training the dynamic routing network. Figure 4 As shown, the cross-modal image registration device can input the image pair into the dynamic routing network to obtain the registration effect indicators of the four registration networks, and the registration network with the highest matching degree with the input data can be determined through the registration effect indicators.
[0045] Optionally, the loss function of the dynamic routing network is a cross entropy loss function, which can be expressed as: ; in, It is through The mutual information between the deformed image and the target image after translation by the layer feature translation network, Normalized weights predicted for a dynamic routing network.
[0046] In an embodiment of the present application, image registration of different modalities can be achieved based on feature translation. Compared with image translation, not only can the alignment of modalities be achieved with fewer model parameters and less computing power, but also the problem of loss of image detail information can be avoided, thereby achieving higher registration accuracy.
[0047] Alternatively, if Figure 5 As shown, the embodiment of the present application provides a cross-modal image registration method further including S103-S105: S103: The cross-modal image registration device obtains the image to be registered and the target image.
[0048] The image to be registered is a first modality image, and the target image is a second modality image.
[0049] S104: The cross-modal image registration device inputs the image to be registered and the target image into the dynamic routing network to determine a target registration network from the registration network.
[0050] The target registration network is a registration network with the highest registration effect index.
[0051] Since image pairs of different individuals often have individual differences, even for the same individual, image slices may differ in texture and structure. Therefore, in order to improve the registration effect, the cross-modal image registration device can adaptively select the feature transformation branch that best matches the input data through a dynamic router, thereby assisting the model in predicting a more accurate registration field.
[0052] S105. The cross-modal image registration device inputs the image to be registered and the target image into the target registration network to obtain a registered image.
[0053] The registered image is a registered first modality image.
[0054] Specifically, continue to refer to Figure 3 , the cross-modal image registration device can apply the predicted registration field to the image to be registered m for spatial transformation to obtain the registered image , the registered image It can be used for subsequent image fusion of different modalities to provide supplementary information for clinical applications.
[0055] In the embodiment of the present application, since the registration network can be dynamically selected according to the sample input, the prediction effect of the registration field can be improved.
[0056] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.
[0057] The cross-modal image registration method provided in the embodiment of the present application may be executed by a cross-modal image registration device, or a control module for cross-modal image registration in the cross-modal image registration device. In the embodiment of the present application, the cross-modal image registration method performed by the cross-modal image registration device is taken as an example to illustrate the cross-modal image registration device provided in the embodiment of the present application.
[0058] It should be noted that the embodiment of the present application can divide the functional modules of the cross-modal image registration device according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. Optionally, the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.
[0059] like Figure 6 As shown, an embodiment of the present application provides a cross-modal image registration device 600. The cross-modal image registration device 600 includes: an acquisition module 601 and a training module 602. The acquisition module 601 can be used to acquire first training data; the training module 602 is used to iteratively train the registration network based on the first training data until the network converges; wherein the registration network is used to achieve cross-modal registration of images, the registration network includes a deformation network and a feature translation network, and the feature translation network is arranged in the encoder of the deformation network; the deformation network is used to achieve modal deformation of the image, and the feature translation network is used to align image features of images of different modalities; the iterative training includes training the feature translation network while freezing the network parameters of the deformation network; and training the deformation network while freezing the network parameters of the feature translation network.
[0060] Optionally, the encoder of the deformable network includes multiple feature layers, the feature translation network is arranged after the target feature layer of the deformable network, and the target feature layer is one of the multiple feature layers.
[0061] Optionally, the multiple feature layers include a first feature layer, a second feature layer, a third feature layer and a fourth feature layer; the registration network includes a first registration network, a second registration network, a third registration network and a fourth registration network; the feature translation network of the first registration network is set after the first feature layer, the feature translation network of the second registration network is set after the second feature layer, the feature translation network of the third registration network is set after the third feature layer, and the feature translation network of the fourth registration network is set after the fourth feature layer.
[0062] Optionally, the acquisition module 601 is used to acquire second training data; the training module 602 is used to iteratively train the dynamic routing network based on the second training data until the network converges; wherein the dynamic routing network includes the first registration network, the second registration network, the third registration network and the fourth registration network, and the dynamic routing network is used to determine the registration effect index of the input data and each registration network.
[0063] Optionally, the cross-modal image registration device 600 may also include a prediction module 603; the acquisition module 601 is used to acquire the image to be registered and the target image, the image to be registered is the first modality image, and the target image is the second modality image; the prediction module 603 is used to input the image to be registered and the target image into the dynamic routing network to determine the target registration network from the registration network, and the target registration network is the registration network with the highest registration effect index; the image to be registered and the target image are input into the target registration network to obtain a registered image, and the registered image is the first modality image after registration.
[0064] Optionally, the training module 602 is used to randomly initialize network parameters of the feature translation network; and when the network parameters of the feature translation network are frozen, pre-train the deformable network to determine the initialization parameters of the deformable network.
[0065] In an embodiment of the present application, image registration of different modalities can be achieved based on feature translation. Compared with image translation, not only can the alignment of modalities be achieved with fewer model parameters and less computing power, but also the problem of loss of image detail information can be avoided, thereby achieving higher registration accuracy.
[0066] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the cross-modal image registration method, the method comprising: obtaining first training data; iteratively training the registration network based on the first training data until the network converges; wherein the registration network is used to realize cross-modal registration of the image, the registration network comprises a deformation network and a feature translation network, the feature translation network is set in the encoder of the deformation network; the deformation network is used to realize the modal deformation of the image, and the feature translation network is used to align the image features of different modal images; the iterative training comprises training the feature translation network under the condition of freezing the network parameters of the deformation network; and training the deformation network under the condition of freezing the network parameters of the feature translation network.
[0067] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0068] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the cross-modal image registration method provided by the above-mentioned methods, and the method includes: obtaining first training data; iteratively training a registration network based on the first training data until the network converges; wherein the registration network is used to achieve cross-modal registration of images, and the registration network includes a deformation network and a feature translation network, and the feature translation network is arranged in the encoder of the deformation network; the deformation network is used to achieve modal deformation of the image, and the feature translation network is used to align image features of images of different modalities; the iterative training includes training the feature translation network while freezing the network parameters of the deformation network; and training the deformation network while freezing the network parameters of the feature translation network.
[0069] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the cross-modal image registration method provided by the above-mentioned methods, the method comprising: obtaining first training data; iteratively training a registration network based on the first training data until the network converges; wherein the registration network is used to achieve cross-modal registration of images, the registration network comprises a deformation network and a feature translation network, and the feature translation network is arranged in an encoder of the deformation network; the deformation network is used to achieve modal deformation of the image, and the feature translation network is used to align image features of images of different modalities; the iterative training comprises training the feature translation network while freezing the network parameters of the deformation network; and training the deformation network while freezing the network parameters of the feature translation network.
[0070] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0071] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A cross-modal image registration method, characterized in that: include: Acquire first training data; Iteratively training the registration network based on the first training data until the network converges; The registration network is used to realize cross-modal registration of images, and the registration network includes a deformation network and a feature translation network, and the feature translation network is set in the encoder of the deformation network; the deformation network is used to realize the modal deformation of the image, and the feature translation network is used to align the image features of images of different modalities; The iterative training includes training the feature translation network while freezing the network parameters of the deformable network; and training the deformable network while freezing the network parameters of the feature translation network.
2. The cross-modal image registration method according to claim 1, characterized in that: The encoder of the deformation network includes multiple feature layers, and the feature translation network is arranged after the target feature layer of the deformation network, and the target feature layer is one of the multiple feature layers.
3. The cross-modal image registration method according to claim 2, characterized in that: The plurality of feature layers include a first feature layer, a second feature layer, a third feature layer and a fourth feature layer; The registration network includes a first registration network, a second registration network, a third registration network and a fourth registration network; The feature translation network of the first registration network is set after the first feature layer, the feature translation network of the second registration network is set after the second feature layer, the feature translation network of the third registration network is set after the third feature layer, and the feature translation network of the fourth registration network is set after the fourth feature layer.
4. The cross-modal image registration method according to claim 3, characterized in that: The method further comprises: Acquire second training data; Iteratively training the dynamic routing network based on the second training data until the network converges; The dynamic routing network includes the first registration network, the second registration network, the third registration network and the fourth registration network, and the dynamic routing network is used to determine the registration effect index between the input data and each registration network.
5. The cross-modal image registration method according to claim 4, characterized in that: The method further comprises: Acquire an image to be registered and a target image, wherein the image to be registered is a first modality image and the target image is a second modality image; Inputting the image to be registered and the target image into the dynamic routing network to determine a target registration network from the registration networks, wherein the target registration network is a registration network with the highest registration effect index; The image to be registered and the target image are input into the target registration network to obtain a registered image, where the registered image is the first modality image after registration.
6. The cross-modal image registration method according to any one of claims 1 to 5, characterized in that: The iterative training of the registration network based on the first training data until the network converges includes: Randomly initializing network parameters of the feature translation network; When the network parameters of the feature translation network are frozen, the deformable network is pre-trained to determine the initialization parameters of the deformable network.
7. A cross-modal image registration device, characterized in that: include: Acquisition module and training module; The acquisition module is used to acquire first training data; The training module is used to iteratively train the registration network based on the first training data until the network converges; Among them, the registration network is used to achieve cross-modal registration of images, the registration network includes a deformation network and a feature translation network, and the feature translation network is set in the encoder of the deformation network; the deformation network is used to achieve modal deformation of the image, and the feature translation network is used to align image features of images of different modalities; the iterative training includes training the feature translation network while freezing the network parameters of the deformation network; and training the deformation network while freezing the network parameters of the feature translation network.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the cross-modal image registration method according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the cross-modal image registration method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the cross-modal image registration method according to any one of claims 1 to 6 is implemented.
Citation Information
Cited By
Registration method, device, equipment, medium and product
CN121564050A
