Gradient surgery based self-distillation image registration method, system and medium

Abstract

The application discloses a kind of self-distillation image registration methods, systems and media based on gradient surgery, which is designed by self-distillation registration paradigm, so that the teacher network is directly copied from the student network, without pre-design and careful selection;And is removed after the end of the training phase, without increasing any additional parameters and inference time, the whole architecture is more lightweight, reduces the difficulty of model deployment;Again introduce gradient surgery optimization strategy, by projecting the conflicting gradient onto the normal plane of the dominant gradient, solve the potential gradient conflict problem between student and teacher network, so that the update direction of student network gradually converges to the update direction of teacher network during the training process, so that the registration performance ultimately approximates the registration performance of teacher network, also further ensures the effectiveness of self-distillation paradigm, ensures the effectiveness of image registration.The technical scheme can be especially applied to the registration of medical images.

Description

Technical Field

[0001] This invention relates to the technical fields of computer-aided planning of surgical procedures, medical image processing, computer vision, and knowledge distillation. Specifically, it relates to a self-distillation image registration method, system, and medium based on gradient surgery, and particularly to a rapid and lightweight self-distillation registration method for medical images. Background Technology

[0002] Unsupervised medical image registration techniques based on deep learning have significantly accelerated traditional methods, compressing registration time to the second level. In recent years, cascaded networks and multi-resolution pyramid networks, with their core idea of ​​coarse-to-fine registration, have effectively improved registration accuracy and have gradually become research hotspots. However, cascaded networks have a large number of parameters and require multiple iterations during training; multi-resolution pyramid networks can use end-to-end training strategies, resulting in fewer model parameters and shorter training and inference times compared to cascaded networks, but their registration performance often falls short of that of cascaded networks. Therefore, achieving a balance between computational complexity and registration accuracy remains a challenge.

[0003] Some existing techniques introduce knowledge distillation into registration networks, enabling student networks with fewer parameters to achieve performance comparable to teacher networks with more parameters. For example, see Reference 1. LDR uses adversarial distillation loss to transfer knowledge from the teacher network to the student network. Other typical methods, such as References 2 and 3, utilize the concept of self-distillation to train low-resolution layers under the guidance of high-resolution layers. However, these existing techniques still have some problems.

[0004] First, they require pre-design or careful selection of suitable teacher networks, and their knowledge distillation can only be applied to network structures with multi-resolution layers, lacking generality. Second, they only constrain the similarity between the deformation fields of teacher and student networks through simple mean square error (MSE) loss, ignoring potential gradient conflicts between student and teacher networks, and there is still room for improvement in registration accuracy.

[0005] Designing a universal distillation paradigm that enables higher registration accuracy across a wide variety of student networks, and reducing potential gradient conflicts between student and teacher networks to ensure effective distillation, remain challenging problems that current technologies cannot solve.

[0006] References:

[0007] [1] Tran, MQ, Do, T., Tran, H., Tjiputra, E., Tran, QD, Nguyen, A.: Lightweight deformable registration using adversarial learning with distilling knowledge. IEEE transactions on medical imaging 41(6), 1443–1453(2022).

[0008] [2]Hu, B., Zhou, S., Xiong, Z., Wu, F.: Cross-resolution distillation for efficient 3D medical image registration. IEEE Transactions on Circuits andSystems for Video Technology 32(10),7269–7283(2022).

[0009] [3]Zhou, S., Hu, B., Xiong, Z., Wu, F.: Self-distilled hierarchical network for unsupervised deformable image registration. IEEE transactions on medical imaging (2023). Summary of the Invention

[0010] To address the shortcomings of existing technologies, the technical problem to be solved by this invention is to provide a self-distillation image registration method, system, and medium based on gradient surgery, thereby achieving the universality of the distillation paradigm and improving image registration performance by utilizing gradient surgery without increasing the number of additional parameters or inference time.

[0011] The technical solution provided by this invention is as follows:

[0012] On the one hand, a self-distillation image registration method based on gradient surgery includes:

[0013] (1) Construct an improved self-distillation registration network;

[0014] The teacher network in the self-distillation registration network is replaced by a copied student network and concatenated after the student network. The teacher network takes the deformed image and fixed image output by the student network as input, and uses the deformation field obtained by adding the residual deformation field output by the teacher network and the deformation field of the student network point by point as the deformation field of the teacher network, thus obtaining the improved self-distillation registration network.

[0015] (2) The improved self-distillation registration network is trained to obtain the trained improved self-distillation registration network;

[0016] (3) Input the image to be registered into the trained improved self-distillation registration network to perform image registration.

[0017] Teacher networks do not need to be redesigned or carefully selected; instead, student networks can be directly copied, which improves the versatility of self-distillation registration networks.

[0018] After training, the improved self-distillation registration network removes the teacher network and retains only the lightweight student network. Compared to the simple student network without the self-distillation paradigm, it does not increase the number of parameters or introduce additional inference time. However, the registration accuracy of the lightweight improved self-distillation registration network is higher than that of the simple student network itself.

[0019] Furthermore, the teacher network in the trained self-distillation registration network is removed to obtain a lightweight improved self-distillation registration network.

[0020] After training, the teacher network is removed, leaving only the lightweight student network. Compared to the simple student network without the introduction of the self-distillation paradigm, this approach does not add any additional parameters or introduce additional inference time, but the registration accuracy is higher than that of the simple student network.

[0021] Furthermore, the training of the improved self-distillation registration network is based on gradient surgery to iteratively optimize and train the improved self-distillation registration network, thereby obtaining a trained gradient surgery-based self-distillation registration network.

[0022] During training, each iteration checks whether the gradient update direction of the student network gS conflicts with the dominant gradient. If a gradient conflict exists, the gradient of the student network is projected onto the normal plane of the dominant gradient; if no gradient conflict exists, the gradient gS of the student network remains unchanged.

[0023] A gradient surgery optimization strategy is introduced: by projecting conflicting gradients onto the normal plane of the dominant gradient, gradient conflicts are resolved, ensuring the effectiveness of the self-distillation paradigm.

[0024] Furthermore, the current training iteration is compared with the threshold. If it is less than the threshold, it is in the early training stage, and the dominant gradient is the gradient gT of the teacher network. If it is greater than or equal to the threshold, it is in the later training stage, and the dominant gradient is the gradient gT of the teacher network and the distillation gradient gD. That is, it is determined in turn whether the update direction of the gradient gS of the student network will conflict with the gradient gT of the teacher network and the distillation gradient gD.

[0025] The phased processing approach ensures the accuracy of the teacher network's deformation field used for distillation by setting a threshold at which the similarity loss of the teacher network almost converges. Only when the teacher network's deformation field is sufficiently accurate is the distillation gradient included in the dominant gradient, guaranteeing that the guidance received by the student network is accurate and reliable.

[0026] Furthermore, whether the gradient update direction of the student network gS conflicts with the dominant gradient refers to whether the product of the student network gradient gS and the dominant gradient is greater than 0. If it is greater than 0, then there is a gradient conflict.

[0027] The projection of the student network's gradient onto the normal plane of the dominant gradient means that, in the early training phase, if the product of gS and gT is greater than 0, then gS = gS - [(gS·gT) / ||gT|| 2 In the later training phase, if the product of gS and gT is greater than 0, then gS = gS - [(gS·gT) / ||gT|| 2 ]·gT, or the product of gS and gT is less than or equal to 0, and the product of gS and gD is greater than 0, then gS=gS-[(gS·gD) / ||gD|| 2 ]·gD.

[0028] Furthermore, the loss functions used when training the improved self-distillation registration network are as follows:

[0029] Similarity loss:

[0030] Regularization loss:

[0031] Distillation loss:

[0032] Where Lsim_stu and Lsim_tea represent the similarity loss of the student network and the teacher network, respectively, and Lreg_stu and Lreg_tea represent the regularization loss of the student network and the teacher network, respectively. and Let Iw_stu and Iw_tea represent the deformation fields of the student network and teacher network, respectively, and let Ldis represent the images after deformation of the moving image by applying the deformation fields of the student network and teacher network. and These represent point-by-point gradient calculation operations on the deformation fields of students and teachers, respectively. Represents the square of the L2 norm;

[0033] The gradient reflects the rate of change between each point in the deformation field and its neighboring points. Minimizing the gradient is to make the rate of change between points as small as possible, thus maintaining the smoothness of the deformation field.

[0034] The total loss for each term is obtained from the similarity loss and the regularization loss:

[0035] Total loss of the student network: LS = Lsim_stu + λr Lreg_stu

[0036] Total loss of the teacher network: LT = Lsim_tea + λr Lreg_tea

[0037] Total distillation loss: LD = λd Ldis

[0038] Where λr represents the loss weights of the student network and the teacher network, and λd represents the total loss weight of the distillation, λr = 0.01 and λd = 0.001.

[0039] Because we need to consider the order of magnitude of multiple losses, we weight each loss. The weight values ​​are set based on multiple experiments. The standard for selecting these weight coefficients is that each loss function can decrease and converge normally, and the network can be trained normally. If the weights are set improperly, the loss function may oscillate, surge, or fail to decrease, and it will not be able to converge normally, thus making it impossible to train the network.

[0040] After replacing the teacher network with a copied student network, during gradient backpropagation and updates in the training process, the distillation gradient gD is obtained from the distillation loss. The backpropagation of distillation gD flows through both the student and teacher networks, updating their parameters. The purpose of this parameter update is to make the student's deformation field as similar to the teacher's deformation field as possible, thereby minimizing the total distillation loss. When the total distillation loss is minimized, it means that the deformation field generated by the student network is most "similar" to the deformation field generated by the teacher, thus allowing the teacher network's deformation field to guide the student network's deformation field.

[0041] The total distillation loss is calculated by multiplying the distillation loss by a weight coefficient to obtain the total distillation loss, which is used for gradient backpropagation and updates, considering that the network needs to calculate many loss functions at once. The magnitudes of the loss functions need to be kept balanced so that the loss can decrease and converge and the network can be trained normally.

[0042] The student network uses either VoxelMorph based on convolutional neural networks or TransMorph based on Transformers.

[0043] Secondly, a gradient surgery-based self-distillation image registration system includes:

[0044] Improved self-distillation registration network module: The teacher network in the self-distillation registration network is replaced by a copied student network and concatenated after the student network; the teacher network takes the deformed image and fixed image output by the student network as input, and uses the deformation field obtained by adding the residual deformation field output by the teacher network and the deformation field of the student network point by point as the deformation field of the teacher network, thus obtaining the improved self-distillation registration network.

[0045] Network training: The improved self-distillation registration network is trained to obtain the improved self-distillation registration network;

[0046] Image registration: Input the image to be registered into the improved distillation registration network to perform image registration.

[0047] The teacher network is removed from the trained self-distillation registration network to obtain a lightweight improved self-distillation registration network.

[0048] Furthermore, the training of the improved self-distillation registration network is based on gradient surgery to iteratively optimize and train the improved self-distillation registration network, thereby obtaining a trained gradient surgery-based self-distillation registration network.

[0049] During training, each iteration checks whether the gradient update direction of the student network gS conflicts with the dominant gradient. If a gradient conflict exists, the gradient of the student network is projected onto the normal plane of the dominant gradient; if no gradient conflict exists, the gradient gS of the student network remains unchanged.

[0050] Thirdly, a computer storage medium storing a computer program, said computer program being invoked by a processor for execution:

[0051] The steps of the above gradient surgery-based self-distillation image registration method.

[0052] Fourthly, a lightweight method for medical image registration, wherein the method applies the gradient-based self-distillation image registration method described above, and the method is applied to intelligent planning of medical image-guided surgery.

[0053] Beneficial effects

[0054] This invention provides a self-distillation image registration method, system, and medium based on gradient surgery. This registration method designs a self-distillation registration paradigm, allowing the teacher network to be directly copied from the student network without pre-design or careful selection. The teacher network is removed after the training phase, without adding any extra parameters or inference time, resulting in a lighter architecture and reduced model deployment difficulty. Furthermore, a gradient surgery optimization strategy is introduced, projecting conflicting gradients onto the normal plane of the dominant gradient to resolve potential gradient conflicts between the student and teacher networks. This ensures that during training, the update direction of the student network gradually converges with that of the teacher network, ultimately approaching the registration performance of the teacher network. This further ensures the effectiveness of the self-distillation paradigm, thus guaranteeing the effectiveness of image registration.

[0055] Furthermore, the technical solution provided by this invention can significantly reduce the time cost of image registration in computer-aided planning of surgical procedures, improve the efficiency of computer-aided planning in surgical procedures, and make it more convenient and simple to use. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the overall implementation process of the registration method of the technical solution of the present invention;

[0057] Figure 2 This is a schematic diagram of the training process using gradient surgery in the technical solution of the present invention;

[0058] Figure 3 This is a schematic diagram of gradient conflict projection processing;

[0059] Figure 4 This diagram illustrates the registration effect on different anatomical structures using the technical solution of this invention and existing technologies. Detailed Implementation

[0060] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments:

[0061] Example 1

[0062] like Figure 1 As shown, a self-distillation image registration method based on gradient surgery includes:

[0063] (1) Construct an improved self-distillation registration network;

[0064] The teacher network in the self-distillation registration network is replaced by a copied student network and concatenated after the student network. The teacher network takes the deformed image and fixed image output by the student network as input, and uses the deformation field obtained by adding the residual deformation field output by the teacher network and the deformation field of the student network point by point as the deformation field of the teacher network, thus obtaining the improved self-distillation registration network.

[0065] (2) The improved self-distillation registration network is trained to obtain the trained improved self-distillation registration network;

[0066] Two 3D medical images are randomly selected from the training dataset. One image is used as the fixed image to be aligned (denoted as If), and the other image is used as the moving image to be deformed (denoted as Im). These images serve as the input information for the improved self-distillation registration network.

[0067] The student network learns the spatial correspondence between Im and If, and predicts the deformation field of the student network.

[0068] application The image of the student network is warped to obtain the warped image of the student network, denoted as Iw_stu.

[0069] Inputting Iw_stu and If into the teacher network, the residual deformation field is predicted.

[0070] The residual deformation field is added point-by-point to the deformation field of the student network to obtain the deformation field of the teacher network.

[0071] application The image of Im is warped to obtain the warped image of the teacher network, denoted as Iw_tea;

[0072] The spatial correspondence between two 3D medical images in the training dataset is known, which is equivalent to knowing the deformation field between the two images. That is, the deformation field can be used to deform one image and then register it with the other image. During the training process, the deformation field output by the improved self-distillation registration network is made to infinitely approximate the spatial correspondence between the images in the training dataset, so that the deformation field output by the trained improved self-distillation registration network can accurately register the input image.

[0073] (3) Input the image to be registered into the trained improved self-distillation registration network to perform image registration.

[0074] Teacher networks do not need to be redesigned or carefully selected; instead, student networks can be directly copied, which improves the versatility of self-distillation registration networks.

[0075] If, after the improved self-distillation registration network has been trained, the teacher network is removed and only the lightweight student network is retained, there is no increase in the number of parameters or the introduction of additional inference time compared to the simple student network without the introduction of the self-distillation paradigm. However, the registration accuracy of the lightweight improved self-distillation registration network is higher than that of the simple student network itself.

[0076] When the teacher network is removed from the trained self-distillation registration network, a lightweight improved self-distillation registration network is obtained. After training, the teacher network is removed, leaving only the lightweight student network. Compared to the simple student network without introducing the self-distillation paradigm, this scheme does not increase the number of additional parameters or introduce additional inference time, but the registration accuracy is higher than that of the simple student network.

[0077] like Figure 2 As shown, optimization is performed during the training process. The improved self-distillation registration network is trained by iteratively optimizing the improved self-distillation registration network based on gradient surgery to obtain a well-trained self-distillation registration network based on gradient surgery.

[0078] During training, each iteration checks whether the gradient update direction of the student network gS conflicts with the dominant gradient. If a gradient conflict exists, the gradient of the student network is projected onto the normal plane of the dominant gradient; if no gradient conflict exists, the gradient gS of the student network remains unchanged.

[0079] A gradient surgery optimization strategy is introduced: by projecting conflicting gradients onto the normal plane of the dominant gradient, gradient conflicts are resolved, ensuring the effectiveness of the self-distillation paradigm.

[0080] The current training iteration is compared with the threshold. If the current iteration is less than the threshold, it is an early training phase, and the dominant gradient is the gradient gT of the teacher network. If the current iteration is greater than or equal to the threshold, it is a later training phase, and the dominant gradient is the gradient gT of the teacher network and the distillation gradient gD. In other words, it is determined whether the update direction of the student network's gradient gS conflicts with the gradient gT of the teacher network and the distillation gradient gD.

[0081] Phased processing: By setting a threshold at which the similarity loss of the teacher network almost converges, the accuracy of the teacher network deformation field used for distillation is ensured. Only when the deformation field of the teacher network is sufficiently accurate is the distillation gradient included in the dominant gradient, ensuring that the guidance received by the student network is accurate and reliable.

[0082] Whether the gradient update direction of the student network gS conflicts with the dominant gradient refers to whether the product of the student network gradient gS and the dominant gradient is greater than 0. If it is greater than 0, then there is a gradient conflict.

[0083] like Figure 3 As shown, projecting the gradient of the student network onto the normal plane of the dominant gradient means that in the early training phase, if the product of gS and gT is greater than 0, gS = gS - [(gS·gT) / ||gT|| 2 In the later training phase, if the product of gS and gT is greater than 0, then gS = gS - [(gS·gT) / ||gT|| 2 ]·gT, or the product of gS and gT is less than or equal to 0, and the product of gS and gD is greater than 0, then gS=gS-[(gS·gD) / ||gD|| 2 ]·gD.

[0084] The loss functions used when training the improved self-distillation registration network are as follows:

[0085] Similarity loss:

[0086] Regularization loss:

[0087] Distillation loss:

[0088] Where Lsim_stu and Lsim_tea represent the similarity loss of the student network and the teacher network, respectively, and Lreg_stu and Lreg_tea represent the regularization loss of the student network and the teacher network, respectively. and Let Iw_stu and Iw_tea represent the deformation fields of the student network and teacher network, respectively, and let Ldis represent the images after deformation of the moving image by applying the deformation fields of the student network and teacher network. and These represent point-by-point gradient calculation operations on the deformation fields of students and teachers, respectively. Represents the square of the L2 norm;

[0089] The gradient reflects the rate of change between each point in the deformation field and its neighboring points. Minimizing the gradient is to make the rate of change between points as small as possible, thus maintaining the smoothness of the deformation field.

[0090] The total loss for each term is obtained from the similarity loss and the regularization loss:

[0091] Total loss of the student network: LS = Lsim_stu + λr Lreg_stu

[0092] Total loss of the teacher network: LT = Lsim_tea + λr Lreg_tea

[0093] Total distillation loss: LD = λd Ldis

[0094] Where λr represents the loss weights of the student network and the teacher network, and λd represents the total loss weight of the distillation, λr = 0.01 and λd = 0.001.

[0095] Because we need to consider the order of magnitude of multiple losses, we weight each loss. The weight values ​​are set based on multiple experiments. The standard for selecting these weight coefficients is that each loss function can decrease and converge normally, and the network can be trained normally. If the weights are set improperly, the loss function may oscillate, surge, or fail to decrease, and it will not be able to converge normally, thus making it impossible to train the network.

[0096] After replacing the teacher network with a copied student network, during gradient backpropagation and updates in the training process, the distillation gradient gD is obtained from the distillation loss. The backpropagation of distillation gD flows through both the student and teacher networks, updating their parameters. The purpose of this parameter update is to make the student's deformation field as similar to the teacher's deformation field as possible, thereby minimizing the total distillation loss. When the total distillation loss is minimized, it means that the deformation field generated by the student network is most "similar" to the deformation field generated by the teacher, thus allowing the teacher network's deformation field to guide the student network's deformation field.

[0097] The total distillation loss is calculated by multiplying the distillation loss by a weight coefficient to obtain the total distillation loss, which is used for gradient backpropagation and updates, considering that the network needs to calculate many loss functions at once. The magnitudes of the loss functions need to be kept balanced so that the loss can decrease and converge and the network can be trained normally.

[0098] In this example, the student network can be either VoxelMorph, a convolutional neural network, or TransMorph, a Transformer-based network. However, this approach is not limited to these two networks and can be applied to various neural networks that can be used as learning networks.

[0099] The method of this invention was evaluated on two publicly available 3D brain magnetic resonance imaging datasets (OASIS and Mindboggle-101). Two classic networks in the field of unsupervised medical image registration—VoxelMorph (VM) based on convolutional neural networks (CNN) and TransMorph (TM) based on Transformer—were used as student networks. The average Dice coefficient (DSC%) and average symmetric surface distance (ASSD) were used as evaluation metrics for registration accuracy, and the percentage of non-positive Jacobian determinants (|Jφ≤0|%) was used as the evaluation metric for deformation field smoothness. The improvement effect of the present invention on the baseline student network is shown in Table 1.

[0100] On the OASIS and Mindboggle-101 datasets, quantitative results of average Dice coefficients (DSC(%)), average surface distance (ASSD(mm)), percentage of non-positive Jacobian determinants (|Jφ≤0|(%)), model inference time (Inference time(s)), and number of model parameters (Params(M)) on two baseline methods (VM and TM) are presented, along with corresponding results of the technical solutions of this invention (SelfDisVM-GS and SelfDisTM-GS). SelfDisVM-GS and SelfDisTM-GS refer to gradient-based self-distillation registration networks that use VM and TM as student networks, respectively.

[0101] Table 1. Comparison of Quantitative Results between Baseline Method and the Technical Solution of this Invention

[0102]

[0103] The SelfDisTM-GS technical solution of this invention achieves a DSC of 79.06% on the OASIS dataset and 70.87% on the Mindboggle-101 dataset, which are 1.14% and 0.81% higher than the baseline method TM, respectively. A similar trend was observed in SelfDisVM-GS, where DSC and ASSD were improved by 1.10% and 0.04 mm, respectively, on the OASIS dataset, and by 0.74% and 0.03 mm, respectively, on the Mindboggle-101 dataset.

[0104] The technical solution of this invention can effectively improve registration accuracy without increasing the number of additional network parameters and inference time, achieving a balance between registration accuracy and computational complexity.

[0105] Furthermore, the registration effect of the technical solution of the present invention on certain specific anatomical structures is superior to that of baseline comparison methods, such as... Figure 4 As shown, the brain region indicated by the arrow is often a challenge for unsupervised image registration due to low imaging quality and a small number of voxels. The technical solution of this invention can improve upon the baseline comparison method. Specifically, in this brain region, the shape result obtained by the method described in this invention is closest to the fixed image. In addition, the white numbers in the lower right corner represent the Dice coefficient (DSC(%)) for each method. The method described in this invention also has the highest score, fully demonstrating the effectiveness of this technical solution.

[0106] Example 2

[0107] Corresponding to the method described in this embodiment, this example also provides a self-distillation image registration system based on gradient surgery, including:

[0108] Improved self-distillation registration network module: The teacher network in the self-distillation registration network is replaced by a copied student network and concatenated after the student network; the teacher network takes the deformed image and fixed image output by the student network as input, and uses the deformation field obtained by adding the residual deformation field output by the teacher network and the deformation field of the student network point by point as the deformation field of the teacher network, thus obtaining the improved self-distillation registration network.

[0109] Network training: The improved self-distillation registration network is trained to obtain the improved self-distillation registration network;

[0110] Image registration: Input the image to be registered into the improved distillation registration network to perform image registration.

[0111] The teacher network can also be removed from the trained self-distillation registration network to obtain a lightweight improved self-distillation registration network.

[0112] Training the improved self-distillation registration network can also be done by iteratively optimizing the improved self-distillation registration network based on gradient surgery to obtain a well-trained gradient surgery-based self-distillation registration network.

[0113] During training, each iteration checks whether the gradient update direction of the student network gS conflicts with the dominant gradient. If a gradient conflict exists, the gradient of the student network is projected onto the normal plane of the dominant gradient; if no gradient conflict exists, the gradient gS of the student network remains unchanged.

[0114] It should be understood that the above module division is merely an example. In some embodiments, the above functional modules are implemented in hardware, while in other embodiments, the above functional modules are implemented in software. This invention does not impose any specific limitations on this.

[0115] Example 3

[0116] Corresponding to the method described in this embodiment, this example also provides a computer storage medium storing a computer program, which is invoked by a processor for execution:

[0117] The steps of the above gradient surgery-based self-distillation image registration method.

[0118] The specific implementation process of each step can be referred to the detailed process of the aforementioned method, and will not be elaborated in detail in this invention.

[0119] Example 4

[0120] A lightweight method for medical image registration, wherein the method applies the gradient-based self-distillation medical image registration method described above, and the method is applied to intelligent planning of medical image-guided surgery;

[0121] The registration method provides a detailed and continuous anatomical reference for surgery by precisely aligning image data from different time points or modalities (such as CT, MRI, and PET). This technology helps surgeons accurately plan incision locations and surgical paths before surgery, optimize surgical approaches, and minimize damage to surrounding healthy tissues.

[0122] For example, in neurosurgery, by registering high-resolution MRI images acquired preoperatively with real-time intraoperative images, surgeons can track the precise location of surgical instruments in real time, ensuring accurate surgical execution. Furthermore, image registration can reveal the relative positions of tumors and important blood vessels, helping surgeons avoid critical structures and improving surgical safety and success rates.

[0123] In summary, the application of medical image registration technology in surgery not only improves surgical precision and safety but also significantly enhances therapeutic outcomes and patient recovery speed by optimizing surgical planning. This integration of technology is a key step in driving the development of surgical navigation systems towards greater intelligence and precision.

[0124] Similarly, the specific implementation process of each step can be referred to the detailed process of the aforementioned method, and this invention will not elaborate on it in detail.

[0125] The technical solution provided by this invention can significantly reduce the time cost of image registration in computer-aided planning for surgical procedures, improve the efficiency of computer-aided planning in surgical procedures, and make it more convenient and simple to use. Furthermore, the technical solution provided by this invention can effectively improve registration accuracy, providing clinicians with more reliable comparison results. In addition, in actual clinical model deployment, the technical solution provided by this invention requires less computing resources, is easier to deploy, and has higher versatility.

[0126] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0127] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0128] It should be emphasized that the examples described in this invention are illustrative rather than limiting. Therefore, this invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of this invention, without departing from the spirit and scope of this invention, whether modifications or substitutions, are also within the protection scope of this invention.

Claims

1. A self-distillation image registration method based on gradient surgery, characterized in that, include: (1) Construct an improved self-distillation registration network; The teacher network in the self-distillation registration network is replaced by a copied student network and concatenated after the student network. The teacher network takes the deformed image and fixed image output by the student network as input, and uses the deformation field obtained by adding the residual deformation field output by the teacher network and the deformation field of the student network point by point as the deformation field of the teacher network, thus obtaining the improved self-distillation registration network. (2) The improved self-distillation registration network is trained to obtain the trained improved self-distillation registration network; (3) Input the image to be registered into the trained improved self-distillation registration network to perform image registration; The training of the improved self-distillation registration network is based on gradient surgery to iteratively optimize and train the improved self-distillation registration network, resulting in a well-trained gradient surgery-based self-distillation registration network. During training, each iteration checks whether the gradient update direction of the student network gS conflicts with the dominant gradient. If a gradient conflict exists, the gradient of the student network is projected onto the normal plane of the dominant gradient; if no gradient conflict exists, the gradient gS of the student network remains unchanged. The current training iteration is compared with the threshold. If it is less than the threshold, it is the early training stage, and the dominant gradient is the gradient gT of the teacher network. If it is greater than or equal to the threshold, it is the later training stage, and the dominant gradient is the gradient gT of the teacher network and the distillation gradient gD. That is, it is determined whether the update direction of the gradient gS of the student network will conflict with the gradient gT and the distillation gradient gD of the teacher network. Whether the gradient update direction of the student network gS conflicts with the dominant gradient refers to whether the product of the student network gradient gS and the dominant gradient is greater than 0. If it is greater than 0, then there is a gradient conflict. The projection of the student network's gradient onto the normal plane of the dominant gradient refers to: in the early training phase, if the product of gS and gT is greater than 0, In the later training phase, if the product of gS and gT is greater than 0, then gS = Alternatively, the product of gS and gT is less than or equal to 0, and the product of gS and gD is greater than 0. .

2. The method according to claim 1, characterized in that, The teacher network is removed from the trained self-distillation registration network to obtain a lightweight improved self-distillation registration network.

3. The method according to any one of claims 1-2, characterized in that, The loss functions used when training the improved self-distillation registration network are as follows: Similarity loss: , ; Regularization loss: , ; Distillation loss: ; Where Lsim_stu and Lsim_tea represent the similarity loss of the student network and the teacher network, respectively, and Lreg_stu and Lreg_tea represent the regularization loss of the student network and the teacher network, respectively. and This represents the deformation field of the student network and the teacher network. and Let Ldis represent the images after the moving image has been deformed by applying the deformation fields of the student network and the teacher network, respectively, and let Ldis represent the distillation loss. and These represent point-by-point gradient calculation operations on the deformation fields of students and teachers, respectively. Let represent the square of the L2 norm; randomly select two 3D medical images from the training dataset, one of which is used as a fixed image to be aligned, denoted as If; The total loss for each term is obtained from the similarity loss and the regularization loss: Total loss of student network access: ; Total losses in the teacher network: ; Total distillation loss: ; in, This represents the loss weights of the student network and the teacher network. This represents the total loss weight of distillation. , .

4. A self-distillation image registration system based on gradient surgery using the method described in claim 1 or 2, characterized in that, include: Improved self-distillation registration network module: The teacher network in the self-distillation registration network is replaced by a copied student network and concatenated after the student network; the teacher network takes the deformed image and fixed image output by the student network as input, and uses the deformation field obtained by adding the residual deformation field output by the teacher network and the deformation field of the student network point by point as the deformation field of the teacher network, thus obtaining the improved self-distillation registration network. Network training: The improved self-distillation registration network is trained to obtain the improved self-distillation registration network; Image registration: Input the image to be registered into the improved distillation registration network to perform image registration.

5. The system according to claim 4, characterized in that, The training of the improved self-distillation registration network is based on gradient surgery to iteratively optimize and train the improved self-distillation registration network, resulting in a well-trained gradient surgery-based self-distillation registration network. During training, each iteration checks whether the gradient update direction of the student network gS conflicts with the dominant gradient. If a gradient conflict exists, the gradient of the student network is projected onto the normal plane of the dominant gradient; if no gradient conflict exists, the gradient gS of the student network remains unchanged.

6. A computer storage medium, characterized in that, The computer program is stored and is invoked by the processor for execution. The steps of the gradient surgery-based self-distillation image registration method according to any one of claims 1-3.

7. A lightweight method for medical image registration, characterized in that, The method employs the gradient surgery-based self-distillation image registration method as described in any one of claims 1-3.

Images