High-resolution reconstruction method and device for CT-MR images of the knee joint
By employing bone-locked registration and cross-layer coupling super-resolution methods, and utilizing CT bone structures and bone tags as priors, high-definition reconstruction of knee joint CT-MR images was achieved. This addresses the problem of insufficient description of bone structure and soft tissue motion in existing technologies, and improves the quality of image registration and reconstruction.
Patent Information
- Application Number
- CN202611423031.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-14
- Publication Date
- 2026-11-17
AI Technical Summary
In existing CT-MR image processing workflows, single rigid or non-rigid registration is insufficient to describe the motion changes of knee joint bone structure and soft tissues. Furthermore, ordinary super-resolution networks cannot fully utilize the high-frequency structural constraints provided by CT bone edges and bone labels, resulting in registration errors affecting super-resolution results and making it difficult to achieve complementary high-definition reconstruction.
A bone-locked registration and cross-layer coupled super-resolution method is adopted. The bone structure and bone tags in CT images are used as anatomical priors to perform rigid motion constraints and non-rigid displacement modeling on MR images. High-definition registered MR images are output through a cross-layer coupled super-resolution network. High-frequency residual prediction is performed by combining CT intensity, CT edge and bone tag surface priors.
While maintaining the rigidity of the bone structure, it significantly improves the registration accuracy of soft tissues and the high-definition reconstruction quality of MR images, making it suitable for multimodal medical image analysis and preoperative planning of the knee joint.
Smart Images

Figure CN123118360A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of medical image processing, three-dimensional medical image registration, and medical image super-resolution reconstruction, and in particular to a method and apparatus for high-definition reconstruction of knee joint CT-MR images based on bone-locking registration and cross-layer coupling super-resolution. Background Technology
[0002] In the diagnosis of knee joint diseases, preoperative planning, cartilage and ligament assessment, prosthesis design, and postoperative follow-up, CT and MR images are significantly complementary. CT images can clearly display the bone cortex, bone boundaries, and bony morphology, making them suitable for establishing bone structure coordinates and anatomical constraints. MR images can display soft tissue information such as cartilage, meniscus, ligaments, and bone marrow, but their spatial resolution, slice thickness, field of view, and geometric consistency are generally inferior to CT. Therefore, in clinical and research settings, it is often necessary to accurately map MR soft tissue information to the CT bone structure space and further obtain high-resolution registered MR images.
[0003] Current CT-MR processing workflows typically treat image registration and super-resolution reconstruction as two separate steps: first, rigid or non-rigid registration is performed on the MR image, and then the registration result is upsampled or enhanced by super-resolution. This sequential workflow has shortcomings: on the one hand, the knee joint bone structure should approximately satisfy rigid body motion, while soft tissues exhibit local deformation. A single rigid registration is insufficient to describe soft tissue changes, while a single non-rigid registration may disrupt the rigidity of the bone structure. On the other hand, ordinary super-resolution networks tend to only increase pixel size, producing smoothing or false details, and failing to fully utilize the high-frequency structural constraints provided by CT bone edges and bone tags. Furthermore, registration errors affect super-resolution results, and the quality of super-resolution recovery also reflects the reliability of local registration; without internal interaction, these two processes cannot mutually promote each other.
[0004] Therefore, how to provide a technical solution that simultaneously considers the rigidity of bone structure, the non-rigidity of soft tissue, the complementarity of CT-MR modalities, and the requirements of high-definition reconstruction, so that the registration branch and the super-resolution branch can work in parallel within the same framework and interact step by step at the feature level, thereby obtaining MR images that both conform to the anatomical structure of the knee joint and have high spatial resolution, is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of the above problems, the present invention provides a method and apparatus for high-definition reconstruction of knee joint CT-MR images to overcome or at least partially solve the above problems. This method utilizes bone structures and bone tags in CT images as anatomical priors to rigidly constrain the motion of bony structures such as the femur and tibia in MR images, while simultaneously performing non-rigid displacement modeling of soft tissue regions. Finally, it outputs a high-definition registered MR image consistent with the CT space through a cross-layer coupled super-resolution network.
[0006] This invention provides the following solution: A high-resolution reconstruction method for knee joint CT-MR images includes: S1: Acquire CT images, MR images, CT bone tags, and MR bone tags of the same patient's knee joint; using the CT images as a reference space, resample the MR images and MR bone tags to the CT grid to obtain resampled MR images and MR bone tags; perform robust normalization on the CT images and MR images to obtain normalized CT images and normalized MR images; construct CT edge maps, bone tag surface maps, soft tissue masks, MR overlay masks, and displacement amplitude maps. S2: Input the normalized CT image, normalized MR image, CT bone label, MR bone label, original bone cavity mask and soft tissue mask into the low-resolution BLHR bone locking registration branch. The BLHR bone locking registration branch adopts a three-dimensional encoder-decoder structure. After the encoder extracts the joint spatial features, the decoder outputs the femoral three-dimensional rigidity transformation parameters, tibial three-dimensional rigidity transformation parameters, soft tissue three-dimensional displacement field and multi-scale registration features. S3: Based on the three-dimensional rigidity transformation parameters of the femur and the tibial bone, bone-locking rigidity transformation synthesis is performed on the normalized MR image to obtain a registered MR image; the registered MR image, the normalized CT image, the CT edge map, and the bone tag surface map are used as input to extract local image blocks within the target ROI region. A degraded MR input is constructed using a high-resolution ROI super-resolution branch through downsampling, upsampling, and restoration. High-frequency residuals are predicted using MR low-frequency information, CT intensity, CT edges, and bone tag surface priors. The high-frequency residuals are added back to the degraded MR to obtain a high-resolution MR image block. S4: Extract multi-scale registration features at different scales of the encoder and decoder of the BLHR bone-locked registration branch. Based on the spatial position of each high-resolution ROI image patch in the original image, crop the multi-scale registration features and interpolate them to the same size as the corresponding ROI image patch to obtain cropped registration features. Through the cross-layer bidirectional Cross-Stitch feature fusion module, the cropped registration features of each level are bidirectionally updated with the corresponding super-resolution branch features. The bone-locked geometric features of the BLHR bone-locked registration branch guide MR high-frequency recovery, and the local detail features of the high-resolution ROI super-resolution branch inversely supplement the registration representation. S5: The high-resolution MR image blocks of each ROI region are weighted and fused through an overlapping sliding window to output a high-resolution registered MR image consistent with the CT space.
[0007] Preferably, in step S2, the rigid transformation can be expressed as:
[0008] In the formula, This represents the set of bone structures to be rigidly modeled. Represents the rotation matrix. Represents the translation vector; The input channel of the three-dimensional encoder-decoder structure consists of normalized CT, normalized MR, CT femoral label, CT tibial label, MR femoral label, MR tibial label, original bone cavity mask, and soft tissue mask.
[0009] Preferably, in step S3, the inputs of the high-resolution ROI super-resolution branch include degraded MR, CT intensity, CT edge and bone surface prior; The degraded MR image is constructed by downsampling, upsampling and restoration of the registered MR image, the CT intensity comes from the normalized CT image, the CT edge comes from the CT edge map, and the bone surface prior comes from the bone tag surface map.
[0010] Preferably, in step S4, the cross-layer bidirectional Cross-Stitch feature fusion module sets up a bidirectional cross-fusion unit in each super-resolution module and updates the features according to the following formula:
[0011]
[0012] in, and They represent the first Super-resolution features and registration features of the layer and For learnable gating parameters; The cross-convolution in the bidirectional cross-fusion unit has zero weights during initialization, making the initial state of the module equivalent to an identity mapping.
[0013] Preferably, in the cross-layer bidirectional Cross-Stitch feature fusion module, the multi-scale registration features come from the encoder outputs and decoder outputs of the BLHR bone-locked registration branch at each scale; the corresponding super-resolution branch features come from the SR modules in the high-resolution ROI super-resolution branch that correspond to each scale of the registration branch encoder-decoder; the bidirectional interactive update is completed within each SR module, and the updated super-resolution features continue to propagate to the next layer SR module.
[0014] Preferably, step S5 includes a rigid security judgment step before weighted fusion: Rigid safety judgment is performed based on the change in the overlap between the predicted bone tag and the reference bone tag. When a rigid transformation causes the bone tag overlap to decrease by more than a preset threshold, the rigid transformation is rejected and excluded from the final fusion result.
[0015] Preferably, in step S5, the weighted fusion of overlapping sliding windows specifically includes: Upsample MR, CT, and bone priors within each ROI region to the target resolution; By using overlapping sliding windows, each image block is sequentially input into the coupled SR branch for prediction, resulting in multiple overlapping high-resolution MR prediction blocks. A weighted fusion method is used to reduce block artifacts in the multiple overlapping high-definition MR prediction blocks, and the final high-definition registered MR image is output.
[0016] Preferably, it also includes a joint training step: Using the soft tissue displacement field predicted in step S2 by the reference displacement field and the reference bone tag constraint, a registration geometric consistency loss is constructed. ; Constructing a voxel-level image alignment loss using local normalized cross-correlation ; Constructing high-resolution MR reconstruction loss using reconstruction error, gradient or high-frequency error, and registration-guided consistency. ; Using the overall loss function The BLHR bone-locking registration branch and the high-resolution ROI super-resolution branch are jointly optimized; These represent the corresponding weight coefficients; In the initial stage of training, the existing independent SR model weights are loaded, so that the Cross-Stitch module starts training from the identity mapping, and then the feature interaction between the registration branch and the SR branch is gradually released.
[0017] Preferably: the CT edge map constructed in step S1 is obtained by edge detection of the normalized CT image, the bone tag surface map is obtained by surface extraction of the CT bone tag, the soft tissue mask is obtained by the reverse region of the CT bone tag, the MR overlay mask is obtained by the non-zero region of the normalized MR image, and the displacement amplitude map is calculated by the amplitude of the soft tissue displacement field.
[0018] A high-resolution reconstruction device for knee joint CT-MR images, used to perform the above-described high-resolution reconstruction method for knee joint CT-MR images, the device comprising: An image preprocessing unit is used to acquire CT images, MR images, CT bone tags, and MR bone tags of the same patient's knee joint; using the CT images as a reference space, the MR images and MR bone tags are resampled onto a CT grid to obtain resampled MR images and MR bone tags; robust normalization is performed on the CT images and MR images to obtain normalized CT images and normalized MR images; and CT edge maps, bone tag surface maps, soft tissue masks, MR overlay masks, and displacement amplitude maps are constructed. The bone locking registration unit is used to input the normalized CT image, normalized MR image, CT bone label, MR bone label, original bone cavity mask and soft tissue mask into the low-resolution BLHR bone locking registration branch. The BLHR bone locking registration branch adopts a three-dimensional encoder-decoder structure. After the encoder extracts the joint spatial features, the decoder outputs the three-dimensional rigidity transformation parameters of the femur, the three-dimensional rigidity transformation parameters of the tibia, the three-dimensional displacement field of the soft tissue and the multi-scale registration features. The super-resolution reconstruction unit is used to perform bone-locking rigid transformation synthesis on the normalized MR image according to the femoral three-dimensional rigidity transformation parameters and the tibial three-dimensional rigidity transformation parameters to obtain a registered MR image; taking the registered MR image, the normalized CT image, the CT edge map, and the bone tag surface map as input, it extracts local image blocks within the target ROI region, constructs degraded MR input through downsampling, upsampling, and restoration using a high-resolution ROI super-resolution branch, and predicts high-frequency residuals using MR low-frequency information, CT intensity, CT edges, and bone tag surface priors, and adds the high-frequency residuals back to the degraded MR to obtain a high-resolution MR image block; A cross-layer bidirectional feature fusion unit is used to extract multi-scale registration features at different scales of the encoder and decoder of the BLHR bone-locked registration branch. Based on the spatial position of each high-resolution ROI image patch in the original image, the multi-scale registration features are cropped and interpolated to the same size as the corresponding ROI image patch to obtain cropped registration features. Through a cross-layer bidirectional Cross-Stitch feature fusion module, the cropped registration features of each level are bidirectionally updated with the corresponding super-resolution branch features. The bone-locked geometric features of the BLHR bone-locked registration branch guide MR high-frequency recovery, and the local detail features of the high-resolution ROI super-resolution branch inversely supplement the registration representation. The result output unit is used to weighted fuse the high-definition MR image blocks of each ROI region through an overlapping sliding window, and output a high-definition registered MR image consistent with the CT space.
[0019] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This application provides a method and apparatus for high-resolution reconstruction of knee joint CT-MR images. The method uses CT, MR, and bone tags as inputs. In the low-resolution branch, it predicts the rigidity transformation of the femur and tibia, as well as the non-rigid displacement field of soft tissues. In the high-resolution ROI branch, it utilizes MR and CT intensity, CT edges, and bone tag surface priors for high-resolution MR reconstruction. A bidirectional Cross-Stitch module with identity initialization enables layer-by-layer interaction between registration features and super-resolution features. This method can improve the quality of soft tissue registration and high-resolution MR reconstruction while maintaining bone structure rigidity, and is suitable for scenarios such as multimodal medical image analysis, three-dimensional reconstruction, and preoperative planning of the knee joint.
[0020] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0022] Figure 1 This is a flowchart of the high-resolution reconstruction method for knee joint CT-MR images provided in this embodiment of the invention; Figure 2 This is a schematic diagram of the overall process of the method provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the cross-layer coupling structure of the low-resolution BLHR bone-locking registration branch and the high-resolution ROI super-resolution branch provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the joint training framework of registration supervision, voxel-level alignment supervision and super-resolution supervision provided in the embodiments of the present invention; Figure 5 This is a schematic diagram of the bone-locking synthesis and ROI sliding window super-resolution process during the inference stage provided in this embodiment of the invention; Figure 6 This is a schematic diagram of the high-definition reconstruction device for knee joint CT-MR images provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the high-definition reconstruction device for knee joint CT-MR images provided in an embodiment of the present invention. Detailed Implementation
[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.
[0024] See Figure 1 , Figure 2 This invention provides a method for high-resolution reconstruction of knee joint CT-MR images, such as... Figure 1 , Figure 2 As shown, the method may include: S1: Acquire CT images, MR images, CT bone tags, and MR bone tags of the same patient's knee joint; using the CT images as a reference space, resample the MR images and MR bone tags to a CT grid to obtain resampled MR images and MR bone tags; perform robust normalization on the CT images and MR images to obtain normalized CT images and normalized MR images; construct a CT edge map, a bone tag surface map, a soft tissue mask, an MR overlay mask, and a displacement amplitude map; in specific implementations, embodiments of the present invention may provide that the constructed CT edge map is obtained from the normalized CT image through edge detection, the bone tag surface map is obtained from the CT bone tag through surface extraction, the soft tissue mask is obtained from the reverse region of the CT bone tag, the MR overlay mask is obtained from the non-zero region of the normalized MR image, and the displacement amplitude map is calculated from the amplitude of the soft tissue displacement field.
[0025] In practical applications, the input data of this invention includes at least CT images, MR images, CT bone tags, and MR bone tags. During the training phase, existing reference registration MR images, reference bone tags, reference displacement fields, original bone cavity masks, and soft tissue masks can also be used as supervision or guidance information. The aforementioned reference data can be generated from existing registration processes, manual quality control correction results, or reliable registration algorithms; not all of it is required during the inference phase.
[0026] During preprocessing, CT images were used as the reference space, and MR images, MR labels, and optional reference registration data were resampled to the CT grid. Subsequently, robust normalization was performed on CT, MR, and reference MR to reduce grayscale differences between different cases and different scan sequences.
[0027] Anatomical prior construction: Based on the above, CT edge maps, bone tag surface maps, soft tissue masks, MR overlay masks, and displacement amplitude maps are constructed. Low-resolution registration inputs can consist of normalized CT, normalized MR, CT femoral tag, CT tibia tag, MR femoral tag, MR tibia tag, original bone cavity mask, and soft tissue mask; high-resolution SR inputs include degraded MR, CT intensity, CT edge and bone surface prior information.
[0028] S2: Input the normalized CT image, normalized MR image, CT bone label, MR bone label, original bone cavity mask and soft tissue mask into the low-resolution BLHR bone locking registration branch. The BLHR bone locking registration branch adopts a three-dimensional encoder-decoder structure. After the encoder extracts the joint spatial features, the decoder outputs the femoral three-dimensional rigidity transformation parameters, tibial three-dimensional rigidity transformation parameters, soft tissue three-dimensional displacement field and multi-scale registration features. In its implementation, the low-resolution BLHR bone-locking registration network employs a three-dimensional encoder-decoder structure. The encoder extracts joint spatial features from CT-MR and label priors, while the decoder recovers the spatial displacement representation at the same scale as the input. The network has three main outputs: a soft tissue displacement field prediction head, a femoral rigidity parameter prediction head, and a tibial rigidity parameter prediction head.
[0029] The soft tissue displacement field prediction head outputs a three-channel displacement field and utilizes a soft tissue mask to limit its primary effect on non-bone regions. The femoral and tibial rigidity parameter prediction heads output translational and rotational parameters, respectively, to describe the three-dimensional rigid motion of individual bone structures. For the first... The rigidity transformation of a bone structure can be expressed as:
[0030] in, This represents the set of bone structures to be rigidly modeled. Represents the rotation matrix. This represents the translation vector. This design ensures that the bone region adheres to rigid body motion and the soft tissue region adheres to continuous deformation, thereby avoiding unreasonable deformation of the bone structure caused by traditional non-rigid registration.
[0031] S3: Based on the three-dimensional rigidity transformation parameters of the femur and the tibial bone, bone-locking rigidity transformation synthesis is performed on the normalized MR image to obtain a registered MR image; the registered MR image, the normalized CT image, the CT edge map, and the bone tag surface map are used as input to extract local image blocks within the target ROI region. A degraded MR input is constructed using a high-resolution ROI super-resolution branch through downsampling, upsampling, and restoration. High-frequency residuals are predicted using MR low-frequency information, CT intensity, CT edges, and bone tag surface priors. The high-frequency residuals are added back to the degraded MR to obtain a high-resolution MR image block. The inputs to the high-resolution ROI super-resolution branch include degraded MR, CT intensity, CT margins, and bone surface priors; The degraded MR image is constructed by downsampling, upsampling and restoration of the registered MR image, the CT intensity comes from the normalized CT image, the CT edge comes from the CT edge map, and the bone surface prior comes from the bone tag surface map.
[0032] like Figure 3 As shown in the lower part, the high-resolution ROI super-resolution branch takes a local image patch of the target region as input and constructs a degraded MR input through a "downsampling-upsampling-recovery" method, using either a reference-registered MR or the original high-resolution MR image patch as the recovery target. This branch uses MR low-frequency information, CT intensity, CT edges, and bone tag surface priors to predict high-frequency residuals, and adds the residuals back into the degraded MR to obtain a high-resolution MR output.
[0033] To improve the reconstruction quality of bone boundaries and registration-sensitive areas, the SR branch incorporates CT feature flow and registration guidance maps, enabling the network to obtain stronger structural constraints on bone surfaces, near original bone cavities, and in areas with significant soft tissue displacement. This reduces the smoothing issues caused by conventional upsampling, allowing the output MR to more closely approximate the anatomical boundaries under CT guidance in terms of spatial detail.
[0034] S4: Extract multi-scale registration features at different scales of the encoder and decoder of the BLHR bone-locked registration branch. Based on the spatial position of each high-resolution ROI image patch in the original image, crop the multi-scale registration features and interpolate them to the same size as the corresponding ROI image patch to obtain cropped registration features. Through the cross-layer bidirectional Cross-Stitch feature fusion module, the cropped registration features of each level are bidirectionally updated with the corresponding super-resolution branch features. The bone-locked geometric features of the BLHR bone-locked registration branch guide MR high-frequency recovery, and the local detail features of the high-resolution ROI super-resolution branch inversely supplement the registration representation. In specific implementation, the cross-layer bidirectional Cross-Stitch feature fusion module sets up a bidirectional cross-fusion unit in each super-resolution module and updates features according to the following formula:
[0035]
[0036] in, and They represent the first Super-resolution features and registration features of the layer and For learnable gating parameters; The cross-convolution in the bidirectional cross-fusion unit has zero weights during initialization, making the initial state of the module equivalent to an identity mapping.
[0037] In the cross-layer bidirectional Cross-Stitch feature fusion module, the multi-scale registration features come from the encoder output and decoder output of the BLHR bone-locked registration branch at each scale; the corresponding super-resolution branch features come from the SR modules in the high-resolution ROI super-resolution branch that correspond to each scale of the registration branch encoder-decoder; the bidirectional interactive update is completed within each SR module, and the updated super-resolution features continue to propagate to the next layer SR module.
[0038] The core of this invention is the layer-by-layer mutual promotion between registration and super-resolution. The BLHR branch generates multi-scale features at different scales in the encoder and decoder; for each high-resolution ROI image patch, the BLHR features are cropped and interpolated to the same size according to their spatial position in the original image, and then sent to the corresponding SR module. Each SR module is equipped with a bidirectional cross-fusion unit to update the SR features with the registration features.
[0039] Cross-convolution is initialized to zero, so the initial state of the module approximately maintains the output of the original SR network; after training, the network can automatically learn the interaction strength between registration information and super-resolution information.
[0040] S5: The high-resolution MR image blocks of each ROI region are weighted and fused using an overlapping sliding window to output a high-resolution registered MR image consistent with the CT space. Specifically, embodiments of the present invention may include a rigid safety judgment step before weighted fusion. Rigid safety judgment is performed based on the change in the overlap between the predicted bone tag and the reference bone tag. When a rigid transformation causes the bone tag overlap to decrease by more than a preset threshold, the rigid transformation is rejected and excluded from the final fusion result.
[0041] The weighted fusion of overlapping sliding windows specifically includes: Upsample MR, CT, and bone priors within each ROI region to the target resolution; By using overlapping sliding windows, each image block is sequentially input into the coupled SR branch for prediction, resulting in multiple overlapping high-resolution MR prediction blocks. A weighted fusion method is used to reduce block artifacts in the multiple overlapping high-definition MR prediction blocks, and the final high-definition registered MR image is output.
[0042] It also includes joint training steps: Using the soft tissue displacement field predicted in step S2 by the reference displacement field and the reference bone tag constraint, a registration geometric consistency loss is constructed. ; Constructing a voxel-level image alignment loss using local normalized cross-correlation ; Constructing high-resolution MR reconstruction loss using reconstruction error, gradient or high-frequency error, and registration-guided consistency. ; Using the overall loss function The BLHR bone-locking registration branch and the high-resolution ROI super-resolution branch are jointly optimized; These represent the corresponding weight coefficients; In the initial stage of training, the existing independent SR model weights are loaded, so that the Cross-Stitch module starts training from the identity mapping, and then the feature interaction between the registration branch and the SR branch is gradually released.
[0043] During the training phase, the registration branch and the super-resolution branch are optimized together. For example... Figure 4 As shown, the registration branch uses the reference displacement field and reference bone label to constrain the predicted soft tissue displacement field and bone rigidity transformation; the voxel-level alignment constraint uses image similarity such as local normalized cross-correlation to make the predicted transformed MR close to the reference registered MR in terms of intensity; the super-resolution branch uses reconstruction error, gradient or high-frequency error and registration-guided consistency to constrain the high-definition MR recovery quality.
[0044] To reduce training risks, existing independent SR model weights can be loaded first, allowing the Cross-Stitch module to start training from the identity mapping, and then the feature interactions between the registration branch and the SR branch can be gradually released.
[0045] like Figure 5 As shown, the inference stage first performs low-resolution BLHR registration prediction on the input cases to obtain the soft tissue displacement field, femoral stiffness parameters, tibial stiffness parameters, and multi-scale registration features. Subsequently, a stiffness safety judgment is performed based on the change in the overlap between the predicted bone label and the reference or target bone label to avoid stiffness transformations that significantly reduce the overlap of bone labels from entering the final result.
[0046] Within the Region of Interest (ROI), the reference registered MR image can be used directly as the super-resolution input source, or a registered source MR image can be synthesized based on the bone locking transform predicted by the network. Subsequently, the MR, CT, and bone priors within the ROI are upsampled to the target resolution, coupled to the SR branch through an overlapping sliding window input, and weighted fusion is used to reduce block artifacts. The final output is a high-resolution registered MR image consistent with the CT space, and can also output a baseline SR comparison image, an upsampled source image, an SR residual image, and a registration auxiliary image for quality control and experimental analysis.
[0047] As can be seen, the high-resolution reconstruction method for knee joint CT-MR images provided in this embodiment of the invention does not simply adopt a serial strategy of "registration first, then independent super-resolution," but instead constructs a parallel network consisting of a low-resolution BLHR bone-locking registration branch and a high-resolution ROI super-resolution branch. The registration branch is responsible for predicting the independent three-dimensional rigid motion of the femur and tibia, as well as the non-rigid displacement field of soft tissues; the super-resolution branch is responsible for recovering high-resolution MR images within the target area of the knee joint using MR, CT intensity, CT edges, bone tag surface, and registration guidance information.
[0048] A bidirectional Cross-Stitch feature fusion module with identity initialization is set between the two branches. This allows the bone-locking geometric features of the registration branch to guide high-frequency MR recovery layer by layer, while the local detail features of the super-resolution branch inversely supplement the registration-related representation. This module is approximately equivalent to the identity mapping during initialization, thus it is compatible with existing independent super-resolution models. As training progresses, the network gradually learns the synergistic relationship between registration and super-resolution.
[0049] A low-resolution bone-locking hybrid registration model was used, with CT, MR and their bone tags as inputs, to predict the six-degree-of-freedom rigid transformation of the femur and tibia, and to predict the three-dimensional displacement field of the soft tissue region. A high-resolution ROI super-resolution reconstruction model was adopted, and high-resolution MR within the target area was recovered by utilizing degraded MR, CT intensity, CT edges and bone tag surface priors. The provided cross-layer bidirectional feature interaction mechanism crops the multi-scale features of the BLHR registration branch to high-resolution ROI image patches and sends them to the corresponding SR module for fusion. Joint training constraints are used to jointly optimize the registration branch and the super-resolution branch by utilizing the reference displacement field, reference bone label, image similarity, and SR reconstruction (super-Resolution Reconstruction) error. A high-definition inference method at the ROI level was implemented, which performs bone-locking synthesis, sliding window prediction and weighted fusion within the target area, and outputs a high-definition registered MR image.
[0050] Experimental data comparison and effect verification: This experiment used paired CT and MR images of a knee joint. The CT resolution was approximately 0.5mm × 0.5mm × 0.5mm, and the original MR resolution was approximately 1.0mm × 1.0mm × 1.5mm. The training, validation, and test sets were divided in a 7:1:2 ratio to ensure no overlap between subsets. All data underwent ethical review and patient-identified information was removed.
[0051] Comparison method: To fully verify the technical effects of the present invention, the following comparison method is set up: Baseline A (serial rigid registration + independent super-resolution): First, perform global rigid registration of MR to CT space, and then use a conventional SRCNN network for super-resolution reconstruction.
[0052] Baseline B (serial non-rigid registration + independent super-resolution): VoxelMorph is used to perform non-rigid registration of MR, followed by super-resolution reconstruction.
[0053] Baseline C (end-to-end registration and super-resolution joint network, no bone locking): The registration branch outputs a single displacement field, the super-resolution branch has no CT edge and bone label guidance, and no cross-stitch interaction.
[0054] Evaluation indicators: Registration accuracy was quantified using the Dice similarity coefficient (Dice, higher is better), mean surface distance (ASD, lower is better), and Hausdorff distance (HD, lower is better) for the femoral / tibial bone structures. Reconstruction quality was quantified using the peak signal-to-noise ratio (PSNR, higher is better) and structural similarity index (SSIM, higher is better). Efficiency metrics included peak training memory usage (GB, lower is better) and single-instance inference time (s, lower is better).
[0055] Quantitative comparison results: Table 1 presents the quantitative comparison results of different methods on the test set. The last row represents the method of the present invention and its results.
[0056] Table 1
[0057] Bone structure preservation effect: Analysis: Baseline B (non-rigid registration) resulted in a significant decrease in bone structure Dice to 91.5% / 90.3% due to the lack of independent rigid constraints on the femur and tibia, indicating that a single non-rigid registration can cause unreasonable deformation of the bone structure. This invention, by predicting the six-degree-of-freedom rigid transformations of the femur and tibia separately, improved the bone structure Dice to 97.8% / 97.1%, demonstrating that the bone locking mechanism effectively avoids deformation and damage to the bone structure.
[0058] Technical results: Compared with Baseline B, the present invention improves femoral Dice by 6.3 percentage points and tibial Dice by 6.8 percentage points.
[0059] Soft tissue registration and reconstruction quality: Analysis: This invention outperforms all comparable methods in both soft tissue alignment accuracy (ASD=1.15mm) and image reconstruction quality (PSNR=32.56dB, SSIM=0.928). This demonstrates that cross-layer bidirectional Cross-Stitch feature interaction synergistically promotes registration information and super-resolution information, an effect that is not simply a superposition of existing technologies.
[0060] Technical results: Compared to the optimal baseline, PSNR is improved by 2.44dB and SSIM is improved by 0.037.
[0061] Video memory efficiency: Analysis: This invention completes registration at low resolution and reconstruction within a high-resolution ROI, with a significantly lower memory usage (9.8GB) than Baseline B (14.6GB), proving that this parallel framework achieves high-definition output while controlling hardware consumption.
[0062] Technical Results: Compared to Baseline B, memory usage is reduced by 32.9%, and inference time is shortened by 46.3%. Ablation Experiment Results To verify the independent contribution of each core module, ablation experiments were conducted by removing key modules one by one from the complete model. The results are shown in Table 2, where the first row of Table 2 represents the complete model (this invention), and the remaining rows represent the results after removing the corresponding modules.
[0063] Table 2
[0064] The necessity of the Cross-Stitch module: Analysis: Removing the Cross-Stitch module decreased the PSNR by 2.32 dB to 30.24 dB, and the femoral Dice decreased slightly to 97.5%. This indicates that cross-layer bidirectional feature interaction plays a crucial role in high-frequency detail recovery, and that this module cannot be simply replaced by a non-interactive serial structure. The identity initialization design of Cross-Stitch ensures training stability, allowing the module to gradually learn the synergistic relationship between registration and super-resolution starting from the identity mapping.
[0065] The necessity of bone locking mechanisms: Analysis: After removing bone locking (using only a single non-rigid displacement field), the femoral Dice dropped sharply to 92.3%, a decrease of 5.5 percentage points. This demonstrates that separately modeling bone rigidity and soft tissue non-rigidity is a necessary means to address the anatomical constraints of the knee joint, and a single non-rigid registration cannot simultaneously meet the needs of both bone structural rigidity and soft tissue deformation.
[0066] The necessity of prior knowledge of CT margins and bone label surfaces: Analysis: After removing the CT edge and bone tag surface prior, the PSNR decreased to 30.87 dB, indicating that the mechanism of high-frequency residual prediction using CT intensity, CT edge and bone tag surface prior in step S3 plays an important role in restoring the true anatomical boundary.
[0067] The necessity of ROI sliding window fusion: Analysis: After removing the ROI sliding window fusion (direct super-resolution of the entire image), the PSNR dropped to 31.45dB, and the visualization results showed obvious blocky artifacts. This proves that the overlapping sliding window input and weighted fusion mechanism have an irreplaceable role in reducing blocky artifacts and improving output quality.
[0068] Module cooperative gain: Analysis: Each module contributes positively to the final result, and there is synergistic gain among the modules. The PSNR of the complete model (32.56dB) is higher than the sum of the individual improvements, indicating that there is a nonlinear synergistic effect among bone locking registration, cross-stitch coupling, CT prior guidance, and sliding window fusion, which produces unexpected technical results.
[0069] Qualitative results explanation: In the visual comparison, the output features of each method are as follows: Baseline A: The MR image shows significant blurring at the bone cortex boundary, with unclear bone edges.
[0070] Baseline B: Local distortion artifacts appear in the femoral region, and the bone structure morphology is distorted.
[0071] Baseline C: The cartilage and ligament areas are excessively smoothed, resulting in the loss of high-frequency details.
[0072] This invention: The high-definition registered MR images output by this invention are superior to the comparative methods in terms of the sharpness of the bone cortex edges and the clarity of the meniscus and ligament textures, and the bone structure morphology is highly consistent with the CT bone labels.
[0073] in conclusion: The experimental data above demonstrate that the present invention, through the synergistic design of bone-locked hybrid registration and cross-layer coupled super-resolution techniques, significantly improves soft tissue registration accuracy and high-frequency reconstruction quality while maintaining bone structure rigidity, without significantly increasing the memory burden. This effect is not a simple superposition of existing technologies, but rather an unexpected technical effect resulting from the interaction of various technical features, demonstrating full inventiveness.
[0074] In summary, the high-definition reconstruction method for knee joint CT-MR images provided by this invention separates the modeling of rigid bone structures from that of non-rigid soft tissues, avoiding unreasonable deformation of bone structures such as the femur and tibia caused by single non-rigid registration; it completes bone-locking registration at low resolution and MR reconstruction within a high-resolution ROI, achieving high-definition output while controlling memory consumption; it utilizes CT intensity, CT edges, and bone tag surface priors to guide high-frequency MR recovery, making the super-resolution results more focused on the true anatomical boundaries; it achieves layer-by-layer interaction between registration features and SR features through a Cross-Stitch module, enabling registration information to guide super-resolution, and the high-definition reconstruction target to inversely constrain registration-related representations; and it sets rigidity safety judgment and ROI sliding window fusion mechanisms in the inference stage, improving the stability and visualization quality of prediction results.
[0075] See Figure 6 The present invention can also provide a high-definition reconstruction device for CT-MR images of the knee joint, such as... Figure 6 As shown, the device may include: The image preprocessing unit 601 is used to acquire CT images, MR images, CT bone tags, and MR bone tags of the same patient's knee joint; using the CT images as a reference space, the MR images and MR bone tags are resampled to a CT grid to obtain resampled MR images and MR bone tags; robustly normalizes the CT images and MR images to obtain normalized CT images and normalized MR images; and constructs CT edge maps, bone tag surface maps, soft tissue masks, MR overlay masks, and displacement amplitude maps. The bone locking registration unit 602 is used to input the normalized CT image, normalized MR image, CT bone label, MR bone label, original bone cavity mask and soft tissue mask into the low-resolution BLHR bone locking registration branch. The BLHR bone locking registration branch adopts a three-dimensional encoder-decoder structure. After the encoder extracts the joint spatial features, the decoder outputs the three-dimensional rigidity transformation parameters of the femur, the three-dimensional rigidity transformation parameters of the tibia, the three-dimensional displacement field of the soft tissue and the multi-scale registration features. The super-resolution reconstruction unit 603 is used to perform bone-locking rigid transformation synthesis on the normalized MR image according to the femoral three-dimensional rigid transformation parameters and the tibial three-dimensional rigid transformation parameters to obtain a registered MR image; taking the registered MR image, the normalized CT image, the CT edge map and the bone tag surface map as input, it extracts local image blocks within the target ROI region, constructs degraded MR input by using a high-resolution ROI super-resolution branch through downsampling, upsampling and restoration, and uses MR low-frequency information, CT intensity, CT edge and bone tag surface prior to predict high-frequency residuals, adds the high-frequency residuals back to the degraded MR, and obtains high-resolution MR image blocks; The cross-layer bidirectional feature fusion unit 604 is used to extract multi-scale registration features at different scales of the encoder and decoder of the BLHR bone-locked registration branch. Based on the spatial position of each high-resolution ROI image patch in the original image, the multi-scale registration features are cropped and interpolated to the same size as the corresponding ROI image patch to obtain cropped registration features. Through the cross-layer bidirectional Cross-Stitch feature fusion module, the cropped registration features of each level are bidirectionally updated with the corresponding super-resolution branch features. The bone-locked geometric features of the BLHR bone-locked registration branch guide MR high-frequency recovery, and the local detail features of the high-resolution ROI super-resolution branch inversely supplement the registration representation. The result output unit 605 is used to weighted fuse the high-definition MR image blocks of each ROI region through an overlapping sliding window, and output a high-definition registered MR image consistent with the CT space.
[0076] This invention can also provide a high-resolution reconstruction device for knee joint CT-MR images, the device including a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of the above-described high-resolution reconstruction method for knee joint CT-MR images according to the instructions in the program code.
[0077] like Figure 7As shown in the figure, a high-definition reconstruction device for knee joint CT-MR images provided in this embodiment of the invention may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.
[0078] In this embodiment of the invention, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.
[0079] The processor 10 can call the program stored in the memory 11. Specifically, the processor 10 can execute the operations in the embodiment of the high-definition reconstruction method of knee joint CT-MR images.
[0080] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 11 stores at least a program for implementing the following functions: S1: Acquire CT images, MR images, CT bone tags, and MR bone tags of the same patient's knee joint; using the CT images as a reference space, resample the MR images and MR bone tags to the CT grid to obtain resampled MR images and MR bone tags; perform robust normalization on the CT images and MR images to obtain normalized CT images and normalized MR images; construct CT edge maps, bone tag surface maps, soft tissue masks, MR overlay masks, and displacement amplitude maps. S2: Input the normalized CT image, normalized MR image, CT bone label, MR bone label, original bone cavity mask and soft tissue mask into the low-resolution BLHR bone locking registration branch. The BLHR bone locking registration branch adopts a three-dimensional encoder-decoder structure. After the encoder extracts the joint spatial features, the decoder outputs the femoral three-dimensional rigidity transformation parameters, tibial three-dimensional rigidity transformation parameters, soft tissue three-dimensional displacement field and multi-scale registration features. S3: Based on the three-dimensional rigidity transformation parameters of the femur and the tibial bone, bone-locking rigidity transformation synthesis is performed on the normalized MR image to obtain a registered MR image; the registered MR image, the normalized CT image, the CT edge map, and the bone tag surface map are used as input to extract local image blocks within the target ROI region. A degraded MR input is constructed using a high-resolution ROI super-resolution branch through downsampling, upsampling, and restoration. High-frequency residuals are predicted using MR low-frequency information, CT intensity, CT edges, and bone tag surface priors. The high-frequency residuals are added back to the degraded MR to obtain a high-resolution MR image block. S4: Extract multi-scale registration features at different scales of the encoder and decoder of the BLHR bone-locked registration branch. Based on the spatial position of each high-resolution ROI image patch in the original image, crop the multi-scale registration features and interpolate them to the same size as the corresponding ROI image patch to obtain cropped registration features. Through the cross-layer bidirectional Cross-Stitch feature fusion module, the cropped registration features of each level are bidirectionally updated with the corresponding super-resolution branch features. The bone-locked geometric features of the BLHR bone-locked registration branch guide MR high-frequency recovery, and the local detail features of the high-resolution ROI super-resolution branch inversely supplement the registration representation. S5: The high-resolution MR image blocks of each ROI region are weighted and fused through an overlapping sliding window to output a high-resolution registered MR image consistent with the CT space.
[0081] In one possible implementation, the memory 11 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function (such as file creation or data read / write). The data storage area may store data created during use, such as initialization data.
[0082] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.
[0083] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.
[0084] Of course, it should be noted that, Figure 7 The structure shown does not constitute a limitation on the high-definition reconstruction device for knee CT-MR images in the embodiments of the present invention. In practical applications, the high-definition reconstruction device for knee CT-MR images may include more than Figure 7 More or fewer components as shown, or combinations of certain components.
[0085] This invention can also provide a computer-readable storage medium for storing program code for executing the steps of the above-described high-resolution reconstruction method for knee joint CT-MR images.
[0086] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0087] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0088] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0089] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. A knee joint CT-MR image high-definition reconstruction method, characterized in that, The method includes: S1: Acquire CT images, MR images, CT bone tags, and MR bone tags of the same patient's knee joint; using the CT images as a reference space, resample the MR images and MR bone tags to the CT grid to obtain resampled MR images and MR bone tags; perform robust normalization on the CT images and MR images to obtain normalized CT images and normalized MR images; construct CT edge maps, bone tag surface maps, soft tissue masks, MR overlay masks, and displacement amplitude maps. S2: Input the normalized CT image, normalized MR image, CT bone label, MR bone label, original bone cavity mask and soft tissue mask into the low-resolution BLHR bone locking registration branch. The BLHR bone locking registration branch adopts a three-dimensional encoder-decoder structure. After the encoder extracts the joint spatial features, the decoder outputs the femoral three-dimensional rigidity transformation parameters, tibial three-dimensional rigidity transformation parameters, soft tissue three-dimensional displacement field and multi-scale registration features. S3: Based on the three-dimensional rigidity transformation parameters of the femur and the tibial bone, bone-locking rigidity transformation synthesis is performed on the normalized MR image to obtain a registered MR image; the registered MR image, the normalized CT image, the CT edge map, and the bone tag surface map are used as input to extract local image blocks within the target ROI region. A degraded MR input is constructed using a high-resolution ROI super-resolution branch through downsampling, upsampling, and restoration. High-frequency residuals are predicted using MR low-frequency information, CT intensity, CT edges, and bone tag surface priors. The high-frequency residuals are added back to the degraded MR to obtain a high-resolution MR image block. S4: Extract multi-scale registration features at different scales of the encoder and decoder of the BLHR bone-locked registration branch. Based on the spatial position of each high-resolution ROI image patch in the original image, crop the multi-scale registration features and interpolate them to the same size as the corresponding ROI image patch to obtain cropped registration features. Through the cross-layer bidirectional Cross-Stitch feature fusion module, the cropped registration features of each level are bidirectionally updated with the corresponding super-resolution branch features. The bone-locked geometric features of the BLHR bone-locked registration branch guide MR high-frequency recovery, and the local detail features of the high-resolution ROI super-resolution branch inversely supplement the registration representation. S5: The high-resolution MR image blocks of each ROI region are weighted and fused through an overlapping sliding window to output a high-resolution registered MR image consistent with the CT space.
2. The knee CT-MR image high-definition reconstruction method of claim 1, characterized in that, In step S2, the rigid transformation can be expressed as: wherein denotes a set of bone structures to be rigidly modeled, denotes a rotation matrix, denotes a translation vector; The input channel of the three-dimensional encoder-decoder structure consists of normalized CT, normalized MR, CT femoral label, CT tibial label, MR femoral label, MR tibial label, original bone cavity mask, and soft tissue mask. 3.The knee joint CT-MR image high-definition reconstruction method according to claim 1, characterized in that, In step S3, the inputs to the high-resolution ROI super-resolution branch include degraded MR, CT intensity, CT margins, and bone surface priors; The degraded MR image is constructed by downsampling, upsampling and restoration of the registered MR image, the CT intensity comes from the normalized CT image, the CT edge comes from the CT edge map, and the bone surface prior comes from the bone tag surface map.
4. The knee CT-MR image high-definition reconstruction method of claim 1, characterized in that, In step S4, the cross-layer bidirectional Cross-Stitch feature fusion module sets up a bidirectional cross-fusion unit in each super-resolution module and updates the features according to the following formula: wherein, and respectively represent the super-resolution feature and the registration feature of the first layer, and are learnable gating parameters; The cross-convolution in the bidirectional cross-fusion unit has zero weights during initialization, making the initial state of the module equivalent to an identity mapping.
5. The knee CT-MR image high-definition reconstruction method according to claim 4, characterized in that, In the cross-layer bidirectional Cross-Stitch feature fusion module, the multi-scale registration features come from the encoder output and decoder output of the BLHR bone-locked registration branch at each scale; the corresponding super-resolution branch features come from the SR modules in the high-resolution ROI super-resolution branch that correspond to each scale of the registration branch encoder-decoder; the bidirectional interactive update is completed within each SR module, and the updated super-resolution features continue to propagate to the next layer SR module.
6. The knee CT-MR image high-definition reconstruction method according to claim 1, characterized in that, Step S5 includes a rigid security judgment step before weighted fusion: Rigid safety judgment is performed based on the change in the overlap between the predicted bone tag and the reference bone tag. When a rigid transformation causes the bone tag overlap to decrease by more than a preset threshold, the rigid transformation is rejected and excluded from the final fusion result.
7. The knee CT-MR image high-definition reconstruction method according to claim 1, characterized in that, In step S5, the weighted fusion of overlapping sliding windows specifically includes: Upsample MR, CT, and bone priors within each ROI region to the target resolution; By using overlapping sliding windows, each image block is sequentially input into the coupled SR branch for prediction, resulting in multiple overlapping high-resolution MR prediction blocks. A weighted fusion method is used to reduce block artifacts in the multiple overlapping high-definition MR prediction blocks, and the final high-definition registered MR image is output. 8.The knee CT-MR image high-definition reconstruction method of claim 1, characterized in that, It also includes joint training steps: The soft tissue displacement field predicted by the step S2 is constrained with the reference displacement field and the reference bone labels to build a registration geometry consistency loss ; Constructing a voxel-level image alignment loss using local normalized cross-correlation ; Constructing high-resolution MR reconstruction loss using reconstruction error, gradient or high-frequency error, and registration-guided consistency. ; Using the overall loss function The BLHR bone-locking registration branch and the high-resolution ROI super-resolution branch are jointly optimized; These represent the corresponding weight coefficients; In the initial stage of training, the existing independent SR model weights are loaded, so that the Cross-Stitch module starts training from the identity mapping, and then the feature interaction between the registration branch and the SR branch is gradually released.
9. The method for high-resolution reconstruction of knee joint CT-MR images according to claim 1, characterized in that, The CT edge map constructed in step S1 is obtained by edge detection from the normalized CT image; the bone tag surface map is obtained by surface extraction from the CT bone tag; the soft tissue mask is obtained from the reverse region of the CT bone tag; the MR overlay mask is obtained from the non-zero region of the normalized MR image; and the displacement amplitude map is calculated from the amplitude of the soft tissue displacement field.
10. A high-definition reconstruction device for CT-MR images of the knee joint, characterized in that, The apparatus for performing the high-resolution reconstruction method of knee joint CT-MR images according to any one of claims 1-9, the apparatus comprising: An image preprocessing unit is used to acquire CT images, MR images, CT bone tags, and MR bone tags of the same patient's knee joint; using the CT images as a reference space, the MR images and MR bone tags are resampled onto a CT grid to obtain resampled MR images and MR bone tags; robust normalization is performed on the CT images and MR images to obtain normalized CT images and normalized MR images; and CT edge maps, bone tag surface maps, soft tissue masks, MR overlay masks, and displacement amplitude maps are constructed. The bone locking registration unit is used to input the normalized CT image, normalized MR image, CT bone label, MR bone label, original bone cavity mask and soft tissue mask into the low-resolution BLHR bone locking registration branch. The BLHR bone locking registration branch adopts a three-dimensional encoder-decoder structure. After the encoder extracts the joint spatial features, the decoder outputs the three-dimensional rigidity transformation parameters of the femur, the three-dimensional rigidity transformation parameters of the tibia, the three-dimensional displacement field of the soft tissue and the multi-scale registration features. The super-resolution reconstruction unit is used to perform bone-locking rigid transformation synthesis on the normalized MR image according to the femoral three-dimensional rigidity transformation parameters and the tibial three-dimensional rigidity transformation parameters to obtain a registered MR image; taking the registered MR image, the normalized CT image, the CT edge map, and the bone tag surface map as input, it extracts local image blocks within the target ROI region, constructs degraded MR input through downsampling, upsampling, and restoration using a high-resolution ROI super-resolution branch, and predicts high-frequency residuals using MR low-frequency information, CT intensity, CT edges, and bone tag surface priors, and adds the high-frequency residuals back to the degraded MR to obtain a high-resolution MR image block; A cross-layer bidirectional feature fusion unit is used to extract multi-scale registration features at different scales of the encoder and decoder of the BLHR bone-locked registration branch. Based on the spatial position of each high-resolution ROI image patch in the original image, the multi-scale registration features are cropped and interpolated to the same size as the corresponding ROI image patch to obtain cropped registration features. Through a cross-layer bidirectional Cross-Stitch feature fusion module, the cropped registration features of each level are bidirectionally updated with the corresponding super-resolution branch features. The bone-locked geometric features of the BLHR bone-locked registration branch guide MR high-frequency recovery, and the local detail features of the high-resolution ROI super-resolution branch inversely supplement the registration representation. The result output unit is used to weighted fuse the high-definition MR image blocks of each ROI region through an overlapping sliding window, and output a high-definition registered MR image consistent with the CT space.
