Multi-modal image fusion cylindrical skeleton repair detection method and system
By acquiring multimodal image data of tubular skeletons, spatial registration and feature fusion are performed to construct a three-dimensional skeleton model, which solves the problem of insufficient accuracy of two-dimensional imaging technology in tubular skeleton repair detection and realizes multi-dimensional repair detection and evaluation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-28
- Publication Date
- 2026-04-24
AI Technical Summary
In existing technologies, the detection of tubular bone repair relies on a single two-dimensional X-ray imaging technique, which makes it difficult to clearly distinguish the three-dimensional structure of the bone and subtle repair changes, resulting in inaccurate analysis results.
Multimodal imaging data (CT, MRI, and X-ray images) of tubular bones are acquired. Modal spatial registration is performed by identifying trabecular bone structural feature points to obtain spatially registered images. Multidimensional tubular bone features are identified and feature-level fusion is performed to construct a three-dimensional bone model and finally generate a repair detection report.
It enables multi-dimensional feature capture and three-dimensional display of tubular bone repair detection, improving the accuracy of repair detection and providing a comprehensive and accurate basis for repair assessment.
Smart Images
Figure CN121921240A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method and system for detecting and repairing tubular bones using multimodal image fusion, belonging to the field of image analysis technology. Background Technology
[0002] Tubular bones are bones with a long tubular structure, usually with a long diaphysis and epiphyses at both ends. The diaphysis is a tubular structure composed of cortical bone, containing a medullary cavity. They play important roles in supporting the body, transmitting force, and participating in hematopoiesis. They are more prone to fractures and other injuries when subjected to external impacts. Monitoring their repair process is crucial for adjusting treatment plans and predicting prognosis.
[0003] Currently, the detection of tubular bone repair primarily relies on single-imaging techniques, such as X-ray imaging. The principle is to use X-rays to project images of the patient's bone and other high-density tissues onto film or a digital detector, creating a contrasting image of the tubular bone. This reveals the overall shape of the tubular bone, the location of the fracture line, and the approximate growth of callus, helping doctors assess the progress of the repair. However, because X-ray imaging is two-dimensional, it can cause image overlap in the three-dimensional structure of the tubular bone, making it difficult to clearly distinguish the bone's three-dimensional structure and subtle repair changes. For example, it cannot accurately determine the distribution and growth direction of the callus in three-dimensional space, leading to less precise analysis results. Summary of the Invention
[0004] This invention provides a method and system for detecting tubular bone repair using multimodal image fusion, the main purpose of which is to improve the accuracy of tubular bone repair detection.
[0005] To achieve the above objectives, the present invention provides a method for detecting and repairing tubular bones using multimodal image fusion, comprising: Multimodal image data of a tubular skeleton is acquired, including CT images, MRI images and X-ray images. Trabecular bone structure feature points of each modality in the multimodal image data are identified. Modal spatial registration is performed on the multimodal image data using the trabecular bone structure feature points to obtain spatially registered images. Using the spatially registered images, the cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour, and fracture line location of the tubular skeleton are identified to obtain multidimensional tubular skeleton features; The multidimensional tubular skeleton features are fused at the feature level to obtain a fused feature map of the tubular skeleton. The fused feature map is then used to construct a three-dimensional skeleton model of the tubular skeleton. Using the three-dimensional model of the skeleton, the callus volume and cortical continuity of the corresponding repair area of the tubular skeleton are identified. Based on the callus volume and cortical continuity, the integrity of the skeleton and the degree of fusion of adjacent bones in the repair area are analyzed. Based on the integrity of the skeleton and the degree of fusion of adjacent bones, a repair inspection report of the tubular skeleton is constructed.
[0006] Optionally, based on the callus volume and the continuity of the bone cortex, the skeletal integrity and adjacent bone fusion of the repaired area are analyzed, including: Based on the callus volume, the mean volume of callus defect and the mean callus density in the repair area were analyzed; Based on the volume of the callus defect and the average callus density, the callus filling efficiency index of the repair area is calculated; The bone cortex fracture characteristics corresponding to the continuous state of the bone cortex are queried, and the bone integrity of the repair area is calculated by combining the bone cortex fracture characteristics and the callus filling efficiency index. Based on the fracture characteristics of the cortical bone, the degree of fusion of adjacent bones in the repair area was analyzed.
[0007] Optionally, the analysis of the degree of fusion of adjacent bones in the repair area based on the fracture characteristics of the cortical bone includes: Based on the fracture characteristics of the cortical bone, the number of fusion feature points and the contact area with adjacent bone in the repair area are identified; The degree of fusion of adjacent bones in the repaired area is calculated based on the number of fusion feature points and the contact area of adjacent bones.
[0008] Optionally, modal spatial registration is performed on the multimodal image data using the trabecular bone structure feature points to obtain a spatially registered image, including: The local structural complexity of the trabecular bone structure feature points is analyzed, and based on the local structural complexity, structural points are screened to obtain baseline structural points. The reference structural points are spatially mapped to obtain the mapped structural points; The spatial coordinates of the mapped structural points are transformed to obtain the coordinates of the structural points; Based on the coordinates of the structural points, modal spatial registration is performed on the multimodal image data to obtain a spatially registered image.
[0009] Optionally, using the spatially registered image, the cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour, and fracture line location of the tubular skeleton are identified to obtain multidimensional tubular skeleton features, including: Query the CT image region, MRI image region, and X-ray image region in the spatially registered image; Using the CT image region, the cortical bone thickness and trabecular bone density of the tubular bone are identified to obtain CT image recognition features; Using the MRI image region, the morphology of the medullary cavity and the soft tissue edema area of the tubular bone are identified, and MRI image recognition features are obtained; The overall outline of the tubular skeleton and the location of the fracture line are identified using the X-ray image region to obtain X-ray image recognition features; Based on the CT image recognition features, the MRI image recognition features, and the X-ray image recognition features, the multidimensional tubular bone features of the tubular skeleton are determined.
[0010] Optionally, using the three-dimensional model of the skeleton, the callus volume and cortical continuity of the corresponding repair area of the tubular bone are identified, including: Using the three-dimensional model of the skeleton, the boundary surface between the repair area and the normal area of the tubular skeleton is identified; The callus region within the boundary surface is segmented using voxelization to obtain three-dimensional volume data of the callus. The three-dimensional volume data of the callus described in the gene are used to calculate the callus volume of the corresponding repair area of the tubular bone. Based on the three-dimensional model of the skeleton, the surface curvature of the cortical layer corresponding to the tubular skeleton is identified; Based on the surface curvature, the fracture characteristics of the cortical bone in the repaired area are analyzed to determine the continuity of the cortical bone in the repaired area according to the fracture characteristics.
[0011] Optionally, feature-level fusion is performed on the multidimensional tubular skeleton features to obtain a fused feature map of the tubular skeleton, including: The multidimensional tubular skeleton features are normalized in spatial coordinates to obtain a standard parameter set, and the standard parameter set is then subjected to principal component dimensionality reduction to obtain a dimensionality-reduced feature vector. Adaptive weight allocation is performed on the dimensionality-reduced feature vectors to obtain the weighting coefficient matrix of each feature vector in the dimensionality-reduced feature vectors; The weighted coefficient matrix and the standard parameter set are convolved and fused to obtain an initial fused feature map; The initial fusion feature map is reconstructed using a three-dimensional surface to obtain the final fusion feature map of the tubular skeleton.
[0012] Optionally, using the fused feature map, a 3D skeletal model of the tubular skeleton is constructed, including: Extract the isosurface of the fused feature map to obtain the initial point cloud data of the bone tissue corresponding to the tubular skeleton; The initial point cloud data is triangulated to obtain a preliminary three-dimensional mesh model; The preliminary three-dimensional mesh model is subjected to normal vector consistency optimization to obtain a smooth surface model; The smooth surface model is topologically calibrated to obtain a skeletal 3D model.
[0013] Optionally, based on the bone integrity and the degree of fusion of adjacent bones, a repair inspection report for the tubular bone is constructed, including: Based on the bone integrity and the degree of fusion of adjacent bones, the bone integrity index and the percentage of fusion of adjacent bones of the tubular bone are determined; The overall repair score of the tubular bone is calculated based on the bone integrity index and the percentage of fusion between adjacent bones. Based on the comprehensive repair score, the bone healing status of the tubular bone is assessed, and a repair detection report of the tubular bone is constructed based on the bone healing status.
[0014] To address the aforementioned problems, the present invention also provides a multimodal image fusion-based tubular bone repair and detection system, the system comprising: The image preprocessing module is used to acquire multimodal image data of the tubular skeleton, wherein the multimodal image data includes CT images, MRI images and X-ray images, identify the trabecular bone structure feature points of each modality in the multimodal image data, and use the trabecular bone structure feature points to perform modal spatial registration on the multimodal image data to obtain spatially registered images. The feature recognition module is used to identify the cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour and fracture line position of the tubular bone using the spatially registered image, so as to obtain multidimensional tubular bone features. The model building module is used to perform feature-level fusion on the multidimensional tubular skeleton features to obtain a fused feature map of the tubular skeleton, and to construct a three-dimensional skeleton model of the tubular skeleton using the fused feature map. The repair detection module is used to identify the callus volume and cortical continuity of the corresponding repair area of the tubular bone using the three-dimensional model of the skeleton. Based on the callus volume and cortical continuity, it analyzes the integrity of the skeleton and the fusion of adjacent bones in the repair area. Based on the integrity of the skeleton and the fusion of adjacent bones, it constructs a repair detection report for the tubular bone.
[0015] This invention first acquires multimodal image data of tubular bones using CT, MRI, and X-ray imaging. By identifying trabecular bone structural feature points in each modality, modal spatial registration is achieved, resulting in spatially registered images. This transforms the originally independent multi-source image data into a unified spatial coordinate system, providing a consistent spatial basis for subsequent feature extraction and avoiding feature misjudgment caused by image misalignment. This ensures data reliability from the source and lays a spatially consistent foundation for a comprehensive analysis of bone repair status. Then, this invention utilizes the spatially registered images to extract multidimensional tubular bone features such as cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour, and fracture line location. This achieves comprehensive capture of bone repair-related features, compensating for the lack of information dimensions in a single modality. It provides rich and complementary feature materials for subsequent fusion analysis, enabling the assessment of repair status to move beyond local information and acquire multidimensional reference data. Next, this invention performs feature-level fusion of multidimensional tubular bone features to obtain a fused feature map and constructs a three-dimensional bone model. This transforms two-dimensional planar features into a three-dimensional spatial structure, not only visually displaying the overall morphology of the tubular bone but also presenting the spatial distribution of the repair area from a three-dimensional perspective. This allows users to observe repair details from multiple angles, accurately determine the spatial relationship between the callus and surrounding bones, and provides intuitive and accurate model support for assessing the three-dimensional integrity of the repair. Finally, based on the three-dimensional bone model, the callus volume and cortical continuity of the repair area are identified, and the bone integrity and adjacent bone fusion are analyzed to generate a repair detection report. This step realizes the transformation from image features to clinical evaluation indicators, quantifying abstract image information into specific repair parameters. Therefore, this invention can improve the accuracy of tubular bone repair detection. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating a method for detecting and repairing tubular bones using multimodal image fusion, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a module for implementing a multimodal image fusion method for detecting and repairing tubular bones, according to an embodiment of the present invention.
[0017] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0019] This application provides a method for detecting and repairing tubular bones using multimodal image fusion. The execution entity of this method includes, but is not limited to, at least one electronic device configured to execute the method provided in this application, such as a server or a terminal. In other words, the method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.
[0020] Reference Figure 1 The diagram shown is a flowchart illustrating a method for detecting and repairing tubular bones using multimodal image fusion according to an embodiment of the present invention. In this embodiment, the method for detecting and repairing tubular bones using multimodal image fusion includes: S1. Acquire multimodal image data of a tubular skeleton, wherein the multimodal image data includes CT images, MRI images and X-ray images, identify the trabecular bone structure feature points of each modality in the multimodal image data, and use the trabecular bone structure feature points to perform modal spatial registration on the multimodal image data to obtain a spatially registered image.
[0021] This invention, through the acquisition of multimodal imaging data of tubular bones, can obtain multidimensional information such as bone structure details, soft tissue and bone marrow status, and overall morphology via CT, MRI, and X-ray, providing comprehensive original evidence for repair detection.
[0022] The multimodal image data includes CT images, MRI images, and X-ray images.
[0023] Optionally, the multimodal image data can be obtained by acquiring corresponding CT images, MRI images, and X-ray images through a CT scanner, magnetic resonance imaging equipment, and X-ray machine, respectively.
[0024] Furthermore, by identifying the trabecular bone structural feature points of each modality in the multimodal image data, embodiments of the present invention can locate corresponding common skeletal structural markers in CT, MRI, and X-ray images, providing a unified reference for the association of images from different modalities. For example, in tibial repair detection, the same trabecular bifurcation points in the midshaft of the tibia can be identified as common feature markers in the three types of images.
[0025] The locations of the trabecular bone structure within the cancellous bone of the skeleton that have distinctive morphological features, such as the bifurcation, intersection, and endpoints of the trabecular bone.
[0026] In practice, edge detection and feature matching algorithms are used for CT, MRI and X-ray images respectively to locate the bifurcation and intersection points of trabecular bone in the images, thereby identifying the structural feature points of trabecular bone in each modality.
[0027] Furthermore, this embodiment of the invention utilizes the feature points of the trabecular bone structure to perform modal spatial registration on the multimodal image data. The resulting spatially registered image can eliminate spatial positional deviations in CT, MRI, and X-ray images, aligning different modal images in the same coordinate system and ensuring the corresponding positions of bone structures in the images. For example, in tibial repair detection, this step allows the trabecular bone structure at the same location in the tibia to accurately overlap in the three images, avoiding structural misjudgments caused by spatial misalignment.
[0028] As an embodiment of the present invention, modal spatial registration is performed on the multimodal image data using the feature points of the trabecular bone structure to obtain a spatially registered image, including: The local structural complexity of the trabecular bone structure feature points is analyzed, and based on the local structural complexity, structural points are screened to obtain baseline structural points. The reference structural points are spatially mapped to obtain the mapped structural points; The spatial coordinates of the mapped structural points are transformed to obtain the coordinates of the structural points; Based on the coordinates of the structural points, modal spatial registration is performed on the multimodal image data to obtain a spatially registered image.
[0029] The local structural complexity refers to the structural complexity within the neighborhood of a trabecular structural feature point. Specifically, it is comprehensively reflected by indicators such as the number of branches, intersection angle, and gray-scale entropy value of the trabecular bone in that region, reflecting the stability and recognizability of the feature point in spatial distribution. The benchmark structural point refers to the feature point selected from the trabecular structural feature points that has high local structural complexity (e.g., local entropy value > 0.6 and number of branches ≥ 3).
[0030] In specific implementation, when analyzing the local structural complexity of the trabecular bone feature points, this can be achieved by calculating the number of branches, intersection angles, and gray-level entropy values of the trabecular bone within the neighborhood of the feature point. Feature points with a local entropy value greater than 0.6 (the threshold can be adjusted according to the bone characteristics of different parts such as the femur and tibia) and a number of branches ≥3 are defined as high-complexity points, and these points are selected as baseline structural points. For example, in femoral CT images, the local entropy value is calculated for feature points in the trabecular bone intersection area, and points with entropy values of 0.7~0.9 are retained as baseline structural points to ensure stable spatial recognition. A thin-plate spline interpolation algorithm is used to establish a mapping relationship between different modal images (such as CT and MRI). The coordinates of the baseline structural points in the CT image are used as input, and the mapped structural points are generated by matching the trabecular bone texture features of the corresponding region in the MRI image. For example, the baseline points selected in the tibial CT are mapped to the same anatomical location in the MRI image through texture similarity matching, achieving cross-modal spatial correspondence. Based on a rigid body transformation model (rotation matrix + translation vector), the coordinate transformation parameters are solved using the least squares method to transform the mapped structural points from the original modal coordinate system to the target modal coordinate system (e.g., transforming MRI mapping points to the CT coordinate system), ensuring that the coordinate deviation is controlled within 0.5mm. For example, in radial multimodal image registration, this transformation ensures that the three-dimensional coordinate error between the MRI mapping point and the CT reference point is ≤0.3mm. When performing modal spatial registration of the multimodal image data based on the coordinates of the structural points, the Iterative Closest Point (ICP) algorithm can be used to iteratively optimize the image position, minimizing the sum of squared spatial distances between corresponding structural points in different modalities (with a convergence threshold set to 10). -6 This process ultimately yields spatially registered images. For example, in CT and MRI registration of the ulna, this method reduces the average spatial deviation of the trabecular endpoints and intersections from an initial 2.1 mm to 0.8 mm, achieving precise alignment.
[0031] S2. Using the spatially registered image, identify the cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour, and fracture line location of the tubular skeleton to obtain multidimensional tubular skeleton features.
[0032] This invention utilizes the spatially registered images to identify the cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour, and fracture line location of the tubular skeleton. This multidimensional tubular skeleton feature can integrate the advantages of multimodal imaging, comprehensively capturing key repair information such as bone structure (e.g., cortical bone thickness) and tissue state (e.g., soft tissue edema), avoiding the one-sidedness of single features, and thus providing multidimensional basis for repair assessment.
[0033] The cortical bone thickness refers to the average distance between the outer and inner edges of the cortical bone in the shaft of the tubular bone, reflecting the density and structural strength of the cortical bone. The trabecular bone density refers to the volume ratio of trabecular bone structure per unit volume within the cancellous bone region of the tubular bone, reflecting the supporting capacity and bone condition of the cancellous bone. The medullary cavity morphology refers to the three-dimensional spatial morphological characteristics of the medullary cavity inside the tubular bone, including parameters such as the longest axis length, cross-sectional area, and volume, reflecting the structural integrity of the medullary cavity. The soft tissue edema area refers to the abnormal edema area in the soft tissue surrounding the tubular bone caused by inflammation or injury, defined by the grayscale features of MRI images, and characterized by volume parameters. The overall bone contour refers to the overall outer boundary of the tubular bone as presented in X-ray images, including the contour morphology of the shaft and the epiphyses at both ends, reflecting the macroscopic morphological characteristics of the bone. The fracture line position refers to the spatial coordinates and direction of the fracture line at the fracture site of the tubular bone in the image, determined by edge detection and linear feature recognition, indicating the specific location and morphology of the fracture.
[0034] As an embodiment of the present invention, the spatially registered image is used to identify the cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour, and fracture line location of the tubular skeleton, thereby obtaining multidimensional tubular skeleton features, including: Query the CT image region, MRI image region, and X-ray image region in the spatially registered image; Using the CT image region, the cortical bone thickness and trabecular bone density of the tubular bone are identified to obtain CT image recognition features; Using the MRI image region, the morphology of the medullary cavity and the soft tissue edema area of the tubular bone are identified, and MRI image recognition features are obtained; The overall outline of the tubular skeleton and the location of the fracture line are identified using the X-ray image region to obtain X-ray image recognition features; Based on the CT image recognition features, the MRI image recognition features, and the X-ray image recognition features, the multidimensional tubular bone features of the tubular skeleton are determined.
[0035] In specific implementation, when identifying cortical bone thickness and trabecular bone density using the CT image region, the Canny operator (threshold set to 50-150 HU) can be used to extract the inner and outer edges of the cortical bone in the CT region, and the vertical distance can be calculated to obtain the thickness value (normal threshold 2-6 mm). The trabecular bone region can be segmented using a threshold of 150-400 HU, and its proportion can be calculated as the density value. The medullary cavity can be segmented using a T1-weighted sequence (signal intensity threshold set to 200-400), and the edema region can be segmented using a T2-weighted sequence (signal intensity 20% higher than the normal soft tissue mean as the threshold). Morphological processing can then be used to extract the three-dimensional morphological parameters of both. For example, in the tibial MRI region, the extracted midline length of the medullary cavity is 32 cm, and the volume of the edema region is 6.8 cm³. 3 Adaptive threshold binarization (with the threshold adjusted based on the image's average grayscale value ±30) was used to extract the bone contour in the X-ray image region. Fracture lines were identified using Canny edge detection combined with Hough transform (with a straight line detection threshold set to 80). For example, in the radius X-ray region, the extracted bone contour matched the anatomical structure, and a fracture line approximately 10 mm long was detected. When determining multidimensional tubular bone features based on three image recognition features, coordinate mapping was used to associate each feature with a unified spatial coordinate system. Abnormal features with spatial position deviations >1 mm were removed, and six types of features, including cortical bone thickness and trabecular bone density, were integrated to form structured data.
[0036] S3. Perform feature-level fusion on the multidimensional tubular skeleton features to obtain a fused feature map of the tubular skeleton, and use the fused feature map to construct a three-dimensional skeleton model of the tubular skeleton.
[0037] This invention, through feature-level fusion of the multidimensional tubular bone features, yields a fused feature map that integrates multi-dimensional information such as bone structure and tissue state. This preserves the unique information of each feature while strengthening the correlation between different features, avoiding information fragmentation. For example, in tibial repair detection, the fused feature map can clearly show the correspondence between the fracture line location and the density changes of surrounding trabecular bone, allowing the structural and functional state of bone repair to form an intuitive association in the same image, providing a more complete feature basis for subsequent analysis.
[0038] The fused feature map refers to a feature representation formed by integrating feature information from multiple different dimensions or sources through a feature-level fusion algorithm. It integrates the effective information of each individual feature and can more comprehensively reflect the attributes or state of the target object.
[0039] As an embodiment of the present invention, feature-level fusion is performed on the multidimensional tubular skeleton features to obtain a fused feature map of the tubular skeleton, including: The multidimensional tubular skeleton features are normalized in spatial coordinates to obtain a standard parameter set, and the standard parameter set is then subjected to principal component dimensionality reduction to obtain a dimensionality-reduced feature vector. Adaptive weight allocation is performed on the dimensionality-reduced feature vectors to obtain the weighting coefficient matrix of each feature vector in the dimensionality-reduced feature vectors; The weighted coefficient matrix and the standard parameter set are convolved and fused to obtain an initial fused feature map; The initial fusion feature map is reconstructed using a three-dimensional surface to obtain the final fusion feature map of the tubular skeleton.
[0040] The weighted coefficient matrix refers to the matrix obtained by an adaptive weight allocation algorithm (such as particle swarm optimization algorithm) and used to characterize the importance of each feature in the dimensionality reduction feature vector. The initial fusion feature map refers to the feature map generated by fusing the weighted coefficient matrix and the standard parameter set through convolution operation (such as 3×3×3 three-dimensional convolution kernel), which contains the spatial correlation information of multidimensional tubular skeleton features.
[0041] In practice, a unified coordinate system can be established based on the skeletal midline. Affine transformations are used to map the coordinates of features such as cortical bone thickness and medullary cavity morphology to this coordinate system (coordinate deviation controlled within 0.3 mm) to obtain a standard parameter set. Principal component analysis (PCA) is then used to retain principal components with a cumulative contribution rate ≥95%, reducing the 6-dimensional original features to 3-dimensional reduced feature vectors. Particle swarm optimization iteratively optimizes the weights, using the feature's contribution to repair assessment (e.g., the contribution of fracture line location is set to 0.35) as the objective function, resulting in a weighted coefficient matrix for each feature vector. For example, in femoral features, the coefficient corresponding to cortical bone thickness is set to 0.28, and the coefficient for soft tissue edema region is set to 0.12. A 3×3×3 three-dimensional convolution kernel (with a stride of 1) is used to convolve the weighted reduced feature vectors with the original features in the standard parameter set, outputting an initial fused feature map containing spatial correlation information. For example, in radius feature fusion, the convolved feature map can simultaneously retain details of trabecular bone density and the spatial location of the fracture line. When reconstructing the initial fusion feature map into a three-dimensional surface, the Moving Cube Algorithm (MC algorithm) can be used. The isosurface threshold is set to 0.6 (adjusted according to the grayscale range of the feature map [0, 1]), and the two-dimensional feature map is converted into a three-dimensional mesh model. Finally, a tubular bone fusion feature map that can intuitively display the spatial distribution of multiple features is obtained. For example, the reconstructed tibial fusion feature map can three-dimensionally present the correlation between the changes in cortical bone thickness and the morphology of the medullary cavity.
[0042] Furthermore, by utilizing the fused feature map, the embodiments of the present invention construct a three-dimensional model of the tubular skeleton, which can accurately present the spatial correlation of multi-dimensional features such as cortical bone thickness and trabecular bone density in a three-dimensional form, and break through the perspective limitations of two-dimensional images, allowing the structural details and feature distribution of the skeleton to form an interactive and intuitive form in three-dimensional space.
[0043] The 3D skeletal model refers to a three-dimensional digital model constructed based on the fusion feature map of a tubular skeleton.
[0044] As an embodiment of the present invention, a three-dimensional skeletal model of the tubular skeleton is constructed using the fused feature map, including: Extract the isosurface of the fused feature map to obtain the initial point cloud data of the bone tissue corresponding to the tubular skeleton; The initial point cloud data is triangulated to obtain a preliminary three-dimensional mesh model; The preliminary three-dimensional mesh model is subjected to normal vector consistency optimization to obtain a smooth surface model; The smooth surface model is topologically calibrated to obtain a skeletal 3D model.
[0045] The initial point cloud data refers to a set of three-dimensional discrete points generated from the fused feature map after voxelization. Each point contains spatial coordinates (such as X, Y, and Z axis coordinates) and corresponding skeletal feature information (such as cortical bone thickness and trabecular bone density encoding value). It is the basic data for constructing a three-dimensional skeletal model. The smooth surface model refers to a continuous surface model obtained after surface extraction and optimization of the initial point cloud data.
[0046] In specific implementation, the Moving Cube Algorithm (MC algorithm) can be used, setting the isosurface threshold to 0.55 (based on the grayscale range of the fused feature map [0, 1], this threshold corresponds to the boundary between skeletal and non-skeletal tissues). The feature map voxel data is traversed to generate a discrete point set, obtaining the initial point cloud data. For example, in the femoral fusion feature map, the initial point cloud extracted by this method can accurately cover the skeletal tissue regions such as the cortex and trabeculae, with a point cloud density of 80 points per cubic millimeter. Using the greedy projection triangulation algorithm, triangular patches are constructed with a neighborhood search radius of 0.3 mm (adjusted according to the accuracy requirements of the skeletal location), connecting the discrete points into a continuous mesh to obtain a preliminary three-dimensional mesh model. For example, after processing, the initial tibial point cloud generates a mesh model containing 2 million triangular patches, which can basically represent the overall shape of the bone. When optimizing the normal vector consistency of the initial 3D mesh model, the normal vector of each triangular facet can be calculated using the least squares method. Normal vectors with conflicting directions (such as those with an angle greater than 90° between the normal vectors of adjacent facets) are flipped for correction. Then, the Laplacian smoothing algorithm is iterated three times (with a smoothing factor of 0.2) to eliminate mesh burrs, resulting in a smooth surface model. For example, in the initial femoral mesh, after optimization, the normal vector consistency of adjacent facets in the articular surface region reaches 98%, and the surface smoothness is improved by 40%. When performing topology calibration on the smooth surface model, a topology check algorithm can be used to identify non-manifold edges (such as edges shared by three or more facets) and isolated facets (area < 0.1 mm). 2 The non-manifold structure is repaired by edge splitting or merging algorithms, isolated facets are deleted, and the model topology consistency is ensured to finally obtain the skeletal 3D model.
[0047] S4. Using the three-dimensional model of the skeleton, identify the callus volume and cortical continuity of the corresponding repair area of the tubular skeleton. Based on the callus volume and cortical continuity, analyze the integrity of the skeleton and the fusion of adjacent bones in the repair area. Based on the integrity of the skeleton and the fusion of adjacent bones, construct a repair detection report for the tubular skeleton.
[0048] This invention utilizes a three-dimensional skeletal model to identify the callus volume and cortical continuity in the corresponding repair area of the tubular bone. This allows for precise quantification of the actual volume of callus growth using complete three-dimensional data, avoiding volume misjudgments caused by angular deviations in two-dimensional image slice measurements. Furthermore, the assessment of cortical continuity covers the entire repair area, rather than a localized observation on a single plane under a two-dimensional perspective, providing a more comprehensive reflection of whether the cortex has truly formed a continuous load-bearing structure. For example, in tibial fracture repair, the three-dimensional model can accurately calculate the three-dimensional distribution volume of callus around the fracture ends and clearly demonstrate whether the cortex has formed continuous connections on the anterior, posterior, medial, and lateral sides, providing a more realistic basis for evaluating the repair effect.
[0049] The bone callus volume refers to the volume of newly formed bony connective tissue (callus) around the fracture ends during the fracture repair process in three-dimensional space. It is an important quantitative indicator reflecting the bone healing process. The bone cortical continuity refers to whether the dense bone cortical tissue on the outer layer of the tubular bone has restored complete continuity after the fracture, including whether there are any breaks, defects, and the integrity of the connection.
[0050] As an embodiment of the present invention, the three-dimensional model of the skeleton is used to identify the callus volume and cortical continuity of the corresponding repair area of the tubular bone, including: Using the three-dimensional model of the skeleton, the boundary surface between the repair area and the normal area of the tubular skeleton is identified; The callus region within the boundary surface is segmented using voxelization to obtain three-dimensional volume data of the callus. The three-dimensional volume data of the callus described in the gene are used to calculate the callus volume of the corresponding repair area of the tubular bone. Based on the three-dimensional model of the skeleton, the surface curvature of the cortical layer corresponding to the tubular skeleton is identified; Based on the surface curvature, the fracture characteristics of the cortical bone in the repaired area are analyzed to determine the continuity of the cortical bone in the repaired area according to the fracture characteristics.
[0051] The term "boundary surface" refers to a smooth surface in a 3D bone model that separates the repair area (such as the callus growth area) from the normal bone area by using a region growth algorithm (with the fracture line as the seed point and based on the bone density difference threshold). This surface can accurately define the spatial range of callus growth. The term "callus 3D volume data" refers to a set of 3D voxels containing callus spatial distribution information obtained after voxelizing the callus area within the boundary surface. The term "surface curvature" refers to the degree of curvature of each vertex of the cortical bone surface calculated by principal component analysis in the 3D bone model. The term "cortical bone fracture characteristics" refers to the specific manifestations of cortical bone continuity interruption obtained based on surface curvature analysis, including the location of curvature abrupt changes, the length of fracture gaps (distance between consecutive abrupt changes), and the number of fractures. For example, "two fracture gaps (1.8mm and 0.5mm)" in the radius repair area are cortical bone fracture characteristics and can be used to determine the continuity of the cortical bone.
[0052] In practice, a region growth algorithm can be used, with the fracture line location as the seed point (e.g., coordinates of the mid-shaft femoral fracture line). A bone density difference threshold is set (the difference in bone density between normal bone tissue and the repair area > 150 HU). The boundary is determined through iterative growth, and then a B-spline curve is used to fit the boundary to generate a smooth boundary surface. The region enclosed by the boundary surface is discretized into voxels of 0.1 mm × 0.1 mm × 0.1 mm. A threshold method is used (the gray value of the callus voxels is set to 200-350 HU, excluding normal bone tissue and soft tissue) to filter the callus voxels, obtaining the three-dimensional volume data of the callus. When calculating the callus volume based on the three-dimensional volume data, the total number of callus voxels can be counted and multiplied by the volume of a single voxel (e.g., 0.001 mm). 3 The total volume is obtained, while volumes <0.5mm are discarded. 3 Isolated callus fragments (considered noise). For example, the effective callus volume for the tibial repair area is calculated to be 8.6 cm³. 3 The curvature values are within the expected range for the clinical healing stage. Principal component analysis was used to calculate the curvature value of each vertex on the cortical bone surface. A sampling interval of 0.3 mm was set, and then Gaussian filtering (standard deviation 0.5 mm) was applied to smooth the noise, resulting in a continuous curvature distribution. For example, in the calculation of the curvature of the femoral cortical bone surface, the curvature values in the normal area are mostly within ±5 mm. -1 Within the fracture zone, abrupt changes may occur due to the fracture. When analyzing cortical bone fracture characteristics based on surface curvature, a curvature abrupt change threshold can be set (curvature difference between adjacent vertices > 10 mm). -1 The mutation locations were marked and the length of the fracture gaps (such as the distance between consecutive mutation points) was counted. If the fracture gaps were ≤2mm and the number was <3, it was considered basically continuous; otherwise, it was considered discontinuous. For example, in the radial bone repair area, two fracture gaps were detected (1.8mm and 0.5mm respectively), and the cortical bone was determined to be in a basically continuous state.
[0053] Furthermore, this embodiment of the invention, by analyzing the skeletal integrity and adjacent bone fusion degree of the repaired area based on the callus volume and the continuity of the bone cortex, can comprehensively judge the repair effect from both aspects of new bone volume and structural continuity, avoiding the one-sidedness of evaluation by a single indicator. For example, in the repair of ulnar fractures, the degree of healing can be judged by the callus volume, and the fusion between the fracture ends and adjacent bones can be analyzed by combining the continuity of the bone cortex, thus comprehensively reflecting the repair status.
[0054] The bone integrity refers to the degree to which the structure of the tubular bone repair area closely resembles that of normal bone, and the adjacent bone fusion degree refers to the degree of tightness between adjacent bones in the repair area (such as bone segments on both sides of a fracture end) through callus and other bony tissues.
[0055] As an embodiment of the present invention, based on the callus volume and the continuity of the bone cortex, the skeletal integrity and adjacent bone fusion of the repaired area are analyzed, including: Based on the callus volume, the mean volume of callus defect and the mean callus density in the repair area were analyzed; Based on the volume of the callus defect and the average callus density, the callus filling efficiency index of the repair area is calculated; The bone cortex fracture characteristics corresponding to the continuous state of the bone cortex are queried, and the bone integrity of the repair area is calculated by combining the bone cortex fracture characteristics and the callus filling efficiency index. Based on the fracture characteristics of the cortical bone, the degree of fusion of adjacent bones in the repair area was analyzed.
[0056] The bone callus defect volume refers to the difference between the theoretically required callus volume (i.e., the total volume of the bone defect space) and the actual callus volume. The average callus density refers to the average value obtained by statistically analyzing the CT values (HU) of all voxels in the callus area using a three-dimensional bone model, reflecting the density and mineralization level of the callus. The callus filling efficiency index is a quantitative indicator that comprehensively considers the amount and quality of callus filling; a higher value indicates better callus filling effect and quality.
[0057] In practice, when analyzing the callus defect volume and average callus density based on the aforementioned callus volume, the theoretical callus requirement volume (i.e., defect space volume) of the repair area can be measured using a three-dimensional bone model. The callus defect volume = theoretical requirement volume - actual callus volume. Simultaneously, the voxel density statistical method is used to calculate the average CT value of all voxels in the callus area as the average callus density (the average density of normal bone tissue is set to 800-1200 HU). For example, the theoretical requirement volume for the tibial repair area is 20 cm³. 3 The actual callus volume was 15cm. 3 The callus defect volume is 5cm. 3 The mean voxel CT value of the callus region was 800 HU, which means the mean callus density was 800 HU.
[0058] Furthermore, as another embodiment of the present invention, the formula for calculating the callus filling efficacy index is as follows: ; in, Indicates the callus filling efficacy index, Indicates the volume of callus. Indicates the volume of the defect. This represents the mean callus density. This represents the mean normal bone mineral density. This represents the standard deviation of the spatial distribution of callus volume in the repair area. denoted by , where represents the average volume of voxels in the repaired region, and k represents the distribution uniformity adjustment coefficient.
[0059] It should be further explained that the calculation of the callus filling efficacy index comprehensively considers the quantity, quality, and spatial distribution characteristics of the callus. Firstly, the sufficiency of callus filling is reflected by comparing the actual growth volume of the callus with the defect volume. Then, the ratio of callus density to normal bone density reflects the mineralization quality and structural strength of the callus. Simultaneously, the spatial uniformity of callus distribution in the repair area is taken into account to avoid uneven support caused by local aggregation or sparseness. The more dispersed the distribution, the more significant the weakening of efficacy. The adjustment coefficient balances the impact of this dispersion based on the characteristics of the bone location. For example, if two cases have the same callus volume and density in the repair area, but one case has callus concentrated at the edge while the other has a uniform distribution, the former's efficacy index will be lower due to the uneven distribution.
[0060] This formula more comprehensively quantifies the repair value of callus, overcoming the limitations of assessment based solely on volume or density. It can distinguish between cases of "sufficient volume but low density" (e.g., insufficient mineralization of new callus) and "slightly insufficient volume but high density and uniform distribution" (e.g., good callus quality). It can also identify callus that meets volume requirements but has localized gaps (e.g., sparse callus at fracture gaps), thus more accurately reflecting the actual contribution of callus to bone repair. For example, in two clinical cases of tibial repair, the callus volume reached 80% of the defect volume, but in one case, the density was only 60% of normal bone and the distribution was uneven. Its efficacy index would be significantly lower than the case with 80% density and uniform distribution, providing a more detailed basis for healing assessment.
[0061] Furthermore, in yet another embodiment of the present invention, the formula for calculating skeletal integrity is as follows: ; in, Indicates skeletal integrity. Indicates the callus filling efficacy index, This represents the influence coefficient of the fracture gap. This indicates the maximum fracture gap of the tubular skeleton. Indicates the total length of the tubular bone repair area. This represents the influence coefficient of fracture length in tubular skeletons. Indicates the total fracture length of the tubular skeleton. This represents the total length of the cortical bone of the tubular skeleton. This indicates the rate of change in cortical bone thickness.
[0062] It should be explained that the calculation of skeletal integrity is based on callus filling efficacy, comprehensively considering the impact of cortical fracture on the overall structure. Callus filling efficacy directly reflects the contribution of callus in terms of quantity, quality, and distribution to the repair of the defect area, and is the core foundation of integrity. Simultaneously, the formula incorporates the size of the fracture gap—the larger the gap, the more significant the damage to skeletal continuity, weakening overall integrity; the longer the total fracture length, the wider the extent of cortical damage, which also reduces integrity. Furthermore, the degree of cortical thickness recovery is also considered; the closer the thickness is to the normal state, the better the structural stability of the bone, and the greater the positive contribution to integrity. These factors interact to constitute a comprehensive assessment of skeletal structural integrity. For example, even with good callus filling efficacy, if there is a large fracture gap or a long fracture length, skeletal integrity will still be significantly affected.
[0063] This formula overcomes the limitations of single-indicator assessment, providing a more accurate reflection of the actual structural state after bone repair. It can distinguish between "good callus filling but severe fractures" and "moderate callus filling but minor fractures"—the former may have lower integrity due to insufficient structural continuity, while the latter may have higher integrity due to less fracture impact. Simultaneously, by integrating the recovery of cortical bone thickness, the formula can identify potential structural weaknesses such as "sufficient callus but thinner cortex," avoiding misjudgments of repair effectiveness based solely on callus volume. For example, in two femoral repair cases, one showed high callus filling efficiency but had three obvious fractures, while the other showed slightly lower callus filling efficiency but fewer fractures and better cortical thickness recovery. The formula clearly demonstrates the latter's more stable structure through its integrity value, providing a more realistic basis for clinical assessment.
[0064] Preferably, the step of analyzing the degree of fusion of adjacent bones in the repair area based on the fracture characteristics of the cortical bone includes: Based on the fracture characteristics of the cortical bone, the number of fusion feature points and the contact area with adjacent bone in the repair area are identified; The degree of fusion of adjacent bones in the repaired area is calculated based on the number of fusion feature points and the contact area of adjacent bones.
[0065] The number of fusion feature points refers to the total number of feature points representing bony fusion that are selected by the bone density threshold method (e.g., bone density ≥ 70% of normal bone density) within the fracture gap of the bone cortex in the repair area. These points can reflect the specific number of locations where adjacent bones are connected by callus. The contact area between adjacent bones refers to the surface area of the interface between adjacent bones on both sides of the fracture (e.g., bone segments on both sides of the fracture end) in three-dimensional space within the repair area.
[0066] In practice, when identifying the number of fusion feature points and the contact area between adjacent bones based on the fracture characteristics of the cortical bone, the fracture gap can be used as the analysis area in the 3D bone model. Fusion feature points are screened using a bone density threshold method (points with bone density ≥ 70% of normal bone density are considered bony fusion points), and their number is counted using a point cloud clustering algorithm. Simultaneously, 3D surface reconstruction technology (such as Poisson reconstruction) is used to extract the contact interface between adjacent bones on both sides of the fracture, and the spatial surface area of the interface is calculated as the contact area between adjacent bones. For example, in the femoral repair area, there are 90 fusion feature points with bone density ≥ 700 HU (normal bone density 1000 HU) within the fracture gap, and the measured contact area between adjacent bones is 60 mm. 2 .
[0067] Furthermore, as another embodiment of the present invention, the formula for calculating the degree of fusion of adjacent bones is as follows: ; in, Indicates the degree of fusion of adjacent bones. Indicates the number of fused feature points. Indicates the contact area between adjacent bones. This represents the standard deviation of bone mineral density in the fusion region. This represents the mean bone mineral density in the fusion region. This indicates the angle between the midlines of adjacent bones.
[0068] It should be noted that the calculation of adjacent bone fusion degree integrates four core elements: the quantity, extent, quality, and alignment of bony connections. The ratio of the number of fusion feature points to the contact area reflects the density of bony connections within a unit contact area—more fusion points within the same contact area indicate a more complete connection. The mean and standard deviation of bone density in the fusion area reflect the fusion quality; the closer the density is to normal bone and the more uniform the distribution, the better the callus mineralization and the more stable the structure. The angle between the midlines of adjacent bones considers the alignment of the bones; the smaller the angle, the more regular the arrangement of adjacent bones, and the stronger the mechanical stability after fusion. These factors are mutually balancing; for example, even with a large contact area and many fusion points, if the density is uneven or the bone alignment is large, the fusion degree will still be affected. It can distinguish between "large contact area but sparse and uneven fusion points" and "medium contact area but dense, uniform, and well-aligned fusion points"—the former will have a lower fusion degree due to insufficient quality and stability. Meanwhile, by incorporating alignment relationships, potential biomechanical risks such as "multiple fusion points but significant bone misalignment" can be identified, avoiding the neglect of overall stability while focusing solely on local connections. For example, in two cases of radius repair, one had a large contact area but significant density fluctuations and a 15° bone angle, while the other had a slightly smaller contact area but uniform density and a 5° angle. The formula clearly demonstrates through the fusion degree value that the latter actually has a better fusion effect, providing a more comprehensive basis for clinical judgment.
[0069] Furthermore, this embodiment of the invention, by constructing a repair inspection report for the tubular skeleton based on the bone integrity and the degree of fusion of adjacent bones, can systematically integrate key repair indicators into intuitive conclusions, and quantify the repair status through standardized presentation formats (such as data charts and graded assessments), avoiding subjective judgment bias. For example, after femoral fracture repair, the report can clearly mark the cortical bone areas where integrity did not meet the standards and the weak points of adjacent bone fusion, allowing doctors to quickly grasp the overall repair level and accurately locate details that require special attention, providing a concrete basis for formulating subsequent treatment plans.
[0070] As an embodiment of the present invention, a repair inspection report for the tubular skeleton is constructed based on the bone integrity and the degree of fusion of adjacent bones, including: Based on the bone integrity and the degree of fusion of adjacent bones, the bone integrity index and the percentage of fusion of adjacent bones of the tubular bone are determined; The overall repair score of the tubular bone is calculated based on the bone integrity index and the percentage of fusion between adjacent bones. Based on the comprehensive repair score, the bone healing status of the tubular bone is assessed, and a repair detection report of the tubular bone is constructed based on the bone healing status.
[0071] The bone integrity index refers to the overall integrity of the bone after repair. For example, in tibial repair, if the callus achievement rate is 85% and the cortical continuity rate is 90%, the integrity index is 87%. The adjacent bone fusion percentage refers to the ratio of the area of the adjacent bone fusion zone measured by a three-dimensional model to the total contact area of the two bones, directly reflecting the degree of fusion between the fracture ends or adjacent bones. The comprehensive repair score is a score (range 0-100 points) calculated by a weighted summation formula based on the bone integrity index (weight 0.6) and the adjacent bone fusion percentage (weight 0.4), comprehensively quantifying the bone repair effect. The bone healing status refers to the healing level divided according to the threshold of the comprehensive repair score, usually divided into "excellent" (≥85 points, complete healing), "good" (70-84 points, basic healing), "moderate" (50-69 points, delayed healing), and "poor" (<50 points, no healing), intuitively reflecting the clinical healing stage of bone repair. For example, when the comprehensive score is 74 points, the corresponding healing status is "good".
[0072] In practice, when determining the bone integrity index and the percentage of fusion between adjacent bones, the following can be selected as integrity indicators: the callus volume achievement rate (e.g., actual callus volume / expected volume × 100%) and the proportion of continuous cortical length (continuous segment length / total repair area length × 100%). These are weighted to obtain the bone integrity index (weights of 0.6 and 0.4, respectively). The fusion percentage is obtained by measuring the ratio of the area of the fusion zone between adjacent bones to the total contact area using a three-dimensional model. For example, in tibial repair, with a callus achievement rate of 85% and a continuous cortical length of 90%, the integrity index is 87%; the proportion of fusion area between adjacent bones is 72%, i.e., the fusion percentage is 72%. When calculating the overall repair score, a weighted summation formula can be used: Overall score = Bone integrity index × 0.6 + Percentage of fusion between adjacent bones × 0.4, with a score range of 0-100. For example, in femoral repair, with an integrity index of 80% and a fusion percentage of 65%, the overall score = 80 × 0.6 + 65 × 0.4 = 74 points. When assessing bone healing status, scoring thresholds can be set: ≥85 points for "Excellent" (complete healing), 70-84 points for "Good" (basic healing), 50-69 points for "Moderate" (delayed healing), and <50 points for "Poor" (non-healing). A report is then generated based on these results, including detailed scoring, 3D model annotations (e.g., low-scoring areas highlighted in red), and clinical recommendations. For example, a composite score of 74 points is considered "Good," and the report uses a heatmap to show callus distribution and recommends continued protective weight-bearing training.
[0073] like Figure 2 The diagram shown is a functional block diagram of a multimodal image fusion-based tubular bone repair and detection system according to the present invention.
[0074] The multimodal image fusion-based tubular bone repair and detection system 200 described in this invention can be installed in an electronic device. Depending on the functions implemented, the multimodal image fusion-based tubular bone repair and detection system may include an image preprocessing module 201, a feature recognition module 202, a model construction module 203, and a repair detection module 204. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.
[0075] In this embodiment of the invention, the functions of each module / unit are as follows: The image preprocessing module 201 is used to acquire multimodal image data of the tubular skeleton, wherein the multimodal image data includes CT images, MRI images and X-ray images, identify the trabecular bone structure feature points of each modality in the multimodal image data, and use the trabecular bone structure feature points to perform modal spatial registration on the multimodal image data to obtain a spatially registered image. The feature recognition module 202 is used to identify the cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour and fracture line position of the tubular bone using the spatial registration image, so as to obtain multidimensional tubular bone features. The model building module 203 is used to perform feature-level fusion on the multidimensional tubular skeleton features to obtain a fused feature map of the tubular skeleton, and to construct a three-dimensional skeleton model of the tubular skeleton using the fused feature map. The repair detection module 204 is used to identify the callus volume and cortical continuity of the corresponding repair area of the tubular bone using the three-dimensional model of the skeleton, analyze the bone integrity and adjacent bone fusion of the repair area based on the callus volume and cortical continuity, and construct a repair detection report of the tubular bone based on the bone integrity and adjacent bone fusion.
[0076] In detail, the modules in the multimodal image fusion tubular bone repair and detection system 200 described in this embodiment of the invention employ the same methods as described above during use. Figure 1 The method used is the same as the multimodal image fusion method for tubular bone repair and detection described in the article, and can produce the same technical effect, so it will not be repeated here.
[0077] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0078] Finally, it should be noted that in the above embodiments, each embodiment can be combined with each other or independent. Deleting any one of them will not affect the technical implementation of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for detecting tubular bone repair using multimodal image fusion, characterized in that, The method includes: Multimodal image data of a tubular skeleton is acquired, including CT images, MRI images and X-ray images. Trabecular bone structure feature points of each modality in the multimodal image data are identified. Modal spatial registration is performed on the multimodal image data using the trabecular bone structure feature points to obtain spatially registered images. Using the spatially registered images, the cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour, and fracture line location of the tubular skeleton are identified to obtain multidimensional tubular skeleton features; The multidimensional tubular skeleton features are fused at the feature level to obtain a fused feature map of the tubular skeleton. The fused feature map is then used to construct a three-dimensional skeleton model of the tubular skeleton. Using the three-dimensional model of the skeleton, the callus volume and cortical continuity of the corresponding repair area of the tubular skeleton are identified. Based on the callus volume and cortical continuity, the integrity of the skeleton and the degree of fusion of adjacent bones in the repair area are analyzed. Based on the integrity of the skeleton and the degree of fusion of adjacent bones, a repair inspection report of the tubular skeleton is constructed.
2. The method for detecting and repairing tubular bones using multimodal image fusion as described in claim 1, characterized in that, Based on the callus volume and the continuity of the bone cortex, the skeletal integrity and adjacent bone fusion of the repaired area are analyzed, including: Based on the callus volume, the mean volume of callus defect and the mean callus density in the repair area were analyzed; Based on the volume of the callus defect and the average callus density, the callus filling efficiency index of the repair area is calculated; The bone cortex fracture characteristics corresponding to the continuous state of the bone cortex are queried, and the bone integrity of the repair area is calculated by combining the bone cortex fracture characteristics and the callus filling efficiency index. Based on the fracture characteristics of the cortical bone, the degree of fusion of adjacent bones in the repair area was analyzed.
3. The method for detecting and repairing tubular bones using multimodal image fusion as described in claim 2, characterized in that, The analysis of adjacent bone fusion in the repair area based on the fracture characteristics of the cortical bone includes: Based on the fracture characteristics of the cortical bone, the number of fusion feature points and the contact area with adjacent bone in the repair area are identified; The degree of fusion of adjacent bones in the repaired area is calculated based on the number of fusion feature points and the contact area of adjacent bones.
4. The method for detecting and repairing tubular bones using multimodal image fusion as described in claim 1, characterized in that, Using the trabecular bone structure feature points, modal spatial registration is performed on the multimodal image data to obtain a spatially registered image, including: The local structural complexity of the trabecular bone structure feature points is analyzed, and based on the local structural complexity, structural points are screened to obtain baseline structural points. The reference structural points are spatially mapped to obtain the mapped structural points; The spatial coordinates of the mapped structural points are transformed to obtain the coordinates of the structural points; Based on the coordinates of the structural points, modal spatial registration is performed on the multimodal image data to obtain a spatially registered image.
5. The method for detecting and repairing tubular bones using multimodal image fusion as described in claim 1, characterized in that, Using the spatially registered images, the cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour, and fracture line location of the tubular skeleton are identified to obtain multidimensional tubular skeleton features, including: Query the CT image region, MRI image region, and X-ray image region in the spatially registered image; Using the CT image region, the cortical bone thickness and trabecular bone density of the tubular bone are identified to obtain CT image recognition features; Using the MRI image region, the morphology of the medullary cavity and the soft tissue edema area of the tubular bone are identified, and MRI image recognition features are obtained; The overall outline of the tubular skeleton and the location of the fracture line are identified using the X-ray image region to obtain X-ray image recognition features; Based on the CT image recognition features, the MRI image recognition features, and the X-ray image recognition features, the multidimensional tubular bone features of the tubular skeleton are determined.
6. The method for detecting tubular bone repair using multimodal image fusion as described in claim 1, characterized in that, Using the three-dimensional model of the skeleton, the callus volume and cortical continuity of the corresponding repair area of the tubular bone are identified, including: Using the three-dimensional model of the skeleton, the boundary surface between the repair area and the normal area of the tubular skeleton is identified; The callus region within the boundary surface is segmented using voxelization to obtain three-dimensional volume data of the callus. The three-dimensional volume data of the callus described in the gene are used to calculate the callus volume of the corresponding repair area of the tubular bone. Based on the three-dimensional model of the skeleton, the surface curvature of the cortical layer corresponding to the tubular skeleton is identified; Based on the surface curvature, the fracture characteristics of the cortical bone in the repaired area are analyzed to determine the continuity of the cortical bone in the repaired area according to the fracture characteristics.
7. The method for detecting and repairing tubular bones using multimodal image fusion as described in claim 1, characterized in that, The multidimensional tubular skeleton features are fused at the feature level to obtain a fused feature map of the tubular skeleton, including: The multidimensional tubular skeleton features are normalized in spatial coordinates to obtain a standard parameter set, and the standard parameter set is then subjected to principal component dimensionality reduction to obtain a dimensionality-reduced feature vector. Adaptive weight allocation is performed on the dimensionality-reduced feature vectors to obtain the weighting coefficient matrix of each feature vector in the dimensionality-reduced feature vectors; The weighted coefficient matrix and the standard parameter set are convolved and fused to obtain an initial fused feature map; The initial fusion feature map is reconstructed using a three-dimensional surface to obtain the final fusion feature map of the tubular skeleton.
8. The method for detecting and repairing tubular bones using multimodal image fusion as described in claim 1, characterized in that, Using the fused feature map, a 3D skeletal model of the tubular skeleton is constructed, including: Extract the isosurface of the fused feature map to obtain the initial point cloud data of the bone tissue corresponding to the tubular skeleton; The initial point cloud data is triangulated to obtain a preliminary three-dimensional mesh model; The preliminary three-dimensional mesh model is subjected to normal vector consistency optimization to obtain a smooth surface model; The smooth surface model is topologically calibrated to obtain a skeletal 3D model.
9. The method for detecting and repairing tubular bones using multimodal image fusion as described in claim 1, characterized in that, Based on the bone integrity and the degree of fusion of adjacent bones, the repair inspection report for the tubular bone includes: Based on the bone integrity and the degree of fusion of adjacent bones, the bone integrity index and the percentage of fusion of adjacent bones of the tubular bone are determined; The overall repair score of the tubular bone is calculated based on the bone integrity index and the percentage of fusion between adjacent bones. Based on the comprehensive repair score, the bone healing status of the tubular bone is assessed, and a repair detection report of the tubular bone is constructed based on the bone healing status.
10. A multimodal image fusion-based tubular bone repair and detection system, characterized in that, The system includes: The image preprocessing module is used to acquire multimodal image data of the tubular skeleton, wherein the multimodal image data includes CT images, MRI images and X-ray images, identify the trabecular bone structure feature points of each modality in the multimodal image data, and use the trabecular bone structure feature points to perform modal spatial registration on the multimodal image data to obtain spatially registered images. The feature recognition module is used to identify the cortical bone thickness, trabecular bone density, medullary cavity morphology, soft tissue edema area, overall bone contour and fracture line position of the tubular bone using the spatially registered image, so as to obtain multidimensional tubular bone features. The model building module is used to perform feature-level fusion on the multidimensional tubular skeleton features to obtain a fused feature map of the tubular skeleton, and to construct a three-dimensional skeleton model of the tubular skeleton using the fused feature map. The repair detection module is used to identify the callus volume and cortical continuity of the corresponding repair area of the tubular bone using the three-dimensional model of the skeleton. Based on the callus volume and cortical continuity, it analyzes the integrity of the skeleton and the fusion of adjacent bones in the repair area. Based on the integrity of the skeleton and the fusion of adjacent bones, it constructs a repair detection report for the tubular bone.