Fracture intelligent reduction navigation evaluation analysis method and system based on multi-modal image
By constructing a bone fragment-fiber bundle representation and embedding multimodal constraints on multi-order units, combined with fracture end pairing diagrams and regular constraints, the instability problem caused by modal differences in fracture reduction navigation was solved, achieving accurate fracture reduction navigation and assessment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-27
- Publication Date
- 2026-03-17
AI Technical Summary
Existing fracture reduction navigation technology suffers from unstable registration results due to differences in spatial resolution, grayscale dynamic range, and noise characteristics between preoperative computed tomography data, intraoperative fluoroscopic images, and ultrasound images. This makes it difficult to accurately reflect the alignment relationship of the fracture ends. Furthermore, traditional methods lack the ability to depict the local morphological features of the fracture fissure triangular facets, which can easily lead to interpenetration or misalignment between bone fragments.
By constructing a bone fragment-fiber bundle representation, introducing connection and parallel movement rules, and establishing convolution kernels on point, edge, and surface multi-level units to embed multimodal constraint weights and gating parameters, combined with fracture end pairing diagrams, non-penetrating barrier constraints, and normal continuity regularization constraints, a repositioning solution conforming to anatomical relationships is generated, and navigation prompts and real-time evaluation indicators are output.
It improves the stability and accuracy of cross-modal reduction calculation, generates reduction solutions that conform to anatomical relationships, provides reliable intelligent assistance for fracture reduction surgery, and enhances the rationality of reduction path and the reliability of surgical navigation.
Smart Images
Figure CN121685877A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image processing and surgical navigation technology, and in particular to a method and system for intelligent fracture reduction navigation assessment and analysis based on multimodal imaging. Background Technology
[0002] Precise fracture reduction is a crucial step in orthopedic surgery, directly impacting the speed of fracture healing and postoperative functional recovery. Current fracture reduction navigation typically relies on three-dimensional reconstruction from preoperative computed tomography images and registration with intraoperative fluoroscopic images to assist surgeons in determining bone fragment positions and performing the reduction procedure. However, existing technologies have the following limitations:
[0003] Preoperative computed tomography data, intraoperative fluoroscopic images, and ultrasound images differ significantly in spatial resolution, grayscale dynamic range, and noise characteristics. Directly using conventional registration methods can easily lead to inconsistencies in intensity and artifact interference, resulting in unstable registration results.
[0004] Traditional methods are mostly based on the geometric shape of the whole bone fragment to calculate the reduction path, but lack detailed depiction of the local morphological features of the fracture fissure triangular facet, making it difficult to accurately reflect the alignment relationship of the fracture ends.
[0005] Existing repositioning path planning methods mostly rely on single geometric constraints or point-edge relationship modeling, failing to fully consider the topological continuity and physical parameter consistency of the fracture surface in the repositioning calculation, which can easily lead to interpenetration or misalignment between bone pieces.
[0006] Therefore, how to provide a method and system for intelligent fracture reduction navigation assessment and analysis based on multimodal imaging is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0007] One objective of this invention is to propose a method and system for intelligent fracture reduction navigation assessment and analysis based on multimodal imaging. This invention constructs a bone fragment-fiber bundle representation and introduces connection and parallel movement rules within it, achieving joint modeling of the overall geometry of the bone fragment and the local state of the fissure. Convolution kernels are established on point, edge, and surface multi-level units, and multimodal constraint weights and gating parameters based on computed tomography, fluoroscopic images, and ultrasound images are embedded, enabling reduction calculations to take into account multimodal information and improve the stability and accuracy of cross-modal registration. Simultaneously, in the reduction solution, the fracture end pairing diagram, non-penetration barrier constraints, and normal continuity regularization constraints are combined to generate a reduction solution that conforms to anatomical relationships, and navigation prompts and real-time evaluation indicators are output, providing reliable intelligent assistance for fracture reduction surgery.
[0008] The fracture intelligent reduction navigation evaluation and analysis method based on multimodal imaging according to embodiments of the present invention includes the following steps:
[0009] Acquire preoperative computed tomography scan data, intraoperative fluoroscopic sequence and intraoperative ultrasound images, perform intensity normalization and artifact suppression on the multimodal images, and reconstruct the bone fragment surface mesh and fracture fissure triangular facet of the target bone.
[0010] The bottom space is formed by the mesh on the surface of the bone plate, and the local state vectors of the fissures attached to each discrete point of the bottom space are used to form fibers. The fiber space is composed of all fiber end states, and a bone plate-fissure fiber bundle representation is established.
[0011] Define connection and parallel movement rules on the bone-fiber bundle;
[0012] Multi-level convolutional kernels are constructed on point units, edge units, and surface units in the bottom space and fiber space, respectively, and multimodal constraint weights and gating parameters are embedded in the convolution update process;
[0013] After completing one or more steps of multi-level convolutional propagation, a fracture end pairing map is constructed in the candidate region of the bottom space based on the normal complementarity of the fiber ends, roughness compatibility and compressibility threshold, the fracture end pairing probability is calculated, and the pairing path and bone fragment repositioning operation sequence are determined.
[0014] During the solution process of the resetting operation sequence, non-penetrating barrier constraints and normal continuity regularization constraints are applied to the relative motion of the bone fragments to obtain the resetting solution;
[0015] Intraoperative navigation prompts are generated based on the repositioning solution, and real-time evaluation indicators are output.
[0016] Furthermore, the bone fragment surface mesh reconstruction of the target bone includes:
[0017] The standardized and artifact-suppressed volumetric data were input into a 3D segmentation network to extract the cortical and cancellous bone regions.
[0018] Triangular meshes of bone fragments are generated based on the segmentation results, and the manifold properties of the mesh are ensured through boundary smoothing and topology repair.
[0019] The reconstruction of the fracture gap triangular facet includes:
[0020] Detect grayscale gradients and areas of abnormal curvature along the mesh on the bone fragment surface to locate potential crack lines;
[0021] Triangulation is generated around the fracture line to form fracture patches;
[0022] The crack normal, roughness, and local curvature are calculated during the crack patch reconstruction process.
[0023] Furthermore, the establishment of the bone plate-fiber bundle representation includes:
[0024] The vertices, edges, and faces of the bone fragment surface mesh are used as discrete units of the bottom space;
[0025] A local state vector of the crack is attached to each discrete unit. The local state vector of the crack includes: the normal direction of the crack patch, the surface roughness value, the local radius of curvature, the cortical bone thickness value, the measured bone density value, the friction coefficient, and the compressive modulus.
[0026] The local state vector of the fissure is indexed and bound to the corresponding discrete unit of the bottom space to form a one-to-one correspondence between the bottom space and the fiber, thus forming a bone plate-fissure fiber bundle.
[0027] Furthermore, defining the connection and parallel movement rules on the bone-fiber plexus includes:
[0028] Establish adjacent element relationships between vertices, edges, and faces of the mesh on the bone fragment surface;
[0029] A connection operator is set between each adjacent unit. The connection operator is represented by an affine transformation matrix and is used to describe the coordinate transformation relationship of the fiber end state vector between adjacent units.
[0030] During parallel movement, the normal direction, radius of curvature, and roughness values of the local state vector of the crack are linearly transformed according to the communication operator, and the cortical bone thickness, bone density, friction coefficient, and compressive modulus are numerically transferred to obtain the updated state vector on the target element.
[0031] Furthermore, the convolutional update process includes:
[0032] Convolutional kernels are constructed in the point unit layer to process the local state vectors of the vertices on the bone slice surface and the cracks to which they are attached;
[0033] Convolutional kernels are constructed in the edge unit layer to process edges composed of adjacent vertices and their corresponding crack local state vectors;
[0034] Convolutional kernels are constructed in the face unit layer to process the crack region defined by triangular facets and its corresponding crack local state vector;
[0035] The multi-level convolutional kernel is multiplied with multimodal constraint weights during the update process. The multimodal constraint weights are derived from the intensity gradient of the computed tomography data, the residual value of the digital reconstructed ray projection, and the echo confidence of the ultrasound image.
[0036] A gating parameter is introduced when outputting the convolution result. The gating parameter is used to adjust the weights of different modal constraints. Specifically, the convolution results of the three modalities are proportionally allocated through adjustable weight coefficients.
[0037] Furthermore, the construction of the end-pairing diagram includes:
[0038] In the bottom space grid, triangular facets located at the fracture boundary are selected as candidate elements for incoherence;
[0039] The fiber end state vectors of the candidate units are compared with parameters, wherein the normal direction is calculated by the vector angle, the roughness is calculated by the surface roughness difference, and the compressibility is calculated by the compressive modulus difference.
[0040] When the normal direction, roughness, and compressibility parameters meet the preset threshold conditions, pairing edges are established between the corresponding candidate units to obtain a broken pairing diagram with candidate units as nodes and pairing edges as connections.
[0041] On the pairing diagram of the broken ends, the pairing probability is calculated based on the number of paired edges and the degree of parameter compatibility, and the pairing path is determined based on the magnitude of the pairing probability;
[0042] A bone fragment repositioning operation sequence is generated based on the alignment path, and the operation sequence includes translation, rotation, and compression.
[0043] Furthermore, the application of non-penetrating barrier constraints and normal continuity regularization constraints to the relative motion of the bone fragments includes:
[0044] Set the nearest point distance constraint between the meshes on the bone slice surface. When the nearest point distance between any two bone slice surfaces is less than zero, apply a penalty to the corresponding operation sequence through a barrier function.
[0045] Normal continuity constraints are set between bone fragment fissure patches. When the angle between the normal directions of adjacent fissure patches is greater than a preset threshold, the cost of the corresponding operation sequence is increased through regularization terms.
[0046] During the solution process of the reset operation sequence, the non-penetrating barrier constraint and the normal continuity regularity constraint are introduced simultaneously to obtain a reset solution that satisfies the constraint conditions.
[0047] Furthermore, the step of generating intraoperative navigation prompts based on the reset solution and outputting real-time evaluation indicators includes:
[0048] By combining the translation, rotation, and clamping amounts in the repositioning solution with the spatial parameters of the intraoperative imaging device, navigation prompts for the fluoroscopic imaging angle and the ultrasound probe posture are generated.
[0049] The navigation prompts are output to the intraoperative display terminal so that the operator can adjust the position of the imaging device and the repositioning instrument.
[0050] During the solution process, real-time evaluation metrics are calculated, including the mating gap, residual rotation angle, crack closure consistency score, and multimodal observation residual image.
[0051] The real-time evaluation metrics and navigation prompts are output together.
[0052] The fracture intelligent reduction navigation assessment and analysis system based on multimodal imaging according to an embodiment of the present invention includes the following modules:
[0053] The image acquisition and preprocessing module is used to acquire preoperative computed tomography scan data, intraoperative fluoroscopic sequences and intraoperative ultrasound images, and to perform intensity normalization and artifact suppression on the multimodal images.
[0054] The bone fragment and fracture reconstruction module is used to reconstruct the bone fragment surface mesh and fracture fracture triangular facets of the target bone based on preprocessed multimodal images.
[0055] The bone-fracture fiber bundle construction module is used to construct a bottom space with a mesh on the surface of the bone slice, and attach the local state vector of the fracture to each discrete unit of the bottom space to form fibers, thereby establishing the bone-fracture fiber bundle representation.
[0056] The communication and translation module is used to define communication operators on the bone-fiber bundle and perform parallel translation of fiber end state vectors between adjacent cells in the bottom space.
[0057] The multi-level convolution computation module is used to construct multi-level convolution kernels on point units, edge units, and surface units in the bottom space and fiber space, respectively, and embed multi-modal constraint weights and gating parameters during the convolution update process;
[0058] The fracture end pairing and repositioning solution module is used to establish a fracture end pairing map based on the fiber end state vector in the candidate region of the bottom space after completing one or more steps of multi-level convolution propagation, calculate the pairing probability, and determine the pairing path and bone fragment repositioning operation sequence.
[0059] The constraint solving module is used to apply non-penetrating barrier constraints and normal continuity regularization constraints to the relative motion of the bone pieces during the solution process of the reset operation sequence, so as to obtain the reset solution;
[0060] The navigation prompt and evaluation module is used to generate intraoperative navigation prompt information based on the reset solution and output real-time evaluation indicators including the alignment gap, residual rotation angle, gap closure consistency score and multimodal observation residual image.
[0061] The beneficial effects of this invention are:
[0062] This invention constructs a bone fragment-fiber bundle representation, combining the overall geometric structure of the bone fragment with the local parameters of the fissure, and introduces connection and parallel movement rules in the fiber bundle to maintain parameter consistency of the fissure state across different bottom space units, thereby achieving more refined modeling of the fracture area.
[0063] This invention establishes multi-level convolution kernels at three levels: point units, edge units, and surface units. It also embeds multimodal constraint weights and gating parameters generated from computed tomography, fluoroscopic images, and ultrasound images into the convolution calculation, enabling the reset solution to utilize multimodal information simultaneously and improving the stability and accuracy of cross-modal reset calculation.
[0064] This invention combines the fracture end pairing diagram, non-penetrating barrier constraint, and normal continuity regularization constraint during the repositioning process to output a repositioning solution that conforms to the anatomical relationship, and generates intraoperative navigation prompts and real-time evaluation indicators, thereby providing quantitative reference for surgical operations and enhancing the rationality of the repositioning path and the reliability of surgical navigation. Attached Figure Description
[0065] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0066] Figure 1 This is a flowchart of the intelligent fracture reduction navigation evaluation and analysis method based on multimodal imaging proposed in this invention;
[0067] Figure 2 This is a schematic diagram of the bone fragment-fiber tuft structure of the fracture intelligent reduction navigation assessment and analysis method based on multimodal imaging proposed in this invention;
[0068] Figure 3 This is a schematic diagram of fracture end pairing for the intelligent fracture reduction navigation evaluation and analysis method based on multimodal imaging proposed in this invention. Detailed Implementation
[0069] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0070] refer to Figure 1-3 A method for intelligent fracture reduction navigation assessment and analysis based on multimodal imaging includes the following steps:
[0071] Acquire preoperative computed tomography scan data, intraoperative fluoroscopic sequence and intraoperative ultrasound images, perform intensity normalization and artifact suppression on the multimodal images, and reconstruct the bone fragment surface mesh and fracture fissure triangular facet of the target bone.
[0072] The bottom space is formed by the mesh on the surface of the bone plate, and the local state vectors of the fissures attached to each discrete point of the bottom space are formed by the fiber. The fiber space is composed of all fiber end states, and a bone plate-fissure fiber bundle representation is established. The local state vectors of the fissures include at least one or more of the following: normal, roughness, microcurvature, cortical thickness and compressibility.
[0073] Connectivity and parallel movement rules are defined on the bone-fiber bundle to ensure that the local state of the fissure at the fiber end remains physically consistent as it propagates along adjacent units in the bottom space.
[0074] Multi-level convolutional kernels are constructed on point units, edge units, and surface units in the bottom space and fiber space, respectively. Multimodal constraint weights and gating parameters are embedded in the convolution update process. The multimodal constraint weights are generated based on the intensity gradient of computed tomography scan, the residual of differentiable digital reconstruction ray projection, and the confidence of ultrasound echo. The gating parameters are used to perform weighted selection and suppress inconsistent information between different modes.
[0075] After completing one or more steps of multi-level convolutional propagation, a fracture end pairing map is constructed in the candidate region of the bottom space based on the normal complementarity of the fiber ends, roughness compatibility and compressibility threshold, the fracture end pairing probability is calculated, and the pairing path and bone fragment repositioning operation sequence are determined.
[0076] During the solution process of the reset operation sequence, non-penetrating barrier constraints and normal continuity regularization constraints are applied to the relative motion of the bone fragments to obtain a reset solution that includes translation, rotation and clamping.
[0077] Based on the repositioning solution, intraoperative navigation prompts are generated, including recommended fluoroscopic angles, ultrasound probe postures, and instrument operation windows, and real-time evaluation indicators are output. The evaluation indicators include at least one or more of the following: engagement gap, residual torsion, gap closure consistency score, and multimodal observation residual heatmap.
[0078] In this embodiment, the reconstruction of the bone fragment surface mesh of the target bone includes:
[0079] The volume data after intensity normalization and artifact suppression is input into a three-dimensional segmentation network. The input of the three-dimensional segmentation network is a computed tomography image voxel, and the output is a label volume of the cortical bone region and the cancellous bone region.
[0080] Based on the segmentation results, an isosurface extraction algorithm is used to generate triangular meshes of bone fragments. The vertex coordinates of the triangular meshes correspond to voxel coordinates, and the edges and faces are automatically formed by the isosurface extraction rules.
[0081] The triangular mesh is smoothed at the boundary to reduce jagged boundaries caused by segmentation and discretization; and overhanging surfaces, duplicate edges and non-manifold points are detected and removed by a topology repair algorithm to ensure that the mesh meets the topological requirements of a continuous manifold.
[0082] The reconstruction of the fracture gap triangular facet includes:
[0083] The gray-level gradient field and curvature distribution are calculated on the mesh of the bone fragment surface. The gray-level gradient field is calculated as follows: at the voxel position corresponding to each vertex of the mesh on the bone fragment surface, the partial derivatives of the gray values in the X, Y and Z directions are calculated using a three-dimensional difference operator, and the square root of the sum of squares is taken as the gray-level gradient magnitude of that vertex.
[0084] The curvature distribution is obtained by calculating the average curvature at each vertex of the mesh on the bone slice surface based on the difference in the normal vectors of adjacent triangular faces. Specifically, it is obtained by fitting a local quadratic surface with least squares and solving for the principal curvature radius.
[0085] Identify gray-level abrupt change regions and curvature abnormal regions. If the gray-level gradient magnitude in a local neighborhood exceeds 1.5 times the global average gray-level gradient, the region is marked as a gray-level abrupt change region.
[0086] If the average radius of curvature in a local neighborhood is less than a preset threshold (e.g., less than 2.0 mm), the area is marked as a curvature anomalous area.
[0087] Regions with abrupt changes in grayscale and regions with abnormal curvature are considered as potential crack line candidates.
[0088] Closed fracture lines are extracted within the candidate fracture line region using a region growing and boundary tracking method.
[0089] Local triangulation is performed around the fracture line to form fracture triangular facets, thus subdividing the fracture line into multiple continuous triangular facets.
[0090] During the reconstruction of the fracture triangular facet, the normal vector is calculated face by face, the surface roughness index is extracted, and the local radius of curvature is calculated as the initial feature parameter of the fracture local state vector.
[0091] In this embodiment, the establishment of the bone plate-fiber bundle representation includes:
[0092] The vertices, edges, and faces of the bone fragment surface mesh are used as discrete units of the bottom space. Each unit is assigned a unique index number to correspond with the fiber end state vector.
[0093] A local state vector of the crack is attached to each discrete element, and the local state vector of the crack includes:
[0094] Normal direction: Calculated from the vertex coordinates of the triangular facet to obtain the normal vector of the triangular facet;
[0095] Surface roughness value: obtained by calculating the standard deviation of the vertex height difference within the local neighborhood of the triangular facet;
[0096] Local radius of curvature: Calculated based on the principal radius of curvature of the quadratic surface fitted by least squares.
[0097] Cortical bone thickness value: obtained by grayscale conversion of computed tomography images and combined with known grayscale-thickness calibration curves;
[0098] Bone mineral density values were obtained by comparing image grayscale with a calibrated bone mineral density table.
[0099] Friction coefficient: Obtained by referring to the table of correspondence between roughness and friction coefficient;
[0100] Compression modulus: Calculated by converting bone mineral density into an empirical formula;
[0101] The local state vector of the fracture is bound to the corresponding bottom space cell index and stored in the fiber bundle data structure in the form of key-value pairs to form a bone plate-fracture fiber bundle representation.
[0102] In this embodiment, defining the connection and parallel movement rules on the bone fragment-fiber bundle includes:
[0103] A connection operator is defined between each pair of adjacent units. The connection operator is represented by an affine transformation matrix, and the matrix elements are obtained in the following way:
[0104] For the normal direction, the rotation matrix is calculated using the difference in the normal vectors of adjacent triangular facets;
[0105] For the radius of curvature, a scaling factor is used for transformation;
[0106] For roughness values, cortical bone thickness values, bone mineral density values, friction coefficient and compressive modulus, a weighted average method is used for transmission;
[0107] During parallel movement, the local state vector of the crack is mapped from the starting element to the target element through the affine transformation matrix, thereby obtaining the updated state parameters;
[0108] The communication and parallel movement rules remain consistent throughout the entire bone-fiber bundle, and are invoked through a unified adjacency relationship and operator table, enabling the state vector to propagate in an orderly manner between different bottom space units.
[0109] In this embodiment, the convolution update process includes:
[0110] In the point unit layer, the convolution kernel takes the crack local state vector of the target vertex and its adjacent vertices as input to calculate the updated features of the target vertex;
[0111] In the edge unit layer, the convolution kernel takes the crack local state vector corresponding to the target edge and its adjacent edges as input to calculate the updated features of the target edge;
[0112] In the face unit layer, the convolution kernel takes the crack local state vectors corresponding to the target triangular facet and its adjacent facets as input to calculate the updated features of the target triangular facet.
[0113] The multi-level convolutional kernel is multiplied by multimodal constraint weights during the update process. These multimodal constraint weights are derived from the intensity gradients of computed tomography (CT) scan data. At each vertex of the bone slice surface grid, the grayscale values of the data in three directions are differentiated to obtain the grayscale change amplitude at that location. The grayscale change amplitudes at all locations are normalized and used as the CT scan constraint weights. The residual values of the digitally reconstructed ray projection are obtained by using the intraoperative fluoroscopic image and the digitally reconstructed ray projection generated based on the CT scan data as inputs. The difference values of pixels at the same projection angle are calculated one by one, and the average value is taken within a local range as the residual index. After inverse normalization, the residual index is used to obtain the fluoroscopic image constraint weights. The echo confidence of the ultrasound image is also obtained by selecting a bone surface region in the ultrasound image and comparing the signal intensity of the pixel grayscale with the neighborhood noise level to obtain the signal-to-noise ratio (SNR). If the SNR is high, a larger confidence score is assigned; if the SNR is low, a smaller confidence score is assigned. The confidence score is normalized and used as the ultrasound image constraint weights.
[0114] A gating parameter is introduced when outputting the convolution update result. The gating parameter consists of learnable weight coefficients and is used to proportionally distribute the convolution results corresponding to the three modalities. Specifically, the gating parameter coefficients are multiplied in the convolution outputs of point units, edge units, and surface units, and the weighted results are summed to obtain the fused update features.
[0115] The updated features are passed to the next convolutional layer until the preset number of convolutional layers is completed.
[0116] In this embodiment, constructing the broken end pairing diagram includes:
[0117] In the bottom space grid, triangular facets located at the fracture boundary are selected as candidate elements for incoherence;
[0118] The crack local state vector of each candidate element is compared. The comparison includes: whether the normal directions are nearly opposite directions, whether the difference in surface roughness values is below a preset threshold, and whether the difference in compressive modulus is within the allowable range. If all the above conditions are met, the candidate elements are determined to be paired.
[0119] Pairing edges are established between candidate units. With candidate units as nodes and pairing edges as lines, a broken-end pairing diagram is formed. In this pairing diagram, each pairing edge is associated with a pairing score, which is determined by the difference in normal angle, roughness, and compressive modulus.
[0120] The pairing probability of a candidate unit is obtained by summing the pairing scores of each candidate unit in the pairing graph and normalizing the sum.
[0121] Based on the pairing probability, the optimal pairing edge set is selected, and the fracture end alignment path and bone fragment repositioning operation sequence are determined accordingly.
[0122] In this embodiment, applying non-penetrating barrier constraints and normal continuity regularization constraints to the relative motion of bone fragments includes:
[0123] Establish non-intersecting barrier constraints. Real-time detection of the spatial distance between closest points on the bone fragment surface meshes is performed. When the distance between the closest points on any two bone fragment surfaces is less than zero, geometric intersecting is considered to have occurred. During the solution process, this situation is recorded as an infeasible state, and the cost of the corresponding solution is increased in subsequent iterations, thereby eliminating motion paths involving intersecting.
[0124] Establish a normal continuity regularization constraint. Compare the included angle of the normal directions between adjacent triangular facets at the fracture boundary. When the included angle is greater than a preset threshold, the normal direction is considered discontinuous. During the reset solution process, add a cost to the operation sequence corresponding to this discontinuity, and gradually compress the angle difference during iterative updates to make the normal direction of the fracture boundary tend to be continuous.
[0125] During the iterative solution of the reset operation sequence, non-penetration barrier constraints and normal continuity regularization constraints are introduced simultaneously. Each translation, rotation and clamping operation is detected and adjusted until a reset solution that satisfies the constraints is generated.
[0126] In this embodiment, the step of generating intraoperative navigation prompts based on the reset solution and outputting real-time evaluation indicators includes:
[0127] The translation, rotation and clamping amounts recorded in the repositioning solution are combined with the spatial parameters of the intraoperative imaging device to calculate the required fluoroscopic imaging angle and ultrasound probe posture, and then displayed in numerical and graphical form in the navigation interface as intraoperative navigation prompts.
[0128] The repositioning quality is quantitatively evaluated based on the alignment gap between bone fragments, the residual rotation angle, and the fissure closure consistency score. Specifically, the alignment gap is calculated from the average distance between the triangular facets at the fissure boundary; the residual rotation angle is calculated from the difference between the local coordinate system and the target coordinate system of the bone fragments; and the fissure closure consistency score is determined by the average pairing probability of corresponding points at the fissure boundary.
[0129] Based on the differences between intraoperative images and predicted images, multimodal observation residual images are generated to demonstrate the degree of consistency between the repositioning results and the actual observation data;
[0130] The navigation prompts and real-time evaluation indicators are displayed on the intraoperative terminal for the operator to refer to and adjust the operation in real time.
[0131] The intelligent fracture reduction navigation assessment and analysis system based on multimodal imaging includes the following modules:
[0132] The image acquisition and preprocessing module is used to acquire preoperative computed tomography scan data, intraoperative fluoroscopic sequences and intraoperative ultrasound images, and to perform intensity normalization and artifact suppression on the multimodal images.
[0133] The bone fragment and fracture reconstruction module is used to reconstruct the bone fragment surface mesh and fracture fracture triangular facets of the target bone based on preprocessed multimodal images.
[0134] The bone-fracture fiber bundle construction module is used to construct a bottom space with a mesh on the surface of the bone slice, and attach the local state vector of the fracture to each discrete unit of the bottom space to form fibers, thereby establishing the bone-fracture fiber bundle representation.
[0135] The communication and translation module is used to define communication operators on the bone-fiber bundle and perform parallel translation of fiber end state vectors between adjacent cells in the bottom space.
[0136] The multi-level convolution computation module is used to construct multi-level convolution kernels on point units, edge units, and surface units in the bottom space and fiber space, respectively, and embed multi-modal constraint weights and gating parameters during the convolution update process;
[0137] The fracture end pairing and repositioning solution module is used to establish a fracture end pairing map based on the fiber end state vector in the candidate region of the bottom space after completing one or more steps of multi-level convolution propagation, calculate the pairing probability, and determine the pairing path and bone fragment repositioning operation sequence.
[0138] The constraint solving module is used to apply non-penetrating barrier constraints and normal continuity regularization constraints to the relative motion of the bone pieces during the solution process of the reset operation sequence, so as to obtain the reset solution;
[0139] The navigation prompt and evaluation module is used to generate intraoperative navigation prompt information based on the reset solution and output real-time evaluation indicators including the alignment gap, residual rotation angle, gap closure consistency score and multimodal observation residual image.
[0140] Example 1:
[0141] To verify the feasibility of this invention in practice, it was applied to a preoperative-intraoperative combined navigation scenario for 10 cases of distal radius fractures (AO23-A2 / A3 / C1 / C2 type) in a hospital. The implementation environment and process are as follows:
[0142] Preoperatively, computed tomography (CT) scans with a slice thickness of 0.6 mm were performed, with an acquisition voltage of 120 kVp and a current of 200 mAs. The output pixel pitch was 0.5–0.8 mm. After metal artifact suppression and intensity normalization, volumetric data were obtained. Intraoperative fluoroscopic images were acquired using a C-arm, with anteroposterior and lateral angular sequences of ±10°, and compared with digitally reconstructed X-ray projections generated based on CT data. Intraoperative ultrasound images were acquired using a 7.5–12 MHz linear array probe, with the cortical bone echo region as the region of interest, and the signal-to-noise ratio was calculated to select effective frames.
[0143] After outputting cortical and cancellous bone tags via a 3D segmentation network, an isosurface extraction algorithm is used to generate a mesh for the bone slice surface. Jagged boundaries and overhangs are eliminated through boundary smoothing and topology repair. During crack extraction, the 3D grayscale gradient magnitude is calculated and normalized at the mesh vertices, and the principal radius of curvature is obtained by least-squares quadratic surface fitting. If a local grayscale gradient exceeds 1.5 times the global average or the radius of curvature is less than 2.0 mm, it is identified as a candidate crack region. Region growing and boundary tracking are performed on the candidate regions to extract closed crack lines, and then local triangulation is used to form crack triangular facets.
[0144] Points, edges, and faces of the bone fragment surface mesh serve as the bottom space elements. The local state vector of the fissure includes parameters such as normal, roughness, local radius of curvature, cortical bone thickness, bone density, friction coefficient, and compressive modulus. The friction coefficient is obtained from the roughness-friction coefficient correspondence table, and the compressive modulus is calculated from bone density. A unique index number is used to establish the binding relationship between the state vector and the bottom space elements. An affine transformation matrix is defined as a connection operator between adjacent elements, where the normal is transmitted through a rotation matrix, the curvature is transmitted using a scaling factor, and the remaining parameters are transmitted using a weighted average. During parallel movement, the state vector is mapped from the starting element to the target element, achieving consistent propagation.
[0145] Convolutional kernels are constructed at three levels: point, edge, and surface. The input is the state vector of the target unit and its neighboring units, and the output is the updated feature. Three types of modality constraint weights are introduced into the convolutional update:
[0146] Computed tomography constraint weights: At the vertex position, the gray values of the volume data in the three directions are differentiated, the gray value change amplitude is calculated and normalized;
[0147] Fluoroscopy image constraint weight: Calculate the pixel-by-pixel difference between intraoperative fluoroscopy image and DRR under the same projection angle, and obtain it by local averaging and inverse normalization;
[0148] Ultrasound image constraint weights: The mean gray value of pixels in the bone surface region is compared with the variance of the neighborhood to obtain the signal-to-noise ratio and normalize it into a confidence level.
[0149] The constraint weights are multiplied one by one with the corresponding input features; in the output stage of the convolution result, a gating parameter is introduced to weight the three modal results to obtain the fused updated features.
[0150] Candidate elements are selected from the triangular facets of the fracture boundary. Elements with approximately opposite normal directions, roughness differences not exceeding 10%, and compressive modulus differences not exceeding 15% are considered to have a pairing relationship. A fracture end pairing graph is constructed using candidate elements as nodes and pairing relationships as edges, and a pairing score is assigned to each edge. The pairing scores are normalized to obtain the pairing probability of each candidate element. The optimal set of paired edges is selected based on the probability results, generating a reset operation sequence. Non-penetrating barrier constraints and normal continuity regularization constraints are introduced into the reset solution. Translation, rotation, and compaction operations are iteratively solved until a reset solution satisfying the constraints is generated.
[0151] After obtaining the reset solution, the recommended C-arm angle and ultrasound probe orientation are calculated based on the recorded translation, rotation, and clamping amounts, and displayed on the intraoperative navigation interface. Evaluation metrics include the apposition gap, residual rotation angle, gap closure consistency score, and multimodal observation residual heatmap.
[0152] The computing workstation is configured with a 16-core CPU, 64GB of RAM, and a 24GB GPU. The DRR stride is set to 0.8mm, and the US signal-to-noise ratio threshold is set to 10dB. The convolutional network is a three-layer structure, with the number of channels progressively increasing from 32 to 64 to 64.
[0153] Table 1. Test results of ten cases of distal radius fractures.
[0154] Case AO type CT slice thickness (mm) Perspective angle group Ultrasonic frequency (MHz) Mating gap (mm) Residual torsion (°) Closedness score Online latency (s) #01 23-A2 0.6 Front / Side ±10° 10 0.9 3.2 0.87 19 #02 23-A3 0.6 Front / Side ±10° 12 0.7 2.4 0.91 17 #03 23-C1 0.6 Front / Side ±10° 9 1.1 3.8 0.86 23 #04 23-A2 0.6 Front / Side ±10° 10 0.8 2.9 0.89 18 #05 23-C2 0.6 Front / Side ±10° 7.5 0.6 2.1 0.92 16 #06 23-C1 0.6 Front / Side ±10° 10 1.2 4.4 0.85 24 #07 23-A3 0.6 Front / Side ±10° 9 0.9 3.0 0.88 20 #08 23-A2 0.6 Front / Side ±10° 12 0.7 2.6 0.90 18 #09 23-C2 0.6 Front / Side ±10° 10 1.0 3.3 0.87 21 #10 23-A3 0.6 Front / Side ±10° 9 0.8 2.7 0.91 17
[0155] As shown in Table 1, the validation results of ten cases of distal radius fractures demonstrate that the method of this invention achieved small alignment gaps and residual torsion in all cases, with average values controlled within the millimeter and several degrees ranges, respectively. The closure consistency scores were all at a high level, indicating stable and reliable gap pairing relationships. The online inference latency was maintained within twenty seconds, meeting the requirements for real-time intraoperative interaction. Overall, the results show that this invention can achieve fracture reduction navigation and assessment under multimodal imaging in actual surgical environments, possessing high accuracy and feasibility.
[0156] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for intelligent reduction navigation evaluation and analysis of fracture based on multi-modal images, characterized in that, The method comprises the following steps: obtaining preoperative computer tomography data, intraoperative fluoroscopy sequence and intraoperative ultrasound image, performing intensity standardization and artifact suppression on the multi-modal images, reconstructing a bone surface mesh of the target bone and a fracture gap triangular mesh; forming a bottom space with the bone surface mesh, forming fibers with the fracture local state vectors attached to each discrete point in the bottom space, forming a fiber space with the end state of all fibers, and establishing a bone fragment-fracture fiber cluster representation; defining a connection and parallel movement rule on the bone fragment-fracture fiber cluster; constructing multi-order convolution kernels on the point unit, edge unit and surface unit of the bottom space and fiber space, and embedding multi-modal constraint weights and gating parameters in the convolution update process; after one or more steps of multi-order convolution propagation, constructing a broken end pairing graph in the symmetrical candidate region of the bottom space according to the normal complement, roughness compatibility and compressibility threshold of the fiber end, calculating the broken end pairing probability, determining the symmetrical path and the bone fragment reduction operation sequence; in the solving process of the reduction operation sequence, applying non-penetration barrier constraint and normal continuity regularization constraint to the relative movement of the bone fragments to obtain a reduction solution; generating intraoperative navigation prompt information according to the reduction solution, and outputting real-time evaluation indicators.
2. The multi-modal image-based intelligent reduction navigation evaluation and analysis method for bone fracture according to claim 1, characterized in that, The bone surface mesh reconstruction of the target bone comprises: inputting the standardized and artifact-suppressed volume data into a three-dimensional segmentation network to extract the bone cortex and cancellous bone regions; generating a triangular mesh of the bone fragment based on the segmentation result, and ensuring the manifold characteristics of the mesh through boundary smoothing and topological repair; The reconstruction of the fracture gap triangular mesh comprises: detecting the gray scale gradient and curvature abnormal area along the bone surface mesh to locate the potential fracture line; generating a triangular partition around the fracture line to form a fracture mesh; and calculating the fracture normal, roughness and local curvature during the fracture mesh reconstruction process. 3.The multi-modal image-based intelligent fracture reduction navigation evaluation and analysis method according to claim 1, characterized in that, The establishment of the bone fragment-fracture fiber cluster representation comprises: the vertices, edges and faces of the bone surface mesh are taken as discrete units of the bottom space; attaching a fracture local state vector to each discrete unit, which includes the normal direction, surface roughness value, local curvature radius, bone cortex thickness value, bone density measurement value, friction coefficient and compression modulus of the fracture mesh; index binding the fracture local state vector and the corresponding bottom space discrete unit to form a one-to-one correspondence between the bottom space and the fiber, and to form a bone fragment-fracture fiber cluster.
4. The multi-modal image-based intelligent reduction navigation evaluation and analysis method for bone fracture according to claim 1, characterized in that, The connection and parallel movement rule defined on the bone fragment-fracture fiber cluster comprises: establishing adjacent unit relationships between the vertices, edges and faces of the bone surface mesh; setting a connection operator between each adjacent unit, which is represented by an affine transformation matrix and is used to describe the coordinate transformation relationship of the fiber end state vector between adjacent units; when performing parallel movement, the normal direction, curvature radius and roughness value of the fracture local state vector are linearly transformed according to the connection operator, and the bone cortex thickness value, bone density measurement value, friction coefficient and compression modulus are numerically transmitted to obtain the updated state vector on the target unit.
5. The multi-modal image based intelligent fracture reduction navigation evaluation and analysis method according to claim 1, wherein, The convolution update process comprises: A convolution kernel is constructed at the point element layer to process the surface vertex of the bone fragment and the local state vector of the fracture attached thereto; A convolution kernel is constructed at the edge element layer to process the edge composed of adjacent vertices and the corresponding local state vector of the fracture; A convolution kernel is constructed at the surface element layer to process the fracture region defined by the triangular surface patch and the corresponding local state vector of the fracture; The multi-order convolution kernel is multiplied by a multi-modal constraint weight in the updating process, and the multi-modal constraint weight is derived from the intensity gradient of the computed tomography data, the residual value of the digital reconstructed radiograph, and the echo confidence of the ultrasound image; A gating parameter is introduced when the convolution result is output, and the gating parameter is used for weighting adjustment between different modal constraint weights, specifically, the convolution results of the three modes are proportionally distributed through adjustable weight coefficients.
6. The multi-modal image based intelligent fracture reduction navigation evaluation and analysis method according to claim 1, wherein, The construction of the end-to-end pairing graph includes: Selecting triangular surface patches located on the fracture boundary as the candidate elements for coaptation in the bottom space grid; Parameter comparison is performed on the fiber end state vectors of the coaptation candidate elements, in which the normal direction is calculated by the vector angle, the roughness is calculated by the surface roughness difference, and the compressibility is calculated by the compression modulus difference; When the normal direction, roughness, and compressibility parameters meet the preset threshold conditions, a pairing edge is established between the corresponding candidate elements to obtain an end-to-end pairing graph with candidate elements as nodes and pairing edges as lines; The pairing probability is calculated according to the number of pairing edges and the parameter compatibility degree on the end-to-end pairing graph, and the coaptation path is determined according to the size of the pairing probability; The bone fragment reduction operation sequence is generated according to the coaptation path, and the operation sequence includes translation, rotation, and compression.
7. The multi-modal image based intelligent fracture reduction navigation evaluation and analysis method according to claim 1, wherein, The non-interpenetration barrier constraint and normal continuity regularization constraint applied to the relative motion of the bone fragments include: A nearest point distance constraint is set between the bone fragment surface grids, and when the nearest point distance of any two bone fragment surfaces is less than zero, a penalty is applied to the corresponding operation sequence through a barrier function; A normal continuity constraint is set between the bone fragment fracture surface patches, and when the normal direction angle of adjacent fracture surface patches is greater than a preset threshold, the cost of the corresponding operation sequence is increased through a regularization term; The non-interpenetration barrier constraint and normal continuity regularization constraint are introduced simultaneously in the reduction operation sequence solving process to obtain a reduction solution that satisfies the constraint conditions.
8. The multi-modal image based intelligent fracture reduction navigation evaluation and analysis method according to claim 1, wherein, The generation of intraoperative navigation prompt information and the output of real-time evaluation indicators according to the reduction solution include: The translation, rotation, and compression in the reduction solution are combined with the spatial parameters of the intraoperative imaging device to generate navigation prompt information of the perspective imaging angle and the ultrasound probe attitude; The navigation prompt information is output to the intraoperative display terminal for the operator to adjust the positions of the imaging device and the reduction instrument; Real-time evaluation indicators are calculated during the reduction solution solving process, including coaptation gap, residual rotation angle, fracture closure consistency score, and multi-modal observation residual image; The real-time evaluation indicators and the navigation prompt information are output together.
9. The multi-modal image-based intelligent fracture reduction navigation evaluation and analysis system, which executes the multi-modal image-based intelligent fracture reduction navigation evaluation and analysis method of claims 1 to 8, characterized in that, The method includes the following modules: An image acquisition and preprocessing module is configured to acquire preoperative computed tomography (CT) data, intraoperative fluoroscopy sequences and intraoperative ultrasound images, and perform intensity normalization and artifact suppression on the multi-modal images; A bone fragment and fracture reconstruction module is configured to reconstruct a bone fragment surface mesh and a fracture surface mesh based on the preprocessed multi-modal images; A bone fragment-fracture fiber bundle construction module is configured to construct a bottom space with the bone fragment surface mesh, attach a local state vector to each discrete unit in the bottom space to form a fiber, and establish a bone fragment-fracture fiber bundle representation; A connection and translation module is configured to define a connection operator on the bone fragment-fracture fiber bundle, and perform parallel movement of the fiber end state vector between adjacent units in the bottom space; A multi-order convolution calculation module is configured to construct multi-order convolution kernels on the point units, edge units and surface units in the bottom space and the fiber space, and embed multi-modal constraint weights and gating parameters in the convolution update process; A broken end pairing and reduction calculation module is configured to establish a broken end pairing graph based on the fiber end state vector in the symmetrization candidate region of the bottom space after one or more steps of multi-order convolution propagation, calculate a pairing probability, determine a symmetrization path and a bone fragment reduction operation sequence, and calculate a reduction solution by imposing a non-penetration barrier constraint and a normal continuity regularization constraint on the relative motion of the bone fragments during the solution process of the reduction operation sequence; A navigation prompt and evaluation module is configured to generate intraoperative navigation prompt information based on the reduction solution, and output real-time evaluation indicators including a symmetrization gap, a residual rotation angle, a fracture closure consistency score and a multi-modal observation residual image.