Bone joint three-dimensional reconstruction method and system based on deep learning

Abstract

The invention relates to the technical field of medical image reconstruction, and discloses a bone joint three-dimensional reconstruction method and system based on deep learning. The method comprises the steps that a bone joint tomography image set is input, and multi-scale density slices and edge contour information are analyzed; deducing the two types of information space structures, and establishing an initial space field containing a bone tissue main body frame and a joint lacuna form; separating articular surface structure details and bone surface texture veins from the initial field; generating a network by using the three-dimensional structure which is obtained through separation and input and is trained to obtain a grid body with anatomical topological connection; and performing curved surface iterative smoothing based on biomechanical characteristics on the grid body, and outputting a final bone joint three-dimensional model.

Description

A Deep Learning-Based Method and System for 3D Reconstruction of Bone and Joint Technical Field

[0001] This invention relates to the field of medical image reconstruction technology, specifically to a method and system for three-dimensional reconstruction of bones and joints based on deep learning. Background Technology

[0002] Current methods for 3D reconstruction of bones and joints often involve direct processing of single-modal data from tomographic scans. Conventional techniques frequently neglect the combined use of multi-scale density and edge contour information, relying solely on overall grayscale distribution to deduce spatial structure. This makes it difficult to simultaneously present the spatial location of the main framework of bone tissue and the specific morphology of the joint cavity, resulting in a blurred relative spatial relationship between bone and joint cavity in the reconstruction results, and failing to accurately reflect the nested characteristics of anatomical structures.

[0003] Conventional reconstruction processes fail to separate the structural details of the articular surfaces from the overall structure, often generating them along with the main bone structure. This makes the fine morphology of the articular surfaces—such as their elevation, curvature, and other features—often obscured by bone texture, as these are the functional load-bearing parts of the joint. Furthermore, the bone surface texture loses its independent anatomical characteristics due to its mixing with the articular surface details. Existing methods lack spatial structural extrapolation of multi-scale density slices and edge contour information to establish an initial spatial field containing the main bone framework and joint cavity morphology. They also fail to separate the structural details of the articular surfaces from the bone surface texture in the initial field. Consequently, the reconstructed model deviates from true anatomy in its representation of joint function-related structures and bone surface features, failing to meet the dual requirements of bone and joint morphology analysis for both local details and overall spatial relationships. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for three-dimensional reconstruction of bones and joints based on deep learning, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, this invention provides a deep learning-based three-dimensional reconstruction method for bone and joints, the method comprising:

[0006] Input a set of tomographic scan images of bones and joints, and parse out multi-scale density slices and edge contour information;

[0007] Spatial structure deduction is performed on multi-scale density slices and edge contour information to establish an initial spatial field containing the main framework of bone tissue and the morphology of joint cavities;

[0008] The structural details of the articular surfaces and the texture of the bone surface are separated from the initial spatial field;

[0009] The separated structural details and texture details are input into a 3D structure generation network trained on samples to obtain a mesh with anatomical topological connections.

[0010] Perform iterative smoothing of the mesh based on biomechanical features to output the final three-dimensional model of the bone joint.

[0011] Preferably, the step of performing spatial structure deduction on multi-scale density slices and edge contour information to establish an initial spatial field containing the main framework of bone tissue and the morphology of joint cavities includes the following steps:

[0012] The gray-level gradient change data is extracted layer by layer from the tomographic image set, and the boundary transition region of bone tissue is located based on the gray-level gradient change data;

[0013] Within the boundary transition area, discontinuous point sets in the edge contour information are identified, and interpolation bridging of the discontinuous point sets is performed according to the principle of tissue continuity to form a closed bone tissue outer contour line.

[0014] The outer contour lines of bone tissue on adjacent tomographic slices were stacked with voxels and morphologically expanded along the scanning axis to construct a continuous spatial skeleton that represents the main framework of bone tissue.

[0015] Simultaneously, regions with lower density than bone tissue are identified in multi-scale density slices and mapped into the continuous spatial skeleton, marked as the initial occupancy space of the joint cavity;

[0016] By integrating the initial occupancy space of the continuous spatial skeleton and the joint cavity, an initial spatial field containing a solid frame and a hollow cavity structure is generated.

[0017] Preferably, the separation of the structural details of the articular surfaces and the texture of the bone surface from the initial spatial field is specifically achieved as follows:

[0018] In the initial spatial field, a series of virtual cutting planes are set along the preset tangential direction of the joint motion trajectory;

[0019] Calculate the curvature distribution spectrum of the intersection line between the virtual cutting plane and the main framework of bone tissue, extract local high curvature feature points exceeding a predetermined threshold from the curvature distribution spectrum, and the set of local high curvature feature points constitutes the structural details of the articular surface;

[0020] A diffusion field simulating tissue growth is applied to the surface of the main bone tissue framework, and the equipotential line distribution pattern of the diffusion field is traced.

[0021] The waveform features exhibiting periodic density variations in the equipotential line distribution pattern are extracted and quantified into the texture of the bone surface. The texture contains information on the orientation of trabeculae and the fluctuation of cortical bone thickness.

[0022] Preferably, the step of inputting the separated structural details and texture details into a 3D structure generation network trained on samples to obtain a mesh with anatomically connected topology includes the following processing steps:

[0023] The three-dimensional structure generation network includes a cascaded feature extractor and a mesh generator, wherein the feature extractor receives the structural details of the articular surfaces and the texture of the bone surface;

[0024] The feature extractor performs multi-level convolution and pooling operations on the structural details of the joint surfaces, and outputs the geometric constraint feature vector of the joint surface contact area.

[0025] Meanwhile, the feature extractor performs directional filtering and feature encoding on the texture of the bone surface, and outputs the morphological constraint feature vector of the bone surface microstructure.

[0026] The geometric constraint feature vector and the morphological constraint feature vector are input into the mesh generator, which generates a set of triangular facets connecting the articular surfaces and bone surfaces in three-dimensional space based on prior knowledge of anatomical atlases.

[0027] The triangular facet assembly is subjected to manifold inspection and hole repair to form a closed mesh that satisfies anatomical connectivity.

[0028] Preferably, the step of performing iterative smoothing of the mesh based on biomechanical features to output the final three-dimensional model of the bone joint is accomplished through the following steps:

[0029] Obtain the stress distribution map of the joint under typical load conditions, and map the stress distribution map onto the surface of the mesh;

[0030] Using stress concentration regions as control anchor points, a virtual elastic film model is constructed on the surface of the mesh. The equilibrium position of the elastic film model is determined by the mapped stress value.

[0031] Initiate an iterative smoothing process. In each iteration, calculate the displacement vector of each vertex on the mesh surface under the action of the elastic thin film model, and adjust the position according to the geometric relationship of adjacent vertices so that the surface curvature change matches the stress distribution trend.

[0032] During the iteration process, the volume conservation of the mesh and the continuity of the joint surface curvature are checked in real time. If the preset geometric constraints are violated, a penalty term is introduced to correct the displacement vector.

[0033] The iteration terminates when the average displacement of all vertices on the mesh surface is less than the convergence threshold, and the smooth mesh that conforms to biomechanical characteristics at this point is output as the final 3D model of the bone joint.

[0034] Preferably, the construction and calculation process of building a virtual elastic thin film model on the surface of the mesh specifically includes:

[0035] Each triangular facet of the mesh is considered as a discrete element of an elastic thin film, and a local stiffness matrix based on Young's modulus and Poisson's ratio is defined for each element.

[0036] Based on the stress values ​​at the corresponding positions in the stress distribution diagram, calculate the equivalent nodal force acting on each discrete element node;

[0037] By integrating the local stiffness matrices and equivalent nodal forces of all discrete elements, a global stiffness equation describing the equilibrium state of the entire elastic membrane is formed.

[0038] Solving the global stiffness equation yields the initial displacement field of each vertex of the mesh under static equilibrium, which serves as a reference for subsequent iterative smoothing.

[0039] Preferably, during the iterative smoothing process, the position is adjusted according to the geometric relationship between adjacent vertices, and the adjustment strategy adopted is as follows:

[0040] For any vertex on the surface of the mesh, find all its directly connected adjacent vertices and calculate the edge vectors from that vertex to each of its adjacent vertices.

[0041] Based on the edge vector, the local average curvature normal vector at the vertex is calculated, and the local average curvature normal vector characterizes the concave and convex trends of the surface at that point.

[0042] The local average curvature normal vector and the displacement vector generated by the elastic thin film model at that point are weighted and synthesized to generate the comprehensive adjustment direction of the vertex;

[0043] The vertex is moved along the comprehensive adjustment direction, and the moving distance is determined by the difference between the current curvature of the point and the target smooth curvature, as well as the normalized result of the stress value.

[0044] Preferably, the feature extractor further includes an optimization step for the geometric constraint feature vector during the process of processing the structural details of the joint surfaces:

[0045] After obtaining the geometric constraint feature vector, an attention weighting mechanism is introduced, which scans the regions with drastic curvature changes in the details of the joint surface structure.

[0046] Higher weight coefficients are assigned to features in regions of drastic curvature change, while feature weights are suppressed in regions of gentle curvature.

[0047] The geometric constraint feature vector is recalculated using the weighted features, enabling the subsequent mesh generator to generate a high-resolution mesh that fits the anatomical structure more accurately in the corresponding region.

[0048] Preferably, the method further includes, after outputting the final three-dimensional model of the bone joint, performing a process of model accuracy verification and adaptive calibration, which includes:

[0049] Laser three-dimensional scanning point cloud data from the same specimen as the tomographic image set is acquired and used as a high-precision reference model;

[0050] The final three-dimensional model of the bone joint is rigidly registered with the high-precision reference model in a unified coordinate system;

[0051] Calculate the shortest distance between corresponding surfaces of the two registered models to form the overall error distance field;

[0052] Analyze the statistical distribution of the error distance field to identify systematic deviation regions where the error continuously exceeds the tolerance threshold;

[0053] The spatial coordinates and deviation direction of the systematic deviation region are fed back into the training process of the three-dimensional structure generation network to adjust the network parameters and achieve adaptive calibration of the accuracy of the subsequent reconstruction model.

[0054] Preferably, when the processor executes the computer program, it implements the steps of the deep learning-based three-dimensional reconstruction method for bone joints as described in any of the above-mentioned methods.

[0055] Compared with the prior art, the beneficial effects of the present invention are:

[0056] By analyzing a collection of bone and joint tomographic images, multi-scale density slices and edge contour information are extracted. An initial spatial field containing the main framework of bone tissue and the morphology of joint cavity is established through spatial structure deduction. This allows the overall spatial occupancy of the main bone tissue and the width and direction of the joint cavity, as well as their relative position to the bone structure, to be directly integrated into the construction logic of the initial field. Unlike the conventional method of deducing space based on a single gray value, this method allows the initial field to simultaneously carry the dual elements of bone and joint cavity. In the subsequent generation process, the spatial positioning of the main bone framework and the morphology of the joint cavity can maintain the same anatomical correlation as the original scan, avoiding morphological misalignment caused by the lack of a relationship between bone and joint cavity.

[0057] By separating the structural details of the articular surfaces and the texture of the bone surface from the initial spatial field, the fine local morphology such as the undulations of the articular surfaces and the curvature changes of the contact surfaces, which were originally mixed in the overall structure, as well as the surface features such as the orientation of bone trabeculae and the texture of cortical bone grooves, are extracted separately. This changes the situation where the details of the articular surfaces and bone textures are mutually obscured in the overall reconstruction. This allows the structural details and textures to enter the three-dimensional structure generation network as independent information. When generating a mesh with anatomical topological connections, the functional morphological features of the articular surfaces and the texture of the bone surface can be accurately mapped separately. The mesh representation of the articular surfaces and bone surfaces is closer to the local differences and overall coordination of real anatomy. Attached Figure Description

[0058] Figure 1 is a schematic diagram illustrating the working principle of the deep learning-based three-dimensional reconstruction method for bone and joints described in this invention.

[0059] Figure 2 is a flowchart of the initial spatial field construction;

[0060] Figure 3 is a flowchart of the three-dimensional structure generation network;

[0061] Figure 4 is a heat map of the error distance field distribution after iterative smoothing of the three-dimensional model of the bone and joint;

[0062] Figure 5 shows a comparison of the cumulative error distribution function before and after calibration of the three-dimensional reconstruction model of the bone and joint. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Referring to Figure 1, this invention provides a deep learning-based method for 3D reconstruction of bone and joints. The method includes: acquiring a set of tomographic scan images of the bone and joint; extracting multi-scale density slices and edge contour information through a parsing module; subsequently, a spatial structure deduction module uses the above information to establish an initial spatial field, which simultaneously includes the main framework of bone tissue and the morphology of joint cavities; and accurately separating the structural details of the articular surfaces and the texture of the bone surface from the initial spatial field through a feature separation module. The obtained data is input into a 3D structure generation network pre-trained with a large number of labeled samples, which outputs a triangular mesh with correct anatomical topological connections. Finally, an iterative surface smoothing algorithm based on biomechanical features is applied to the mesh to optimize the model surface to conform to physiological stress states, outputting the final 3D model of the bone and joint.

[0065] In one embodiment of the present invention, referring to Figure 2, grayscale gradient change data is extracted layer by layer from a set of tomographic images, and the boundary transition region of bone tissue is located based on this data. Within the boundary transition region, discontinuous point sets in the edge contour information are identified, and interpolation bridging operations are performed on these discontinuous point sets strictly according to the principle of tissue continuity, thereby forming a closed outer contour line of bone tissue. The outer contour lines of bone tissue on adjacent tomographic slices are subjected to voxel stacking and morphological dilation operations along the scanning axis to construct a continuous spatial skeleton representing the main framework of bone tissue. Simultaneously, regions with lower density than bone tissue are identified in multi-scale density slices, and these regions are mapped to the interior of the continuous spatial skeleton and marked as the initial occupancy space of the joint cavity. The continuous spatial skeleton and the initial occupancy space of the joint cavity are fused to generate an initial spatial field that simultaneously contains a solid frame and a hollow cavity structure. In the initial spatial field, a series of virtual cross-sections are set along the preset tangential direction of the joint movement trajectory. The curvature distribution spectrum of the intersection line between the virtual section plane and the main bone tissue framework is calculated. Locally high curvature feature points exceeding a predetermined threshold are extracted from this spectrum; the set of these locally high curvature feature points constitutes the structural details of the articular surface. A diffusion field simulating tissue growth is applied to the surface of the main bone tissue framework, and the equipotential line distribution pattern of this diffusion field is tracked. Waveform features exhibiting periodic density variations in the equipotential line distribution pattern are extracted and quantified into the texture network of the bone surface. This texture network contains information on the orientation of trabecular bone and the fluctuation of cortical bone thickness.

[0066] In practical implementation, taking a set of knee joint CT tomographic images as an example, the image sequence has 500 layers, a layer thickness of 0.625 mm, and a pixel resolution of 512x512. The parsing process calculates the gray-level gradient changes of each pixel along the X and Y directions for each tomographic image. The gray-level gradient changes are obtained through Sobel convolution, and the calculation formula is as follows:

[0067] in: The magnitude of the grayscale gradient change data of a pixel. and These represent the gradient components obtained after convolution by the Sobel operators in the horizontal and vertical directions, respectively. In specific implementations, based on the magnitude of the grayscale gradient change data, pixel regions with magnitudes within a preset range are positioned as boundary transition regions of bone tissue. This region corresponds to the density change zone from compact bone to surrounding soft tissue or joint cavity.

[0068] In some embodiments, within the identified boundary transition region, the Canny edge detection algorithm is further applied to extract single-pixel-wide edge contour information. In specific implementations, the edge contour information may contain discontinuous point sets due to image noise or partial volume effects, which manifest as breaks or gaps on the edge lines. Based on the principle of tissue continuity, cubic spline interpolation is used to bridge the gaps between adjacent discontinuous point sets, making the outer contour line of bone tissue on each slice a closed, continuous curve. It can be understood that when the outer contour lines of bone tissue on adjacent slices are stacked along the scanning axis, a three-dimensional voxel binary mask is formed. A morphological dilation operation is performed on this mask, using a sphere with a radius of 2 voxels as the structural element. After the dilation operation, a continuous and smooth spatial skeleton representing the main framework of bone tissue can be constructed.

[0069] Simultaneously, a density threshold is set in the multi-scale density slices to distinguish bone tissue from low-density regions. In specific implementations, all regions with voxel values ​​below this threshold are identified, and the spatial coordinates of these regions are mapped to the volume defined by the aforementioned continuous spatial skeleton. The mapped low-density regions are marked as the initial occupancy space of the joint cavity, such as the gap between the femur and tibia and the space behind the patella. The initial occupancy space of the joint cavity is fused by marking the voxels representing the continuous spatial skeleton as solids and the voxels representing the initial occupancy space of the joint cavity as cavities, thereby generating a binary three-dimensional matrix containing a solid framework and a hollow cavity structure. This matrix is ​​the initial spatial field.

[0070] In the initial spatial field, a series of virtual cross-sections are pre-defined along the tangent direction of the instantaneous rotation center, based on the biomechanical axis of knee flexion and extension. This can be understood as calculating the cut-off line generated by the intersection of each virtual cross-section with the main bone structure, and analyzing the curvature of each point on the cut-off line to form a curvature distribution spectrum. In specific implementation, local high-curvature feature points with curvature values ​​exceeding a predetermined threshold are extracted from the curvature distribution spectrum. These local high-curvature feature points are concentrated in the bulges and depressions of the articular surface of the femoral condyle, and the set of local high-curvature feature points constitutes the structural details of the articular surface. Optionally, an initial seed point is defined on the surface of the main bone structure. A diffusion field simulating tissue growth is applied from the seed point, and the diffusion field iteratively propagates according to the heat conduction equation. The equipotential line distribution pattern formed by the diffusion field on the bone surface is tracked, and the waveform features of the periodic density changes consistent with the trabecular bone arrangement direction presented in the equipotential line distribution pattern are extracted. Optionally, this waveform feature can be quantized into spatial frequency and amplitude information through Fourier transform. This information is the texture of the bone surface, which contains information about the orientation of trabeculae and the variation in cortical bone thickness.

[0071] In one embodiment of the present invention, referring to Figure 3, the three-dimensional structure generation network includes a cascaded feature extractor and a mesh generator. The feature extractor receives the structural details of the articular surfaces and the texture of the bone surface obtained by the aforementioned process. The feature extractor performs multi-level convolution and pooling operations on the structural details of the articular surfaces, outputting geometric constraint feature vectors of the articular surface contact areas. Simultaneously, the feature extractor performs directional filtering and feature encoding on the texture of the bone surface, outputting morphological constraint feature vectors of the bone surface microstructures. The geometric constraint feature vectors and morphological constraint feature vectors are input into the mesh generator, which generates a set of triangular facets connecting the articular surfaces and the bone surface in three-dimensional space based on prior knowledge of anatomical atlases. The triangular facet set is subjected to manifold checking and hole repair to form a closed mesh that satisfies anatomical connectivity. During the processing of the structural details of the articular surfaces by the feature extractor, an attention weighting mechanism is introduced, which scans regions of drastic curvature changes in the structural details of the articular surfaces. Higher weight coefficients are assigned to features in regions of drastic curvature changes, while suppressing feature weights in regions of gentle curvature. The geometric constraint feature vector is recalculated using the weighted features, enabling the subsequent mesh generator to generate a high-resolution mesh that fits the anatomical structure more accurately in the corresponding region.

[0072] In its implementation, the 3D structure generation network comprises a cascaded feature extractor and a mesh generator. The input to the feature extractor is the structural details of the articular surfaces and the texture of the bone surface derived from the aforementioned separation process. The structural details of the articular surfaces are represented as 3D point clouds, while the texture of the bone surface is represented as a feature map attached to the main framework surface of the bone tissue. Taking a knee joint dataset as an example, the input to the structural details of the articular surfaces is a point cloud containing approximately 50,000 high-curvature feature points, and the input to the texture of the bone surface is a 256x256x32 multi-channel feature map covering the distal femur surface.

[0073] In its implementation, the feature extractor performs multi-level convolution and pooling operations on the structural details of the articular surfaces. These multi-level convolution and pooling operations are implemented through three 3D convolutional modules, each containing a convolutional layer, a batch normalization layer, and a ReLU activation layer, followed by a max-pooling layer with a stride of 2. After these multi-level convolution and pooling operations, the articular surface point cloud is encoded into a 1024-dimensional geometric constraint feature vector of the articular surface contact region. Simultaneously, the feature extractor performs directional filtering and feature encoding on the texture veins of the bone surface. The directional filtering uses a set of eight Gabor filter kernels for convolution to enhance the texture features of trabecular bone in different orientations. Subsequently, the filtered feature map passes through an encoder network, outputting a 512-dimensional morphological constraint feature vector of the bone surface microstructure. It can be understood that after obtaining the geometric constraint feature vector, an attention weighting mechanism is introduced. This attention weighting mechanism works by scanning the local curvature changes of the point cloud in the structural details of the articular surfaces. The attention weighting mechanism calculates the curvature variance in the neighborhood of each feature point in the point cloud and inputs the curvature variance into a fully connected network to generate weight coefficients.

[0074] In some embodiments, the attention weighting mechanism assigns higher weight coefficients to features in regions of rapid curvature change, while suppressing feature weights in regions of gentle curvature. The formula for calculating the weight coefficients is as follows:

[0075] in: These represent the weight coefficients assigned to the features of the i-th local region. This represents the curvature variance measure of the local region. and These are the scaling and bias parameters learned by the fully connected network. This represents the Sigmoid activation function. The geometric constraint feature vector is recalculated using the weighted features; specifically, the original geometric constraint feature vector is combined with the features weighted by the coefficients. The reweighted features are fused to obtain the optimized geometric constraint feature vector.

[0076] In practice, the optimized geometric constraint feature vector and morphological constraint feature vector are concatenated along the channel dimension to form a 1536-dimensional joint feature vector, which is then input into the mesh generator. The mesh generator is a graph convolution-based neural network that, based on parameters learned from a prior knowledge base of anatomical atlases, progressively generates a set of triangular facets connecting articular surfaces and bone surfaces in three-dimensional space. The output of the mesh generator is an initial mesh containing approximately 100,000 vertices and 300,000 triangular facets. This set of triangular facets undergoes manifold checking and hole repair. Manifold checking verifies that each edge is shared by at most two triangular facets, while hole repair involves finding boundary loops and filling missing facets using the Delaunay triangulation algorithm, ultimately forming a closed and watertight mesh that satisfies anatomical connectivity. Optionally, the mesh generation process can be iterative to progressively refine the mesh's topology.

[0077] In one embodiment of the present invention, a stress distribution map of the joint under typical load conditions is obtained and mapped onto the surface of a mesh. Using stress concentration regions as control anchor points, a virtual elastic membrane model is constructed on the mesh surface. The equilibrium position of the elastic membrane model is determined by the mapped stress values. An iterative smoothing process is initiated. In each iteration, the displacement vector of each vertex on the mesh surface under the action of the elastic membrane model is calculated, and the position is adjusted according to the geometric relationship of adjacent vertices to make the surface curvature change consistent with the stress distribution trend. During the iteration process, the volume conservation of the mesh and the continuity of the joint surface curvature are checked in real time. If a preset geometric constraint is violated, a penalty term is introduced to correct the displacement vector. When the average displacement of all vertices on the mesh surface is less than the convergence threshold, the iteration is terminated, and the smooth mesh that conforms to biomechanical characteristics is output as the final three-dimensional model of the bone joint. The construction and calculation process of constructing the virtual elastic membrane model on the mesh surface involves treating each triangular facet of the mesh as a discrete unit of the elastic membrane and defining a local stiffness matrix based on Young's modulus and Poisson's ratio for each unit. Based on the stress values ​​at the corresponding locations in the stress distribution diagram, the equivalent nodal forces acting on each discrete element node are calculated. The local stiffness matrices and equivalent nodal forces of all discrete elements are integrated to form a global stiffness equation describing the equilibrium state of the entire elastic membrane. Solving this global stiffness equation yields the initial displacement field of each vertex of the mesh under static equilibrium; this initial displacement field serves as a reference for subsequent iterative smoothing.

[0078] In practice, stress distribution maps of the joint under typical load conditions are obtained. A typical load condition is, for example, the knee joint bearing weight on one leg during the mid-standing phase. The stress distribution map is calculated using finite element analysis software by applying a load of 1.5 times the body weight to the initial mesh and stored as a scalar field. The stress distribution map is mapped to the surface of the mesh through vertex attribute mapping, so that each vertex of the mesh is associated with a stress scalar value. Stress concentration regions are used as control anchor points. A stress concentration region is defined as a set of vertices with stress values ​​more than twice the average stress value. A virtual elastic membrane model is constructed on the mesh surface using these control anchor points as constraint points. The equilibrium position of the virtual elastic membrane model is determined by the stress value mapped to each vertex; the higher the stress value, the more pronounced the tendency for the equilibrium position to concave inwards towards the model.

[0079] In practical implementation, the computational process of constructing a virtual elastic membrane model on the mesh surface involves treating each triangular facet of the mesh as a discrete element of the elastic membrane, and defining a local stiffness matrix for each discrete element based on Young's modulus and Poisson's ratio. The Young's modulus used is 1.5 MPa, and the Poisson's ratio is 0.45. Based on the stress values ​​at corresponding locations in the stress distribution diagram, the equivalent nodal forces acting on each discrete element node are calculated. These equivalent nodal forces transform the stress field within the element into a nodal force vector according to the static equivalence principle. The local stiffness matrices and equivalent nodal forces of all discrete elements are integrated to form a global stiffness equation describing the equilibrium state of the entire elastic membrane. The global stiffness equation is then solved; its form is:

[0080] in: This represents the overall stiffness matrix formed after integrating the local stiffness matrices of all discrete elements. The vector representing the displacement of all vertices under static equilibrium. This represents the overall load vector formed after integrating all equivalent nodal forces. Solving this system of linear equations yields the initial displacement field of each vertex of the mesh under static equilibrium. This initial displacement field serves as the reference for subsequent iterative smoothing. The iterative smoothing process is then initiated. In each iteration, the displacement vector of each vertex on the mesh surface under the action of the elastic thin film model is calculated. The displacement vector is determined by the force generated by the elastic thin film model and the internal damping of the mesh.

[0081] In some embodiments, the volume conservation of the mesh and the continuity of the articular surface curvature are checked in real time during the iteration process. Volume conservation is monitored by comparing the percentage change in the volume enclosed by the mesh before and after the iteration, and the continuity of the articular surface curvature is monitored by evaluating the change in the angle between the normal vectors of adjacent triangular facets in the articular surface region. If a preset geometric constraint is violated, such as a single iteration volume change exceeding 0.5% or a sudden change in the angle between the normal vectors of adjacent facets exceeding 30 degrees, a penalty term is introduced to correct the displacement vector. The penalty term is a vector proportional to the degree of constraint violation and has the opposite direction to the original displacement vector. It is understood that the iterative smoothing process continues, and when the average displacement of all vertices on the mesh surface is less than a preset convergence threshold, for example, when the average displacement is less than 0.01 mm, the iteration loop is terminated, and the smooth mesh that conforms to biomechanical characteristics at this point is output as the final 3D model of the bone joint. Optionally, the convergence threshold can be scaled proportionally according to the actual size of the model. Optionally, the physical simulation of calculating the displacement vector during the iteration process can employ an explicit time integration method. In the iterative smoothing process, when the explicit time integration method is chosen to calculate the displacement vector, the implementation of this method is based on the equivalent nodal forces and vertex motion relationship defined in the elastic thin film model. The explicit time integration updates the vertex displacement step by step through discrete time steps. In each time step, the acceleration and velocity changes of the vertex are directly calculated based on the current equivalent nodal forces, and the displacement vector increment is derived. There is no need to repeatedly solve the global stiffness equation, thus adapting to the need for real-time displacement adjustment in iterative smoothing. In the implementation of this method, the position is dynamically updated according to the mechanical constraints on the vertex, ensuring that the smoothing process takes into account both computational efficiency and biomechanical characteristics.

[0082] In one embodiment of the present invention, the strategy adopted for position adjustment based on the geometric relationship of adjacent vertices during iterative smoothing is as follows: For any vertex on the surface of the mesh, find all its directly connected adjacent vertices and calculate the edge vectors from the vertex to each adjacent vertex. Based on these edge vectors, calculate the local average curvature normal vector at the vertex, which characterizes the concavity and convexity trend of the surface at that point. The local average curvature normal vector is weighted and synthesized with the displacement vector generated at that point by the elastic thin film model to generate the comprehensive adjustment direction of the vertex. The vertex is moved along this comprehensive adjustment direction, and the moving distance is determined by the difference between the current curvature and the target smoothed curvature of the point, as well as the normalized result of the stress value.

[0083] In specific implementation, taking the smoothing of a knee joint mesh containing approximately 100,000 vertices as an example, the position is adjusted according to the geometric relationship of adjacent vertices during the iterative smoothing process. For any vertex on the surface of the mesh, such as a vertex located on the medial femoral condyle articular surface, all its directly connected adjacent vertices are found. The directly connected adjacent vertices are obtained by traversing the triangular facets with shared edges. This vertex has 6 adjacent vertices. The edge vectors from this vertex to each adjacent vertex are calculated. The edge vectors are obtained by subtracting the coordinates of the central vertex from the three-dimensional coordinates of the adjacent vertices.

[0084] Based on the edge vector, the local average curvature normal vector at the vertex is calculated. The calculation of the local average curvature normal vector uses a weighted method based on the area of ​​adjacent triangular facets. The local average curvature normal vector characterizes the concavity / convexity trend of the surface at that point. If the local average curvature normal vector points outside the mesh, it indicates that the region is a convex region. In specific implementation, the local average curvature normal vector and the displacement vector generated by the elastic thin film model at that point are weighted and synthesized. The displacement vector generated by the elastic thin film model is first normalized to obtain its direction vector. The overall adjustment direction is generated by linear interpolation between the normalized displacement direction vector and the local average curvature normal vector. The formula for generating the overall adjustment direction is:

[0085] in, Representing the overall adjustment direction, it is a unit direction vector. This represents the displacement direction vector obtained by normalizing the displacement vector generated by the elastic thin film model. This represents the normalized local mean curvature normal vector. This represents a weighting combination coefficient between 0 and 1. Along the direction of comprehensive adjustment. The distance the vertex moves is determined by the difference between the current curvature of the point and the target smooth curvature, as well as the normalized result of the stress value.

[0086] In some embodiments, the weight combination coefficient The value of depends on the vertex classification, which is based on its local curvature and mapped stress value. It can be understood that different vertices of different categories employ different weight combination strategies to achieve a balance between preserving anatomical features and smoothing stress concentration areas. See Table 1, which shows a weight combination coefficient for different surface feature regions of the knee joint mesh. The value of .

[0087] Table 1: Weighting Combination Coefficients for Different Surface Feature Regions

[0088] Surface feature region local curvature feature stress level weighting combination coefficient The joint contact high-stress zone has a high curvature (0.8); the non-load-bearing joint surface zone has a high curvature (0.4); the bone diaphysis cortex has a low curvature (moderate curvature) (0.6); and the tendon attachment zone (rough surface) has a drastic change in local curvature (0.3). surface

[0089] Optionally, the difference between the current curvature and the target smooth curvature is obtained by comparing the average Gaussian curvature in the vertex neighborhood with the curvature value of an ideal smooth model. Stress normalization involves linearly scaling the original stress value to a range of 0 to 1. The formula for calculating the moving distance is linearly proportional to the difference and the normalized stress value. Optionally, weight combination coefficients... The calculation can dynamically depend on the current iteration number and the local curvature gradient at the vertex. In some embodiments, the movement distance of all vertices in each iteration is recorded and used to monitor the stability of the smoothing process. Through the above adjustment strategy, the mesh surface can evolve in a direction that conforms to both biomechanical stress distribution and maintains a reasonable anatomical geometry during the iteration process.

[0090] Referring to Figure 4, the core of the error distance field distribution of the 3D bone joint model is the visualization of the surface deviation between the iteratively smoothed model and the high-precision reference point cloud. In the figure, the X and Y axes represent the normalized spatial position of the model surface, and the color coding corresponds to the deviation distance (unit: mm). The gradient change in color from dark blue (-0.025 mm) to dark red (0.175 mm) intuitively reflects the error magnitude in different regions. Specifically, through the statistical distribution of the error distance field, the "systematic deviation region" (highlighted in red in the figure) is located. The error in this type of region continuously exceeds the tolerance threshold, and its spatial coordinates and deviation direction can be fed back to the training process of the 3D structure generation network for parameter adjustment to achieve adaptive calibration of model accuracy. Most areas in the figure are light blue and yellow, indicating that the error in most areas is within a relatively small range of 0.025 mm to 0.075 mm. The red hue in the systematic deviation region corresponds to a higher deviation value, reflecting the structural deviation characteristics that still exist in local areas of the model after iterative smoothing. This visualization result provides precise spatial guidance for subsequent model calibration.

[0091] In one embodiment of the present invention, after outputting the final 3D model of the bone joint, a process of model accuracy verification and adaptive calibration is performed. This process includes acquiring laser 3D scanning point cloud data from the same specimen as the tomographic image set, as a high-precision reference model. The final 3D model of the bone joint and the high-precision reference model are rigidly registered in a unified coordinate system. The shortest distance between corresponding surfaces of the two models after registration is calculated to form an overall error distance field. The statistical distribution of the error distance field is analyzed to identify systematic deviation regions where the error continuously exceeds the tolerance threshold. The spatial coordinates and deviation directions of the systematic deviation regions are fed back to the training process of the 3D structure generation network to adjust the network parameters and achieve adaptive calibration of the accuracy of the subsequent reconstructed model.

[0092] In practical implementation, taking a reconstructed 3D knee joint model as an example, after outputting the final 3D bone joint model, a model accuracy verification and adaptive calibration process is performed. Laser 3D scan point cloud data, derived from the same human knee joint specimen as the CT tomographic image set used as the input source, is acquired. The laser 3D scan is performed using a 3D scanner with an accuracy of 0.02 mm. The generated high-precision reference model contains approximately 2 million surface point cloud data. The final 3D bone joint model and the high-precision reference model are rigidly registered in a unified coordinate system. The rigid registration uses an iterative nearest-point algorithm, aligning the final 3D bone joint model as the source model and the high-precision reference model as the target model. The shortest distance between corresponding surfaces of the two models after registration is calculated, forming an overall error distance field. For each vertex on the surface of the final 3D bone joint model, the Euclidean distance to all points in the point cloud of the high-precision reference model is calculated, and the minimum value is taken as the error distance at that vertex.

[0093] The statistical distribution of the error distance field is analyzed. This distribution is characterized by calculating the average, standard deviation, and histogram of the error distances at all vertices. Systematic deviation regions where the error consistently exceeds a tolerance threshold are identified. The tolerance threshold is set to 0.5 mm. Systematic deviation regions refer to surface areas with a continuous spatial range exceeding 10 square millimeters and an average error distance consistently exceeding the tolerance threshold. The spatial coordinates and deviation directions of these systematic deviation regions are fed back into the training process of the 3D structure generation network. The spatial coordinates of the systematic deviation regions are recorded as a set of 3D vertex indices, and the deviation direction is represented by the average normal vector pointing from the reconstructed model surface to the high-precision reference model surface. It is understood that the feedback strength coefficient, used to quantify the degree of deviation and guide network parameter adjustments, needs to be a dimensionless quantity. The feedback strength coefficient is jointly determined by the average error distance within the systematic deviation region and the feature length of the model. The calculation of the feedback strength coefficient... The formula is:

[0094] in: This represents the feedback strength coefficient for the current systematic deviation region. Represents the region of systematic deviation The average of the absolute values ​​of the error distances of all vertices within the range. This represents the feature length of the final 3D model of the bone joint, which is the diagonal length of the model's bounding box. In some embodiments, the feedback intensity coefficient... The spatial coordinates and deviation direction vector of the systematic deviation region are constructed together to form an additional training sample, which is used to adjust the network parameters of the 3D structure generation network. The focus of the network parameter adjustment is the weights of the feature extractor and mesh generator layers in the 3D structure generation network that are responsible for generating the geometry of the corresponding deviation region.

[0095] It is understandable that the adaptive calibration process can be performed periodically in batches after reconstructing multiple different sample models. The identification of systematic bias regions and the calculation of feedback intensity coefficients are performed independently for each model, but the collected feedback data are merged for centralized parameter updates of the network. Optional, feature length... Alternatively, anatomically standard dimensions, such as the width of the distal femoral condyle, can be used as a normalization benchmark. Optionally, in constructing training samples, in addition to the feedback intensity coefficient... The area and spatial location of the systematic bias region will also be encoded as input features.

[0096] Referring to Figure 5, the cumulative error distribution function (CDF) characteristics of the initial model and the model after four calibrations are presented in the accuracy verification and adaptive calibration process of the 3D bone and joint reconstruction model, and a tolerance threshold is introduced as the accuracy evaluation benchmark. In the figure, the horizontal axis represents the error distance (unit: mm), and the vertical axis represents the cumulative probability that the error does not exceed the corresponding distance; the red curve represents the cumulative error distribution of the initial model, the green curve represents the cumulative error distribution of the model after four calibrations, and the black dashed line represents the tolerance threshold of 0.5 mm. Specifically, the cumulative error distribution curve of the initial model shows a relatively gentle upward trend, with a cumulative probability of approximately 0.7 at the tolerance threshold (0.5 mm); while the cumulative error distribution curve of the model after four calibrations rises rapidly when the error distance is close to 0, and the cumulative probability approaches 1.0 at a distance much smaller than 0.5 mm (approximately 0.25 mm). This characteristic indicates that after four adaptive calibrations, the model error can cover the vast majority of samples within a smaller distance range, the systematic bias area is effectively corrected, and the model accuracy is significantly improved and far exceeds the tolerance threshold requirement.

[0097] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0098] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for three-dimensional reconstruction of bone joints based on deep learning, characterized in that, The process includes the following steps: inputting a set of tomographic scan images of the bone joint and resolving multi-scale density slices and edge contour information; performing spatial structure deduction on the multi-scale density slices and edge contour information to establish an initial spatial field containing the main framework of bone tissue and the morphology of joint cavities; separating the structural details of the articular surfaces and the texture of the bone surface from the initial spatial field; inputting the separated structural details and texture into a three-dimensional structure generation network trained on samples to obtain a mesh with anatomically topological connections; performing iterative surface smoothing based on biomechanical features on the mesh to output the final three-dimensional model of the bone joint.

2. The method for three-dimensional reconstruction of bone joints based on deep learning according to claim 1, characterized in that, The process of spatially extrapolating the structure of multi-scale density slices and edge contour information to establish an initial spatial field containing the main framework of bone tissue and the morphology of joint cavities includes the following steps: extracting gray-level gradient change data layer by layer from the tomographic image set, and locating the boundary transition region of bone tissue based on the gray-level gradient change data; identifying discontinuous point sets in the edge contour information within the boundary transition region, and interpolating and bridging the discontinuous point sets according to the principle of tissue continuity to form a closed outer contour line of bone tissue; performing voxel stacking and morphological dilation operations on the outer contour lines of bone tissue on adjacent tomographic slices along the scanning axis to construct a continuous spatial skeleton representing the main framework of bone tissue; simultaneously, identifying regions with lower density than bone tissue in the multi-scale density slices, and mapping these regions into the continuous spatial skeleton, marking them as the initial occupancy space of the joint cavity; fusing the continuous spatial skeleton and the initial occupancy space of the joint cavity to generate an initial spatial field containing a solid framework and a hollow cavity structure.

3. The method for three-dimensional reconstruction of bone joints based on deep learning according to claim 2, characterized in that, The specific implementation of separating the structural details of the articular surface and the texture of the bone surface from the initial spatial field is as follows: In the initial spatial field, a series of virtual cross-sections are set along the preset tangential direction of the joint movement trajectory; the curvature distribution spectrum of the intersection line between the virtual cross-sections and the main framework of the bone tissue is calculated, and local high curvature feature points exceeding a predetermined threshold are extracted from the curvature distribution spectrum. The set of local high curvature feature points constitutes the structural details of the articular surface; a diffusion field simulating tissue growth is applied to the surface of the main framework of the bone tissue, and the equipotential line distribution pattern of the diffusion field is tracked; the waveform features exhibiting periodic density changes in the equipotential line distribution pattern are extracted, and these waveform features are quantified into the texture of the bone surface. The texture contains information on the orientation of bone trabeculae and the fluctuation of cortical bone thickness.

4. The method for three-dimensional reconstruction of bone joints based on deep learning according to claim 3, characterized in that, The process of inputting the separated structural details and texture details into a 3D structure generation network trained on samples to obtain a mesh with anatomical topological connections includes the following steps: The 3D structure generation network contains a cascaded feature extractor and a mesh generator. The feature extractor receives the structural details of the articular surfaces and the texture details of the bone surfaces. The feature extractor performs multi-level convolution and pooling operations on the structural details of the articular surfaces, outputting geometric constraint feature vectors for the contact areas of the articular surfaces. Simultaneously, the feature extractor performs directional filtering and feature encoding on the texture details of the bone surfaces, outputting morphological constraint feature vectors for the microstructures of the bone surfaces. The geometric constraint feature vectors and morphological constraint feature vectors are input into the mesh generator. Based on prior knowledge of anatomical atlases, the mesh generator generates a set of triangular facets connecting the articular surfaces and the bone surfaces in 3D space. The set of triangular facets undergoes manifold checking and hole repair to form a closed mesh that satisfies anatomical connectivity.

5. The method for three-dimensional reconstruction of bone joints based on deep learning according to claim 4, characterized in that, The process of performing biomechanical feature-based surface iterative smoothing on the mesh to output the final 3D model of the bone joint is accomplished through the following steps: obtaining the stress distribution map of the joint under typical load conditions and mapping the stress distribution map onto the surface of the mesh; using the stress concentration region as the control anchor point, constructing a virtual elastic membrane model on the surface of the mesh, the equilibrium position of the elastic membrane model being determined by the mapped stress value; Initiate an iterative smoothing process. In each iteration, calculate the displacement vector of each vertex on the mesh surface under the action of the elastic thin film model, and adjust the position according to the geometric relationship of adjacent vertices so that the surface curvature change matches the stress distribution trend. During the iteration process, the volume conservation of the mesh and the continuity of the joint surface curvature are checked in real time. If the preset geometric constraints are violated, a penalty term is introduced to correct the displacement vector. When the average displacement of all vertices on the mesh surface is less than the convergence threshold, the iteration is terminated, and the smooth mesh that conforms to biomechanical characteristics is output as the final three-dimensional model of the bone joint.

6. The method for three-dimensional reconstruction of bone joints based on deep learning according to claim 5, characterized in that, The construction and calculation process of building a virtual elastic thin film model on the surface of the mesh includes: treating each triangular facet of the mesh as a discrete unit of the elastic thin film, and defining a local stiffness matrix for each unit based on Young's modulus and Poisson's ratio; calculating the equivalent nodal force acting on each discrete unit node according to the stress value at the corresponding position in the stress distribution diagram; integrating the local stiffness matrix and equivalent nodal force of all discrete units to assemble a global stiffness equation describing the equilibrium state of the entire elastic thin film; solving the global stiffness equation to obtain the initial displacement field of each vertex of the mesh under static equilibrium, and the initial displacement field serves as a reference for subsequent iterative smoothing.

7. The method for three-dimensional reconstruction of bone joints based on deep learning according to claim 6, characterized in that, During the iterative smoothing process, the position is adjusted according to the geometric relationship between adjacent vertices. The adjustment strategy adopted is as follows: for any vertex on the surface of the mesh, find all its directly connected adjacent vertices and calculate the edge vector from the vertex to each adjacent vertex; based on the edge vector, calculate the local average curvature normal vector at the vertex, which represents the concavity and convexity trend of the surface at that point; and weight and synthesize the local average curvature normal vector with the displacement vector generated by the elastic thin film model at that point to generate the comprehensive adjustment direction of the vertex. The vertex is moved along the comprehensive adjustment direction, and the moving distance is determined by the difference between the current curvature of the point and the target smooth curvature, as well as the normalized result of the stress value.

8. The method for three-dimensional reconstruction of bone joints based on deep learning according to claim 4, characterized in that, During the process of the feature extractor processing the structural details of the joint surface, an optimization step for the geometric constraint feature vector is also included: after obtaining the geometric constraint feature vector, an attention weighting mechanism is introduced, which scans the regions with drastic curvature changes in the structural details of the joint surface; higher weight coefficients are assigned to features in the regions with drastic curvature changes, while suppressing the feature weights in regions with gentle curvature; the geometric constraint feature vector is recalculated using the weighted features, so that the subsequent mesh generator can more accurately generate high-resolution meshes that fit the anatomical structure in the corresponding regions.

9. The method for three-dimensional reconstruction of bone joints based on deep learning according to claim 1, characterized in that, The method further includes a process of model accuracy verification and adaptive calibration after outputting the final bone and joint 3D model. This process includes: acquiring laser 3D scanning point cloud data from the same specimen as the tomographic image set as a high-precision reference model; rigidly registering the final bone and joint 3D model and the high-precision reference model in a unified coordinate system; calculating the shortest distance between corresponding surfaces of the two models after registration to form an overall error distance field; analyzing the statistical distribution of the error distance field to identify systematic deviation regions where the error continuously exceeds the tolerance threshold; and feeding back the spatial coordinates and deviation direction of the systematic deviation regions to the training process of the 3D structure generation network to adjust the network parameters and achieve adaptive calibration of the accuracy of the subsequent reconstructed model.

10. A deep learning-based three-dimensional reconstruction system for bone joints, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the deep learning-based three-dimensional reconstruction method for bone joints as described in any one of claims 1 to 9.

Images