Method, apparatus and device for image registration of bone structures

Abstract

The application is suitable for the field of computer-aided surgery technology and medical image processing technology, and provides an image registration method, device and equipment of bone structure, which comprises the following steps: acquiring a first pose of a first bone structure in a first coordinate system corresponding to a first image; constructing a joint kinematics model based on the positional relationship between the first bone structure and a second bone structure in a second image; generating a candidate pose of the second bone structure in the first coordinate system according to the joint kinematics model; establishing an interosseous collision and contact constraint term based on the candidate pose; and constructing an image response index based on a candidate bone surface of the second bone structure under the candidate pose; establishing an objective function according to the image response index and the interosseous collision and contact constraint term; and obtaining a registration result of the first bone structure and the second bone structure in the first image by solving the objective function. By using the above method, the degrees of freedom of the bone structure can be reduced, and the consistency and realizability of the multi-bone structure registration process can be improved.

Description

Technical Field

[0001] This application relates to the fields of computer-aided surgical technology and medical image processing technology, and in particular to an image registration method, apparatus and equipment for bone structures. Background Technology

[0002] Image registration is a common procedure in the medical field. It aligns multiple images, facilitating comprehensive observation by physicians. Operations involving image registration typically include different methods such as unimodal registration and multimodal registration. Multimodal registration refers to fusing images from different imaging modalities. For example, registering and fusing magnetic resonance imaging (MRI), which displays soft tissue details, with computed tomography (CT), which displays bony structures, generates an MRI / CT image.

[0003] For example, in scenarios such as preoperative planning for bone tumor surgery, lesion extent assessment, and limb-sparing surgical design, it is usually necessary to utilize both CT and MRI images. CT images can clearly reflect the bony structure and bone surface geometry, facilitating the creation of three-dimensional models of bone structures such as the femur and tibia; MRI images can clearly reflect the tumor extent, medullary cavity invasion, and signal changes in surrounding soft tissues. Therefore, in clinical workflows, it is usually necessary to spatially align the bone model in CT images with the corresponding anatomical region in MRI images to observe the positional relationship between bone structures and tumor regions in a unified coordinate system.

[0004] Existing technologies can employ registration methods based on artificial markers or artificial interactive point clouds, cross-modal registration methods based on contour, segmentation, or surface extraction, or registration methods based on whole-image similarity to register CT and MRI images. Each of these methods has its advantages and disadvantages, making it difficult to meet actual clinical needs. Summary of the Invention

[0005] In view of this, embodiments of this application provide an image registration method, apparatus, and device for bone structures, which can utilize joint connection relationships to transform the multi-bone structure cross-modal registration problem into a constrained low-dimensional optimization problem, thereby reducing the degree of freedom of solving bone structures, reducing manual operations in the registration process, improving the consistency and feasibility of the multi-bone structure registration process, and improving the registration speed and accuracy.

[0006] The first aspect of this application provides an image registration method for bone structures, including:

[0007] Obtain the first pose of the first bone structure in the first coordinate system corresponding to the first image;

[0008] Based on the positional relationship between the first bone structure and the second bone structure in the second image, a joint kinematic model is constructed. The joint kinematic model is used to represent the second pose of the second bone structure as a function of the first pose of the first bone structure and joint parameters. The first image and the second image both include the first bone structure and the second bone structure, and the first bone structure and the second bone structure are connected by the same joint.

[0009] Based on the joint kinematic model, candidate poses of the second bone structure in the first coordinate system are generated;

[0010] Based on the candidate pose, interbone collision and contact constraints are established; and based on the candidate bone surface of the second bone structure under the candidate pose, an image response index is constructed to reflect the degree of bone interface matching.

[0011] Based on the image response index and the interosseous collision and contact constraints, an objective function representing the second pose of the second bone structure is established.

[0012] The registration results of the first bone structure and the second bone structure in the first image are obtained by solving the objective function.

[0013] Optionally, the step of constructing a joint kinematic model based on the positional relationship between the first bone structure and the second bone structure in the second image includes:

[0014] Determine the initial joint relative pose from the first bone structure to the second bone structure in the second coordinate system corresponding to the second image;

[0015] Multiple joint parameters are determined, and a relative motion transformation from the second bone structure to the first bone structure is constructed based on the multiple joint parameters.

[0016] Based on the initial joint relative pose and the relative motion transformation, a joint kinematic model is constructed;

[0017] The joint parameters include at least the flexion-extension angle, internal and external rotation angle, and internal and external rotation angle.

[0018] Optionally, the relative motion transformation from the second bone structure to the first bone structure, determined by a plurality of said joint parameters, includes:

[0019] Extract three-dimensional models of the first bone structure and the second bone structure from the second image;

[0020] A joint local coordinate system is established based on the three-dimensional models of the first bone structure and the second bone structure; wherein, the Z-axis of the joint local coordinate system is determined based on the major axis of the first bone structure;

[0021] Based on the multiple joint parameters, determine multiple rotation matrices obtained by rotating the second bone structure around each axis of the joint local coordinate system;

[0022] Based on multiple rotation matrices, a representation of the relative motion transformation from the second bone structure to the first bone structure is constructed.

[0023] Optionally, generating the candidate pose of the second bone structure in the first coordinate system based on the joint kinematic model includes:

[0024] For any candidate parameter value of the joint parameter, calculate the candidate pose of the second bone structure corresponding to the candidate parameter value based on the joint kinematic model;

[0025] Based on the candidate pose, the mesh vertices of the second bone structure are transformed into the first coordinate system.

[0026] Optionally, establishing interosseous collision and contact constraints based on the candidate pose includes:

[0027] Calculate the penetration relationship and distance relationship between the first bone structure and the second bone structure when the second bone structure is in the candidate pose;

[0028] Based on the penetration relationship and the distance relationship, establish interosseous collision and contact constraints between the first bone structure and the second bone structure;

[0029] Wherein, when there is bone surface penetration between the first bone structure and the second bone structure, and / or the surface distance between the first bone structure and the second bone structure exceeds a preset range, the inter-bone collision and contact constraint term is given a penalty value.

[0030] Optionally, the step of constructing an image response index reflecting the degree of bone interface matching based on the candidate bone surface of the second bone structure in the candidate pose includes:

[0031] The bone surface of the second bone structure is sampled to obtain a set of sampling points and a corresponding set of normal vectors;

[0032] For each sampling point in the set of sampling points, one-dimensional sampling of gray values ​​is performed along the normal vector of the sampling point and the vector in the opposite direction of the normal vector to obtain a local gray value sequence;

[0033] Based on the local grayscale sequence, the local response value of each sampling point is calculated, and the local response value is used to represent the grayscale change characteristics of the corresponding sampling point.

[0034] Based on the local response values ​​of multiple sampling points, an image response index reflecting the overall matching degree of the bone interface is constructed.

[0035] Optionally, obtaining the registration result of the first bone structure and the second bone structure in the first image by solving the objective function includes:

[0036] Determine the optimal joint parameters, which are the joint parameters that minimize the objective function value;

[0037] The second pose of the second bone structure in the first coordinate system is calculated based on the optimal joint parameters.

[0038] Based on the first pose of the first bone structure in the first coordinate system and the second pose of the second bone structure in the first coordinate system, the registration result of the first bone structure and the second bone structure in the first image is determined.

[0039] Optionally, determining the optimal joint parameters includes:

[0040] Multiple candidate initial values ​​are obtained through discrete sampling, and the objective function values ​​corresponding to each candidate initial value are calculated.

[0041] Multiple candidate parameters are determined based on the objective function values, and continuous optimization is performed using the candidate parameters as the starting point for local optimization to obtain the optimal joint parameters.

[0042] A second aspect of this application provides an image registration apparatus for bone structures, comprising:

[0043] The first pose acquisition module is used to acquire the first pose of the first bone structure in the first coordinate system corresponding to the first image;

[0044] The kinematic model construction module is used to construct a joint kinematic model based on the positional relationship between the first bone structure and the second bone structure in the second image. The joint kinematic model is used to represent the second pose of the second bone structure as a function of the first pose of the first bone structure and joint parameters. The first image and the second image both include the first bone structure and the second bone structure, and the first bone structure and the second bone structure are connected by the same joint.

[0045] The candidate pose generation module is used to generate candidate poses of the second bone structure in the first coordinate system based on the joint kinematic model.

[0046] A constraint construction module is used to establish inter-bone collision and contact constraints based on the candidate pose.

[0047] The image response index construction module is used to construct an image response index that reflects the degree of bone interface matching based on the candidate bone surface of the second bone structure in the candidate pose.

[0048] The objective function establishment module is used to establish an objective function representing the second pose of the second bone structure based on the image response index and the interosseous collision and contact constraint terms.

[0049] The registration module is used to obtain the registration results of the first bone structure and the second bone structure in the first image by solving the objective function.

[0050] A third aspect of this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer device performs the method described in the first aspect above.

[0051] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a computer, implements the method described in the first aspect above.

[0052] A fifth aspect of this application provides a computer program product, including a computer program that, when run, causes the method described in the first aspect to be executed.

[0053] Compared with the prior art, the embodiments of this application have the following beneficial effects:

[0054] 1. By introducing joint kinematic constraints, the pose solution of the second bone structure can be transformed from a six-degree-of-freedom rigid body optimization to a low-dimensional parametric optimization, which helps to speed up the optimization solution and improve the efficiency of registration.

[0055] 2. Existing multi-bone registration methods typically require solving each bone structure as an independent rigid body, necessitating a search within a large parameter space, resulting in low efficiency. This application's embodiment, by adding a "joint kinematic modeling" step, represents the candidate pose of the second bone structure as a function of the pose of the first bone structure and a small number of joint parameters. This significantly reduces the search space and the number of unreasonable candidate poses in the solution space, enabling automatic solving of another bone structure without relying on markers, automatic segmentation, or deep learning models.

[0056] 3. In existing technologies, without automatic segmentation or manual point matching, the pose of another bone structure is often difficult to reliably obtain. The embodiments of this application utilize a combined modeling approach of "interosseous joint relationships + MRI surface response + contact constraints," enabling the second bone structure to be solved without additional manual point selection, reducing reliance on manual interaction and complex training data. Furthermore, interosseous collision detection and contact constraints help improve the anatomical rationality of candidate poses.

[0057] 4. The embodiments of this application add a "bone collision and contact constraint" step, which applies a penalty to candidate poses that have bone surface penetration or bone distance that significantly exceeds the physiological range, thereby eliminating solutions that are anatomically unreasonable and reducing the probability of erroneous registration results.

[0058] 5. Compared with the prior art that uses intensity similarity of the entire image for registration, the embodiments of this application utilize the geometric information of the bone surface of the second bone structure, perform local sampling only within the neighborhood of the bone surface, and construct response indicators based on normal grayscale changes, reducing the interference of irrelevant regions on the objective function, and are more suitable for cross-modal matching between CT bone models and MRI bone interfaces. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a schematic diagram of an image registration method for bone structures provided in an embodiment of this application;

[0061] Figure 2 This is a schematic diagram of another image registration method for bone structures provided in an embodiment of this application;

[0062] Figure 3 This is a schematic diagram of an image registration process for a bone structure provided in an embodiment of this application;

[0063] Figure 4 This is a schematic diagram of the modeling process of a joint kinematic model provided in an embodiment of this application;

[0064] Figure 5 This is a schematic diagram of a collision detection and contact constraint provided in an embodiment of this application;

[0065] Figure 6 This is a schematic diagram illustrating sampling along the normal direction of the bone surface of a second bone structure, provided in an embodiment of this application.

[0066] Figure 7 This is a schematic diagram of MRI volume data sampling provided in an embodiment of this application;

[0067] Figure 8 This is a schematic diagram of an optimization process for an objective function provided in an embodiment of this application;

[0068] Figure 9 This is a schematic diagram of an image registration device for bone structures provided in an embodiment of this application;

[0069] Figure 10 This is a schematic diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0070] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0071] As described in the background section, in scenarios such as preoperative planning for bone tumor surgery, lesion extent assessment, and limb-sparing surgical design, it is usually necessary to spatially align the bone model in CT images with the corresponding anatomical region in MRI images in order to observe the positional relationship between the bone structure and the tumor region in a unified coordinate system. For ease of understanding, before introducing the technical solutions provided in the embodiments of this application, several types of multimodal registration methods in the prior art will be briefly introduced.

[0072] The first category is registration methods based on manually selected markers or manually interacted point clouds. The main implementation involves the operator manually clicking several points on the bone cortex boundary in the MRI image to form a sparse point cloud. These sparse point clouds are then rigidly matched with a bone surface model extracted from the CT image to determine the pose of the bone structure in the MRI coordinate system. This method works by using manually selected corresponding anatomical points or boundary points as geometric constraints for registration, and then solving for rigid body transformations through point-to-surface distance optimization. Because manual point selection is required, the registration results obtained using this method are greatly affected by the operator's experience, point distribution, and click stability, making it difficult to guarantee the stability and consistency of the registration process.

[0073] The second category comprises cross-modal registration methods based on contour, segmentation, or surface extraction. The main implementation involves first extracting bone contours or surface models from CT images, and then extracting the corresponding bone boundaries, contours, or segmentation results from MRI images; finally, registration is performed based on contour similarity, surface distance, or image similarity. This method works by transforming the cross-modal problem into a geometric matching problem through segmentation or edge extraction. However, this method requires automatic segmentation or deep learning and is highly dependent on training data, model generalization ability, and image quality. Consequently, it suffers from low clinical acceptance in practical medical engineering applications and has not been widely adopted.

[0074] The third category is registration methods based on the similarity of the entire image. Its main implementation involves constructing a projection or resampling result from the CT image to the MRI image, measuring its similarity to the MRI image, and solving for the pose parameters using an optimization algorithm. This method works by establishing and solving a registration objective function based on image intensity, edges, or gradient distribution. However, automatic registration based on the similarity of the entire image has a large search space, typically requiring optimization in a six-degree-of-freedom rigid body space. It is susceptible to local extrema and suffers from insufficient computational stability.

[0075] Furthermore, in bone structures connected by joints, such as the femur and tibia, the aforementioned methods typically treat each bone structure as an independent object, failing to fully utilize the kinematic constraints and contact relationships inherent in the joint itself. This results in problems such as high solution dimensionality, high optimization difficulty, and complex registration process.

[0076] To address the aforementioned issues, embodiments of this application provide an image registration method, apparatus, and device for bone structures. This method can transform the multi-bone structure cross-modal registration problem into a constrained low-dimensional optimization problem by utilizing joint connections, without relying on automatic segmentation, deep learning, or manual markers. This reduces the solution degrees of freedom for another bone structure, decreases manual operations, and improves the consistency and feasibility of the multi-bone structure registration process.

[0077] Specifically, this method can be applied to the registration of images from different modalities containing at least two bone structures connected by joints. For example, this method can be used to register CT images and MRI images containing at least two bone structures connected by joints. The main process is as follows: First, the known or prior pose of the first bone structure in the MRI coordinate system is obtained. Then, based on the relative anatomical relationship between the first and second bone structures in the CT image, a joint kinematic model is constructed, and the pose of the second bone structure in the MRI coordinate system is expressed as a function of the pose of the first bone structure and joint parameters. On this basis, a target function is further constructed based on the image response of the bone surface of the second bone structure in the MRI image, and the joint parameters are optimized by combining inter-bone collision and contact constraints to obtain the registration pose of the second bone structure in the MRI image.

[0078] This method not only solves the problems of existing multi-bone cross-modal registration, which requires independent manual registration for each bone structure, resulting in repetitive processes and complex operations; it also addresses the issue that existing registration methods heavily rely on manual markers, automatic segmentation results, or deep learning models, leading to complex engineering implementation and limited clinical acceptance. Furthermore, this method also solves the problem that existing registration methods require optimizing another bone structure as a six-degree-of-freedom independent rigid body during registration, resulting in a large search space, susceptibility to local extrema, and insufficient registration stability.

[0079] The technical solution of this application will be described below through specific embodiments.

[0080] Reference Figure 1 The diagram illustrates an image registration method for bone structures provided in an embodiment of this application, which may specifically include the following steps:

[0081] S101. Obtain the first pose of the first bone structure in the first coordinate system corresponding to the first image.

[0082] It should be noted that the embodiments of this application can be applied to computer devices, that is, the executing entity of this method can be a computer device. The computer device can achieve registration of multimodal image data by executing the various steps of this method. The aforementioned computer device can be a desktop computer, a cloud server, or other similar devices; the embodiments of this application do not limit the specific type of computer device.

[0083] In this embodiment, multimodal image data can refer to two different types of image data, namely a first image and a second image. Both the first and second images can be three-dimensional images, and both include at least two bone structures connected by the same joint, such as a first bone structure and a second bone structure. In other words, both the first and second images include at least a first bone structure and a second bone structure, which are connected by the same joint.

[0084] In one example, the first image may be an MRI image, and the second image may be a CT image. Alternatively, the first image may be a positron emission tomography (PET) image, and the second image may be an MRI image. This application does not limit the specific types of the first and second images in its embodiments.

[0085] The purpose of applying the embodiments of this application is to register bone structures in a first image with bone structures in a second image. For example, when the first image is an MRI image and the second image is a CT image, applying the embodiments of this application can register bone structures in the MRI image with bone structures in the CT image.

[0086] The bone structures in the first and second images mentioned above include a first bone structure and a second bone structure, which are connected by the same joint. For example, the first and second bone structures can be the femur and tibia, the humerus and radius and ulna, or adjacent bone structures such as the femur and patella.

[0087] To better illustrate the image registration method for bone structures provided in the embodiments of this application, the following descriptions will use MRI images as the first image and CT images as the second image as examples.

[0088] In the embodiments of this application, the coordinate system corresponding to any image can refer to a coordinate system constructed based on that image. For example, the first coordinate system corresponding to a first image can refer to a coordinate system constructed based on the first image. For instance, if the first image is an MRI image, the first coordinate system corresponding to the first image refers to a coordinate system constructed based on the MRI image, which can be represented as the MRI coordinate system. Correspondingly, the second coordinate system corresponding to a second image can refer to a coordinate system constructed based on the second image. For instance, if the second image is a CT image, the second coordinate system corresponding to the second image refers to a coordinate system constructed based on the CT image, which can be represented as the CT coordinate system.

[0089] Therefore, when the first image is an MRI image, the purpose of obtaining the first pose of the first bone structure in the first coordinate system corresponding to the first image is to obtain the first pose of the first bone structure in the MRI coordinate system. The aforementioned first pose can refer to a known or prior pose of the first bone structure in the MRI coordinate system. This first pose can originate from manual registration results, the results of other known registration processes, or the optimization results of previous steps in the system. This application does not limit the specific method for obtaining the first pose of the first bone structure in the first coordinate system corresponding to the first image.

[0090] S102. Based on the positional relationship between the first and second bone structures in the second image, construct a joint kinematic model.

[0091] In this embodiment, a joint kinematic model can be constructed based on the positional relationship between the first and second bone structures in the second image. This kinematic model represents the pose of the second bone structure as a function of the pose of the first bone structure and joint parameters. The aforementioned positional relationship can refer to the relative anatomical relationship between the bone structures.

[0092] To distinguish them, in this embodiment, the pose of the first bone structure is represented as the first pose, and the pose of the second bone structure is represented as the second pose. Therefore, by constructing a joint kinematic model, the second pose of the second bone structure can be represented as a function of the first pose of the first bone structure and joint parameters.

[0093] In other words, taking the second image as a CT image as an example, a joint kinematic model can be constructed based on the relative anatomical relationship between the first and second bone structures in the CT image, thereby establishing the transformation relationship between the first and second bone structures.

[0094] In one possible implementation of this application embodiment, taking the knee joint application scenario as an example, the joint kinematic model can be represented as:

[0095]

[0096] in: This represents the unknown pose of the second bone structure in the MRI coordinate system; This indicates the known pose of the first bone structure in the MRI coordinate system; This indicates the initial relative joint pose from the first bone structure to the second bone structure in the CT coordinate system; Indicated by joint motion parameters The determined relative motion transformation; This is a vector of joint parameters.

[0097] Therefore, when constructing a joint kinematic model, the initial relative joint pose from the first bone structure to the second bone structure can be determined first in the second coordinate system corresponding to the second image, i.e., the MRI coordinate system. Then, by determining multiple joint parameters, the relative motion transformation from the second bone structure to the first bone structure, determined by these parameters, can be constructed. Based on this, a joint kinematic model can be constructed according to the aforementioned initial relative joint pose and relative motion transformation.

[0098] In one possible implementation of this application embodiment, the joint parameters may include the following parameters:

[0099] 1. Flexion-extension motion parameters, i.e., flexion-extension angles;

[0100] 2. Internal and external rotation motion parameters, i.e., internal and external rotation angles;

[0101] 3. Inversion / exversion movement parameters, i.e., inversion / exversion angles;

[0102] 4. Small translation compensation parameters, i.e. translation amount.

[0103] Among them, the small translation compensation parameter can be used as an option.

[0104] Therefore, the joint parameter vector in the aforementioned example can be defined as:

[0105]

[0106] in, The angle of flexion and extension; Inward and outward rotation angles; For both inner and outer corners; This is a translation compensation item.

[0107] In this embodiment, to reflect the physiological rationality of joint connection, reasonable constraint ranges can be pre-set for the aforementioned joint parameters. These constraint ranges can be set according to joint type, patient position, or clinically preset parameters. For example, in the knee joint, the flexion-extension angle can be limited to a preset angle range, the internal and external rotation angles and the varus-valgus angle can be limited to a smaller angle range, and the translation compensation term can be limited to a millimeter-level range. By limiting the range of joint parameters as described above, the solution of the second bone structure pose can be transformed from a six-degree-of-freedom rigid body search problem into a low-dimensional joint parameter optimization problem, thereby improving the computational speed in the registration process.

[0108] In one possible implementation of this application embodiment, the relative motion transformation from the second bone structure to the first bone structure can be achieved based on an established joint local coordinate system.

[0109] Specifically, for the second image, the computer device can extract three-dimensional models of the first and second bone structures from the second image. These three-dimensional models can be triangular mesh models, point cloud models, or implicit surface models; this embodiment does not limit the specific model. Based on this, a joint local coordinate system can be established according to the three-dimensional models of the first and second bone structures. The Z-axis of the joint local coordinate system can be determined based on the major axis of the first bone structure.

[0110] For example, taking a CT image as the second image, the CT coordinate system can be considered the standard coordinate system. By applying principal component analysis (PCA) to the three-dimensional model of the first bone structure, the major axis of the first bone structure was obtained, and this major axis can be identified as... Then, matrix multiplication is used to... In Align to To obtain the local coordinate system In this way, a relatively accurate joint coordinate system can be constructed without the use of landmarks.

[0111] Based on the aforementioned determined joint parameters, multiple rotation matrices can be determined by rotating the second bone structure around each axis of the joint's local coordinate system. Therefore, based on these multiple rotation matrices, a representation of the relative motion transformation from the second bone structure to the first bone structure can be constructed. This is illustrated in the aforementioned example. .

[0112] S103. Generate candidate poses of the second bone structure in the first coordinate system based on the joint kinematic model.

[0113] In this embodiment of the application, the candidate pose can be the pose of the second bone structure in a first coordinate system, such as the MRI coordinate system, calculated based on the corresponding candidate parameter values.

[0114] Specifically, based on the determination of multiple joint parameters, multiple candidate parameter values ​​can be obtained by combining the constraint range of the joint parameters. Any candidate parameter value can include a set of specific parameter values ​​of the joint parameters. For example, any candidate parameter value can include a set of values ​​consisting of flexion-extension angle, internal / external rotation angle, internal / external rotation angle, and translation compensation term as described in the previous example.

[0115] For any candidate parameter value of the joint parameter, the computer device can calculate the candidate pose of the second bone structure corresponding to the candidate parameter value based on the joint kinematic model. The candidate pose can refer to the pose directly calculated based on the corresponding candidate parameter value. Then, based on the candidate pose, the mesh vertices of the second bone structure can be transformed to the first coordinate system, i.e., the MRI coordinate system, thereby obtaining the candidate pose of the second bone structure in the MRI coordinate system.

[0116] S104. Establish inter-bone collision and contact constraints based on candidate poses.

[0117] In this embodiment, since the candidate pose is calculated based on the candidate parameter values ​​and is located in the coordinate system corresponding to the first image (i.e., the pose of the second bone structure in the MRI coordinate system), and the pose of the first bone structure in the MRI coordinate system has already been obtained in the aforementioned steps, collision detection and surface distance detection can be performed on the poses of the first and second bone structures in the MRI coordinate system to construct inter-bone collision and contact constraints. The collision detection can be used to determine whether the first and second bone structures collide when the second bone structure is in the corresponding candidate pose; the surface distance detection can determine the distance between the bone surfaces of the first and second bone structures.

[0118] Both the collision detection and surface distance detection described above can be implemented based on the bone surfaces of the first and second bone structures. Collision detection can be determined by detecting the penetration relationship between the bone surfaces of the first and second bone structures. When there is bone surface penetration between the first and second bone structures, it can be considered that a collision has occurred.

[0119] Therefore, when establishing inter-bone collision and contact constraints based on candidate poses, we can first calculate the penetration relationship and distance relationship between the first and second bone structures when the second bone structure is in the candidate pose, and then establish inter-bone collision and contact constraints between the first and second bone structures based on the penetration relationship and distance relationship.

[0120] Specifically, when there is bone surface penetration between the first bone structure and the second bone structure, and / or the surface distance between the first bone structure and the second bone structure exceeds a preset range, the aforementioned inter-bone collision and contact constraint terms can be assigned a penalty value.

[0121] In one possible implementation of this application embodiment, collision detection can be implemented using any of the following methods:

[0122] 1. Collision detection of triangular facets based on bounding box hierarchy;

[0123] 2. Penetration detection based on signed range field;

[0124] 3. Minimum surface distance detection based on nearest point search.

[0125] Specifically, this can be done for each set of candidate parameter values ​​for the joint parameters. Calculate the candidate pose of the second bone structure in the MRI coordinate system, and further calculate the minimum distance between the bone surfaces of the first and second bone structures. Average neighborhood distance between the two bone surfaces Penetration volume or penetration depth .

[0126] When there is bone surface penetration, the candidate parameter value can be penalized; when the surface distance between the first bone structure and the second bone structure exceeds the preset physiological range, the candidate parameter value can also be penalized.

[0127] As an example of an embodiment of this application, the above-mentioned interosseous collision and contact constraint terms can be expressed as:

[0128]

[0129] in, This indicates a penetrating penalty (interosseous impact penalty). This refers to a distance penalty that exceeds the range of physical contact.

[0130] In another possible implementation of this application embodiment, a weighting system can be introduced for the aforementioned penetration penalty and distance penalty, so that the aforementioned interbone collision and contact constraint terms can be further expressed as:

[0131]

[0132] in, These are the weighting coefficients for penetration penalty and distance penalty, respectively.

[0133] The embodiments of this application introduce inter-bone collision and contact constraints, which can exclude candidate poses of second bone structures that are mathematically feasible but anatomically unreasonable, thereby accelerating the calculation process of the registration process.

[0134] S105. Based on the candidate bone surface of the second bone structure in the candidate pose, construct an image response index to reflect the degree of bone interface matching.

[0135] In this embodiment, the candidate bone surface of the second bone structure in the candidate pose can refer to the bone surface of the second bone structure when it is in the candidate pose. Based on this candidate bone surface, an image response index can be constructed. This image response index can reflect the degree of matching between the second bone structure and the bone surface of the first bone structure in the coordinate system corresponding to the first image.

[0136] In one possible implementation of this application, the aforementioned image response index can be determined based on the response index of each sampling point. That is, the overall image response index can be obtained by calculating the local response index of each sampling point on the bone surface.

[0137] Specifically, when constructing an image response index to reflect the degree of bone interface matching, the candidate bone surface of the second bone structure can first be sampled to obtain a set of sampling points and a corresponding set of normal vectors. Then, for each sampling point in the set of sampling points, one-dimensional sampling of gray values ​​can be performed along the normal vector of that sampling point and the vector in the opposite direction of the normal vector to obtain a local gray-level sequence. For example, for sampling points... Along its normal vector and the opposite direction vector One-dimensional sampling is performed on the MRI volume data to obtain a local grayscale sequence. Based on this, the local response value of each sampling point can be calculated according to the local grayscale sequence. This local response value is the local response index of each sampling point in the aforementioned example, and it can represent the grayscale change characteristics of the corresponding sampling point. Based on the local response values ​​of multiple sampling points, an image response index reflecting the overall matching degree of the bone interface can be constructed. For example, the local response values ​​of all sampling points can be summarized into an overall image response index.

[0138] Therefore, the imaging response index reflecting the degree of bone interface matching can be expressed as:

[0139]

[0140] in, The number of sampling points; For the first Each bone surface sampling point at candidate parameter value The local response value under [condition].

[0141] In one possible implementation of this application embodiment, the local response value of each sampling point can be represented in one or more of the following forms:

[0142] 1. Maximum absolute value of the normal gradient;

[0143] 2. Normal grayscale jump amplitude;

[0144] 3. Local extreme response near the target interface.

[0145] S106. Based on the image response index and the interosseous collision and contact constraints, establish an objective function representing the second pose of the second bone structure.

[0146] In this embodiment, an objective function for solving the pose of the second bone structure can be established by combining image response metrics and interosseous collision and contact constraints. This objective function can be expressed as:

[0147]

[0148] in: This means that the problem of maximizing the image response is transformed into a problem of minimizing the objective function; To constrain collisions and contact between joint bones; This is a penalty term for parameters exceeding the bounds. This is an optional smoothing regularization term used to limit abnormal parameter changes or suppress numerical jitter.

[0149] After constructing the objective function described above, the registration of multimodal images is transformed into solving the objective function. Therefore, step S107 can be executed to obtain the registration results of the first and second bone structures in the first image by solving the objective function.

[0150] S107. The registration results of the first bone structure and the second bone structure in the first image are obtained by solving the objective function.

[0151] In this embodiment, to solve the objective function, the optimal joint parameters can first be determined. These optimal joint parameters are those that minimize the value of the objective function. Based on this, the second pose of the second bone structure in the first coordinate system can be calculated using the optimal joint parameters. Then, based on the first pose of the first bone structure in the first coordinate system and the second pose of the second bone structure in the first coordinate system, the registration result of the first and second bone structures in the first image can be determined.

[0152] In other words, taking the first image as an MRI image and the second image as a CT image as an example, based on the determination of the optimal joint parameters, the pose of the second bone structure in the MRI coordinate system can be calculated according to the optimal joint parameters. Then, based on the pose of the first bone structure and the second bone structure in the MRI coordinate system, the registration result of the first bone structure and the second bone structure in the MRI image can be determined.

[0153] In one possible implementation of this application embodiment, the optimal joint parameters can be determined using numerical optimization. For example, methods such as mesh search with local refinement, Powell's direction set method, gradient-free local optimization, and a combination of genetic algorithms and local optimization can be used to determine the optimal joint parameters.

[0154] As an example of an embodiment of this application, the optimization process for determining the optimal joint parameters described above can first obtain multiple candidate initial values ​​through discrete sampling, and then calculate the objective function values ​​corresponding to each candidate initial value. Multiple candidate parameters are then determined based on these multiple objective function values, and continuous optimization is performed using these candidate parameters as the starting point for local optimization until the optimal joint parameters are obtained.

[0155] Specifically, multiple objective function values ​​can be calculated from the candidate initial values ​​obtained through discrete sampling. Several optimal candidate parameters can be selected from these objective function values ​​as the starting point for local optimization until the stopping condition is met, thus obtaining the optimal joint parameters. The optimal objective function value calculated based on multiple candidate initial values ​​can be the minimum value among the calculated objective function values.

[0156] In this embodiment of the application, the stopping condition for continuous optimization may include the following conditions:

[0157] 1. The parameter update amount is less than the threshold;

[0158] 2. The change in the objective function between adjacent iterations is less than the threshold;

[0159] 3. The maximum number of iterations has been reached.

[0160] In this way, the final pose of the second bone structure in the MRI coordinate system (i.e., the coordinate system corresponding to the first image) can be calculated based on the obtained optimal joint parameters, thereby completing the registration of the bone structure in the first and second images.

[0161] It should be noted that the foregoing embodiments use two bone structures connected to the same joint as an example to introduce the image registration method for bone structures provided in this application. It should be understood that this method can also be applied to achieve fast and accurate registration for a larger number of bone structures connected to the same joint. For example, when more than two bone structures are connected to the same joint, this method can also be used for registration.

[0162] Based on the foregoing introduction, such as Figure 2 As shown, the image registration method for bone structures provided in this application embodiment can be summarized as including the following steps S201-S206. In Figure 2 In the illustrated embodiments, the MRI image is the same as the first image in the aforementioned embodiments, and the CT image is the same as the second image in the aforementioned embodiments. The method specifically includes:

[0163] Step S201: Obtain input data and build an initial model.

[0164] In this embodiment of the application, the input data may include the following data:

[0165] 1. CT three-dimensional image data, namely the CT images in the aforementioned embodiments;

[0166] 2. MRI three-dimensional image data, i.e., the MRI images in the aforementioned embodiments;

[0167] 3. The three-dimensional model of the first bone structure and the three-dimensional model of the second bone structure extracted from the CT three-dimensional image (initial model), that is, the three-dimensional model of the first bone structure and the second bone structure extracted from the second image as described in the above embodiments. The three-dimensional model can be a triangular mesh model, a point cloud model or an implicit surface model.

[0168] 4. The current pose of the first bone structure in the MRI coordinate system (first pose), which may be derived from manual registration results, the output of other known registration processes, or the optimization results of previous steps in the system.

[0169] The first and second bone structures mentioned above are adjacent bone structures connected by the same joint. For example, adjacent bone structures such as the femur and tibia, the humerus and radius and ulna, and the femur and patella.

[0170] Step S202: Construct a joint kinematic constraint model.

[0171] The joint kinematic constraint model in this application embodiment is the same as the joint kinematic model in the aforementioned embodiments.

[0172] In this embodiment of the application, a joint local coordinate system can be established in the CT coordinate system based on the three-dimensional model of the first bone structure and the three-dimensional model of the second bone structure, and the joint motion parameters of the second bone structure relative to the first bone structure can be defined.

[0173] Specifically, the CT coordinate system can be considered the standard coordinate system. Using the 3D model of the first bone structure and the PCA algorithm, the major axis of the first bone structure was obtained and identified as... Matrix multiplication can be used to... In Align to In this way, a local coordinate system can be obtained. Through the above processing, a relatively accurate joint coordinate system can be constructed without the landmark points.

[0174] The joint motion parameters in this application embodiment (i.e., the joint parameters in the aforementioned embodiments) include the following four types:

[0175] 1. Flexion and extension motion parameters;

[0176] 2. Internal and external rotation motion parameters;

[0177] 3. Inversion / exversion movement parameters;

[0178] 4. Small translation compensation parameters.

[0179] In knee joint applications, the pose of the second bone structure relative to the first bone structure can be represented as:

[0180]

[0181] in: This represents the unknown pose of the second bone structure in the MRI coordinate system; This indicates the known pose of the first bone structure in the MRI coordinate system; This indicates the initial relative joint pose from the first bone structure to the second bone structure in the CT coordinate system; This represents the relative motion transformation determined by the joint motion parameter q; q is the joint parameter vector.

[0182] The joint parameter vector mentioned above can be defined as:

[0183]

[0184] in: The angle of flexion and extension; Inward and outward rotation angles; For both inner and outer corners; This is a translation compensation item.

[0185] To reflect the physiological rationality of joint connections, embodiments of this application can set constraint ranges for joint parameters. These constraint ranges can be set according to joint type, patient position, or clinically preset parameters. For example, in the knee joint, the flexion-extension angle is limited to a preset angle range, the internal / external rotation angle and the varus / valgus angle are limited to a smaller angle range, and the translation compensation term is limited to a millimeter-level range. Through this method, the second bone structure pose solution can be transformed from a six-degree-of-freedom rigid body search problem into a low-dimensional joint parameter optimization problem.

[0186] Step S203: Establish interosseous collision and contact constraints.

[0187] In this embodiment, collision detection and surface distance detection can be performed in the MRI coordinate system based on the mesh surfaces of the first and second bone structures to construct inter-bone collision and contact constraints.

[0188] Collision detection in this application embodiment may include any of the following methods:

[0189] 1. Collision detection of triangular facets based on bounding box hierarchy;

[0190] 2. Penetration detection based on signed range field;

[0191] 3. Minimum surface distance detection based on nearest point search.

[0192] Specifically, for each set of candidate joint parameters q, the candidate pose of the second bone structure in the MRI coordinate system can be calculated, and the minimum distance between the two bone surfaces can be further calculated. Average neighborhood distance between the two bone surfaces Penetration volume or penetration depth .

[0193] When bone surface penetration occurs, the current candidate joint parameters can be penalized; when the surface distance exceeds the preset physiological range, the current candidate joint parameters can also be penalized.

[0194] The above collision and contact constraints can be expressed as:

[0195]

[0196] in, Indicates a penalty for penetration; This refers to a penalty imposed for exceeding the physical contact limit; These are the weighting coefficients.

[0197] The embodiments of this application introduce joint contact constraints to exclude candidate poses of second bone structures that are mathematically feasible but anatomically unreasonable.

[0198] Step S204: Construct MRI image response indices based on the surface of the second bone structure.

[0199] In this embodiment, the candidate surface of the second bone structure can be transformed into the MRI coordinate system, and the MRI image can be sampled in the normal direction of its surface points or surface sampling points to construct an image response index that reflects the degree of bone interface matching.

[0200] The specific steps for constructing image response metrics include:

[0201] 1. Sample the surface of the second bone structure to obtain a set of sampling points. and the corresponding set of normal vectors ;

[0202] 2. For each sampling point Along its normal vector and reverse One-dimensional sampling is performed on the MRI volume data to obtain a local grayscale sequence;

[0203] 3. Based on the common grayscale change characteristics of the bone cortex boundary, bone marrow boundary, or bone-soft tissue interface in MRI images, calculate the local response value of each sampling point;

[0204] 4. The local responses of all sampling points are summarized into an overall image matching index, i.e., an image response index.

[0205] The local response values ​​in the embodiments of this application may take one or more of the following forms:

[0206] 1. Maximum absolute value of the normal gradient;

[0207] 2. Normal grayscale jump amplitude;

[0208] 3. Local extreme response near the target interface;

[0209] As an example, the overall image response index can be expressed as:

[0210]

[0211] For sampling points For the first Surface sampling points in candidate Local image response (local response value) under the condition.

[0212] In this embodiment, the sampling points can be weighted according to surface curvature, regional uniformity, or joint-related regions to reduce the impact of noisy regions or poorly imaged regions on the overall objective function.

[0213] Step S205: Construct the objective function and optimize the parameters.

[0214] By combining image response metrics and interosseous collision and contact constraints, an objective function can be established to solve for the pose of the second bone structure. This objective function can be expressed as:

[0215]

[0216] This means that the problem of maximizing the image response is transformed into a problem of minimizing the objective function; This refers to joint contact constraints. This is a penalty term for parameters exceeding the bounds. This is an optional smoothing regularization term used to limit abnormal parameter changes or suppress numerical jitter.

[0217] In this embodiment, the optimization process for the objective function described above can employ traditional numerical optimization methods, including but not limited to:

[0218] Grid search with local refinement; Powell's direction set method; gradient-free local optimization method; method combining genetic algorithm and local optimization, etc.

[0219] The specific optimization process is as follows:

[0220] 1. Perform coarse-grained discrete sampling based on the physiological range of joint parameters to obtain several candidate initial values;

[0221] 2. Calculate the objective function value for each candidate initial value;

[0222] 3. Select several candidate joint parameters with the optimal objective function values ​​as the starting point for local optimization;

[0223] 4. Perform continuous optimization on the local starting point until the stopping condition is met;

[0224] 5. Select the joint parameter that minimizes the final objective function. Optimal joint parameters for the second bone structure.

[0225] The stopping conditions in the above optimization process may include:

[0226] 1. The parameter update amount is less than the threshold;

[0227] 2. The change in the objective function between adjacent iterations is less than the threshold;

[0228] 3. The maximum number of iterations has been reached.

[0229] In this way, ultimately, according to Calculate the final pose of the second bone structure in the MRI coordinate system. .

[0230] Step S206: Output the registration result.

[0231] After performing the aforementioned steps S201 to S205, the output results may include:

[0232] 1. Rigid body pose of the second bone structure in the MRI coordinate system;

[0233] 2. Joint registration results of the first and second bone structures in MRI images;

[0234] 3. Optimal solution for joint parameters (i.e., optimal joint parameters);

[0235] 4. Image response index, collision detection results, and objective function value, etc.

[0236] This completes the image registration of the multi-bone structure across modalities.

[0237] This application embodiment introduces joint kinematic constraints, which can transform the pose solution of the second bone structure from six-degree-of-freedom rigid body optimization to low-dimensional parameter optimization, thus helping to accelerate the optimization solution and improve the registration efficiency.

[0238] Secondly, existing multi-bone registration methods typically require solving each bone structure as an independent rigid body, necessitating a search within a large parameter space, resulting in low efficiency. This application's embodiment, by adding a "joint kinematic modeling" step, represents the candidate pose of the second bone structure as a function of the pose of the first bone structure and a small number of joint parameters. This significantly reduces the search space and the number of unreasonable candidate poses in the solution space, enabling automatic solving of another bone structure without relying on markers, automatic segmentation, or deep learning models.

[0239] Third, in existing technologies, without automatic segmentation or manual point matching, the pose of another bone structure is often difficult to obtain reliably. The embodiments of this application, through joint modeling of "interosseous joint relationship + MRI surface response + contact constraint", enable the second bone structure to be solved without additional manual point selection, reducing the dependence on manual interaction and complex training data; through interosseous collision detection and contact constraint, it helps to improve the anatomical rationality of candidate poses.

[0240] Fourth, the embodiments of this application add a "bone collision and contact constraint" step, which punishes candidate poses that have bone surface penetration or bone distance that significantly exceeds the physiological range, thereby eliminating solutions that are anatomically unreasonable and reducing the probability of erroneous registration results.

[0241] Fifth, compared with the prior art which uses intensity similarity of the entire image for registration, the embodiments of this application utilize the geometric information of the bone surface of the second bone structure, perform local sampling only within the neighborhood of the bone surface, and construct response indicators based on normal grayscale changes, reducing the interference of irrelevant regions on the objective function, and are more suitable for cross-modal matching between CT bone models and MRI bone interfaces.

[0242] To facilitate understanding, the image registration method for bone structures provided in this application embodiment will be described below with specific examples. In this example, the first image is an MRI image, the second image is a CT image, the first bone structure is the femur, and the second bone structure is the tibia. That is, this example applies the registration method described above to register CT images and MRI images containing the femur and tibia.

[0243] In this example, the input data includes:

[0244] 1. CT scan data of the patient's knee joint area;

[0245] 2. MRI volumetric data of the corresponding region from the same patient;

[0246] 3. Femoral triangular mesh model extracted from CT images;

[0247] 4. Tibial triangular mesh model extracted from CT images;

[0248] 5. The pose matrix of the registered femur in the MRI coordinate system, i.e., the first pose in the aforementioned embodiments.

[0249] Below, we will combine Figure 3 The registration process shown provides a detailed introduction to the registration of CT and MRI images containing the femur and tibia. Specifically, in... Figure 3 The diagram illustrates the overall process from inputting CT images, MRI images, and the pose of the first bone structure, to establishing a joint kinematic model, constructing an objective function, optimizing joint parameters, and outputting the pose of the second bone structure. Specifically:

[0250] Step 1: Data Preparation

[0251] 1.1 Read CT volume data and MRI volume data, which can be DICOM sequence data;

[0252] 1.2 Extract the femoral and tibial surface mesh models from CT scan data. The mesh data structure is a vertex array V={ } and triangular facet index array ;

[0253] 1.3 Obtaining the pose matrix of the femur in the MRI coordinate system ;

[0254] 1.4 A trilinear interpolation accessor is established for the MRI volume data to enable subsequent continuous grayscale sampling along the normal direction.

[0255] Step 2: Establish the local coordinate system of the knee joint

[0256] 2.1 Calculate the femoral long axis in the CT coordinate system and treat it as a local coordinate system. Axis, using the CT coordinate system In Axis, pair After rotation, the local coordinate system is obtained. ;

[0257] The aforementioned local coordinate system includes: the principal axis of flexion and extension; the medial and lateral directions; and the proximal and distal directions.

[0258] 2.2 Initial relative transformation from femur to tibia in CT body data As the reference relative pose of the second bone structure, and define the joint parameter vector:

[0259]

[0260] in: The angle of flexion and extension; Inward and outward rotation angles; For both inner and outer corners.

[0261] In this example, independent translation compensation parameters are not introduced, or the translation is limited to a preset minimum range as an option.

[0262] See Figure 4 The diagram illustrates the modeling process of a joint kinematic model provided in an embodiment of this application. Figure 4 The example shown includes establishing a local coordinate system for the knee joint and performing joint kinematic modeling based on this local coordinate system. Specifically, Figure 4 This illustrates the initial relative positions of the first and second bone structures in the CT coordinate system, the local joint coordinate system, and the definition of joint parameters. Specifically, the initial coordinate system is generated based on the CT image, and the Z-axis of the skeletal coordinate system is generated from the major axis obtained by PCA. The skeletal coordinate system is generated by aligning the Z-axis of the CT coordinate system to the Z-axis of the skeletal coordinate system. Figure 4 In the figure, (a) represents the CT coordinate system. Figure 4 In the diagram, (b) represents the skeletal coordinate system. Figure 4 (c) in the figure represents constrained joint movement.

[0263] Step 3: Generate candidate tibial poses based on joint parameters

[0264] 3.1 For any candidate joint parameter Construct the incremental transformation:

[0265]

[0266] in, These represent the rotation matrices obtained by rotating the tibia around the corresponding axes of the local coordinate system of the joint;

[0267] 3.2 Calculate candidate tibial poses:

[0268]

[0269] 3.3 Transform the tibial mesh vertices to the MRI coordinate system:

[0270]

[0271] in,

[0272] Step 4: Calculate interosseous collision and contact constraints

[0273] 4.1 Construct bounding box hierarchical structures for the femoral and tibial meshes respectively;

[0274] 4.2 Record the initial point cloud distance between the femur and tibia, and set the allowable range of contact distance. ;

[0275] 4.3 Collision detection is performed between the tibial candidate pose mesh and the femoral mesh;

[0276] 4.4 If there are penetrating triangular faces, record the number of penetrating triangular faces, the penetration depth, or the estimated penetration volume;

[0277] 4.5 For cases with no penetration or slight contact, calculate the distribution of the closest point distance between the tibial plateau surface and the distal femoral surface; such as Figure 5 The diagram shown illustrates a collision detection and contact constraint method according to an embodiment of this application. Figure 5 As shown, CT data can record the initial state. The distance value D obtained by detection can be compared with the above value to confirm whether there is excessive separation on the bone surface.

[0278] 4.6 Constructing the penalty function:

[0279] a) When the nearest distance is less than If an abnormal squeezing or penetration trend is detected, a penalty will be imposed.

[0280] b) When the nearest distance is greater than When a joint is deemed to be outside the normal contact range, a punishment is imposed.

[0281] The following contact cost function can be used:

[0282]

[0283] in: Penalty for penetration; The surface distance of the k-th contact sampling point; This is a piecewise penalty function.

[0284] Step 5: Calculate MRI image response indices

[0285] 5.1 A set of sampling points was obtained by uniformly sampling the tibial grid surface { The number of samples is twice the number of points in the model point cloud;

[0286] 5.2 Calculate the surface normal vector for each sampling point { };

[0287] 5.3 For each sampling point, perform one-dimensional discrete sampling along the normal direction. The sampling position can be represented as:

[0288]

[0289] The sampling sequence number; This is the sampling step size.

[0290] like Figure 6 The diagram shown is a schematic diagram of sampling along the normal direction of the bone surface of a second bone structure according to an embodiment of this application. Figure 6 Point 601 shown is a sampling point, and the normal vector at this sampling point is as follows: Figure 6 The vector 602 in the diagram is shown.

[0291] 5.4 The gray values ​​at each sampling location are read using an MRI interpolation accessor to form a local gray value sequence;

[0292] 5.5 Calculate the local boundary response values, for example, using the maximum normal gradient:

[0293]

[0294] in, Indicates MRI grayscale value;

[0295] 5.6 Calculate the average, weighted average, or quantile statistics of all local response values ​​to obtain the overall image response index:

[0296]

[0297] in, The sampling point weight can be set based on the importance of the surface region, local curvature, or normal stability.

[0298] like Figure 7 The diagram shown is a schematic diagram of MRI volume data sampling provided in an embodiment of this application. Figure 7 The superimposed display effect of the second bone structure in the MRI image is shown. The green line 701 represents the bone surface, and the computer device can sample the pixel intensity of the MRI along the normal vector.

[0299] Step 6: Construct the objective function

[0300] 6.1 Set parameter range penalty items:

[0301]

[0302] in This is the preset feasible domain (range of joint parameters).

[0303] 6.2 Constructing the overall objective function:

[0304]

[0305] The objective is to solve:

[0306]

[0307] Step 7: Optimize the solution

[0308] like Figure 8 The diagram shown is a schematic representation of an optimization process for an objective function provided in an embodiment of this application. Figure 8 The process shown can include optimizing the objective function as follows:

[0309] 7.1 Perform a coarse search within the parameter range for the flexion / extension angle, internal / external rotation angle, and internal / external flip angle to form an initial set of candidate parameters;

[0310] 7.2 Calculate the objective function value for each set of candidate parameters;

[0311] 7.3 Select the first few optimal candidate parameters as initial values ​​for local optimization;

[0312] 7.4 Local continuous optimization is performed using the Powell method;

[0313] 7.5 Stop optimization when any of the following conditions are met:

[0314] a. The parameter changes are below the threshold for two consecutive iterations;

[0315] b. The objective function changes below a threshold in two consecutive iterations;

[0316] c. The number of iterations reaches its maximum value.

[0317] 7.6 Output the optimal parameter q* and the corresponding tibial pose. .

[0318] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0319] Reference Figure 9 This diagram illustrates an image registration device for bone structures provided in an embodiment of this application. Specifically, it may include a first pose acquisition module 901, a kinematic model construction module 902, a candidate pose generation module 903, a constraint term construction module 904, an image response index construction module 905, an objective function establishment module 906, and a registration solution module 907, wherein:

[0320] The first pose acquisition module 901 is used to acquire the first pose of the first bone structure in the first coordinate system corresponding to the first image.

[0321] The kinematic model construction module 902 is used to construct a joint kinematic model based on the positional relationship between the first bone structure and the second bone structure in the second image. The joint kinematic model is used to represent the second pose of the second bone structure as a function of the first pose of the first bone structure and joint parameters. The first image and the second image both include the first bone structure and the second bone structure, and the first bone structure and the second bone structure are connected by the same joint.

[0322] The candidate pose generation module 903 is used to generate candidate poses of the second bone structure in the first coordinate system based on the joint kinematics model.

[0323] The constraint construction module 904 is used to establish inter-bone collision and contact constraints based on the candidate pose.

[0324] The image response index construction module 905 is used to construct an image response index that reflects the degree of bone interface matching based on the candidate bone surface of the second bone structure in the candidate pose.

[0325] The objective function establishment module 906 is used to establish an objective function representing the second pose of the second bone structure based on the image response index and the interosseous collision and contact constraint terms.

[0326] The registration module 907 is used to obtain the registration results of the first bone structure and the second bone structure in the first image by solving the objective function.

[0327] In this embodiment of the application, the kinematic model construction module 902 can be specifically used for:

[0328] Determine the initial joint relative pose from the first bone structure to the second bone structure in the second coordinate system corresponding to the second image;

[0329] Multiple joint parameters are determined, and a relative motion transformation from the second bone structure to the first bone structure is constructed based on the multiple joint parameters.

[0330] Based on the initial joint relative pose and the relative motion transformation, a joint kinematic model is constructed;

[0331] The joint parameters include at least the flexion-extension angle, internal and external rotation angle, and internal and external rotation angle.

[0332] In one possible implementation of this application embodiment, the kinematic model construction module 902 may also be used for:

[0333] Extract three-dimensional models of the first bone structure and the second bone structure from the second image;

[0334] A joint local coordinate system is established based on the three-dimensional models of the first bone structure and the second bone structure; wherein, the Z-axis of the joint local coordinate system is determined based on the major axis of the first bone structure;

[0335] Based on the multiple joint parameters, determine multiple rotation matrices obtained by rotating the second bone structure around each axis of the joint local coordinate system;

[0336] The relative motion transformation from the second bone structure to the first bone structure is constructed based on multiple rotation matrices.

[0337] In this embodiment of the application, the candidate pose generation module 903 can be specifically used for:

[0338] For any candidate parameter value of the joint parameter, calculate the candidate pose of the second bone structure corresponding to the candidate parameter value based on the joint kinematic model;

[0339] Based on the candidate pose, the mesh vertices of the second bone structure are transformed into the first coordinate system.

[0340] In this embodiment of the application, the constraint construction module 904 can be specifically used for:

[0341] Calculate the penetration relationship and distance relationship between the first bone structure and the second bone structure when the second bone structure is in the candidate pose;

[0342] Based on the penetration relationship and the distance relationship, establish interosseous collision and contact constraints between the first bone structure and the second bone structure;

[0343] Wherein, when there is bone surface penetration between the first bone structure and the second bone structure, and / or the surface distance between the first bone structure and the second bone structure exceeds a preset range, the inter-bone collision and contact constraint term is given a penalty value.

[0344] In this embodiment of the application, the image response index construction module 905 can be specifically used for:

[0345] The bone surface of the second bone structure is sampled to obtain a set of sampling points and a corresponding set of normal vectors;

[0346] For each sampling point in the set of sampling points, one-dimensional sampling of gray values ​​is performed along the normal vector of the sampling point and the vector in the opposite direction of the normal vector to obtain a local gray value sequence;

[0347] Based on the local grayscale sequence, the local response value of each sampling point is calculated, and the local response value is used to represent the grayscale change characteristics of the corresponding sampling point.

[0348] Based on the local response values ​​of multiple sampling points, an image response index reflecting the overall matching degree of the bone interface is constructed.

[0349] In this embodiment of the application, the registration solving module 907 can be specifically used for:

[0350] Determine the optimal joint parameters, which are the joint parameters that minimize the objective function value;

[0351] The second pose of the second bone structure in the first coordinate system is calculated based on the optimal joint parameters.

[0352] Based on the first pose of the first bone structure in the first coordinate system and the second pose of the second bone structure in the first coordinate system, the registration result of the first bone structure and the second bone structure in the first image is determined.

[0353] In one possible implementation of this application embodiment, the registration solving module 907 can also be used for:

[0354] Multiple candidate initial values ​​are obtained through discrete sampling, and the objective function values ​​corresponding to each candidate initial value are calculated.

[0355] Multiple candidate parameters are determined based on the objective function values, and continuous optimization is performed using the candidate parameters as the starting point for local optimization to obtain the optimal joint parameters.

[0356] This application provides an image registration device for bone structures. This device can be a computer device or one or more modules, components, or units within a computer device. Using this device, the steps in the aforementioned method embodiments can be implemented.

[0357] As the apparatus embodiments are basically similar to the method embodiments, they are described in a relatively simple manner. For relevant details, please refer to the description in the method embodiment section.

[0358] Reference Figure 10 The diagram illustrates a computer device provided in an embodiment of this application. Figure 10 As shown, the computer device 1000 in this embodiment includes: a processor 1010, a memory 1020, and a computer program 1021 stored in the memory 1020 and executable on the processor 1010. When the processor 1010 executes the computer program 1021, it implements the steps in various embodiments of the above-described image registration method for bone structures, for example... Figure 1 The steps S101 to S107 are shown. Alternatively, when the processor 1010 executes the computer program 1021, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 9 The functions of modules 901 to 907 are shown.

[0359] For example, the computer program 1021 can be divided into one or more modules / units, which are stored in the memory 1020 and executed by the processor 1010 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which can be used to describe the execution process of the computer program 1021 in the computer device 1000. For example, the computer program 1021 can be divided into a first pose acquisition module, a kinematic model construction module, a candidate pose generation module, a constraint term construction module, an image response index construction module, an objective function establishment module, and a registration solution module. The specific functions of each module are as follows:

[0360] The first pose acquisition module is used to acquire the first pose of the first bone structure in the first coordinate system corresponding to the first image.

[0361] The kinematic model construction module is used to construct a joint kinematic model based on the positional relationship between the first bone structure and the second bone structure in the second image. The joint kinematic model is used to represent the second pose of the second bone structure as a function of the first pose of the first bone structure and joint parameters. The first image and the second image both include the first bone structure and the second bone structure, and the first bone structure and the second bone structure are connected by the same joint.

[0362] The candidate pose generation module is used to generate candidate poses of the second bone structure in the first coordinate system based on the joint kinematic model.

[0363] A constraint construction module is used to establish inter-bone collision and contact constraints based on the candidate pose.

[0364] The image response index construction module is used to construct an image response index that reflects the degree of bone interface matching based on the candidate bone surface of the second bone structure in the candidate pose.

[0365] The objective function establishment module is used to establish an objective function representing the second pose of the second bone structure based on the image response index and the interosseous collision and contact constraint terms.

[0366] The registration module is used to obtain the registration results of the first bone structure and the second bone structure in the first image by solving the objective function.

[0367] The computer device 1000 may be a device capable of implementing the relevant methods or functions in the foregoing method embodiments. The computer device 1000 may be a desktop computer, a cloud server, or other similar devices. The computer device 1000 may include, but is not limited to, a processor 1010 and a memory 1020. Those skilled in the art will understand that... Figure 10 This is merely one example of computer device 1000 and does not constitute a limitation on computer device 1000. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device 1000 may also include input / output devices, network access devices, buses, etc.

[0368] The processor 1010 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0369] The memory 1020 may be an internal storage unit of the computer device 1000, such as a hard disk or memory of the computer device 1000. The memory 1020 may also be an external storage device of the computer device 1000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 1000. Furthermore, the memory 1020 may include both internal storage units and external storage devices of the computer device 1000. The memory 1020 is used to store the computer program 1021 and other programs and data required by the computer device 1000. The memory 1020 can also be used to temporarily store data that has been output or will be output.

[0370] This application also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the methods described in the foregoing embodiments.

[0371] This application also discloses a computer-readable storage medium storing a computer program that, when executed by a computer, implements the methods described in the foregoing embodiments.

[0372] This application also discloses a computer program product, including a computer program that, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.

[0373] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for image registration of bone structures, characterized in that, include: Obtain the first pose of the first bone structure in the first coordinate system corresponding to the first image; Based on the positional relationship between the first bone structure and the second bone structure in the second image, a joint kinematic model is constructed. The joint kinematic model is used to represent the second pose of the second bone structure as a function of the first pose of the first bone structure and joint parameters. The first image and the second image both include the first bone structure and the second bone structure, and the first bone structure and the second bone structure are connected by the same joint. Based on the joint kinematic model, candidate poses of the second bone structure in the first coordinate system are generated; Based on the candidate pose, interbone collision and contact constraints are established; and based on the candidate bone surface of the second bone structure in the candidate pose, an image response index is constructed to reflect the degree of bone interface matching. Based on the image response index and the interosseous collision and contact constraints, an objective function representing the second pose of the second bone structure is established. The registration results of the first bone structure and the second bone structure in the first image are obtained by solving the objective function.

2. The method according to claim 1, characterized in that, The construction of a joint kinematic model based on the positional relationship between the first and second bone structures in the second image includes: Determine the initial joint relative pose from the first bone structure to the second bone structure in the second coordinate system corresponding to the second image; Multiple joint parameters are determined, and a relative motion transformation from the second bone structure to the first bone structure is constructed based on the multiple joint parameters. Based on the initial joint relative pose and the relative motion transformation, a joint kinematic model is constructed; The joint parameters include at least the flexion-extension angle, internal and external rotation angle, and internal and external rotation angle.

3. The method according to claim 2, characterized in that, The construction, determined by a plurality of joint parameters, involves a relative motion transformation from the second bone structure to the first bone structure, including: Extract three-dimensional models of the first bone structure and the second bone structure from the second image; A joint local coordinate system is established based on the three-dimensional models of the first bone structure and the second bone structure; wherein, the Z-axis of the joint local coordinate system is determined based on the major axis of the first bone structure; Based on the multiple joint parameters, determine multiple rotation matrices obtained by rotating the second bone structure around each axis of the joint local coordinate system; Based on multiple rotation matrices, a representation of the relative motion transformation from the second bone structure to the first bone structure is constructed.

4. The method according to claim 1, characterized in that, The step of generating candidate poses of the second bone structure in the first coordinate system based on the joint kinematic model includes: For any candidate parameter value of the joint parameter, calculate the candidate pose of the second bone structure corresponding to the candidate parameter value based on the joint kinematic model; Based on the candidate pose, the mesh vertices of the second bone structure are transformed into the first coordinate system.

5. The method according to any one of claims 1 to 4, characterized in that, The process of establishing interosseous collision and contact constraints based on the candidate pose includes: Calculate the penetration relationship and distance relationship between the first bone structure and the second bone structure when the second bone structure is in the candidate pose; Based on the penetration relationship and the distance relationship, establish interosseous collision and contact constraints between the first bone structure and the second bone structure; Wherein, when there is bone surface penetration between the first bone structure and the second bone structure, and / or the surface distance between the first bone structure and the second bone structure exceeds a preset range, the inter-bone collision and contact constraint term is given a penalty value.

6. The method according to claim 5, characterized in that, The construction of an image response index reflecting the degree of bone interface matching based on the candidate bone surface of the second bone structure in the candidate pose includes: The bone surface of the second bone structure is sampled to obtain a set of sampling points and a corresponding set of normal vectors; For each sampling point in the set of sampling points, one-dimensional sampling of gray values ​​is performed along the normal vector of the sampling point and the vector in the opposite direction of the normal vector to obtain a local gray value sequence; Based on the local grayscale sequence, the local response value of each sampling point is calculated, and the local response value is used to represent the grayscale change characteristics of the corresponding sampling point. Based on the local response values ​​of multiple sampling points, an image response index reflecting the overall matching degree of the bone interface is constructed.

7. The method according to any one of claims 1 to 4 or 6, characterized in that, The process of obtaining the registration results of the first bone structure and the second bone structure in the first image by solving the objective function includes: Determine the optimal joint parameters, which are the joint parameters that minimize the objective function value; The second pose of the second bone structure in the first coordinate system is calculated based on the optimal joint parameters. Based on the first pose of the first bone structure in the first coordinate system and the second pose of the second bone structure in the first coordinate system, the registration result of the first bone structure and the second bone structure in the first image is determined.

8. The method according to claim 7, characterized in that, Determining the optimal joint parameters includes: Multiple candidate initial values ​​are obtained through discrete sampling, and the objective function values ​​corresponding to each candidate initial value are calculated. Multiple candidate parameters are determined based on the objective function values, and continuous optimization is performed using the candidate parameters as the starting point for local optimization to obtain the optimal joint parameters.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the computer device to implement the method as described in any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is run, the method as described in any one of claims 1 to 8 is performed.

Images