Bone landmark detection method and device based on image registration

Abstract

The present invention relates to the technical field of image registration, and provides a method and device for detecting bone landmarks based on image registration. The method comprises: determining the centroid coordinates of a scapula mask based on a bone joint image to be measured; aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured based on the centroid offset between the scapula segmentation mask and the scapula mask in the annotation template, and transforming the coordinates of each landmark in the annotation template to obtain an aligned annotation template; respectively registering the scapula segmentation region with the scapula region, and the humeral segmentation region with the humeral region in the aligned annotation template to obtain a registration result; and spatially transforming the coordinates of each landmark in the aligned annotation template based on the registration result to obtain target coordinates for navigation and planning of shoulder replacement surgery. The method of the present invention improves the efficiency and accuracy of bone landmark detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image registration, and in particular to a bone landmark detection method and device based on image registration. Background Art

[0002] Shoulder replacement surgery is a procedure that implants an artificial prosthesis for patients with shoulder joint injuries. Accurate detection and positioning of bony landmarks in the shoulder joint are crucial for preoperative navigation and planning.

[0003] In related technologies, three-dimensional reconstruction is usually performed on preoperative or intraoperative medical images first, and then the anatomical information is transferred to the point cloud data of the human body or key parts of the human body through a registration algorithm based on image grayscale values ​​to achieve shoulder joint landmark positioning. This method has strict requirements on the initial registration conditions. If the initial registration is inaccurate, the accuracy of the registration results will be reduced, thereby reducing the point accuracy of the shoulder joint landmarks. At the same time, the algorithm is easily affected by image noise and outliers when calculating the image grayscale values, which will also lead to inaccurate registration results. In addition, the existing technology also uses deep learning methods to automatically learn key features in the image and use them for bony landmark detection. Since this method requires a large amount of labeled data for model training, it takes a lot of time and cost, and the decision-making process during model training lacks transparency, resulting in low reliability of the model output shoulder joint detection results. Summary of the Invention

[0004] The present invention provides a bone landmark detection method and device based on image registration, which is used to solve the problems in the existing technology that when shoulder joint landmarks are located by a registration algorithm based on image grayscale values, they are easily affected by initial registration inaccuracy or image noise and outliers, resulting in inaccurate final registration results. When bone landmarks are detected by a deep learning method, data labeling is time-consuming and the decision-making process during model training is opaque, resulting in inefficient bone landmark detection and inaccurate detection results. The method improves the efficiency and accuracy of bone landmark detection.

[0005] The present invention provides a bone landmark detection method based on image registration, comprising:

[0006] Determining the centroid coordinates of the scapula mask based on the bone and joint image to be measured; wherein the bone and joint image to be measured includes the scapula region, the scapula mask, the humeral region, and the humeral mask;

[0007] A center of mass offset is obtained according to the center of mass coordinates of the scapula segmentation mask in the annotation template and the center of mass coordinates of the scapula mask; the origin coordinates of the annotation template are aligned with the origin coordinates of the bone joint image to be measured according to the center of mass offset, and the coordinates of each landmark point in the annotation template are converted to obtain an aligned annotation template; wherein the annotation template includes a scapula segmentation area, a scapula segmentation mask, a humerus segmentation area and a humerus segmentation mask; the humerus segmentation area includes a plurality of humeral landmark points, and the scapula segmentation area includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates;

[0008] The scapula segmentation area in the aligned annotation template is aligned with the scapula area to obtain a scapula alignment result, and the humerus segmentation area in the aligned annotation template is aligned with the humerus area to obtain a humerus alignment result; the coordinates of each landmark point in the aligned annotation template are spatially transformed according to the scapula alignment result and the humerus alignment result to obtain target coordinates for navigation and planning of shoulder replacement surgery.

[0009] According to a bone landmark detection method based on image registration provided by the present invention, the annotation template is obtained through the following steps:

[0010] acquiring a plurality of sets of template images by scanning the target subject from the neck to the chest on the left side of the body, from the neck to the chest on the right side of the body, from the neck to the waist on the left side of the body, and from the neck to the waist on the right side of the body using a computed tomography (CT) device;

[0011] For each set of template images, a humeral segmentation mask and a scapula segmentation mask are annotated from the template images based on expert experience, and the landmark points and coordinates to be identified are marked on the humerus, and the landmark points and coordinates to be identified are marked on the scapula to obtain the annotated template.

[0012] According to a bone landmark detection method based on image registration provided by the present invention, registering the scapula segmentation region in the aligned annotation template with the scapula region to obtain a scapula registration result includes:

[0013] Extracting a template scapula foreground region from the scapula segmentation region of the aligned annotation template; extracting a scapula foreground region to be measured from the scapula region;

[0014] Performing an affine transformation on the template scapula foreground region and the scapula foreground region to be measured to obtain a scapula affine transformation result, and performing a non-rigid transformation on the scapula affine transformation result and the scapula foreground region to be measured to obtain the scapula registration result;

[0015] The registering the humeral segmentation region in the aligned annotation template with the humeral region to obtain a humeral registration result includes:

[0016] Extracting a template humerus foreground region from the humerus segmentation region in the aligned annotation template, and extracting a humerus foreground region to be measured from the humerus region in the bone joint image to be measured;

[0017] Affine transformation is performed on the template humerus foreground area and the humerus foreground area to be measured to obtain a humerus affine transformation result; and non-rigid transformation registration is performed on the humerus affine transformation result and the humerus foreground area to be measured to obtain the humerus registration result.

[0018] According to a bone landmark detection method based on image registration provided by the present invention, the humerus registration result includes a humerus transformation matrix and a humerus deformation field; the scapula registration result includes a scapula transformation matrix and a scapula deformation field; the humerus transformation matrix and the humerus deformation field are obtained by sequentially applying affine transformation and non-rigid transformation to the humerus, and the scapula transformation matrix and the scapula deformation field are obtained by sequentially applying affine transformation and non-rigid transformation to the scapula;

[0019] The spatial transformation of the coordinates of each landmark point in the aligned annotation template according to the scapula registration result and the humerus registration result to obtain the target coordinates includes:

[0020] The coordinates of each landmark point on the humerus in the aligned annotation template are transformed according to the humeral transformation matrix to obtain the coordinates of the template humeral landmark points; the coordinates of each landmark point on the scapula in the aligned annotation template are transformed according to the scapula transformation matrix to obtain the coordinates of the template scapula landmark points;

[0021] The coordinates of the template humeral landmark points are transformed according to the humeral deformation field to obtain the coordinates of the target humeral landmark points; the coordinates of the template scapula landmark points are transformed according to the scapula deformation field to obtain the coordinates of the target scapula landmark points.

[0022] According to a bone landmark detection method based on image registration provided by the present invention, the scapula segmentation region includes the glenoid cavity, the humeral segmentation region includes the humeral head, humeral neck and humeral medullary cavity; the target coordinates include the target humeral landmark coordinates and the target scapula landmark coordinates;

[0023] After obtaining the target coordinates, the method further includes:

[0024] Calculating glenoid prosthesis characteristics based on the coordinates of the landmark points corresponding to the glenoid, and calculating humeral prosthesis characteristics based on the coordinates of the landmark points corresponding to the humeral head, humeral neck, and humeral medullary cavity; wherein the glenoid prosthesis characteristics include the size, model, and initial placement position of the glenoid prosthesis; and the humeral prosthesis characteristics include the size, model, and initial placement position of the humeral prosthesis;

[0025] estimating the postoperative offset of the scapula according to the glenoid prosthesis characteristics and the position of the scapula rotation center, and estimating the postoperative offset of the humerus according to the humeral prosthesis characteristics and the position of the greater tuberosity of the humerus;

[0026] Preoperative navigation of the shoulder replacement surgery is performed according to the postoperative offset of the scapula and the postoperative offset of the humerus.

[0027] According to a bone landmark detection method based on image registration provided by the present invention, after obtaining the target coordinates, the method further includes:

[0028] A bone surface model of a target object is determined based on the target coordinates, and when bone surface point cloud data of the target object is collected through bone surface surgery, the bone surface point cloud data is registered with the bone surface model to obtain a registration result:

[0029] The registration accuracy of each landmark point in the registration result is calculated based on the target coordinates to adjust one of the surgical navigation parameters or the positioning parameters of the robot-assisted system.

[0030] The present invention also provides a bone landmark detection device based on image registration, comprising:

[0031] A coordinate calculation module, configured to determine the centroid coordinates of the scapula mask based on the bone and joint image to be measured; wherein the bone and joint image to be measured includes the scapula region, the scapula mask, the humeral region, and the humeral mask;

[0032] An alignment module is configured to obtain a center of mass offset based on the center of mass coordinates of the scapula segmentation mask in the annotation template and the center of mass coordinates of the scapula mask; align the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured based on the center of mass offset, and convert the coordinates of each landmark point in the annotation template to obtain an aligned annotation template; wherein the annotation template includes a scapula segmentation region, a scapula segmentation mask, a humerus segmentation region, and a humerus segmentation mask; the humerus segmentation region includes a plurality of humeral landmark points, and the scapula segmentation region includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates;

[0033] A registration module is used to align the scapula segmentation area in the aligned annotation template with the scapula area to obtain a scapula registration result, and to align the humerus segmentation area in the aligned annotation template with the humerus area to obtain a humerus registration result; and to spatially transform the coordinates of each landmark point in the aligned annotation template according to the scapula registration result and the humerus registration result to obtain target coordinates for navigation and planning of shoulder replacement surgery.

[0034] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the bone landmark detection method based on image registration as described above is implemented.

[0035] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for detecting bone landmarks based on image registration.

[0036] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for detecting bone landmarks based on image registration.

[0037] The bone landmark point detection method and device based on image registration provided by the present invention calculate the center of mass offset through the center of mass coordinates corresponding to the scapula mask in the bone joint image to be measured and the scapula segmentation mask in the annotation template, and align the origin coordinates and the landmark point coordinates of the annotation template and the bone joint image to be measured according to the center of mass offset to obtain an aligned annotation template. Finally, the shoulder joint area in the aligned annotation template and the shoulder joint area in the bone joint image to be measured are aligned, and then the coordinates of each landmark point in the aligned annotation template are transformed into the coordinate space of the bone joint image to be measured according to the alignment result, so that the operator can quickly and accurately complete the landmark point detection, thereby improving the efficiency and accuracy of bone landmark point detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is one of the flow charts of the bone landmark detection method based on image registration provided by the present invention.

[0040] Figure 2This is the second flow chart of the bone landmark detection method based on image registration provided by the present invention.

[0041] Figure 3 It is a structural schematic diagram of the bone landmark point detection device based on image registration provided by the present invention.

[0042] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0044] The following combination Figure 1-Figure 3 The present invention describes a bone landmark detection method and device based on image registration.

[0045] Figure 1 This is one of the flow charts of the bone landmark detection method based on image registration provided by the present invention, such as Figure 1 As shown, the bone landmark detection method based on image registration includes the following steps:

[0046] Step 110: Determine the centroid coordinates of the scapula mask based on the bone joint image to be measured; wherein the bone joint image to be measured includes the scapula region, the scapula mask, the humerus region, and the humerus mask.

[0047] In this step, the bone and joint images to be measured include CT image data obtained by scanning the target object with a CT (Computed Tomography) device.

[0048] In this step, the target object includes a human body or a vertebrate.

[0049] In this step, the bone joint image to be measured includes bone joint related data of the target object, and the bone joints include but are not limited to bone areas such as shoulder joints, hip joints, knee joints and elbow joints.

[0050] The following takes the bone joint images to be tested including shoulder joint image data, where the shoulder joint includes the scapula and humerus, as an example to illustrate the specific implementation of the bone landmark point detection method based on image registration in this application.

[0051] In this embodiment, a mask segmentation tool or methods including but not limited to threshold segmentation, region growing, edge detection, etc. are used to segment the humerus mask and the scapula mask from the patient's CT image data.

[0052] Specifically, the CT image data is converted into a binary mask, and a region of interest (ROI) is set. For example, the ROI in the binary mask is set to 1 or True, and the background is set to 0 or False. Then, the region of interest is obtained by threshold segmentation to obtain a scapula mask or a humeral mask.

[0053] In this embodiment, after obtaining the humerus mask and the scapula mask, morphological operations (such as dilation, erosion, opening, and closing) are used to remove noise from the masks, and the total number of pixels and centroid coordinates in the masks are calculated.

[0054] Specifically, the patient's humeral mask is obtained from the image of the bone joint to be tested. and scapula mask , get the humeral segmentation mask from the annotation template and scapula segmentation mask , respectively calculated Center of mass and The center of mass ; The center of mass coordinates The calculation method is:

[0055] ;

[0056] in, is the total number of pixels in the image, 、 and are the coordinates of each pixel respectively.

[0057] Step 120: Obtain a center of mass offset based on the center of mass coordinates of the scapula segmentation mask and the center of mass coordinates of the scapula mask in the annotation template; align the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured based on the center of mass offset, and transform the coordinates of each landmark point in the annotation template to obtain an aligned annotation template; wherein, the annotation template includes a scapula segmentation area, a scapula segmentation mask, a humerus segmentation area, and a humerus segmentation mask; the humerus segmentation area includes multiple humeral landmark points, and the scapula segmentation area includes multiple scapula landmark points, and different landmark points correspond to different coordinates.

[0058] In this step, since the scapula is a relatively stable anatomical structure in the shoulder joint area, the position and morphology of the scapula have certain similarities between different individuals. Based on this similarity, the template images are preliminarily aligned. and patient images (i.e. images of the bone and joint to be tested) The position of the image can provide a more accurate starting point for subsequent registration, thereby improving the overall registration accuracy.

[0059] In this embodiment, the center of mass offset is calculated by the following formula: :

[0060] .

[0061] in, is the centroid coordinate of the scapula segmentation mask, is the centroid coordinate of the scapula mask.

[0062] In this embodiment, the origin coordinates of the annotation template are the same as the origin coordinates of the scapula segmentation mask; the origin coordinates corresponding to the aligned annotation template are calculated using the following formula:

[0063] ;

[0064] in, is the origin coordinate of the annotation template, is the origin coordinate of the aligned annotation template; the coordinates of the landmark points on the scapula in the annotation template are converted using the following formula to obtain the aligned landmark point coordinates:

[0065] ;

[0066] in, is the coordinate of the scapula landmark point after alignment, is the coordinate of the scapula landmark point; the coordinate of the landmark point on the humerus in the annotation template is converted by the following formula to obtain the coordinate of the aligned humerus landmark point:

[0067] ;

[0068] in, are the coordinates of the humeral landmark points after alignment, are the coordinates of the humeral landmark points; the aligned annotation template includes the aligned coordinates of the humeral landmark points and the scapula landmark points, as well as the new origin coordinates.

[0069] In this embodiment, the origin coordinates of the aligned scapula segmentation mask are calculated using the following formula:

[0070] ;

[0071] in, is the origin coordinate of the annotation template (that is, the origin coordinate of the scapula segmentation mask), is the origin coordinate of the aligned annotation template (that is, the origin coordinate of the aligned scapula segmentation mask).

[0072] In this embodiment, the stability of the scapula is utilized to perform preliminary alignment on the annotation template and the segmentation mask respectively through the centroid offset, which significantly improves the starting point accuracy of the subsequent fine registration and reduces the registration error.

[0073] Step 130: align the scapula segmentation region and the scapula region in the aligned annotation template to obtain a scapula registration result; align the humerus segmentation region and the humerus region in the aligned annotation template to obtain a humerus registration result; and spatially transform the coordinates of each landmark point in the aligned annotation template based on the scapula registration result and the humerus registration result to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0074] In this step, the coordinates of each landmark point on the shoulder joint in the aligned annotation template and the coordinates of each landmark point on the shoulder joint in the bone joint image to be measured are converted to the same spatial coordinate system through the aligned segmentation mask, so that the operator can quickly and accurately complete the landmark point detection later, avoiding the tedious manual annotation process in the traditional method.

[0075] For example, the aligned scapula segmentation mask is used to extract the foreground area corresponding to the aligned scapula from the aligned annotation template, and the scapula mask is used to extract the foreground area corresponding to the scapula from the bone joint image to be measured. These two foreground areas are then used to perform one or more coordinate system transformations to transform the coordinates of each landmark point in the aligned annotation template into the coordinate space of the bone joint image to be measured. This can quickly extract different bone feature points of the shoulder joint and apply them to surgical navigation or surgical robot-assisted systems to perform preoperative planning and intraoperative registration and positioning operations.

[0076] The bone landmark point detection method based on image registration provided by an embodiment of the present invention calculates the center of mass offset through the center of mass coordinates corresponding to the scapula mask in the bone joint image to be measured and the scapula segmentation mask in the annotation template, and aligns the origin coordinates and the landmark point coordinates of the annotation template and the bone joint image to be measured according to the center of mass offset to obtain an aligned annotation template. Finally, the shoulder joint area in the aligned annotation template and the shoulder joint area in the bone joint image to be measured are aligned, and then the coordinates of each landmark point in the aligned annotation template are transformed into the coordinate space of the bone joint image to be measured according to the alignment result, so that the operator can complete the landmark point detection quickly and accurately, thereby improving the efficiency and accuracy of bone landmark point detection.

[0077] In some embodiments, the annotation template is obtained by the following steps: multiple sets of template images are obtained by scanning the target object from the neck to the chest on the left side of the body, from the neck to the chest on the right side of the body, from the neck to the waist on the left side of the body, and from the neck to the waist on the right side of the body through a computed tomography (CT) device; for each set of template images, a humeral segmentation mask and a scapula segmentation mask are annotated from the template image based on expert experience, and the landmark points and coordinates to be identified are marked on the humerus, and the landmark points and coordinates to be identified are marked on the scapula to obtain the annotation template.

[0078] In this embodiment, four sets of registration template images (i.e., annotation templates) are produced based on the common clinical shoulder joint surgery CT scanning method; the four sets of registration template images respectively include: a CT image obtained by the CT device scanning from the neck to the chest on the left side of the body, a CT image obtained by the CT device scanning from the neck to the chest on the right side of the body, a CT image obtained by the CT device scanning from the neck to the waist on the left side of the body, and a CT image obtained by the CT device scanning from the neck to the waist on the right side of the body.

[0079] In this embodiment, an orthopedic surgeon annotates each set of template images based on expert experience or visual judgment. The annotation content includes: outlining the scapula segmentation mask. and humeral segmentation mask , mark the coordinates of the landmark points on the humerus and scapula to be identified and record them as and That is, the annotation template includes a scapula segmentation mask area and a humerus segmentation mask area, the scapula includes multiple landmark points and their coordinates, and the humerus includes multiple landmark points and their coordinates.

[0080] In this embodiment, the annotation template may include one or more sets of template images, and a suitable template image may be selected as the annotation template according to user needs.

[0081] The bone landmark detection method based on image registration provided by an embodiment of the present invention scans the body of the target object through a CT device to obtain four sets of registration templates with different scanning ranges, which comprehensively cover the anatomical areas required for shoulder joint surgery, ensure the wide applicability and high precision of the registration, and for each set of template images, mark the humerus segmentation mask, scapula segmentation mask, each landmark point to be identified and its coordinates according to expert experience, thereby constructing a marked template, improving the reliability and accuracy of the template image, and providing a reliable benchmark for subsequent registration.

[0082] In some embodiments, registering the scapula segmentation region and the scapula region in the aligned annotation template to obtain a scapula registration result includes:

[0083] (1) The template scapula foreground region is extracted from the scapula segmentation region of the aligned annotation template, and the scapula foreground region to be measured is extracted from the scapula region.

[0084] Specifically, the scapula segmentation mask is used From the aligned annotation template Extract the template scapula foreground area , using the scapula mask from Extract the foreground area of ​​the scapula to be measured .

[0085] (2) Perform affine transformation on the template scapula foreground area and the scapula foreground area to be measured to obtain the scapula affine transformation result.

[0086] Specifically, yes and Perform affine transformation registration; affine transformation can be expressed by the following formula:

[0087] ;

[0088] in, It is the annotation template after alignment The coordinate points in is the coordinate point after affine transformation, is the linear transformation matrix, is the translation vector; the registration goal is to find the optimal affine transformation parameters and , and use mutual information to measure similarity, and then iteratively update the affine transformation parameters through the gradient descent algorithm to minimize the registration objective function, and finally obtain the scapula affine transformation result. .

[0089] (3) Perform non-rigid transformation on the template scapula affine transformation result and the scapula foreground area to be measured to obtain the scapula registration result.

[0090] In this embodiment, and Perform a non-rigid transformation; the non-rigid transformation can be expressed as follows:

[0091] ;

[0092] in, It is the aligned annotation template after affine transformation The coordinate points in is the coordinate point after transformation, is a nonlinear transformation function.

[0093] It should be noted that It is composed of a series of BSpline basis functions defined by control points. The registration goal of non-rigid transformation is to find the position of the optimal control point and measure it using mutual information. Then, the non-rigid transformation parameters are updated by the gradient descent algorithm. By adjusting the position of the control point, a continuous and smooth deformation field is generated. , thereby achieving local and global deformation of the moving image, and completing the non-rigid registration, the scapula registration result is obtained .

[0094] In this embodiment, the humeral segmentation region and the humeral region in the aligned annotation template are registered to obtain a humeral registration result including:

[0095] (1) The template humerus foreground region is extracted from the humerus segmentation region in the aligned annotation template, and the humerus foreground region to be measured is extracted from the humerus region in the bone joint image to be measured.

[0096] Specifically, using the humeral segmentation mask from Extract the template humerus foreground area , using the humeral mask from Extract the foreground area of ​​the humerus to be measured .

[0097] (2) Perform affine transformation on the template humerus foreground area and the humerus foreground area to be measured to obtain the affine transformation result of the humerus.

[0098] In this embodiment, and Perform affine transformation registration.

[0099] Specifically, the affine transformation can be expressed as follows:

[0100] ;

[0101] in, It is the annotation template after alignment The coordinate points in is the coordinate point after affine transformation, is the linear transformation matrix, is the translation vector; the registration goal is to find the optimal affine transformation parameters and , and use mutual information to measure similarity, and then iteratively update the affine transformation parameters through the gradient descent algorithm to minimize the registration objective function, and finally obtain the affine transformation result of the humerus. .

[0102] (3) Perform non-rigid transformation registration on the affine transformation result of the humerus and the foreground area of ​​the humerus to be measured to obtain the humerus registration result.

[0103] In this embodiment, and Perform a non-rigid transformation; the non-rigid transformation can be expressed as follows:

[0104] ;

[0105] in, It is the aligned annotation template after affine transformation The coordinate points in is the coordinate point after transformation, is a nonlinear transformation function.

[0106] The registration goal of the non-rigid transformation in this embodiment is to find the position of the optimal control point and measure it using mutual information. Then, the non-rigid transformation parameters are updated by the gradient descent algorithm, and a continuous and smooth deformation field is generated by adjusting the position of the control point. , thereby achieving local and global deformation of the moving image, and completing the non-rigid registration, the humeral registration result is obtained .

[0107] The bone landmark detection method based on image registration provided by the embodiment of the present invention performs global alignment through affine transformation and then performs local fine adjustment through non-rigid transformation, thereby realizing the registration of the aligned template image with the humerus and scapula areas in the patient image, thereby improving the registration accuracy.

[0108] In some embodiments, the humerus registration result includes a humeral transformation matrix and a humeral deformation field; the scapula registration result includes a scapula transformation matrix and a scapula deformation field; the humeral transformation matrix and the humeral deformation field are obtained by applying affine transformation and non-rigid transformation to the humerus in sequence, and the scapula transformation matrix and the scapula deformation field are obtained by applying affine transformation and non-rigid transformation to the scapula in sequence.

[0109] In this embodiment, the template humerus foreground area and the humerus foreground area to be measured are affine transformed and registered to obtain the humerus affine transformation result. and the humeral transformation matrix ; Perform affine transformation registration on the template scapula foreground area and the scapula foreground area to be measured to obtain the scapula affine transformation result and the scapula transformation matrix ; Perform non-rigid transformation registration on the affine transformation result of the humerus and the foreground area of ​​the humerus to be measured to obtain the humerus registration result and humeral deformation field ; Perform non-rigid transformation registration on the affine transformation result of the scapula and the scapula foreground area to be measured to obtain the scapula registration result and scapula deformation field .

[0110] Furthermore, the coordinates of each landmark point in the aligned annotation template are spatially transformed according to the scapula registration results and the humerus registration results, and the target coordinates obtained include:

[0111] (1) According to the humeral transformation matrix, the coordinates of each landmark point on the humerus in the aligned annotation template are transformed to obtain the coordinates of the template humeral landmark points; according to the scapula transformation matrix, the coordinates of each landmark point on the scapula in the aligned annotation template are transformed to obtain the coordinates of the template scapula landmark points.

[0112] Specifically, using the scapula transformation matrix Transform the coordinates of the landmark points on the pre-aligned scapula to obtain the coordinates of the template scapula landmark points , using the humeral transformation matrix Transform the coordinates of the landmark points on the pre-aligned humerus to obtain the coordinates of the template humerus landmark points .

[0113] (2) The coordinates of the template humeral landmark points are transformed according to the humeral deformation field to obtain the coordinates of the target humeral landmark points; the coordinates of the template scapula landmark points are transformed according to the scapula deformation field to obtain the coordinates of the target scapula landmark points.

[0114] Specifically, using the scapula deformation field right Perform the transformation to obtain the coordinates of the target scapula landmark point ; Using the humeral deformation field right Perform the transformation to obtain the coordinates of the target humeral landmark point .

[0115] The pre-aligned template landmarks are transformed into the coordinate space of the patient to be tested using the affine transformation matrix and deformation field.

[0116] The bone landmark point detection method based on image registration provided by an embodiment of the present invention transforms the coordinates of each landmark point in the aligned annotation template through the change matrix obtained by affine transformation, and then further transforms the transformed landmark point coordinates through the deformation field obtained by rigid transformation to obtain the target coordinates, thereby realizing multi-level fine registration of the standardized registration template image and the patient image, and improving the accuracy and efficiency of landmark point detection.

[0117] In some embodiments, the scapula segmentation area includes the glenoid cavity, and the humeral segmentation area includes the humeral head, humeral neck and humeral medullary cavity; the target coordinates include the target humeral landmark coordinates and the target scapula landmark coordinates; after obtaining the target coordinates, the bone landmark detection method based on image registration also includes: calculating the glenoid prosthesis features according to the landmark coordinates corresponding to the glenoid cavity, and calculating the humeral prosthesis features according to the landmark coordinates corresponding to the humeral head, humeral neck and humeral medullary cavity; wherein the glenoid prosthesis features include the size, model and initial placement position of the glenoid prosthesis; the humeral prosthesis features include the size, model and initial placement position of the humeral prosthesis; estimating the postoperative offset of the scapula according to the glenoid prosthesis features and the position of the scapula rotation center, and estimating the postoperative offset of the humerus according to the humeral prosthesis features and the position of the greater tuberosity of the humerus; and performing preoperative navigation for shoulder replacement surgery based on the postoperative offset of the scapula and the postoperative offset of the humerus.

[0118] In this embodiment, multiple landmark points on the scapula, such as the superior pole, inferior pole, anterior edge and posterior edge of the glenoid, are used to determine the size of the glenoid and select the appropriate glenoid prosthesis model. At the same time, through these landmark points, the doctor can also determine the center point position of the glenoid, and then calculate key data such as the patient's original posterior tilt angle and superior tilt angle on the glenoid side, so as to determine the initial placement position of the glenoid prosthesis and ensure that the shoulder joint function is restored to the best state after surgery.

[0119] In this embodiment, data such as the initial neck-shaft angle of the humerus is calculated on the humeral side based on characteristic points such as the center of the humeral head, the center of the humeral neck, the proximal center of the humeral medullary cavity, and the distal center, so as to select a suitable humeral prosthesis model and determine its initial placement position. In the preoperative planning process, the position information of key points such as the scapular rotation center and the greater tuberosity of the humerus is used to estimate the postoperative humeral displacement of the patient's shoulder joint according to the planned position, thereby providing a basis for postoperative rehabilitation.

[0120] The bone landmark detection method based on image registration provided by the embodiment of the present invention calculates the glenoid prosthesis characteristics through the landmark coordinates corresponding to the glenoid, calculates the humeral prosthesis characteristics according to the landmark coordinates corresponding to the humeral head, humeral neck and humeral medullary cavity, and estimates the postoperative offset of the scapula according to the glenoid prosthesis characteristics and the position of the scapula rotation center, and estimates the postoperative offset of the humerus according to the humeral prosthesis characteristics and the position of the greater tuberosity of the humerus, providing reliable data support for preoperative navigation of shoulder replacement surgery.

[0121] In some embodiments, after obtaining the target coordinates, the bone landmark detection method based on image registration further includes: determining the bone surface model of the target object based on the target coordinates, and when the bone surface point cloud data of the target object is collected through bone surface surgery, aligning the bone surface point cloud data with the bone surface model to obtain a registration result: calculating the registration accuracy of each landmark point in the registration result based on the target coordinates to adjust one of the surgical navigation parameters or the positioning parameters of the robot-assisted system.

[0122] In this embodiment, the shoulder joint landmarks are used for the coarse registration positioning and registration verification process of the navigation and robot-assisted system.

[0123] In this embodiment, the ICP (Iterative Closest Point) algorithm is used to roughly register the bone surface point cloud data with the bone surface model, which can effectively improve the registration accuracy and efficiency.

[0124] In this embodiment, after the registration is completed, since the bony landmarks on the patient's bone surface are easy to identify, the registration accuracy can be judged by comparing the difference between the corresponding positions of the bony landmarks on the bone surface and the CT image to verify the registration accuracy of the surgical navigation or robotic system, and then adjust the registration accuracy to meet the surgical requirements.

[0125] The bone landmark detection method based on image registration provided by the embodiment of the present invention realizes the coarse registration of bone surface point cloud data and bone surface model through target coordinates, and verifies the registration accuracy of surgical navigation or robotic system to adjust the surgical navigation parameters or the positioning parameters of the robotic-assisted system, thereby enhancing the safety of surgery and the efficiency of surgical operation.

[0126] Figure 2 This is the second flow chart of the bone landmark detection method based on image registration provided by the present invention. Figure 2 In the embodiment shown, in the preparation stage, four sets of standard templates and corresponding segmentation masks and landmark coordinates are obtained through template production and template annotation; in the clinical process, the preoperative image is first obtained, and the template to be aligned is selected from the four sets of standard templates, and the centroid of the scapula area in the template to be aligned is aligned with the centroid of the scapula area in the preoperative image to obtain the aligned template, and the aligned template is subjected to affine transformation and non-rigid registration to obtain the registration result, and finally, according to the registration result, the landmark points in the template are converted to the spatial coordinate system of the corresponding landmark points in the preoperative image to obtain the coordinates of the patient's scapula and humerus landmark points.

[0127] The bone landmark detection device based on image registration provided by the present invention is described below. The bone landmark detection device based on image registration described below and the bone landmark detection method based on image registration described above can refer to each other.

[0128] Figure 3 FIG. 1 is a structural diagram of a bone landmark detection device based on image registration provided by the present invention, as shown in FIG. Figure 3 As shown, the bone landmark detection device based on image registration includes: a coordinate calculation module 310 , an alignment module 320 and a registration module 330 .

[0129] A coordinate calculation module 310 is used to determine the centroid coordinates of the scapula mask based on the bone joint image to be measured; wherein the bone joint image to be measured includes the scapula region, the scapula mask, the humeral region and the humeral mask;

[0130] An alignment module 320 is configured to obtain a center of mass offset based on the center of mass coordinates of the scapula segmentation mask and the center of mass coordinates of the scapula mask in the annotation template; align the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured based on the center of mass offset, and convert the coordinates of each landmark point in the annotation template to obtain an aligned annotation template; wherein the annotation template includes a scapula segmentation region, a scapula segmentation mask, a humerus segmentation region, and a humerus segmentation mask; the humerus segmentation region includes a plurality of humeral landmark points, and the scapula segmentation region includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates;

[0131] The registration module 330 is used to align the scapula segmentation area with the scapula area in the aligned annotation template to obtain a scapula registration result, and to align the humerus segmentation area with the humerus area in the aligned annotation template to obtain a humerus registration result; based on the scapula registration result and the humerus registration result, the coordinates of each landmark point in the aligned annotation template are spatially transformed to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0132] The bone landmark point detection device based on image registration provided by an embodiment of the present invention calculates the center of mass offset through the center of mass coordinates corresponding to the scapula mask in the bone joint image to be measured and the scapula segmentation mask in the annotation template, and aligns the origin coordinates and the landmark point coordinates of the annotation template and the bone joint image to be measured according to the center of mass offset to obtain an aligned annotation template. Finally, the shoulder joint area in the aligned annotation template and the shoulder joint area in the bone joint image to be measured are aligned, and then the coordinates of each landmark point in the aligned annotation template are transformed into the coordinate space of the bone joint image to be measured according to the alignment result, so that the operator can complete the landmark point detection quickly and accurately, thereby improving the efficiency and accuracy of bone landmark point detection.

[0133] Figure 4 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communications interface 420 and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute a bone landmark detection method based on image registration, the method comprising: determining the centroid coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes a scapula region, a scapula mask, a humeral region and a humeral mask; obtaining a centroid offset according to the centroid coordinates of the scapula segmentation mask and the centroid coordinates of the scapula mask in the annotation template; aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured according to the centroid offset, and converting the coordinates of each landmark point in the annotation template to obtain an aligned annotation template; wherein the annotation template includes Scapula segmentation region, scapula segmentation mask, humerus segmentation region and humerus segmentation mask; the humerus segmentation region includes multiple humerus landmark points, and different landmark points correspond to different coordinates; the scapula segmentation region and the scapula region in the aligned annotation template are aligned to obtain a scapula alignment result, and the humerus segmentation region and the humerus region in the aligned annotation template are aligned to obtain a humerus alignment result; according to the scapula alignment result and the humerus alignment result, the coordinates of each landmark point in the aligned annotation template are spatially transformed to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0134] Furthermore, the logic instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0135] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the bone landmark point detection method based on image registration provided by the above methods, the method including: determining the center of mass coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes a scapula area, a scapula mask, a humeral area and a humeral mask; obtaining a center of mass offset according to the center of mass coordinates of the scapula segmentation mask and the center of mass coordinates of the scapula mask in the annotation template; aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured according to the center of mass offset, and aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured in the annotation template. The coordinates of each landmark point are converted to obtain an aligned annotation template; wherein the annotation template includes a scapula segmentation region, a scapula segmentation mask, a humerus segmentation region and a humerus segmentation mask; the humerus segmentation region includes multiple humerus landmark points, and the scapula segmentation region includes multiple scapula landmark points, and different landmark points correspond to different coordinates; the scapula segmentation region and the scapula region in the aligned annotation template are aligned to obtain a scapula alignment result, and the humerus segmentation region and the humerus region in the aligned annotation template are aligned to obtain a humerus alignment result; according to the scapula alignment result and the humerus alignment result, the coordinates of each landmark point in the aligned annotation template are spatially transformed to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0136] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the bone landmark point detection method based on image registration provided by the above-mentioned methods, the method comprising: determining the centroid coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes a scapula region, a scapula mask, a humeral region and a humeral mask; obtaining a centroid offset according to the centroid coordinates of the scapula segmentation mask and the centroid coordinates of the scapula mask in the annotation template; aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured according to the centroid offset, and converting the coordinates of each landmark point in the annotation template to obtain to the aligned annotation template; wherein the annotation template includes a scapula segmentation region, a scapula segmentation mask, a humerus segmentation region and a humerus segmentation mask; the humerus segmentation region includes multiple humerus landmark points, and the scapula segmentation region includes multiple scapula landmark points, and different landmark points correspond to different coordinates; the scapula segmentation region and the scapula region in the aligned annotation template are aligned to obtain a scapula alignment result, and the humerus segmentation region and the humerus region in the aligned annotation template are aligned to obtain a humerus alignment result; according to the scapula alignment result and the humerus alignment result, the coordinates of each landmark point in the aligned annotation template are spatially transformed to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0137] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0138] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A bone landmark detection method based on image registration, characterized in that: include: Determining the centroid coordinates of the scapula mask based on the bone and joint image to be measured; wherein the bone and joint image to be measured includes the scapula region, the scapula mask, the humeral region, and the humeral mask; A center of mass offset is obtained according to the center of mass coordinates of the scapula segmentation mask in the annotation template and the center of mass coordinates of the scapula mask; the origin coordinates of the annotation template are aligned with the origin coordinates of the bone joint image to be measured according to the center of mass offset, and the coordinates of each landmark point in the annotation template are converted to obtain an aligned annotation template; wherein the annotation template includes a scapula segmentation area, a scapula segmentation mask, a humerus segmentation area and a humerus segmentation mask; the humerus segmentation area includes a plurality of humeral landmark points, and the scapula segmentation area includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates; Aligning the scapula segmentation region in the aligned annotation template with the scapula region to obtain a scapula registration result, and aligning the humerus segmentation region in the aligned annotation template with the humerus region to obtain a humerus registration result; performing spatial transformation on the coordinates of each landmark point in the aligned annotation template according to the scapula registration result and the humerus registration result to obtain a target coordinate; The humerus registration result includes a humerus transformation matrix and a humerus deformation field; the scapula registration result includes a scapula transformation matrix and a scapula deformation field; the humerus transformation matrix and the humerus deformation field are obtained by sequentially applying affine transformation and non-rigid transformation to the humerus, and the scapula transformation matrix and the scapula deformation field are obtained by sequentially applying affine transformation and non-rigid transformation to the scapula; The spatial transformation of the coordinates of each landmark point in the aligned annotation template according to the scapula registration result and the humerus registration result to obtain the target coordinates includes: The coordinates of each landmark point on the humerus in the aligned annotation template are transformed according to the humeral transformation matrix to obtain the coordinates of the template humeral landmark points; the coordinates of each landmark point on the scapula in the aligned annotation template are transformed according to the scapula transformation matrix to obtain the coordinates of the template scapula landmark points; The coordinates of the template humeral landmark points are transformed according to the humeral deformation field to obtain the coordinates of the target humeral landmark points; the coordinates of the template scapula landmark points are transformed according to the scapula deformation field to obtain the coordinates of the target scapula landmark points.

2. The bone landmark detection method based on image registration according to claim 1, characterized in that: The annotation template is obtained through the following steps: acquiring a plurality of sets of template images by scanning the target subject from the neck to the chest on the left side of the body, from the neck to the chest on the right side of the body, from the neck to the waist on the left side of the body, and from the neck to the waist on the right side of the body using a computed tomography (CT) device; For each set of template images, a humerus segmentation mask and a scapula segmentation mask are marked from the template images, and landmark points and coordinates to be identified are marked on the humerus, and landmark points and coordinates to be identified are marked on the scapula to obtain the marked template.

3. The bone landmark detection method based on image registration according to claim 1, characterized in that: The registering the scapula segmentation region in the aligned annotation template with the scapula region to obtain a scapula registration result includes: Extracting a template scapula foreground region from the scapula segmentation region of the aligned annotation template; extracting a scapula foreground region to be measured from the scapula region; Performing an affine transformation on the template scapula foreground region and the scapula foreground region to be measured to obtain a scapula affine transformation result, and performing a non-rigid transformation on the scapula affine transformation result and the scapula foreground region to be measured to obtain the scapula registration result; The registering the humeral segmentation region in the aligned annotation template with the humeral region to obtain a humeral registration result includes: Extracting a template humerus foreground region from the humerus segmentation region in the aligned annotation template, and extracting a humerus foreground region to be measured from the humerus region in the bone joint image to be measured; Affine transformation is performed on the template humerus foreground area and the humerus foreground area to be measured to obtain a humerus affine transformation result; and non-rigid transformation registration is performed on the humerus affine transformation result and the humerus foreground area to be measured to obtain the humerus registration result.

4. The bone landmark detection method based on image registration according to claim 1, characterized in that: The scapula segmentation area includes the glenoid cavity, and the humeral segmentation area includes the humeral head, humeral neck, and humeral medullary cavity; the target coordinates include target humeral landmark coordinates and target scapula landmark coordinates; After obtaining the target coordinates, the method further includes: Calculating glenoid prosthesis characteristics based on the coordinates of the landmark points corresponding to the glenoid, and calculating humeral prosthesis characteristics based on the coordinates of the landmark points corresponding to the humeral head, humeral neck, and humeral medullary cavity; wherein the glenoid prosthesis characteristics include the size, model, and initial placement position of the glenoid prosthesis; and the humeral prosthesis characteristics include the size, model, and initial placement position of the humeral prosthesis; The postoperative offset of the scapula is estimated according to the glenoid prosthesis characteristics and the position of the scapula rotation center, and the postoperative offset of the humerus is estimated according to the humeral prosthesis characteristics and the position of the greater tuberosity of the humerus.

5. The bone landmark detection method based on image registration according to claim 1, characterized in that: After obtaining the target coordinates, the method further includes: A bone surface model of a target object is determined based on the target coordinates, and when bone surface point cloud data of the target object is collected through bone surface surgery, the bone surface point cloud data is registered with the bone surface model to obtain a registration result: The registration accuracy of each landmark point in the registration result is calculated based on the target coordinates to adjust one of the surgical navigation parameters or the positioning parameters of the robot-assisted system.

6. A bone landmark detection device based on image registration, using the bone landmark detection method based on image registration as claimed in claim 1, characterized in that: include: A coordinate calculation module, configured to determine the centroid coordinates of the scapula mask based on the bone and joint image to be measured; wherein the bone and joint image to be measured includes the scapula region, the scapula mask, the humeral region, and the humeral mask; An alignment module is configured to obtain a center of mass offset based on the center of mass coordinates of the scapula segmentation mask in the annotation template and the center of mass coordinates of the scapula mask; align the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured based on the center of mass offset, and convert the coordinates of each landmark point in the annotation template to obtain an aligned annotation template; wherein the annotation template includes a scapula segmentation region, a scapula segmentation mask, a humerus segmentation region, and a humerus segmentation mask; the humerus segmentation region includes a plurality of humeral landmark points, and the scapula segmentation region includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates; A registration module is used to align the scapula segmentation area in the aligned annotation template with the scapula area to obtain a scapula registration result, and to align the humerus segmentation area in the aligned annotation template with the humerus area to obtain a humerus registration result; and to perform spatial transformation on the coordinates of each landmark point in the aligned annotation template according to the scapula registration result and the humerus registration result to obtain the target coordinates.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the bone landmark detection method based on image registration according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the bone landmark detection method based on image registration according to any one of claims 1 to 5 is implemented.

Images