Bone mark point detection method and device based on image registration

Abstract

The invention relates to the technical field of image registration, and provides a bone mark point detection method and device based on image registration, and the method comprises the steps: determining the mass center coordinates of a scapula mask according to a to-be-detected bone joint image; according to the centroid offset between the scapula segmentation mask and the scapula mask in the labeling template, aligning the origin coordinates of the labeling template with the origin coordinates of the bone joint image to be detected, and converting the coordinates of each mark point in the labeling template to obtain an aligned labeling template; respectively registering the shoulder blade segmentation region and the shoulder blade region and the humerus segmentation region and the humerus region in the aligned marking template to obtain registration results; and according to a registration result, performing spatial transformation on the coordinates of the mark points in the aligned marking template to obtain target coordinates for navigation and planning of shoulder joint replacement surgery. According to the method, the bone mark point detection efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image registration, and in particular to a bone landmark detection method and device based on image registration. Background Art

[0002] Shoulder replacement surgery is a procedure that implants an artificial prosthesis for patients with shoulder joint injuries. Accurate detection and positioning of bony landmarks of the shoulder joint are crucial for preoperative navigation and planning.

[0003] In the related art, it is usually necessary to first perform three-dimensional reconstruction of preoperative or intraoperative medical images, and then transfer the anatomical information to the point cloud data of the human body or key parts of the human body through a registration algorithm based on image grayscale values ​​to achieve the positioning of shoulder joint landmark points. This method has strict requirements on the initial registration conditions. If the initial registration is inaccurate, the accuracy of the registration results will be reduced, thereby reducing the point accuracy of the shoulder joint landmark points. At the same time, the algorithm is easily affected by image noise and outliers when calculating the image grayscale value, which will also lead to inaccurate registration results. In addition, the existing technology also automatically learns key features in the image through deep learning methods, and uses them for bony landmark point detection. Since this method requires a large amount of labeled data for model training, it takes a lot of time and cost, and the decision-making process during model training lacks transparency, resulting in low reliability of the shoulder joint detection results output by the model. Summary of the invention

[0004] The present invention provides a bone landmark detection method and device based on image registration, so as to solve the problem that the existing technology uses a registration algorithm based on image gray value to locate the shoulder joint landmarks, which is easily affected by inaccurate initial registration or image noise and outliers, resulting in inaccurate final registration results; and when using a deep learning method to detect bone landmarks, due to the time-consuming data labeling and the opaque decision-making process during model training, the bone landmark detection is inefficient and the detection results are inaccurate, thereby improving the efficiency and accuracy of bone landmark detection.

[0005] The present invention provides a bone landmark detection method based on image registration, comprising: Determine the centroid coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes a scapula region, a scapula mask, a humeral region and a humeral mask; The center of mass offset is obtained according to the center of mass coordinates of the scapula segmentation mask in the annotation template and the center of mass coordinates of the scapula mask; the origin coordinates of the annotation template are aligned with the origin coordinates of the bone joint image to be measured according to the center of mass offset, and the coordinates of each landmark point in the annotation template are converted to obtain the aligned annotation template; wherein the annotation template includes a scapula segmentation area, a scapula segmentation mask, a humerus segmentation area and a humerus segmentation mask; the humerus segmentation area includes a plurality of humerus landmark points, and the scapula segmentation area includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates; The scapula segmentation area in the aligned annotation template is aligned with the scapula area to obtain a scapula alignment result, and the humerus segmentation area in the aligned annotation template is aligned with the humerus area to obtain a humerus alignment result; the coordinates of each landmark point in the aligned annotation template are spatially transformed according to the scapula alignment result and the humerus alignment result to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0006] According to a bone landmark detection method based on image registration provided by the present invention, the annotation template is obtained through the following steps: Acquire multiple sets of template images by scanning the target subject from the neck to the chest on the left side of the body, from the neck to the chest on the right side of the body, from the neck to the waist on the left side of the body, and from the neck to the waist on the right side of the body using a computed tomography (CT) device; For each set of template images, a humerus segmentation mask and a scapula segmentation mask are annotated from the template images according to expert experience, and landmark points and coordinates to be identified are marked on the humerus, and landmark points and coordinates to be identified are marked on the scapula to obtain the annotated template.

[0007] According to a bone landmark detection method based on image registration provided by the present invention, the scapula segmentation region in the aligned annotation template is registered with the scapula region to obtain a scapula registration result, which includes: Extracting a template scapula foreground region from the scapula segmentation region of the aligned annotation template; extracting a scapula foreground region to be measured from the scapula region; Performing an affine transformation on the template scapula foreground area and the scapula foreground area to be measured to obtain a scapula affine transformation result, and performing a non-rigid transformation on the scapula affine transformation result and the scapula foreground area to be measured to obtain the scapula registration result; The registering the humerus segmentation region in the aligned annotation template with the humerus region to obtain a humerus registration result comprises: Extracting a template humerus foreground region from the humerus segmentation region in the aligned annotation template, and extracting a humerus foreground region to be measured from the humerus region in the bone joint image to be measured; Affine transformation is performed on the template humerus foreground area and the humerus foreground area to be measured to obtain a humerus affine transformation result; non-rigid transformation registration is performed on the humerus affine transformation result and the humerus foreground area to be measured to obtain the humerus registration result.

[0008] According to a bone landmark detection method based on image registration provided by the present invention, the humerus registration result includes a humerus transformation matrix and a humerus deformation field; the scapula registration result includes a scapula transformation matrix and a scapula deformation field; the humerus transformation matrix and the humerus deformation field are obtained by sequentially applying affine transformation and non-rigid transformation to the humerus, and the scapula transformation matrix and the scapula deformation field are obtained by sequentially applying affine transformation and non-rigid transformation to the scapula; The spatial transformation of the coordinates of each landmark point in the aligned annotation template according to the scapula registration result and the humerus registration result to obtain the target coordinates includes: The coordinates of each landmark point on the humerus in the aligned annotation template are transformed according to the humerus transformation matrix to obtain the coordinates of the humerus landmark points of the template; the coordinates of each landmark point on the scapula in the aligned annotation template are transformed according to the scapula transformation matrix to obtain the coordinates of the scapula landmark points of the template; The coordinates of the template humerus landmark points are transformed according to the humerus deformation field to obtain the coordinates of the target humerus landmark points; the coordinates of the template scapula landmark points are transformed according to the scapula deformation field to obtain the coordinates of the target scapula landmark points.

[0009] According to a bone landmark detection method based on image registration provided by the present invention, the scapula segmentation area includes the glenoid cavity, the humeral segmentation area includes the humeral head, the humeral neck and the humeral medullary cavity; the target coordinates include the target humeral landmark coordinates and the target scapula landmark coordinates; After obtaining the target coordinates, the method further includes: The glenoid prosthesis features are calculated according to the coordinates of the landmark points corresponding to the glenoid, and the humeral prosthesis features are calculated according to the coordinates of the landmark points corresponding to the humeral head, humeral neck and humeral medullary cavity; wherein the glenoid prosthesis features include the size, model and initial placement position of the glenoid prosthesis; and the humeral prosthesis features include the size, model and initial placement position of the humeral prosthesis; estimating the postoperative displacement of the scapula according to the glenoid prosthesis characteristics and the position of the scapula rotation center, and estimating the postoperative displacement of the humerus according to the humeral prosthesis characteristics and the position of the greater tuberosity of the humerus; Preoperative navigation is performed on the shoulder replacement surgery according to the postoperative offset of the scapula and the postoperative offset of the humerus.

[0010] According to a bone landmark detection method based on image registration provided by the present invention, after obtaining the target coordinates, the method further includes: The bone surface model of the target object is determined based on the target coordinates, and when the bone surface point cloud data of the target object is collected through bone surface surgery, the bone surface point cloud data is registered with the bone surface model to obtain a registration result: The registration accuracy of each landmark in the registration result is calculated based on the target coordinates to adjust one of the surgical navigation parameters or the positioning parameters of the robot-assisted system.

[0011] The present invention also provides a bone landmark detection device based on image registration, comprising: A coordinate calculation module, used to determine the centroid coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes the scapula region, the scapula mask, the humerus region and the humerus mask; An alignment module, used for obtaining a center of mass offset according to the center of mass coordinates of the scapula segmentation mask in the annotation template and the center of mass coordinates of the scapula mask; aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured according to the center of mass offset, and converting the coordinates of each landmark point in the annotation template to obtain an aligned annotation template; wherein the annotation template includes a scapula segmentation region, a scapula segmentation mask, a humerus segmentation region and a humerus segmentation mask; the humerus segmentation region includes a plurality of humerus landmark points, and the scapula segmentation region includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates; A registration module is used to align the scapula segmentation area in the aligned annotation template with the scapula area to obtain a scapula registration result, and to align the humerus segmentation area in the aligned annotation template with the humerus area to obtain a humerus registration result; spatially transform the coordinates of each landmark point in the aligned annotation template according to the scapula registration result and the humerus registration result to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the bone landmark detection method based on image registration as described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the bone landmark detection method based on image registration as described in any one of the above is implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for detecting bone landmarks based on image registration.

[0015] The bone landmark point detection method and device based on image registration provided by the present invention calculate the center of mass offset through the center of mass coordinates corresponding to the scapula mask in the bone joint image to be measured and the scapula segmentation mask in the annotation template, and align the origin coordinates and the landmark point coordinates of the annotation template and the bone joint image to be measured according to the center of mass offset to obtain the aligned annotation template, and finally align the shoulder joint area in the aligned annotation template and the shoulder joint area in the bone joint image to be measured, and then transform the coordinates of each landmark point in the aligned annotation template into the coordinate space of the bone joint image to be measured according to the registration result, so that the operator can quickly and accurately complete the landmark point detection, thereby improving the efficiency and accuracy of bone landmark point detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is one of the flow charts of the bone landmark detection method based on image registration provided by the present invention.

[0018] Figure 2 This is the second flow chart of the bone landmark detection method based on image registration provided by the present invention.

[0019] Figure 3 It is a structural schematic diagram of a bone landmark point detection device based on image registration provided by the present invention.

[0020] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Combine the following Figure 1-Figure 3 The present invention describes a bone landmark detection method and device based on image registration.

[0023] Figure 1 is one of the flow charts of the bone landmark detection method based on image registration provided by the present invention, such as Figure 1 As shown, the bone landmark detection method based on image registration includes the following steps: Step 110, determining the centroid coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes the scapula region, the scapula mask, the humerus region and the humerus mask.

[0024] In this step, the bone and joint images to be measured include CT image data obtained by scanning the target object with a CT (Computed Tomography) device.

[0025] In this step, the target object includes a human body or a vertebrate.

[0026] In this step, the bone joint image to be measured includes bone joint related data of the target object, and the bone joints include but are not limited to bone areas such as shoulder joints, hip joints, knee joints and elbow joints.

[0027] The following uses the bone joint images to be tested including shoulder joint image data, where the shoulder joint includes the scapula and the humerus as an example to illustrate the specific implementation of the bone landmark point detection method based on image registration in the present application.

[0028] In this embodiment, a mask segmentation tool or methods including but not limited to threshold segmentation, region growing, edge detection, etc. are used to segment the humerus mask and the scapula mask from the patient's CT image data.

[0029] Specifically, the CT image data is converted into a binary mask, and a region of interest (ROI) is set. For example, the ROI in the binary mask is set to 1 or True, and the background is set to 0 or False. Then, the region of interest is obtained by using threshold segmentation to obtain a scapula mask or a humerus mask.

[0030] In this embodiment, after the humerus mask and the scapula mask are obtained, morphological operations (such as dilation, erosion, opening operation, and closing operation) are used to remove noise in the above masks, and the total number of pixels and centroid coordinates in the masks are calculated.

[0031] Specifically, the patient's humeral mask is obtained from the image of the bone joint to be tested. and scapula mask , get the humerus segmentation mask from the annotation template and scapula segmentation mask , respectively calculated Centroid and The centroid ; The centroid coordinates The calculation method is: ; in, is the total number of pixels in the image, , and are the coordinates of each pixel respectively.

[0032] Step 120, obtaining a center of mass offset according to the center of mass coordinates of the scapula segmentation mask and the center of mass coordinates of the scapula mask in the annotation template; aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured according to the center of mass offset, and transforming the coordinates of each landmark point in the annotation template to obtain an aligned annotation template; wherein the annotation template includes a scapula segmentation region, a scapula segmentation mask, a humerus segmentation region and a humerus segmentation mask; the humerus segmentation region includes multiple humerus landmark points, and the scapula segmentation region includes multiple scapula landmark points, and different landmark points correspond to different coordinates.

[0033] In this step, since the scapula is a relatively stable anatomical structure in the shoulder joint area, the position and morphology of the scapula have certain similarities between different individuals. Based on this similarity, the template images are preliminarily aligned. and patient images (i.e. images of the bone and joint to be tested) The position can provide a more accurate starting point for subsequent registration, thereby improving the overall registration accuracy.

[0034] In this embodiment, the center of mass offset is calculated by the following formula: : .

[0035] in, is the centroid coordinate of the scapula segmentation mask, are the centroid coordinates of the scapula mask.

[0036] In this embodiment, the origin coordinates of the annotation template are the same as the origin coordinates of the scapula segmentation mask; the origin coordinates corresponding to the aligned annotation template are calculated by the following formula: ; in, is the origin coordinate of the annotation template, is the origin coordinate of the aligned annotation template; the coordinates of the landmark points on the scapula in the annotation template are converted by the following formula to obtain the aligned landmark point coordinates: ; in, are the coordinates of the scapula landmark points after alignment, is the coordinate of the scapula landmark point; the coordinate of the landmark point on the humerus in the annotation template is converted by the following formula to obtain the coordinate of the aligned humerus landmark point: ; in, are the coordinates of the humeral landmark points after alignment, are the coordinates of the humeral landmark points; the aligned annotation template includes the aligned humeral landmark point coordinates and scapula landmark point coordinates, as well as the new origin coordinates.

[0037] In this embodiment, the origin coordinates of the aligned scapula segmentation mask are calculated by the following formula: ; in, is the origin coordinate of the annotation template (that is, the origin coordinate of the scapula segmentation mask), is the origin coordinate of the aligned annotation template (that is, the origin coordinate of the aligned scapula segmentation mask).

[0038] In this embodiment, the stability of the scapula is utilized to perform preliminary alignment of the annotation template and the segmentation mask through the centroid offset, which significantly improves the starting point accuracy of the subsequent fine registration and reduces the registration error.

[0039] Step 130, align the scapula segmentation area with the scapula area in the aligned annotation template to obtain a scapula alignment result, align the humerus segmentation area with the humerus area in the aligned annotation template to obtain a humerus alignment result; according to the scapula alignment result and the humerus alignment result, spatially transform the coordinates of each landmark point in the aligned annotation template to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0040] In this step, the coordinates of each landmark point on the shoulder joint in the aligned annotation template and the coordinates of each landmark point on the shoulder joint in the bone joint image to be tested are converted to the same spatial coordinate system through the aligned segmentation mask, so that the operator can quickly and accurately complete the landmark point detection later, avoiding the tedious manual labeling process in the traditional method.

[0041] For example, the aligned scapula segmentation mask is used to extract the foreground area corresponding to the aligned scapula from the aligned annotation template, and the scapula mask is used to extract the foreground area corresponding to the scapula from the bone joint image to be tested. These two foreground areas are then used to perform one or more coordinate system conversions to transform the coordinates of each landmark point in the aligned annotation template into the coordinate space of the bone joint image to be tested. This allows for rapid extraction of different bone feature points of the shoulder joint and applies them to surgical navigation or surgical robot-assisted systems to perform preoperative planning and intraoperative registration and positioning.

[0042] The bone landmark point detection method based on image registration provided by the embodiment of the present invention calculates the center of mass offset through the center of mass coordinates corresponding to the scapula mask in the bone joint image to be measured and the scapula segmentation mask in the annotation template, and aligns the origin coordinates and the landmark point coordinates of the annotation template and the bone joint image to be measured according to the center of mass offset to obtain an aligned annotation template, and finally aligns the shoulder joint area in the aligned annotation template and the shoulder joint area in the bone joint image to be measured, and then transforms the coordinates of each landmark point in the aligned annotation template into the coordinate space of the bone joint image to be measured according to the registration result, so that the operator can quickly and accurately complete the landmark point detection, thereby improving the efficiency and accuracy of bone landmark point detection.

[0043] In some embodiments, the annotation template is obtained by the following steps: multiple sets of template images are acquired by scanning the target object from the neck to the chest on the left side of the body, from the neck to the chest on the right side of the body, from the neck to the waist on the left side of the body, and from the neck to the waist on the right side of the body through a computed tomography (CT) device; for each set of template images, a humeral segmentation mask and a scapula segmentation mask are annotated from the template image according to expert experience, and the landmark points and coordinates to be identified are marked on the humerus, and the landmark points and coordinates to be identified are marked on the scapula to obtain the annotation template.

[0044] In this embodiment, four sets of registration template images (i.e., annotation templates) are prepared according to the commonly used clinical shoulder joint surgery CT scanning method; the four sets of registration template images respectively include: a CT image obtained by the CT device scanning from the neck to the chest on the left side of the body, a CT image obtained by the CT device scanning from the neck to the chest on the right side of the body, a CT image obtained by the CT device scanning from the neck to the waist on the left side of the body, and a CT image obtained by the CT device scanning from the neck to the waist on the right side of the body.

[0045] In this embodiment, an orthopedic surgeon annotates each set of template images based on expert experience or visual judgment. The annotation content includes: outlining the scapula segmentation mask and humerus segmentation mask , mark the coordinates of the landmark points on the humerus and scapula to be identified and record them as and That is, the annotation template includes a scapula segmentation mask area and a humerus segmentation mask area, the scapula includes a plurality of landmark points and their coordinates, and the humerus includes a plurality of landmark points and their coordinates.

[0046] In this embodiment, the annotation template may include one or more sets of template images, and a suitable template image may be selected as the annotation template according to user needs.

[0047] The bone landmark detection method based on image registration provided in an embodiment of the present invention scans the body of the target object through a CT device to obtain four sets of registration templates with different scanning ranges, which comprehensively cover the anatomical areas required for shoulder joint surgery, ensure the wide applicability and high precision of the registration, and for each set of template images, annotate the humerus segmentation mask, scapula segmentation mask, each landmark point to be identified and its coordinates according to expert experience, thereby constructing a labeled template, improving the reliability and accuracy of the template image, and providing a reliable benchmark for subsequent registration.

[0048] In some embodiments, registering the scapula segmentation region and the scapula region in the aligned annotation template to obtain a scapula registration result includes: (1) The template scapula foreground region is extracted from the scapula segmentation region of the aligned annotation template, and the scapula foreground region to be measured is extracted from the scapula region.

[0049] Specifically, the scapula segmentation mask is used From the aligned annotation template Extract the scapula foreground area of ​​the template , using the scapula mask from Extract the foreground area of ​​the scapula to be measured .

[0050] (2) Perform affine transformation on the template scapula foreground area and the scapula foreground area to be measured to obtain the scapula affine transformation result.

[0051] Specifically, yes and Perform affine transformation registration; affine transformation can be expressed by the following formula: ; in, This is the aligned annotation template The coordinate points in is the coordinate point after affine transformation, is the linear transformation matrix, is the translation vector; the registration goal is to find the optimal affine transformation parameters and , and use mutual information to measure similarity, then iteratively update the affine transformation parameters through the gradient descent algorithm to minimize the registration objective function, and finally obtain the affine transformation result of the scapula .

[0052] (3) Perform non-rigid transformation on the template scapula affine transformation result and the scapula foreground area to be measured to obtain the scapula registration result.

[0053] In this embodiment, and Perform a non-rigid transformation; the non-rigid transformation can be expressed by the following formula: ; in, It is the aligned annotation template after affine transformation The coordinate points in is the coordinate point after transformation, is a nonlinear transformation function.

[0054] It should be noted that It is composed of a series of BSpline basis functions defined by control points. The registration goal of non-rigid transformation is to find the position of the optimal control point and measure it using mutual information. Then, the non-rigid transformation parameters are updated by the gradient descent algorithm. A continuous and smooth deformation field is generated by adjusting the position of the control point. , thereby realizing local and global deformation of the moving image, and completing the non-rigid registration, the scapula registration result is obtained .

[0055] In this embodiment, the humerus segmentation region and the humerus region in the aligned annotation template are registered to obtain a humerus registration result including: (1) The template humerus foreground region is extracted from the humerus segmentation region in the aligned annotation template, and the humerus foreground region to be measured is extracted from the humerus region in the bone joint image to be measured.

[0056] Specifically, using the humeral segmentation mask from Extract the template humerus foreground area , using the humeral mask from Extract the foreground area of ​​the humerus to be measured .

[0057] (2) Perform affine transformation on the template humerus foreground area and the humerus foreground area to be measured to obtain the humerus affine transformation result.

[0058] In this embodiment, and Perform affine transformation registration.

[0059] Specifically, the affine transformation can be expressed as follows: ; in, This is the aligned annotation template The coordinate points in is the coordinate point after affine transformation, is the linear transformation matrix, is the translation vector; the registration goal is to find the optimal affine transformation parameters and , and use mutual information to measure similarity, then iteratively update the affine transformation parameters through the gradient descent algorithm to minimize the registration objective function, and finally obtain the affine transformation result of the humerus .

[0060] (3) Perform non-rigid transformation registration on the affine transformation result of the humerus and the foreground area of ​​the humerus to be measured to obtain the humerus registration result.

[0061] In this embodiment, and Perform a non-rigid transformation; the non-rigid transformation can be expressed by the following formula: ; in, It is the aligned annotation template after affine transformation The coordinate points in is the coordinate point after transformation, is a nonlinear transformation function.

[0062] The registration goal of the non-rigid transformation in this embodiment is to find the position of the optimal control point, measure it using mutual information, and then update the non-rigid transformation parameters through the gradient descent algorithm to generate a continuous and smooth deformation field by adjusting the position of the control point. , thereby realizing local and global deformation of the moving image, and completing the non-rigid registration, the humeral registration result is obtained .

[0063] The bone landmark detection method based on image registration provided in an embodiment of the present invention performs global alignment through affine transformation and then performs local fine adjustment through non-rigid transformation, thereby realizing the registration of the aligned template image with the humerus and scapula areas in the patient image, thereby improving the registration accuracy.

[0064] In some embodiments, the humerus registration result includes a humerus transformation matrix and a humerus deformation field; the scapula registration result includes a scapula transformation matrix and a scapula deformation field; the humerus transformation matrix and the humerus deformation field are obtained by applying affine transformation and non-rigid transformation to the humerus in sequence, and the scapula transformation matrix and the scapula deformation field are obtained by applying affine transformation and non-rigid transformation to the scapula in sequence.

[0065] In this embodiment, the template humerus foreground area and the humerus foreground area to be measured are registered by affine transformation to obtain the humerus affine transformation result. and the humeral transformation matrix ; Perform affine transformation registration on the template scapula foreground area and the scapula foreground area to be tested to obtain the scapula affine transformation result and the scapula transformation matrix ; Perform non-rigid transformation registration on the affine transformation result of the humerus and the foreground area of ​​the humerus to be measured to obtain the humerus registration result and humeral deformation field ; Perform non-rigid transformation registration on the affine transformation result of the scapula and the scapula foreground area to be measured to obtain the scapula registration result and scapula deformation field .

[0066] Furthermore, according to the scapula registration result and the humerus registration result, the coordinates of each landmark point in the aligned annotation template are spatially transformed to obtain the target coordinates including: (1) The coordinates of each landmark point on the humerus in the aligned annotation template are transformed according to the humerus transformation matrix to obtain the coordinates of the template humerus landmark points; the coordinates of each landmark point on the scapula in the aligned annotation template are transformed according to the scapula transformation matrix to obtain the coordinates of the template scapula landmark points.

[0067] Specifically, use the scapula transformation matrix Transform the coordinates of the landmark points on the pre-aligned scapula to obtain the coordinates of the landmark points on the template scapula , using the humeral transformation matrix Transform the coordinates of the landmark points on the pre-aligned humerus to obtain the coordinates of the landmark points on the template humerus .

[0068] (2) The coordinates of the humeral landmark points of the template are transformed according to the humeral deformation field to obtain the coordinates of the target humeral landmark points; the coordinates of the scapula landmark points of the template are transformed according to the scapula deformation field to obtain the coordinates of the target scapula landmark points.

[0069] Specifically, using the scapula deformation field right Transform to obtain the coordinates of the target scapula landmark point ; Using the humeral deformation field right Transform to obtain the coordinates of the target humeral landmark point .

[0070] The pre-aligned template landmarks are transformed into the coordinate space of the patient to be tested using the affine transformation matrix and the deformation field.

[0071] The bone landmark point detection method based on image registration provided by an embodiment of the present invention transforms the coordinates of each landmark point in the aligned annotation template through the change matrix obtained by affine transformation, and further transforms the transformed landmark point coordinates through the deformation field obtained by rigid transformation to obtain the target coordinates, thereby realizing multi-level fine registration of the standardized registration template image with the patient image, and improving the accuracy and efficiency of landmark point detection.

[0072] In some embodiments, the scapula segmentation area includes the glenoid cavity, and the humeral segmentation area includes the humeral head, humeral neck and humeral medullary cavity; the target coordinates include the target humeral landmark coordinates and the target scapula landmark coordinates; after obtaining the target coordinates, the bone landmark detection method based on image registration also includes: calculating the glenoid prosthesis features according to the landmark coordinates corresponding to the glenoid cavity, and calculating the humeral prosthesis features according to the landmark coordinates corresponding to the humeral head, humeral neck and humeral medullary cavity; wherein the glenoid prosthesis features include the size, model and initial placement position of the glenoid prosthesis; the humeral prosthesis features include the size, model and initial placement position of the humeral prosthesis; the postoperative offset of the scapula is estimated according to the glenoid prosthesis features and the position of the scapula rotation center, and the postoperative offset of the humerus is estimated according to the humeral prosthesis features and the position of the greater tuberosity of the humerus; and preoperative navigation of shoulder replacement surgery is performed according to the postoperative offset of the scapula and the postoperative offset of the humerus.

[0073] In this embodiment, multiple landmark points on the scapula, such as the superior pole, inferior pole, anterior edge and posterior edge of the glenoid, are used to determine the size of the glenoid and select the appropriate glenoid prosthesis model. At the same time, through these landmark points, the doctor can also determine the center point position of the glenoid, and then calculate key data such as the original posterior tilt angle and superior tilt angle of the patient on the glenoid side, so as to determine the initial placement position of the glenoid prosthesis and ensure that the shoulder joint function is restored to the optimal state after surgery.

[0074] In this embodiment, data such as the initial neck-shaft angle of the humerus is calculated on the humeral side based on feature points such as the center of the humeral head, the center of the humeral neck, the proximal center of the humeral medullary cavity, and the distal center, so as to select a suitable humeral prosthesis model and determine its initial placement position. In the preoperative planning process, the position information of key points such as the scapular rotation center and the greater tuberosity of the humerus is used to estimate the postoperative humeral displacement of the patient's shoulder joint according to the planned position, so as to provide a basis for postoperative rehabilitation.

[0075] The bone landmark detection method based on image registration provided in an embodiment of the present invention calculates the glenoid prosthesis characteristics through the landmark coordinates corresponding to the glenoid, calculates the humeral prosthesis characteristics according to the landmark coordinates corresponding to the humeral head, humeral neck and humeral medullary cavity, and estimates the postoperative offset of the scapula according to the glenoid prosthesis characteristics and the position of the scapula rotation center, and estimates the postoperative offset of the humerus according to the humeral prosthesis characteristics and the position of the greater tuberosity of the humerus, thereby providing reliable data support for preoperative navigation of shoulder replacement surgery.

[0076] In some embodiments, after obtaining the target coordinates, the bone landmark detection method based on image registration also includes: determining the bone surface model of the target object based on the target coordinates, and when the bone surface point cloud data of the target object is collected through bone surface surgery, registering the bone surface point cloud data with the bone surface model to obtain a registration result: calculating the registration accuracy of each landmark point in the registration result based on the target coordinates to adjust one of the surgical navigation parameters or the positioning parameters of the robot-assisted system.

[0077] In this embodiment, the shoulder joint landmarks are used for the rough registration positioning and registration verification process of the navigation and robot-assisted system.

[0078] In this embodiment, the ICP (Iterative Closest Point) algorithm is used to roughly align the bone surface point cloud data with the bone surface model, which can effectively improve the alignment accuracy and efficiency.

[0079] In this embodiment, after the registration is completed, since the bony landmarks on the patient's bone surface are easy to identify, the registration accuracy can be determined by comparing the difference between the corresponding positions of the bony landmarks on the bone surface and the CT image to verify the registration accuracy of the surgical navigation or robotic system, and then adjust the registration accuracy to meet the surgical requirements.

[0080] The bone landmark detection method based on image registration provided in the embodiment of the present invention realizes the coarse registration of bone surface point cloud data and bone surface model through target coordinates, and verifies the registration accuracy of surgical navigation or robot system to adjust the surgical navigation parameters or the positioning parameters of the robot-assisted system, thereby enhancing the safety of surgery and the efficiency of surgical operation.

[0081] Figure 2This is the second flow chart of the bone landmark detection method based on image registration provided by the present invention. Figure 2 In the illustrated embodiment, in the preparation stage, four sets of standard templates and corresponding segmentation masks and landmark coordinates are obtained through template making and template annotation; in the clinical process, the preoperative image is first obtained, and the template to be aligned is selected from the four sets of standard templates, and the centroid of the scapula region in the template to be aligned is aligned with the centroid of the scapula region in the preoperative image to obtain the aligned template, and the aligned template is subjected to affine transformation and non-rigid registration to obtain the registration result, and finally, according to the registration result, the landmark points in the template are converted to the spatial coordinate system of the corresponding landmark points in the preoperative image to obtain the coordinates of the landmark points of the patient's scapula and humerus.

[0082] The bone landmark detection device based on image registration provided by the present invention is described below. The bone landmark detection device based on image registration described below and the bone landmark detection method based on image registration described above can be referenced to each other.

[0083] Figure 3 is a structural schematic diagram of a bone landmark detection device based on image registration provided by the present invention, such as Figure 3 As shown, the bone landmark detection device based on image registration includes: a coordinate calculation module 310, an alignment module 320 and a registration module 330.

[0084] A coordinate calculation module 310 is used to determine the centroid coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes the scapula region, the scapula mask, the humerus region and the humerus mask; The alignment module 320 is used to obtain the centroid offset according to the centroid coordinates of the scapula segmentation mask and the centroid coordinates of the scapula mask in the annotation template; align the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured according to the centroid offset, and transform the coordinates of each landmark point in the annotation template to obtain the aligned annotation template; wherein the annotation template includes a scapula segmentation region, a scapula segmentation mask, a humerus segmentation region and a humerus segmentation mask; the humerus segmentation region includes a plurality of humerus landmark points, and the scapula segmentation region includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates; The registration module 330 is used to align the scapula segmentation area with the scapula area in the aligned annotation template to obtain a scapula registration result, and to align the humerus segmentation area with the humerus area in the aligned annotation template to obtain a humerus registration result; according to the scapula registration result and the humerus registration result, the coordinates of each landmark point in the aligned annotation template are spatially transformed to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0085] The bone landmark point detection device based on image registration provided by the embodiment of the present invention calculates the center of mass offset through the center of mass coordinates corresponding to the scapula mask in the bone joint image to be measured and the scapula segmentation mask in the annotation template, and aligns the origin coordinates and the landmark point coordinates of the annotation template and the bone joint image to be measured according to the center of mass offset to obtain an aligned annotation template, and finally aligns the shoulder joint area in the aligned annotation template and the shoulder joint area in the bone joint image to be measured, and then transforms the coordinates of each landmark point in the aligned annotation template into the coordinate space of the bone joint image to be measured according to the alignment result, so that the operator can quickly and accurately complete the landmark point detection, thereby improving the efficiency and accuracy of bone landmark point detection.

[0086] Figure 4 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute a bone landmark detection method based on image registration, the method comprising: determining the centroid coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes a scapula region, a scapula mask, a humeral region and a humeral mask; obtaining a centroid offset according to the centroid coordinates of the scapula segmentation mask and the centroid coordinates of the scapula mask in the annotation template; aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured according to the centroid offset, and transforming the coordinates of each landmark in the annotation template to obtain an aligned annotation template; wherein the annotation template includes Scapula segmentation region, scapula segmentation mask, humerus segmentation region and humerus segmentation mask; the humerus segmentation region includes multiple humerus landmark points, and different landmark points correspond to different coordinates; the scapula segmentation region and the scapula region in the aligned annotation template are registered to obtain a scapula registration result, and the humerus segmentation region and the humerus region in the aligned annotation template are registered to obtain a humerus registration result; according to the scapula registration result and the humerus registration result, the coordinates of each landmark point in the aligned annotation template are spatially transformed to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0087] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0088] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the bone landmark point detection method based on image registration provided by the above-mentioned methods, and the method includes: determining the center of mass coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes a scapula area, a scapula mask, a humeral area and a humeral mask; obtaining a center of mass offset according to the center of mass coordinates of the scapula segmentation mask and the center of mass coordinates of the scapula mask in the annotation template; aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured according to the center of mass offset, and The coordinates of each landmark point are converted to obtain an aligned annotation template; wherein the annotation template includes a scapula segmentation region, a scapula segmentation mask, a humerus segmentation region and a humerus segmentation mask; the humerus segmentation region includes multiple humerus landmark points, and the scapula segmentation region includes multiple scapula landmark points, and different landmark points correspond to different coordinates; the scapula segmentation region and the scapula region in the aligned annotation template are registered to obtain a scapula registration result, and the humerus segmentation region and the humerus region in the aligned annotation template are registered to obtain a humerus registration result; according to the scapula registration result and the humerus registration result, the coordinates of each landmark point in the aligned annotation template are spatially transformed to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0089] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the bone landmark point detection method based on image registration provided by the above-mentioned methods, the method comprising: determining the center of mass coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes a scapula region, a scapula mask, a humeral region and a humeral mask; obtaining a center of mass offset according to the center of mass coordinates of the scapula segmentation mask and the center of mass coordinates of the scapula mask in the annotation template; aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured according to the center of mass offset, and converting the coordinates of each landmark point in the annotation template to obtain to the aligned annotation template; wherein the annotation template includes a scapula segmentation area, a scapula segmentation mask, a humerus segmentation area and a humerus segmentation mask; the humerus segmentation area includes a plurality of humerus landmark points, and the scapula segmentation area includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates; the scapula segmentation area and the scapula area in the aligned annotation template are registered to obtain a scapula registration result, and the humerus segmentation area and the humerus area in the aligned annotation template are registered to obtain a humerus registration result; according to the scapula registration result and the humerus registration result, the coordinates of each landmark point in the aligned annotation template are spatially transformed to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

[0090] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0091] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A bone landmark detection method based on image registration, characterized in that: include: Determine the centroid coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes a scapula region, a scapula mask, a humeral region and a humeral mask; The center of mass offset is obtained according to the center of mass coordinates of the scapula segmentation mask in the annotation template and the center of mass coordinates of the scapula mask; the origin coordinates of the annotation template are aligned with the origin coordinates of the bone joint image to be measured according to the center of mass offset, and the coordinates of each landmark point in the annotation template are converted to obtain the aligned annotation template; wherein the annotation template includes a scapula segmentation area, a scapula segmentation mask, a humerus segmentation area and a humerus segmentation mask; the humerus segmentation area includes a plurality of humerus landmark points, and the scapula segmentation area includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates; The scapula segmentation area in the aligned annotation template is aligned with the scapula area to obtain a scapula alignment result, and the humerus segmentation area in the aligned annotation template is aligned with the humerus area to obtain a humerus alignment result; the coordinates of each landmark point in the aligned annotation template are spatially transformed according to the scapula alignment result and the humerus alignment result to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

2. The bone landmark detection method based on image registration according to claim 1, characterized in that: The annotation template is obtained through the following steps: Acquire multiple sets of template images by scanning the target subject from the neck to the chest on the left side of the body, from the neck to the chest on the right side of the body, from the neck to the waist on the left side of the body, and from the neck to the waist on the right side of the body using a computed tomography (CT) device; For each set of template images, a humerus segmentation mask and a scapula segmentation mask are annotated from the template images according to expert experience, and landmark points and coordinates to be identified are marked on the humerus, and landmark points and coordinates to be identified are marked on the scapula to obtain the annotated template.

3. The bone landmark detection method based on image registration according to claim 1, characterized in that: The registering the scapula segmentation region in the aligned annotation template with the scapula region to obtain a scapula registration result includes: Extracting a template scapula foreground region from the scapula segmentation region of the aligned annotation template; extracting a scapula foreground region to be measured from the scapula region; Performing an affine transformation on the template scapula foreground area and the scapula foreground area to be measured to obtain a scapula affine transformation result, and performing a non-rigid transformation on the scapula affine transformation result and the scapula foreground area to be measured to obtain the scapula registration result; The registering the humerus segmentation region in the aligned annotation template with the humerus region to obtain a humerus registration result comprises: Extracting a template humerus foreground region from the humerus segmentation region in the aligned annotation template, and extracting a humerus foreground region to be measured from the humerus region in the bone joint image to be measured; Affine transformation is performed on the template humerus foreground area and the humerus foreground area to be measured to obtain a humerus affine transformation result; non-rigid transformation registration is performed on the humerus affine transformation result and the humerus foreground area to be measured to obtain the humerus registration result.

4. The bone landmark detection method based on image registration according to claim 1, characterized in that: The humerus registration result includes a humerus transformation matrix and a humerus deformation field; the scapula registration result includes a scapula transformation matrix and a scapula deformation field; the humerus transformation matrix and the humerus deformation field are obtained by sequentially applying affine transformation and non-rigid transformation to the humerus, and the scapula transformation matrix and the scapula deformation field are obtained by sequentially applying affine transformation and non-rigid transformation to the scapula; The spatial transformation of the coordinates of each landmark point in the aligned annotation template according to the scapula registration result and the humerus registration result to obtain the target coordinates includes: The coordinates of each landmark point on the humerus in the aligned annotation template are transformed according to the humerus transformation matrix to obtain the coordinates of the humerus landmark points of the template; the coordinates of each landmark point on the scapula in the aligned annotation template are transformed according to the scapula transformation matrix to obtain the coordinates of the scapula landmark points of the template; The coordinates of the template humerus landmark points are transformed according to the humerus deformation field to obtain the coordinates of the target humerus landmark points; the coordinates of the template scapula landmark points are transformed according to the scapula deformation field to obtain the coordinates of the target scapula landmark points.

5. The bone landmark detection method based on image registration according to claim 1, characterized in that: The scapula segmentation area includes the glenoid cavity, and the humerus segmentation area includes the humeral head, humeral neck and humeral medullary cavity; the target coordinates include target humeral landmark point coordinates and target scapula landmark point coordinates; After obtaining the target coordinates, the method further includes: The glenoid prosthesis features are calculated according to the coordinates of the landmark points corresponding to the glenoid, and the humeral prosthesis features are calculated according to the coordinates of the landmark points corresponding to the humeral head, humeral neck and humeral medullary cavity; wherein the glenoid prosthesis features include the size, model and initial placement position of the glenoid prosthesis; and the humeral prosthesis features include the size, model and initial placement position of the humeral prosthesis; estimating the postoperative displacement of the scapula according to the glenoid prosthesis characteristics and the position of the scapula rotation center, and estimating the postoperative displacement of the humerus according to the humeral prosthesis characteristics and the position of the greater tuberosity of the humerus; Preoperative navigation is performed on the shoulder replacement surgery according to the postoperative offset of the scapula and the postoperative offset of the humerus.

6. The bone landmark detection method based on image registration according to claim 1, characterized in that: After obtaining the target coordinates, the method further includes: The bone surface model of the target object is determined based on the target coordinates, and when the bone surface point cloud data of the target object is collected through bone surface surgery, the bone surface point cloud data is registered with the bone surface model to obtain a registration result: The registration accuracy of each landmark in the registration result is calculated based on the target coordinates to adjust one of the surgical navigation parameters or the positioning parameters of the robot-assisted system.

7. A bone landmark detection device based on image registration, characterized in that: include: A coordinate calculation module, used to determine the centroid coordinates of the scapula mask according to the bone joint image to be measured; wherein the bone joint image to be measured includes the scapula region, the scapula mask, the humerus region and the humerus mask; An alignment module, used for obtaining a center of mass offset according to the center of mass coordinates of the scapula segmentation mask in the annotation template and the center of mass coordinates of the scapula mask; aligning the origin coordinates of the annotation template with the origin coordinates of the bone joint image to be measured according to the center of mass offset, and converting the coordinates of each landmark point in the annotation template to obtain an aligned annotation template; wherein the annotation template includes a scapula segmentation region, a scapula segmentation mask, a humerus segmentation region and a humerus segmentation mask; the humerus segmentation region includes a plurality of humerus landmark points, and the scapula segmentation region includes a plurality of scapula landmark points, and different landmark points correspond to different coordinates; A registration module is used to align the scapula segmentation area in the aligned annotation template with the scapula area to obtain a scapula registration result, and to align the humerus segmentation area in the aligned annotation template with the humerus area to obtain a humerus registration result; spatially transform the coordinates of each landmark point in the aligned annotation template according to the scapula registration result and the humerus registration result to obtain the target coordinates for navigation and planning of shoulder replacement surgery.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the bone landmark detection method based on image registration as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting bone landmarks based on image registration as claimed in any one of claims 1 to 6 is implemented.

Images