Implant implantation correction method
By selecting the target implant on a two-dimensional image of the Kirschner wire, performing masking and connected component filtering, extracting line segments, and calculating angular deviations, the problem of multiple adjustments to the Kirschner wire was solved, achieving a high-accuracy one-time successful implantation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING TUODAO MEDICAL TECHNOLOGY CO LTD
- Filing Date
- 2024-12-25
- Publication Date
- 2026-06-23
AI Technical Summary
In existing technologies, the implantation of Kirschner wires requires multiple adjustments to achieve the planned angle, which prolongs the operation time and may cause harm to the human body.
By selecting the target implant on a 2D image of the implant, mask acquisition and candidate connected component filtering are performed, candidate line segments are extracted, line segments are combined and implantation angle deviation is calculated, and angle adjustment is prompted to achieve successful implantation in one attempt.
It achieves high accuracy, low false negative rate and low false positive rate in implant identification in complex scenarios, ensuring that the implant reaches the planned angle on the first attempt, reducing operation time and harm to the human body.
Smart Images
Figure CN119850560B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and more particularly to an implantation correction method. Background Technology
[0002] Kirschner wires are widely used medical devices in surgical procedures, with a wide range of applications. They can be used not only as implants, such as to fix fractures and aid healing, but also as tools, such as guides for subsequent drilling and screw insertion, improving success rates.
[0003] As a guide tool for metal implantation, Kirschner wires need to be inserted into the body in advance at a planned angle to reach the designated position. If it is found to be inconsistent with the plan after implantation, it needs to be removed and re-implanted until the requirements are met. Multiple implantations not only prolong the operation time but may also cause greater harm to the human body. Summary of the Invention
[0004] Purpose of the invention: To address the above-mentioned shortcomings, this invention proposes an implantation correction method. When an implant is inserted, it is identified and extended at both ends to calculate the deviation between the implantation angle and the planned angle. When the deviation exceeds the limit range, the method prompts for angle adjustment, thereby achieving successful implantation on the first attempt.
[0005] Technical solution:
[0006] This invention provides a method for correcting implant placement, comprising the following steps:
[0007] (1) Select the target implant on the original two-dimensional image containing the implant to obtain its mask;
[0008] (2) Obtain and filter candidate connected components that pass through the mask region in the original two-dimensional image, extract their corresponding candidate binary outer contours, perform line search on the candidate binary outer contours to obtain several single line segments, and filter to obtain candidate line segments.
[0009] (3) Combine the candidate line segments obtained in step (2), calculate the intersection of each line segment combination with the mask area, and retain the line segment combination with the longest total length as the line segment combination corresponding to the target implant.
[0010] (4) Draw all the line segments in the combination of line segments obtained in step (3) on the binary image, obtain the smallest bounding rectangle that contains all the line segments, extend the smallest bounding rectangle along the length direction until it intersects the edge of the binary image, subtract it from the candidate connected component to obtain the target connected component, obtain the smallest bounding rectangle of the target connected component, and use the midpoint of the short side of the smallest bounding rectangle as the endpoint of the extension line to be drawn.
[0011] (5) Connect the endpoints of the extension line obtained in step (4), and calculate the deviation between the implantation angle and the planned angle of the implant. When the deviation exceeds the limit, it is corrected.
[0012] Specifically, in step (2), obtaining and filtering the candidate connected components in the mask involves:
[0013] (21) Obtain the gradient binary image corresponding to the original two-dimensional image, and perform connected component search to obtain several connected components;
[0014] (22) Perform intersection calculations between each connected component obtained in step (21) and the mask region, and retain the connected components in which the intersection pixels are greater than the first threshold.
[0015] (23) Perform line detection on the gradient binary image to obtain several candidate line segments, calculate the intersection of the line segments with the connected components retained in step (22), and retain the connected components whose intersection pixels are greater than the second threshold as candidate connected components.
[0016] More specifically, step (21) includes:
[0017] Obtain the gradient binary image corresponding to the original two-dimensional image, and perform connected component search and filtering on the gradient binary image to obtain the first filtered image;
[0018] The original two-dimensional image is binarized, and then connected component search and filtering are performed on it. The intersection of this filtered image with the first filtered image is then taken to obtain the second filtered image, which in turn yields several connected components.
[0019] Furthermore, obtaining the gradient binary image corresponding to the original two-dimensional image specifically involves:
[0020] Gradient detection is performed on the original two-dimensional image to obtain a gradient image, and an adaptive binarization method is used to binarize the gradient image to obtain the corresponding gradient binary image.
[0021] Furthermore, the condition for filtering connected components is: removing connected components in the binary image obtained after binarization that have a width and height of less than 5 pixels, are within 10 pixels from the image edge, or have an area of less than 100.
[0022] More specifically, in step (23), before performing line detection on the gradient binary image, an edge detection step is also included.
[0023] More specifically, in step (2), the extraction of candidate binary outer contours corresponding to candidate connected components is as follows:
[0024] Calculate the intersection of each candidate connected component with each candidate line segment, calculate the ratio between the area of the intersection and the area of the corresponding candidate connected component, retain the candidate connected components whose ratio is greater than a set value, and use their outer contours as candidate binary outer contours.
[0025] Specifically, in step (2), the selection of candidate line segments is as follows:
[0026] The slope range of all single line segments is statistically analyzed to obtain the slope distribution range corresponding to the single line segments with a set proportion. Single line segments outside the slope distribution range are removed to obtain candidate line segments.
[0027] Specifically, in step (3), when combining the candidate line segments, it is determined whether the two candidate line segments belong to the same implant based on whether the distance between the extension lines of the two candidate line segments in the mask area is within a set distance. If it is, then the two candidate line segments belong to the same implant; otherwise, they do not.
[0028] Specifically, in step (1), after selecting the target implant, the step further includes removing the area outside the selected area within a set range.
[0029] Beneficial Effects: This invention identifies and extends the implant by selecting the target area and replacing it with a set of line segments. This method is suitable for complex clinical applications involving significant differences in image brightness or contrast, various metal adhesions and intersections, and bent implants. It simultaneously meets the requirements of high accuracy, low false negative rate, and low false positive rate. The invention identifies the implant during implantation and then extends it at both ends to calculate the deviation between the implantation angle and the planned angle. When the deviation exceeds a certain range, an angle adjustment prompt is provided, achieving successful implantation on the first attempt. Attached Figure Description
[0030] To more clearly illustrate the technical solutions in this invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0031] Figure 1 This is a flowchart illustrating the implantation correction method of the present invention.
[0032] Figure 2 To obtain the original two-dimensional image containing Kirschner wires;
[0033] Figure 3 In order to be in Figure 2 Example diagram of Kirschner wire selection;
[0034] Figure 4 for Figure 3 Binary image of the masked region selected by the Kirschner wire frame;
[0035] Figure 5 To Figure 2 Example graph obtained by gradient lookup;
[0036] Figure 6 To Figure 5 Example image of the binarization result;
[0037] Figure 7 To Figure 6 Example image of the first filtered image obtained by contour filtering;
[0038] Figure 8 To Figure 2 Example image of the binarization result;
[0039] Figure 9 To Figure 8 Perform contour filtering and match Figure 7 Example image of the second filtered image obtained by performing intersection calculation;
[0040] Figure 10 To Figure 2 Example image of the results obtained from edge detection;
[0041] Figure 11 To Figure 10 Example image of the results obtained from line detection;
[0042] Figure 12 Example graph of candidate connected components;
[0043] Figure 13 Example image of candidate contours;
[0044] Figure 14 Based on Figure 13 Example image showing the results of line detection and single-segment filtering and extraction;
[0045] Figure 15 To Figure 14 Example image of the target Kirschner wires obtained through screening;
[0046] Figure 16 Based on Figure 15 Example diagram of finding the endpoint of the extension line of the target Kirschner wire;
[0047] Figure 17 This is an example diagram of the extension line of the Kirschner wire. Detailed Implementation
[0048] To make the objectives, technical solutions and advantages of the present invention clearer, the present application will be further described in detail below with reference to specific embodiments and accompanying drawings.
[0049] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of the present invention should have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.
[0050] The implantation correction method of the present invention is as follows: Figure 1 As shown, the steps include:
[0051] S1. Select the target implant on the original two-dimensional image containing the implant, and fill the selected area to obtain a mask;
[0052] Obtain a raw two-dimensional image containing at least one implant. In a specific embodiment of the present invention, the implant is a Kirschner wire, such as... Figure 2 As shown, a closed bounding box is used to select the target implant on the original 2D image. Further, a rectangular bounding box can be used to select a portion of the target implant, such as... Figure 3 As shown, the selected area is filled to obtain a mask;
[0053] Specifically, the process of filling the selected area to obtain a mask involves setting the pixel value of pixels within the selected area to 255 and the pixel value of pixels outside the selected area to 0, thereby obtaining the corresponding mask. Figure 4 As shown.
[0054] Furthermore, in order to remove edge interference, in this invention, a region outside the selected area is removed within a set range. Specifically, the set range is a region 10 pixels wide outside the selected area.
[0055] Generally, in this invention, there is usually only one implant within the selected area. The selected area may contain other interference, but when selecting the area, it is necessary to ensure that the foreground of the target implant occupies the largest proportion to improve the overall accuracy of implant recognition.
[0056] In addition, if the target implant is bent, the selection should be as close as possible to the end of the target implant to be extended. With this design, the slope of the line segment actually retained can be similar to the slope of the extension line of the target implant to be extended, thus solving the problem of inaccurate extension of the implant when it is bent.
[0057] If the target implant is not selected, all metal implants in the image need to be automatically identified and extended, which increases the complexity of the algorithm and makes it impossible to simultaneously achieve a low false negative rate and a high recognition accuracy. Furthermore, too many detected targets will interfere with the user's vision, and in the future, the user will most likely still need to specify and only display the extension line of a certain implant.
[0058] S2. Perform gradient detection on the original two-dimensional image obtained in S1, and binarize it to obtain the corresponding gradient binary image. Perform connected component search and filtering on the gradient binary image to obtain the first filtered image.
[0059] In this invention, the Sobel operator is used to perform gradient detection on the original two-dimensional image acquired by S1, such as... Figure 5 As shown, it can produce good edge detection results and find the edges of Kirschner wires when the edges are relatively indistinct, thereby reducing the false negative rate.
[0060] In this invention, during actual clinical surgery, the exposure imaging conditions and the location of the implant are unknown. In this scenario, the intensity values of each pixel in the gradient image obtained from gradient detection vary significantly. If a fixed threshold binarization method such as OTSU is used, the resulting target edge image may be fragmented, increasing the difficulty for subsequent algorithms. Therefore, this invention employs an adaptive binarization method, determining the segmentation threshold for each pixel by a weighted average of the gradient intensities in the neighborhood of each pixel. Figure 6 As shown, a more complete target edge can be obtained, which facilitates the search for candidate lines. In this invention, the kernel function size of the adaptive binarization method can be 3, 5, or 7; in this embodiment, 5 is preferred.
[0061] In this invention, the gradient binary image is subjected to connected component search and filtering to remove interference from regions with small width and height dimensions, located at image edges, or with small contour areas. A first filtered image is obtained by drawing with coarse edges, such as... Figure 7 As shown, the present invention uses a coarse edge design to ensure that the actual edge is preserved.
[0062] In this invention, the filtering conditions for connected component search and filtering of the gradient binary image obtained in S2 are: width and height dimensions less than 5 pixels, within 10 pixels of the image edge, or contour area less than 100. Connected components meeting these conditions need to be removed. This setting reduces missed detections. Furthermore, this invention can further reduce these conditions, such as width and height dimensions less than 3 pixels, within 0 pixels of the edge, or contour area less than 10, but this will correspondingly increase the subsequent filtering time.
[0063] S3. The original two-dimensional image obtained in S1 is binarized, and after connecting component search and filtering, the intersection with the first filtered image is taken to obtain the second filtered image.
[0064] In this invention, the binarization of the original two-dimensional image acquired by S1 can also be performed using adaptive binarization, such as... Figure 8 As shown, specifically, the kernel function for adaptive binarization in this step can be an odd number between 9 and 17, and in this embodiment, 15 is preferred.
[0065] In this invention, connected component search and filtering are performed on the original two-dimensional image after binarization to remove interference that has small width and height dimensions, is located at the image edge, or has a small connected component area. The conditions for contour search and filtering can be the same as those in S2.
[0066] In this invention, after performing contour search and filtering on the original two-dimensional image after binarization, the intersection of this image with the first filtered image is taken to obtain the second filtered image, as shown below. Figure 9 As shown, the second filtered image obtained needs to meet the following condition: the number of pixels in the intersection is greater than a set value; in this invention, the set value can be 50-100, and in this embodiment, it is preferred to be 100.
[0067] S4. Calculate the intersection of each connected component in the second filtered image obtained in S3 with the mask region obtained in S1, and retain the connected components in which the intersection pixels are greater than the first threshold.
[0068] In this invention, the first threshold can be 20-100, and in this embodiment, it is preferably 100.
[0069] S5. Perform edge detection on the gradient binary image obtained in S2, and perform line detection to obtain several candidate line segments. Calculate the intersection of the connected components obtained in S4 with the aforementioned candidate line segments, and retain the connected components where the number of pixels in the intersection is greater than the second threshold as candidate connected components. Figure 12 As shown;
[0070] In this invention, edge detection of the gradient binary image obtained from S2 can be performed using the Canny algorithm, such as... Figure 10 As shown; furthermore, the setting of low and high thresholds in the Canny algorithm can ensure that there is a large threshold range, thereby reducing the false negative rate.
[0071] In this invention, line detection can employ a Hough transform based on cumulative probability, thereby obtaining a series of line segments, such as... Figure 11 As shown.
[0072] In this invention, the second threshold can be between 20 and 100, and in this embodiment, it is preferably 100.
[0073] S6. Calculate the intersection of each candidate connected component with each candidate line segment obtained in S5. Calculate the ratio between the area corresponding to the intersection and the area of the corresponding candidate connected component. Keep the candidate connected components whose ratio is greater than the set ratio and use their outer contours as candidate binary outer contours. If there are no candidate binary outer contours, return to S1.
[0074] In this invention, after obtaining candidate connected components that meet the requirements, the outer contours of the aforementioned candidate connected components can be redrawn on a new image, thereby obtaining candidate binary outer contours, such as... Figure 13 As shown.
[0075] In this invention, the area of a line segment is the number of pixels in the region corresponding to that line segment.
[0076] In this embodiment, the ratio can be set to 0.2-0.4, preferably 0.4.
[0077] In this invention, there is generally one and only one candidate binary outer contour; otherwise, a pop-up prompt is displayed to prompt the user to re-select the target implant on the original two-dimensional image obtained in S1. At this time, the following possibilities exist: the selected area does not contain the target implant, the selected area is too large or too small, or the selected area contains too much interference. In this case, a relatively clean area can be selected and the selection can be re-selected, i.e., return to S1.
[0078] S7. Perform a line search on the candidate binary outer contour obtained in S6 to obtain several single line segments. Filter to obtain candidate line segments. If there are no at least two single line segments among the candidate line segments, return to S1.
[0079] Taking the Kirschner wire as an example, an implant has at least two line segments located on both sides of the Kirschner wire. These line segments belong to the connected region corresponding to the same binary outer contour in the binary image, and the slopes of the line segments are similar. If there are no at least two single line segments, a pop-up window prompts the user to re-select the target implant on the original two-dimensional image obtained by S1, i.e., return to S1.
[0080] In this invention, after obtaining several single line segments, the slope range of all single line segments is statistically analyzed. Based on the principle of majority rule, a retention ratio can be set according to actual needs. Based on the aforementioned statistical slope range of all single line segments and the retention ratio, single line segments with slopes significantly different from the majority of single line segments (those outside the retention ratio) are removed. These single line segments may be due to interference from the threshold setting during line detection. Candidate line segments are obtained, where different line segments are represented by different colors, such as... Figure 14 The blue and purple line segments in the diagram.
[0081] S8. Combine the candidate line segments obtained in S7, calculate the intersection of each line segment combination with the mask obtained in S1, and retain the line segment combination with the longest total line segment length as the line segment combination corresponding to the target implant.
[0082] In this invention, when combining candidate line segments, the thickness of the implant, such as a Kirschner wire, can be determined empirically. When the extended lines on both sides of the Kirschner wire are within the mask, the distance between corresponding line segments on the extended lines should be within a set range. Therefore, based on whether the distance between the extended lines of two candidate line segments in the mask area is within the set distance, it can be determined whether the two candidate line segments belong to the same implant. If they are, then the two candidate line segments belong to the same implant; otherwise, they do not. The set distance depends on the detection content. If only thinner implants are detected, the set range can be set smaller (e.g., 1-20mm) based on prior information. If other relatively thicker implants need to be identified, the set range can be set larger (e.g., 1-40mm).
[0083] In this embodiment, although only one candidate contour is retained, the adhesion between metals means that a single line segment in the found contour may belong to multiple objects. Therefore, different line segments belonging to the same object can be combined based on the slope information. The intersection of each line segment combination with the mask obtained in S1 is calculated, and the combination with the longest total line segment length is retained as the line segment combination corresponding to the target implant. Figure 15 As shown.
[0084] In this invention, there is generally at least one set of line segments; otherwise, a pop-up prompt is displayed to remind the user to re-select the target implant on the original two-dimensional image obtained in S1, i.e., return to S1.
[0085] S9. Draw all line segments in the combination of line segments retained in S8 on the binary image. Obtain the minimum bounding rectangle containing all line segments. Extend the minimum bounding rectangle along the length direction until it intersects the edge of the binary image, obtaining an image region. Subtract the connected components corresponding to the candidate contours obtained in S6 from this image region, perform morphological operations on it, and then perform contour search to obtain the target connected components connected to the image edge. Obtain the minimum bounding rectangle of each target connected component. The midpoint of the short side of the minimum bounding rectangle is the endpoint of the extension line to be drawn. Example: Figure 16 As shown.
[0086] In this invention, there is generally at least one set of endpoints; otherwise, a pop-up prompt is displayed to remind the user to re-select the target implant on the original two-dimensional image obtained in S1, i.e., return to S1.
[0087] S10, Connect the endpoints of the extension line obtained from S9, for example... Figure 17 As shown, the deviation between the implantation angle and the planned angle is calculated. When the deviation exceeds the limit, the implantation angle is adjusted to achieve successful implantation on the first attempt.
[0088] In this invention, there may be a situation where the mask obtained in S1 is very close to the edge of the image. In this case, the difference between the aforementioned image region and the connected component corresponding to the candidate contour obtained in S6 is calculated. The length of the resulting connected component may be very short, that is, only a small area at the edge of the image. This part of the area does not meet the aforementioned set length range and can be eliminated. That is, the connected component in the target connected component obtained in S9 that is close to the image edge and whose distance between the midpoints of the two short sides of the smallest bounding rectangle, i.e., the endpoints of the two extended lines, does not exceed the set value is eliminated. The final target connected component is obtained, and the deviation between the implantation angle and the planned angle of the implant is calculated using the endpoints of its corresponding extended lines.
[0089] This invention identifies and extends the implant by selecting the target area and replacing it with a set of line segments. It is suitable for complex clinical applications involving significant differences in image brightness or contrast, various metal adhesions and intersections, and bent implants. It simultaneously meets the requirements of high accuracy, low false negative rate, and low false positive rate. The invention identifies the implant during implantation and then extends it at both ends to calculate the deviation between the implantation angle and the planned angle. When the deviation exceeds a certain range, it prompts the implant angle to be adjusted, achieving successful implantation on the first attempt.
[0090] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of the invention as described above, which are not provided in the details for the sake of brevity.
[0091] The embodiments of this invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this invention should be included within the protection scope of this invention.
Claims
1. A method for correcting implant placement, characterized in that, Including the following steps: (1) Select the target implant on the original two-dimensional image containing the implant to obtain a mask of the selected area; (2) Obtain and filter candidate connected regions in the original two-dimensional image that pass through the mask region, extract candidate binary outer contours of the candidate connected regions, perform line search on the candidate binary outer contours to obtain single line segments, perform distribution statistics on the slope range of all single line segments to obtain the slope distribution range corresponding to the single line segments with a set proportion, remove single line segments outside the slope distribution range to obtain candidate line segments; (3) Determine whether the two candidate line segments belong to the same implant based on whether the distance between the extension lines of the two candidate line segments in the mask area is within the set distance. If they are, the two candidate line segments belong to the same implant; otherwise, they do not. Combine the candidate line segments that belong to the same implant. The intersection of each line segment combination with the mask area is calculated, and the line segment combination with the longest total length is retained as the line segment combination corresponding to the target implant. (4) Draw all the line segments in the combination of line segments obtained in step (3) on the binary image, obtain the smallest bounding rectangle that contains all the line segments, extend the smallest bounding rectangle along the length direction until it intersects with the edge of the binary image, subtract it from the candidate connected component to obtain the target connected component, obtain the smallest bounding rectangle of the target connected component, and use the midpoint of the short side of the smallest bounding rectangle as the endpoint of the extension line to be drawn; (5) Connect the endpoints of the extension line obtained in step (4), and calculate the deviation between the implantation angle and the planned angle of the implant. When the deviation exceeds the limit, it is corrected.
2. The implantation correction method according to claim 1, characterized in that, In step (2), the acquisition and filtering of candidate connected components in the masked region of the original two-dimensional image specifically involves: (21) Obtain the gradient binary image corresponding to the original two-dimensional image, and perform connected component lookup to obtain the connected components; (22) Perform intersection calculations on each connected component obtained in step (21) and the mask region, and retain the connected components in which the intersection pixels are greater than the first threshold. (23) Perform line detection on the gradient binary image to obtain candidate line segments, calculate the intersection of the candidate line segments with the connected components retained in step (22), and retain the connected components whose intersection pixels are greater than the second threshold as candidate connected components.
3. The implantation correction method according to claim 2, characterized in that, Step (21) includes: Obtain the gradient binary image corresponding to the original two-dimensional image, and perform connected component search and filtering on the gradient binary image to obtain the first filtered image; The original two-dimensional image is binarized, and its connected components are searched and filtered. The intersection of this filtered image with the first filtered image is then taken to obtain the second filtered image, which is the connected component obtained therein.
4. The implantation correction method according to claim 3, characterized in that, The specific steps for obtaining the gradient binary image corresponding to the original two-dimensional image are as follows: Gradient detection is performed on the original two-dimensional image to obtain a gradient image, and an adaptive binarization method is used to binarize the gradient image to obtain the corresponding gradient binary image.
5. The implantation correction method according to claim 3, characterized in that, The conditions for filtering connected components are: removing connected components in the binary image obtained after binarization that have a width and height of less than 5 pixels, are within 10 pixels from the image edge, or have an area of less than 100 pixels.
6. The implantation correction method according to claim 2, characterized in that, In step (23), before performing line detection on the gradient binary image, an edge detection step is also included.
7. The implantation correction method according to claim 1, characterized in that, In step (2), the extraction of candidate binary outer contours of candidate connected components is specifically as follows: Calculate the intersection of each candidate connected component with each candidate line segment, calculate the ratio between the area of the intersection and the area of the corresponding candidate connected component, retain the candidate connected components whose ratio is greater than a set value, and use their outer contours as candidate binary outer contours.
8. The implantation correction method according to claim 1, characterized in that, In step (1), after selecting the target implant, the step further includes removing the area outside the selected area within a set range.