A bone stitching method based on X-ray images

Abstract

The application discloses an X-ray image bone splicing method, which comprises the following steps: acquiring X-ray images to be spliced; pre-processing the X-ray images to remove preset information in the X-ray images; converting the processed X-ray images into binary images; extracting bone contours in the binary images; segmenting the bone contours from muscles and other tissues to obtain bone contour lines; extracting feature points in the bone contour lines; extracting key points in the feature points; using the Euclidean distance of the key point feature vectors as a similarity judgment measurement of the key points in two images to be spliced; obtaining a transformation matrix by using an improved RANSAC algorithm to remove error matching points, registering the images through the transformation matrix, obtaining a corresponding coordinate system, splicing the images, and obtaining a registered image; and judging whether the accuracy of the registered image meets the requirements, and if not, adjusting the position of the spliced image.

Description

Technical Field

[0001] This invention relates to the field of medical technology, specifically to a method for bone stitching based on X-ray images. Background Technology

[0002] X-ray equipment has a limited field of view (FOV), and for longer bones in the human body, multiple scans are required to obtain a complete image. Doctors need to review different images separately to gain a comprehensive understanding of the entire bone structure during diagnosis or surgical planning, which is inconvenient for doctors.

[0003] For stitching different images, current methods mainly rely on grayscale theory to verify grayscale correlation in order to achieve image stitching.

[0004] While existing methods can stitch together different images, they face difficulties in stitching together images with large distortions, discontinuities, and significant variations in brightness. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a method for bone stitching of X-ray images.

[0006] This invention provides a method for bone stitching in X-ray images, the method comprising:

[0007] S1: Acquire the X-ray images to be stitched;

[0008] S2: Preprocess the X-ray image to remove preset information from the X-ray image;

[0009] S3: Convert the processed X-ray image into a binary image;

[0010] S4: Extract the skeletal contours from the binary image;

[0011] S5: Separate the skeletal outline from the muscles and other tissues to obtain the skeletal outline line;

[0012] S6: Extract feature points from the skeletal outline;

[0013] S7: Extract key points from feature points;

[0014] S8: The Euclidean distance of the key point feature vectors is used as the similarity metric for key points in two images to be stitched. When the distance between two key points in two images to be stitched is less than the threshold, they are judged as a pair of matching points; otherwise, they are considered as mismatched points and are removed.

[0015] S9: After removing erroneous matching points using the improved RANSAC algorithm, the transformation matrix is ​​obtained. The image is then registered using the transformation matrix to obtain the corresponding coordinate system. Finally, the images are stitched together to obtain the registered image.

[0016] S10: Determine whether the accuracy of the registered image meets the requirements. If it does not meet the requirements, adjust the position of the stitched image.

[0017] In one embodiment, removing preset information from the X-ray image includes removing image noise and artifacts from the X-ray image.

[0018] In one embodiment, extracting the skeletal contour from the binary image includes: extracting the skeletal contour from the binary image using edge detection and connectivity analysis algorithms.

[0019] In one embodiment, segmenting the skeletal contour from muscles and other tissues to obtain the skeletal contour line includes:

[0020] Threshold segmentation and edge detection are used to separate the extracted bone contour from other tissues to obtain the bone contour line.

[0021] In one embodiment, the extraction of feature points in the skeletal contour includes: constructing the DOG scale space of the skeletal contour using the SIFT algorithm;

[0022] The extreme points in the DOG scale space are detected. When an extreme point is the maximum or minimum value in the current layer and the two layers above and below the DOG scale space, it is determined to be a feature point.

[0023] In one embodiment, the extraction of key points from feature points includes: establishing a spatial scale function based on the DOG scale space, taking the derivative of the spatial scale function, and setting it to zero to obtain the location of the key points;

[0024] Remove low-contrast key points and unstable edge response points;

[0025] The location and scale of key points are accurately determined using a three-dimensional quadratic function.

[0026] In one embodiment, the improved RANSAC algorithm includes: S901: randomly selecting four pairs of nonlinear feature point matching pairs from the coarse matching results to form a set M;

[0027] S902: Calculate the homography matrix H based on set M;

[0028] S903: Verify all matching pairs in the coarse matching results using the homography matrix H, and add the point pairs that are less than the threshold to the set M;

[0029] S904: Determine whether the number of point pairs in set M has increased. If it has increased, return to step S902 to continue execution; if set M remains unchanged, proceed to the next step.

[0030] S905: If the number of point pairs in set M is greater than the number of point pairs in the current optimal homography matrix, then update the current homography matrix; otherwise, do not update it.

[0031] S906: Update the total number of iterations based on the number of interior points of the current optimal homography matrix. If the current number of iterations is less than the total number of iterations, return to step S901; otherwise, the current optimal homography matrix is ​​the final result.

[0032] In one embodiment, step S9, the stitching of the images to obtain the registered image, includes:

[0033] The obtained coordinate system is mapped to the image coordinates and then stitched together to obtain a complete skeletal image.

[0034] In one embodiment, determining whether the accuracy of the registered image meets the requirements, and if not, adjusting the position of the stitched image, includes:

[0035] The stitched images are displayed separately, with the upper image adjusted to 50%-55% transparency.

[0036] Observe the accuracy between the upper and lower images. If it does not meet the requirements, manually adjust the position of the stitched images.

[0037] The beneficial effects of this invention include:

[0038] This invention provides a method for stitching bones in X-ray images. By processing the images to be stitched, the method extracts bone contours and feature points, thereby reducing matching complexity. An improved RANSAC algorithm is used to remove mismatches, further enhancing the accuracy of feature point matching. The method calculates the transformation relationship of the image by determining the positional relationships of the image feature points, and then stitches the images according to this transformation relationship, effectively solving the problem of stitching discontinuous images. This method is applicable to medical imaging, orthopedics, and radiology, and can quickly and accurately stitch together a complete bone image, bringing great convenience to doctors. Attached Figure Description

[0039] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0040] Figure 1 A schematic flowchart of the X-ray image bone stitching method provided in an embodiment of the present invention is shown;

[0041] Figure 2 A flowchart illustrating another X-ray image bone stitching method provided in an embodiment of the present invention is shown;

[0042] Figure 3This diagram illustrates the scale-space extreme point search provided by an embodiment of the present invention.

[0043] Figure 4 A schematic diagram of the SIFT feature point extraction results provided by an embodiment of the present invention is shown;

[0044] Figure 5 A flowchart illustrating the improved RANSAC algorithm provided in an embodiment of the present invention is shown.

[0045] Figure 6 A schematic diagram of the matching and splicing results provided by an embodiment of the present invention is shown. Detailed Implementation

[0046] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0047] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0048] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0049] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0050] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0051] It should be noted that when an element is referred to as being "fixed to" or "set on" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementation.

[0052] Please refer to Figure 1 , Figure 1 This invention illustrates an X-ray image bone stitching method provided by an embodiment of the present invention, the method comprising:

[0053] S1: Acquire the X-ray images to be stitched;

[0054] S2: Preprocess the X-ray image to remove preset information from the X-ray image;

[0055] S3: Convert the processed X-ray image into a binary image;

[0056] S4: Extract the skeletal contours from the binary image;

[0057] S5: Separate the skeletal outline from the muscles and other tissues to obtain the skeletal outline line;

[0058] S6: Extract feature points from the skeletal outline;

[0059] S7: Extract key points from feature points;

[0060] S8: The Euclidean distance of the key point feature vectors is used as the similarity metric for key points in two images to be stitched. When the distance between two key points in two images to be stitched is less than the threshold, they are judged as a pair of matching points; otherwise, they are considered as mismatched points and are removed.

[0061] S9: After removing erroneous matching points using the improved RANSAC algorithm, the transformation matrix is ​​obtained. The image is then registered using the transformation matrix to obtain the corresponding coordinate system. Finally, the images are stitched together to obtain the registered image.

[0062] S10: Determine whether the accuracy of the registered image meets the requirements. If it does not meet the requirements, adjust the position of the stitched image.

[0063] Specifically, such as Figure 1 and combined Figure 2 As shown, in step S1, the X-ray images to be stitched are acquired, including: acquiring X-ray plain film image data in DICOM format, for example, acquiring left and right images in DICOM format. DICOM (Digital Imaging and Communications in Medicine) is an international standard for medical images and related information. It defines a medical image format that can be used for data exchange and whose quality meets clinical needs.

[0064] In step S2, the X-ray image is preprocessed to remove preset information from the X-ray image, including processing the obtained X-ray film image data to remove image noise and artifacts from the X-ray film image. Since the technology for removing image noise and artifacts from X-ray film images is existing technology, it will not be described in detail here.

[0065] In step S3, the processed X-ray image is converted into a binary image, that is, each pixel in the X-ray image conversion is converted into a black or white image.

[0066] In step S4, the skeletal contours in the binary image are extracted, including extracting the skeletal contours from the binary image using edge detection and connectivity analysis algorithms. Edge detection is a fundamental problem in image processing and computer vision; its purpose is to identify points in a digital image where brightness changes significantly.

[0067] In step S5, the skeletal contour is segmented from muscles and other tissues to obtain the skeletal contour line. This includes using threshold segmentation and edge detection to separate the extracted skeletal contour from other tissues to obtain the skeletal contour line.

[0068] In step S6, feature points are extracted from the skeletal contour line. The feature point extraction results are as follows: Figure 4 As shown, the extraction method includes constructing the DOG scale space of the skeletal contour using the SIFT algorithm; detecting the extreme points in the DOG scale space; and determining a feature point when an extreme point is the maximum or minimum value in the preset neighborhood of the current layer and the two layers above and below in the DOG scale space.

[0069] Specifically, (1) the generation of scale space

[0070] The purpose of scale space theory is to simulate the multi-scale features of images. The Gaussian convolution kernel is the only linear kernel that realizes scale transformation. The scale space of a two-dimensional image is defined as: L(x, y, σ) = G(x, y, σ) * I(x, y), where G(x, y, σ) is a scale-variable Gaussian function. (x, y) are spatial coordinates, and σ· is the scale coordinate, which determines the smoothness of the image.

[0071] The Gaussian difference scale space is constructed as follows:

[0072] D(x,y,σ)=(G(x,y,kσ)-G(x,y,σ))*I(x,y)=L(x,y,kσ)-L(x,y,σ)

[0073] (2) Detecting scale space extreme points

[0074] To find the extreme points in scale space, each sampled point is compared with its neighbors to see if it is larger or smaller than the contiguous points in both the image domain and scale domain. For example... Figure 3 As shown, the detection point in the middle is compared with its 8 neighboring points at the same scale and 9 x 2 points corresponding to the adjacent scales above and below, a total of 26 points, to ensure that extreme points can be detected in both scale space and two-dimensional image space. If a point is the maximum or minimum value in the 26 neighborhoods of the DOG scale space in the current layer and the two layers above and below, it is considered to be a feature point at that scale of the image.

[0075] For example, a 16*16 neighborhood centered on the feature point is obtained as a sampling window. The relative directions between the sampling point and the feature point are weighted by Gaussian and then incorporated into a direction histogram containing 8 bins to obtain a 128-dimensional feature descriptor. The algorithm process is as follows.

[0076] Determine the image region required for calculating the descriptor

[0077] The descriptor gradient orientation histogram is generated from the blurred image at the scale of the keypoints, and the radius of the image region is calculated as follows:

[0078]

[0079] (1) Align the main directions of key points with consistent coordinates

[0080] The rotated coordinates are represented as follows:

[0081]

[0082] (2) Calculate the gradient magnitude and direction for each pixel within the image radius region, and then multiply each gradient magnitude by a Gaussian weight parameter to generate a direction map, as shown in the following formula:

[0083]

[0084] (3) Calculate an 8-direction histogram within a 2x2 window area, plot the cumulative gradient direction values ​​to form seeds, and calculate in sequence to generate a total of 16 seed points.

[0085] (4) Descriptor element gateification and gated descriptor normalization.

[0086] Descriptor vector element normalization:

[0087] W = (w1, w2, ..., w 128 ) represents the resulting 128 descriptor vectors.

[0088] L = (l1, l2, ..., l) 128 () is a normalized vector

[0089]

[0090] In step S7, key points are extracted from the feature points, including: establishing a spatial scale function based on the DOG scale space, taking the derivative of the spatial scale function and setting it to zero to obtain the location of the key points; removing low-contrast key points and unstable edge response points; and accurately determining the location and scale of the key points through a three-dimensional quadratic function.

[0091] For example:

[0092] a. Spatial scale function

[0093]

[0094] Take the derivative and set it to 0 to obtain the precise position.

[0095]

[0096] b. Among the detected feature points, remove low-contrast feature points and unstable edge response points, substitute (2) into (1), and take the first two terms:

[0097]

[0098] like The feature is retained if it is not retained, otherwise it is discarded.

[0099] c. Edge response removal: The principal curvature is calculated using a 2x2 Hessian matrix H.

[0100] The derivative is estimated by interpolation between adjacent sampling points, and finally the location and scale of the key points are accurately determined by fitting a three-dimensional quadratic function.

[0101] In step S9, after removing erroneous matching points using the improved RANSAC algorithm, a transformation matrix is ​​obtained. The image is then registered using the transformation matrix to obtain the corresponding coordinate system, and the images are stitched together to obtain the registered image. Figure 5 As shown, this includes randomly selecting four pairs of nonlinear feature point matching pairs from the coarse matching results to form a set M;

[0102] Calculate the homography matrix H based on set M;

[0103] All matching pairs in the coarse matching results are verified using the homography matrix H, and pairs with values ​​less than a threshold are added to set M.

[0104] Determine whether the number of point pairs in set M has increased. If it has increased, return to step S902 to continue execution; if set M remains unchanged, proceed to the next step.

[0105] If the number of point pairs in set M is greater than the number of point pairs in the current optimal homography matrix, then the current homography matrix is ​​updated; otherwise, it is not updated.

[0106] The total number of iterations is updated based on the number of interior points of the current optimal homography matrix. If the number of current iterations is less than the total number of iterations, the process returns to step S901; otherwise, the current optimal homography matrix is ​​the final result.

[0107] To improve the accuracy of the homography matrix, a suitable threshold must be selected. Since the feature points obtained in SIFT feature point detection are approximate rather than exact pixel values, an allowable error exists. As the SIFT algorithm shows, the difference between this approximation and the exact value is within one pixel. Therefore, the threshold can be set to 1, changing the method for determining inliers from Euclidean distance to area-based determination.

[0108] d 2 x+d 2 y <= 1

[0109] Revised to:

[0110]

[0111] Where: dx is the difference between the actual value and the calculated value on the x-axis, and dy is the difference between the actual value and the calculated value on the y-axis.

[0112] Finally, the optimal matching point and the corresponding H transformation matrix are obtained. For adjacent images, the two images are registered using the obtained H transformation matrix. Through bilinear interpolation, the corresponding positions of them in the same coordinate system are found.

[0113] In step S9, it is determined whether the accuracy of the registered image meets the requirements. If it does not, the position of the stitched image is adjusted, including mapping the registered coordinate system to the image coordinates for stitching, to obtain a complete skeletal image. Figure 6 As shown. Then display the stitched images separately, adjusting the transparency of the top image to 50% to observe the registration accuracy. If you are not satisfied with the stitching accuracy, make manual fine-tuning until you are satisfied.

[0114] The above technical solution reduces matching complexity by processing the images to be stitched, extracting skeletal contours and feature points; it further improves the accuracy of feature point matching by removing mismatches through an improved RANSAC algorithm; and it solves the image transformation relationship by calculating the positional relationships of image feature points, then stitches the images together based on this transformation relationship, effectively solving the problem of stitching discontinuous images. This method is applicable to medical imaging, orthopedics, and radiology, and can quickly and accurately stitch together a complete skeletal image, bringing great convenience to doctors.

[0115] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0116] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. An X-ray image bone stitching method, characterized by, The method comprises: S1: obtaining X-ray images to be spliced; S2: preprocessing the X-ray images to remove preset information in the X-ray images; S3: converting the processed X-ray images into binary images; S4: extracting a bone contour in the binary images; S5: segmenting the bone contour from muscles and other tissues to obtain a bone contour line; S6: extracting feature points in the bone contour line; S7: extracting key points in the feature points; S8: using the Euclidean distance of the key point feature vectors as a similarity judgment measurement of the key points in two images to be spliced, and determining that two key points in the two images to be spliced are a pair of matching points when the distance between the two key points is less than a threshold value, and determining that the two key points are wrong matching points and removing the wrong matching points when the distance between the two key points is greater than the threshold value; S9: obtaining a transformation matrix by using an improved RANSAC algorithm after removing the wrong matching points, registering the images by using the transformation matrix, obtaining a corresponding coordinate system, splicing the images, and obtaining a registered image; S10: judging whether the accuracy of the registered image meets a requirement, and adjusting the position of the spliced image when the accuracy of the registered image does not meet the requirement. The improved RANSAC algorithm comprises: S901: randomly selecting four pairs of non-linear feature point matching pairs from coarse matching results to form a set M; S902: calculating a homography matrix H according to the set M; S903: verifying all matching pairs in the coarse matching results by using the homography matrix H, and adding pairs of points less than a threshold value to the set M; S904: determining whether the number of pairs of points in the set M increases, and returning to step S902 for continuous execution when the number of pairs of points in the set M increases; otherwise, the next step is executed; S905: updating the current homography matrix when the number of pairs of points in the set M is greater than the number of pairs of points in a current optimal homography matrix, and not updating the current homography matrix when the number of pairs of points in the set M is less than the number of pairs of points in the current optimal homography matrix; S906: updating the total number of iterations according to the number of inliers of the current optimal homography matrix, and returning to step S901 for execution when the number of iterations of the current iteration is less than the total number of iterations; otherwise, the current optimal homography matrix is the final result.

2. The X-ray image bone stitching method according to claim 1, characterized in that, The preset information in the X-ray images is removed, including removing image noise and artifacts in the X-ray images.

3. The X-ray image bone stitching method of claim 1, wherein, The bone contour in the binary image is extracted, including extracting the bone contour from the binary image by using an edge detection and connectivity analysis algorithm.

4. The X-ray image bone stitching method of claim 1, wherein, The bone contour is segmented from muscles and other tissues to obtain the bone contour line, including: using threshold segmentation and edge detection to separate the extracted bone contour from other tissues to obtain the bone contour line.

5. The X-ray image bone stitching method of claim 1, wherein, The feature points in the bone contour line are extracted, including: constructing a DOG scale space of the bone contour line by using a SIFT algorithm; detecting extreme points of the DOG scale space, and determining that a feature point is an extreme point when the extreme point is the maximum or minimum value in a preset number of fields of the current layer and the upper and lower layers of the DOG scale space.

6. The X-ray image bone stitching method according to claim 5, characterized in that, The key points in the feature points are extracted, including: establishing a spatial scale function according to the DOG scale space, deriving the spatial scale function, and obtaining the positions of the key points by setting the derived spatial scale function to zero; removing low-contrast key points and unstable edge response points; accurately determining the positions and scales of the key points by using a three-dimensional quadratic function.

7. The X-ray image bone stitching method of claim 1, wherein, In step S9, the spliced images are obtained, and the registered images are obtained. The obtained coordinate system is mapped to image coordinates for splicing, and a complete bone image is obtained.

8. The X-ray image bone stitching method of claim 1, wherein, The accuracy of the registered images is judged to determine whether the accuracy meets the requirements, and if the accuracy does not meet the requirements, the position of the spliced image is adjusted. The spliced images are displayed respectively, and the upper image is displayed with a transparency of 50%-55%. The accuracy between the upper image and the lower image is observed, and if the accuracy does not meet the requirements, the position of the spliced image is manually adjusted.

Images