Medical image feature point recognition method, recognition system and readable storage medium
By segmenting and filtering feature point groups and utilizing the relative positional relationship of calibrated imaging elements, the robustness and accuracy problems of feature point recognition in existing technologies are solved, achieving efficient feature point recognition in occluded and noisy environments, which is suitable for surgical robot navigation and positioning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU MICROPORT ORTHOBOT CO LTD
- Filing Date
- 2022-08-15
- Publication Date
- 2026-06-05
AI Technical Summary
Existing medical image feature point recognition methods are not robust enough in environments with interference such as occlusion and noise, making it difficult to accurately identify feature points. Furthermore, they have high requirements for the angle between the scale plane and the imaging plane, which makes them unsuitable for the current scale design structure.
This paper provides a method for identifying feature points in medical images. By segmenting and identifying feature point groups, and using the relative positional relationship of calibrated imaging elements, the method predicts and filters local image groups, ultimately obtaining an accurate feature point sequence. This method eliminates the influence of occlusion and noise, and improves the recognition accuracy and robustness.
It accurately identifies feature points in interference environments, improves recognition efficiency, avoids missed identification, and ensures the smoothness and accuracy of the surgical procedure. It is applicable to two-dimensional medical fluoroscopic images.
Smart Images
Figure CN115631342B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical device technology, and in particular to a method, system and readable storage medium for identifying medical image feature points. Background Technology
[0002] During navigation and positioning, some surgical robots need to use transmitted rays (such as X-rays) to take fluoroscopic images of the patient by transmitting light through a calibration scale containing imaging markers. By identifying the feature points of the imaging markers in the fluoroscopic image, the transformation relationship between the surgical robot coordinate system and the surgical space coordinate system is established based on the coordinates of the identified feature points, and then surgical path planning and navigation positioning are performed.
[0003] Currently, the identification and localization of feature points in fluoroscopic images of medical imaging generally adopts traditional image detection methods, such as circle detection algorithms like Hough transform; or it adopts a method of pre-setting feature points and angle matching; or it divides the feature points into multiple regions, first extracts a specific region, then generates a sub-region, and finally extracts feature points from the sub-region.
[0004] However, these identification methods all have certain defects or shortcomings, such as:
[0005] 1. Traditional image detection methods are not very robust under relatively strong interference, and are prone to missed identification and false identification in environments with interference such as occlusion and noise;
[0006] 2. The method based on preset feature points and angle matching has high requirements for the angle between the scale plane and the imaging plane. If the angle is greater than 5 degrees, recognition cannot be completed.
[0007] 3. Methods such as dividing feature points into regions are not suitable for the current scale design structure and cannot complete the recognition task. Summary of the Invention
[0008] The purpose of this invention is to provide a medical image feature point recognition method, recognition system, and readable storage medium to solve the problems existing in the current feature point recognition methods.
[0009] To address the aforementioned technical problems, the first aspect of this invention provides a method for identifying feature points in medical images, applicable to feature point identification in two-dimensional medical fluoroscopic images. The method includes:
[0010] Provide a medical image with a set of feature points, the set of feature points including multiple feature points, each feature point corresponding to a calibration imaging element, the multiple calibration imaging elements having a known relative positional relationship;
[0011] Initial segmentation and identification are performed on multiple feature points in the medical image to obtain an initial feature point set;
[0012] The feature points in the initial feature point set are grouped to obtain the prediction point set;
[0013] For one of the prediction point groups in the prediction point group set, based on the relative positional relationship of the multiple calibration imaging elements, the prediction local image group corresponding to the prediction point group is obtained; all the prediction point groups in the prediction point group set are traversed to obtain the prediction local image group set.
[0014] For one of the predicted local image groups in the set of predicted local image groups, feature points are identified based on a local segmentation and recognition algorithm to obtain a predicted feature point sequence; all predicted local image groups in the set of predicted local image groups are traversed to obtain a set of predicted feature point sequences;
[0015] The predicted feature point sequences in the predicted feature point sequence set are filtered to obtain the final feature point recognition sequence.
[0016] Optionally, in the medical image feature point recognition method, the step of obtaining the predicted local image group corresponding to one of the predicted point groups in the predicted point group set based on the relative positional relationship of the multiple calibrated imaging elements includes:
[0017] For one of the prediction point groups in the set of prediction point groups, a prediction coordinate sequence group is obtained based on the relative positional relationship of the calibration developing elements;
[0018] The predicted local image group is obtained based on the distance between the feature points in the predicted point group and the predicted coordinate sequence group.
[0019] Optionally, in the medical image feature point recognition method, the step of obtaining a predicted local image group based on the distance between the feature points in the predicted point group and the predicted coordinate sequence group includes:
[0020] Obtain the number of the feature point in the predicted coordinate sequence group in the predicted point group, and calculate the projection ratio of the distance between the feature points in the medical image and the distance between the actual calibration imaging elements based on the number of the feature point and the relative position relationship between the corresponding calibration imaging elements.
[0021] Based on the projection ratio, the length and width of the predicted local image corresponding to a certain feature point are obtained;
[0022] The center point of the predicted local image is obtained based on the relative positional relationship of the calibrated developing elements corresponding to the feature points and the projection ratio.
[0023] The predicted local image is obtained from the center point, length, and width of the predicted local image.
[0024] Optionally, in the medical image feature point recognition method, each predicted point group includes two of the feature points.
[0025] Optionally, in the medical image feature point recognition method, the calibrated imaging element is spherical; the step of performing initial segmentation and recognition on multiple feature points in the medical image to obtain an initial feature point set includes:
[0026] The initial segmentation threshold is obtained based on the diameter, number, and image resolution of the calibrated developing elements;
[0027] The medical image is segmented based on an initial segmentation threshold to obtain the image coordinates of feature points;
[0028] Based on the radius of the identified feature points, the feature points are categorized into an initial feature point set.
[0029] Optionally, in the medical image feature point recognition method, the step of obtaining the initial feature point set further includes:
[0030] If the ratio of the number of feature points to the target number is less than a preset value, the initial segmentation threshold is adjusted, and the medical image is re-detected and segmented based on the adjusted initial segmentation threshold.
[0031] Optionally, in the medical image feature point recognition method, the calibrated imaging element is spherical, and the local segmentation recognition algorithm includes:
[0032] The local segmentation threshold is obtained by statistically analyzing the radii of the feature points detected in the medical image.
[0033] The predicted local image is segmented based on the local segmentation threshold to obtain the segmentation result;
[0034] Traverse all connected regions in the segmentation result and statistically obtain the aspect ratio, roundness, radius, and center of the circle;
[0035] If the aspect ratio, roundness, and radius meet the preset requirements of the currently detected feature point, the connected region is determined as a feature point, and the center of the feature point is added to the predicted feature point sequence.
[0036] Optionally, in the medical image feature point recognition method, the step of filtering all the predicted feature point sequences in the predicted feature point sequence set to obtain the final feature point recognition sequence includes:
[0037] Iterate through all the predicted feature point sequences in the predicted feature point sequence set;
[0038] If the number of feature points in a certain predicted feature point sequence does not match the expected number of feature points, then the predicted feature point sequence is deleted.
[0039] Calculate the average error between the predicted feature point sequence and the predicted center point of the local image;
[0040] The predicted feature point sequence with the smallest average error is determined as the final feature point recognition sequence.
[0041] To address the aforementioned technical problems, a second aspect of the present invention provides a readable storage medium having a program stored thereon, which, when executed, implements the steps of the medical image feature point recognition method as described above.
[0042] To address the aforementioned technical problems, a third aspect of the present invention provides a medical image feature point recognition system, comprising: a medical imaging device, a ruler tool, and a readable storage medium as described above; the medical imaging device includes a transmitting end and a receiving end; the ruler tool includes a plurality of calibration imaging elements, and there is a known relative positional relationship between the plurality of calibration imaging elements; the ruler tool is disposed between the transmitting end and the receiving end.
[0043] Optionally, in the medical image feature point recognition system, the ruler tool includes at least two planes and at least two different sizes of calibration imaging elements, wherein the calibration imaging elements of the same size are arranged on the same plane, and the number of calibration imaging elements of each size is not less than three.
[0044] Optionally, in the medical image feature point recognition system, the arrangement of the calibration imaging elements on the two planes is different.
[0045] Optionally, in the medical image feature point recognition system, the ruler tool further includes a rod, the two planes being non-coplanar and distributed on both sides of the rod.
[0046] In summary, the medical image feature point recognition method, recognition system, and readable storage medium provided by this invention include: providing a medical image with a feature point group, wherein the feature point group includes multiple feature points, each feature point corresponding to a calibration imaging element, and the multiple calibration imaging elements having a known relative positional relationship; performing initial segmentation and recognition on the multiple feature points in the medical image to obtain an initial feature point set; grouping the feature points in the initial feature point set to obtain a prediction point group set; for one prediction point group in the prediction point group set, obtaining a prediction local image group corresponding to the prediction point group based on the relative positional relationship of the multiple calibration imaging elements; traversing all prediction point groups in the prediction point group set to obtain a prediction local image group set; for one prediction local image group in the prediction local image group set, recognizing the feature points therein based on a local segmentation and recognition algorithm to obtain a prediction feature point sequence; traversing all prediction local image groups in the prediction local image group set to obtain a prediction feature point sequence set; and filtering all prediction feature point sequences in the prediction feature point sequence set to obtain a final feature point recognition sequence.
[0047] This configuration, utilizing the known relative positional relationships between feature points, predicts the possible regions of feature points in medical images. It can accurately obtain the predicted local image containing the feature points, and further perform local segmentation and recognition of feature points within the predicted local image range. This effectively eliminates the influence of occlusion, noise, and different exposure intensities, improving the accuracy and robustness of the recognition method, avoiding missed recognition of feature points, and increasing recognition efficiency. It does not require manual interaction, ensuring a smooth surgical process and improving surgical efficiency. Attached Figure Description
[0048] Those skilled in the art will understand that the accompanying drawings are provided to better understand the invention and do not constitute any limitation on the scope of the invention. Wherein:
[0049] Figure 1 This is a schematic diagram of a surgical robot system for registration using medical images, according to an embodiment of the present invention.
[0050] Figure 2 This is a schematic diagram of the end effector region of the robotic arm according to an embodiment of the present invention;
[0051] Figure 3 This is a schematic diagram of the ruler tool according to an embodiment of the present invention;
[0052] Figure 4 This is a top view of the ruler tool according to an embodiment of the present invention;
[0053] Figure 5 This is a schematic diagram of medical images according to an embodiment of the present invention;
[0054] Figure 6 This is a schematic diagram of the initial segmentation result according to an embodiment of the present invention;
[0055] Figure 7 This is a schematic diagram of a predicted local image group corresponding to metal balls L1 to L9 in an embodiment of the present invention;
[0056] Figure 8 This is a schematic diagram of a certain predicted local image group corresponding to metal balls S1 to S9 in an embodiment of the present invention;
[0057] Figure 9 This is a schematic diagram of a local predicted image of the metal ball L8 and its corresponding segmentation result in an embodiment of the present invention;
[0058] Figure 10 This is a schematic diagram of a local predicted image corresponding to the metal ball S9 in an embodiment of the present invention and its corresponding segmentation result;
[0059] Figure 11 This is a schematic diagram of the final feature point recognition sequence in an embodiment of the present invention;
[0060] Figure 12 This is a schematic diagram of a ruler tool according to another embodiment of the present invention;
[0061] Figure 13 This is a top view of a ruler tool according to another embodiment of the present invention;
[0062] Figure 14 This is a flowchart of a medical image feature point recognition method according to an embodiment of the present invention. Detailed Implementation
[0063] To make the objectives, advantages, and features of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and are not drawn to scale, and are only used to facilitate and clarify the explanation of the embodiments of this invention. Furthermore, the structures shown in the drawings are often part of the actual structures. In particular, different figures may emphasize different aspects and may sometimes use different scales.
[0064] As used in this invention, the singular forms “a,” “an,” and “the” include plural objects; the term “or” is generally used to mean “and / or”; the term “a number” is generally used to mean “at least one”; and the term “at least two” is generally used to mean “two or more”. Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first,” “second,” or “third” may explicitly or implicitly include one or at least two of that feature; “one end” and “the other end,” and “proximal end” and “distal end” generally refer to two corresponding parts, which include not only endpoints. Furthermore, the terms "installed," "connected," and "attached," as used in this invention, and the term "set" on one element from another, should be interpreted broadly. They generally only indicate a connection, coupling, cooperation, or transmission relationship between the two elements, which can be direct or indirect through an intermediate element. They should not be construed as indicating or implying a spatial relationship between the two elements, meaning one element can be located inside, outside, above, below, or to one side of another element, unless otherwise explicitly stated. Those skilled in the art can understand the specific meaning of these terms in this invention based on the specific circumstances. Additionally, directional terms such as above, below, up, down, upward, downward, left, and right are used relative to exemplary embodiments as shown in the figures, with upward or upper directions pointing towards the top of the corresponding figure, and downward or lower directions pointing towards the bottom of the corresponding figure.
[0065] The purpose of this invention is to provide a medical image feature point recognition method, recognition system, and readable storage medium to solve the problems existing in the current feature point recognition methods.
[0066] The following description refers to the accompanying drawings.
[0067] Please refer to Figure 1This invention illustrates a surgical robot system for registration using medical images, comprising: a robotic arm 1, a ruler tool 2, a medical imaging device 3, and a navigation device 4. The medical imaging device 3 includes a transmitter 31 and a receiver 32. In an alternative example, the medical imaging device 3 may be an X-ray machine, with the transmitter 31 being an X-ray tube for emitting X-rays toward the receiver 32, which is an imaging plate. The X-rays pass through the ruler tool 2 and the surgical object 5 before reaching the receiver 32, where they are imaged to obtain a medical image. Of course, the medical imaging device 3 is not limited to an X-ray machine; those skilled in the art can configure it as a device such as a CT scanner, based on existing technology. The navigation device 4 includes a positioning device 41 (such as an optical locator) and several trackable elements 42 (such as optical targets). The positioning device 41 is paired with the trackable elements 42, thereby enabling the positioning device 41 to track and acquire the pose information of the trackable elements 42. Of course, the positioning device 41 and the trackable element 42 are not limited to optical positioning instruments and optical targets. Those skilled in the art can also configure them as magnetic positioning devices, etc., according to existing technology, and the present invention is not limited thereto. In one example, the trackable element 42 can be installed on the robotic arm 1 and the surgical subject 5 respectively. With this configuration, the positioning device 41 can obtain the pose information of the robotic arm 1 and the pose information of the surgical subject 5 by tracking and acquiring the pose of the trackable element 42.
[0068] For further details, please refer to... Figure 2 This illustrates an example of the end effector region of a robotic arm 1, where a ruler tool 2 is mounted on the robotic arm 1, and a tracking element 42 is also mounted on the robotic arm 1, thus the relative positional relationship between the ruler tool 2 and the tracking element 42 is known and fixed. Furthermore, the ruler tool 2 includes at least two planes 20 and at least two different sizes of calibration developing elements 21. Calibration developing elements 21 of the same size are disposed on the same plane 20, and the number of each size of calibration developing element 21 is not less than three. Preferably, the calibration developing elements 21 are spherical. The diameters of the calibration developing elements 21 of different sizes are different. Further, each size of calibration developing element 21 has a fixed arrangement on its corresponding plane 20; preferably, the arrangement of the calibration developing elements 21 on the two planes 20 is different; more preferably, the two planes 20 are parallel to each other. Please refer to... Figure 3The illustration shows an example of a ruler tool 2, which includes two planes 20 and two different sizes of calibration imaging elements 21. Thus, in the medical image obtained at the receiving end 32, the images of the calibration imaging elements 21 of different sizes can be distinguished. Based on the principle that at least three points determine a plane, the relationship between the two planes 20 in the medical image can be determined according to the images of the two sizes of calibration imaging elements 21, thereby calculating the projection matrix of the medical image. This allows for the calculation of three-dimensional spatial coordinates, registration of the medical image coordinate system and the navigation coordinate system, and, based on this, surgical path planning and surgical operations. Specifically, the surgical object 5 can be a patient, but is not limited to a patient; it can also be a model prosthesis, etc., which can be used by the operator for training, calibration, or verification of the surgery. This invention does not limit the application scenarios of this surgical robot system.
[0069] A demonstrative medical image acquisition process includes:
[0070] Step Sa1: Position the surgical subject 5 in a suitable location;
[0071] Step Sa2: The robotic arm 1, medical imaging device 3, and positioning device 41 are positioned appropriately;
[0072] Step Sa3: Fix the tracking element 42 on the surgical object 5 and the robotic arm 1 respectively; and fix the ruler tool 2 on the robotic arm 1.
[0073] Step Sa4: The C-arm of the medical imaging device 3 is adjusted to the appropriate position according to the type and location of the surgery;
[0074] Step Sa5: Adjust the robotic arm 1 and place the ruler tool 2 close to the surgical object 5, with the plane 20 on the ruler tool 2 as parallel as possible to the imaging plate of the receiving end 32;
[0075] Step Sa6: Obtain medical images.
[0076] After obtaining the medical image, the images of the calibrated contrast agents 21 need to be segmented and extracted. Ideally, segmenting and extracting the medical image should identify all the images of the calibrated contrast agents 21, allowing for accurate calculation of the projection matrix of the medical image. However, in practice, due to interference from factors such as occlusion and exposure noise, segmenting and extracting the medical image often fails to identify all the images of the calibrated contrast agents 21, or may contain a certain number of interference points. If the number of identified images of the calibrated contrast agents 21 is less than expected, errors will occur, and in some cases, calculation may become impossible.
[0077] To address this problem, this invention provides a method for identifying feature points in medical images, applicable to feature point identification in two-dimensional medical fluoroscopy images. For ease of description, the image of the calibration imaging element 21 is abstracted as feature points. It is understood that feature points can be defined based on the characteristics of the image of the calibration imaging element 21. For example, in some embodiments, the image of the calibration imaging element 21 is circular or elliptical, then a feature point can refer to such a circular or elliptical image region. Furthermore, for ease of description, several feature points corresponding to several calibration imaging elements 21 of the same specifications in a medical image are defined as a feature point group. For example... Figure 14 As shown, the medical image feature point recognition method includes:
[0078] Step S1: Provide a medical image with a set of feature points, the set of feature points including multiple feature points, each feature point corresponding to a calibration imaging element 21, and the multiple calibration imaging elements 21 having a known relative positional relationship;
[0079] Step S2: Perform initial segmentation and identification on multiple feature points in the medical image to obtain an initial feature point set;
[0080] Step S3: Group the feature points in the initial feature point set to obtain a prediction point set; understandably, each prediction point set contains at least two feature points.
[0081] Step S4: For one of the prediction point groups in the prediction point group set, based on the relative positional relationship of the multiple calibration imaging elements 21, obtain the prediction local image group corresponding to the prediction point group; traverse all the prediction point groups in the prediction point group set to obtain the prediction local image group set.
[0082] Step S5: For one of the predicted local image groups in the set of predicted local image groups, based on the local segmentation and recognition algorithm, identify the feature points therein to obtain a predicted feature point sequence; traverse all the predicted local image groups in the set of predicted local image groups to obtain a set of predicted feature point sequences;
[0083] Step S6: Filter all the predicted feature point sequences in the predicted feature point sequence set to obtain the final feature point recognition sequence.
[0084] The following example, with reference to the accompanying drawings, illustrates this point.
[0085] Please refer to Figure 3 and Figure 4The scale tool 2 includes a scale base 22 and a rod 23. The scale base 22 is made of an X-ray resistant material. The rod 23 is connected to the scale base 22. The scale base 22 has two planes 20, namely a first plane 201 and a second plane 202. Preferably, the two planes 20 are not coplanar and are distributed on both sides of the rod 23. The first plane 201 and the second plane 202 each have nine mounting holes for mounting the calibration developing element 21. The mounting holes on the two planes 20 have different diameters and are arranged differently. The calibration developing element 21 consists of two different diameter metal balls, nine of each type, which are installed in the mounting holes on the two planes 20. Furthermore, the metal balls on the first plane 201 and the second plane 202 are numbered in a specific order, so that each metal ball has its own unique number. For ease of description, the nine metal spheres on the first plane 201 are numbered L1 to L9, and the nine metal spheres on the second plane 201 are numbered S1 to S9. This establishes a fixed and known relative positional relationship between each metal sphere and the others. After the medical image is obtained through scanning, the relative image coordinates of the feature points of each metal sphere in the medical image are also fixed, such as... Figure 5 As shown. Understandably, step S1, based on the ruler tool 2 as described above, results in a medical image containing two feature point groups, each of which includes 9 feature points.
[0086] Optionally, step S2, which involves initial segmentation and identification of multiple feature points in the medical image to obtain an initial feature point set, includes:
[0087] Step S21: Obtain the initial segmentation threshold based on the diameter, number, and image resolution of the calibrated developing element 21; Step S21 may employ traditional image processing algorithms, such as adaptive threshold segmentation methods or object detection algorithms based on machine learning or deep learning, etc., which can be selected by those skilled in the art based on existing technologies.
[0088] Step S22: Detect and segment the medical image according to the initial segmentation threshold to obtain the image coordinates of the feature points. It should be noted that since the calibration imaging element 21 is spherical, the feature points should ideally appear as circles (or approximately circular ellipses, etc.). The image coordinates of the feature points include the center coordinates and the radius of the feature points. The initial segmentation result is as follows... Figure 6 As shown. It should be noted that the initial segmentation may only identify some of the feature points, and it is not required to identify all feature points. Figure 6In the example shown, the initial segmentation identified 8 feature points corresponding to metal spheres L1, L2, L3, L4, L5, L6, L7, and L9 on plane 201; and 5 feature points corresponding to metal spheres S1, S2, S3, S4, and S5 on plane 202. Feature points corresponding to metal spheres L8 and S6–S9 were not identified.
[0089] Step S23: Based on the radius of the identified feature points, classify the feature points into an initial feature point set. It is understood that since the calibration developing element 21 is spherical, the radius of the feature points can also be obtained based on the initial segmentation in step S21. If there is only one type of calibration developing element 21, the radius of the feature points should also be the same, and then an initial feature point set can be obtained after classification. However, based on the ruler tool 2 in the above embodiment, which includes two different specifications of calibration developing elements 21, the radius of the identified feature points will also be two different types. Therefore, based on the different radii, the feature points are classified into two initial feature point sets respectively.
[0090] Optionally, before step S21, a two-dimensional image histogram can be calculated to obtain an initial segmentation threshold. Optionally, before step S23, a clustering algorithm can be used to filter out noise points. Further, after step S23, the number of identified feature points can be counted. If the ratio of the number of feature points to the target number is less than a preset value (e.g., 60% of the total number of calibrated imaging elements 21), the initial segmentation threshold is adjusted, and the medical image is re-detected and segmented according to the adjusted initial segmentation threshold. If the ratio of the number of feature points to the target number is not less than the preset value, the initial segmentation is completed.
[0091] In steps S3 and S4, for an initial set of feature points, there are known and definite relative positional relationships among the feature points it contains. Therefore, the positions of all other feature points can be predicted using any two or more feature points and their relative positional relationships. Thus, the feature points in the initial set of feature points can be grouped, with each group called a prediction point group. Based on each prediction point group, the positions of all other feature points in the initial set of feature points can be predicted. It is understood that at least two feature points are sufficient for prediction; therefore, preferably, each prediction point group includes two feature points. Of course, in other embodiments, a larger number of feature points can be used for prediction, and this invention does not limit this.
[0092] Taking the prediction of two feature points as an example, the relative positional relationship between these two feature points is assumed to correspond to the relative positional relationship between two calibration developing elements 21 (for ease of description, this is called the assumed correspondence). Thus, the relative positions of all feature points corresponding to each specification of calibration developing element 21 (for ease of description, this is called the predicted positions of the feature points) can be predicted based on the two feature points in the aforementioned prediction point set. That is, based on an assumed correspondence between the feature point and the calibration developing element 21 (i.e., based on a prediction point set), a predicted position of the feature point can be obtained; furthermore, by exhaustively enumerating the assumed correspondences and traversing all possible assumed correspondences, all possible predicted positions can be obtained.
[0093] Understandably, all possible predicted locations contain a large number of unwanted spurious results. Therefore, it is necessary to eliminate these spurious results. To do this, we can segment the image into local prediction groups based on any predicted location to eliminate spurious results.
[0094] Optionally, in step S4, the step of obtaining the predicted local image group corresponding to one of the predicted point groups in the set of predicted point groups, based on the relative positional relationship of the multiple calibration imaging elements 21, includes:
[0095] Step S41: For one of the prediction point groups in the prediction point group set, a prediction coordinate sequence group is obtained based on the relative positional relationship of the calibration developing element 21; this step, based on the number of the calibration developing element 21 corresponding to two feature points in the prediction point group, can predict the number of the calibration developing element 21 corresponding to the other 7 feature points, that is, a prediction coordinate sequence group is obtained.
[0096] Step S42: Obtain the predicted local image group based on the distance between the feature points in the predicted point group and the predicted coordinate sequence group. Based on the above explanation, it can be understood that, according to the predicted coordinate sequence group, the positions of all other feature points can be predicted based on two feature points in the predicted point group. This results in the predicted local image corresponding to each feature point, and these predicted local images are categorized into a predicted local image group.
[0097] Please refer to Figure 7 This illustrates a set of predicted local images corresponding to metal spheres L1 to L9 obtained according to the aforementioned steps, containing nine predicted local images L1' to L9'. In one example, the feature point corresponding to metal sphere L8 was not successfully identified in the initial segmentation of the aforementioned steps; therefore, the predicted local image L8' corresponding to metal sphere L8 was obtained by segmenting based on the predicted position of the feature point corresponding to metal sphere L8. Please refer to... Figure 8This illustrates a set of predicted local images corresponding to metal spheres S1 to S9 obtained according to the aforementioned steps, containing nine predicted local images S1' to S9'. In one example, the feature points corresponding to metal spheres S6 to S9 were not successfully identified in the initial segmentation of the aforementioned steps. Therefore, the predicted local images S6' to S9' corresponding to metal spheres S6 to S9 were obtained by segmentation based on the predicted positions of the feature points corresponding to metal spheres S6 to S9.
[0098] Furthermore, step S42, the step of obtaining a predicted local image, includes:
[0099] Step S421: Obtain the number of the feature point in the predicted coordinate sequence group in the predicted point group. Based on the number of the feature point and its corresponding relative position relationship with the calibration imaging element 21, calculate the projection ratio of the distance between the feature points in the medical image and the distance between the calibration imaging elements 21 in reality. In this step, the distance between two feature points in a predicted point group is taken as the image distance. Assuming that the two feature points in the predicted point group have a one-to-one correspondence with two calibration imaging elements 21 in reality (i.e., an assumed correspondence), the distance between these two calibration imaging elements 21 is taken as the template distance. The projection ratio can be calculated by using the image distance and the template distance.
[0100] Step S422: Based on the projection ratio, obtain the length and width of the predicted local image corresponding to a certain feature point;
[0101] Step S423: Based on the relative positional relationship of the calibrated developing element 21 corresponding to the feature points and the projection ratio, obtain the center point of the predicted local image;
[0102] Step S424: Obtain the predicted local image from the center point, length, and width of the predicted local image.
[0103] Furthermore, by iterating through all hypothetical correspondences—that is, pairing each predicted point group with any two calibrated developing elements 21—multiple projection ratios can be obtained. Based on these projection ratios, a predicted local image group corresponding to the predicted point group can be obtained. Understandably, further iterating through all predicted point groups yields multiple predicted local image groups. These predicted local image groups are then categorized to obtain a set of predicted local image groups. Understandably, this set of predicted local image groups contains a large number of useless spurious results. The local segmentation and recognition algorithm in step S5 is then used to perform local segmentation and recognition on the predicted local images, eliminating these useless spurious results.
[0104] Optionally, in step S5, the local segmentation recognition algorithm includes:
[0105] Step S51: Calculate the local segmentation threshold based on the radius of the detected feature points in the medical image (e.g., which can be identified in step S22);
[0106] Step S52: Segment the predicted local image according to the local segmentation threshold to obtain the segmentation result; it is understood that steps S51 and S52 can be a circle detection method based on Hough transform, or other circle recognition methods, and those skilled in the art can choose according to the existing technology.
[0107] Step S53: Traverse all connected regions in the segmentation result and obtain the aspect ratio, roundness, radius and center of the circle;
[0108] Step S54: If the aspect ratio, roundness, and radius meet the preset requirements of the currently detected feature point, then the connected region is determined as a feature point, and the center of the feature point is added to the predicted feature point sequence.
[0109] Please refer to Figure 9 and Figure 10 ,in Figure 9 This shows a local prediction image L8' corresponding to the metal sphere L8. Figure 9 The left region), and its corresponding segmentation result L8”. Figure 9 (Right side area); Figure 10 This shows a local prediction image S9' corresponding to the metal sphere S9. Figure 10 The left region), and its corresponding segmentation result S9”. Figure 10 (Right side area)
[0110] Optionally, in step S54, if the aspect ratio, roundness, and radius do not meet the preset requirements of the currently detected feature points, the segmentation is deemed a failure, meaning no feature points or their corresponding center points are obtained. Thus, the predicted feature point sequence corresponding to this set of predicted local images may only contain the center points of the initial two feature points in the predicted point group. Optionally, before step S51, the histogram of the local two-dimensional image can be calculated to obtain the local segmentation threshold.
[0111] In fact, steps S51 to S54 are processes that eliminate invalid predicted local images by verifying whether feature points exist in the predicted local images from the previous step. For example, among all possible predicted locations, there may be some invalid results, that is, the assumed correspondence between two feature points in the predicted point group and the calibration developing element 21 is not true. In this case, multiple predicted local images in the predicted local image group obtained according to the projection relationship may not actually contain feature points. Therefore, executing steps S51 to S54 at this time will not yield valid results.
[0112] Furthermore, step S6 includes:
[0113] Step S61: Traverse all the predicted feature point sequences in the predicted feature point sequence set;
[0114] Step S62: If the number of feature points in a certain predicted feature point sequence does not match the expected number of feature points, then delete the predicted feature point sequence; if the number of feature points in a certain predicted feature point sequence is less than expected, it indicates that there are invalid predicted local images in the predicted local image group corresponding to this predicted feature point sequence, that is, no valid feature points were obtained in them, further indicating that this is a set of invalid false results, which should be excluded.
[0115] Step S63: Calculate the average error between the predicted feature point sequence and the center point of the predicted local image; it can be understood that the data in the predicted feature point sequence is the set of center points of the connected regions obtained in the aforementioned steps S53 and S54. The closer the center point is to the center point of the predicted local image, the more accurate the position of the previously predicted feature point is, and the higher its confidence level is.
[0116] Step S64: Determine the predicted feature point sequence with the smallest average error as the final feature point recognition sequence.
[0117] Since the predicted point groups are obtained through traversal, they contain several sets of correct hypothesis correspondences, thus also yielding several sets of effective predicted local image groups. Steps S63 and S64 aim to determine the predicted feature point sequence corresponding to the predicted local image group with the smallest error as the final feature point recognition sequence. This yields the final sequence of feature points to be recognized. The final feature point recognition sequence is as follows: Figure 11 As shown.
[0118] It should be noted that the ruler tool 2, which includes two sets of nine metal balls each, is only used to illustrate the medical image feature point recognition method provided in this embodiment, and does not limit the specific shape and structural composition of the ruler tool 2. Figure 12 and Figure 13As shown, in some other embodiments, the ruler tool 2 may also include two sets of five metal balls each. It is understood that fewer calibration imaging elements 21 result in fewer shadows in the medical image, reducing the amount of shadows obscuring the patient's area and minimizing the impact of the calibration imaging elements 21's shadows on diagnosis and surgical planning; however, fewer calibration imaging elements 21 will reduce positioning accuracy. The specific number of calibration imaging elements 21 can be set according to the required accuracy in the actual application scenario. In particular, the number of planes 20 included in the ruler tool 2 is not limited to two, and the number of calibration imaging elements 21 on each plane 20 is not necessarily the same; those skilled in the art can configure them according to actual needs.
[0119] Based on the medical image feature point recognition method described above, embodiments of the present invention also provide a readable storage medium storing a program thereon, which, when executed, implements the steps of the medical image feature point recognition method described above. Furthermore, embodiments of the present invention also provide a medical image feature point recognition system, which includes a medical imaging device 3, a ruler tool 2, and the readable storage medium described above. It is understood that the readable storage medium can be set independently or integrated into the medical image feature point recognition system, such as by integrating it into the medical imaging device 3; the present invention is not limited in this regard.
[0120] In summary, the medical image feature point recognition method, recognition system, and readable storage medium provided by this invention include: providing a medical image with a feature point group, wherein the feature point group includes multiple feature points, each feature point corresponding to a calibration imaging element, and the multiple calibration imaging elements having a known relative positional relationship; performing initial segmentation and recognition on the multiple feature points in the medical image to obtain an initial feature point set; grouping the feature points in the initial feature point set to obtain a prediction point group set; for one prediction point group in the prediction point group set, obtaining a prediction local image group corresponding to the prediction point group based on the relative positional relationship of the multiple calibration imaging elements; traversing all prediction point groups in the prediction point group set to obtain a prediction local image group set; for one prediction local image group in the prediction local image group set, recognizing the feature points therein based on a local segmentation and recognition algorithm to obtain a prediction feature point sequence; traversing all prediction local image groups in the prediction local image group set to obtain a prediction feature point sequence set; and filtering all prediction feature point sequences in the prediction feature point sequence set to obtain a final feature point recognition sequence. This configuration, utilizing the known relative positional relationships between feature points, predicts the possible regions of feature points in medical images. It can accurately obtain the predicted local image containing the feature points, and further perform local segmentation and recognition of feature points within the predicted local image range. This effectively eliminates the influence of occlusion, noise, and different exposure intensities, improving the accuracy and robustness of the recognition method, avoiding missed recognition of feature points, and increasing recognition efficiency. It does not require manual interaction, ensuring a smooth surgical process and improving surgical efficiency.
[0121] It should be noted that the above embodiments can be combined with each other. The above description is only a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.
Claims
1. A method for identifying feature points in medical images, applicable to feature point identification in two-dimensional medical fluoroscopic images, characterized in that, include: Provide a medical image with a set of feature points, the set of feature points including multiple feature points, each feature point corresponding to a calibration imaging element, the multiple calibration imaging elements having a known relative positional relationship; Initial segmentation and identification are performed on multiple feature points in the medical image to obtain an initial feature point set; The feature points in the initial feature point set are grouped to obtain the prediction point set; For one of the prediction point groups in the prediction point group set, based on the relative positional relationship of the multiple calibration imaging elements, the prediction local image group corresponding to the prediction point group is obtained; all the prediction point groups in the prediction point group set are traversed to obtain the prediction local image group set. For one of the predicted local image groups in the set of predicted local image groups, feature points are identified based on a local segmentation and recognition algorithm to obtain a predicted feature point sequence; all predicted local image groups in the set of predicted local image groups are traversed to obtain a set of predicted feature point sequences; The calibrated imaging element is spherical. The local segmentation recognition algorithm includes: obtaining a local segmentation threshold based on the radius of the detected feature points in the medical image; segmenting the predicted local image based on the local segmentation threshold to obtain a segmentation result; traversing all connected regions in the segmentation result and obtaining the aspect ratio, roundness, radius, and center; if the aspect ratio, roundness, and radius meet the preset requirements of the currently detected feature point, then the connected region is determined as a feature point, and the center of the feature point is added to the predicted feature point sequence. The predicted feature point sequences in the predicted feature point sequence set are filtered to obtain the final feature point recognition sequence.
2. The medical image feature point recognition method according to claim 1, characterized in that, For a given set of predicted point groups, the step of obtaining the predicted local image group corresponding to that predicted point group based on the relative positional relationship of the multiple calibration imaging elements includes: For one of the prediction point groups in the set of prediction point groups, a prediction coordinate sequence group is obtained based on the relative positional relationship of the calibration developing elements; The predicted local image group is obtained based on the distance between the feature points in the predicted point group and the predicted coordinate sequence group.
3. The medical image feature point recognition method according to claim 2, characterized in that, The step of obtaining a predicted local image from the step of obtaining the predicted local image group based on the distance between the feature points in the predicted point group and the predicted coordinate sequence group includes: Obtain the number of the feature point in the predicted coordinate sequence group in the predicted point group, and calculate the projection ratio of the distance between the feature points in the medical image and the distance between the actual calibration imaging elements based on the number of the feature point and the relative position relationship between the corresponding calibration imaging elements. Based on the projection ratio, the length and width of the predicted local image corresponding to a certain feature point are obtained; The center point of the predicted local image is obtained based on the relative positional relationship of the calibrated developing elements corresponding to the feature points and the projection ratio. The predicted local image is obtained from the center point, length, and width of the predicted local image.
4. The medical image feature point recognition method according to claim 1, characterized in that, Each group of predicted points includes two of the feature points.
5. The medical image feature point recognition method according to claim 1, characterized in that, The calibration imaging element is spherical; the step of performing initial segmentation and identification of multiple feature points in the medical image to obtain an initial feature point set includes: The initial segmentation threshold is obtained based on the diameter, number, and image resolution of the calibrated developing elements; The medical image is segmented based on an initial segmentation threshold to obtain the image coordinates of feature points; Based on the radius of the identified feature points, the feature points are categorized into an initial feature point set.
6. The medical image feature point recognition method according to claim 5, characterized in that, The steps to obtain the initial set of feature points also include: If the ratio of the number of feature points to the target number is less than a preset value, the initial segmentation threshold is adjusted, and the medical image is re-detected and segmented based on the adjusted initial segmentation threshold.
7. The medical image feature point recognition method according to claim 1, characterized in that, The step of filtering all the predicted feature point sequences in the predicted feature point sequence set to obtain the final feature point recognition sequence includes: Iterate through all the predicted feature point sequences in the predicted feature point sequence set; If the number of feature points in a certain predicted feature point sequence does not match the expected number of feature points, then the predicted feature point sequence is deleted. Calculate the average error between the predicted feature point sequence and the predicted center point of the local image; The predicted feature point sequence with the smallest average error is determined as the final feature point recognition sequence.
8. A readable storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the steps of the medical image feature point recognition method according to any one of claims 1 to 7.
9. A medical image feature point recognition system, characterized in that, include: A medical imaging device, a ruler tool, and a readable storage medium according to claim 8, wherein the medical imaging device includes a transmitter and a receiver, the ruler tool includes a plurality of calibration imaging elements, and there is a known relative positional relationship between the plurality of calibration imaging elements; the ruler tool is disposed between the transmitter and the receiver.
10. The medical image feature point recognition system according to claim 9, characterized in that, The ruler tool includes at least two planes and at least two different sizes of calibration developing elements, wherein the calibration developing elements of the same size are arranged on the same plane, and the number of calibration developing elements of each size is not less than three.
11. The medical image feature point recognition system according to claim 10, characterized in that, The calibration developing elements on the two planes are arranged differently.
12. The medical image feature point recognition system according to claim 10, characterized in that, The ruler tool also includes a rod, and the two planes are not coplanar and are distributed on both sides of the rod.