A surgical navigation robot landmark point recognition registration method, system and device

Abstract

The application discloses a surgical navigation robot mark point identification registration method, system and device, and relates to the technical field of surgical navigation robots. The method comprises the following steps: acquiring an image of a scale placed at a preset position on a human body surface as a scale image; extracting a connected domain of the scale image by using a connected domain extraction algorithm and screening a plurality of steel ball identification regions; determining the gravity center coordinates of each steel ball identification region as a steel ball identification point; determining an optimal conversion matrix according to the plurality of steel ball identification points; and completing coordinate conversion from an image space to an actual space based on the optimal conversion matrix. By determining the optimal conversion matrix, the application can improve the accuracy of surgical navigation robot mark point identification registration.

Description

Technical Field

[0001] This invention relates to the field of surgical navigation robot technology, and in particular to a method, system and device for marker recognition and registration of surgical navigation robots. Background Technology

[0002] In 3D surgical robot navigation technology, intraoperative landmarks are used to determine the pre-defined positions of surgical instruments and the patient's body surface, thereby assisting the robot in performing surgical operations. The principle of this technology is to convert the image information of intraoperative landmarks into 3D coordinate information using computer vision and machine learning techniques, and then align it with the patient's CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) images to achieve high-precision surgical navigation. 3D surgical robot navigation has gradually developed alongside the rapid advancements in robotics, computer vision, and medical imaging technologies. Traditional surgical navigation methods typically require complex preoperative planning and real-time operation and adjustments during the procedure, but these often introduce errors. 3D surgical robot navigation technology, by combining intraoperative landmarks with patient image information, achieves a more accurate and convenient navigation method, becoming one of the important development directions of modern surgical techniques. Summary of the Invention

[0003] The purpose of this invention is to provide a method, system, and device for marker recognition and registration in surgical navigation robots, which can improve the accuracy of marker recognition and registration in surgical navigation robots.

[0004] To achieve the above objectives, the present invention provides the following solution:

[0005] A method for marker recognition and registration in a surgical navigation robot includes:

[0006] The image obtained when the ruler is placed at a preset position on the human body surface is the ruler image; the ruler is provided with multiple steel balls; the number of steel balls is not less than 3; the multiple steel balls are not collinear;

[0007] The connected components of the ruler image are extracted using a connected component extraction algorithm;

[0008] Multiple steel ball recognition regions are selected from the multiple connected domains;

[0009] The centroid coordinates of each steel ball recognition area are determined as the steel ball recognition point;

[0010] Determine the optimal transformation matrix based on multiple steel ball identification points;

[0011] The coordinate transformation from image space to real space is completed based on the optimal transformation matrix.

[0012] Optionally, the step of extracting the connected components of the ruler image using a connected component extraction algorithm includes:

[0013] Obtain the initial isosurface threshold, step size, and isosurface threshold;

[0014] The initial isosurface threshold is set as the current isosurface threshold.

[0015] Based on the current isosurface threshold, the undetermined connected components of the scale image are extracted using a connected component extraction algorithm;

[0016] Determine whether the number of undetermined connected components is within the range of connected component numbers to obtain a first determination result; the lower limit of the range of connected component numbers is 3; the upper limit of the range of connected component numbers is twice the number of steel balls.

[0017] If the first judgment result is negative, then the value of the current isosurface threshold is increased by the step size;

[0018] Determine whether the current isosurface threshold has reached the isosurface threshold, and obtain a second determination result;

[0019] If the second judgment result is negative, then return to the step "Extract the undetermined connected components of the scale image using the connected component extraction algorithm based on the current isosurface threshold";

[0020] If the second judgment result is yes, then update the step size and return to the step "determine the initial isosurface threshold as the current isosurface threshold";

[0021] If the first judgment result is yes, then the multiple undetermined connected components extracted based on the current isosurface threshold are determined to be connected components.

[0022] Optionally, multiple steel ball recognition regions are filtered from the multiple connected components, including:

[0023] Determine any of the aforementioned connected components as the current connected component;

[0024] The number of points contained in the current connected component is obtained as the current decision parameter;

[0025] Determine whether the current judgment parameter is within the point range to obtain a third judgment result;

[0026] If the third judgment result is negative, then the current connected region is determined to be a non-steel ball recognition region;

[0027] If the third judgment result is yes, then the current connected region is determined to be the steel ball recognition region;

[0028] Update the current connected component and return to the step "determine whether the current judgment parameter is in the point range and obtain the third judgment result" until all connected components are traversed to obtain multiple steel ball recognition regions.

[0029] Optionally, based on multiple ball recognition points, the optimal transformation matrix is ​​determined, including:

[0030] Identify any three steel balls on the scale as undetermined steel balls;

[0031] Construct the current measurement coordinate system based on the undetermined steel ball;

[0032] The current steel ball and the two undetermined steel balls are identified as the establishing steel balls of the current measurement coordinate system;

[0033] Obtain the measurement coordinates of multiple steel balls in the current measurement coordinate system;

[0034] Determine any steel ball identification point as the current steel ball identification point;

[0035] Determine any two steel ball identification points other than the current steel ball identification point as undetermined steel ball identification points;

[0036] Using the current steel ball identification point as the origin, construct the current image coordinate system based on the undetermined steel ball identification points;

[0037] The current steel ball identification point and the two undetermined steel ball identification points are determined as the steel ball identification points for establishing the current image coordinate system;

[0038] Obtain the image coordinates of multiple steel ball identification points in the current image coordinate system;

[0039] Based on the measurement coordinates of multiple system-establishing steel balls in the current measurement coordinate system and the image coordinates of multiple system-establishing steel ball identification points in the current image coordinate system, a current transformation matrix between the current measurement coordinate system and the current image coordinate system is constructed.

[0040] Based on the current transformation matrix, the measurement coordinates of the multiple system steel ball identification points in the current measurement coordinate system are the system steel ball identification point measurement coordinates.

[0041] Based on the measurement coordinates of multiple system-establishing steel ball identification points and the measurement coordinates of each system-establishing steel ball in the current measurement coordinate system, the transformation error value of the current transformation matrix is ​​determined.

[0042] Update the undetermined steel ball identification points and return to the step "Construct the current image coordinate system based on the undetermined steel ball identification points with the current steel ball identification point as the origin" until all combinations of undetermined steel ball identification points are traversed to obtain the transformation error value of the transformation matrix between the current measurement coordinate system and the multiple image coordinate systems constructed with the current steel ball identification point as the origin;

[0043] Update the current steel ball identification point and return to the step "determine any two steel ball identification points other than the current steel ball identification point as undetermined steel ball identification points" until all steel ball identification points are traversed to obtain the transformation error value of the transformation matrix between the current measurement coordinate system and multiple image coordinate systems;

[0044] Update the undetermined steel ball and return to the step "Construct the current measurement coordinate system based on the undetermined steel ball" until all combinations of undetermined steel balls are traversed to obtain the transformation error values ​​of the transformation matrices between different measurement coordinate systems and multiple image coordinate systems;

[0045] The transformation matrix corresponding to the minimum transformation error value is determined as the optimal transformation matrix.

[0046] Optionally, after determining that the transformation matrix corresponding to the minimum transformation error value is the optimal transformation matrix, the following steps are also included:

[0047] The measurement coordinate system corresponding to the optimal transformation matrix is ​​determined to be the actual surgical space coordinate system;

[0048] The image coordinate system corresponding to the optimal transformation matrix is ​​determined as the image space coordinate system.

[0049] A surgical navigation robot marker recognition and registration system includes:

[0050] The ruler image acquisition module is used to acquire an image of the ruler when it is placed at a preset position on the human body surface; the ruler is provided with multiple steel balls; the number of steel balls is not less than 3; the multiple steel balls are not collinear;

[0051] A connected component extraction module is used to extract the connected components of the ruler image using a connected component extraction algorithm;

[0052] A steel ball recognition region acquisition module is used to filter multiple steel ball recognition regions from multiple connected domains;

[0053] The steel ball identification point determination module is used to determine the centroid coordinates of each steel ball identification area as the steel ball identification point;

[0054] The optimal transformation matrix determination module is used to determine the optimal transformation matrix based on multiple steel ball identification points;

[0055] The coordinate transformation module is used to perform coordinate transformation from image space to actual space based on the optimal transformation matrix.

[0056] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor running the computer program to enable the electronic device to perform the aforementioned surgical navigation robot marker recognition and registration method.

[0057] Optionally, the memory is a readable storage medium.

[0058] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0059] This invention provides a method, system, and device for identifying and registering landmarks in a surgical navigation robot. Through image segmentation algorithms and pairing calculations, it achieves automatic identification and registration of landmarks, reducing manual intervention and time consumption, and improving efficiency and accuracy. Employing multiple threshold filtering and multiple screening methods, it can eliminate interference factors, accurately locate target landmarks, and improve registration accuracy. It does not limit the algorithms and data types used and can be adjusted and changed according to actual needs, making it suitable for different types of models and image data, and exhibiting good robustness. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart of the marker recognition and registration method for surgical navigation robots in Embodiment 1 of the present invention;

[0062] Figure 2 This is a schematic diagram of the marker recognition and registration method for surgical navigation robots in Embodiment 1 of the present invention;

[0063] Figure 3 This is a diagram of the surgical navigation device in Embodiment 1 of the present invention;

[0064] Figure 4 This is a plan view of the positioning scale in Embodiment 1 of the present invention;

[0065] Figure 5 This is a three-dimensional reconstruction model of the flat plate positioning scale in Embodiment 1 of the present invention;

[0066] Figure 6 This is a registration and matching completion diagram from Embodiment 1 of the present invention;

[0067] Figure 7 This is a diagram showing the error distribution of registration markers in Embodiment 1 of the present invention.

[0068] Explanation of the attached diagram labels: First marker point -1; Second marker point -2; Third marker point -3; Fourth marker point -4; Fifth marker point -5; Binocular vision module -10; Ruler -11; Robotic arm -12; C-arm -13; Metal frame -14. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] The purpose of this invention is to provide a method, system, and device for marker recognition and registration in surgical navigation robots, which can improve the accuracy of marker recognition and registration in surgical navigation robots.

[0071] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0072] Example 1

[0073] like Figure 1 As shown, this embodiment provides a method for marker recognition and registration in a surgical navigation robot. The method is applied to a surgical navigation device, such as... Figure 3 The device includes: a robotic arm 12, a ruler 11, a binocular vision module 10, an image acquisition module, and a C-arm 13, as well as a metal frame 14 for fixing it. The robotic arm is used to carry the ruler and place it at a preset position on the human body surface. The C-arm is a separate device. The operating instruments are surgical instruments or radiographic equipment. The binocular vision module is used to acquire the ruler's posture; the image acquisition module acquires images of the ruler when it is placed at the preset position on the human body surface.

[0074] The methods include:

[0075] Step 101: Obtain an image of the ruler placed at a preset position on the human body surface. The ruler has multiple steel balls. The number of steel balls is no less than 3. The steel balls are not collinear.

[0076] Step 102: Extract the connected components of the ruler image using a connected component extraction algorithm.

[0077] Step 103: Select multiple steel ball recognition regions from multiple connected components.

[0078] Step 104: Determine the centroid coordinates of each steel ball recognition area as the steel ball recognition point.

[0079] Step 105: Determine the optimal transformation matrix based on multiple steel ball identification points.

[0080] Step 106: Complete the coordinate transformation from image space to real space based on the optimal transformation matrix.

[0081] Step 102 includes:

[0082] Step 102-1: Obtain the initial isosurface threshold, step size, and isosurface threshold.

[0083] Step 102-2: Determine the initial isosurface threshold as the current isosurface threshold.

[0084] Step 102-3: Based on the current isosurface threshold, extract the undetermined connected components of the scale image using a connected component extraction algorithm.

[0085] Step 102-4: Determine whether the number of undetermined connected components is within the range of connected component counts to obtain the first determination result. The lower limit of the range of connected component counts is 3. The upper limit of the range of connected component counts is twice the number of steel balls. If the first determination result is negative, proceed to step 102-5; if the first determination result is positive, proceed to step 102-9.

[0086] Step 102-5: Increase the current isosurface threshold value by the step size.

[0087] Step 102-6: Determine whether the current isosurface threshold has been reached, and obtain the second determination result. If the second determination result is negative, proceed to step 102-7; if the second determination result is positive, proceed to step 102-8.

[0088] Step 102-7: Return to step 102-3.

[0089] Step 102-8: Update the step size and return to step 102-2.

[0090] Step 102-9: Determine the multiple undetermined connected components extracted based on the current isosurface threshold as connected components.

[0091] Step 103 includes:

[0092] Step 103-1: Determine any connected component as the current connected component.

[0093] Step 103-2: Obtain the number of points contained in the current connected component as the current decision parameter.

[0094] Step 103-3: Determine whether the current judgment parameter is within the point range to obtain the third judgment result. If the third judgment result is negative, proceed to step 103-4; if the third judgment result is positive, proceed to step 103-5.

[0095] Step 103-4: Determine that the current connected component is a non-ball recognition region.

[0096] Step 103-5: Determine that the current connected domain is the steel ball recognition region.

[0097] Step 103-6: Update the current connected component and return to step 103-3 until all connected components are traversed, resulting in multiple ball recognition regions.

[0098] Step 105 includes:

[0099] Step 105-1: Determine any 3 steel balls on the scale as the steel balls to be determined.

[0100] Step 105-2: Construct the current measurement coordinate system based on the undetermined steel ball.

[0101] Step 105-3: Determine the current steel ball and the two undetermined steel balls as the establishing steel balls of the current measurement coordinate system.

[0102] Step 105-4: Obtain the measurement coordinates of multiple steel balls in the current measurement coordinate system.

[0103] Step 105-5: Determine any steel ball identification point as the current steel ball identification point.

[0104] Step 105-6: Determine any two steel ball recognition points other than the current steel ball recognition point as undetermined steel ball recognition points.

[0105] Step 105-7: Using the current steel ball recognition point as the origin, construct the current image coordinate system based on the undetermined steel ball recognition point.

[0106] Step 105-8: Determine the current steel ball recognition point and two undetermined steel ball recognition points as the steel ball recognition points for establishing the current image coordinate system.

[0107] Step 105-9: Obtain the image coordinates of multiple steel ball recognition points in the current image coordinate system.

[0108] Steps 105-10: Based on the measurement coordinates of multiple system-establishing steel balls in the current measurement coordinate system and the image coordinates of multiple system-establishing steel ball identification points in the current image coordinate system, construct the current transformation matrix between the current measurement coordinate system and the current image coordinate system.

[0109] Step 105-11: Based on the current transformation matrix, the measurement coordinates of multiple system-established steel ball identification points in the current measurement coordinate system are used as the measurement coordinates of the system-established steel ball identification points.

[0110] Steps 105-12: Based on the measurement coordinates of multiple system-establishing steel ball identification points and the measurement coordinates of each system-establishing steel ball in the current measurement coordinate system, determine the transformation error value of the current transformation matrix.

[0111] Step 105-13: Update the undetermined steel ball identification points and return to step 105-7 until all combinations of undetermined steel ball identification points are traversed, and the transformation error value of the transformation matrix between the current measurement coordinate system and multiple image coordinate systems constructed with the current steel ball identification point as the origin is obtained.

[0112] Step 105-14: Update the current steel ball identification point and return to step 105-6 until all steel ball identification points are traversed, and obtain the transformation error value of the transformation matrix between the current measurement coordinate system and multiple image coordinate systems.

[0113] Step 105-15: Update the undetermined steel balls and return to step 105-2 until all combinations of undetermined steel balls are traversed, and the transformation error values ​​of the transformation matrices between different measurement coordinate systems and multiple image coordinate systems are obtained.

[0114] Steps 105-16: Determine the transformation matrix corresponding to the minimum transformation error value as the optimal transformation matrix.

[0115] Steps 105-17: Determine the measurement coordinate system corresponding to the optimal transformation matrix as the actual surgical space coordinate system.

[0116] Steps 105-18: Determine the image coordinate system corresponding to the optimal transformation matrix as the image space coordinate system.

[0117] This invention does not fix the origin of the marker points, allowing for arbitrary values. The algorithm can automatically calculate the optimal registration parameters through permutations and combinations in subsequent calculations. Furthermore, this method has strong scalability for the number of marker points, requiring only N≥3. This embodiment aims to identify and locate marker points in a 3D image. First, image preprocessing is performed, and a connected component extraction algorithm is used to identify all regions that could potentially contain steel balls. Then, the centroid coordinates of each region are calculated and saved. Image preprocessing includes thresholding and Gaussian smoothing to improve data processing efficiency and accuracy, simplify the data structure, and facilitate further data analysis and visualization. After image preprocessing, the image is converted from vtkImageData type data to vtkPolyData type data to facilitate connected component extraction.

[0118] like Figure 2When extracting connected domains, the following three-step operation is performed: (1) setting a threshold range [T2, T3] for an isosurface to control the number of connected domains; (2) assuming that when the number of extracted connected domains satisfies 3 < NUM < M×N, all marker point regions are necessarily included, and the number of finally obtained regions is at least more than twice the number of steel balls to meet the requirement of containing at least three steel balls; (3) automatically removing interference regions that cannot be steel ball regions. The purpose of these operations is to improve the execution efficiency of the algorithm. The search is stopped when the number of regions meeting the conditions is obtained, and the centroid coordinates of each region are calculated and stored; the number of regions containing marker points is definitely greater than the actual number of marker points, therefore, a permutation and combination method is adopted for registration with actual marker points.

[0119] The specific operation of the permutation and combination method is as follows.

[0120] (1) storing these point coordinates respectively by Vector, then for N calibrated steel ball coordinates and each identified coordinate, there are the following pairing situations.

[0121] (2) According to the right-hand rule, three points can construct a spatial Cartesian coordinate system, so selecting three points from N steel balls has C(N, 3) combinations. Considering the order problem of identification points, selecting 3 points from the identification points has A(FNUM, 3) selection methods, wherein FNUM represents the number of finally identified regions.

[0122] (3) using vector <vector3d>Store all possible combinations. Assuming the first point of each possibility is the origin, there are a total of C(N,3)*A(FNUM,3) matching relationships, resulting in C(N,3)*A(FNUM,3) sets of matching parameters.

[0123] (4) Using the coordinates of the three selected points, calculate the theoretical coordinates of each point in reverse with the matching parameters, and calculate the distance error between the theoretical coordinates and the measured coordinates. Add the error values ​​together. After all the calculations are completed, there will be C(N,3)×A(FNUM,3) sets of errors. The matching parameter corresponding to the smallest error value is the required matching parameter. In fact, theoretically there will be C(N,3) sets of parameters, all of which are correct because N points may be the origin. However, considering the influence of point layout, only one set is the most accurate.

[0124] like Figures 4-5 Taking a flat plate positioning ruler with five steel ball markers of the same radius as an example, this embodiment will be specifically explained.

[0125] First, a flat positioning ruler with five steel ball markers of equal radius is designed. These five points are coplanar, but any three points are not collinear. Then, the surgical model and the positioning ruler are scanned using CBCT or CT to obtain a set of standard DICOM images. These images are then reconstructed in three dimensions and saved as vtkImageData. The flowchart of the method provided in this embodiment is as follows: Figure 2 As shown, the specific steps include:

[0126] Step 1. Load the standard DICOM sequence image into the developed marker recognition software. Use the algorithm classes encapsulated in the image processing and visualization toolkit VTK to perform thresholding and Gaussian smoothing on the image to improve data processing efficiency and simplify the data structure. At the same time, for easy visualization, the threshold is set to 100 based on empirical values.

[0127] Step 2. Convert the preprocessed image from Step 1 (data of type vtkImageData, defined in the VTK image processing and visualization toolkit) to vtkPolyData (data of type vtkPolyData, defined in the VTK image processing and visualization toolkit) using a pre-defined algorithm class within VTK. For the resulting vtkpolydata data, first set the upper limit of the isosurface threshold T for the steel ball to 300 based on experience. Then, specify an initial threshold greater than zero (t0 = 40 in this experiment) and set the step size t = 10. Next, write a loop statement to update the value in each iteration. The threshold is input into the VTK-encapsulated isosurface extraction algorithm, and then the VTK-encapsulated connected component extraction algorithm is used to count the number of connected components. This algorithm can obtain the number of points contained in each connected component and the coordinate value of each point. For a certain region, the coordinate values ​​of the points constituting the region are added together and then divided by the number of points to obtain the centroid coordinates of the connected region. Since the steel ball used in the experiment is very small, based on multiple experimental measurements, a threshold [20, 800] for the number of points contained in a steel ball was set. When the number of points in a certain connected component is >800 or <20, the region is considered not to be a steel ball region. At this time, the program can automatically remove this region so that it does not participate in the subsequent calculation.

[0128] Step 3. In Step 2, when the loop reaches a certain step and finds that the number of connected components extracted (3 > NUM) or NUM > 2 × 5, it immediately jumps out of the current loop and proceeds to the next loop. This avoids the time overhead of calculating the centroid coordinates. The specific implementation is as follows: As mentioned above, a step size is set in the loop statement, so the loop will execute multiple times if the condition is met. In order to save time, a threshold range of [3, 2 × N] for the number of connected components is set, where N represents the number of steel balls on the scale. Through experiments, it is found that when the number of connected components obtained meets this range, it must contain at least 3 steel ball points. Therefore, the program will calculate the number of connected components obtained in each loop and determine whether it meets the above range. If it does not meet the range, the continue statement is used to immediately jump out of the current loop and no longer execute the subsequent functions, directly starting the next loop.

[0129] Step 4. For all centroid regions extracted in Step 1, the number must be greater than or equal to 3. Therefore, in order to register with the actual marker points, a permutation and combination method is adopted. According to the right-hand rule, three points can construct a spatial Cartesian coordinate system. Then, there are C(5,3) = 10 combinations of selecting three from the given 5 steel ball coordinates. Considering the order of the identification points, there are A(8,3) = 336 ways to select 3 from the identification points. In this embodiment, 8 regions are finally obtained. If we assume that the first coordinate point of each possibility is the origin, then there are a total of C(5,3) × A(8,3) = 3360 matching relationships, which will result in 3360 sets of registration parameters.

[0130] Step 5. For the 3360 sets of registration parameters mentioned in Step 4, theoretically, there would be C(5,3) = 10 sets of parameters that are correct because all 5 points could be the origin. However, considering the influence of point layout, only one set is the most accurate. To find the most suitable set of parameters, a cyclic back-calculation of theoretical values ​​is used. In this experiment, the scale has 5 marker points, so there are 5 sets of measured coordinate values. When identifying the marker points, 8 regions were finally selected, so there are 8 corresponding centroid coordinates of the regions. Knowing any 3 points in space, a spatial coordinate system can be established. Therefore, when calculating the transformation relationship (registration matrix) between the scale's real space and the scale's model space, only 3 points need to be taken from each. Here, there are C(5,3) ways to take 3 points from the 5 points in the real space. When considering the order of the points, there are A(8,3) ways to take points from the 5 points in the model space. When using 3 pairs of points to calculate the transformation relationship between the real space and the model space, there are C(5,3) × A(8,3) sets of parameters. This experiment calculates the transformation relationship from the model space to the model space. The transformation relationship in real space allows for the calculation of a registration matrix for each pair of points, resulting in a total of C(5,3)×A(8,3) registration matrices. Multiplying the three coordinate values ​​of each selected model point by this registration matrix yields a set of three theoretical measurements. Calculating the distance between the points corresponding to these theoretical and actual measurements gives the distance error between the points. Adding the errors of the three points together gives the total error, which is then divided by 3 to obtain the average error of the pair. After calculating each matching relationship, C(5,3)×A(8,3) error values ​​are obtained. The matching parameter corresponding to the smallest set of error values ​​is the required matching parameter.

[0131] Step 6. Regarding the inverse calculation of theoretical values ​​mentioned in Step 5, the specific implementation is as follows: for any set of selected measurement and identification values ​​(both consisting of 3 points), a spatial rectangular coordinate system can be constructed by first calculating the unit vector and then using two cross products. Then, two 4×4 matrices M1 and M2 are used to represent the real space and model space coordinate systems respectively. Finally, M2×M... T =M1 can transform the model space to the real space, where M T This represents a 4×4 transformation matrix from model space to real space. With this transformation matrix, knowing the coordinates P2 of any point in model space allows us to use P1 = M... T ×P2 calculates the corresponding real space value P1. In this embodiment, the inverse calculation is to use the calculated transformation matrix to calculate the measured value of the real space corresponding to the recognition point in the model by left multiplying the transformation matrix; that is, to use the set of transformation matrices and the corresponding recognition coordinates to inversely calculate its theoretically corresponding calibration value.

[0132] Step 7. Using the obtained transformation matrix, calculate the calibration values ​​in the model space and use VTK visualization technology. When the coordinates of the corresponding point in the model space are calculated, a small-radius sphere can be drawn at that coordinate location to indicate the position of the point. This completes the identification and registration of the positioning scale markers. Figure 6 As shown, the identification and matching of 5 landmark points in the 3D reconstructed image were completed.

[0133] In specific step 6, a portion of the data was recorded using multiple models for three-stage recognition and registration. The results are recorded below.

[0134] (1) Automatically select matching points as marker point 2, marker point 4 and marker point 5.

[0135] The coordinates of marker points 2, 4, and 5 are determined by coordinate measuring machine (CMM) measurement.

[0136] (62.865, -34.988, -27.170).

[0137] (91.106, -20.803, -42.200).

[0138] (105.108, 4.917, -48.480).

[0139] The coordinates of the matched marker points 2, 4, and 5, transformed into a unified coordinate system, are as follows:

[0140] (62.866, -34.987, -27.171).

[0141] (91.106, -20.803, -42.200).

[0142] (105.073, 4.867, -48.480).

[0143] After matching, the deviation of marker 2 is 0.017mm, marker 4 is the aligned origin with no deviation, and the deviation of marker 5 is 0.061mm.

[0144] (2) Automatically select matching points as marker point 1, marker point 2, and marker point 4. Through coordinate measuring machine (CMM) measurement, the coordinates of marker point 1, marker point 2, and marker point 4 are...

[0145] (57.447, 9.106, -20.768).

[0146] (63.336, -34.933, -26.135).

[0147] (91.920, -20.809, -40.677).

[0148] The coordinates of the matched marker points 1, 2, and 4, transformed into a unified coordinate system, are as follows:

[0149] (57.435, 9.129, -20.760).

[0150] (63.400, -34.901, -26.168).

[0151] (91.920, -20.809, -40.677).

[0152] After matching, the deviation of marker point 1 is 0.027mm, the deviation of marker point 2 is 0.071mm, and the origin of marker point 4 is aligned without deviation.

[0153] (3) Automatically select matching points as marker point 1, marker point 2, and marker point 5. Through coordinate measuring machine (CMM) measurement, the coordinates of marker point 1, marker point 2, and marker point 5 are:

[0154] (23.631, -40.775, 21.108).

[0155] (66.039, -34.724, 40.030).

[0156] (13.275, -49.198, 38.105).

[0157] The coordinates of the matched marker points 1, 2, and 5, transformed into a unified coordinate system, are as follows:

[0158] (23.631, -40.775, 21.108).

[0159] (66.022, -34.726, 40.022).

[0160] (13.322, -49.152, 38.005).

[0161] After matching, marker point 1 is aligned with the origin without deviation, marker point 2 has a deviation of 0.018mm, and marker point 3 has a deviation of 0.072mm.

[0162] (4) Automatically select matching points as marker point 1, marker point 2, and marker point 5. Through coordinate measuring machine (CMM) measurement, the coordinates of marker point 1, marker point 2, and marker point 5 are:

[0163] (57.135, 9.119, -21.318).

[0164] (62.865, -34.988, -27.170).

[0165] (105.108, 4.917, -48.480).

[0166] The coordinates of the matched marker points 1, 2, and 5, transformed into a unified coordinate system, are as follows:

[0167] (57.135, 9.041, -21.322).

[0168] (62.865, -34.988, -27.170).

[0169] (105.066, 4.878, -48.459).

[0170] After matching, the deviation of marker point 1 is 0.080mm, marker point 2 is the aligned origin with no deviation, and the deviation of marker point 5 is 0.063mm.

[0171] (5) Automatically select matching points as marker point 1, marker point 3, and marker point 4. Through coordinate measuring machine (CMM) measurement, the coordinates of marker point 1, marker point 3, and marker point 4 are...

[0172] (57.135, 9.119, -21.318).

[0173] (84.035, -2.788, -37.211).

[0174] (91.106, -20.803, -42.200).

[0175] The coordinates of the matched marker points 1, 3, and 4, transformed into a unified coordinate system, are as follows:

[0176] (57.135, 9.119, -21.318).

[0177] (84.002, -2.773, -37.192).

[0178] (91.107, -20.781, -42.200).

[0179] After matching, marker 1 is aligned with the origin without deviation, marker 3 has a deviation of 0.107mm, and marker 4 has a deviation of 0.022mm.

[0180] Excluding the aligned origin, the above 6 sets of data contain 10 points. The average deviation of these 10 points is 0.071 mm, and the variance is 0.00147. The data deviation distribution is as follows: Figure 7 As shown.

[0181] Example 2

[0182] In order to implement the system corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a surgical navigation robot marker recognition and registration method is provided below, including:

[0183] The ruler image acquisition module is used to acquire an image of the ruler when it is placed at a preset position on the human body surface. The ruler has multiple steel balls. The number of steel balls is no less than 3. The steel balls are not collinear.

[0184] The connected component extraction module is used to extract connected components from the ruler image using a connected component extraction algorithm.

[0185] The steel ball recognition region acquisition module is used to filter multiple steel ball recognition regions from multiple connected domains.

[0186] The steel ball identification point determination module is used to determine the centroid coordinates of each steel ball identification area as the steel ball identification point.

[0187] The optimal transformation matrix determination module is used to determine the optimal transformation matrix based on multiple steel ball identification points.

[0188] The coordinate transformation module is used to perform coordinate transformation from image space to real space based on the optimal transformation matrix.

[0189] Example 3

[0190] This embodiment provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the aforementioned surgical navigation robot marker recognition and registration method. The memory is a readable storage medium.

[0191] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0192] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for marker recognition and registration in a surgical navigation robot, characterized in that, The methods include: The image obtained when the ruler is placed at a preset position on the human body surface is the ruler image; The scale is equipped with multiple steel balls; The number of steel balls is not less than 3; The multiple steel balls are not collinear; The connected components of the ruler image are extracted using a connected component extraction algorithm; Multiple steel ball recognition regions are selected from the multiple connected domains; The centroid coordinates of each steel ball recognition area are determined as the steel ball recognition point; Determine the optimal transformation matrix based on multiple steel ball identification points; Based on the optimal transformation matrix, the coordinate transformation from image space to actual surgical space is completed; The step of extracting connected components from the ruler image using a connected component extraction algorithm includes: Obtain the initial isosurface threshold, step size, and isosurface threshold; The initial isosurface threshold is set as the current isosurface threshold. Based on the current isosurface threshold, the undetermined connected components of the scale image are extracted using a connected component extraction algorithm; Determine whether the number of undetermined connected components is within the range of connected component numbers to obtain a first determination result; the lower limit of the range of connected component numbers is 3; the upper limit of the range of connected component numbers is twice the number of steel balls. If the first judgment result is negative, then the value of the current isosurface threshold is increased by the step size; Determine whether the current isosurface threshold has reached the isosurface threshold, and obtain a second determination result; If the second judgment result is negative, then return to the step "Based on the current isosurface threshold, use the connected component extraction algorithm to extract the undetermined connected components of the scale image"; If the second judgment result is yes, then update the step size and return to the step "determine the initial isosurface threshold as the current isosurface threshold"; If the first judgment result is yes, then the multiple undetermined connected components extracted based on the current isosurface threshold are determined to be connected components.

2. The method for marker recognition and registration of a surgical navigation robot according to claim 1, characterized in that, Filtering multiple steel ball recognition regions from multiple connected components includes: Determine any of the aforementioned connected components as the current connected component; The number of points contained in the current connected component is obtained as the current decision parameter; Determine whether the current judgment parameter is within the point range to obtain a third judgment result; If the third judgment result is negative, then the current connected region is determined to be a non-steel ball recognition region; If the third judgment result is yes, then the current connected region is determined to be the steel ball recognition region; Update the current connected component and return to the step "determine whether the current judgment parameter is in the point range and obtain the third judgment result" until all connected components are traversed to obtain multiple steel ball recognition regions.

3. The method for marker recognition and registration of a surgical navigation robot according to claim 1, characterized in that, Based on multiple steel ball recognition points, the optimal transformation matrix is ​​determined, including: Identify any three steel balls on the scale as undetermined steel balls; Construct the current measurement coordinate system based on the undetermined steel ball; The current steel ball and the two undetermined steel balls are identified as the establishing steel balls of the current measurement coordinate system; Obtain the measurement coordinates of multiple steel balls in the current measurement coordinate system; Determine any steel ball identification point as the current steel ball identification point; Determine any two steel ball identification points other than the current steel ball identification point as undetermined steel ball identification points; Using the current steel ball identification point as the origin, construct the current image coordinate system based on the undetermined steel ball identification points; The current steel ball identification point and the two undetermined steel ball identification points are determined as the steel ball identification points for establishing the current image coordinate system; Obtain the image coordinates of multiple steel ball identification points in the current image coordinate system; Based on the measurement coordinates of multiple system-establishing steel balls in the current measurement coordinate system and the image coordinates of multiple system-establishing steel ball identification points in the current image coordinate system, a current transformation matrix between the current measurement coordinate system and the current image coordinate system is constructed. Based on the current transformation matrix, the measurement coordinates of the multiple system steel ball identification points in the current measurement coordinate system are the system steel ball identification point measurement coordinates. Based on the measurement coordinates of multiple system-establishing steel ball identification points and the measurement coordinates of each system-establishing steel ball in the current measurement coordinate system, the transformation error value of the current transformation matrix is ​​determined. Update the undetermined steel ball identification points and return to the step "Construct the current image coordinate system based on the undetermined steel ball identification points with the current steel ball identification point as the origin" until all combinations of undetermined steel ball identification points are traversed to obtain the transformation error value of the transformation matrix between the current measurement coordinate system and the multiple image coordinate systems constructed with the current steel ball identification point as the origin; Update the current steel ball identification point and return to the step "determine any two steel ball identification points other than the current steel ball identification point as undetermined steel ball identification points" until all steel ball identification points are traversed to obtain the transformation error value of the transformation matrix between the current measurement coordinate system and multiple image coordinate systems; Update the undetermined steel ball and return to the step "Construct the current measurement coordinate system based on the undetermined steel ball" until all combinations of undetermined steel balls are traversed to obtain the transformation error values ​​of the transformation matrices between different measurement coordinate systems and multiple image coordinate systems; The transformation matrix corresponding to the minimum transformation error value is determined as the optimal transformation matrix.

4. The method for marker recognition and registration of a surgical navigation robot according to claim 3, characterized in that, After determining that the transformation matrix corresponding to the minimum transformation error value is the optimal transformation matrix, the following steps are also included: The measurement coordinate system corresponding to the optimal transformation matrix is ​​determined to be the actual surgical space coordinate system; The image coordinate system corresponding to the optimal transformation matrix is ​​determined as the image space coordinate system.

5. A marker recognition and registration system for a surgical navigation robot, characterized in that, The system uses the surgical navigation robot marker recognition and registration method as described in any one of claims 1-4, and the system includes: The ruler image acquisition module is used to acquire an image of the ruler when it is placed at a preset position on the human body surface; the ruler is provided with multiple steel balls; the number of steel balls is not less than 3; the multiple steel balls are not collinear; A connected component extraction module is used to extract the connected components of the ruler image using a connected component extraction algorithm; A steel ball recognition region acquisition module is used to filter multiple steel ball recognition regions from multiple connected domains; The steel ball identification point determination module is used to determine the centroid coordinates of each steel ball identification area as the steel ball identification point; The optimal transformation matrix determination module is used to determine the optimal transformation matrix based on multiple steel ball identification points; The coordinate transformation module is used to perform coordinate transformation from image space to actual space based on the optimal transformation matrix.

6. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program and the processor runs the computer program to enable the electronic device to perform a surgical navigation robot marker recognition and registration method according to any one of claims 1 to 4.

7. An electronic device according to claim 6, characterized in that, The memory is a readable storage medium.

Images