A robot execution precision evaluation method

Abstract

The application discloses a robot execution precision evaluation method, comprising the following steps: obtaining preoperative images containing a planned channel and postoperative images containing a surgical needle; obtaining an optimal registration matrix of the preoperative images and the postoperative images; extracting a postoperative needle channel in the postoperative images; converting the planned channel into the postoperative images or converting the postoperative needle channel into the preoperative images according to the optimal registration matrix; and calculating execution precision according to position information of the planned channel and the postoperative needle channel in the same images. The application considers the interference of dynamic conditions on robot execution precision, can effectively judge the error between the planned channel and the robot execution result, and better guides doctors to perform surgery.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a method for evaluating the execution accuracy of a robot. Background Technology

[0002] In recent years, surgical robot technology has developed rapidly. Robot navigation systems make full use of patients' medical imaging information to provide doctors with more three-dimensional navigation and precise angle positioning. They have the characteristics of high operational precision, good repeatability and high stability, making the application of robot navigation systems in the medical field increasingly popular.

[0003] When a robotic navigation system is in operation, it first establishes spatial coordinate transformation relationships between the robot, imaging equipment, and tracking equipment through registration using markers or anatomical features. The doctor then uses these coordinate transformation relationships, combined with the pose of the planned pathway on the image, to determine the target pose the robot must reach. The doctor then manipulates the robot to achieve this target pose. These spatial transformation relationships are established under static conditions. External disturbances, such as human breathing, human movement, or bed movement, will alter these relationships. Therefore, if the human body and imaging equipment are not rigidly bound together, the actual puncture process involves unconsidered dynamic conditions. Under these dynamic conditions, preoperative and postoperative images taken by the same imaging equipment at different times will exhibit varying degrees of deviation.

[0004] Currently available robots of this type fully consider the accuracy of each device under static conditions to achieve precise puncture, but they do not take into account the interference of various dynamic conditions on accuracy during actual surgery, resulting in a large error in the accuracy evaluation of the robot's execution results. Summary of the Invention

[0005] Purpose of the invention: To address the above-mentioned problems, this invention provides a method for evaluating the execution accuracy of a robot, which can effectively and accurately assess the error between the planned channel and the robot's execution result, and is of great significance for improving the success rate of needle insertion on the first attempt.

[0006] A method for evaluating robot execution accuracy includes the following steps:

[0007] Acquire preoperative images containing the planned access route and postoperative images containing the surgical needle;

[0008] Find the optimal registration matrix between preoperative and postoperative images;

[0009] Extract the postoperative needle tract from the postoperative images;

[0010] The planned channel is converted to the postoperative image or the postoperative needle path is converted to the preoperative image according to the optimal registration matrix.

[0011] The execution accuracy is calculated based on the position information of the planned channel and the postoperative needle path in the same image.

[0012] Specifically, the execution accuracy is calculated based on the angle between the vectors of the planned channel and the postoperative needle path, and the distance between the planned channel and the postoperative needle path.

[0013] More specifically, the distance is the Euclidean distance between the end point of the postoperative needle path and the end point of the planned channel.

[0014] Furthermore, the optimal registration matrix for the preoperative and postoperative images is obtained as follows:

[0015] (1) Establish the Euler transformation matrix of the preoperative image and the postoperative image;

[0016] (2) Transform the preoperative image using the Euler transformation matrix to obtain the preoperative transformed image;

[0017] (3) Calculate the mutual information evaluation measure between the preoperative transformed images and the postoperative images;

[0018] (4) Calculate the current Euler transform matrix using the mutual information evaluation measure as the loss function, and update the Euler transform matrix of step (1) to the current Euler transform matrix;

[0019] (5) Repeat steps (2)-(4) until the number of iterations reaches the maximum set number of iterations or the iteration step size reaches the minimum step size of the last stage. Then stop the iteration and output the Euler transformation matrix corresponding to the minimum value of the mutual information evaluation measure as the best registration matrix.

[0020] Furthermore, the mutual information evaluation measure F(img) is calculated. A ′,img B Specifically:

[0021] F(img A ′,img B )=-H(img B )+H(img A ′) / H(img B ,img A ′)

[0022]

[0023] Where H(x,y) represents the mutual information between image x and image y, H(img A ′), H(img B () represent preoperative transformed images (img) A Postoperative images (img)B With respect to its own mutual information, p(x,y) is the joint histogram of image x and image y, and p(x) or p(y) represents the histogram of image x or image y.

[0024] Furthermore, the current Euler transformation matrix H euler t The calculation is as follows:

[0025] H euler t =H euler t-1 -λ*α*g t-1

[0026]

[0027]

[0028] Among them, H euler t Let represent the transformation matrix obtained in the t-th calculation; λ is the penalty factor, which is reduced when the gradient direction of the variable changes at different resolution stages; α is the learning rate, which is set according to different resolution stages; (x,y,z,rx,ry,rz) is the vector representation of the corresponding Euler transformation matrix.

[0029] More specifically, each resolution stage has a minimum iteration step size, and the minimum iteration step size of the subsequent resolution stage is smaller than the minimum iteration step size of the previous resolution stage. The initial learning rate of the subsequent resolution stage is the iteration step size of the previous resolution stage.

[0030] More specifically, three resolution stages are set;

[0031] In the first stage, the image is reduced by a factor of 8, α is 2, λ is 0.4, and the minimum iteration step size is 1;

[0032] The image is reduced by a factor of 4 in the second stage, α is the iteration step size of the previous stage, λ is 0.8, and the minimum iteration step size is 0.2 times that of the previous stage;

[0033] In the third stage, no reduction processing is performed. α is the iteration step size of the previous stage, λ is 0.8, and the minimum iteration step size is 0.2 times that of the previous stage.

[0034] Furthermore, the extraction of the puncture needle from the postoperative images specifically involves:

[0035] Calculate the segmentation threshold of the postoperative needle tract in the postoperative image;

[0036] The image is segmented based on the calculated segmentation threshold and then binarized to obtain the needle path region.

[0037] Through connectivity analysis, the region with the largest aspect ratio among the connected blocks was identified as the needle channel region.

[0038] Furthermore, the planned channels in the preoperative images are transformed into the postoperative images using the optimal registration matrix. The execution accuracy is calculated based on the information of the planned channels and the postoperative needle paths in the postoperative images. Specifically:

[0039] [p1; p2] = H euler *[p A1 ;p A2 ]

[0040] θ=atan2(|v t ×v|,v t T *v)

[0041] d = ||p B2 p2‖ 2

[0042] Among them, H euler For the optimal registration matrix, p A1 p A2 p1 and p2 represent the positions of the two endpoints of the planned channel in the preoperative image, respectively, and p2 and p3 represent the positions of the two endpoints of the planned channel after transformation to the postoperative image. B2 v is the end point of the postoperative needle tract. t denoted by , v represents the vector where the planned channel is located in the postoperative image, and v represents the vector where the postoperative needle path is located.

[0043] Specifically, before the step of obtaining the optimal transformation matrix, the method further includes the step of segmenting the preoperative image and the postoperative image to obtain the human body region image, wherein the optimal registration matrix is ​​the optimal registration matrix between the human body region images corresponding to the preoperative image and the postoperative image.

[0044] More specifically, the step of segmenting to obtain the human body region image is as follows:

[0045] Extract the axial slices of the image to be segmented, compress and map the voxel values ​​of the voxel points in the segmented image data to (0~255) using the voxel value range [-800,100], and apply adaptive thresholding to obtain the optimal segmentation threshold;

[0046] The slice images are binarized according to the optimal segmentation threshold to separate the background and foreground;

[0047] Connectivity analysis was performed on each slice image after binarization to obtain each isolated sub-region. The perimeter of each sub-region was calculated, and the sub-region with the largest perimeter was taken as the human body region.

[0048] Beneficial effects: The robot execution accuracy evaluation method of the present invention takes into account the interference of dynamic conditions on the robot execution accuracy, and can effectively and accurately evaluate the error between the planned channel and the robot execution result, which is of great significance for improving the success rate of needle insertion in one attempt. Attached Figure Description

[0049] Figure 1 A flowchart of the robot execution accuracy evaluation method of the present invention;

[0050] Figure 2 A flowchart for human body region image segmentation;

[0051] Figure 3 This is a schematic diagram of a preoperative image obscured by a mask.

[0052] Figure 4 This is a schematic diagram of a postoperative image obscured by a mask. Detailed Implementation

[0053] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0054] The flowchart of the robot execution accuracy evaluation method of the present invention is as follows: Figure 1 As shown, it includes the following steps:

[0055] (1) Obtain preoperative images containing the planned channel and postoperative images containing the surgical needle;

[0056] Preoperative images are obtained by scanning with imaging equipment, and the preoperative images containing the planned channels are obtained by planning on the preoperative images; after the operation, postoperative images containing the surgical needles are obtained by scanning with imaging equipment.

[0057] (2) The preoperative and postoperative images are segmented to obtain the corresponding human body region images.

[0058] Because the human body is lying on the equipment, the surrounding equipment, bed board, clothes, air noise, etc., will cause certain interference to the images that need to be processed. The interference area is not directly connected to the human body area image and accounts for a small proportion. In order to reduce the amount of computation and improve the accuracy of computation, the human body area image is segmented for image processing. In other embodiments, this step can be omitted and the preoperative and postoperative images can be processed directly.

[0059] There are many types of algorithms for segmenting human body regions in images. This invention uses, for example... Figure 2 The method shown is used for human body region image segmentation, specifically as follows:

[0060] (11) Extract axial slices of the image to be segmented, and use the voxel value range [-800, 100] to compress and map the voxel values ​​of the voxel points in the image to be segmented to the U8 data range, that is, to the range of (0~255), thereby enhancing the regional image of the lungs.

[0061] (12) The optimal segmentation threshold is obtained by adaptive thresholding. The image slices are binarized according to the optimal segmentation threshold to segment the background and foreground.

[0062] (13) Perform connectivity analysis on each slice image after binarization to obtain each isolated sub-region;

[0063] (14) Calculate the perimeter of each isolated sub-region, and the isolated sub-region with the largest perimeter is the human body region.

[0064] The preoperative and postoperative images were processed using the above steps to obtain the preoperative human region image (img). A and postoperative human body region images (img) B Then, by using a mask, areas other than the human body area are obscured, such as... Figure 3 , 4 As shown, subsequent calculations skip the parts obscured by the mask to reduce interference and improve calculation speed and accuracy;

[0065] (3) Obtain the optimal registration matrix for the preoperative and postoperative human body region images, including the following steps:

[0066] (31) Establish the Eulerian transformation relationship between preoperative and postoperative human body region images;

[0067] The transformation relationship between preoperative and postoperative human region images can be represented by an Euler transformation matrix. The vector form (tx, ty, tz, rx, ry, rz) represents the Euler transformation relation, which includes rotation and translation transformations, i.e., corrections for rotation and translation. Here, tx, ty, and tz are the translations in the x, y, and z directions of the transformation from the geometric center of the preoperative to the postoperative human region image, respectively; rx, ry, and rz are the Euler angles of the rotation about the x, y, and z axes of the transformation from the preoperative to the postoperative human region image, respectively. ij These are the elements corresponding to the rotation transformation matrix; image transformation is performed using the above matrix, and parameter optimization is achieved using the above vector.

[0068] (32) Using the Euler transformation matrix to transform the preoperative human region image (img) A The preoperative transformed image (img) was obtained by performing the transformation. A ′;

[0069] (33) Calculate the preoperative transformed image (img) A ′ and postoperative human body area images (img) B Mutual information evaluation measure F(img) A ′,img B );

[0070] This invention uses the standard normalized mutual information calculation formula for calculation, as follows:

[0071] F(img A ′,img B )=-H(img B )+H(img A ′) / H(img B ,img A ′)

[0072]

[0073] Where H(x,y) represents the mutual information between image x and image y, H(img A ′), H(img B ) represent the mutual information between the preoperative transformed image, the postoperative human body region image and itself, respectively, and p(x,y) is the joint histogram of image x and image y. p(x) or p(y) represents the histogram of image x or image y. The number of histogram groups used in this invention is 256.

[0074] (34) Calculate the optimal Euler transformation matrix for preoperative and postoperative human body region images.

[0075] This invention uses a regularized gradient descent algorithm, with the mutual information evaluation measure F(img) obtained in step (33) as the standard. A ′,img B The current Euler transform matrix H is calculated using this as the loss function. euler t The Euler transformation matrix H from step (31) euler Updated to the current Euler transformation matrix H euler t Repeat steps (32)-(34) until the number of iterations reaches the set maximum number of iterations or the iteration step size reaches the minimum step size of the last stage. Then stop the iteration and output the mutual information evaluation measure F(img). A ′,img B The Euler transformation matrix corresponding to the minimum value is used as the optimal registration matrix;

[0076] Current Euler transformation matrix H euler t The calculation is as follows:

[0077] H euler t =H euler t-1 -λ*λ*g t-1

[0078]

[0079]

[0080] Among them, H euler t Let represent the Euler transformation matrix obtained in the t-th update; λ is the penalty factor, which decreases when the gradient direction of the variable changes at different resolution stages; α is the learning rate, which is set according to different resolution stages.

[0081] The learning rate varies at different resolution stages. The mutual information evaluation measure F(img) is obtained by iteratively reducing α using an iteration step size l, with each resolution stage being determined. A ′,img B The minimum value is obtained, thus yielding the optimal Euler transform matrix for each resolution stage. The initial learning rate for the next resolution stage is the iteration step size l of the previous resolution stage.

[0082] In this invention, each resolution stage is further provided with a minimum iteration step size, and the minimum iteration step size of the later resolution stage is smaller than the minimum iteration step size of the earlier resolution stage.

[0083] In a specific embodiment of the present invention, three resolution stages are set;

[0084] In the first stage, the image is reduced by a factor of 8, α is 2, λ is 0.4, and the minimum iteration step size is 1; in the second stage, the image is reduced by a factor of 4, α is the iteration step size of the previous stage, λ is 0.8, and the minimum iteration step size is 0.2 times that of the previous stage; in the third stage, the image is not processed, α is the iteration step size of the previous stage, λ is 0.8, and the minimum iteration step size is 0.2 times that of the previous stage.

[0085] (4) Extract the postoperative needle tract from the postoperative images;

[0086] To evaluate the accuracy of robot execution, the extraction of the needle path after surgery is crucial. The needle itself is straight after surgery and can be regarded as a spatial line segment. However, due to the influence of its metal material or surrounding accessories, many metal artifacts are generated. This invention segments the postoperative image to obtain the needle path region by calculating the segmentation threshold of the puncture needle path. Through connectivity analysis, the region with the largest aspect ratio among each connected block is identified as the needle path region. The needle path is fitted to the identified needle path region to obtain the endpoints of the needle path, thus obtaining the two endpoints of the needle path.

[0087] (5) Calculate execution accuracy;

[0088] The preoperative image containing the planned channel is loaded from the local image. The planned channel is converted into the postoperative image using Equation (1). The angle θ between the vector of the planned channel in the postoperative image and the vector of the postoperative needle path in the postoperative image is calculated using Equation (2). The Euclidean distance d between the end point of the postoperative needle path and the end point of the planned channel is calculated using Equation (3).

[0089] [p1; p2] = H euler *[p A1 ;p A2 (1)

[0090] θ=atan2(|v t ×v|,v t T *v) (2)

[0091] d = ||p B2 p2‖ 2 (3)

[0092] Among them, H euler p is the optimal registration matrix obtained after step (3). A1 p A2 p1 and p2 represent the positions of the planned channel's entry point on the body surface and its endpoint within the body in the preoperative image, respectively. p1 and p2 represent the positions of the planned channel's entry point on the body surface and its endpoint within the body in the postoperative image after transforming the preoperative image to a postoperative image, respectively. B2 The end point of the postoperative needle path obtained by extracting the postoperative needle path from the postoperative image in step (4), v t represents the vector of the planned channel, and v represents the vector of the postoperative needle path in the postoperative image;

[0093] In other embodiments, the step of segmenting the preoperative and postoperative images to obtain the corresponding human body region images can be omitted. Instead, the preoperative and postoperative images can be processed directly, that is, step (2) can be skipped, and step (3) can be used directly to process the preoperative and postoperative images and calculate the optimal registration matrix of the preoperative and postoperative images. The calculation method used is the same as the above method, and will not be repeated here.

[0094] In the embodiments disclosed in this invention, the planned channel in the preoperative image is transformed into the postoperative image using an optimal registration matrix. The robot execution accuracy is evaluated based on the angle θ between the vectors containing the planned channel and the postoperative needle path in the postoperative image, and the Euclidean distance d between the end point of the postoperative needle path and the end point of the planned channel. In other embodiments, the postoperative needle path in the postoperative image can be transformed into the preoperative image using an optimal registration matrix. The robot execution accuracy is evaluated based on the angle θ between the vectors containing the planned channel and the postoperative needle path in the preoperative image, and the Euclidean distance d between the end point of the postoperative needle path and the end point of the planned channel. The calculation steps are the same as above and will not be repeated here.

[0095] This invention considers the impact of dynamic conditions on the evaluation of execution accuracy, calculates the optimal registration matrix of preoperative and postoperative images, and converts the planned channel or postoperative needle path into the same image based on the optimal registration matrix. The robot execution accuracy is calculated based on the position information of the two in the same image, which can effectively judge the error between the planned channel and the robot execution result, and is of great significance to improving the success rate of needle insertion in one attempt.

[0096] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations (such as quantity, shape, position, etc.) can be made to the technical solution of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A method for evaluating the execution accuracy of a robot, characterized in that: Including the following steps: Acquire preoperative images containing the planned access route and postoperative images containing the surgical needle; The Euler transformation matrix of the preoperative and postoperative images is established and the preoperative image is transformed. The mutual information evaluation measure between the transformed preoperative and postoperative images is used as the loss function. The Euler transformation matrix is ​​iteratively calculated and updated until the number of iterations reaches the set maximum number of iterations or the iteration step size reaches the minimum step size of the last stage. The optimal registration matrix of the preoperative and postoperative images is thus obtained. The Euler transformation matrix The calculation is as follows: ； ； ； in, denoted as the transformation matrix obtained in the t-th calculation; λ is the penalty factor, which is reduced when the gradient direction of the variable changes at different resolution stages; α is the learning rate, which is set according to different resolution stages. The minimum value of the mutual information evaluation measure at each resolution stage is obtained by iterating by setting the iteration step size l and reducing α. The initial learning rate of the next resolution stage is the iteration step size of the previous resolution stage. A measure of mutual information between the transformed preoperative and postoperative images; The vector representation of the corresponding Euler transform matrix is ​​used; the postoperative needle tract is extracted from the postoperative image. The planned channel is converted to the postoperative image or the postoperative needle path is converted to the preoperative image according to the optimal registration matrix. The execution accuracy is calculated based on the position information of the planned channel and the postoperative needle path in the same image.

2. The robot execution accuracy evaluation method according to claim 1, characterized in that: The execution accuracy is calculated based on the angle between the planned channel and the vector of the postoperative needle path, and the distance between the planned channel and the postoperative needle path.

3. The robot execution accuracy evaluation method according to claim 2, characterized in that: The distance is the Euclidean distance between the end point of the postoperative needle path and the end point of the planned channel.

4. The robot execution accuracy evaluation method according to claim 1, characterized in that: Calculate mutual information evaluation measure Specifically: ； ； in, Represents the mutual information between image x and image y. , These represent preoperative transformed images (img). A Postoperative images (img) B With respect to its own mutual information, p(x,y) is the joint histogram of image x and image y, and p(x) or p(y) represents the histogram of image x or image y.

5. The robot execution accuracy evaluation method according to claim 1, characterized in that: Each resolution stage has a minimum iteration step size, and the minimum iteration step size of the next resolution stage is smaller than the minimum iteration step size of the previous resolution stage. The initial learning rate of the next resolution stage is the iteration step size of the previous resolution stage.

6. The robot execution accuracy evaluation method according to claim 5, characterized in that: Set three resolution stages; In the first stage, the image is reduced by a factor of 8, α is 2, λ is 0.4, and the minimum iteration step size is 1; The image is reduced by a factor of 4 in the second stage, α is the iteration step size of the previous stage, λ is 0.8, and the minimum iteration step size is 0.2 times that of the previous stage; In the third stage, no reduction processing is performed. α is the iteration step size of the previous stage, λ is 0.8, and the minimum iteration step size is 0.2 times that of the previous stage.

7. The robot execution accuracy evaluation method according to claim 1, characterized in that: The puncture needle used to extract the postoperative images is specifically: Calculate the segmentation threshold of the postoperative needle tract in the postoperative image; The image is segmented based on the calculated segmentation threshold and then binarized to obtain the needle path region. Through connectivity analysis, the region with the largest aspect ratio among the connected blocks was identified as the needle channel region.

8. The robot execution accuracy evaluation method according to claim 3, characterized in that: The planned channels in the preoperative images are transformed into the postoperative images using the optimal registration matrix. The execution accuracy is calculated based on the information of the planned channels and the postoperative needle paths in the postoperative images. Specifically: ； ； ； in, For the optimal registration matrix, , These represent the positions of the two endpoints of the planned access route in the preoperative images. These represent the positions of the two endpoints after the planned channel has been transformed into the postoperative image. v is the end point of the postoperative needle tract. t denoted by , v represents the vector where the planned channel is located in the postoperative image, and v represents the vector where the postoperative needle path is located.

9. The robot execution accuracy evaluation method according to any one of claims 1 to 8, characterized in that: Before the step of obtaining the optimal registration matrix, the method further includes the step of segmenting the preoperative image and the postoperative image to obtain human body region images, wherein the optimal registration matrix is ​​the optimal registration matrix between the human body region images corresponding to the preoperative image and the postoperative image.

10. The robot execution accuracy evaluation method according to claim 9, characterized in that: The specific steps for segmenting to obtain human body region images are as follows: Extract the axial slices of the image to be segmented, compress and map the voxel values ​​of the voxel points in the segmented image data to the range of (0~255) using the voxel value range [-800, 100], and apply adaptive thresholding to obtain the optimal segmentation threshold; The slice images are binarized according to the optimal segmentation threshold to separate the background and foreground; Connectivity analysis was performed on each slice image after binarization to obtain each isolated sub-region. The perimeter of each sub-region was calculated, and the sub-region with the largest perimeter was taken as the human body region.

Images