Automatic kidney stone tracking system and method of flexible ureteroscope surgical robot based on optimal visual servo

Abstract

The invention discloses an automatic kidney stone tracking system and method for a flexible ureteroscope surgical robot based on optimal visual servo. The system comprises a target recognition module, a feature extraction module, a posture solving module and a posture adjusting module. The target recognition module is used for acquiring image data of a ureter cavity and kidney stone and processing the image data to obtain a stone target area; the feature extraction module is used for performing feature extraction on the stone target area to obtain an actual image feature and an actual interaction matrix of an actual image pose, and is also used for calculating an expected image feature and an expected interaction matrix of an expected image pose; the attitude solving module is used for calculating the optimal solution of the motion attitude of each joint of the robot by using an optimal control algorithm and a control barrier function based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix; and the posture adjustment module is used for controlling the posture of the robot based on the motion posture optimal solution.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical robots, and in particular relates to a kidney stone automatic tracking system and method of a ureteroscopic surgical robot based on optimal visual servoing. Background Art

[0002] In recent years, the number of patients with kidney stones has been increasing. Ureteroscope surgical robots for kidney stones have also been gradually used in surgery. However, endoscopic surgery often requires doctors to concentrate highly, and frequent surgical operations can easily cause mental fatigue, thereby reducing the accuracy of the surgery. It may even cause the observation target to be lost during the operation, causing the surgery to be delayed or fail.

[0003] The ureteroscopic surgical robot can be used for ureteroscopic lithotripsy. The automatic tracking method and system for kidney stones applied to the ureteroscopic surgical robot can achieve the function of automatically tracking kidney stone lesions, improve the success rate of kidney stone crushing surgery and the efficiency of surgical operators. This new technology should be able to reduce the energy consumed by doctors when performing ureteroscopic examinations, while maintaining reliability in various complex situations. In addition, the technology also designs visibility constraints for kidney stone targets, thereby reducing the risk of kidney stone targets being lost in the endoscopic field of view during surgery and reducing operational risks.

[0004] There are currently the following defects in ureteroscopic surgery. First, traditional ureteroscopic surgery requires doctors to concentrate more, and frequent operations can easily cause doctors to become fatigued, endangering the safety and efficiency of the operation. Secondly, the automatic tracking of kidney stones by general ureteral robots requires images with high integrity, and the intracavitary environment often cannot provide good image information. Finally, due to the presence of disturbances, the kidney stone target may be out of the camera's field of view when the ureteroscopic robot is tracking kidney stones. Summary of the invention

[0005] The present invention aims to solve the deficiencies of the prior art and provides the following solutions:

[0006] The kidney stone automatic tracking system of the ureteroscopic surgical robot based on optimal visual servoing includes: a target recognition module, a feature extraction module, a posture solving module and a posture adjustment module;

[0007] The target recognition module is used to obtain image data of the ureteral cavity and kidney stones, and process the image data to obtain a target area of ​​stones;

[0008] The feature extraction module is used to extract features from the target area of ​​the stone to obtain actual image features and actual interaction matrix of actual image posture, and is also used to calculate expected image features and expected interaction matrix of expected image posture;

[0009] The posture solving module is used to calculate the optimal solution of the motion posture of each joint of the robot by using the optimal control algorithm and the control obstacle function based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix;

[0010] The posture adjustment module is used to control the posture of the robot based on the optimal solution of the motion posture.

[0011] Preferably, the target recognition module includes: an image acquisition unit and an image processing unit;

[0012] The image acquisition unit is used to obtain the image data of the ureteral cavity and kidney stones;

[0013] The image processing unit uses a neural network algorithm to identify the contour of the kidney stone in the image data, and uses an adaptive threshold method to binarize the image data to obtain a binary image, and then marks the contour in the binary image to obtain the stone target area.

[0014] Preferably, the feature extraction module includes: an actual image extraction unit and a desired image extraction unit;

[0015] The actual image extraction unit is used to extract features from the binarized target stone area to obtain the actual image features of the actual image posture and the actual interaction matrix:

[0016] s1=[x n1 y n1 a n1 s x1 s y1 α1] T ,

[0017]

[0018] Among them, s1 represents the actual image features, L1 represents the actual interaction matrix, and x n1 represents the actual image feature that controls the translational freedom in the X direction, y n1 represents the actual image feature that controls the Y-direction translational freedom, a n1 represents the actual image feature that controls the Z-direction translational freedom, s x1 represents the actual characteristics of rotation around the X axis, s y1represents the actual feature rotated around the Y axis, α1 represents the actual direction angle of the target area, and T represents the transpose of the matrix;

[0019] The expected image extraction unit calculates the expected image features and the expected interaction matrix of the expected image pose based on the preset expected image:

[0020] s0=[x n0 y n0 a n0 s x0 s y0 α0] T ,

[0021]

[0022] Among them, s0 represents the expected image features, L0 represents the expected interaction matrix, and x n0 represents the desired image feature that controls the translational freedom in the X direction, y n0 represents the desired image feature that controls the Y-direction translational freedom, a n0 represents the desired image feature that controls the Z-direction translational freedom, s x0 represents the desired characteristics of rotation around the X axis, s y0 represents the desired feature of rotation around the Y axis, and α0 represents the desired direction angle of the target area.

[0023] Preferably, the posture solving module includes: a camera expected speed calculation unit, an obstacle function setting unit and a robot posture calculation unit;

[0024] The camera expected speed calculation unit calculates the camera expected speed based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix:

[0025]

[0026] e=s1-s0,

[0027]

[0028] Among them, v d represents the expected speed of the camera, λ represents a constant, e represents the image feature error, and L s represents the interaction matrix, represents the inverse matrix of the interaction matrix;

[0029] The barrier function setting unit is used to set the barrier function:

[0030]

[0031] Where B represents the barrier function, represents the image feature difference that controls the translational freedom in the X direction, represents the image feature difference controlling the Y-direction translational freedom, and ρ represents the visibility constraint radius;

[0032] The robot posture calculation unit is used to calculate the optimal solution of the robot's motion posture based on the optimal control algorithm and in combination with the obstacle function:

[0033] B (1) ≥-k b B,

[0034] min M=||v d -v c || 2 ,

[0035] stv c =J robot Δq,

[0036]

[0037] Among them, B (1) represents the first-order derivative of the barrier function, k b represents the proportionality constant, J robot represents the Jacobian matrix of the robot, v c represents the actual speed of the camera, Δq represents the optimal solution for the motion posture, represents the rotation speed of the robot, represents the bending speed of the robot, Indicates the feed speed of the robot.

[0038] The present invention also provides a method for automatically tracking kidney stones by a ureteroscopic surgical robot based on optimal visual servoing. The method is applied to any of the above-mentioned systems and comprises the following steps:

[0039] Acquire image data of the ureteral cavity and kidney stones, and process the image data to obtain a target area of ​​stones;

[0040] Extracting features of the target area of ​​the stone to obtain actual image features and actual interaction matrix of the actual image position, which are also used to calculate expected image features and expected interaction matrix of the expected image position;

[0041] Based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix, an optimal solution for the motion posture of each joint of the robot is calculated using an optimal control algorithm and a control obstacle function;

[0042] The posture of the robot is controlled based on the optimal solution of the motion posture.

[0043] Preferably, the method for obtaining the stone target area includes:

[0044] Acquiring the image data of the ureteral cavity and kidney stones;

[0045] The outline of the kidney stone in the image data is identified by using a neural network algorithm, and the image data is binarized by using an adaptive threshold method to obtain a binary image, and then the outline is marked in the binary image to obtain the stone target area.

[0046] Preferably, the method for performing the feature extraction includes:

[0047] Feature extraction is performed on the binarized target stone area to obtain the actual image features and the actual interaction matrix of the actual image posture:

[0048] s1=[x n1 y n1 a n1 s x1 s y1 α1] T ,

[0049]

[0050] Among them, s1 represents the actual image features, L1 represents the actual interaction matrix, and x n1 represents the actual image feature that controls the translational freedom in the X direction, y n1 represents the actual image feature that controls the Y-direction translational freedom, a n1 represents the actual image feature that controls the Z-direction translational freedom, s x1 Represents the actual characteristics of rotation around the X axis, s y1 represents the actual feature rotated around the Y axis, α1 represents the actual direction angle of the target area, and T represents the transpose of the matrix;

[0051] Based on the preset expected image, the expected image features and the expected interaction matrix of the expected image pose are calculated:

[0052] s0=[x n0 y n0 a n0 s x0 s y0 α0] T ,

[0053]

[0054] Among them, s0 represents the expected image features, L0 represents the expected interaction matrix, and x n0represents the desired image feature that controls the translational freedom in the X direction, y n0 represents the desired image feature that controls the Y-direction translational freedom, a n0 represents the desired image feature that controls the Z-direction translational freedom, s x0 represents the desired characteristics of rotation around the X axis, s y0 represents the desired feature of rotation around the Y axis, and α0 represents the desired direction angle of the target area.

[0055] Preferably, the method for obtaining the optimal solution of the motion posture includes:

[0056] The camera expected speed calculation unit calculates the camera expected speed based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix:

[0057]

[0058] e=s1-s0,

[0059]

[0060] Among them, v d represents the expected speed of the camera, λ represents a constant, e represents the image feature error, and L s represents the interaction matrix, represents the inverse matrix of the interaction matrix;

[0061] The barrier function setting unit is used to set the barrier function:

[0062]

[0063] Where B represents the barrier function, represents the image feature difference that controls the translational freedom in the X direction, represents the image feature difference controlling the Y-direction translational freedom, and ρ represents the visibility constraint radius;

[0064] The robot posture calculation unit is used to calculate the optimal solution of the robot's motion posture based on the optimal control algorithm and in combination with the obstacle function:

[0065] B (1) ≥-k b B,

[0066] min M=||v d -v c || 2 ,

[0067] stv c =J robot Δq,

[0068]

[0069] Among them, B (1) represents the first-order derivative of the barrier function, k b represents the proportionality constant, J robot represents the Jacobian matrix of the robot, v c represents the actual speed of the camera, Δq represents the optimal solution for the motion posture, represents the rotation speed of the robot, represents the bending speed of the robot, Indicates the feed speed of the robot.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] The technical solution of the present invention innovatively proposes to compare the expected image with the actual image by calculating the moment characteristics and interaction matrix of the kidney stone image, and use a nonlinear solver to solve the optimized joint speed, so as to realize the automatic tracking function of the ureteroscopic robot for kidney stones. And when solving the optimal joint speed, a visibility constraint based on the control obstacle function is set to ensure that the kidney stone target always remains within the endoscope field of view to prevent the loss of the target. The present invention can realize the control of the ureteroscopic surgical robot when the image is incomplete, reduce the risk of losing the tracking target, and improve the safety and efficiency of the ureteroscopic surgical robot when in use. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0073] Figure 1 A schematic diagram of the system structure of an embodiment of the present invention;

[0074] Figure 2 A schematic diagram of a method flow of an embodiment of the present invention;

[0075] Figure 3 The figure is a flowchart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0078] Embodiment 1

[0079] In this embodiment, if Figure 1 As shown, the kidney stone automatic tracking system of the ureteroscopic surgical robot based on optimal visual servoing includes: a target recognition module, a feature extraction module, a posture solution module and a posture adjustment module.

[0080] The target recognition module is used to obtain image data of the ureteral cavity and kidney stones, and process the image data to obtain the stone target area.

[0081] The target recognition module includes: an image acquisition unit and an image processing unit. The image acquisition unit is used to obtain image data of the ureteral cavity and kidney stones; the image processing unit uses a neural network algorithm to identify the contour of kidney stones in the image data, and uses an adaptive threshold method to binarize the image data to obtain a binary image, and then marks the contour in the binary image to obtain the stone target area.

[0082] The feature extraction module is used to extract features of the target area of ​​the stone to obtain the actual image features and actual interaction matrix of the actual image posture, and is also used to calculate the expected image features and expected interaction matrix of the expected image posture.

[0083] The feature extraction module includes: an actual image extraction unit and an expected image extraction unit. The actual image extraction unit is used to extract features from the binarized target area of ​​the stone to obtain the actual image features and actual interaction matrix of the actual image posture:

[0084] s1=[x n1 y n1 a n1 s x1 s y1 α1] T ,

[0085]

[0086] Among them, s1 represents the actual image features, L1 represents the actual interaction matrix, and xn1 represents the actual image feature that controls the translational freedom in the X direction, y n1 represents the actual image feature that controls the Y-direction translational freedom, a n1 represents the actual image feature that controls the Z-direction translational freedom, s x1 represents the actual characteristics of rotation around the X axis, s y1 represents the actual feature rotated around the Y axis, α1 represents the actual direction angle of the target area, and T represents the transpose of the matrix; the expected image extraction unit calculates the expected image features and the expected interaction matrix of the expected image posture based on the preset expected image:

[0087] s0=[x n0 y n0 a n0 s x0 s y0 α0] T ,

[0088]

[0089] Among them, s0 represents the expected image features, L0 represents the expected interaction matrix, and x n0 represents the desired image feature that controls the translational freedom in the X direction, y n0 represents the desired image feature that controls the Y-direction translational freedom, a n0 represents the desired image feature that controls the Z-direction translational freedom, s x0 represents the desired characteristics of rotation around the X axis, s y0 represents the desired feature of rotation around the Y axis, and α0 represents the desired direction angle of the target area.

[0090] In this embodiment, the specific process of calculating the actual image features and the expected image features is as follows: First, the actual image features and the expected image features are both represented as features s:

[0091] s=[x n y n a n s x s y α] T ;

[0092] Among them, x n Represents the image features that control the translational freedom in the X direction, y n represents the image feature that controls the Y-direction translational freedom, a n represents the image feature that controls the Z-direction translational freedom, s x Indicates the characteristics of rotation around the X axis, s yrepresents the feature of rotation around the Y axis, and α represents the direction angle of the target area; then, the moment center (x g ,y g ):

[0093] x g =m 10 / m 00 ,

[0094] y g =m 01 / m 00 ,

[0095] Among them, m 00 is the area of ​​the target area, and the image moment m ij With the central moment μ ij They are defined as follows:

[0096] m ij =∫∫ o x i y j f(x,y)dxdy,

[0097] μ ij =∫∫ o (xx g ) i (yy g ) j f(x,y)dxdy,

[0098]

[0099] Among them, i and j represent natural numbers; f() represents the grayscale of the two-dimensional image, x represents the horizontal coordinate of the pixel point in the two-dimensional image, and y represents the vertical coordinate of the pixel point in the two-dimensional image; in order to reduce the coupling of motion, the features are normalized, and the processing method is as follows:

[0100]

[0101] x n =a n x g ,

[0102] y n =a n y g ,

[0103] Wherein, z* represents the desired depth, a represents the image area of ​​the actual image area, a* represents the image area of ​​the desired image area, and the area calculation formula of the image area of ​​the actual image area is:

[0104] a=m 00 =∫∫ of(x,y)dxdy,

[0105] The calculation formula of the direction angle α of the target object is:

[0106]

[0107] The calculation formulas for the features rotating around the X axis and the features rotating around the Y axis are:

[0108] s x =(c2c3+s2 s3) / K,

[0109] s y =(s2c3-c2s3) / K,

[0110] The formula contains:

[0111]

[0112] In this embodiment, the specific process of calculating the actual interaction matrix and the expected interaction matrix is ​​as follows: First, the actual interaction matrix and the expected interaction matrix are both expressed as a matrix L:

[0113]

[0114] The formula contains:

[0115]

[0116]

[0117]

[0118]

[0119]

[0120]

[0121]

[0122]

[0123]

[0124]

[0125]

[0126]

[0127]

[0128]

[0129]

[0130]

[0131]

[0132]

[0133] n ij =μ ij / m 00 ,

[0134] The interaction matrix in the above formula is and for:

[0135]

[0136]

[0137] The formula contains:

[0138] m vx =-i(Am ij +Bm i-1，j+1 +Cm i-1，j )-Am ij ,

[0139] m vy =-j(Am i+1，j +Bm ij +Cm i，j-1 )-Bm ij ,

[0140] m vz =(i+j+3)(Am i+1，j +Bm i，j+1 +Cm ij )-Cm ij ,

[0141] m wx =(i+j+3)m i，j+1 +jm i，j-1 ,

[0142] m wy =-(i+j+3)m i+1，j -im i-1，j ,

[0143] m wz =im i-1，j+1 -jm i+1，j-1 ,

[0144] μ vx =-(i+1)Aμ ij -iBμ i-1，j+1 ,

[0145] μ vy =-jAμ i+1，j-1 -(j+1)Bμ ij ,

[0146]

[0147]

[0148] μ wz =iμ i-1，j+1 -Jμ i+1，j-1 ,

[0149] Among them, A, B and C are all constants.

[0150] The posture solving module is used to calculate the optimal solution of the motion posture of each joint of the robot based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix, using the optimal control algorithm and the control obstacle function.

[0151] The posture solving module includes: a camera expected speed calculation unit, an obstacle function setting unit and a robot posture calculation unit. The camera expected speed calculation unit calculates the camera expected speed based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix:

[0152] v d =[v dx v dy v dz ω dx ω dy ω dz ] T ,

[0153]

[0154] e=S1-S0,

[0155]

[0156] Among them, v d represents the expected speed of the camera, λ represents a constant, e represents the image feature error, and L s represents the interaction matrix, Represents the inverse matrix of the interaction matrix; the barrier function setting unit is used to set the barrier function:

[0157]

[0158]

[0159]

[0160] Where B represents the barrier function, represents the image feature difference that controls the translational freedom in the X direction, represents the image feature difference controlling the Y-direction translational freedom, and ρ represents the visibility constraint radius; the robot posture calculation unit is used to calculate the optimal solution of the robot's motion posture based on the optimal control algorithm and the obstacle function:

[0161] B (1) ≥-k b B,

[0162] min M=||v d -v c || 2 ,

[0163] stv c =J robot Δq,

[0164]

[0165]

[0166]

[0167]

[0168] Among them, B (1) represents the first-order derivative of the barrier function, k b represents the proportionality constant, j robot represents the Jacobian matrix of the robot, v c represents the actual speed of the camera, Δq represents the optimal solution for the motion posture, represents the rotation speed of the robot, represents the bending speed of the robot, represents the feeding speed of the robot, l represents the length of the flexible segment, represents the angle of rotation and θ represents the angle of bending.

[0169] The posture adjustment module is used to control the posture of the robot based on the optimal solution of the motion posture.

[0170] In this embodiment, the calculated rotation speed, bending speed and feeding speed of the ureteral robot are The movement speed control amount required by the motor is converted and sent to the ureteroscope surgical robot for motor driving to adjust the posture of the ureteroscope terminal.

[0171] Embodiment 2

[0172] In this embodiment, if Figure 2 , Figure 3 As shown, the automatic tracking method of kidney stones by a ureteroscopic surgical robot based on optimal visual servoing includes the following steps:

[0173] S1. Obtain image data of the ureteral cavity and kidney stones, and process the image data to obtain a target area of ​​stones.

[0174] The method for obtaining a target area for stones includes: obtaining image data of the ureteral cavity and kidney stones; using a neural network algorithm to identify the contour of kidney stones in the image data, and using an adaptive threshold method to binarize the image data to obtain a binary image, and then marking the contour in the binary image to obtain a target area for stones.

[0175] S2. Feature extraction is performed on the target area of ​​the stone to obtain actual image features and actual interaction matrix of the actual image posture, which are also used to calculate expected image features and expected interaction matrix of the expected image posture.

[0176] The method for performing feature extraction includes: performing feature extraction on the binarized target area of ​​the stone to obtain actual image features and an actual interaction matrix of the actual image position:

[0177] s1=[x n1 y n1 a n1 s x1 s y1 α1] T ,

[0178]

[0179] Among them, s1 represents the actual image features, L1 represents the actual interaction matrix, and x n1 represents the actual image feature that controls the translational freedom in the X direction, y n1 represents the actual image feature that controls the Y-direction translational freedom, a n1 represents the actual image feature that controls the Z-direction translational freedom, s x1 represents the actual characteristics of rotation around the X axis, s y1 represents the actual feature rotated around the Y axis, α1 represents the actual direction angle of the target area, and T represents the transpose of the matrix; based on the preset expected image, the expected image features and expected interaction matrix of the expected image pose are calculated:

[0180] s0=[x n0 y n0 a n0s x0 s y0 α0] T ,

[0181]

[0182] Among them, s0 represents the expected image features, L0 represents the expected interaction matrix, and x n0 represents the desired image feature that controls the translational freedom in the X direction, y n0 represents the desired image feature that controls the Y-direction translational freedom, a n0 represents the desired image feature that controls the Z-direction translational freedom, s x0 represents the desired characteristics of rotation around the X axis, s y0 represents the desired feature of rotation around the Y axis, and α0 represents the desired direction angle of the target area.

[0183] S3. Based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix, the optimal control algorithm and the control obstacle function are used to calculate the optimal solution for the motion posture of each joint of the robot.

[0184] The method for obtaining the optimal solution of the motion posture includes: the camera expected speed calculation unit calculates the camera expected speed based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix:

[0185]

[0186] e=S1-s0,

[0187]

[0188] Among them, v d represents the expected speed of the camera, λ represents a constant, e represents the image feature error, and L s represents the interaction matrix, Represents the inverse matrix of the interaction matrix; the barrier function setting unit is used to set the barrier function:

[0189]

[0190] Where B represents the barrier function, represents the image feature difference that controls the translational freedom in the X direction, represents the image feature difference controlling the Y-direction translational freedom, and ρ represents the visibility constraint radius; the robot posture calculation unit is used to calculate the optimal solution of the robot's motion posture based on the optimal control algorithm and the obstacle function:

[0191] B (1) ≥-k b B,

[0192] min M=||v d -v c || 2 ,

[0193] stv c =J robot Δq,

[0194]

[0195] Among them, B (1) represents the first-order derivative of the barrier function, k b represents the proportionality constant, J robot represents the Jacobian matrix of the robot, v c represents the actual speed of the camera, Δq represents the optimal solution for the motion posture, represents the rotation speed of the robot, represents the bending speed of the robot, Indicates the feed speed of the robot.

[0196] S4. Control the robot's posture based on the optimal solution of motion posture.

[0197] The calculated rotation speed, bending speed and feeding speed of the ureteral robot are The required motion speed control amount of the motor is converted and sent to the ureteroscope surgical robot for motor drive to adjust the posture of the ureteroscope terminal.

[0198] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. An automatic kidney stone tracking system for a ureteroscopic surgical robot based on optimal visual servoing, characterized in that: include: Target recognition module, feature extraction module, attitude solution module and attitude adjustment module; The target recognition module is used to obtain image data of the ureteral cavity and kidney stones, and process the image data to obtain a target area of ​​stones; The feature extraction module is used to extract features from the target area of ​​the stone to obtain actual image features and actual interaction matrix of actual image posture, and is also used to calculate expected image features and expected interaction matrix of expected image posture; The posture solving module is used to calculate the optimal solution of the motion posture of each joint of the robot by using the optimal control algorithm and the control obstacle function based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix; The posture adjustment module is used to control the posture of the robot based on the optimal solution of the motion posture.

2. The kidney stone automatic tracking system of the ureteroscopic surgical robot based on optimal visual servoing according to claim 1 is characterized in that: The target recognition module includes: an image acquisition unit and an image processing unit; The image acquisition unit is used to obtain the image data of the ureteral cavity and kidney stones; The image processing unit uses a neural network algorithm to identify the contour of the kidney stone in the image data, and uses an adaptive threshold method to binarize the image data to obtain a binary image, and then marks the contour in the binary image to obtain the stone target area.

3. The kidney stone automatic tracking system of the ureteroscopic surgical robot based on optimal visual servoing according to claim 1 is characterized in that: The feature extraction module includes: an actual image extraction unit and an expected image extraction unit; The actual image extraction unit is used to extract features from the binarized target stone area to obtain the actual image features of the actual image posture and the actual interaction matrix: s1=[x n1 y n1 a n1 s x1 s y1 α1] T , Among them, s1 represents the actual image features, L1 represents the actual interaction matrix, and x n1 represents the actual image feature that controls the translational freedom in the X direction, y n1 represents the actual image feature that controls the Y-direction translational freedom, a n1 represents the actual image feature that controls the Z-direction translational freedom, s x1 represents the actual characteristics of rotation around the X axis, s y1 represents the actual feature rotated around the Y axis, α1 represents the actual direction angle of the target area, and T represents the transpose of the matrix; The expected image extraction unit calculates the expected image features and the expected interaction matrix of the expected image pose based on the preset expected image: s0=[x n0 y n0 a n0 s x0 s y0 α0] T , Among them, s0 represents the expected image features, L0 represents the expected interaction matrix, and x n0 represents the desired image feature that controls the translational freedom in the X direction, y n0 represents the desired image feature that controls the Y-direction translational freedom, a n0 represents the desired image feature that controls the Z-direction translational freedom, s x0 represents the desired characteristics of rotation around the X axis, s y0 represents the desired feature of rotation around the Y axis, and α0 represents the desired direction angle of the target area.

4. The kidney stone automatic tracking system of the ureteroscopic surgical robot based on optimal visual servoing according to claim 3 is characterized in that: The posture solving module includes: a camera expected speed calculation unit, an obstacle function setting unit and a robot posture calculation unit; The camera expected speed calculation unit calculates the camera expected speed based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix: e=S1-s0, Among them, v d represents the expected speed of the camera, λ represents a constant, e represents the image feature error, and L s represents the interaction matrix, represents the inverse matrix of the interaction matrix; The barrier function setting unit is used to set the barrier function: Where B represents the barrier function, represents the image feature difference that controls the translational freedom in the X direction, represents the image feature difference controlling the Y-direction translational freedom, and ρ represents the visibility constraint radius; The robot posture calculation unit is used to calculate the optimal solution of the robot's motion posture based on the optimal control algorithm and in combination with the obstacle function: B (1) ≥-k b B, min M=||v d -v c || 2 , s.t.v c =J robot Δq, Among them, B (1) represents the first-order derivative of the barrier function, k b represents the proportionality constant, J robot represents the Jacobian matrix of the robot, v c represents the actual speed of the camera, Δq represents the optimal solution for the motion posture, represents the rotation speed of the robot, represents the bending speed of the robot, Indicates the feed speed of the robot.

5. A method for automatically tracking kidney stones by a ureteroscopic surgical robot based on optimal visual servoing, the method being applied to the system according to any one of claims 1 to 4, characterized in that: The following steps are involved: Acquire image data of the ureteral cavity and kidney stones, and process the image data to obtain a target area of ​​stones; Extracting features of the target area of ​​the stone to obtain actual image features and actual interaction matrix of the actual image position, which are also used to calculate expected image features and expected interaction matrix of the expected image position; Based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix, an optimal solution for the motion posture of each joint of the robot is calculated using an optimal control algorithm and a control obstacle function; The posture of the robot is controlled based on the optimal solution of the motion posture.

6. The method for automatically tracking kidney stones by a ureteroscopic surgical robot based on optimal visual servoing according to claim 5, characterized in that: The method for obtaining the stone target area includes: Acquiring the image data of the ureteral cavity and kidney stones; The outline of the kidney stone in the image data is identified by using a neural network algorithm, and the image data is binarized by using an adaptive threshold method to obtain a binary image, and then the outline is marked in the binary image to obtain the stone target area.

7. The method for automatically tracking kidney stones by a ureteroscopic surgical robot based on optimal visual servoing according to claim 5, characterized in that: The method for performing the feature extraction comprises: Feature extraction is performed on the binarized target stone area to obtain the actual image features and the actual interaction matrix of the actual image posture: s1=[x n1 y n1 a n1 s x1 s y1 α1] T , Among them, s1 represents the actual image features, L1 represents the actual interaction matrix, and x n1 represents the actual image feature that controls the translational freedom in the X direction, y n1 represents the actual image feature that controls the Y-direction translational freedom, a n1 represents the actual image feature that controls the Z-direction translational freedom, s x1 represents the actual characteristics of rotation around the X axis, s y1 represents the actual feature rotated around the Y axis, α1 represents the actual direction angle of the target area, and T represents the transpose of the matrix; Based on the preset expected image, the expected image features and the expected interaction matrix of the expected image pose are calculated: s0=[x n0 y n0 a n0 s x0 s y0 α0] T , Among them, s0 represents the expected image features, L0 represents the expected interaction matrix, and x n0 represents the desired image feature that controls the translational freedom in the X direction, y n0 represents the desired image feature that controls the Y-direction translational freedom, a n0 represents the desired image feature that controls the Z-direction translational freedom, s x0 represents the desired characteristics of rotation around the X axis, s y0 represents the desired feature of rotation around the Y axis, and α0 represents the desired direction angle of the target area.

8. The method for automatically tracking kidney stones by a ureteroscopic surgical robot based on optimal visual servoing according to claim 7, characterized in that: The method for obtaining the optimal solution of the motion posture includes: The camera expected speed calculation unit calculates the camera expected speed based on the actual image features, the actual interaction matrix, the expected image features and the expected interaction matrix: e=S1-s0, Among them, v d represents the expected speed of the camera, λ represents a constant, e represents the image feature error, and L s represents the interaction matrix, represents the inverse matrix of the interaction matrix; The barrier function setting unit is used to set the barrier function: Where B represents the barrier function, represents the image feature difference that controls the translational freedom in the X direction, represents the image feature difference controlling the Y-direction translational freedom, and ρ represents the visibility constraint radius; The robot posture calculation unit is used to calculate the optimal solution of the robot's motion posture based on the optimal control algorithm and in combination with the obstacle function: B (1) ≥-k b B, min M=||v d -v c || 2 , s.t.v c =J robot Δq, Among them, B (1) represents the first-order derivative of the barrier function, k b represents the proportionality constant, J robot represents the Jacobian matrix of the robot, v c represents the actual speed of the camera, Δq represents the optimal solution for the motion posture, represents the rotation speed of the robot, represents the bending speed of the robot, Indicates the feed speed of the robot.

Images