Optimal visual control method and system of flexible endoscope robot based on kinematic model online estimation

Abstract

The invention discloses an optimal visual control method and system of a flexible endoscope robot for online estimation of a kinematic model, and belongs to the field of medical robotics.The method comprises the following steps that a cavity image is collected based on the endoscope robot, and the cavity image is processed to obtain cavity center coordinates and visual features; calculating a Jacobian matrix of the endoscope robot in each posture based on the center coordinate of the cavity and the joint speed of the endoscope robot; image center coordinates are obtained, image feature errors are obtained based on the cavity center coordinates and the image center coordinates, and the optimal solution of the movement speed of each joint of the endoscope robot is calculated based on the image feature errors and a Jacobian matrix; and inputting the optimal solution of the movement speed of each joint of the endoscope robot into a driving motor of each joint of the endoscope robot to realize optimal visual control. According to the invention, high-precision and stable control of the flexible endoscope robot is realized, and the quality and the efficiency of robot endoscopic surgery are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical robots, and in particular relates to an optimal visual control method and system for a flexible endoscope robot with online kinematic model estimation. Background Art

[0002] Endoscopic surgery involves using an endoscope (a thin, flexible tube with a lens and light on the end) to view and perform surgery through small incisions or natural openings. This procedure has many advantages over open surgery, including fewer incisions, less blood loss, faster recovery time, fewer side effects, and a reduced likelihood of problems.

[0003] An endoscopic robot is a robot with flexible deformation capabilities. Due to its high flexibility and strong adaptability, it can move in a limited space and adapt to various complex constrained environments. In the medical field, the emergence and application of this robot has improved the accuracy and efficiency of traditional surgery. The endoscope can directly reduce bleeding and tissue damage during surgery, and can also shorten the patient's recovery time.

[0004] There are some challenges when it comes to modeling and controlling endoscopic robots. On the one hand, the flexible and deformable nature of endoscopes is difficult to accurately model; on the other hand, they are subject to multiple constraints during movement. First, the nonlinearity and high degree of freedom of the endoscope make its kinematic modeling more difficult and challenging. Second, the disturbance of the external environment causes the actual position of the endoscopic robot to deviate from the theoretical position, thereby generating outliers, which can lead to singularities in the kinematic modeling and control instability. Third, in the control process of the endoscopic robot, it is necessary to consider the physical constraints of the robot system and the control constraints required by the task to ensure the safety of the operation. Therefore, in order to solve these problems, it is necessary to provide an innovative control method for the endoscopic robot to overcome the limitations of traditional methods and achieve efficient, stable and precise control of the endoscopic robot in a constrained environment to reduce the risks and complications during surgery. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes an optimal visual control method and system for a flexible endoscopic robot with online estimation of a kinematic model to solve the problems existing in the above prior art.

[0006] To achieve the above object, the present invention provides an optimal visual control method for a flexible endoscope robot with online estimation of a kinematic model, comprising the following steps:

[0007] Acquiring cavity images based on an endoscope robot, and processing the cavity images to obtain cavity center coordinates and visual features;

[0008] Calculating the Jacobian matrix of the endoscope robot at each posture based on the center coordinates of the cavity and the joint velocities of the endoscope robot;

[0009] Acquire the image center coordinates, obtain the image feature error based on the cavity center coordinates and the image center coordinates, and calculate the optimal solution of the movement speed of each joint of the endoscope robot based on the image feature error and the Jacobian matrix;

[0010] The optimal solution of the movement speed of each joint of the endoscopic robot is input into the driving motor of each joint of the endoscopic robot to achieve optimal visual control.

[0011] Optionally, the process of processing the lumen image to obtain the lumen center coordinates includes:

[0012] Adopting an adaptive threshold method to binarize the cavity image to obtain a binary image;

[0013] Segmenting the cavity contour in the binary image, and extracting the maximum inscribed circle of the cavity contour to obtain the cavity center;

[0014] The cavity center coordinates are determined based on the cavity center.

[0015] Optionally, the process of calculating the optimal solution for the movement speed of each joint of the endoscope robot includes:

[0016] Obtaining a motion speed of a desired visual feature based on the image feature error;

[0017] Constructing a quadratic programming model based on the Jacobian matrix and the movement speed of the desired visual feature;

[0018] The quadratic programming model is solved to obtain the optimal control quantity of each joint of the endoscope robot.

[0019] Optionally, the cost function of the quadratic programming model includes: an error between an expected speed and an actual speed of the visual feature and a regularization term for completing a target task with a minimum control amount.

[0020] Optionally, the expression of the quadratic programming model is:

[0021]

[0022] subject to q min ≤q≤q max

[0023]

[0024] In the formula, β is a positive constant, is the joint velocity, represents the motion model of the endoscope robot, J represents the Jacobian matrix of the endoscope robot, |e| represents the absolute distance between the target point and the center of the image, k1 and k2 represent the vertical axis and horizontal axis of the scaling function, r0 represents the influence radius of the pixel coordinates of the target point, represents the constraint function, q min ,q max denote the lower and upper limits of q respectively, Respectively indicate speed lower and upper limits.

[0025] Optionally, the optimal visual control method further comprises calculating the environmental disturbance amount based on the optimal solution of the movement speed of each joint of the endoscope robot and the visual feature;

[0026] The process of calculating the environmental disturbance includes:

[0027] Establish the state space equations and observers of the endoscope robot;

[0028] Establishing an observer equation based on the observation data of the observer and the state-space equation;

[0029] An environmental disturbance amount is determined based on the observer equation.

[0030] Optionally, the expression of the environmental disturbance amount is:

[0031] ω=z2=∫-β2e1dt

[0032] Where ω represents the environmental disturbance, β2 represents the observed gain coefficient, and e1 represents the observed output error.

[0033] The present invention also provides an optimal visual control system for a flexible endoscope robot with online kinematic model estimation, comprising: an image processing module, a Jacobian matrix robust estimation module, an optimal visual control module, a disturbance observation module and an endoscope driving module;

[0034] The image processing module is used to collect the cavity image of the human body and calculate the center coordinates of the cavity based on the cavity image;

[0035] The Jacobian matrix robust estimation module is used to calculate the Jacobian matrix of the endoscope robot in various postures based on the cavity center coordinates and the joint speeds of the endoscope robot;

[0036] The optimal visual control module is used to calculate the optimal solution of the movement speed of each joint of the endoscope robot based on the Jacobian matrix and the image feature error;

[0037] The disturbance observation module is used to calculate the environmental disturbance amount;

[0038] The endoscope driving module is used to drive the joint driving motors of the endoscope robot based on the optimal solution of the movement speed of each joint of the endoscope robot to achieve posture control of the end camera of the endoscope robot.

[0039] The present invention also provides a computer terminal device, comprising:

[0040] one or more processors;

[0041] A memory, coupled to the processor, for storing one or more programs;

[0042] When the one or more programs are executed by the one or more processors, the one or more processors implement an optimal visual control method for a flexible endoscopic robot with online estimation of a kinematic model.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, an optimal visual control method for a flexible endoscope robot with online estimation of a kinematic model is implemented.

[0044] Compared with the prior art, the present invention has the following advantages and technical effects:

[0045] This invention innovatively proposes a robust estimation method for the Jacobian matrix of an endoscopic robot. Considering the influence of sensor noise and environmental disturbance on the system, the Jacobian matrix of the endoscopic robot is approximated online based on the improved iterative reweighted least squares algorithm. In addition, the physical and control constraints of the robot are considered in the control process, which not only improves the stability and robustness of the control, but also ensures the safety during the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0047] Figure 1 A schematic diagram of the system structure of an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of the system operation flow of an embodiment of the present invention;

[0049] Figure 3 A driving-visual mapping diagram according to an embodiment of the present invention;

[0050] Figure 4 Schematic diagram of the motion control of an endoscope within a cavity according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0052] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0053] Embodiment 1

[0054] The present invention proposes a more stable and reliable autonomous operation method for an endoscopic robot, which aims to solve the problem that the kinematic model of the endoscope becomes singular due to external disturbances, thereby improving the robustness of the entire control system, integrating the robot's physical and control constraints into a quadratic programming framework, and improving the control stability and safety of the endoscopic robot, making it easier to implement in endoscopic surgery and enabling the endoscopic robot to automatically adjust the posture of the endoscope.

[0055] like Figure 2 As shown, this embodiment provides an optimal visual control method for a flexible endoscopic robot with online estimation of a kinematic model, comprising the following steps: collecting cavity images based on the endoscopic robot, processing the cavity images to obtain cavity center coordinates and visual features; calculating the Jacobian matrix of the endoscopic robot in various postures based on the cavity center coordinates and the joint speed of the endoscopic robot; acquiring the image center coordinates, obtaining the image feature error based on the cavity center coordinates and the image center coordinates, and calculating the optimal solution for the movement speed of each joint of the endoscopic robot based on the image feature error and the Jacobian matrix; inputting the optimal solution for the movement speed of each joint of the endoscopic robot into the drive motors of each joint of the endoscopic robot to achieve optimal visual control.

[0056] S1: Acquire image information of a human body cavity, and process the image information to obtain the center coordinates of the cavity;

[0057] In this embodiment, S1 specifically includes: first, the endoscope collects image information in the human airway in real time, extracts cavity information in the image by deep learning, and uses an adaptive threshold method to binarize the original image to obtain a binary image. (where (x,y) are the image coordinates of the image plane).

[0058] The cavity contour in the binary image is segmented, and the maximum inscribed circle of the cavity contour is extracted to obtain the cavity center; and the cavity center coordinates are determined based on the cavity center.

[0059] S2: Calculate the Jacobian matrix of the endoscope robot at each posture during the control process based on the processed cavity center and joint speed;

[0060] In this embodiment, S2 specifically includes: obtaining the center position of the cavity image at time t-1 [u t-1 ,v t-1 ], obtain the center position of the cavity image at time t [u t ,v t ], the Jacobian matrix J of the endoscope robot is obtained based on the center position of the cavity image at time t-1 and t and the expected speed of each joint.

[0061] Prepare a container to store the motion speed Δs, joint speed Δq and observed disturbance ω of the visual feature at each moment. Take the last k groups of data in the container (excluding outliers) for calculation, and define the estimation of the Jacobian matrix J of the endoscope robot as an optimization problem. The cost function includes the estimated output error Δs-JΔq-ω of the visual feature and the regularization term of the Jacobian matrix change ||ΔJ||2, and calculate the optimal Jacobian matrix J of the endoscope robot.

[0062] S3: Based on the physical and control constraints of the endoscope robot, the optimal solution of the joint speed of the endoscope robot is obtained through the optimization algorithm with the goal of minimizing the visual tracking error;

[0063] In this embodiment, S3 specifically includes: comparing the cavity center coordinates under the current endoscopic robot posture with the image center coordinates to calculate the image feature error. Building a quadratic programming framework, constructing an objective function based on the image feature error and the Jacobian matrix of the endoscopic robot, and combining the physical and control constraints of the endoscopic robot to calculate the optimal solution for the movement speed of each joint of the robot.

[0064] Using the current visual features s t With the image center s d The error is used to obtain the movement speed of the desired visual feature. Where t is the movement time of the robot in each iteration. Combined with the Jacobian matrix J of the endoscope robot obtained above, an optimal problem based on the quadratic programming framework is constructed. The cost function contains the error between the expected speed and the actual speed of the visual feature. And the regularization term that ensures the minimum control amount to complete the required task Calculate the optimal control quantity of each joint of the endoscope robot, including the rotation speed of the robot Bending speed and feed speed

[0065] S4: Based on the optimal solution of joint velocity and the change of visual features, estimate the disturbance caused by the environment to the system;

[0066] In this embodiment, S4: according to the Jacobian matrix J of the endoscope robot solved in S2 and the optimal control amount of each joint solved in S3 combined with the visual feature s, the disturbance amount ω generated by the environment to the system is estimated; specifically including: establishing the state space equation of the endoscope robot: in is the expected motion speed of the visual feature, is the joint velocity, x2 is the augmented state variable, which represents the disturbance of the model due to the interference of the environment.

[0067] Establish an observer, let the observed visual feature be z1, the observed environmental disturbance be z2, and the output error of the observed visual feature be e=z1-x1;

[0068] Setting up the observer equation and Among them, β1, β2 represent the observed gain coefficients, β1 = 2w0, β2 = w0 2 ;

[0069] The environmental disturbance observed by the observer is ω=z2=∫-β2e1dt.

[0070] S5: The optimal solution of the joint speed of the endoscope robot is input to the driving motors of each joint of the endoscope robot, thereby realizing the posture control of the endoscope robot's terminal camera.

[0071] In this embodiment, S5 specifically includes: linearly converting the optimal solution of the joint movement speed into a motor rotation speed, and sending the speed to the endoscope robot for motor driving to adjust the posture of the endoscope robot terminal camera.

[0072] If the calculated image feature error is never less than the threshold, repeat the above steps until the error is less than the threshold.

[0073] The specific definitions are as follows:

[0074] The interaction matrix of the endoscope robot is defined as:

[0075]

[0076] Where: L is the length of the continuum actuator. The symbols c(·) and s(·) are the abbreviations of cos(·) and sin(·), respectively.

[0077] The motion model of the endoscope robot is defined as:

[0078] in:

[0079] J0=L s ·J r

[0080]

[0081] Among them J r represents the Jacobian matrix of the continuum actuator, L s represents the image Jacobian matrix, and J0 represents the Jacobian matrix of the endoscope robot at the initial moment. and Represents the pixel error between the visual feature coordinates and the image center coordinates. Among them, f and ρ represent the focal length and pixel ratio of the camera respectively.

[0082] The estimation of the Jacobian matrix of the subsequent endoscope robot is defined as an optimization problem, specifically defined as:

[0083]

[0084] subject to J k+1 =J k +ΔJ

[0085] in:

[0086]

[0087] e i =||Δs i -J k+1 Δq i -ω i ||2

[0088]

[0089] Where ΔJ represents the optimization variable to be solved. k and J k+1 Respectively represent the Jacobian matrix at the kth and k+1th moments. i (e) represents the weight of each data pair in the container, and the GM estimator is used for weight assignment. γ<1 is a forgetting factor with a positive constant, and σ represents an extended metric factor used to quantify the probability distribution, which is calculated using the median absolute deviation function. B is a constant that is selected so that the MAD is consistent with the normal distribution using the cumulative normal distribution function Φ(·): y i Represents the L2 norm of the visual feature, y i=||Δs i ||2.

[0090] The linear extended state observer is defined as:

[0091]

[0092] Where e1 represents the observed output error, β1 and β2 represent the observed gains. z1 and z2 represent the states of the observer. Where z1 represents the visual feature s, and z2 represents the observed perturbation ω.

[0093] Specifically, the input compensation can be expressed as:

[0094]

[0095] The lifting form is expressed as:

[0096]

[0097] in Represents the error between the estimated value and the true value.

[0098] The observed disturbance ω can be expressed as:

[0099] ω=z2=∫-β2e1dt

[0100] In order to minimize the visual tracking error and satisfy the constraints of the endoscope robot, a QP control framework is constructed, which is defined as:

[0101]

[0102] subject to q min ≤q≤q max

[0103]

[0104] Among them, the objective function in the above formula consists of two terms, and β is a positive constant used to adjust the weights of these two objective functions. The first term quantifies the error between the expected time derivative and the actual time derivative of the visual target in the image space. The goal of the second term is to enable the endoscope robot to complete the required task with the minimum control amount. The constraint equation must comply with the physical constraints of the robot, and Respectively represent q(speed )’s lower and upper limits. The repulsion field function is modified, |e| represents the absolute distance between the target point and the center of the image, and r0 represents the influence radius of the pixel coordinates of the target point. Constraint function Follows a normal distribution. The vertical and horizontal axes of the function are scaled using parameters k1 and k2, respectively.

[0105] When controlling the endoscope robot, the calculated movement speed of each joint of the robot is transmitted to each drive motor, and then the position of the end camera of the endoscope robot is adjusted. The error between the visual feature and the center of the image will become smaller and smaller as the robot is controlled. At the same time, the endoscope can be inserted into the human airway.

[0106] Embodiment 2

[0107] The present invention provides an optimal visual control system for a flexible endoscope robot with online estimation of kinematic models, such as Figure 1 As shown, the system includes: an image processing module, a Jacobian matrix robust estimation module, an optimal visual control module, a disturbance observation module and an endoscope drive module; the image processing module uses a micro camera to collect the cavity image of the human body in real time and calculate the center position of the cavity; the Jacobian matrix robust estimation module is based on an improved iterative reweighted least squares algorithm to accurately and robustly estimate the Jacobian matrix of the endoscope robot. The optimal visual control module fully considers the physical and control constraints of the robot, and takes minimizing the visual tracking error as the goal to obtain the optimal control amount of each joint. The disturbance observation module estimates the amount of disturbance caused by the environment to the system based on the optimal control amount of the joint and the change of the visual features. The endoscope drive module dynamically adjusts the posture of the endoscope robot according to the optimization results, so that the target feature is quickly and accurately close to the center of the image. As a specific implementation method of this embodiment, it includes:

[0108] An image processing module, used for collecting the cavity image of the human body and calculating the pixel coordinates of the cavity center based on the cavity image of the human body;

[0109] Jacobian matrix robust estimation module, used to calculate the Jacobian matrix of the endoscope robot under different postures;

[0110] An optimal visual control module, used to calculate the optimal control amount of each joint of the endoscope robot according to the expected position of the visual feature;

[0111] The disturbance observation module is used to calculate the disturbance caused by the environment to the system and compensate for it;

[0112] The endoscope drive module is used to input the optimal solution of each joint speed obtained by optimization to the drive motor of each joint of the robot, thereby realizing the control of the end camera posture of the endoscope robot.

[0113] An image processing module collects cavity images in real time, segments the cavity contour of the cavity image, and extracts the maximum inscribed circle of the cavity contour, with the center of the maximum inscribed circle as the cavity center;

[0114] The Jacobian matrix robust estimation module is used to calculate the Jacobian matrix of the endoscope robot in different postures during the control process based on the center coordinates of the cavity and the joint speeds of the endoscope robot;

[0115] An optimal visual control module is used to calculate the optimal solution of the movement speed of each joint of the endoscope robot based on the error between the center coordinates of the cavity and the center of the image and the Jacobian matrix of the endoscope robot;

[0116] The disturbance observation module is used to calculate the disturbance amount caused by the environment to the system based on the center coordinates of the cavity and the joint movement speed;

[0117] The endoscope drive module is used to input the optimal solution of the speed of each joint of the endoscope robot to the joint drive motor of the endoscope robot, thereby realizing the posture control of the endoscope robot's terminal camera.

[0118] In this example, the image processing module includes: an image acquisition unit and an image processing unit;

[0119] The image acquisition unit is used for acquiring cavity images;

[0120] The image processing unit is used to segment the cavity contour in the cavity image and extract the maximum inscribed circle of the cavity contour, with the center of the maximum inscribed circle as the cavity center.

[0121] In this example, in the Jacobian matrix robust estimation module, the method for calculating the Jacobian matrix of the endoscope robot in different postures during the control process based on the cavity center coordinates and the joint speeds of the endoscope robot includes:

[0122] Get the center position of the cavity image at time t-1 [u t-1 ,v t-1 ], obtain the center position of the cavity image at time t [u t ,v t ], the Jacobian matrix J of the endoscope robot is obtained based on the center position of the cavity image at time t-1 and t and the expected velocity change of each joint.

[0123] In this example, in the optimal visual control module, based on the physical and control constraints of the endoscope robot and with the goal of minimizing the visual tracking error, the method of obtaining the optimal solution of the motion speed of each joint of the endoscope robot through the optimization algorithm includes:

[0124] The cavity center under the current endoscope robot posture is compared with the image center to calculate the image feature error.

[0125] A quadratic programming framework is constructed to construct the objective function based on the image feature error and the Jacobian matrix of the endoscope robot. The optimal motion speed of each joint of the robot is calculated by combining the physical and control constraints of the endoscope robot.

[0126] In this example, in the disturbance observation module, the method for estimating the disturbance amount caused by the environment to the system based on the optimal control amount of the joint and the change of the visual features includes:

[0127] Establish the state space equation of the endoscope robot: in is the expected motion speed of the visual feature, is the joint velocity, x2 is the augmented state variable, which represents the estimation error of the model due to environmental interference.

[0128] Establish an observer, let the observed visual feature be z1, the observed environmental disturbance be z2, and the output error of the observed visual feature be e=z1-x1;

[0129] Setting up the observer equation and Among them, β1, β2 represent the observed gain coefficients, β1 = 2w0, β2 = w0 2 ;

[0130] The environmental disturbance observed by the observer is ω=z2=∫-β2e1dt.

[0131] In this example, in the endoscope driving module, the optimal solution of each joint speed of the endoscope robot is input to the joint driving motor of the endoscope robot, and the method for realizing the posture control of the endoscope robot terminal camera includes:

[0132] The optimal control amount of the joint is linearly converted into the motor rotation speed, and the speed is sent to the endoscope robot for motor drive to adjust the posture of the endoscope robot's terminal camera.

[0133] Figure 3 The schematic diagram of the drive-vision mapping of the endoscope in the optimal vision control system of the flexible endoscope robot with online kinematic model estimation of the present invention is shown in FIG. Among them, {B} is the base coordinate system of the endoscope robot, {C} is the camera coordinate system of the endoscope robot, {S} is the image coordinate system of the endoscope robot, and P is the position of the target feature in three-dimensional space. There is a linear mapping between the coordinate system {B} and the coordinate system {S} with respect to the Jacobian matrix of the endoscope robot.

[0134] Figure 4This is a schematic diagram of the motion control of the endoscope robot in the cavity in the optimal visual control system of the flexible endoscope robot with online estimation of the kinematic model of the present invention. In actual control, the endoscope drive module can realize the rotational motion of the endoscope. The bending motion θ and feeding motion z ensure that the endoscopic robot can perform navigation tasks in restricted cavities.

[0135] The technical solution of the present invention calculates the pixel coordinates of the center of the cavity by processing the image, and then calculates the Jacobian matrix of the endoscope robot by combining the visual feature motion speed and the joint speed, and then uses the quadratic programming framework to solve the optimal value of the joint speed, and adjusts the state of each joint through the motor to reduce the visual feature error. The present invention can achieve accurate and stable control of the endoscope when the endoscope is subjected to external forces, while improving the safety and efficiency of the endoscope when in use.

[0136] The present invention innovatively proposes a model-free optimal visual motion control method, which does not rely on precise analytical solutions to the kinematic model. It estimates the kinematic model online from sensor data and takes into account a variety of constraints, thereby achieving high-precision and stable control of the flexible endoscopic robot, which contributes to the quality and efficiency of robotic endoscopic surgery.

[0137] Embodiment 3

[0138] This embodiment also provides a computer terminal device, including:

[0139] one or more processors;

[0140] A memory, coupled to the processor, for storing one or more programs;

[0141] When one or more programs are executed by one or more processors, the one or more processors implement an optimal visual control method for a flexible endoscopic robot with online estimation of a kinematic model.

[0142] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, an optimal visual control method for a flexible endoscope robot with online estimation of a kinematic model is implemented.

[0143] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An optimal visual control method for a flexible endoscope robot with online estimation of kinematic model, characterized in that: The following steps are involved: Acquiring cavity images based on an endoscope robot, and processing the cavity images to obtain cavity center coordinates and visual features; Calculating the Jacobian matrix of the endoscope robot at each posture based on the center coordinates of the cavity and the joint velocities of the endoscope robot; Acquire the image center coordinates, obtain the image feature error based on the cavity center coordinates and the image center coordinates, and calculate the optimal solution of the movement speed of each joint of the endoscope robot based on the image feature error and the Jacobian matrix; The optimal solution of the movement speed of each joint of the endoscopic robot is input into the driving motor of each joint of the endoscopic robot to achieve optimal visual control.

2. The optimal visual control method of a flexible endoscope robot based on online estimation of the kinematic model according to claim 1 is characterized in that: The process of processing the lumen image to obtain the lumen center coordinates includes: Adopting an adaptive threshold method to binarize the cavity image to obtain a binary image; Segmenting the cavity contour in the binary image, and extracting the maximum inscribed circle of the cavity contour to obtain the cavity center; The cavity center coordinates are determined based on the cavity center.

3. The optimal visual control method of a flexible endoscope robot based on online estimation of the kinematic model according to claim 1, characterized in that: The process of calculating the optimal solution for the motion speed of each joint of the endoscope robot includes: Obtaining a motion speed of a desired visual feature based on the image feature error; Constructing a quadratic programming model based on the Jacobian matrix and the movement speed of the desired visual feature; The quadratic programming model is solved to obtain the optimal control quantity of each joint of the endoscope robot.

4. The optimal visual control method of a flexible endoscope robot based on online estimation of the kinematic model according to claim 3 is characterized in that: The cost function of the quadratic programming model includes: the error between the expected speed and the actual speed of the visual feature and the regularization term of the minimum control amount to complete the target task.

5. The optimal visual control method of a flexible endoscope robot based on online estimation of the kinematic model according to claim 3 is characterized in that: The expression of the quadratic programming model is: In the formula, β is a positive constant, is the joint velocity, represents the motion model of the endoscope robot, J represents the Jacobian matrix of the endoscope robot, |e| represents the absolute distance between the target point and the center of the image, k1 and k2 represent the vertical axis and horizontal axis of the scaling function, r0 represents the influence radius of the pixel coordinates of the target point, represents the constraint function, q min ,q max denote the lower and upper limits of q respectively, Respectively indicate speed lower and upper limits.

6. The optimal visual control method of a flexible endoscope robot based on online estimation of the kinematic model according to claim 3 is characterized in that: The optimal visual control method further includes calculating the environmental disturbance amount based on the optimal solution of the movement speed of each joint of the endoscope robot and the visual features; The process of calculating the environmental disturbance includes: Establish the state space equations and observers of the endoscope robot; Establishing an observer equation based on the observation data of the observer and the state-space equation; An environmental disturbance amount is determined based on the observer equation.

7. The optimal visual control method of a flexible endoscope robot based on online estimation of kinematic model according to claim 6, characterized in that: The expression of the environmental disturbance is: ω=z2=∫-β2e1dt Where ω represents the environmental disturbance, β2 represents the observed gain coefficient, and e1 represents the observed output error.

8. An optimal visual control system for a flexible endoscope robot with online estimation of kinematic models, characterized in that: include: Image processing module, Jacobian matrix robust estimation module, optimal visual control module, disturbance observation module and endoscope driving module; The image processing module is used to collect the cavity image of the human body and calculate the center coordinates of the cavity based on the cavity image; The Jacobian matrix robust estimation module is used to calculate the Jacobian matrix of the endoscope robot in various postures based on the cavity center coordinates and the joint speeds of the endoscope robot; The optimal visual control module is used to calculate the optimal solution of the movement speed of each joint of the endoscope robot based on the Jacobian matrix and the image feature error; The disturbance observation module is used to calculate the environmental disturbance amount; The endoscope driving module is used to drive the joint driving motors of the endoscope robot based on the optimal solution of the movement speed of each joint of the endoscope robot to achieve posture control of the end camera of the endoscope robot.

9. A computer terminal device, characterized in that: include: one or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the optimal visual control method of the flexible endoscopic robot with online estimation of the kinematic model as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, an optimal visual control method for a flexible endoscopic robot with online estimation of a kinematic model as described in any one of claims 1 to 7 is implemented.

Images