An optimal visual control method and system for a flexible endoscopic robot based on online kinematic model estimation.

Abstract

This invention discloses an optimal visual control method and system for a flexible endoscopic robot based on online kinematic model estimation, belonging to the field of medical robotics. The method includes the following steps: acquiring cavity images based on the endoscopic robot; processing the cavity images to obtain the cavity center coordinates and visual features; calculating the Jacobian matrix of the endoscopic robot in various postures based on the cavity center coordinates and the joint velocities of the endoscopic robot; obtaining the image center coordinates; obtaining the image feature error based on the cavity center coordinates and the image center coordinates; calculating the optimal solution for the motion velocity of each joint of the endoscopic robot based on the image feature error and the Jacobian matrix; and inputting the optimal solution for the motion velocity of each joint of the endoscopic robot into the drive motors of each joint to achieve optimal visual control. This invention achieves high-precision and stable control of a flexible endoscopic robot, contributing to the quality and efficiency of robotic endoscopic surgery.

Description

Technical Field

[0001] This invention belongs to the field of medical robot technology, and in particular relates to an optimal visual control method and system for a flexible endoscopic robot based on online kinematic model estimation. Background Technology

[0002] Endoscopic surgery involves using an endoscope (a flexible, thin tube with a lens and light source at its end) to visualize and perform surgery inside the body through a small incision or natural opening. Compared to open surgery, this procedure has many advantages, including fewer incisions, less blood loss, faster recovery time, fewer side effects, and a reduced likelihood of complications.

[0003] Endoscopic robots are robots with flexible deformation capabilities. Due to their high flexibility and adaptability, they can move in limited spaces and adapt to various complex constraints. In the medical field, the emergence and application of such robots have improved the accuracy and efficiency of traditional surgery. Endoscopy can directly reduce bleeding and tissue damage during surgery and shorten the patient's recovery time.

[0004] Modeling and controlling endoscopic robots presents several challenges. First, the flexibility and deformability of endoscopes make precise modeling difficult; second, their motion is subject to various constraints. First, the nonlinearity and high degrees of freedom of endoscopes make kinematic modeling more challenging. Second, external environmental disturbances cause the actual position of the endoscopic robot to deviate from its theoretical position, generating outliers. These outliers lead to singularities in kinematic modeling, resulting in control instability. Third, the control of endoscopic robots requires consideration of both the physical constraints of the robot system and the control constraints required for the task, ensuring operational safety. Therefore, to address these issues, it is necessary to provide an innovative control method for endoscopic robots to overcome the limitations of traditional methods and achieve efficient, stable, and precise control of endoscopic robots in confined environments, thereby reducing risks and complications during surgery. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes an optimal visual control method and system for a flexible endoscopic robot based on online kinematic model estimation, thereby resolving the issues present in the prior art.

[0006] To achieve the above objectives, this invention provides an optimal visual control method for a flexible endoscopic robot based on online kinematic model estimation, comprising the following steps:

[0007] Based on the acquisition of cavity images by an endoscopic robot, the cavity images are processed to obtain the cavity center coordinates and visual features;

[0008] The Jacobian matrix of the endoscopic robot in each posture is calculated based on the coordinates of the cavity center and the joint velocity of the endoscopic robot.

[0009] Obtain the coordinates of the image center, obtain the image feature error based on the cavity center coordinates and the image center coordinates, and calculate the optimal solution for the motion speed of each joint of the endoscopic robot based on the image feature error and the Jacobian matrix;

[0010] The optimal solution of the movement speed of each joint of the endoscope robot is input into the drive motor of each joint of the endoscope robot to achieve optimal visual control.

[0011] Optionally, the process of processing the cavity image to obtain the cavity center coordinates includes:

[0012] The cavity image is binarized using an adaptive thresholding method to obtain a binarized image;

[0013] The cavity contour in the binarized image is segmented, and the maximum inscribed circle of the cavity contour is extracted to obtain the cavity center;

[0014] The coordinates of the cavity center are determined based on the cavity center.

[0015] Optionally, the process of calculating the optimal solution for the motion velocity of each joint of the endoscopic robot includes:

[0016] The motion speed of the desired visual feature is obtained based on the image feature error;

[0017] A quadratic programming model is constructed based on the Jacobian matrix and the motion speed of the desired visual features;

[0018] Solving the quadratic programming model yields the optimal control values ​​for each joint of the endoscopic robot.

[0019] Optionally, the cost function of the quadratic programming model includes: the error between the expected speed and the actual speed of the visual features and a regularization term for achieving the target task with the minimum control input.

[0020] Optionally, the expression for the quadratic programming model is:

[0021]

[0022] subject to q min ≤q≤q max

[0023]

[0024] In the formula, β is a positive constant. For joint velocity, Let J represent the motion model of the endoscopic robot, J represent the Jacobian matrix of the endoscopic robot, |e| represent the absolute distance between the target point and the image center, k1 and k2 represent the vertical and horizontal axes of the scaling function, respectively, and r0 represent the radius of influence of the pixel coordinates of the target point. Represents the constraint function. q min ,q max Let represent the lower and upper limits of q, respectively. They represent speeds respectively. The lower and upper limits.

[0025] Optionally, the optimal visual control method further includes calculating the environmental disturbance based on the optimal solution of the joint movement speed of the endoscope robot and the visual features;

[0026] The process of calculating environmental disturbances includes:

[0027] Establish the state-space equations and observer for the endoscopic robot;

[0028] The observer equation is established based on the observer's observation data and the state-space equation.

[0029] The environmental disturbance is determined based on the observer equation.

[0030] Optionally, the expression for the environmental disturbance quantity is:

[0031] ω=z2=∫-β2e1dt

[0032] In the formula, ω represents the environmental disturbance, β2 represents the observed gain coefficient, and e1 represents the observed output error.

[0033] The present invention also provides an optimal visual control system for a flexible endoscopic robot with online kinematic model estimation, comprising: an image processing module, a Jacobian matrix robust estimation module, an optimal visual control module, a perturbation observation module, and an endoscope driving module;

[0034] The image processing module is used to acquire images of the cavities of the human body and calculate the coordinates of the cavity center based on the images.

[0035] The robust estimation module for the Jacobian matrix is ​​used to calculate the Jacobian matrix of the endoscope robot in various postures based on the cavity center coordinates and the joint velocities of the endoscope robot.

[0036] The optimal vision control module is used to calculate the optimal solution for the motion speed of each joint of the endoscope robot based on the Jacobian matrix and image feature error;

[0037] The disturbance observation module is used to calculate the amount of environmental disturbance.

[0038] The endoscope drive module is used to drive the joint drive motors of the endoscope robot based on the optimal solution of the joint movement speed of each joint of the endoscope robot to achieve attitude control of the end-effector camera of the endoscope robot.

[0039] The present invention also provides a computer terminal device, comprising:

[0040] One or more processors;

[0041] A memory, coupled to the processor, for storing one or more programs;

[0042] When the one or more programs are executed by the one or more processors, the one or more processors implement an optimal visual control method for a flexible endoscopic robot based on online kinematic model estimation.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an optimal visual control method for a flexible endoscopic robot based on online estimation of a kinematic model.

[0044] Compared with the prior art, the present invention has the following advantages and technical effects:

[0045] This invention innovatively proposes a robust estimation method for the Jacobian matrix of an endoscopic robot. Considering the impact of sensor noise and environmental disturbances on the system, an improved iterative reweighted least squares algorithm is used to approximate the Jacobian matrix of the endoscopic robot online. Furthermore, the physical and control constraints of the robot are taken into account during the control process, which not only improves the stability and robustness of the control but also ensures safety during surgery. Attached Figure Description

[0046] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0047] Figure 1 This is a schematic diagram of the system structure according to an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the system operation flow according to an embodiment of the present invention;

[0049] Figure 3 This is a driver-vision mapping diagram according to an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the motion control of the endoscope in the cavity according to an embodiment of the present invention. Detailed Implementation

[0051] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0052] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0053] Example 1

[0054] This invention proposes a more stable and reliable autonomous operation method for endoscopic robots, aiming to solve the problem of singularity in the kinematic model of endoscopes caused by external disturbances. It improves the robustness of the entire control system, integrates the robot's physical and control constraints into a quadratic programming framework, enhances the control stability and safety of endoscopic robots, makes them easier to implement in endoscopic surgery, and enables endoscopic robots to automatically adjust the posture of endoscopes.

[0055] like Figure 2 As shown, this embodiment provides an optimal visual control method for a flexible endoscopic robot based on online kinematic model estimation, including the following steps: acquiring cavity images based on the endoscopic robot, processing the cavity images to obtain cavity center coordinates and visual features; calculating the Jacobian matrix of the endoscopic robot in various postures based on the cavity center coordinates and the joint velocities of the endoscopic robot; obtaining the image center coordinates, obtaining the image feature error based on the cavity center coordinates and the image center coordinates, calculating the optimal solution for the motion velocity of each joint of the endoscopic robot based on the image feature error and the Jacobian matrix; and inputting the optimal solution for the motion velocity of each joint of the endoscopic robot into the drive motor of each joint of the endoscopic robot to achieve optimal visual control.

[0056] S1: Acquire image information of human body cavities and process the image information to obtain the coordinates of the cavity center;

[0057] In this embodiment, S1 specifically includes: First, the endoscope acquires image information within the human airway in real time; deep learning is used to extract cavity information from the image; and an adaptive thresholding method is used to binarize the original image to obtain a binarized image. For the binarized image, (where (x,y) are the image coordinates of the image plane).

[0058] The cavity contour in the binarized image is segmented, and the maximum inscribed circle of the cavity contour is extracted to obtain the cavity center; the coordinates of the cavity center are determined based on the cavity center.

[0059] S2: Based on the processed cavity center and joint velocity, calculate the Jacobian matrix of the endoscope robot in various postures during the control process;

[0060] In this embodiment, S2 specifically includes: obtaining the center position of the cavity image at time t-1 [u t-1 ,v t-1 ], obtain the center position of the cavity image at time t [u t ,v t Based on the cavity image center position at times t-1 and t and the expected velocity of each joint, the Jacobian matrix J of the endoscopic robot is obtained.

[0061] Prepare a container to store the motion velocity Δs, joint velocity Δq, and observed perturbation ω of the visual features at each moment. Take the last k sets of data (excluding outliers) in the container and perform calculations. Define the estimation of the Jacobian matrix J of the endoscopic robot as an optimization problem. The cost function includes the estimated output error of the visual features Δs-JΔq-ω and the regularization term ||ΔJ||2 of the Jacobian matrix change. Calculate the optimal Jacobian matrix J of the endoscopic robot.

[0062] S3: Based on the physical and control constraints of the endoscopic robot, with the goal of minimizing visual tracking error, the optimal solution for the velocity of each joint of the endoscopic robot is obtained through optimization algorithm;

[0063] In this embodiment, S3 specifically includes: comparing the cavity center coordinates with the image center coordinates under the current posture of the endoscopic robot to calculate the image feature error. A quadratic programming framework is constructed, and an objective function is built based on the image feature error and the Jacobian matrix of the endoscopic robot. Combining the physical and control constraints of the endoscopic robot, the optimal solution for the motion speed of each joint of the robot is calculated.

[0064] Using the visual features s at the current moment t With image center s d The error is used to obtain the motion speed of the desired visual feature. Where t represents the robot's motion time in each iteration. Combining the Jacobian matrix J of the endoscopic robot obtained above, an optimization problem based on a quadratic programming framework is constructed. The cost function includes the error between the expected velocity and the actual velocity based on visual features. And regularization terms that ensure the minimum control quantity is required to complete the task. Calculate the optimal control values ​​for each joint of the endoscopic robot, including the robot's rotational speed. Bending speed and feed rate

[0065] S4: Based on the optimal solution of joint velocity and changes in visual features, estimate the amount of disturbance generated by the environment on the system;

[0066] In this embodiment, S4: Based on the Jacobian matrix J of the endoscope robot solved in S2 and the optimal control quantities of each joint solved in S3, combined with visual features s, the disturbance ω caused by the environment to the system is estimated; specifically, this includes: establishing the state-space equation of the endoscope robot: in The expected motion speed for visual features. x1 represents the joint velocity, and x2 represents the augmented state variable, which is the disturbance caused by environmental interference in the model.

[0067] Establish an observer, let the observed visual feature be z1, the observed environmental disturbance be z2, and the output error of the observed visual feature be e = z1 - x1;

[0068] Establish the observer equation and Where β1 and β2 represent the observation gain coefficients, β1 = 2w0, β2 = w0 2 ;

[0069] The environmental disturbance observed by the observer is ω=z2=∫-β2e1dt.

[0070] S5: Input the optimal solution of the joint speed of the endoscope robot to the drive motor of each joint of the endoscope robot, thereby realizing the attitude control of the end-effector camera of the endoscope robot.

[0071] In this embodiment, S5 specifically includes: linearly converting the optimal solution of the joint movement speed into the motor rotation speed, and sending the speed to the endoscope robot for motor drive to adjust the posture of the endoscope robot's end camera.

[0072] If the calculated image feature error is consistently not less than the threshold, repeat the above steps until the error is less than the threshold.

[0073] The specific definition is as follows:

[0074] The interaction matrix of the endoscopic robot is defined as follows:

[0075]

[0076] Where: L is the length of the continuum actuator. The symbols c(·) and s(·) are abbreviations for cos(·) and sin(·), respectively.

[0077] The motion model of the endoscopic robot is defined as follows:

[0078] in:

[0079] J0 = L s ·J r

[0080]

[0081] J r L represents the Jacobian matrix of the continuum actuator. s J0 represents the Jacobian matrix of the image, and J0 represents the Jacobian matrix of the endoscopic robot at the initial time. and This represents the pixel error between the visual feature coordinates and the image center coordinates. Where f and ρ represent the camera's focal length and pixel ratio, respectively.

[0082] The estimation of the Jacobian matrix of the subsequent endoscopic robot is defined as an optimization problem, specifically:

[0083]

[0084] subject to J k+1 =J k +ΔJ

[0085] in:

[0086]

[0087] e i =||Δs i -J k+1 Δq i -ω i ||2

[0088]

[0089] Where ΔJ represents the optimization variable to be solved. k and J k+1 Let ρ represent the Jacobian matrices at times k and k+1, respectively. i (e) represents the weight of each data pair in the container, assigned using the GM estimator. γ<1 is the forgetting factor with normals, and σ represents the expansion measure used to quantify the probability distribution, calculated using the median absolute deviation function. B is a constant chosen such that the MAD conforms to a normal distribution using the cumulative normal distribution function Φ(·). y i The L2 norm representing visual features, y i=||Δs i ||2.

[0090] The linearly extended state observer is defined as:

[0091]

[0092] Where e1 represents the observed output error, β1 and β2 represent the observation gain, and z1 and z2 represent the observer state. z1 represents the visual feature s, and z2 represents the observed perturbation ω.

[0093] Specifically, input compensation can be expressed as:

[0094]

[0095] The promotion form is represented as:

[0096]

[0097] in This indicates the error between the estimated value and the true value.

[0098] The observed perturbation ω can be expressed as:

[0099] ω=z2=∫-β2e1dt

[0100] To minimize visual tracking errors and meet the constraints of the endoscopic robot, a QP control framework is constructed, defined as follows:

[0101]

[0102] subject to q min ≤q≤q max

[0103]

[0104] In the above equation, the objective function consists of two terms, where β is a positive constant used to adjust the weights of these two terms. The first term quantifies the error between the expected and actual time derivatives of the visual target in image space. The second term aims to enable the endoscopic robot to complete the required task with minimal control input. The constraint equations must conform to the robot's physical constraints. and They represent q (velocity) The lower and upper limits of ). The repulsion field function was modified, where |e| represents the absolute distance between the target point and the image center, and r0 represents the radius of influence of the target point's pixel coordinates. Constraint function. It follows a normal distribution. The vertical and horizontal axes are scaled using parameters k1 and k2, respectively.

[0105] When controlling the endoscopic robot, the calculated movement speeds of each joint are transmitted to the various drive motors, thereby adjusting the pose of the end-effector camera. The error between the visual features and the image center decreases with increasing robot control. Simultaneously, the endoscope can be inserted into the human airway.

[0106] Example 2

[0107] This invention provides an optimal vision control system for a flexible endoscopic robot based on online kinematic model estimation, such as... Figure 1 As shown, the system includes: an image processing module, a Jacobi matrix robust estimation module, an optimal visual control module, a disturbance observation module, and an endoscope driving module. The image processing module uses a miniature camera to acquire real-time images of the body's cavities and calculates the cavity center position. The Jacobi matrix robust estimation module uses an improved iterative reweighted least squares algorithm to accurately and robustly estimate the Jacobi matrix of the endoscope robot. The optimal visual control module fully considers the robot's physical and control constraints, aiming to minimize visual tracking errors and obtain the optimal control values ​​for each joint. The disturbance observation module estimates the disturbance amount generated by the environment on the system based on the optimal control values ​​of the joints and changes in visual features. The endoscope driving module dynamically adjusts the posture of the endoscope robot according to the optimization results, enabling the target features to quickly and accurately approach the image center. A specific implementation of this embodiment includes:

[0108] The image processing module is used to acquire images of the cavities of the human body and calculate the pixel coordinates of the cavity center based on the images of the human body cavities.

[0109] The Jacobian matrix robust estimation module is used to calculate the Jacobian matrix of the endoscope robot under different postures.

[0110] The optimal vision control module is used to calculate the optimal control values ​​for each joint of the endoscope robot based on the desired position of the visual features.

[0111] The disturbance observation module is used to calculate and compensate for the disturbances caused by the environment to the system.

[0112] The endoscope drive module is used to input the optimal solutions for the speeds of each joint obtained through optimization into the drive motors of each joint of the robot, thereby realizing the control of the posture of the end-effector camera of the endoscope robot.

[0113] The image processing module acquires cavity images in real time, segments the cavity contour of the cavity image, and extracts the maximum inscribed circle of the cavity contour, using the center of the maximum inscribed circle as the cavity center.

[0114] The Jacobian matrix robust estimation module is used to calculate the Jacobian matrix of the endoscope robot under different postures during the control process based on the cavity center coordinates and the joint velocities of the endoscope robot.

[0115] The optimal vision control module is used to calculate the optimal solution for the movement speed of each joint of the endoscope robot based on the error between the cavity center coordinates and the image center and the Jacobian matrix of the endoscope robot.

[0116] The disturbance observation module is used to calculate the disturbance amount generated by the environment on the system based on the cavity center coordinates and joint motion velocity;

[0117] The endoscope drive module is used to input the optimal solution of the speed of each joint of the endoscope robot to the joint drive motor of the endoscope robot, thereby realizing the attitude control of the end-effector camera of the endoscope robot.

[0118] In this example, the image processing module includes: an image acquisition unit and an image processing unit;

[0119] The image acquisition unit is used to acquire images of the cavity;

[0120] The image processing unit is used to segment the cavity contour in the cavity image and extract the maximum inscribed circle of the cavity contour, with the center of the maximum inscribed circle as the cavity center.

[0121] In this example, the Jacobian matrix robust estimation module includes methods for calculating the Jacobian matrix of the endoscopic robot under different postures during control, based on the cavity center coordinates and joint velocities of the endoscopic robot:

[0122] Obtain the center position of the cavity image at time t-1 [u t-1 ,v t-1 ], obtain the center position of the cavity image at time t [u t ,v t Based on the cavity image center position at times t-1 and t and the expected velocity changes of each joint, the Jacobian matrix J of the endoscopic robot is obtained.

[0123] In this example, the optimal vision control module, based on the physical and control constraints of the endoscopic robot, aims to minimize the visual tracking error. The method for obtaining the optimal solution for the motion velocity of each joint of the endoscopic robot through an optimization algorithm includes:

[0124] The image feature error is calculated by comparing the cavity center with the image center under the current posture of the endoscopic robot.

[0125] A quadratic programming framework is constructed, and an objective function is built based on image feature error and the Jacobian matrix of the endoscopic robot. Combined with the physical and control constraints of the endoscopic robot, the optimal motion speed of each joint of the robot is calculated.

[0126] In this example, the disturbance observation module uses methods to estimate the disturbance caused by the environment on the system based on changes in the optimal control input of the joints and visual characteristics.

[0127] Establish the state-space equations for the endoscopic robot: in The expected motion speed for visual features. Let x1 be the joint velocity, x2 be the augmented state variable, and x3 be the estimation error of the model due to environmental disturbances.

[0128] Establish an observer, let the observed visual feature be z1, the observed environmental disturbance be z2, and the output error of the observed visual feature be e = z1 - x1;

[0129] Establish the observer equation and Where β1 and β2 represent the observation gain coefficients, β1 = 2w0, β2 = w0 2 ;

[0130] The environmental disturbance observed by the observer is ω=z2=∫-β2e1dt.

[0131] In this example, the method of inputting the optimal solution of the joint velocities of the endoscope robot into the joint drive motors of the endoscope robot in the endoscope drive module, thereby realizing the attitude control of the endoscope robot's end-effector camera, includes:

[0132] The optimal joint control value is linearly converted into motor rotation speed, and this speed is sent to the endoscope robot for motor drive to adjust the attitude of the endoscope robot's end camera.

[0133] Figure 3 This diagram illustrates the drive-vision mapping of the endoscope in an optimal vision control system for a flexible endoscopic robot, based on online kinematic model estimation, according to the present invention. In this diagram, {B} represents the base coordinate system of the endoscopic robot, {C} represents the camera coordinate system of the endoscopic robot, {S} represents the image coordinate system of the endoscopic robot, and P represents the position of the target feature in three-dimensional space. Coordinate systems {B} and {S} exhibit a linear mapping with respect to the Jacobian matrix of the endoscopic robot.

[0134] Figure 4This diagram illustrates the motion control of an endoscope robot within a cavity in an optimal visual control system for a flexible endoscope robot, based on online kinematic model estimation according to the present invention. In actual control, the endoscope drive module can realize the rotational movement of the endoscope. The bending motion θ and the feed motion z ensure that the endoscopic robot can perform navigation tasks within the confined cavity.

[0135] This invention calculates the pixel coordinates of the cavity center after image processing, then uses these coordinates with visual feature motion speed and joint speed to obtain the Jacobian matrix of the endoscopic robot. A quadratic programming framework is then used to solve for the optimal joint speed, and the state of each joint is adjusted by motors to reduce visual feature errors. This invention enables precise and stable control of the endoscope even when subjected to external forces, while simultaneously improving the safety and efficiency of endoscope use.

[0136] This invention innovatively proposes a model-free optimal visual motion control method. It does not rely on a precise analytical solution of the kinematic model, but estimates the kinematic model online from sensor data. Taking into account various constraints, it achieves high-precision and stable control of a flexible endoscopic robot, which helps to improve the quality and efficiency of robotic endoscopic surgery.

[0137] Example 3

[0138] This embodiment also provides a computer terminal device, including:

[0139] One or more processors;

[0140] Memory, coupled to the processor, is used to store one or more programs;

[0141] When one or more programs are executed by one or more processors, the one or more processors implement an optimal visual control method for a flexible endoscopic robot based on online estimation of a kinematic model.

[0142] This embodiment also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements an optimal visual control method for a flexible endoscopic robot based on online estimation of a kinematic model.

[0143] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An optimal visual control method for a flexible endoscopic robot based on online kinematic model estimation, characterized in that, Includes the following steps: Based on the acquisition of cavity images by an endoscopic robot, the cavity images are processed to obtain the cavity center coordinates and visual features; The Jacobian matrix of the endoscopic robot in each posture is calculated based on the coordinates of the cavity center and the joint velocity of the endoscopic robot. Obtain the coordinates of the image center, obtain the image feature error based on the cavity center coordinates and the image center coordinates, and calculate the optimal solution for the motion speed of each joint of the endoscopic robot based on the image feature error and the Jacobian matrix; The optimal solution of the movement speed of each joint of the endoscope robot is input into the drive motor of each joint of the endoscope robot to achieve optimal visual control; The process of calculating the optimal solution for the motion speed of each joint of the endoscopic robot includes: obtaining the motion speed of the desired visual feature based on the image feature error; constructing a quadratic programming model based on the Jacobian matrix and the motion speed of the desired visual feature; solving the quadratic programming model to obtain the optimal control quantity for each joint of the endoscopic robot; the cost function of the quadratic programming model includes: the error between the desired speed and the actual speed of the visual feature and a regularization term for achieving the target task with the minimum control quantity; The estimation of the Jacobian matrix of the spycam robot is defined as an optimization problem, specifically: in: in, This represents the optimization variable to be solved. and Let these represent the Jacobian matrices at times k and k+1, respectively. This represents the weight of each data pair in the container. It is a forgetting factor with positive constants. B represents the extended metric used to quantify the probability distribution, and is a constant. The L2 norm represents visual features. ; The expression for the quadratic programming model is: In the formula, It is a positive number. For joint velocity, This represents the motion model of an endoscopic robot. The Jacobian matrix represents the endoscopic robot. This represents the absolute distance between the target point and the image center. and These represent the vertical and horizontal axes of the scaling function, respectively. The radius of influence representing the pixel coordinates of the target point. Represents the constraint function. , They represent The lower and upper limits, , They represent speeds respectively. The lower and upper limits.

2. The optimal visual control method for a flexible endoscopic robot based on online kinematic model estimation according to claim 1, characterized in that, The process of processing the cavity image to obtain the cavity center coordinates includes: The cavity image is binarized using an adaptive thresholding method to obtain a binarized image; The cavity contour in the binarized image is segmented, and the maximum inscribed circle of the cavity contour is extracted to obtain the cavity center; The coordinates of the cavity center are determined based on the cavity center.

3. The optimal visual control method for a flexible endoscopic robot based on online kinematic model estimation according to claim 1, characterized in that, The optimal visual control method further includes calculating the environmental disturbance based on the optimal solution of the joint movement speed of the endoscope robot and the visual features. The process of calculating environmental disturbances includes: Establish the state-space equations and observer for the endoscopic robot; The observer equation is established based on the observer's observation data and the state-space equation. The environmental disturbance is determined based on the observer equation.

4. The optimal visual control method for a flexible endoscopic robot based on online kinematic model estimation according to claim 3, characterized in that, The expression for the environmental disturbance is: In the formula, This indicates the amount of environmental disturbance. Represents the gain coefficient of the observation. This represents the observed output error. This represents the observed environmental disturbance.

5. An optimal vision control system for a flexible endoscopic robot based on online kinematic model estimation, characterized in that, The system for implementing the method as described in claim 1 includes: an image processing module, a Jacobian matrix robust estimation module, an optimal visual control module, a perturbation observation module, and an endoscope driving module; The image processing module is used to acquire images of the cavities of the human body and calculate the coordinates of the cavity center based on the images. The robust estimation module for the Jacobian matrix is ​​used to calculate the Jacobian matrix of the endoscope robot in various postures based on the cavity center coordinates and the joint velocities of the endoscope robot. The optimal vision control module is used to calculate the optimal solution for the motion speed of each joint of the endoscope robot based on the Jacobian matrix and image feature error; The disturbance observation module is used to calculate the amount of environmental disturbance. The endoscope drive module is used to drive the joint drive motors of the endoscope robot based on the optimal solution of the joint movement speed of each joint of the endoscope robot to achieve attitude control of the end-effector camera of the endoscope robot.

6. A computer terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the optimal visual control method for a flexible endoscopic robot based on online kinematic model estimation as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the optimal visual control method for a flexible endoscopic robot based on online estimation of the kinematic model as described in any one of claims 1-4.

Images