Endoscopic robot and method for controlling same, system, electronic device, medium, and program product

Abstract

Provided in the present disclosure are a method for controlling an endoscopic robot, a system, an electronic device, a medium, and a program product. The endoscopic robot comprises a first mechanical joint, a second mechanical joint, a third mechanical joint, a fourth mechanical joint, a first virtual joint, and a second virtual joint. The method for controlling the endoscopic robot comprises: determining a first translation distance, a first rotation angle, a second rotation angle, a third rotation angle, a second translation distance, and a fourth rotation angle based on a target coordinate point of the endoscopic robot; and controlling a position of the endoscopic robot to be updated to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle. According to the present disclosure, the surgery can be observed from different angles by controlling the movement of the endoscopic robot comprising the combination of the mechanical joints and the virtual joints, thereby simplifying the manufacturing process of endoscopic robots and ensuring the stability of the structure of the endoscopic robots without compromising the degree of freedom.

Description

Endoscopic robots and their control methods, systems, electronic devices, media and program products

[0001] This application claims priority to Chinese patent application 2024114409065, filed on 2024 / 10 / 15. The entire contents of the aforementioned Chinese patent application are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of surgical robot technology, and in particular to an endoscopic robot and its control method, system, electronic equipment, medium and program products. Background Technology

[0003] In surgical robotics, robots that operate through natural orifices are equipped with robotic arms featuring multi-degree-of-freedom structures to drive endoscopes (referred to as endoscopic robots) for real-time imaging observation of natural orifices. Existing endoscopic robots generally consist of three subsystems: a vision system, a robotic arm, and a control system. The vision system comprises a lens, a light source, an image processing unit, and a display. The robotic arm system typically consists of a series of joints composed of mechanical links and pulleys, drive cables, lead screws, reducers, and other transmission mechanisms, as well as motors. The control system generally consists of a motion controller, a motor servo controller, and a host computer. The lens is usually mounted at the end of the robotic arm, and its movement is driven by the robotic arm to move the lens, changing its position and observation angle to meet clinical visual observation requirements.

[0004] Due to the elongated and tortuous nature of natural cavities and the need for imaging from different angles during surgery, endoscopic robots are often designed to be flexible, multi-degree-of-freedom, and small in diameter. In current technologies, the number of degrees of freedom and the diameter of the endoscopic robot are often mutually restrictive. More degrees of freedom mean more joints and mechanical structures, requiring more space and increasing the complexity of design and manufacturing processes. Furthermore, the interconnectedness of multiple joints can lead to a decrease in robot performance. Current technologies often employ reducing degrees of freedom to improve structural stability or adding more mechanical structures, but this comes at the cost of reduced performance and structural stability. Summary of the Invention

[0005] The technical problem to be solved by this disclosure is to overcome the defects in the prior art where the number of degrees of freedom and the diameter of the endoscope robot are mutually constrained, the process is complicated and affects the robot's performance, and to provide an endoscope robot and its control method, system, electronic equipment, medium and program products.

[0006] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0007] This disclosure provides an endoscopy robot, which includes at least one mechanical joint, a first virtual joint, and a second virtual joint. The first virtual joint is achieved by scaling an image of the endoscopy robot, and the second virtual joint is achieved by rotating the image of the endoscopy robot.

[0008] The endoscopic robot is configured as follows:

[0009] The translational distance and / or rotational angle of the at least one mechanical joint, the translational distance of the first virtual joint, and the rotational angle of the second virtual joint are determined based on the target coordinates of the endoscopy robot.

[0010] The position of the endoscopic robot is updated to the target coordinate point based on the translation distance and / or rotation angle of the mechanical joint, the translation distance of the first virtual joint, and the rotation angle of the second virtual joint.

[0011] Preferably, the mechanical joint includes a first mechanical joint, a second mechanical joint, a third mechanical joint, and a fourth mechanical joint. The endoscopic robot determines a first translational distance of the first mechanical joint, a first rotational angle of the second mechanical joint, a second rotational angle of the third mechanical joint, a third rotational angle of the fourth mechanical joint, a second translational distance of the first virtual joint, and a fourth rotational angle of the second virtual joint based on the target coordinate point of the endoscopic robot.

[0012] The endoscopic robot is configured as follows:

[0013] The position of the endoscopic robot is updated to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle.

[0014] Preferably, controlling the position update of the endoscopic robot to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle includes:

[0015] Determine the optimal imaging distance based on focal length and image distance;

[0016] The scaling factor is determined based on the optimal imaging distance and the second translation distance;

[0017] The third image coordinates of the first virtual joint after translation are determined based on the scaling factor, the first image coordinates corresponding to the initial position of the first virtual joint, the fourth rotation angle, and the optimal imaging distance; the fourth image coordinates of the second virtual joint after rotation are determined based on the scaling factor, the second image coordinates corresponding to the initial position of the second virtual joint, the fourth rotation angle, and the optimal imaging distance.

[0018] Preferably, determining the third image coordinates of the translated first virtual joint based on the scaling factor, the first image coordinates corresponding to the initial position of the first virtual joint, the fourth rotation angle, and the optimal imaging distance, and determining the fourth image coordinates of the rotated second virtual joint based on the scaling factor, the second image coordinates corresponding to the initial position of the second virtual joint, the fourth rotation angle, and the optimal imaging distance, includes:

[0019] The scaling factor is applied to the first image coordinates to obtain the first intermediate coordinates, and the scaling factor is applied to the second image coordinates to obtain the second intermediate coordinates;

[0020] Based on the fourth rotation angle, the first intermediate coordinates are rotated in two dimensions to obtain the third image coordinates corresponding to the first virtual joint.

[0021] Based on the fourth rotation angle, the second intermediate coordinates are rotated in two dimensions to obtain the fourth image coordinates corresponding to the second virtual joint.

[0022] Preferably, determining the optimal imaging distance based on the focal length and image distance includes:

[0023] The optimal imaging distance is calculated based on the Gaussian imaging formula, using the focal length and the image distance.

[0024] And / or,

[0025] Determining the scaling factor based on the optimal imaging distance and the second translation distance includes:

[0026] The scaling factor is calculated based on the perspective projection scaling model, using the optimal imaging distance and the second translation distance.

[0027] Preferably, determining the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the target coordinates of the endoscopy robot includes:

[0028] A first matrix is ​​determined based on the target coordinate point, and the first matrix is ​​used to characterize the spatial position and orientation of the target coordinate point.

[0029] Obtain a second matrix, which is used to characterize the joint positional relationship between the first mechanical joint, the second mechanical joint, the third mechanical joint, the fourth mechanical joint, the first virtual joint, and the second virtual joint;

[0030] The first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint are determined based on the first matrix and the second matrix.

[0031] Preferably, determining the first matrix based on the target coordinate points includes:

[0032] Based on the target pose corresponding to the target coordinate point, construct a first rotation matrix;

[0033] Wherein, the first rotation matrix represents the orientation of the target coordinate point;

[0034] The first translation vector corresponding to the end of the endoscope robot is determined based on the target coordinate point;

[0035] The first matrix is ​​obtained based on the first rotation matrix and the first translation vector;

[0036] The process of obtaining the second matrix includes:

[0037] Based on the element values ​​in the first matrix, obtain the second matrix corresponding to each element value in the first matrix;

[0038] The step of determining the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the first matrix and the second matrix includes:

[0039] Based on the first matrix and the second matrix, the kinematic equations of the endoscopy robot are solved to determine the third rotation angle of the fourth mechanical joint and the fourth rotation angle of the second virtual joint.

[0040] Based on the third rotation angle of the fourth mechanical joint and the first matrix, the second translation distance of the first virtual joint is determined;

[0041] The first angle variable is obtained based on the third rotation angle of the fourth mechanical joint and the first matrix;

[0042] The first rotation angle of the second mechanical joint is determined using the third rotation angle of the fourth mechanical joint, the first angle variable, the second translation distance of the first virtual joint, and the first matrix.

[0043] Based on the first angle variable and the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint is determined;

[0044] Based on the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint, the third rotation angle of the fourth mechanical joint, the second translation distance of the first virtual joint, and the first matrix, the first translation distance of the first mechanical joint is determined.

[0045] This disclosure provides a method for controlling an endoscopic robot. The endoscopic robot includes a first mechanical joint, a second mechanical joint, a third mechanical joint, a fourth mechanical joint, a first virtual joint, and a second virtual joint. The first virtual joint is achieved by scaling an image of the endoscopic robot, and the second virtual joint is achieved by rotating an image of the endoscopic robot. The method for controlling the endoscopic robot includes:

[0046] The first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint are determined based on the target coordinate point of the endoscopy robot.

[0047] The position of the endoscopic robot is updated to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle.

[0048] Preferably, the step of controlling the position update of the endoscopic robot to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle includes:

[0049] Determine the optimal imaging distance based on focal length and image distance;

[0050] The scaling factor is determined based on the optimal imaging distance and the second translation distance;

[0051] The third image coordinates of the first virtual joint after translation are determined based on the scaling factor, the first image coordinates corresponding to the initial position of the first virtual joint, the fourth rotation angle, and the optimal imaging distance; the fourth image coordinates of the second virtual joint after rotation are determined based on the scaling factor, the second image coordinates corresponding to the initial position of the second virtual joint, the fourth rotation angle, and the optimal imaging distance.

[0052] Preferably, the steps of determining the third image coordinates of the first virtual joint after translation based on the scaling factor, the first image coordinates corresponding to the initial position of the first virtual joint, the fourth rotation angle, and the optimal imaging distance, and determining the fourth image coordinates of the second virtual joint after rotation based on the scaling factor, the second image coordinates corresponding to the initial position of the second virtual joint, the fourth rotation angle, and the optimal imaging distance, include:

[0053] The scaling factor is applied to the first image coordinates to obtain the first intermediate coordinates, and the scaling factor is applied to the second image coordinates to obtain the second intermediate coordinates;

[0054] Based on the fourth rotation angle, the first intermediate coordinates are rotated in two dimensions to obtain the third image coordinates corresponding to the first virtual joint.

[0055] Based on the fourth rotation angle, the second intermediate coordinates are rotated in two dimensions to obtain the fourth image coordinates corresponding to the second virtual joint.

[0056] The coordinates of the third image and the coordinates of the fourth image are determined by the following formula:

[0057] Among them, z c Used to characterize the optimal imaging distance; The x-axis coordinate used to represent the coordinates of the third or fourth image; The y-axis coordinate is used to represent the third or fourth image coordinate; θ is used to represent the fourth rotation angle; k is used to represent the scaling factor; x is used to represent the x-axis coordinate of the first or second image coordinate; y is used to represent the y-axis coordinate of the first or second image coordinate.

[0058] Preferably, the step of determining the optimal imaging distance based on the focal length and image distance includes:

[0059] The optimal imaging distance is calculated based on the Gaussian imaging algorithm, using the focal length and the image distance.

[0060] The optimal imaging distance is determined using the following formula:

[0061] Among them, z c f is used to characterize the optimal imaging distance; f is used to characterize the focal length; d is used to characterize the image distance.

[0062] And / or,

[0063] The step of determining the scaling factor based on the optimal imaging distance and the second translation distance includes:

[0064] The scaling factor is calculated based on the perspective projection scaling model, using the optimal imaging distance and the second translation distance.

[0065] The scaling factor is determined using the following formula:

[0066] Where k represents the scaling factor; z c δ is used to characterize the optimal imaging distance; δ is used to characterize the second translation distance.

[0067] Preferably, the step of determining the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the target coordinates of the endoscopy robot includes:

[0068] A first matrix is ​​determined based on the target coordinate point, and the first matrix is ​​used to characterize the spatial position and orientation of the target coordinate point.

[0069] Obtain a second matrix, which is used to characterize the joint positional relationship between the first mechanical joint, the second mechanical joint, the third mechanical joint, the fourth mechanical joint, the first virtual joint, and the second virtual joint;

[0070] The first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint are determined based on the first matrix and the second matrix.

[0071] Preferably, the step of determining the first matrix based on the target coordinate points includes:

[0072] Based on the target pose corresponding to the target coordinate point, construct a first rotation matrix;

[0073] The rotation matrix represents the orientation of the target coordinate point;

[0074] The first translation vector corresponding to the end of the endoscope robot is determined based on the target coordinate point;

[0075] The first matrix is ​​obtained based on the first rotation matrix and the first translation vector;

[0076] The first matrix, determined based on the target coordinates, is calculated using the following formula: T 11 =-sin(q4)sin(q6)sin(q2+q3)+cos(q6)cos(q2+q3); T 12 =-sin(q4)cos(q6)sin(q2+q3)-sin(q6)cos(q2+q3); T 13 = -sin(q2+q3)cos(q4); T 14 =-L2sin(q2)-L3sin(q2+q3)-(L4+q5)sin(q2+q3)cos(q4); T 21 = sin(q6)cos(q4); T 22 =cos(q4)cos(q6); T 23 = -sin(q4); T 24 = (-L4-q5)sin(q4); T 31 =sin(q4)sin(q6)cos(q2+q3)+sin(q2+q3)cos(q6); T 32 =sin(q4)cos(q6)cos(q2+q3)-sin(q6)sin(q2+q3); T 33 =cos(q4)cos(q2+q3) T 34 =L1+L2cos(q2)+L3cos(q2+q3)+q1+(L4+q5)cos(q4)cos(q2+q3); T 41 =0; T 42 =0; T 43 =0; T 44 =1;

[0077] Where T is used to represent the first matrix, T 11 The formula used to characterize the first row and first column of the first matrix, T 12 The formula used to characterize the first row and second column of the first matrix, T 13 The formula used to characterize the first row and third column of the first matrix, T 14 The formula used to characterize the first row and fourth column of the first matrix, T 21 The formula used to characterize the second row and first column of the first matrix, T 22The formula used to characterize the second row and second column of the first matrix, T 23 The formula used to characterize the second row and third column of the first matrix, T 24 The formula used to characterize the second row and fourth column of the first matrix, T 31 The formula used to characterize the first column of the third row of the first matrix, T 32 The formula used to characterize the third row and second column of the first matrix, T 33 The formula used to characterize the third row and third column of the first matrix, T 34 The formula used to characterize the third row and fourth column of the first matrix, T 41 The formula used to characterize the first column of the fourth row of the first matrix, T 42 The formula used to characterize the fourth row and second column of the first matrix, T 43 The formula used to characterize the third column of the fourth row of the first matrix, T 44 The formula used to characterize the fourth row and fourth column of the first matrix is ​​as follows: q1 represents the first translation distance of the first mechanical joint, q2 represents the first rotation angle of the second mechanical joint, q3 represents the second rotation angle of the third mechanical joint, q4 represents the third rotation angle of the fourth mechanical joint, q5 represents the second translation distance of the first virtual joint, and q6 represents the fourth rotation angle of the second virtual joint; L1 represents the link length between the first and second mechanical joints, L2 represents the link length between the second and third mechanical joints, L3 represents the link length between the third and fourth mechanical joints, and L4 represents the link length between the first and second virtual joints.

[0078] The steps for obtaining the second matrix include:

[0079] Based on the first matrix, each element in the first matrix is ​​assigned a value to obtain the second matrix;

[0080] The second matrix is ​​obtained using the following formula:

[0081] Among them, T act Used to characterize the second matrix, T act11 Used to characterize T 11 The value of T act12 Used to characterize T 12 The value of T act13 Used to characterize T 13 The value of T act14 Used to characterize T 14 The value of T act21 Used to characterize T 21 The value of Tact22 Used to characterize T 22 The value of T act23 Used to characterize T 23 The value of T act24 Used to characterize T 24 The value of T act31 Used to characterize T 31 The value of T act32 Used to characterize T 32 The value of T act33 Used to characterize T 33 The value of T act34 Used to characterize T 34 The value of T act41 Used to characterize T 41 The value of T act42 Used to characterize T 42 The value of T act43 Used to characterize T 43 The value of T act44 Used to characterize T 44 The value;

[0082] The step of determining the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the first matrix and the second matrix includes:

[0083] Based on the first matrix and the second matrix, the kinematic equations of the endoscopy robot are solved to determine the third rotation angle of the fourth mechanical joint and the fourth rotation angle of the second virtual joint.

[0084] Based on the third rotation angle of the fourth mechanical joint and the first matrix, the second translation distance of the first virtual joint is determined;

[0085] The first angle variable is obtained based on the third rotation angle of the fourth mechanical joint and the first matrix;

[0086] The first rotation angle of the second mechanical joint is determined using the third rotation angle of the fourth mechanical joint, the first angle variable, the second translation distance of the first virtual joint, and the first matrix.

[0087] Based on the first angle variable and the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint is determined;

[0088] Based on the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint, the third rotation angle of the fourth mechanical joint, the second translation distance of the first virtual joint, and the first matrix, the first translation distance of the first mechanical joint is determined.

[0089] The first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint are determined by the following formula: q1=T 34 -L1-L2cos(q2)-(L3+(L4+q5)cos(q4))cos(q2+q3); q2=-arcsin((T 14 +((L4+q5)cos(q4)+L3)sin(q2+q3)) / L2);

[0090] q3=arctan(sin(q2+q3) / cos(q2+q3)-q2.

[0091] Where sin(q2+q3)=-T 13 / cos(q4); Cos(q2+q3)=T 33 / cos(q4) q4=-arcsin(T 23 ); q5 = -T 24 / sin(q4)-L4;

[0092] q6 = arctan(T) 21 T 22 );

[0093] Wherein, q1 is used to characterize the first translational distance of the first mechanical joint, q2 is used to characterize the first rotational angle of the second mechanical joint, q3 is used to characterize the second rotational angle of the third mechanical joint, q4 is used to characterize the third rotational angle of the fourth mechanical joint, q5 is used to characterize the second translational distance of the first virtual joint, q6 is used to characterize the fourth rotational angle of the second virtual joint, and sin(q2+q3) and cos(q2+q3) are the first angle variables.

[0094] This disclosure provides an endoscopic robot control system, the endoscopic robot including at least one mechanical joint and at least one virtual joint, the at least one virtual joint being implemented by endoscopic robot image scaling and / or image rotation, the endoscopic robot control system including:

[0095] The determination module is used to determine the translational distance and / or rotational angle of the at least one mechanical joint and the translational distance and / or rotational angle of the at least one virtual joint based on the target coordinate point of the endoscopy robot.

[0096] The position update module is used to control the position update of the endoscopic robot to the target coordinate point based on the translation distance and / or rotation angle of the mechanical joint and the translation distance and / or rotation angle of the virtual joint.

[0097] Preferably, the virtual joint includes a first virtual joint and / or a second virtual joint, wherein the first virtual joint is achieved by scaling the endoscopic robot image and the second virtual joint is achieved by rotating the endoscopic robot image;

[0098] The determining module is used to determine the translation distance of the first virtual joint and / or the rotation angle of the second virtual joint based on the target coordinate point of the endoscopy robot.

[0099] The determining module is used to determine the optimal imaging distance based on the focal length and image distance, and to determine the scaling factor based on the optimal imaging distance and the translation distance of the first virtual joint.

[0100] Preferably, the mechanical joint includes a first mechanical joint, a second mechanical joint, a third mechanical joint, and a fourth mechanical joint;

[0101] The determining module is used to determine the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the target coordinate point of the endoscopy robot.

[0102] The position update module is used to control the position update of the endoscope robot to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle.

[0103] This disclosure provides an endoscopy robot, including at least one mechanical joint and at least one virtual joint, the at least one virtual joint being implemented by endoscopy robot image scaling and / or image rotation, and the endoscopy robot also including an endoscopy robot control system as described above.

[0104] This disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and used to run on the processor, wherein the processor executes the computer program to implement the above-described endoscopic robot control method.

[0105] This disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described endoscopic robot control method.

[0106] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the above-described endoscopic robot control method.

[0107] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.

[0108] The positive and progressive effects of this disclosure are as follows:

[0109] This disclosure allows for observation of surgery from different angles by controlling the movement of an endoscopic robot that combines mechanical and virtual joints, simplifying the manufacturing process of the endoscopic robot while ensuring the structural stability of the endoscopic robot without sacrificing degrees of freedom. Attached Figure Description

[0110] Figure 1 is a structural diagram of an endoscopy robot provided in Embodiment 1 of this disclosure;

[0111] Figure 2 is a flowchart of an endoscopy robot control method provided in Embodiment 1 of this disclosure;

[0112] Figure 3 is a schematic diagram of the transformation from the world coordinate system to the camera coordinate system in a specific example of an endoscopic robot control method provided in Embodiment 1 of this disclosure;

[0113] Figure 4 is a schematic diagram of the transformation from the camera coordinate system to the image coordinate system in a specific example of an endoscopic robot control method provided in Embodiment 1 of this disclosure.

[0114] Figure 5 is a schematic diagram of the transformation from the image coordinate system to the pixel coordinate system in a specific example of an endoscopic robot control method provided in Embodiment 1 of this disclosure;

[0115] Figure 6 is a schematic diagram of the optimal imaging distance for a specific example of an endoscopic robot control method provided in Embodiment 1 of this disclosure;

[0116] Figure 7 is a flowchart of a specific example of an endoscopic robot control method provided in Embodiment 1 of this disclosure;

[0117] Figure 8 is a structural diagram of an endoscopy robot control system provided in Embodiment 2 of this disclosure;

[0118] Figure 9 is a schematic diagram of the electronic device structure of Embodiment 3 of this disclosure. Detailed Implementation

[0119] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0120] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this disclosure does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not be construed as an unnecessary limitation. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0121] Example 1

[0122] This embodiment provides an endoscope robot according to the present disclosure. Referring to Figure 1, the endoscope robot includes at least one mechanical joint, a first virtual joint, and a second virtual joint. The first virtual joint is realized by scaling the endoscope robot image, and the second virtual joint is realized by rotating the endoscope robot image.

[0123] The endoscopic robot is configured as follows:

[0124] Determine the translational distance and / or rotational angle of at least one mechanical joint, the translational distance of the first virtual joint, and the rotational angle of the second virtual joint based on the target coordinates of the endoscopy robot.

[0125] The position of the endoscopic robot is updated to the target coordinate point based on the translation distance and / or rotation angle of the mechanical joint, the translation distance of the first virtual joint, and the rotation angle of the second virtual joint.

[0126] In an alternative implementation, the endoscopic robot may also include at least one mechanical joint and one virtual joint. The endoscopic robot may only include a first virtual joint implemented by image scaling or a second virtual joint implemented by image rotation, and the control of the remaining degrees of freedom may be achieved by the mechanical joint.

[0127] The endoscopic robot is configured as follows:

[0128] Determine the translation distance and / or rotation angle of at least one mechanical joint, the translation distance of the first virtual joint, or the rotation angle of the second virtual joint based on the target coordinates of the endoscopic robot.

[0129] The position of the endoscopic robot is updated to the target coordinate point based on the translation distance and / or rotation angle of the mechanical joint, the translation distance of the first virtual joint, or the rotation angle of the second virtual joint.

[0130] In one optional implementation, the mechanical joint includes a first mechanical joint, a second mechanical joint, a third mechanical joint, and a fourth mechanical joint. The endoscopic robot determines a first translational distance of the first mechanical joint, a first rotational angle of the second mechanical joint, a second rotational angle of the third mechanical joint, a third rotational angle of the fourth mechanical joint, a second translational distance of the first virtual joint, and a fourth rotational angle of the second virtual joint based on the target coordinate point of the endoscopic robot.

[0131] The endoscopic robot is configured as follows:

[0132] The position of the endoscopic robot is updated to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle.

[0133] In one optional implementation, controlling the position update of the endoscopic robot to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle includes:

[0134] Determine the optimal imaging distance based on focal length and image distance;

[0135] The scaling factor is determined based on the optimal imaging distance and the second translation distance;

[0136] The third image coordinates of the first virtual joint after translation are determined based on the scaling factor, the first image coordinates corresponding to the initial position of the first virtual joint, the fourth rotation angle, and the optimal imaging distance; the fourth image coordinates of the second virtual joint after rotation are determined based on the scaling factor, the second image coordinates corresponding to the initial position of the second virtual joint, the fourth rotation angle, and the optimal imaging distance.

[0137] The coordinates of the third and fourth images are determined using the following formula:

[0138] Where zc is used to characterize the optimal imaging distance; The x-axis coordinate used to represent the coordinates of the third or fourth image; The y-axis coordinate is used to represent the third or fourth image coordinate; θ is used to represent the fourth rotation angle; k is used to represent the scaling factor; x is used to represent the x-axis coordinate of the first or second image coordinate; y is used to represent the y-axis coordinate of the first or second image coordinate.

[0139] In an optional implementation, determining the third image coordinates of the translated first virtual joint based on the scaling factor, the first image coordinates corresponding to the initial position of the first virtual joint, the fourth rotation angle, and the optimal imaging distance, and determining the fourth image coordinates of the rotated second virtual joint based on the scaling factor, the second image coordinates corresponding to the initial position of the second virtual joint, the fourth rotation angle, and the optimal imaging distance, includes:

[0140] Applying the scaling factor to the first image coordinates yields the first intermediate coordinates, and applying the scaling factor to the second image coordinates yields the second intermediate coordinates;

[0141] Based on the fourth rotation angle, the first intermediate coordinates are rotated in two dimensions to obtain the third image coordinates corresponding to the first virtual joint.

[0142] Based on the fourth rotation angle, the second intermediate coordinates are rotated in two dimensions to obtain the fourth image coordinates corresponding to the second virtual joint.

[0143] In an optional implementation, determining the optimal imaging distance based on the focal length and image distance includes:

[0144] Based on the Gaussian imaging formula, the optimal imaging distance is calculated using focal length and image distance.

[0145] The optimal imaging distance is determined using the following formula:

[0146] Among them, z c f is used to characterize the optimal imaging distance; f is used to characterize the focal length; d is used to characterize the image distance.

[0147] And / or,

[0148] Determining the scaling factor based on the optimal imaging distance and the second translation distance includes:

[0149] The scaling factor is calculated based on the perspective projection scaling model, using the optimal imaging distance and the second translation distance.

[0150] The scaling factor is determined using the following formula:

[0151] Where k represents the scaling factor; z c δ is used to characterize the optimal imaging distance; δ is used to characterize the second translation distance.

[0152] In an optional implementation, determining the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the target coordinates of the endoscopic robot includes:

[0153] The first matrix is ​​determined based on the target coordinates, and the first matrix is ​​used to characterize the spatial position and orientation of the target coordinates.

[0154] Obtain the second matrix, which is used to characterize the joint positional relationship between the first mechanical joint, the second mechanical joint, the third mechanical joint, the fourth mechanical joint, the first virtual joint, and the second virtual joint;

[0155] The first translation distance of the first mechanical joint, the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint, the third rotation angle of the fourth mechanical joint, the second translation distance of the first virtual joint, and the fourth rotation angle of the second virtual joint are determined based on the first matrix and the second matrix.

[0156] In an optional implementation, determining the first matrix based on the target coordinates includes:

[0157] Based on the target pose corresponding to the target coordinate point, construct the first rotation matrix;

[0158] The first rotation matrix represents the orientation of the target coordinate point;

[0159] Determine the first translation vector corresponding to the end effector of the endoscopic robot based on the target coordinate point;

[0160] The first matrix is ​​obtained based on the first rotation matrix and the first translation vector;

[0161] The first matrix, determined based on the target coordinates, is calculated using the following formula:

[0162] The first matrix, determined based on the target coordinates, is calculated using the following formula: T 11 =-sin(q4)sin(q6)sin(q2+q3)+cos(q6)cos(q2+q3); T 12 =-sin(q4)cos(q6)sin(q2+q3)-sin(q6)cos(q2+q3); T 13 = -sin(q2+q3)cos(q4); T 14 =-L2sin(q2)-L3sin(q2+q3)-(L4+q5)sin(q2+q3)cos(q4); T 21 = sin(q6)cos(q4); T 22 =cos(q4)cos(q6);

[0163] T 23 = -sin(q4); T 24= (-L4-q5)sin(q4); T 31 =sin(q4)sin(q6)cos(q2+q3)+sin(q2+q3)cos(q6); T 32 =sin(q4)cos(q6)cos(q2+q3)-sin(q6)sin(q2+q3); T 33 =cos(q4)cos(q2+q3) T 34 =L1+L2cos(q2)+L3cos(q2+q3)+q1+(L4+q5)cos(q4)cos(q2+q3); T 41 =0; T 42 =0; T 43 =0; T 44 =1;

[0164] Where T is used to represent the first matrix, T 11 The formula used to characterize the first row and first column of the first matrix, T 12 The formula used to characterize the first row and second column of the first matrix, T 13 The formula used to characterize the first row and third column of the first matrix, T 14 The formula used to characterize the first row and fourth column of the first matrix, T 21 The formula used to characterize the first column of the second row of the first matrix, T 22 The formula used to characterize the second row and second column of the first matrix, T 23 The formula used to characterize the second row and third column of the first matrix, T 24 The formula used to characterize the second row and fourth column of the first matrix, T 31 The formula used to characterize the first column of the third row of the first matrix, T 32 The formula T33 is used to characterize the third row and second column of the first matrix. 34 The formula used to characterize the third row and fourth column of the first matrix, T 41 The formula used to characterize the first column of the fourth row of the first matrix, T 42 The formula used to characterize the fourth row and second column of the first matrix, T 43 The formula used to characterize the fourth row and third column of the first matrix, T 44The formula used to characterize the fourth row and fourth column of the first matrix is ​​as follows: q1 represents the first translation distance of the first mechanical joint, q2 represents the first rotation angle of the second mechanical joint, q3 represents the second rotation angle of the third mechanical joint, q4 represents the third rotation angle of the fourth mechanical joint, q5 represents the second translation distance of the first virtual joint, and q6 represents the fourth rotation angle of the second virtual joint; L1 represents the link length of the first and second mechanical joints, L2 represents the link length of the second and third mechanical joints, L3 represents the link length of the third and fourth mechanical joints, and L4 represents the link length of the first and second virtual joints.

[0165] Obtaining the second matrix includes:

[0166] Based on the element values ​​in the first matrix, obtain the second matrix corresponding to each element value in the first matrix;

[0167] The second matrix is ​​obtained using the following formula:

[0168] Among them, T act Used to characterize the second matrix, T act11 Used to characterize T 11 The value of T act12 Used to characterize T 12 The value of T act13 Used to characterize T 13 The value of T act14 Used to characterize T 14 The value of T act21 Used to characterize T 21 The value of T act22 Used to characterize T 22 The value of T act23 Used to characterize T 23 The value of T act24 Used to characterize T 24 The value of T act31 Used to characterize T 31 The value of T act32 Used to characterize T 32 The value of T act33 Used to characterize T 33 The value of T act34 Used to characterize T 34 The value of T act41 Used to characterize T 41 The value of T act42 Used to characterize T 42 The value of T act43 Used to characterize T 43 The value of T act44 Used to characterize T 44 The value;

[0169] The determination of the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the first matrix and the second matrix includes:

[0170] Based on the first and second matrices, the kinematic equations of the endoscope robot are solved to determine the third rotation angle of the fourth mechanical joint and the fourth rotation angle of the second virtual joint.

[0171] Based on the third rotation angle of the fourth mechanical joint and the first matrix, the second translation distance of the first virtual joint is determined;

[0172] The first angle variable is obtained based on the third rotation angle of the fourth mechanical joint and the first matrix;

[0173] The first rotation angle of the second mechanical joint is determined using the third rotation angle of the fourth mechanical joint, the first angle variable, the second translation distance of the first virtual joint, and the first matrix.

[0174] Based on the first angle variable and the first rotation angle of the second mechanical joint, determine the second rotation angle of the third mechanical joint.

[0175] Based on the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint, the third rotation angle of the fourth mechanical joint, the second translation distance of the first virtual joint, and the first matrix, the first translation distance of the first mechanical joint is determined.

[0176] The first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint are determined by the following formula: q1=T 34 -L1-L2cos(q2)-(L3+(L4+q5)cos(q4))cos(q2+q3); q2=-arcsin((T 14 +((L4+q5)cos(q4)+L3)sin(q2+q3)) / L2); q3=arctan(sin(q2+q3) / cos(q2+q3)-q2.

[0177] Where sin(q2+q3)=-T 13 / cos(q4);

[0178] cos(q2+q3)=T 33 / cos(q4) q4=-arcsin(T23 ); q5 = -T 24 / sin(q4)-L4; q6=arctan(T 21 T 22 );

[0179] Where q1 represents the first translational distance of the first mechanical joint, q2 represents the first rotational angle of the second mechanical joint, q3 represents the second rotational angle of the third mechanical joint, q4 represents the third rotational angle of the fourth mechanical joint, q5 represents the second translational distance of the first virtual joint, q6 represents the fourth rotational angle of the second virtual joint, and sin(q2+q3) and cos(q2+q3) are the first angle variables.

[0180] Example 2

[0181] This embodiment provides a control method for an endoscopy robot. Referring to Figure 1, the endoscopy robot includes a first mechanical joint J1, a second mechanical joint J2, a third mechanical joint J3, a fourth mechanical joint J4, a first virtual joint J5, and a second virtual joint J6. J1 is a translational joint implemented by a mechanical structure and driven by a corresponding motor. J2, J3, and J4 are rotational joints implemented by a mechanical structure and driven by corresponding motors. Joint J5 is a virtual translational joint implemented by an image, simulated by image scaling. J6 is a virtual rotational joint implemented by an image, simulated by image rotation. The first and second virtual joints are displayed by mapping to an image coordinate system. The initial position of the first virtual joint corresponds to the first image coordinates, and the initial position of the second virtual joint corresponds to the second image coordinates. The translation of the first virtual joint is achieved by scaling the first image coordinates, and the rotation of the second virtual joint is determined by rotating the second image coordinates.

[0182] In an optional implementation, step S1 is preceded by:

[0183] s0. Determine the target coordinates of the endoscopy robot according to the user's instructions.

[0184] User instructions can be set according to the actual situation.

[0185] Referring to Figure 2, the control method for the endoscopic robot includes:

[0186] S1. Determine the first translation distance of the first mechanical joint, the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint, the third rotation angle of the fourth mechanical joint, the second translation distance of the first virtual joint, and the fourth rotation angle of the second virtual joint based on the target coordinates of the endoscope robot.

[0187] S2. Based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle, control the position of the endoscope robot to update to the target coordinate point.

[0188] In this embodiment, the endoscopic robot consists of 6 serial joints. In the master-slave teleoperation control, the target coordinate points (Xe, Ye, Ze) of the endoscopic robot are inversely solved to calculate the distance or angle that each joint needs to move. The motor or image processor is synchronously controlled to realize the joints, thereby operating the endoscope lens e to observe from different positions and angles. That is, by controlling the movement of the endoscopic robot, which includes a combination of mechanical joints and virtual joints, the surgery can be observed from different angles. This simplifies the manufacturing process of the endoscopic robot and ensures the stability of the endoscopic robot structure without sacrificing the degrees of freedom.

[0189] In an optional implementation, step S2 includes:

[0190] S21. Determine the optimal imaging distance based on the focal length and image distance;

[0191] S22. Determine the scaling factor based on the optimal imaging distance and the second translation distance;

[0192] S23. Determine the third image coordinates corresponding to the first virtual joint after translation based on the scaling factor, the first image coordinates, the fourth rotation angle, and the optimal imaging distance; determine the fourth image coordinates corresponding to the second virtual joint after rotation based on the scaling factor, the second image coordinates, the fourth rotation angle, and the optimal imaging distance.

[0193] In an optional implementation, step S23 includes:

[0194] Applying the scaling factor to the first image coordinates yields the first intermediate coordinates, and applying the scaling factor to the second image coordinates yields the second intermediate coordinates;

[0195] Based on the fourth rotation angle, the first intermediate coordinates are rotated in two dimensions to obtain the third image coordinates corresponding to the first virtual joint.

[0196] Based on the fourth rotation angle, the second intermediate coordinates are rotated in two dimensions to obtain the fourth image coordinates corresponding to the second virtual joint.

[0197] S231. Determine the coordinates of the third and fourth images using the following formula:

[0198] Among them, z c Used to characterize the optimal imaging distance; The x-axis coordinate used to represent the coordinates of the third or fourth image; The y-axis coordinate is used to represent the third or fourth image coordinate; θ is used to represent the fourth rotation angle; k is used to represent the scaling factor; x is used to represent the x-axis coordinate of the first or second image coordinate; y is used to represent the y-axis coordinate of the first or second image coordinate.

[0199] In an optional implementation, step S21:

[0200] Based on the Gaussian imaging algorithm, the optimal imaging distance is calculated using focal length and image distance;

[0201] The optimal imaging distance is determined using the following formula:

[0202] Among them, z c f is used to characterize the optimal imaging distance; f is used to characterize the focal length; d is used to characterize the image distance.

[0203] In an optional implementation, step S22 includes:

[0204] The scaling factor is calculated based on the perspective projection scaling model, using the optimal imaging distance and the second translation distance.

[0205] The scaling factor is determined using the following formula:

[0206] Where k represents the scaling factor; z c δ is used to characterize the optimal imaging distance; δ is used to characterize the second translation distance.

[0207] In an optional implementation, step S1 includes:

[0208] S11. Determine the first matrix based on the target coordinates.

[0209] The first matrix is ​​used to characterize the spatial position and orientation of the target coordinate point.

[0210] S12, Obtain the second matrix.

[0211] The second matrix is ​​used to characterize the joint positional relationships between the first mechanical joint, the second mechanical joint, the third mechanical joint, the fourth mechanical joint, the first virtual joint, and the second virtual joint. The second matrix is ​​represented by a homogeneous matrix.

[0212] S13. Determine the first translation distance of the first mechanical joint, the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint, the third rotation angle of the fourth mechanical joint, the second translation distance of the first virtual joint, and the fourth rotation angle of the second virtual joint based on the first matrix and the second matrix.

[0213] Specifically, step S11 includes:

[0214] Based on the target pose corresponding to the target coordinate point, construct the first rotation matrix;

[0215] The rotation matrix represents the orientation of the target coordinate point;

[0216] Determine the first translation vector corresponding to the end effector of the endoscopic robot based on the target coordinate point;

[0217] The first matrix is ​​obtained based on the first rotation matrix and the first translation vector;

[0218] The first matrix, determined based on the target coordinates, is calculated using the following formula: T 11 =-sin(q4)sin(q6)sin(q2+q3)+cos(q6)cos(q2+q3); T 12 =-sin(q4)cos(q6)sin(q2+q3)-sin(q6)cos(q2+q3); T 13 = -sin(q2+q3)cos(q4); T 14 =-L2sin(q2)-L3sin(q2+q3)-(L4+q5)sin(q2+q3)cos(q4); T 21 = sin(q6)cos(q4); T 22 =cos(q4)cos(q6);

[0219] T 23 = -sin(q4);

[0220] T 24 = (-L4-q5)sin(q4);

[0221] T 31 =sin(q4)sin(q6)cos(q2+q3)+sin(q2+q3)cos(q6);

[0222] T 32 =sin(q4)cos(q6)cos(q2+q3)-sin(q6)sin(q2+q3); T 33 =cos(q4)cos(q2+q3) T 34 =L1+L2cos(q2)+L3cos(q2+q3)+q1+(L4+q5)cos(q4)cos(q2+q3); T 41 =0; T 42 =0; T 43 =0; T 44 =1:

[0223] Where T is used to represent the first matrix, T 11 The formula used to characterize the first row and first column of the first matrix, T 12 The formula used to characterize the first row and second column of the first matrix, T 13 The formula used to characterize the first row and third column of the first matrix, T 14 The formula used to characterize the first row and fourth column of the first matrix, T 21 The formula used to characterize the first column of the second row of the first matrix, T 22 The formula used to characterize the second row and second column of the first matrix, T 23 The formula used to characterize the second row and third column of the first matrix, T 24 The formula used to characterize the second row and fourth column of the first matrix, T 31 The formula used to characterize the first column of the third row of the first matrix, T 32 The formula used to characterize the third row and second column of the first matrix, T 33 The formula used to characterize the third row and third column of the first matrix, T 34 The formula used to characterize the third row and fourth column of the first matrix, T 41 The formula used to characterize the first column of the fourth row of the first matrix, T 42 The formula used to characterize the fourth row and second column of the first matrix, T 43 The formula used to characterize the fourth row and third column of the first matrix, T 44 The formula used to characterize the fourth row and fourth column of the first matrix is ​​as follows: q1 represents the first translation distance of the first mechanical joint, q2 represents the first rotation angle of the second mechanical joint, q3 represents the second rotation angle of the third mechanical joint, q4 represents the third rotation angle of the fourth mechanical joint, q5 represents the second translation distance of the first virtual joint, and q6 represents the fourth rotation angle of the second virtual joint; L1 represents the link length of the first and second mechanical joints, L2 represents the link length of the second and third mechanical joints, L3 represents the link length of the third and fourth mechanical joints, and L4 represents the link length of the first and second virtual joints.

[0224] Step S12 includes:

[0225] Based on the element values ​​in the first matrix, obtain the second matrix corresponding to each element value in the first matrix;

[0226] The second matrix is ​​obtained using the following formula:

[0227] Among them, T act Used to characterize the second matrix, T act11 Used to characterize T 11 The value of Tact12 Used to characterize T 12 The value of T act13 Used to characterize T 13 The value of T act14 Used to characterize T 14 The value of T act21 Used to characterize T 21 The value of T act22 Used to characterize T 22 The value of T act23 Used to characterize T 23 The value of T act24 Used to characterize T 24 The value of T act31 Used to characterize T 31 The value of T act32 Used to characterize T 32 The value of T act33 Used to characterize T 33 The value of T act34 Used to characterize T 34 The value of T act41 Used to characterize T 41 The value of T act42 Used to characterize T 42 The value of T act43 Used to characterize T 43 The value of T act44 Used to characterize T 44 The value;

[0228] Step S13 includes:

[0229] The first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint are determined by the following formula: q1=T 34 -L1-L2cos(q2)-(L3+(L4+q5)cos(q4))cos(q2+q3);

[0230] q2=-arcsin((T 14 +((L4+q5)cos(q4)+L3)sin(q2+q3)) / L2);

[0231] q3=arctan(sin(q2+q3) / cos(q2+q3)-q2.

[0232] Where sin(q2+q3)=-T 13 / cos(q4);

[0233] cos(q2+q3)=T 33 / cos(q4)

[0234] q4=-arcsin(T 23 );

[0235] q5 = -T 24 / sin(q4)-L4;

[0236] q6 = arctan(T) 21 T 22 );

[0237] Where q1 represents the first translational distance of the first mechanical joint, q2 represents the first rotational angle of the second mechanical joint, q3 represents the second rotational angle of the third mechanical joint, q4 represents the third rotational angle of the fourth mechanical joint, q5 represents the second translational distance of the first virtual joint, q6 represents the fourth rotational angle of the second virtual joint, and sin(q2+q3) and cos(q2+q3) are the first angle variables.

[0238] In this embodiment, the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint are determined by inverse calculation of the first matrix and the second matrix, thereby improving the observation efficiency of the endoscopic robot.

[0239] In an optional implementation, step S2 includes:

[0240] Map the second translation distance to the pixel coordinate system.

[0241] The position of the endoscopic robot is updated based on the second translation distance in the pixel coordinate system.

[0242] In an optional implementation, step S2 may further include:

[0243] Map the fourth rotation angle to the pixel coordinate system.

[0244] The position of the endoscopic robot is updated based on the second translation distance in the pixel coordinate system.

[0245] In an optional implementation, step S2 may further include:

[0246] Based on the first translation distance, the first rotation angle, the second rotation angle, and the third rotation angle, the first motor running angle corresponding to the first mechanical joint, the second motor running angle corresponding to the second mechanical joint, the third motor running angle corresponding to the third mechanical joint, and the fourth motor running angle corresponding to the fourth mechanical joint are determined respectively.

[0247] The position of the endoscopic robot is updated based on the operating angles of the first motor, the second motor, the third motor, and the fourth motor.

[0248] In this embodiment, the mechanical joints can control the endoscope robot to reach a preset position by determining the motor's operating angle. The virtual joints can be transformed into an image coordinate system and then into a pixel coordinate system by scaling and rotation methods. The second translation distance and the fourth rotation angle are mapped to the pixel coordinate system, and then the endoscope robot is controlled to reach the preset position based on the second translation distance and the fourth rotation angle determined in the pixel coordinate system. This embodiment achieves a 6-DOF robot through the linkage processing of the mechanical joints and the image virtual joints without increasing the complexity of the mechanical structure, thus balancing the degrees of freedom of the endoscope robot with the stability of the mechanical structure.

[0249] The following is a specific example to illustrate the endoscopic robot control method of this embodiment.

[0250] As shown in Figure 1, the endoscopic robot in this example consists of 4 mechanical joints and 2 virtual joints.

[0251] J1, J2, J3, and J4 are mechanical joints, which respectively achieve:

[0252] J1 is a translational axis (first mechanical joint), realizing the advance and exit degrees of freedom; J2 (second mechanical joint) and J3 (third mechanical joint) are rotational axes, realizing the pitch degree of freedom; J4 (fourth mechanical joint) is a rotational axis, realizing the yaw degree of freedom.

[0253] Additionally, J5 and J6 are virtual image joints, and lens e is mounted on the end link of the endoscopic robot's robotic arm, respectively achieving:

[0254] J5 is the image scaling axis (first virtual joint), realizing the translational degree of freedom of the viewpoint along the view direction.

[0255] J6 is the image rotation axis (second virtual joint), realizing the degree of freedom of rotation along the field of view.

[0256] The virtual coordinate axes of J5 and J6 have equivalent rotation and scaling effects to the image coordinate axes. The specific principle is as follows:

[0257] Without considering distortion, the imaging model for the camera is as follows.

[0258] For a point P in a three-dimensional rectangular coordinate system world coordinate system w (x w y w , z w), mapped to the pixel coordinate system o(u, v), is divided into three parts.

[0259] Part 1, as shown in Figure 3, world coordinate system P w To camera coordinate system P c (x c y c , z c Transformation of )

[0260] Where R is a 3x3 rotation matrix and t is a 3x1 translation vector.

[0261] Part Two: Camera Coordinate System P c The transformation to the image coordinate system P(x, y), as shown in Figure 4, is expressed by the following formula:

[0262] Where f is the camera focal length.

[0263] Part Three: The transformation from the image coordinate system P(x, y) to the pixel coordinate system o(u, v), as shown in Figure 5, has the following formula:

[0264] Where dx is the distance between each pixel in the x-direction and dy is the distance between each pixel in the y-direction.

[0265] Combining the above three parts, we can obtain the mapping from the world coordinate system to the pixel coordinate system, as shown in the following formula:

[0266] Among them, u O v0 is the coordinate value of the pixel at the center of the image.

[0267] The virtual coordinate axes of J5 and J6 can be equivalently rotated and scaled around the center of the image coordinate system at a point P(x, y). The scaled point is... Its rotation angle is θ, and its scaling factor is k:

[0268] The equation is rearranged as follows

[0269] We can assume that the Z-axis of the image coordinate system has been translated, and the Z-axis coordinates after the translation are:

[0270] Combining the above formula for change, then:

[0271] Equivalent to

[0272] in,

[0273] This formula can be understood as rotating and translating a point in the image coordinate system relative to the initial coordinate system. The rotation angle is 0 and the translation amount is δ. This can be equivalent to rotating a point in the camera coordinate system along the Z-axis and scaling it by a factor of k, where the rotation angle is 0 and the scaling factor is δ. This method achieves equivalent motion for the two virtual joints J5 and J6, namely translation and rotation.

[0274] The following describes a specific process for determining the scaling factor k:

[0275] When the virtual joint J5 undergoes translational motion, we can assume that the Z-axis direction of the camera coordinate system corresponding to J5 is translated and scaled by a factor of k. The Z-axis coordinate after translation is:

[0276] That is, if there exists a target position along the z-axis The required scaling amount is Let's assume the translation δ relative to the initial position is:

[0277] Therefore, we can obtain:

[0278] For a scaling rotation angle of 0, since the virtual joint rotation acts in the z-axis direction, the angle of virtual joint rotation is the same as the angle of image rotation.

[0279] When J5 and J6 are translated or rotated, the camera coordinates of J5 and J6 first undergo the following transformations:

[0280] Secondly, the camera coordinate system It will be transformed into an image coordinate system.

[0281] Finally, the image coordinate system will be... Transform into pixel coordinate system

[0282] Where dx is the distance between each pixel in the x-direction; dy is the distance between each pixel in the y-direction, where u0 and v O These are the coordinates of the pixel at the center of the image.

[0283] In virtual joint control, the coordinate system for rotation and translation is used as input conditions, that is, it is necessary to obtain the rotation angle 0 and the scaling factor of k.

[0284] The above Z cThis refers to the Z-coordinate of the object being photographed in the camera coordinate system. Since the Z-coordinate of the object changes with the camera's position and orientation during actual imaging, a virtual object needs to be selected as a reference for the virtual joint motion. In this invention, the optimal imaging distance of the camera is selected as the Z-coordinate. c This is to adapt to most shooting situations. The optimal imaging distance is related to the focal length and image distance, as shown in Figure 6. The specific calculation results are as follows:

[0285] Where f is the camera focal length and d is the camera image distance.

[0286] Figure 7 illustrates the control process of the endoscopic robot in this example. Typically:

[0287] S701, The host computer sends the target coordinates of the endoscope robot as an instruction, that is, the observation position and angle that the endoscope needs to be in.

[0288] S702. Calculate the angles corresponding to each joint by using the inverse kinematics method to obtain the target coordinates of the endoscope robot.

[0289] The specific inverse solution method is as follows:

[0290] As shown in Figure 1, the reference coordinate system for each joint link is set according to the Modified Denavit-Hartenberg method (a parameterized method for describing the geometric relationships of robot joints and links). The desired joint positions J1 to J6 are then defined.

[0291] Let q1, q2, q3, q4, q5, q6 be the references, and let the corresponding link lengths in the diagram be L1, L2, L3, L4.

[0292] Using an analytical method, homogeneous proofs are used to express the relationship between the spatial position and orientation of point e and the positions of each joint: T 11 =-sin(q4)sin(q6)sin(q2+q3)+cos(q6)cos(q2+q3); T 12 =-sin(q4)cos(q6)sin(q2+q3)-sin(q6)cos(q2+q3); T 13 = -sin(q2+q3)cos(q4); T 14 =-L2sin(q2)-L3sin(q2+q3)-(L4+q5)sin(q2+q3)cos(q4); T 21 = sin(q6)cos(q4); T 22 =cos(q4)cos(q6); T 23= -sin(q4); T 24 = (-L4-q5)sin(q4); T 31 =sin(q4)sin(q6)cos(q2+q3)+sin(q2+q3)cos(q6); T 32 =sin(q4)cos(q6)cos(q2+q3)-sin(q6)sin(q2+q3); T 33 =cos(q4)cos(q2+q3) T 34 =L1+L2cos(q2)+L3cos(q2+q3)+q1+(L4+q5)cos(q4)cos(q2+q3); T 41 =0; T 42 =0; T 43 =0; T 44 =1;

[0293] Given the spatial location and orientation of point e, solve for the positions of each joint.

[0294] The range of motion of the mechanical joints and virtual joints is set in conjunction with the actual mechanical limit settings. For example, in this example, q1 is limited to [0, 20] mm; q2, q3, q4 are limited to [-60, 60] degrees; q5 is limited to [0, 10] mm; q6 is limited to [-180, 180] degrees. deg .

[0295] Solving for q1, we get: q1 = T 34 -L1-L2cos(q2)-(L3+(L4+q5)cos(q4))cos(q2+q3); q2=-arcsin((T 14 +((L4+q5)cos(q4)+L3)sin(q2+q3)) / L2); q3=arctan(sin(q2+q3) / cos(q2+q3)-q2.

[0296] Where sin(q2+q3)=-T 13 / cos(q4); cos(q2+q3)=T 33 / cos(q4) q4=-arcsin(T 23 ); q5 = -T 24 / sin(q4)-L4; q6=arctan(T 21 T 22 );

[0297] The above solution values ​​are then restricted based on the range of motion of the mechanical joints and the virtual joints.

[0298] S703. For joints J1, J2, J3, and J4, calculations are performed according to traditional mechanical mechanism relationships to determine the required operating angle of the motor.

[0299] S704 converts the required operating angle of the motor into the required coded number of the motor and sends it to the motor servo as the control target instruction, and outputs the control angle through the motor servo motion unit.

[0300] S705. For joints J5 and J6, the rotation angle and translation distance of the joints are mapped to the image coordinate system using a mapping method.

[0301] S706. The image pixel processing unit then maps the rotation angle and translation distance to the pixel coordinate system and performs corresponding rotation and translation calculations on the image pixels.

[0302] Example 3

[0303] Corresponding to the aforementioned embodiments of the endoscopic robot control method, this disclosure also provides embodiments of the endoscopic robot control system.

[0304] An endoscopic robot control system, the endoscopic robot including at least one mechanical joint and at least one virtual joint, the at least one virtual joint being implemented by endoscopic robot image scaling and / or image rotation, the endoscopic robot control system comprising:

[0305] The determination module 1 is used to determine the translation distance and / or rotation angle of at least one mechanical joint and the translation distance and / or rotation angle of the at least one virtual joint based on the target coordinate point of the endoscopy robot.

[0306] The position update module 2 is used to control the position update of the endoscopy robot to the target coordinate point based on the translation distance and / or rotation angle of the mechanical joint and the translation distance and / or rotation angle of the virtual joint.

[0307] Specifically, the endoscopic robot includes a first mechanical joint, a second mechanical joint, a third mechanical joint, a fourth mechanical joint, a first virtual joint, and a second virtual joint. The first and second virtual joints are displayed by mapping to an image coordinate system. The initial position of the first virtual joint corresponds to the first image coordinates, and the initial position of the second virtual joint corresponds to the second image coordinates. The translation of the first virtual joint is achieved by scaling the first image coordinates, and the rotation of the second virtual joint is determined by rotating the second image coordinates. See Figure 8. The endoscopic robot control system includes:

[0308] The determination module 1 is used to determine the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the target coordinates of the endoscopy robot.

[0309] The position update module 2 is used to control the position update of the endoscope robot to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle.

[0310] In an optional implementation, the determining module 1 is further configured to determine the optimal imaging distance based on the focal length and image distance; further configured to determine the scaling factor based on the optimal imaging distance and the second translation distance; further configured to determine the third image coordinates corresponding to the first virtual joint after translation based on the scaling factor, the first image coordinates, the fourth rotation angle, and the optimal imaging distance; and further configured to determine the fourth image coordinates corresponding to the second virtual joint after rotation based on the scaling factor, the second image coordinates, the fourth rotation angle, and the optimal imaging distance.

[0311] In an optional implementation, the determining module 1 is further configured to determine the third image coordinates and the fourth image coordinates using the following formula:

[0312] Among them, z c Used to characterize the optimal imaging distance; The x-axis coordinate used to represent the coordinates of the third or fourth image; The y-axis coordinate is used to represent the third or fourth image coordinate; θ is used to represent the fourth rotation angle; k is used to represent the scaling factor; x is used to represent the x-axis coordinate of the first or second image coordinate; y is used to represent the y-axis coordinate of the first or second image coordinate.

[0313] In an optional implementation, the determining module 1 is further configured to determine the optimal imaging distance using the following formula:

[0314] Among them, z c f is used to characterize the optimal imaging distance; f is used to characterize the focal length; d is used to characterize the image distance.

[0315] In an optional implementation, the determining module 1 is further configured to determine the scaling factor using the following formula:

[0316] Where k represents the scaling factor; z c δ is used to characterize the optimal imaging distance; δ is used to characterize the second translation distance.

[0317] In an optional implementation, the determining module 1 is further configured to determine a first matrix based on the target coordinate point, the first matrix being used to characterize the spatial position and orientation of the target coordinate point.

[0318] Referring to Figure 8, the endoscopic robot control system also includes:

[0319] Module 3 is used to acquire the second matrix. The second matrix is ​​used to represent the joint positional relationships between the first mechanical joint, the second mechanical joint, the third mechanical joint, the fourth mechanical joint, the first virtual joint, and the second virtual joint.

[0320] The determining module 1 is also used to determine, based on the first matrix and the second matrix, the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint.

[0321] In an optional implementation, the determining module 1 is further configured to determine the first matrix based on the target coordinate points using the following formula: T 11 =-sin(q4)sin(q6)sin(q2+q3)+cos(q6)cos(q2+q3); T 12 =-sin(q4)cos(q6)sin(q2+q3)-sin(q6)cos(q2+q3); T 13 = -sin(q2+q3)cos(q4); T 14 =-L2sin(q2)-L3sin(q2+q3)-(L4+q5)sin(q2+q3)cos(q4); T 21 = sin(q6)cos(q4); T 22 =cos(q4)cos(q6); T 23 = -sin(q4); T 24 = (-L4-q5)sin(q4); T 31 =sin(q4)sin(q6)cos(q2+q3)+sin(q2+q3)cos(q6); T 32 =sin(q4)cos(q6)cos(q2+q3)-sin(q6)sin(q2+q3); T 33 =cos(q4)cos(q2+q3) T 34 =L1+L2cos(q2)+L3cos(q2+q3)+q1+(L4+q5)cos(q4)cos(q2+q3); T 41 =0; T42 =0; T 43 =0; T 44 =1;

[0322] Where T is used to represent the first matrix, T 11 The formula used to characterize the first row and first column of the first matrix, T 12 The formula used to characterize the first row and second column of the first matrix, T 13 The formula used to characterize the first row and third column of the first matrix, T 14 The formula used to characterize the first row and fourth column of the first matrix, T 21 The formula used to characterize the first column of the second row of the first matrix, T 22 The formula used to characterize the second row and second column of the first matrix, T 23 The formula used to characterize the second row and third column of the first matrix, T 24 The formula used to characterize the second row and fourth column of the first matrix, T 31 The formula used to characterize the first column of the third row of the first matrix, T 32 The formula used to characterize the third row and second column of the first matrix, T 33 The formula used to characterize the third row and third column of the first matrix, T 34 The formula used to characterize the third row and fourth column of the first matrix, T 41 The formula used to characterize the first column of the fourth row of the first matrix, T 42 The formula used to characterize the fourth row and second column of the first matrix, T 43 The formula used to characterize the fourth row and third column of the first matrix, T 44 The formula used to characterize the fourth row and fourth column of the first matrix is ​​as follows: q1 represents the first translation distance of the first mechanical joint, q2 represents the first rotation angle of the second mechanical joint, q3 represents the second rotation angle of the third mechanical joint, q4 represents the third rotation angle of the fourth mechanical joint, q5 represents the second translation distance of the first virtual joint, and q6 represents the fourth rotation angle of the second virtual joint; L1 represents the link length of the first and second mechanical joints, L2 represents the link length of the second and third mechanical joints, L3 represents the link length of the third and fourth mechanical joints, and L4 represents the link length of the first and second virtual joints.

[0323] Module 3 is also used to obtain the second matrix using the following formula:

[0324] Among them, T act Used to characterize the second matrix, T act11 Used to characterize T 11 The value of T act12 Used to characterize T 12The value of T act13 Used to characterize T 13 The value of T act14 Used to characterize T 14 The value of T act21 Used to characterize T 21 The value of T act22 Used to characterize T 22 The value of T act23 Used to characterize T 23 The value of T act24 Used to characterize T 24 The value of T act31 Used to characterize T 31 The value of T act32 Used to characterize T 32 The value of T act33 Used to characterize T 33 The value of T act34 Used to characterize T 34 The value of T act41 Used to characterize T 41 The value of T act42 Used to characterize T 42 The value of T act43 Used to characterize T 43 The value of T act44 Used to characterize T 44 The value;

[0325] Module 1 is further configured to determine the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint using the following formula: q1=T 34 -L1-L2Cos(q2)-(L3+(L4+q5)cos(q4))cos(q2+q3); q2=-arcsin((T 14 +((L4+q5)cos(q4)+L3)sin(q2+q3)) / L2); q3=arctan(sin(q2+q3) / cos(q2+q3)-q2.

[0326] Where sin(q2+q3)=-T 13 / cos(q4); cos(q2+q3)=T 33 / cos(q4) q4=-arcsin(T 23 ); q5 = -T 24 / sin(q4)-L4; q6=arctan(T 21 T 22 );

[0327] Where q1 represents the first translational distance of the first mechanical joint, q2 represents the first rotational angle of the second mechanical joint, q3 represents the second rotational angle of the third mechanical joint, q4 represents the third rotational angle of the fourth mechanical joint, q5 represents the second translational distance of the first virtual joint, q6 represents the fourth rotational angle of the second virtual joint, and sin(q2+q3) and cos(q2+q3) are the first angle variables.

[0328] In an optional implementation, referring to Figure 8, the endoscopic robot control system further includes:

[0329] Mapping module 4 is used to map the second translation distance to the pixel coordinate system.

[0330] The position update module 2 is also used to control the position update of the endoscopy robot based on a second translation distance in the pixel coordinate system.

[0331] In an optional implementation, the mapping module 4 is also used to map the fourth rotation angle to the pixel coordinate system.

[0332] The position update module 2 is also used to control the position update of the endoscopy robot based on a second translation distance in the pixel coordinate system.

[0333] In an optional implementation, the determining module 1 is further configured to determine, based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, and the fourth rotation angle, the first motor operating angle corresponding to the first mechanical joint, the second motor operating angle corresponding to the second mechanical joint, the third motor operating angle corresponding to the third mechanical joint, and the fourth motor operating angle corresponding to the fourth mechanical joint, respectively.

[0334] The position update module 2 is also used to control the position update of the endoscopy robot based on the operating angles of the first motor, the second motor, the third motor, and the fourth motor.

[0335] In an optional implementation, the determining module 1 is further configured to determine the target coordinates of the endoscopic robot according to user instructions.

[0336] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0337] Example 4

[0338] Figure 9 is a schematic diagram of an electronic device according to an example embodiment of this disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the endoscopic robot control method of any of the above embodiments. The electronic device 90 shown in Figure 9 is merely an example and should not impose any limitation on the function and scope of use of the embodiments of this disclosure.

[0339] As shown in Figure 9, the electronic device 90 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including memory 92 and processor 91).

[0340] Bus 93 includes a data bus, an address bus, and a control bus.

[0341] The memory 92 may include volatile memory, such as random access memory (RAM) 921 and / or cache memory 922, and may further include read-only memory (ROM) 923.

[0342] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) program module 924, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0343] The processor 91 executes various functional applications and data processing by running computer programs stored in the memory 92, such as the endoscopic robot control method provided in any of the above embodiments.

[0344] Electronic device 90 can also communicate with one or more external devices 94 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 95. Furthermore, electronic device 90 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 96. As shown, network adapter 96 communicates with other modules of electronic device 90 via bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0345] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0346] Example 5

[0347] This disclosure also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the endoscopic robot control method provided in any of the above embodiments.

[0348] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0349] Example 6

[0350] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the endoscopic robot control method described above.

[0351] The program code for executing the computer program product disclosed herein can be written in any combination of one or more programming languages. The program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0352] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. An endoscopic robot, characterized by, The endoscopic robot includes at least one mechanical joint, a first virtual joint, and a second virtual joint. The first virtual joint is achieved by scaling the endoscopic robot image, and the second virtual joint is achieved by rotating the endoscopic robot image. The endoscopic robot is configured as follows: The translational distance and / or rotational angle of the at least one mechanical joint, the translational distance of the first virtual joint, and the rotational angle of the second virtual joint are determined based on the target coordinates of the endoscopy robot. The position of the endoscopic robot is updated to the target coordinate point based on the translation distance and / or rotation angle of the mechanical joint, the translation distance of the first virtual joint, and the rotation angle of the second virtual joint.

2. The endoscopic robot of claim 1, wherein, The mechanical joints include a first mechanical joint, a second mechanical joint, a third mechanical joint, and a fourth mechanical joint. The endoscopic robot determines the first translational distance of the first mechanical joint, the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint, the third rotation angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotation angle of the second virtual joint based on the target coordinate point of the endoscopic robot. The endoscopic robot is configured as follows: The position of the endoscopic robot is updated to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle.

3. The endoscopic robot of claim 2, wherein, The step of controlling the position update of the endoscopic robot to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle includes: Determine the optimal imaging distance based on focal length and image distance; The scaling factor is determined based on the optimal imaging distance and the second translation distance; The third image coordinates of the first virtual joint after translation are determined based on the scaling factor, the first image coordinates corresponding to the initial position of the first virtual joint, the fourth rotation angle, and the optimal imaging distance; the fourth image coordinates of the second virtual joint after rotation are determined based on the scaling factor, the second image coordinates corresponding to the initial position of the second virtual joint, the fourth rotation angle, and the optimal imaging distance.

4. The endoscopic robot of claim 3, wherein, The steps of determining the third image coordinates of the first virtual joint after translation based on the scaling factor, the first image coordinates corresponding to the initial position of the first virtual joint, the fourth rotation angle, and the optimal imaging distance, and determining the fourth image coordinates of the second virtual joint after rotation based on the scaling factor, the second image coordinates corresponding to the initial position of the second virtual joint, the fourth rotation angle, and the optimal imaging distance, include: The scaling factor is applied to the first image coordinates to obtain the first intermediate coordinates, and the scaling factor is applied to the second image coordinates to obtain the second intermediate coordinates; Based on the fourth rotation angle, the first intermediate coordinates are rotated in two dimensions to obtain the third image coordinates corresponding to the first virtual joint. Based on the fourth rotation angle, the second intermediate coordinates are rotated in two dimensions to obtain the fourth image coordinates corresponding to the second virtual joint.

5. The endoscopic robot of claim 3, wherein, Determining the optimal imaging distance based on focal length and image distance includes: The optimal imaging distance is calculated based on the Gaussian imaging formula, using the focal length and the image distance. And / or, Determining the scaling factor based on the optimal imaging distance and the second translation distance includes: The scaling factor is calculated based on the perspective projection scaling model, using the optimal imaging distance and the second translation distance.

6. The endoscopic robot of claim 2, wherein, The step of determining the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the target coordinates of the endoscopy robot includes: A first matrix is ​​determined based on the target coordinate point, and the first matrix is ​​used to characterize the spatial position and orientation of the target coordinate point. Obtain a second matrix, which is used to characterize the joint positional relationship between the first mechanical joint, the second mechanical joint, the third mechanical joint, the fourth mechanical joint, the first virtual joint, and the second virtual joint; The first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint are determined based on the first matrix and the second matrix.

7. The endoscopic robot of claim 6, wherein, Determining the first matrix based on the target coordinate points includes: Based on the target pose corresponding to the target coordinate point, construct a first rotation matrix; Wherein, the first rotation matrix represents the orientation of the target coordinate point; The first translation vector corresponding to the end of the endoscope robot is determined based on the target coordinate point; The first matrix is ​​obtained based on the first rotation matrix and the first translation vector; The process of obtaining the second matrix includes: Based on the element values ​​in the first matrix, obtain the second matrix corresponding to each element value in the first matrix; The step of determining the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the first matrix and the second matrix includes: Based on the first matrix and the second matrix, the kinematic equations of the endoscopy robot are solved to determine the third rotation angle of the fourth mechanical joint and the fourth rotation angle of the second virtual joint. Based on the third rotation angle of the fourth mechanical joint and the first matrix, the second translation distance of the first virtual joint is determined; The first angle variable is obtained based on the third rotation angle of the fourth mechanical joint and the first matrix; The first rotation angle of the second mechanical joint is determined using the third rotation angle of the fourth mechanical joint, the first angle variable, the second translation distance of the first virtual joint, and the first matrix. Based on the first angle variable and the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint is determined; Based on the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint, the third rotation angle of the fourth mechanical joint, the second translation distance of the first virtual joint, and the first matrix, the first translation distance of the first mechanical joint is determined.

8. A control method of an endoscopic robot, characterized by, The endoscopic robot includes at least one mechanical joint, a first virtual joint, and a second virtual joint. The first virtual joint is achieved by scaling an image of the endoscopic robot, and the second virtual joint is achieved by rotating an image of the endoscopic robot. The control method includes: The translational distance and / or rotational angle of the at least one mechanical joint, the translational distance of the first virtual joint, and the rotational angle of the second virtual joint are determined based on the target coordinates of the endoscopy robot. The position of the endoscopic robot is updated to the target coordinate point based on the translation distance and / or rotation angle of the mechanical joint, the translation distance of the first virtual joint, and the rotation angle of the second virtual joint.

9. The control method of the endoscopic robot according to claim 8, wherein The mechanical joints include a first mechanical joint, a second mechanical joint, a third mechanical joint, and a fourth mechanical joint. The endoscopic robot determines a first translational distance of the first mechanical joint, a first rotational angle of the second mechanical joint, a second rotational angle of the third mechanical joint, a third rotational angle of the fourth mechanical joint, a second translational distance of the first virtual joint, and a fourth rotational angle of the second virtual joint based on the target coordinates of the endoscopic robot. The control method includes: The position of the endoscopic robot is updated to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle. 10.The control method of the endoscopic robot according to claim 9, wherein The step of controlling the position update of the endoscopic robot to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle includes: Determine the optimal imaging distance based on focal length and image distance; The scaling factor is determined based on the optimal imaging distance and the second translation distance; The third image coordinates of the first virtual joint after translation are determined based on the scaling factor, the first image coordinates corresponding to the initial position of the first virtual joint, the fourth rotation angle, and the optimal imaging distance; the fourth image coordinates of the second virtual joint after rotation are determined based on the scaling factor, the second image coordinates corresponding to the initial position of the second virtual joint, the fourth rotation angle, and the optimal imaging distance. 11.The control method of the endoscopic robot according to claim 10, wherein The steps of determining the third image coordinates of the first virtual joint after translation based on the scaling factor, the first image coordinates corresponding to the initial position of the first virtual joint, the fourth rotation angle, and the optimal imaging distance, and determining the fourth image coordinates of the second virtual joint after rotation based on the scaling factor, the second image coordinates corresponding to the initial position of the second virtual joint, the fourth rotation angle, and the optimal imaging distance, include: The scaling factor is applied to the first image coordinates to obtain the first intermediate coordinates, and the scaling factor is applied to the second image coordinates to obtain the second intermediate coordinates; Based on the fourth rotation angle, the first intermediate coordinates are rotated in two dimensions to obtain the third image coordinates corresponding to the first virtual joint. Based on the fourth rotation angle, the second intermediate coordinates are rotated in two dimensions to obtain the fourth image coordinates corresponding to the second virtual joint. 12.The control method of the endoscopic robot according to claim 10, wherein Determining the optimal imaging distance based on focal length and image distance includes: The optimal imaging distance is calculated based on the Gaussian imaging formula, using the focal length and the image distance. And / or, Determining the scaling factor based on the optimal imaging distance and the second translation distance includes: The scaling factor is calculated based on the perspective projection scaling model, using the optimal imaging distance and the second translation distance. 13.The control method of the endoscopic robot of claim 9, wherein, The step of determining the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the target coordinates of the endoscopy robot includes: A first matrix is ​​determined based on the target coordinate point, and the first matrix is ​​used to characterize the spatial position and orientation of the target coordinate point. Obtain a second matrix, which is used to characterize the joint positional relationship between the first mechanical joint, the second mechanical joint, the third mechanical joint, the fourth mechanical joint, the first virtual joint, and the second virtual joint; The first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint are determined based on the first matrix and the second matrix. 14.The control method of the endoscopic robot according to claim 13, wherein Determining the first matrix based on the target coordinate points includes: Based on the target pose corresponding to the target coordinate point, construct a first rotation matrix; Wherein, the first rotation matrix represents the orientation of the target coordinate point; The first translation vector corresponding to the end of the endoscope robot is determined based on the target coordinate point; The first matrix is ​​obtained based on the first rotation matrix and the first translation vector; The process of obtaining the second matrix includes: Based on the element values ​​in the first matrix, obtain the second matrix corresponding to each element value in the first matrix; The step of determining the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the first matrix and the second matrix includes: Based on the first matrix and the second matrix, the kinematic equations of the endoscopy robot are solved to determine the third rotation angle of the fourth mechanical joint and the fourth rotation angle of the second virtual joint. Based on the third rotation angle of the fourth mechanical joint and the first matrix, the second translation distance of the first virtual joint is determined; The first angle variable is obtained based on the third rotation angle of the fourth mechanical joint and the first matrix; The first rotation angle of the second mechanical joint is determined using the third rotation angle of the fourth mechanical joint, the first angle variable, the second translation distance of the first virtual joint, and the first matrix. Based on the first angle variable and the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint is determined; The first translation distance of the first mechanical joint is determined based on the first rotation angle of the second mechanical joint, the second rotation angle of the third mechanical joint, the third rotation angle of the fourth mechanical joint, the second translation distance of the first virtual joint, and the first matrix.

15. An endoscopic robotic control system, comprising: The endoscopic robot includes at least one mechanical joint and at least one virtual joint, wherein the at least one virtual joint is achieved by endoscopic robot image scaling and / or image rotation, and the endoscopic robot control system includes: The determination module is used to determine the translation distance and / or rotation angle of the at least one mechanical joint and the translation distance and / or rotation angle of the at least one virtual joint based on the target coordinate point of the endoscopy robot. The position update module is used to control the position update of the endoscopic robot to the target coordinate point based on the translation distance and / or rotation angle of the mechanical joint and the translation distance and / or rotation angle of the virtual joint.

16. The endoscopic robotic control system of claim 15, wherein, The virtual joint includes a first virtual joint and / or a second virtual joint, wherein the first virtual joint is achieved by scaling the endoscopic robot image and the second virtual joint is achieved by rotating the endoscopic robot image. The determining module is used to determine the translation distance of the first virtual joint and / or the rotation angle of the second virtual joint based on the target coordinate point of the endoscopy robot. The determining module is used to determine the optimal imaging distance based on the focal length and image distance, and to determine the scaling factor based on the optimal imaging distance and the translation distance of the first virtual joint.

17. The endoscopic robotic control system of claim 16, wherein, The mechanical joint includes a first mechanical joint, a second mechanical joint, a third mechanical joint, and a fourth mechanical joint; The determining module is used to determine the first translational distance of the first mechanical joint, the first rotational angle of the second mechanical joint, the second rotational angle of the third mechanical joint, the third rotational angle of the fourth mechanical joint, the second translational distance of the first virtual joint, and the fourth rotational angle of the second virtual joint based on the target coordinate point of the endoscopy robot. The position update module is used to control the position update of the endoscope robot to the target coordinate point based on the first translation distance, the first rotation angle, the second rotation angle, the third rotation angle, the second translation distance, and the fourth rotation angle.

18. An endoscopic robot, characterized by The endoscopic robot includes at least one mechanical joint and at least one virtual joint, the at least one virtual joint being achieved by endoscopic robot image scaling and / or image rotation, and the endoscopic robot further includes an endoscopic robot control system as described in any one of claims 15 to 17.

19. An electronic device comprising a memory, a processor, and a computer program stored on the memory for running on the processor, characterized in that, When the processor executes the computer program, it implements the control method for the endoscopic robot according to any one of claims 8 to 14.

20. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the endoscopic robot control method according to any one of claims 8 to 14.

21. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the endoscopic robot control method as described in any one of claims 8 to 14.

Images