An endoscope adjustment device, method and surgical robotic system

Abstract

The application provides an endoscope adjusting device, a method and a surgical robot system. The endoscope adjusting device comprises: a first pose acquisition module configured to acquire a pose of a target tissue relative to a mechanical arm base based on an image collected by a current endoscope; a pose planning module configured to acquire a target pose of a target mechanical arm; a distal end of the target mechanical arm is mounted with a target endoscope; when the target mechanical arm is in the target pose, the target tissue is located in a field of view of the target endoscope; a second pose acquisition module configured to acquire a current position of a joint of the target mechanical arm; and a motion planning module configured to plan a target motion direction of at least one joint of the target mechanical arm based on the current position of the joint of the target mechanical arm and the target pose of the target mechanical arm. The endoscope adjusting device is applied to a surgical robot system. When a minimally invasive interventional surgery is performed by using the surgical robot system, the switching of two endoscopes can be quickly realized through the operation of the endoscope adjusting device.

Description

Technical Field

[0001] This invention belongs to the field of medical device technology, specifically relating to an endoscope adjustment device, method, and surgical robot system. Background Technology

[0002] In minimally invasive surgery, certain target tissues may be difficult to identify or at least partially obscured by other tissues. In such cases, dual endoscopes are often used during the procedure. Typically, one endoscope is a white light endoscope, and the other is a fluorescence endoscope. The use of the fluorescence endoscope helps in identifying the target tissue.

[0003] However, when using dual endoscopes, switching between endoscopes will result in changes in the field of view. In existing technologies, this requires the operator to manually adjust the position of the robotic arm that mounts the endoscope to adjust the field of view. However, manual adjustments rely primarily on visual inspection and repeated adjustments, resulting in poor accuracy and low efficiency. Furthermore, manual adjustments are prone to collisions between the robotic arm and other robotic arms, or between the endoscope and other instruments, potentially leading to surgical risks. Summary of the Invention

[0004] The purpose of this invention is to provide an endoscope adjustment device, method, and surgical robot system, which aims to enable rapid and accurate adjustment of the field of view when switching endoscopes during surgery using dual endoscopes, reduce the possibility of collisions between devices, and improve surgical safety.

[0005] To achieve the above objectives, the present invention provides an endoscope adjustment device, comprising:

[0006] The first pose acquisition module is used to acquire the position of the target tissue in the coordinate system of the robot arm base based on the image acquired by the current endoscope; the target tissue is located within the field of view of the current endoscope, and the current endoscope is mounted on the end of the current robot arm;

[0007] The pose planning module is used to plan the target pose of the target robotic arm with the target endoscope mounted on it based on the position of the target tissue in the coordinate system of the robotic arm base; when the target robotic arm is in the target pose, the target tissue is located within the field of view of the target endoscope, and the target robotic arm is different from the current robotic arm;

[0008] The second pose acquisition module is used to acquire the current position of the joints of the target robotic arm; and,

[0009] A motion planning module is used to plan the target motion direction of at least one joint of the target robotic arm based on the current position of the joints of the target robotic arm and the target pose of the target robotic arm.

[0010] Optionally, the first pose acquisition module is used for:

[0011] The position of the target tissue in the coordinate system of the image acquisition element of the current endoscope is obtained based on the image acquired by the current endoscope.

[0012] Obtain the position of the image acquisition element of the endoscope in the coordinate system of the robotic arm base;

[0013] Based on the position of the target tissue in the coordinate system of the image acquisition element of the current endoscope and the position of the image acquisition element of the current endoscope in the coordinate system of the robotic arm base, the position of the target tissue in the coordinate system of the robotic arm base is obtained.

[0014] Optionally, the first pose acquisition module is further configured to:

[0015] Perform forward kinematic calibration on the current robotic arm.

[0016] Optionally, the motion planning module is used for:

[0017] The target position of the Nth joint is obtained based on the target pose of the target robotic arm.

[0018] Calculate the difference between the target position and the current position of the Nth joint;

[0019] The difference between the target position and the current position of the Nth joint is used to determine whether the Nth joint is in its corresponding target position; if not, the target movement direction of the Nth joint is obtained based on the difference between the target position and the current position.

[0020] Optionally, the endoscope adjustment device further includes a display control module, the display control module being used for:

[0021] The control unit displays the target's direction of motion.

[0022] Optionally, the endoscope adjustment device further includes a motion control module, which is used to control the corresponding joints of the target robotic arm to move in the target motion direction.

[0023] Optionally, the endoscope adjustment device further includes a prompt information generation module and a display control module;

[0024] The prompt information generation module is used to generate a first prompt information when the Nth joint is located at the corresponding target position;

[0025] The display control module is used to: control a prompting device to display the first prompting information; and / or,

[0026] The prompt information generation module is used to generate a second prompt information when the target robotic arm arrives at the target pose.

[0027] The display control module is used to: control a display device to display the second prompt information.

[0028] Optionally, the target motion direction is the rotational or translational direction of the joint.

[0029] To achieve the above objectives, the present invention also provides an endoscope adjustment method, comprising:

[0030] The position of the target tissue in the coordinate system of the robotic arm base is obtained based on the image acquired by the current endoscope; the target tissue is located within the field of view of the current endoscope, which is mounted on the end of the current robotic arm;

[0031] Based on the position of the target tissue in the coordinate system of the robotic arm base, the target pose of the target robotic arm with the target endoscope mounted is planned; when the target robotic arm is in the target pose, the target tissue is located within the field of view of the target endoscope, and the target robotic arm is different from the current robotic arm;

[0032] Obtain the current position of the joints of the target robotic arm, and plan the target motion direction of at least one joint of the target robotic arm based on the current position of the joints of the target robotic arm and the target pose of the target robotic arm.

[0033] To achieve the above objectives, the present invention also provides a surgical robot system, comprising:

[0034] The surgical operating device includes a first robotic arm and a second robotic arm;

[0035] An endoscope assembly for acquiring images of target tissue includes a first endoscope and a second endoscope, the first endoscope being mounted on the end of a first robotic arm, and the second endoscope being mounted on the end of a second robotic arm; one of the first endoscope and the second endoscope is a current endoscope, and the other is a target endoscope; the robotic arm mounting the current endoscope is the current robotic arm, and the robotic arm mounting the target endoscope is the target robotic arm; and...

[0036] As described in any of the preceding claims, the endoscope adjustment device is communicatively connected to the surgical operating device and the endoscope assembly.

[0037] Compared with the prior art, the endoscopic adjustment device, method, and surgical robot system of the present invention have the following advantages:

[0038] The aforementioned endoscope adjustment device includes: a first pose acquisition module, a pose planning module, a second pose acquisition module, and a motion planning module. The first pose acquisition module acquires the position of the target tissue in the coordinate system of the robotic arm base based on an image acquired by the current endoscope. The target tissue is located within the field of view of the current endoscope, which is mounted on the end of the current robotic arm. The pose planning module acquires the target pose of the target robotic arm with the target endoscope mounted on it based on the position of the target tissue in the coordinate system of the robotic arm base. When the target robotic arm is in the target pose, the target tissue is located within the field of view of the target endoscope, and the target robotic arm is different from the current robotic arm. The second pose acquisition module acquires the current position of the joints of the target robotic arm. The motion planning module plans the target motion direction of at least one joint of the target robotic arm based on the current position of the target endoscope and the target pose of the target robotic arm. The endoscope adjustment device is applied to a surgical robot system. Thus, when performing minimally invasive interventional surgery using the surgical robot system, the two endoscopes can be switched quickly through the operation of the endoscope adjustment device, and the collision of devices during endoscope switching can be reduced or even avoided. Attached Figure Description

[0039] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0040] Figure 1 This is a schematic diagram illustrating an application scenario of the surgical robot system provided by the present invention according to an embodiment;

[0041] Figure 2 This invention provides a schematic diagram of a partial application scenario of a surgical robot system according to one embodiment, where an endoscope is not shown.

[0042] Figure 3 This is a schematic diagram of a partial application scenario of the surgical robot system provided by the present invention according to an embodiment. The diagram shows two endoscopes, and;

[0043] Figure 4 This is a schematic diagram of the surgical operation device of the surgical robot system provided according to an embodiment of the present invention;

[0044] Figure 5This is a schematic diagram of the endoscope of the surgical robot system provided by the present invention according to an embodiment;

[0045] Figure 6 This is a schematic diagram of the application scenario of the surgical robot system provided by the present invention according to an embodiment. The diagram shows a schematic diagram when switching endoscopes. The current position of the second endoscope is drawn with a solid line, and the position of the second endoscope after the pose adjustment is drawn with a dashed line.

[0046] Figure 7 This is a partial flowchart of the endoscope adjustment method performed by the endoscope adjustment device when the surgical robot system provided by the present invention is applied according to an embodiment;

[0047] Figure 8 This is a schematic diagram illustrating the mathematical modeling principle of a binocular vision device;

[0048] Figure 9 This is a schematic diagram illustrating the positioning principle of a binocular vision device;

[0049] Figure 10 This is a schematic diagram of the robotic arm of a surgical robot system provided according to an embodiment of the present invention;

[0050] Figure 11 This is a schematic diagram of the robotic arm of a surgical robot system according to an embodiment of the present invention. Figure 11 The movement patterns of the various joints of the robotic arm are schematically indicated by arrows.

[0051] Figure 12 This is a schematic diagram of the adjusting arm of the robotic arm of the surgical robot system provided according to an embodiment of the present invention;

[0052] Figure 13 This is a schematic diagram of the tool arm positioning of the robotic arm of the surgical robot system provided according to an embodiment of the present invention;

[0053] Figure 14 This is a schematic diagram of the movement direction of the first joint of the robotic arm of the surgical robot system provided according to an embodiment of the present invention. In the diagram, "+" represents the positive direction and "-" represents the negative direction.

[0054] Figure 15 This is a partial flowchart of step S30 of the endoscopic method performed by the endoscope adjustment device when the surgical robot system provided by the present invention is applied according to an embodiment;

[0055] Figure 16This is a partial flowchart of the surgical robot system provided by the present invention during the switching of endoscopic field of view during the operation. The diagram shows that the target robotic arm is automatically controlled by the endoscope adjustment device, and the target movement direction information, first prompt information and second prompt information are prompted by the voice mechanism.

[0056] Figure 17 This is a partial flowchart of the surgical robot system provided by the present invention during the switching of endoscopic field of view during the operation, according to an embodiment of the present invention. The target robotic arm is manually driven by the operator in the figure.

[0057] Figure 18 This is a partial flowchart of the surgical robot system provided by the present invention during the switching of endoscopic field of view during the operation. The diagram shows that the operator controls the movement of the target robotic arm by operating the main hand.

[0058] Figure 19 This is a partial structural diagram of a surgical robot system provided according to an embodiment of the present invention, wherein the prompting device in the diagram is a first light prompting mechanism;

[0059] Figure 20 yes Figure 19 A magnified schematic diagram of point A of the surgical robot system shown;

[0060] Figure 21 This is a partial flowchart of the surgical robot system provided by the present invention during the endoscopic field of view switching during the operation. The diagram shows that the target robotic arm is automatically controlled by the endoscope adjustment device, and the target movement direction, first prompt information and second prompt information are indicated by the light changes of the first light prompting mechanism.

[0061] Figure 22 This is a partial structural diagram of a surgical robot system provided according to an embodiment of the present invention, wherein the prompting device in the diagram is a second light prompting mechanism;

[0062] Figure 23 yes Figure 22 Enlarged schematic diagram of section B of the surgical robot system shown.

[0063] Figure 24 This is a partial structural diagram of a surgical robot system according to an embodiment of the present invention, wherein the prompting device in the diagram is a display screen. Detailed Implementation

[0064] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in this embodiment are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show components related to the present invention and are not drawn according to the actual number, shape, and size of components in the actual implementation. In the actual implementation, the type, quantity, and proportion of each component can be arbitrarily changed, and the component layout may also be more complex.

[0065] Furthermore, while each embodiment described below possesses one or more technical features, this does not imply that users of the present invention must simultaneously implement all technical features in any embodiment, or can only separately implement some or all technical features in different embodiments. In other words, provided it is feasible, those skilled in the art can, based on the disclosure of the present invention and depending on design specifications or implementation requirements, selectively implement some or all technical features in any embodiment, or selectively implement a combination of some or all technical features in multiple embodiments, thereby increasing the flexibility in implementing the present invention.

[0066] To make the objectives, advantages, and features of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clearly illustrate the objectives of the embodiments of the present invention. The same or similar reference numerals in the drawings represent the same or similar parts.

[0067] Figures 1 to 3 This diagram illustrates an application scenario of a surgical robot system provided by an embodiment of the present invention. For example... Figures 1 to 3 As shown, the surgical robot system includes a control end and an execution end. The control end includes a doctor's console and a doctor's control device 10 mounted on the doctor's console. The doctor's control device 10 is equipped with an operating hand (as shown in the figure). The execution end includes a patient-side control device (not labeled in the figure), a surgical operation device 20, an image display device 30, and an endoscope assembly (not labeled in the figure). The endoscope assembly is used to enter the body of the target object 1 and acquire images of the target tissue 2 located within the target object 1. The endoscope assembly includes a first endoscope 41 and a second endoscope 42. The image display device 30 is communicatively connected to the endoscope assembly and is used to receive and display the images of the target tissue 2 acquired by the endoscope assembly.

[0068] As in 2 to Figure 4As shown, the surgical device includes an operating platform 21 and multiple robotic arms 22. These robotic arms 22 are referred to as a first robotic arm 22a, a second robotic arm 22b, and a third robotic arm (not labeled in the figure). The first robotic arm 22a is used to mount the first endoscope 41, the second robotic arm 22b is used to mount the second endoscope 42, and there is at least one third robotic arm 22c. Each third robotic arm is used to mount a surgical instrument (not shown in the figure), which is used to enter the interior of the target object 1 to perform surgical operations on the target tissue 2. Here, the target object 1 is, for example, a patient, or in a simulated surgery, a human model, and the guiding device is, for example, a puncture card.

[0069] Usually, such as Figure 5 As shown, the endoscope includes a light source interface 40a, a stylus 40b, and an image acquisition element 40c. The light source interface 40a is located at the proximal end of the stylus 40b and is used to connect to a light source (not shown). The image acquisition element 40c is located at the distal end of the stylus 40b and acquires images of the target tissue 2 based on the illumination light generated by the light source.

[0070] During operation, the light source interface 40a of the first endoscope 41 is connected to a first light source, and the light source interface 40a of the second endoscope 42 is connected to a second light source. The second illumination light generated by the second light source is different from the first illumination light generated by the first light source. In an exemplary embodiment, one of the first and second light sources is a white light source and the other is a fluorescent light source; therefore, one of the first and second illumination lights is white light and the other is fluorescent. Thus, the endoscope connected to the white light source is a white light endoscope, and the endoscope connected to the fluorescent light source is a fluorescent endoscope. When performing surgery using the surgical robot system, the combined use of the first endoscope 41 and the second endoscope 42 can effectively achieve the effect of target tissue identification or enhance the visual salience of the target tissue.

[0071] During the procedure, the operator alternately uses the first endoscope 41 and the second endoscope 42 to observe the target tissue 2. That is, the operator will switch endoscopes as needed during the procedure. It can be understood that if the operator is currently using the first endoscope 41 to observe the target tissue 2, then at that moment, the target tissue 2 must be within the field of view of the first endoscope 41, but it may not be within the field of view of the second endoscope 42. Regarding the situation where the target tissue 2 is not currently within the field of view of the second endoscope 42 (e.g....), Figure 6In the case of the second endoscope 42 (shown as a solid line), if the operator needs to switch to using the second endoscope 42 to observe the target tissue 2, the second robotic arm 22b needs to be moved to adjust the position of the second endoscope 42 so that the second endoscope 42 moves until its field of view covers the target tissue 2 (e.g., as shown in the diagram). Figure 6 The second endoscope 42 (shown as a dashed line) is designed to ensure that the target tissue 2 appears within the field of view of the second endoscope 42.

[0072] Furthermore, the surgical robot system also includes an endoscope adjustment device, which is communicatively connected to the surgical operating device and the endoscope assembly, and is configured to receive images acquired by the endoscope assembly and execute an endoscope adjustment method. The purpose of this endoscope adjustment method is to quickly and accurately adjust the pose of the second robotic arm 22b so that the field of view of the second endoscope 42 covers the target tissue.

[0073] The specific configuration of the endoscope adjustment device in this embodiment of the invention is not limited, as long as it can effectively execute the robotic arm posture adjustment method. Optionally, the endoscope adjustment device is mounted on the doctor's end control device, or the endoscope adjustment device is mounted on the patient's end control device, or a portion of the endoscope adjustment device is mounted on the doctor's end control device and another portion is mounted on the patient's end control device, or at least a portion of the endoscope adjustment device is independent of the doctor's end control device and the patient's end control device.

[0074] The following description uses the example of the first endoscope 41 being the currently used endoscope and the second endoscope 42 being the target endoscope to be switched to to illustrate the robotic arm pose adjustment method. It can be understood that when the second endoscope 42 is the target endoscope to be switched to, the second robotic arm 22b is the target robotic arm requiring pose adjustment, and the first robotic arm 22a is the current robotic arm that mounts the current endoscope. Furthermore, those skilled in the art can make adaptive modifications to the following description to suit situations where the currently used endoscope is the second endoscope 42, the second robotic arm 22b is the current robotic arm, the target endoscope to be switched to is the first endoscope 41, and the target robotic arm requiring pose adjustment is the first robotic arm 22a.

[0075] Figure 7 The overall flowchart of the described endoscope adjustment method is shown. Please refer to it. Figure 7 The endoscope adjustment method includes:

[0076] Step S10: Obtain the position of the target tissue 2 relative to the robotic arm base based on the image acquired by the current endoscope, i.e., the first endoscope 41. It can be understood that the image acquired by the first endoscope 41 displays the image of the target tissue 2.

[0077] Step S20: Obtain the target pose of the target robotic arm, i.e., the second robotic arm 22b, based on the pose of the target tissue 2 relative to the robotic arm base. The target pose is the corresponding pose of the second robotic arm 22b when the target tissue 2 is within the field of view of the second endoscope 42.

[0078] Step S30: Obtain the current position of the joints of the second robotic arm 22b, and plan the target motion direction of at least one joint of the second robotic arm 22b based on the current position of the joints of the second robotic arm 22b and the target pose of the second robotic arm 22b.

[0079] Subsequently, the second robotic arm 22b can be controlled to move in any suitable manner according to the target movement direction of each joint, so that the second robotic arm 22b moves to the target pose, thereby driving the second endoscope 42 to move until its field of view covers the target tissue 2. In practice, the endoscope adjustment device can automatically control the movement of the second robotic arm 22b according to the target movement direction of each joint (e.g., Figure 16 and Figure 21 (as shown), or the second robotic arm 22b can be manually controlled by the operator according to the target movement direction of each joint (as shown). Figure 17 (as shown), or the operator controls the movement of the second robotic arm 22b by manipulating the main hand according to the target movement direction of each joint (as shown). Figure 18 (As shown).

[0080] It should be noted that any moment during the execution of the robotic arm pose adjustment method is considered the current moment.

[0081] Based on this, the endoscopic adjustment device includes at least a first pose acquisition module, a pose planning module, a second pose acquisition module, and a motion planning module. The first pose acquisition module is communicatively connected to the endoscope assembly, the pose planning module is communicatively connected to the first pose acquisition module, and the motion planning module is communicatively connected to both the pose planning module and the second pose acquisition module. The first pose acquisition module is used to execute step S10, the pose planning module is used to execute step S20, and the second pose acquisition module and the motion planning module jointly execute step S30. Furthermore, "pose" includes both position and orientation.

[0082] The following will provide further explanation of steps S10, S20, and S30.

[0083] The specific process of step S10 above includes steps S11, S12 and S13.

[0084] Step S11 includes obtaining the position of the target tissue 2 in the coordinate system of the image acquisition element 40c of the first endoscope 41 based on the image acquired by the first endoscope 41. Those skilled in the art will know that the image acquisition element 40c of the endoscope is a binocular vision device, including two cameras. Therefore, the specific operation of step S11 is to first segment the image acquired by the first endoscope 41 using any suitable image segmentation method to extract the image of the target tissue 2. Then, based on the positioning principle of the binocular vision device, the position of the target tissue 2 in the coordinate system of the image acquisition element 40c of the first endoscope 41 is obtained.

[0085] The binocular vision device uses two cameras to simultaneously acquire two digital images of the object under test from different angles, and recovers the three-dimensional geometric information of the object under test based on the principle of parallax to obtain the position of the object under test. Figure 8 A schematic diagram illustrating the mathematical model of a binocular vision device is shown. Figure 9 A schematic diagram illustrating the positioning principle of a binocular vision device is provided. In this paper, the two cameras of the binocular vision device are referred to as the left camera and the right camera. Figure 8 and Figure 9 As shown, point P((x, y, z) is a feature point on the object being measured, and the coordinate system F l The coordinate system of the left camera is O. l The optical center of the left camera is given by coordinate system F. R The coordinate system of the right camera is O. R It is the optical center of the right camera. If point P is observed using the left camera, the image of point P in the image plane O1 of the left camera is point P. L But we cannot be controlled by P l Knowing the three-dimensional position of P, in fact, in O l P l Any point on the image of the left camera connected by the line is P. l Therefore, by P l The location is known only by the fact that point P lies on line O. l P l Above. Similarly, from the perspective of the right camera, we can only know that point P is located at execution O. r P rAbove. Therefore, when the left camera and the right camera simultaneously capture the same feature point P((x, y, z) of the object being measured, the straight line O... l P l With line O r P r The intersection of these points is the position of point P in space.

[0086] The optical center O of the left camera L With the right camera O R The distance between the optical centers is the baseline b, and the focal lengths of both the left and right cameras are f. The two cameras simultaneously capture feature points P(x, y, z) of the object being measured, and based on the principle of similar triangles, the following relationship is obtained:

[0087]

[0088] Therefore, we get:

[0089]

[0090] Therefore, the coordinates of feature point P on the object under test in the coordinate system of the binocular vision device can be obtained. Similarly, the coordinates of other feature points on the object under test in the coordinate system of the binocular vision device can be obtained, thereby obtaining the position of the object under test in the binocular vision device. In this embodiment of the invention, the position of the target tissue 2 in the coordinate system of the image acquisition element 40c of the first endoscope 41 is obtained.

[0091] Step S12 includes acquiring the position of the image acquisition element 40c of the first endoscope 41 in the coordinate system of the robotic arm base. It is known that the first endoscope 41 is mounted on the end of the first robotic arm 22a, and the image acquisition element 40c is located at the end of the first endoscope 41. Therefore, based on the design parameters of the first robotic arm 22a, the current joint angles of each joint of the first robotic arm 22a, and the design parameters of the first endoscope 41, the position of the image acquisition element 40c of the first endoscope 41 in the coordinate system of the robotic arm base can be obtained. Preferably, while performing step S12, the first pose acquisition module also performs forward kinematic calibration (Denavit-Hartenberg calibration, DH calibration) on the first robotic arm 22a to improve the accuracy of the acquired position of the image acquisition element 40c of the first endoscope 41 in the coordinate system of the robotic arm base.

[0092] Step S13 includes obtaining the position of the target tissue 2 in the coordinate system of the robotic arm base based on the position of the target tissue 2 in the coordinate system of the image acquisition element 40c of the first endoscope 41 and the position of the image acquisition element 40c of the first endoscope 41 in the coordinate system of the robotic arm base.

[0093] Step S20 can be implemented by any suitable method in the prior art, as long as the target pose of the second robotic arm 22b can be obtained when the target tissue 2 is within the field of view of the second endoscope 42. However, it should be understood that when planning the target pose of the second robotic arm 22b, it should be considered that the target pose of the second robotic arm 22b should not interfere with other robotic arms or devices, and that the second robotic arm 22b should not collide with other robotic arms or devices when it moves from the current pose to the target pose.

[0094] The robotic arm 22 may include multiple joints, to Figures 10 to 13As shown in the example, the robotic arm 22 includes seven joints connected in sequence. In some non-limiting embodiments, the robotic arm includes an adjusting arm 221 and a tool arm 222 connected to each other. The adjusting arm 221 is connected to the operating platform 21 and includes four joints, namely a first joint 221a, a second joint 221b, a third joint 221c, and a fourth joint 221d connected in sequence. The tool arm 222 includes three joints, namely a fifth joint 222a, a sixth joint 222b, and a seventh joint 222c connected in sequence. The first joint 221a, the third joint 221c, the fourth joint 221d, the fifth joint 222a, and the sixth joint 222b are rotary joints, while the second joint 221b and the seventh joint 222c are translational joints. For the second robotic arm 22b, its pose adjustment includes adjusting the positions of up to seven joints, namely: rotating the first joint 221a from its current position along a positive or negative direction until reaching the corresponding target position; translating the second joint 221b from its current position along a positive or negative direction until reaching the corresponding target position; rotating the third joint 221c from its current position along a positive or negative direction until reaching the corresponding target position; rotating the fourth joint 221d from its current position along a positive or negative direction until reaching the corresponding target position; rotating the fifth joint 222a from its current position along a positive or negative direction until reaching the corresponding target position; rotating the sixth joint 222b from its current position along a positive or negative direction until reaching the corresponding target position; and translating the seventh joint 222c from its current position along a positive or negative direction until reaching the corresponding target position. Here, for the rotary joints, one of the positive and negative directions is clockwise and the other is counterclockwise. For example, the positive direction is counterclockwise and the negative direction is clockwise (e.g., ...). Figure 14 As shown in the diagram, for translational joints, the positive direction is, for example, the direction closer to target object 1, and the negative direction is the direction farther away from target object 1. It should be understood that each joint, during its movement from its current position to the corresponding target position, should move within permissible limits to ensure safety.

[0095] like Figure 15 As shown, step S30 includes at least steps S31, S32, S33 and S34.

[0096] Step S31 includes acquiring the current position of a specified joint of the second robotic arm 22b, such as the Nth joint, where N is a positive integer greater than or equal to 1. Specifically, the second pose acquisition module receives position data monitored by a position sensor installed at the joint, and then acquires the current position of the corresponding joint based on the position data. For rotary joints, the position sensor can be an angle encoder or other type of angle sensor; for translational joints, the position sensor can be a distance sensor. It is understood that the second pose acquisition module is communicatively connected to the position sensor.

[0097] Step S32 includes obtaining the target positions of each joint of the second robotic arm 22b based on its target pose. The target pose of the second robotic arm 22b is determined by the target positions of its joints. Conversely, given the target pose of the second robotic arm 22b, the target positions of its joints can be deduced from its configuration. In practice, step S32 can be performed by using a robot inverse kinematics algorithm to calculate the target position of each joint of the second robotic arm 22b.

[0098] Step S33 includes calculating the difference between the target position and the current position of the Nth joint.

[0099] Step S34 includes determining whether the Nth joint is at the target position based on the difference between the target position and the current position. If not, step S35 is further executed, which includes obtaining the target movement direction of the Nth joint based on the difference between the target position and the current position.

[0100] In step S34, if the difference between the target position and the current position of the Nth joint is within a preset range, then the Nth joint is determined to be at the target position. If the difference between the target position and the current position of the Nth joint is not within a predetermined range, then the Nth joint is determined not to be at the target pose.

[0101] If the difference between the target position and the current position of the Nth joint is positive, then the target movement direction of the Nth joint obtained in step S35 is positive; conversely, if the difference is negative, the target movement direction of the Nth joint obtained in step S35 is negative. For example, when the Nth joint is a rotary joint, the target position of the Nth joint is θ. goal The current position is θ0, if θ goalWhen -θ0 is positive, the target motion direction of the Nth joint is along the positive direction; conversely, when θgoal-θ0 is negative, the target motion direction of the Nth joint is along the negative direction. When the Nth joint is a translational joint, and the Nth joint translates in the Y direction (with... Figure 11 (As shown in the example), the target position of the Nth joint is Ng. oal (0, y goal Given that the current position is N0(0, y0, 0), the difference between the two can be directly expressed as y. goal -y0, if y goal If -y0 is a positive value, then the target motion direction during the translation of the Nth joint is the positive direction. goal If -y0 is negative, then the target motion direction during the translation of the Nth joint is the reverse direction.

[0102] It is understood that steps S32 to S35 are executed by the motion planning module, and the motion planning module executes step S32 once, and executes steps S33 and S34 at least for each joint.

[0103] In some embodiments, the endoscope adjustment device further includes a motion control module, which is communicatively connected to the motion planning module and the robot. The endoscope adjustment method further includes step S50 executed by the motion control module, wherein step S50 includes controlling each joint of the second robotic arm 22b to move according to a corresponding target motion direction. That is, as... Figure 16 and Figure 21 As shown, after obtaining the target motion direction of the Nth joint, the motion control module automatically controls the Nth joint to move in accordance with its target motion direction.

[0104] In other embodiments, such as Figure 17 As described above, after obtaining the target motion direction of the Nth joint of the second robotic arm 22b, the operator manually controls the Nth joint to move according to its target motion direction, or, as... Figure 18 As shown, the operator controls the Nth joint to move in its target direction by manipulating the main operating hand.

[0105] It is understood that for the joint that has performed step S35, during the movement of the Nth joint of the second robotic arm 22b, steps S33 and S34 are repeated until the corresponding joint reaches the corresponding target position.

[0106] Furthermore, the surgical robot system also includes a display device, and correspondingly, the endoscope adjustment device also includes a display control module, which is communicatively connected to the display device. For example... Figure 15 As shown, the endoscopic adjustment method further includes step S60 executed by the display control module, which includes controlling the display device to display the target movement direction of the Nth joint. Therefore, when the motion control module automatically controls the Nth joint to move in the target movement direction, the operator can confirm whether the movement of the Nth joint is correct through the display on the display device. When the operator manually controls the movement of the Nth joint or by manipulating the main operating hand, the operator can visually see the specific direction of the target movement through the display on the display device.

[0107] Furthermore, the endoscope adjustment device also includes a prompt information generation module, which is communicatively connected to the motion planning module and the display device. The robotic arm pose adjustment method further includes step S70, executed by the prompt information generation module, which generates a first prompt information when the Nth joint is in the target position. Subsequently, the display control module executes step S80, which controls the display device to display the first prompt information.

[0108] Furthermore, the endoscope adjustment method also includes step S01 executed by the prompt information generation module and step S02 executed by the display control module. Step S01 includes generating confirmation information. Step S02 includes controlling the display device to display the confirmation information to prompt the operator to confirm the current Nth joint. It can be understood that when adjusting the pose of the second robotic arm 22b, the adjustment operation is usually performed starting from the first joint 221a and sequentially on all joints according to the connection order. Furthermore, the endoscopic adjustment method further includes step S36, executed by the motion planning module after the position adjustment of each joint is completed. Step S36 includes determining whether all joints of the second robotic arm 22b are in their respective target positions (i.e., determining whether the second robotic arm 22b has reached the target pose). If not, N is updated, and steps S01, S02, and S42 are executed for the updated Nth joint. If yes, it is determined that the second robotic arm 2b has reached the target pose, and steps S90 and S100 are executed. Step S90 includes the prompt information generation module generating a second prompt information. Step S100 includes the display control module controlling the display device to display the second prompt information.

[0109] Therefore, please refer to Figures 16 to 18 and Figure 21 As shown, during the surgical procedure performed using the surgical robot, a specific process in which the operator switches from the currently used first endoscope 41 to the second endoscope 42 is as follows:

[0110] First, steps S10, S20, and S32 are executed, followed by steps S01 and S02. After the operator confirms that the joint to be adjusted is the Nth joint, steps S31, S33, and S34 are executed for the Nth joint. If the result of step S34 is "yes," steps S70, S80, and S36 are executed. If the result of step S36 is "yes," steps S90 and S100 are executed. If the result of step S36 is "no," N is updated, and based on the updated N, step S01 and subsequent steps are returned to be executed. If the result of step S34 is "no," step S35 is executed for the Nth joint, followed by steps S60 and S50. Furthermore, during the execution of step S50 on the Nth joint, step S31 and subsequent steps are repeatedly executed until the Nth joint reaches its target position, that is, the judgment result of step S34 is "yes", and then each step after step S34 is executed.

[0111] As previously mentioned, step S50 can be executed by the motion control module of the endoscope adjustment device, or manually controlled by the operator (e.g., Figure 17 (as shown) or by manipulating the main operating hand to control the Nth joint (e.g. Figure 18 (As shown) it moves in its target direction of motion. All other steps are performed by the endoscope adjustment device.

[0112] In an optional embodiment, the display device is a voice device, which can prompt the operator to confirm the current Nth joint through voice broadcast, and can also display the first prompt information and the second prompt information through voice broadcast. That is, steps S02, S60, S80, and S100 are executed by the voice device.

[0113] In an alternative embodiment, such as Figure 19 and Figure 20As shown, the display device is a first light indicator mechanism, which includes indicator light groups 50, and the number of indicator light groups 50 is at least two. One indicator light group 50 is provided on the first robotic arm 22a, and another indicator light group 50 is provided on the second robotic arm 22b. When the position of the first robotic arm 22a is adjusted, the indicator light group 50 on the first robotic arm 22a is activated; when the position of the second robotic arm 22b is adjusted, the indicator light group 50 on the second robotic arm 22b is activated. Each indicator light group 50 may include a first indicator light 51 and a second indicator light 52. Both the first indicator light 51 and the second indicator light 52 have states such as flashing with different colors, constant on, and constant off. Figure 21 As shown, those skilled in the art can configure the states of the first indicator light 51 and the second indicator light 52 to display corresponding information. For example, by controlling the first indicator light 51 and the second indicator light 52 to flash with corresponding colors, the operator can be prompted to confirm the Nth joint to be adjusted. For instance, controlling both the first indicator light 51 and the second indicator light 52 to flash with a first color can prompt the operator to confirm that the joint to be adjusted is the first joint 221a, and controlling them to flash with a second color different from the first color can prompt the operator to confirm that the joint to be adjusted is the second joint 221b, and so on. The target movement direction can be displayed by controlling one of the first indicator light 51 and the second indicator light 52 to be constantly on and the other to be constantly off. For instance, controlling the first indicator light 51 to be constantly on and the second indicator light 52 to be constantly off indicates a positive target movement direction, and controlling the first indicator light 51 to be constantly off and the second indicator light 52 to be constantly on indicates a negative target movement direction. The first and second prompt messages can be displayed by controlling both the first indicator light 51 and the second indicator light 52 to be constantly off.

[0114] Alternatively, please refer to Figure 22 and Figure 23The display device is a second light indication mechanism, which includes at least two third indicator lights 60. One third indicator light 60 is located on the first robotic arm 22a, and another on the second robotic arm 22b. When the position of the first robotic arm 22a is adjusted, the third indicator light 60 on the first robotic arm 22a is activated; conversely, when the position of the second robotic arm 22b is adjusted, the third indicator light 60 on the second robotic arm 22b is activated. The third indicator light 60 has multiple states, such as flashing with different colors, remaining constantly lit with different colors, or remaining constantly off. Those skilled in the art can also set the states of the third indicator light 60 to display different information. For example, controlling the third indicator light 60 to flash with different colors can prompt the operator to confirm the Nth joint to be adjusted; controlling the third indicator light 60 to remain constantly lit with different colors can indicate the target display direction; and controlling the third indicator light 60 to remain constantly off can indicate the first and second prompt information.

[0115] In another alternative embodiment, such as Figure 24 As shown, the display device includes a display screen 70, and the number of display screens 70 is at least two. One display screen 70 is mounted on the first robotic arm 22a, and another display screen 70 is mounted on the second robotic arm 22b. When the pose of the first robotic arm 22a is adjusted, the display screen 70 on the first robotic arm 22a is active; when the pose of the second robotic arm 22b is adjusted, the display screen 70 on the second robotic arm 22b is active. Each display screen 70 has indicator icons, including a first arrow 71 pointing in the positive direction, a second arrow 72 pointing in the negative direction, and a confirmation symbol 73, such as a "√". The first arrow 71 and the second arrow 72 have constantly lit and constantly off states, and the confirmation symbol 73 can flash in different colors or remain constantly lit or constantly off. During operation, the confirmation symbol is controlled to flash in a corresponding color to prompt the operator to confirm the Nth joint currently awaiting confirmation; the target movement direction is indicated by keeping one of the first arrow 71 and the second arrow 72 constantly lit, for example, keeping the first arrow 71 constantly lit indicates that the target movement direction is positive, and keeping the second arrow 72 constantly lit indicates that the target movement direction is negative; the first prompt information and the second prompt information are displayed by keeping the confirmation symbol 73 constantly lit.

[0116] Furthermore, embodiments of the present invention also provide a computer-readable storage medium having a program stored thereon, which, when executed, performs the aforementioned endoscope adjustment method.

[0117] While the present invention has been disclosed above, it is not limited thereto. Those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims and their equivalents, the present invention also intends to include such modifications and variations.

Claims

1. An endoscope adjustment device, characterized in that, The endoscope adjustment device is applied to a surgical robot system, which includes a first endoscope and a second endoscope, wherein one of the first endoscope and the second endoscope is a white light endoscope and the other is a fluorescence endoscope. The endoscope adjustment device includes: The first pose acquisition module is used to acquire the position of the target tissue in the coordinate system of the robot arm base based on the image acquired by the current endoscope; the target tissue is located within the field of view of the current endoscope, and the current endoscope is mounted on the end of the current robot arm; The pose planning module is used to plan the target pose of the target robotic arm with the target endoscope mounted on it based on the position of the target tissue in the coordinate system of the robotic arm base; when the target robotic arm is in the target pose, the target tissue is located within the field of view of the target endoscope, and the target robotic arm is different from the current robotic arm; The second pose acquisition module is used to acquire the current position of the joints of the target robotic arm; and, A motion planning module is used to plan the target motion direction of at least one joint of the target robotic arm based on the current position of the joints of the target robotic arm and the target pose of the target robotic arm. The current endoscope is one of the first endoscope and the second endoscope, and the target endoscope is the other of the first endoscope and the second endoscope.

2. The endoscope adjustment device according to claim 1, characterized in that, The first pose acquisition module is used for: The position of the target tissue in the coordinate system of the image acquisition element of the current endoscope is obtained based on the image acquired by the current endoscope. Obtain the position of the image acquisition element of the endoscope in the coordinate system of the robotic arm base; Based on the position of the target tissue in the coordinate system of the image acquisition element of the current endoscope and the position of the image acquisition element of the current endoscope in the coordinate system of the robotic arm base, the position of the target tissue in the coordinate system of the robotic arm base is obtained.

3. The endoscope adjustment device according to claim 2, characterized in that, The first pose acquisition module is also used for: Perform forward kinematic calibration on the current robotic arm.

4. The endoscope adjustment device according to claim 1, characterized in that, The motion planning module is used for: The target position of the Nth joint is obtained based on the target pose of the target robotic arm. Calculate the difference between the target position and the current position of the Nth joint; The difference between the target position and the current position of the Nth joint is used to determine whether the Nth joint is in its corresponding target position; if not, the target movement direction of the Nth joint is obtained based on the difference between the target position and the current position.

5. The endoscope adjustment device according to any one of claims 1-4, characterized in that, The endoscope adjustment device further includes a display control module, which is used for: The control unit displays the target's direction of motion.

6. The endoscope adjustment device according to any one of claims 1-4, characterized in that, The endoscope adjustment device further includes a motion control module, which is used to control the corresponding joints of the target robotic arm to move in the target motion direction.

7. The endoscope adjustment device according to claim 4, characterized in that, The endoscope adjustment device also includes a prompt information generation module and a display control module; The prompt information generation module is used to generate a first prompt information when the Nth joint is located at the corresponding target position; The display control module is used to: control a prompting device to display the first prompting information; And / or, The prompt information generation module is used to generate a second prompt information when the target robotic arm arrives at the target pose. The display control module is used to control a display device to display the second prompt information.

8. The endoscope adjustment device according to claim 3, characterized in that, The target motion direction is the rotational or translational direction of the joint.

9. An endoscope adjustment method, characterized in that, The endoscope adjustment method is applied to a surgical robot system, which includes a first endoscope and a second endoscope, wherein one of the first endoscope and the second endoscope is a white light endoscope and the other is a fluorescence endoscope. The endoscope adjustment method includes: The position of the target tissue in the coordinate system of the robotic arm base is obtained based on the image acquired by the current endoscope; the target tissue is located within the field of view of the current endoscope, which is mounted on the end of the current robotic arm; Based on the position of the target tissue in the coordinate system of the robotic arm base, the target pose of the target robotic arm with the target endoscope mounted is planned; when the target robotic arm is in the target pose, the target tissue is located within the field of view of the target endoscope, and the target robotic arm is different from the current robotic arm; Obtain the current position of the joints of the target robotic arm, and plan the target motion direction of at least one joint of the target robotic arm based on the current position of the joints of the target robotic arm and the target pose of the target robotic arm. The current endoscope is one of the first endoscope and the second endoscope, and the target endoscope is the other of the first endoscope and the second endoscope.

10. A surgical robot system, characterized in that, include: The surgical operating device includes a first robotic arm and a second robotic arm; An endoscope assembly for acquiring images of target tissue includes a first endoscope and a second endoscope, wherein the first endoscope is mounted on the end of a first robotic arm and the second endoscope is mounted on the end of a second robotic arm; one of the first endoscope and the second endoscope is the current endoscope and the other is the target endoscope. as well as, The endoscope adjustment device as described in any one of claims 1-8, wherein the endoscope adjustment device is communicatively connected to the surgical operating device and the endoscope assembly.

Images