Endoscope robot and pose adjusting method thereof

Abstract

The invention provides an endoscope robot and a pose adjusting method thereof, and the method comprises the steps: building simulation connection between a mechanical arm connected with an endoscope and a simulation module under the initial pose of the mechanical arm connected with the endoscope and the initial pose of the simulation module; wherein the simulation module comprises a base and an operating rod, and in the simulation connection state, the position posture of the operating rod and the position posture of the endoscope have a corresponding relation; according to the current pose of the operating rod adjusted by the user, the mechanical arm is controlled to move so as to adjust the position pose of the endoscope. The pose of the endoscope is controlled and adjusted through the operating rod of the simulation module, the adjusting precision is improved, the mechanical arm can accurately adjust the endoscope to the position indicated by the operating rod, the mechanical arm can stably hold the endoscope for a long time, the workload of doctors / nurses is reduced, and the working efficiency is improved. And the endoscope is adjusted through the simulation module, so that the operation space near a patient cannot be occupied, and the operation of a doctor is freer.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical devices, in particular to an endoscope robot and a pose adjustment method thereof. BACKGROUND

[0002] Endoscopic neurosurgery is a minimally invasive surgical technique that uses an endoscope to access the brain or spinal cord area through a small incision to treat various nervous system diseases. During endoscopic surgery, the pose of the endoscope often needs to be adjusted to observe different areas as the surgery progresses.

[0003] One common way is to manually adjust the pose of the endoscope, which requires an assistant doctor to hold the mirror to provide assistance to the main doctor. On the one hand, a lot of practice is needed to make the coordination smooth, i.e. a main doctor usually cannot complete the surgery alone. On the other hand, the assistant doctor occupies a certain operating position, increasing the difficulty of the surgery for the main doctor. Thirdly, the adjustment accuracy of manual mirror holding is poor, and long-term holding of the endoscope also brings a large workload to the doctor, and the mirror holding stability is poor. Another common way is to install the endoscope to a mechanical arm, and adjust the pose of the endoscope by dragging the mechanical arm. In order to ensure safety, a fixed "damping" is usually set for the mechanical arm to avoid accidental disturbance causing the mechanical arm to move. The operator needs to overcome the "damping" of the mechanical arm to adjust the pose of the endoscope, which reduces the pose adjustment accuracy.

[0004] To solve the defects of poor accuracy and inconvenient operation of adjusting the pose of the endoscope in the prior art, the present application provides an endoscope robot and a pose adjustment method thereof. SUMMARY

[0005] The present application provides an endoscope robot and a pose adjustment method thereof to solve the defects of poor accuracy and inconvenient operation of adjusting the pose of the endoscope in the prior art.

[0006] The present application provides a pose adjustment method of an endoscope robot, characterized in that it comprises:

[0007] Under the initial pose of the mechanical arm connected with the endoscope and the initial pose of the simulation module, a simulation connection is established between the mechanical arm connected with the endoscope and the simulation module; wherein the simulation module comprises a base and an operating rod, and in the simulation connection state, the position and pose of the operating rod have a corresponding relationship with the position and pose of the endoscope;

[0008] The pose of the operating rod is controlled to control the movement of the mechanical arm, and the position and pose of the endoscope are adjusted.

[0009] establish the simulation connection between the mechanical arm connected with the endoscope and the simulation module under initial poses of the mechanical arm and the simulation module; wherein, the end of the mechanical arm is connected with an endoscope, the simulation module comprises a base and an operating rod, and the position and pose of the operating rod have a corresponding relationship with the position and pose of the endoscope in the simulation connection state;

[0010] control the movement of the mechanical arm according to the pose change of the operating rod to adjust the position and pose of the endoscope.

[0011] Optionally, the establishment of the simulation connection between the mechanical arm connected with the endoscope and the simulation module under initial poses of the mechanical arm and the simulation module comprises:

[0012] an instruction for establishing the simulation connection is acquired;

[0013] the initial pose of the endoscope in the mechanical arm coordinate system is determined according to the initial pose of the mechanical arm and the fixed connection relationship between the end of the mechanical arm and the endoscope;

[0014] the initial pose of the operating rod in the simulation coordinate system is determined;

[0015] the simulation connection is established according to the initial pose of the endoscope in the mechanical arm coordinate system and the initial pose of the operating rod in the simulation coordinate system.

[0016] Further, the establishment of the simulation connection according to the initial pose of the endoscope in the mechanical arm coordinate system and the initial pose of the operating rod in the simulation coordinate system comprises:

[0017] the initial pose of a first mapping reference point in the mechanical arm coordinate system is determined according to the initial pose of the endoscope in the mechanical arm coordinate system and the first mapping reference point in the endoscope;

[0018] the initial pose of a second mapping reference point in the simulation coordinate system is determined according to the initial pose of the operating rod in the simulation coordinate system and the second mapping reference point in the operating rod;

[0019] the initial pose of the first mapping reference point in the mechanical arm coordinate system and the initial pose of the second mapping reference point in the simulation coordinate system are taken as the initial state of the simulation connection.

[0020] In the application, the mapping reference point is a point with a space position indication and has a three-dimensional orthogonal coordinate system with a space reference point as the origin, and preferably, the mapping reference point is a rotationally fixed point, i.e., a point that remains unchanged in the rotation component of the movement of the endoscope and the operating rod.

[0021] Optionally, the controlling the movement of the mechanical arm according to the change of the pose of the operation rod to adjust the position and pose of the endoscope comprises:

[0022] obtaining a current pose of the operation rod in a simulation coordinate system.

[0023] obtaining a change of the pose of the operation rod in the simulation coordinate system according to the current pose and the initial pose of the operation rod in the simulation coordinate system;

[0024] scaling the change of the pose of the operation rod according to a scaling coefficient to obtain a change of the pose of the endoscope in a mechanical arm coordinate system;

[0025] determining a target pose of the end of the mechanical arm according to the change of the pose of the endoscope in the mechanical arm coordinate system, and controlling the movement of the mechanical arm to the target pose.

[0026] Optionally, the scaling the change of the pose of the operation rod according to the scaling coefficient to obtain the change of the pose of the endoscope in the mechanical arm coordinate system comprises:

[0027] decomposing the change of the pose of the operation rod into a displacement change and an angle change;

[0028] scaling the displacement change according to a displacement scaling coefficient and scaling the angle change according to an angle scaling coefficient;

[0029] obtaining the change of the pose of the endoscope in the mechanical arm coordinate system according to the scaled displacement change and the scaled angle change.

[0030] Further, in the case that the user selects a translation mode, the angle change of the operation rod is shielded; in the case that the user selects a rotation mode, the displacement change of the operation rod is shielded.

[0031] Optionally, the scaling coefficient is adjustable.

[0032] Further, the controlling the movement of the mechanical arm according to the change of the pose of the operation rod to adjust the position and pose of the endoscope further comprises:

[0033] controlling the endoscope to be near a target position according to the operation of the user on the operation rod with a first scaling coefficient;

[0034] adjusting the first scaling coefficient to a second scaling coefficient according to a treatment input by the user;

[0035] controlling the endoscope to be at the target position according to the operation of the user on the operation rod with the second scaling coefficient.

[0036] Optionally, a movement boundary of the endoscope is determined in advance, and the method further comprises:

[0037] Based on the relative position of the endoscope and the motion boundary, determine whether to output feedback resistance through the operating lever.

[0038] Further, determining whether to output feedback resistance through the operating lever based on the relative position of the endoscope and the motion boundary includes:

[0039] Based on the simulation connection, the motion boundary in the robotic arm coordinate system is mapped to the simulation coordinate system to obtain the mapped boundary;

[0040] Based on the control lever and the mapping boundary in the simulation coordinate system, determine whether to output feedback resistance.

[0041] Further, the step of mapping the motion boundary in the robotic arm coordinate system to the simulation coordinate system based on the simulation connection to obtain the mapped boundary includes:

[0042] Based on the motion boundary in the robotic arm coordinate system and the pose of the endoscope in the robotic arm coordinate system, the motion boundary of the endoscope in the first mapping reference point coordinate system in the endoscope is obtained.

[0043] Based on the motion boundary and scaling factor in the first mapping reference point coordinate system, the mapping boundary in the joystick coordinate system is obtained;

[0044] Based on the pose of the joystick in the simulation coordinate system and the mapping boundary in the joystick coordinate system, the mapping boundary in the simulation coordinate system is obtained.

[0045] Optionally, the closer the endoscope is to the motion boundary, the greater the feedback resistance output by the operating lever; or, the distance between the endoscope and the motion boundary is divided into segments, and the feedback resistance output by the operating lever is different in different segments.

[0046] Optionally, when the endoscope reaches the operating boundary, an alarm message is also provided via the output module.

[0047] The present invention also provides an endoscope robot, comprising: a processing module, a robotic arm, and a simulation module;

[0048] The end of the robotic arm is connected to an endoscope for adjusting the position and orientation of the endoscope.

[0049] The simulation module includes a base and an operating lever, and the base is capable of detecting the position and pose of the operating lever;

[0050] The processing module includes a memory and a processor. When the processor executes a computer program stored in the memory, it implements the pose adjustment method of the endoscopic robot described above, and controls the movement of the robotic arm according to the pose of the control lever to adjust the pose of the endoscope.

[0051] Optionally, the base includes a wireless detection module, and the control lever is equipped with a wireless positioning module. The wireless detection module determines the spatial orientation of the control lever through the wireless positioning module.

[0052] Optionally, the simulation module includes a base and a retractable operating rod. The orientation of the operating rod on the base is adjustable. The base determines the spatial pose of the operating rod by detecting the orientation of the operating rod and the amount of extension and retraction of the operating rod.

[0053] Optionally, the simulation module includes a connecting rod connected to the base, the operating rod is the end segment of the connecting rod, and the base determines the spatial pose of the operating rod by detecting the angles of each segment of the rod.

[0054] Optionally, the method further includes: obtaining the mapping relationship between the medical image coordinate system and the simulation coordinate system based on the simulation connection, mapping the motion boundary of the medical image coordinate system to the simulation coordinate system to obtain the mapping boundary, and outputting corresponding force feedback based on the distance between the control lever and the mapping boundary.

[0055] The present invention also provides an external viewing mirror robot, comprising: a processing module, a robotic arm, an image acquisition module, and a simulation module;

[0056] The robotic arm is connected to an image acquisition module at its end, which is used to adjust the pose of the endoscope image acquisition module.

[0057] The simulation module includes a base and an operating lever, and the base is capable of detecting the position and pose of the operating lever;

[0058] The processing module includes a memory and a processor. When the processor executes a computer program stored in the memory, it implements the pose adjustment method of the endoscopic robot described above, and controls the movement of the robotic arm according to the pose of the control lever to adjust the pose of the image acquisition module.

[0059] Optionally, the system also includes a navigation module. The distal end of the robotic arm or the endoscope is equipped with positioning markers. The navigation module tracks these positioning markers to determine the position of the endoscope in a navigation coordinate system. That is, the navigation coordinate system can replace the robotic arm coordinate system, defining the position and movement boundaries of the endoscope and providing higher-precision guidance when the robotic arm's own movement accuracy is lower than the visual tracking positioning accuracy.

[0060] The present invention also provides another endoscopic robot, comprising: a processing module, a robotic arm, a simulation module, and a navigation module;

[0061] The simulation module includes a base and an operating lever. An endoscope is connected to the end of the robotic arm. A positioning mark is set on the end of the robotic arm or the endoscope. The navigation module tracks the positioning mark to determine the position and attitude of the endoscope in the navigation coordinate system.

[0062] The processing module is capable of performing the following methods:

[0063] Establish a simulated connection between the endoscope and the control lever, such that the position and orientation of the control lever correspond to the position and orientation of the endoscope;

[0064] The movement of the robotic arm is controlled according to the position and posture changes of the control lever, thereby adjusting the position and posture of the endoscope.

[0065] Furthermore, establishing the simulated connection between the endoscope and the operating lever includes:

[0066] Obtain the command to establish a simulation connection;

[0067] Determine the initial pose of the endoscope in the navigation coordinate system;

[0068] Determine the initial pose of the control lever in the simulation coordinate system;

[0069] A simulation connection is established based on the initial pose of the endoscope in the navigation coordinate system and the initial pose of the control lever in the simulation coordinate system.

[0070] Furthermore, establishing the simulation connection based on the initial pose of the endoscope in the navigation coordinate system and the initial pose of the control lever in the simulation coordinate system includes:

[0071] Based on the initial pose of the endoscope in the navigation coordinate system and the first mapping reference point in the endoscope, determine the initial pose of the first mapping reference point in the navigation coordinate system;

[0072] Based on the initial pose of the control lever in the simulation coordinate system and the second mapping reference point in the control lever, determine the initial pose of the second mapping reference point in the simulation coordinate system.

[0073] The initial pose of the first mapping reference point in the navigation coordinate system and the initial pose of the second mapping reference point in the simulation coordinate system serve as the initial state of the simulation connection.

[0074] The endoscopic robot and its pose adjustment method provided by this invention have at least the following beneficial effects:

[0075] 1. The position and posture of the endoscope at the end of the robotic arm are controlled and adjusted by the joystick of the simulation module, which improves the adjustment accuracy. The robotic arm can accurately adjust the endoscope to the position indicated by the joystick, and the robotic arm can hold the endoscope stably for a long time, reducing the workload of doctors / nurses. In addition, adjusting the endoscope through the simulation module will not occupy the surgical space near the patient, and the doctor's operation is more free.

[0076] 2. A first mapping reference point is set in the endoscope at the end of the robotic arm, and a corresponding second mapping reference point is set in the control lever of the simulation module. The simulation connection between the two is established. The simulation module can accurately determine the pose of the second mapping reference point in the control lever under any posture. The robotic arm can accurately adjust its posture so that the first mapping reference point in the endoscope reaches the corresponding pose, which provides the basis for simulation control and ensures the accuracy of simulation control.

[0077] 3. Simulation control can scale the position and posture changes of the control lever, which can improve the control accuracy and efficiency of the endoscope position and posture adjustment process. In some implementations, the scaling factor is adjustable, and users can select the required scaling factor at different adjustment stages to balance the control accuracy and efficiency of the endoscope position and posture.

[0078] 4. It provides translation mode and rotation mode. Translation mode can shield the rotational changes of the control stick, and rotation mode can shield the displacement changes of the control stick, allowing users to adjust the position and posture of the endoscope more precisely.

[0079] 5. When the endoscope reaches the movement boundary, feedback resistance is output through the operating lever, which improves the safety of use.

[0080] 6. It provides simulation modules in wireless, telescopic pole, and linkage forms to meet users' diverse product needs.

[0081] 7. It can assist in remote assistance, allowing assistant doctors to provide technical support remotely, so that only the surgeon needs to be on-site to complete the surgery, thus improving the utilization rate of medical resources.

[0082] 8. When used in conjunction with the navigation module, it can more accurately locate the position and orientation of the endoscope. It can also work with other functions of surgical navigation, such as displaying the endoscope, patient 3D model, and other surgical instruments together on the display device, improving information integration and providing convenience for doctors' operations. Attached Figure Description

[0083] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0084] Figure 1 This is a system structure diagram of the endoscopic robot provided by the present invention;

[0085] Figure 2 This is a flowchart illustrating the pose adjustment method for the endoscopic robot provided by the present invention.

[0086] Figure 3 This is a schematic diagram of another endoscopic robot system provided by the present invention. Detailed Implementation

[0087] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0088] The following is combined Figures 1-2 This invention describes an endoscopic robot and its pose adjustment method.

[0089] Figure 1 This is a system structure diagram of an endoscopic robot provided by the present invention, as shown below. Figure 1 As shown, the endoscopic robot includes: a processing module 100, a robotic arm 200, an endoscope 300, and a simulation module 400.

[0090] One end of the robotic arm 200 is fixed, for example, to the ground, a hospital bed, a trolley, a wall, a ceiling, etc. The robotic arm 200 can be a robotic arm with 4 degrees of freedom or more, and can adjust the spatial pose of the endoscope 300 connected to its other end.

[0091] Endoscope 300 is a rigid endoscope. The robotic arm 200 can maintain the endoscope 300 in the required position and posture, and acquire images of the target area of ​​interest to the doctor from the appropriate position and angle without hindering the doctor's operation. The robotic arm 200 can stably maintain the position and posture of the endoscope 300 for a long time, and can also adjust the position and posture of the endoscope 300 under the control of medical staff when the target area of ​​interest to the doctor changes.

[0092] The simulation module 400 includes a base 410 and an operating joystick 420. The base 410 can be fixed to the ground, hospital bed, desktop, trolley, etc. The operating joystick 420 can adjust its posture relative to the base 410, and its spatial posture can be detected by the base 410.

[0093] The processing module 100 is communicatively connected to the robotic arm 200 and the simulation module 400 (either via wired or wireless connection). The processing module 100 employs a general computer architecture, including a memory and a processor. The processor executes the computer program stored in the memory to adjust the pose of the endoscope 300. Specifically, the processing module 100 acquires the pose of the control lever 420 transmitted from the simulation module 400, generates control commands based on the pose of the control lever 420, and then sends the control commands to the robotic arm 200. The robotic arm 200 then moves according to the control commands to adjust the endoscope 300 to the corresponding pose.

[0094] It is understandable that the various components of an endoscopic robot can be separate and independent, or they can be partially or completely integrated together. For example, the simulation module base 410 and the robotic arm base can be integrated into the operating trolley, and the processing module 100 can be placed inside the operating trolley. Alternatively, the processing module 100 can be integrated into the robotic arm base, and the simulation module 400 can be set up independently.

[0095] The endoscopic robot provided in this embodiment controls and adjusts the position and posture of the endoscope through the operating lever of the simulation module, which improves the adjustment accuracy. The robotic arm can accurately adjust the endoscope to the position indicated by the operating lever, and the robotic arm can hold the endoscope stably for a long time, reducing the workload of doctors / nurses. Furthermore, adjusting the endoscope through the simulation module does not occupy the surgical space near the patient, giving doctors more freedom of operation.

[0096] Based on any embodiment, in some embodiments, the base 410 includes a wireless detection module, and the operating lever 420 is provided with a wireless positioning module. The wireless detection module determines the spatial orientation of the operating lever 420 through the wireless positioning module. Specifically, the wireless detection module can determine the position of the wireless positioning module based on ultrasonic positioning, infrared positioning, RFID positioning, electromagnetic positioning, etc., thereby determining the spatial orientation of the operating lever 420 (relative to the base 410). Furthermore, an operating lever socket can be provided on the base 410, so that the operating lever 420 can be placed in the socket when not in use and removed when needed.

[0097] In another embodiment, the simulation module 400 includes a base 410 and an operating lever 420. The orientation (posture) of the operating lever 420 on the base 410 is adjustable. The operating lever 420 is equivalent to a rocker arm set on the base 410. The operating lever 420 is telescopic. The base 410 determines the spatial pose of the operating lever 420 by detecting the orientation of the operating lever 420 and the amount of telescopic extension of the operating lever 420.

[0098] In another embodiment, the simulation module 400 includes a connecting rod connected to the base 410. The connecting rod includes multiple rod segments. The operating rod 420 is the end segment of the connecting rod. The base 410 determines the spatial pose of the operating rod 420 by detecting the angles of each rod segment.

[0099] The above provides various simulation modules to meet the needs of different costs and operating methods.

[0100] Based on any embodiment, in some embodiments, the endoscopic robot further includes an interaction module:

[0101] The interaction module may include a display device capable of displaying a 3D model of the patient, or overlaying the endoscope model with the 3D model based on the actual position of the patient and the endoscope. The interaction module may also include command input devices, such as a keyboard, mouse, touchscreen, foot switch, and microphone. Users can input relevant commands to the endoscopic robot through these devices, such as adjusting the display angle, switching between different observation modes that hide or show the endoscope model, inputting commands to establish a simulation connection, or controlling the robotic arm to stop 200 degrees in an emergency using the foot switch.

[0102] The following describes a pose adjustment method for an endoscope robot provided by the present invention. The pose adjustment method for the endoscope robot described below can be referred to in correspondence with the endoscope robot described above.

[0103] Figure 2 This is a flowchart illustrating a method for adjusting the pose of an endoscopic robot provided by the present invention, as shown below. Figure 2 As shown, the method includes:

[0104] S1. In the initial posture of the robotic arm 200 connected to the endoscope 300 and the initial posture of the simulation module 400, a simulation connection is established between the robotic arm 200 connected to the endoscope 300 and the simulation module 400; wherein, the simulation module 400 includes a base 410 and an operating lever 420, and in the simulation connection state, the position and posture of the operating lever 420 correspond to the position and posture of the endoscope 300.

[0105] S2. Based on the positional changes of the control lever 420, control the movement of the robotic arm 200 to adjust the position and orientation of the endoscope 300.

[0106] Specifically, the initial posture of the robotic arm 200 can be a specific posture initialized during robotic arm development, or it can be a posture that has been adjusted. For example, the user can drag the robotic arm 200 to roughly adjust the endoscope 300 to a position near the patient as the initial posture. Similarly, the initial posture of the simulation module 400 can be a specific posture initialized during robotic arm development, or it can be a posture that has been adjusted. For example, the user can adjust the operating lever 420 of the simulation module 400 to be roughly in the same direction as the endoscope 300, and use this as the initial posture of the simulation module 400. After receiving the instruction to establish a simulation connection, the initial postures of the robotic arm 200 and the simulation module 400 are acquired and used as the initial state of the simulation connection. Subsequently, the positional changes of the operating lever 420 in the simulation module 400 will be mapped to the endoscope 300 at the end of the robotic arm 200, that is, there is a corresponding relationship between the positional posture of the operating lever 420 and the positional posture of the endoscope 300.

[0107] Preferably, a simulation connection is established with the endoscope 300 facing the same or approximately the same orientation as the operating lever 420 in the initial posture, which makes it easier for the operator to adjust the endoscope 300 to the required position more intuitively.

[0108] This embodiment obtains the initial pose of the robotic arm and the simulation module to establish a simulation connection, which provides a foundation for the simulation control of the endoscope. The simulation module improves the accuracy and stability of adjusting the endoscope.

[0109] Based on the previous embodiment, in some embodiments, step S1 includes:

[0110] S11. Obtain the command to establish a simulation connection;

[0111] S12. Based on the initial posture of the robotic arm 200 and the fixed connection between the end of the robotic arm 200 and the endoscope 300, determine the initial pose of the endoscope 300 in the robotic arm coordinate system.

[0112] S13. Determine the initial pose of the control lever 420 in the simulation coordinate system.

[0113] S14. Establish a simulation connection based on the initial pose of the endoscope in the robotic arm coordinate system and the initial pose of the control lever in the simulation coordinate system.

[0114] Specifically, the command to establish the simulation connection can be input by the user in any form, such as via a button on the joystick 420, a keyboard, mouse, or foot switch, voice input and recognition, or gesture recognition. In the initial posture of the robotic arm 200, the endoscope at the end of the robotic arm 200 is in a 300° position. It can be determined that the pose of the endoscope 300 in this position will be used as the basis for simulation mapping, along with the initial pose of the control lever 420 of the simulation module 400. A simulation mapping is established. In the subsequent process, the pose change of the control lever 420 will cause the pose change of the endoscope 300, and the poses of the two have a one-to-one correspondence.

[0115] In this embodiment, after obtaining the instruction to establish a simulation mapping, a simulation connection is established based on the initial posture of the endoscope in the robotic arm coordinate system and the initial posture of the control lever in the simulation coordinate system. The initial state of the simulation mapping is recorded, which facilitates the subsequent mapping of the position changes of the control lever to the endoscope.

[0116] Based on the foregoing embodiments, in some embodiments, step S12 includes:

[0117] Based on the initial posture of the robotic arm 200, determine the initial pose of the end effector of the robotic arm 200 in the robotic arm coordinate system. And combined with the pre-calibrated conversion relationship between the endoscope 300 and the end of the robotic arm 200 Obtain the initial pose of endoscope 300 in the robotic arm coordinate system. The specific formula is as follows:

[0118]

[0119] Based on the previous embodiment, in some embodiments, step S14 includes:

[0120] S141. Determine the initial pose of the first mapping reference point in the robotic arm coordinate system based on the initial pose of the endoscope in the robotic arm coordinate system and the first mapping reference point in the endoscope.

[0121] S142. Based on the initial pose of the control stick in the simulation coordinate system and the second mapping reference point in the control stick, determine the initial pose of the second mapping reference point in the simulation coordinate system.

[0122] S143. The initial pose of the first mapping reference point in the robot arm coordinate system and the initial pose of the second mapping reference point in the simulation coordinate system serve as the initial state of the simulation connection.

[0123] Specifically, refer to Figure 1A first mapping reference point was determined in the endoscope, and a second mapping reference point was determined in the control lever. Since the initial pose of the endoscope in the robotic arm coordinate system is known, the pose (including orientation) of the first mapping reference point in the endoscope can be determined. Therefore, the initial pose of the first mapping reference point in the robotic arm coordinate system can be determined, and its actual pose can describe the subsequent pose changes of the endoscope. Similarly, the initial pose (including orientation) of the second mapping reference point in the simulation coordinate system can also be determined, and its actual pose can describe the subsequent pose changes of the control lever. The initial poses of these two mapping reference points are used as the initial state of the simulation connection, facilitating the subsequent mapping of the control lever's pose changes to the endoscope.

[0124] Furthermore, in a specific endoscopic scenario, the doctor places an observation sheath at a location on the skull. The endoscope penetrates the sheath to image the target area. The endoscope can move within the sheath, adjusting its position and angle to observe different target areas. In this scenario, the intersection of the endoscope and the curved surface of the skull can be determined as the first mapping reference point. Even further, during simulation control, if the depth of the endoscope within the observation sheath changes by more than a preset threshold, the intersection of the endoscope with the curved surface of the skull after depth adjustment can be determined as the new first mapping reference point, either based on the doctor's instructions or automatically.

[0125] This embodiment defines a first mapping reference point in the endoscope and a second mapping reference point in the control lever, which facilitates the accurate mapping of the control lever's pose changes to the endoscope.

[0126] Furthermore, the second mapping reference point can be any position on the joystick. It can be rotated by the user according to their usage habits, or a default setting can be used, such as setting the midpoint, quarter point, or center of the connecting end of the joystick as the second mapping reference point. In some optional implementations, the pose of the joystick in the simulation coordinate system is directly described based on the fixed second mapping reference point in the joystick. In this case, the initial pose of the joystick in the simulation coordinate system in the aforementioned step S143 is the initial pose of the second mapping reference point, which is directly used to establish the simulation connection. The following embodiment also uses "directly describing the pose of the joystick in the simulation coordinate system based on the fixed second mapping reference point in the joystick" as an example.

[0127] Those skilled in the art will understand that the position and attitude of the joystick in the simulation coordinate system can also be described based on other positions, such as a coordinate system established based on the end of the joystick, or a coordinate system established based on the spatial position tracking mark in the joystick. In this case, it is necessary to combine the pose of the second mapping reference point in the joystick to further determine the position and attitude (initial pose / subsequent pose) of the second mapping reference point in the simulation coordinate system.

[0128] The position of the joystick in the simulation coordinate system can be described, for example, based on its tracking method. This is understandable.

[0129] Based on the previous embodiment, in some embodiments, step S141 includes:

[0130] Based on the initial pose of the endoscope 300 in the robotic arm coordinate system and the position and orientation of the first mapping reference point in the endoscope 300 Obtain the initial pose of the robotic arm in the coordinate system to the first mapping reference point. The specific formula is as follows:

[0131]

[0132] In this embodiment, the conversion relationship between the endoscope and the robotic arm end effector is described above. The first mapping reference point can be pre-calibrated, and its position and orientation within the endoscope can be determined. For example, the first mapping reference point can be set at the center of the endoscope body, or, based on the positional relationship between the endoscope and the patient, a point on the endoscope can be determined as the first mapping reference point. This embodiment describes the orientation of the first mapping reference point using a coordinate system established based on it, and outlines the transformation relationship between the robotic arm coordinate system and the first mapping reference point coordinate system. This describes the position and orientation of the first mapping reference point in the robotic arm coordinate system.

[0133] Based on any embodiment, in some embodiments, step S2 includes:

[0134] S21. Obtain the current pose of the control lever 420 in the simulation coordinate system.

[0135] S22. Based on the current pose of the control lever 420 in the simulation coordinate system. and initial pose The pose changes of the control stick 420 in the simulation coordinate system were obtained. The specific formula is as follows:

[0136]

[0137] S23. Adjust the pose of the control stick 420 according to the scaling factor. By scaling, the pose change of the first mapping reference point in the robot arm coordinate system is obtained.

[0138] S24. Based on the pose change of the first mapping reference point in the robot arm coordinate system. Determine the target pose of the robotic arm 200 end effector And control the robotic arm 200 to move to the target pose.

[0139] Specifically, steps S21 and S22 above can acquire the pose change of the operating lever 420, and step S23 scales the pose change of the operating lever 420. It can be understood that when the scaling factor is in the "magnified state", a small pose change of the operating lever 420 can adjust the endoscope 300 by a larger extent, improving the adjustment efficiency; when the scaling factor is in the "reduced state", a large pose change of the operating lever 420 can adjust the endoscope 300 by a smaller extent, improving the adjustment accuracy.

[0140] Step S24 performs a scaled pose change in the robotic arm coordinate system. Specifically, this is done based on the initial pose of the first mapping reference point in the robotic arm coordinate system. and pose changes Determine the target pose of the mapping reference point The specific formula is as follows:

[0141]

[0142] Then, based on the position and orientation of the first mapping reference point in the endoscope 300 Determine the target pose of endoscope 300 The specific formula is as follows:

[0143]

[0144] Furthermore, consider the conversion relationship between the endoscope 300 and the end effector of the robotic arm 200. Determine the target pose of the robotic arm 200 end effector The specific formula is as follows:

[0145]

[0146] This embodiment accurately maps the pose changes of the control lever to the pose changes of the mapping reference point in the endoscope, thereby improving the adjustment accuracy of the endoscope.

[0147] Based on the previous embodiment, in some embodiments, step S23 includes:

[0148] Change the position of the control stick 420 It can be decomposed into changes in displacement and changes in angle;

[0149] The displacement change is scaled according to the displacement scaling factor, and the angle change is scaled according to the angle scaling factor.

[0150] Based on the displacement and angle changes after scaling, the pose change of the first mapping reference point in the robotic arm coordinate system is obtained.

[0151] Specifically, in this embodiment, the matrix is ​​decomposed into displacement change and angle change, which makes it easier to scale the displacement and angle to the same or different degrees, and can better meet the position and pose adjustment requirements of the endoscope 300. For example, the endoscope 300 is relatively long and thin. After the endoscope 300 is placed in the observation sheath, even a small angle change of the endoscope 300 will cause a large displacement of the distal end of the endoscope 300. By setting a large angle scaling coefficient, the angle can be adjusted more precisely, and collision between the distal end of the endoscope 300 and the observation sheath can be avoided.

[0152] Based on the previous embodiment, in some embodiments, when the user selects the translation mode, the angular change of the control lever 420 is masked; when the user selects the rotation mode, the displacement change of the control lever 420 is masked.

[0153] Specifically, "shielding the angular change of the operating lever 420" can be achieved by "setting the angular change to zero" or by "setting the angular scaling factor to zero," and the same applies to "shielding the displacement change of the operating lever 420." It is understood that the previous embodiment described the processing flow of the normal mode (free mode). This embodiment also provides translation mode and rotation mode, allowing users to selectively shield displacement / angular changes for more precise adjustment of the endoscope 300's spatial pose. Furthermore, the operating lever is equipped with selection or switching buttons for translation mode, rotation mode, and normal mode, allowing users to conveniently select the desired mode and flexibly adjust the endoscope's pose.

[0154] Based on any embodiment, in some embodiments, the scaling factor is adjustable.

[0155] In one use case, before establishing a simulation connection, the user adjusts the scaling factor to a value that suits their personal usage habits, then establishes the simulation connection and executes simulation control. In another use case, after adjusting endoscope 300 using a certain scaling factor, the user adjusts the scaling factor again and adjusts endoscope 300 according to the new scaling factor.

[0156] Specifically, when the user "sets the displacement scaling factor to a non-zero value and sets the angle scaling factor to zero", the translation mode of the previous embodiment can be realized; when the user "sets the angle scaling factor to a non-zero value and sets the displacement scaling factor to zero", the rotation mode of the previous embodiment can be realized.

[0157] This embodiment allows users to select the appropriate scaling factor according to their needs, improving user convenience.

[0158] Based on the previous embodiment, in some embodiments, the method further includes: after receiving the instruction to adjust the scaling factor, taking the current posture of the robotic arm 200 and the simulation module 400 as the new initial posture, and re-establishing the simulation connection.

[0159] Specifically, in this embodiment, the scaling factor is adjustable during the adjustment of the endoscope 300 after establishing the simulation connection. Upon receiving the instruction to adjust the scaling factor, the connection is re-established based on the current posture of the robotic arm 200 and the current posture of the simulation module 400 as the new initial posture. This avoids "scaling down some displacement and angle changes before the adjustment factor using the new scaling factor" during subsequent adjustments, making the adjustment process of the endoscope 300 before and after the scaling factor adjustment more continuous and preventing accidental damage. Of course, it is also possible to use the old scaling factor to scale down some displacement and angle changes before the adjustment factor, and use the new scaling factor to scale down some displacement and angle changes after the adjustment factor. The above two processing methods are simple equivalent variations, differing only in wording.

[0160] Based on the foregoing embodiments, in some embodiments, step S2 includes:

[0161] Based on the user's operation of the lever 420, the endoscope 300 is moved to the vicinity of the target position with the first zoom factor;

[0162] Based on the user's input, the first scaling factor is adjusted to the second scaling factor;

[0163] Based on the user's operation of the control lever 420, the endoscope 300 is moved to the target position with the second zoom factor.

[0164] In other words, after adjusting the endoscope 300 using a certain scaling factor, the user can switch to another scaling factor to adjust the endoscope 300. For example, the user can first use a (relatively) larger scaling factor to roughly adjust the position / angle of the endoscope 300, and then use a (relatively) smaller scaling factor to finely adjust the position / angle of the endoscope 300.

[0165] Based on the foregoing embodiments, in some embodiments, the motion boundaries of the endoscope 300 are predetermined, and the method further includes:

[0166] S3. Based on the relative position of the endoscope 300 and the movement boundary, determine whether to output feedback resistance through the operating lever 420.

[0167] Specifically, in some application scenarios, the endoscope 300 has a certain safe operating range. Therefore, the motion boundary of the endoscope can be determined in the robotic arm coordinate system, or in other coordinate systems related to the robotic arm, such as the medical imaging coordinate system or the navigation coordinate system. The transformation relationship between these coordinate systems and the robotic arm coordinate system can be determined through preoperative registration. The pose of the endoscope can be calculated under any robotic arm posture. Combining the above relationships, the "endoscope" and "motion boundary" can be transformed into the same coordinate system to determine whether the endoscope is too close to the motion boundary and whether feedback resistance needs to be output through the control lever 420. It can be understood that the medical imaging coordinate system can be a single medical image or a fused medical imaging coordinate system obtained by fusing two or more medical images.

[0168] In a specific endoscopic scenario, the doctor places an observation sheath at a certain point in the skull. The endoscope penetrates the observation sheath to image the target area. The endoscope can move within the observation sheath, adjusting its position and angle to observe different target areas. In this scenario, the area inside the observation sheath is the safe operating range of the endoscope 300, and the corresponding movement boundaries are set according to the inner wall of the observation sheath. More specifically, the position of the motion boundary can be determined based on the surgical plan. For example, the doctor plans the position of the observation sheath on the patient model in the medical imaging coordinate system. Before the endoscopic surgery, the observation sheath is placed in the corresponding position according to the surgical plan. At this time, the motion boundary can be set directly according to the observation sheath in the medical imaging coordinate system. Combined with the endoscope's pose determined based on the robotic arm's posture during the operation, the "endoscope" and "motion boundary" are transformed to the same coordinate system to determine their relative positions and whether feedback resistance needs to be output. For another example, before the endoscopic surgery, the doctor directly places the observation sheath at a certain location on the patient and determines the spatial position of the observation sheath in the navigation coordinate system through 3D imaging, optical navigation, etc. Based on this, the motion boundary in the navigation coordinate system is set. Then, combined with the transformation relationship between the navigation coordinate system and the robotic arm coordinate system, the "endoscope" and "motion boundary" are transformed to the same coordinate system to determine their relative positions and whether feedback resistance needs to be output.

[0169] In another specific endoscopic surgical scenario, the endoscope directly penetrates the target area to acquire images, requiring the setting of preset boundaries based on the patient's medical images. For example, in transnasal endoscopic surgery, the nasal cavity is the endoscope's operating space. A patient model is reconstructed based on the patient's medical images, and the aforementioned preset boundaries are set in the medical image coordinate system according to the spatial position of the patient's nasal cavity. After surgical registration, the motion boundaries in the medical image space can be mapped to the surgical space (e.g., robotic arm coordinate system, navigation coordinate system).

[0170] During the actual adjustment of the endoscope 300, the relative position of the endoscope 300 to the motion boundary determines whether feedback resistance is output through the operating lever 420. For example, when the endoscope 300 reaches the motion boundary, feedback resistance is output through the operating lever 420; otherwise, no feedback resistance is output. Alternatively, when the distance between the endoscope 300 and the motion boundary is less than a preset distance threshold, the closer the endoscope 300 is to the motion boundary, the greater the feedback resistance output by the operating lever 420; when the distance is not less than the preset distance threshold, no feedback resistance is output. Furthermore, the distance between the endoscope 300 and the motion boundary can be divided into multiple intervals, with different feedback resistance output by the operating lever 420 depending on whether the distance falls within those intervals. Additionally, the process of determining the "relative position of the endoscope 300 to the motion boundary" can be performed in the robotic arm coordinate system or transformed to other coordinate systems, such as the medical imaging coordinate system, navigation coordinate system, endoscope coordinate system, first mapping reference point coordinate system, or simulation coordinate system.

[0171] Based on any embodiment, in some embodiments, step S3 includes:

[0172] S31. Based on the simulation connection, map the motion boundary in the robot arm coordinate system to the simulation coordinate system to obtain the mapped boundary;

[0173] S32. Based on the operating lever 420 and the mapping boundary in the simulation coordinate system, determine whether to output feedback resistance.

[0174] Specifically, this embodiment maps the motion boundary in the mechanical coordinate system to the simulation coordinate system based on the simulation connection, obtaining the mapped boundary. Then, it quickly determines whether to output resistance feedback directly in the simulation coordinate system based on the position of the control lever 420 relative to the mapped boundary, resulting in a more timely response and improved safety. Furthermore, combined with the aforementioned embodiments requiring the re-establishment of the simulation connection for actions such as "changing the scaling factor," "entering translation mode," and "entering rotation mode," the mapped boundary in the simulation coordinate system is recalculated based on the new simulation connection, avoiding judgment errors. It is understood that after performing an operation in translation / rotation mode and exiting that mode, it is equivalent to changing the scaling factor again, requiring an update of the simulation connection and recalculation of the mapped boundary in the simulation coordinate system. Optionally, when "entering translation mode" or "entering rotation mode," the mapped boundary can be temporarily not updated. In translation mode, the change in the control lever angle is masked; in rotation mode, the change in the control lever displacement is masked. Based on the remaining changes, a position and attitude of a "virtual control lever" is obtained. The relative position of this "virtual control lever" and the unupdated motion boundary is used to determine whether to output resistance feedback. In another embodiment, when "entering translation mode" or "entering rotation mode", feedback resistance can be directly determined in the robotic arm coordinate system based on the relative position of the endoscope after pose adjustment and the motion boundary.

[0175] Based on the previous embodiment, in some embodiments, step S31 includes:

[0176] Based on the motion boundary in the robotic arm coordinate system and the pose of the first mapping reference point in the endoscope 300 in the robotic arm coordinate system, the motion boundary of the endoscope in the first mapping reference point coordinate system is obtained.

[0177] Based on the motion boundary and scaling factor in the first mapping reference point coordinate system, the mapping boundary in the joystick coordinate system is obtained;

[0178] Based on the pose of the joystick in the simulation coordinate system and the mapping boundary in the joystick coordinate system, the mapping boundary in the simulation coordinate system is obtained.

[0179] Specifically, based on the initial pose of the robotic arm 200, the fixed connection between the end effector of the robotic arm 200 and the endoscope 300, and the pose of the mapping reference point in the endoscope 300, the motion boundary in the first mapping reference point coordinate system (that is, the relative positional relationship between the motion boundary and the mapping reference point) can be obtained. Then, combined with the scaling factor, the motion boundary in the first mapping reference point coordinate system is "inversely scaled" to the joystick coordinate system to obtain the mapping boundary in the joystick coordinate system. Combined with the initial pose of the joystick 420 in the simulation coordinate system, the mapping boundary in the simulation coordinate system can be obtained. Of course, the mapping boundary can also be calculated based on the pose of the robotic arm 200 at any time and the pose of the joystick 420 at that time, and the mapping boundary can be transformed to the simulation coordinate system for subsequent judgment processes.

[0180] Understandably, without re-establishing the simulation connection or changing the scaling factor, the mapping boundary in the simulation coordinate system is fixed. This mapping boundary does not need to be recalculated. In the subsequent adjustment of the endoscope 300, the distance between the endoscope 300 and the boundary can be determined directly based on the calculated mapping boundary and the pose of the operating lever 420, which can improve the efficiency of judging the output feedback resistance.

[0181] Based on any embodiment, in some embodiments, the method further includes: obtaining a mapping relationship between the medical image coordinate system and the simulation coordinate system according to the simulation connection; mapping the motion boundary of the medical image coordinate system to the simulation coordinate system to obtain a mapped boundary; and outputting corresponding force feedback based on the distance between the control lever and the mapped boundary.

[0182] Specifically, unlike the previous two embodiments, this embodiment does not require determining the position of the endoscope and the motion boundary, nor does it require determining the mapping boundary through the endoscope. Since a simulation connection is established, there is a pairwise relationship between points in the medical image coordinate system and points in the simulation coordinate system. This embodiment directly determines the mapping boundary (i.e., the range of motion of the control lever) in the simulation coordinate system based on the motion boundary of the medical image coordinate system combined with the simulation connection, and directly determines whether to output resistance feedback based on the positional relationship between the control lever and the mapping boundary.

[0183] Based on any embodiment, in one embodiment, the method further includes providing alarm information via an output module when the endoscope 300 reaches the operating boundary.

[0184] Specifically, this can be achieved by, for example, displaying character prompts / distance value prompts on a monitor, displaying flashing prompts via indicator lights, or controlling a speaker to output audible prompts. In conjunction with the aforementioned embodiment that "divides the distance between the endoscope and the moving boundary into multiple intervals," different alarm messages are output when the distance between the endoscope 300 and the moving boundary falls within different intervals. For example, the speaker outputs prompts at different frequencies, the indicator lights output different colors of light / flash at different frequencies, etc.

[0185] The present invention also provides an endoscope robot, which can be understood in conjunction with the endoscope robot described above.

[0186] Figure 3 This is a system structure diagram of another endoscopic robot provided by the present invention, as shown below. Figure 3 As shown, the endoscopic robot includes: a processing module 100, a robotic arm 200, an endoscope 300, a simulation module 400, and a navigation module 500.

[0187] One end of the robotic arm 200 is fixed, for example, to the ground, a hospital bed, a trolley, a wall, a ceiling, etc. The robotic arm 200 can be a robotic arm with 4 degrees of freedom or more, and can adjust the spatial pose of the endoscope 300 connected to its other end.

[0188] Endoscope 300 is a rigid endoscope. The robotic arm 200 can maintain the endoscope 300 in the required position and posture, and acquire images of the target area of ​​interest to the doctor from the appropriate position and angle without hindering the doctor's operation. The robotic arm 200 can stably maintain the position and posture of the endoscope 300 for a long time, and can also adjust the position and posture of the endoscope 300 under the control of medical staff when the target area of ​​interest to the doctor changes.

[0189] The simulation module 400 includes a base 410 and an operating joystick 420. The base 410 can be fixed to the ground, a hospital bed, a table, a trolley, or other locations. The operating joystick 420 can adjust its posture relative to the base 410, and its spatial pose can be detected by the base 410. More structural details of the simulation module 400 can be found above.

[0190] The processing module 100 is communicatively connected to the robotic arm 200 and the simulation module 400 (either via wired or wireless connection). The processing module 100 employs a general computer architecture, including a memory and a processor. The processor executes the computer program stored in the memory to adjust the pose of the endoscope 300. Specifically, the processing module 100 establishes a simulation connection between the endoscope 300 and the operating joystick 420 based on their initial poses, ensuring a correspondence between the joystick's position and the endoscope's position. Medical personnel can then adjust the endoscope 300 using the joystick 420. The processing module 100 generates control commands based on the actual pose (pose change) of the joystick 420 and sends these commands to the robotic arm 200. The robotic arm 200 then moves according to the control commands, adjusting the endoscope 300 to the corresponding pose.

[0191] The navigation module 500 can be an electromagnetic navigation module, in which case the positioning mark 310 is an electromagnetic positioning mark. The navigation module 500 can also be an optical navigation module, in which case the positioning mark 310 is an active or passive optical mark. Figure 3 The illustration only shows the state where the positioning mark is set on the endoscope 300. The navigation module 500 can determine the spatial position of the endoscope 300 by tracking the positioning mark 310. The positioning mark 310 can also be set at the end of the robotic arm. Since there is a fixed connection between the end of the robotic arm and the endoscope 300, the spatial pose of the endoscope 300 can be calculated accordingly.

[0192] In a typical use case, the navigation module tracks the position of the endoscope and displays the endoscope and the target position (such as the area to be observed or the area to be avoided) on the display device. After observing the relative position of the endoscope and the target position, the medical staff determines the orientation of the endoscope that needs to be adjusted, and controls the movement of the robotic arm through the joystick of the simulation module to adjust the endoscope to the required position.

[0193] The endoscopic robot provided in this embodiment improves the positioning accuracy of the endoscope through a navigation module and can also be combined with other functions of surgical navigation, such as tracking surgical instruments and displaying them together with a 3D model of the patient on a display device. The robotic arm can accurately adjust the endoscope to the position indicated by the control lever, and the robotic arm can hold the endoscope stably for a long time, reducing the workload of doctors / nurses. The simulation module adjusts the endoscope without occupying surgical space near the patient, giving doctors more freedom of operation.

[0194] It should also be noted that the various components of the endoscopic robot described above can be independent or partially or completely integrated together. For example, the simulation module base 410 and the robotic arm base can be integrated into the operating trolley, and the processing module 100 can be placed inside the operating trolley. Alternatively, the processing module 100 can be integrated into the robotic arm base, and the simulation module 400 and the navigation module 500 can be set up independently. Or, the processing module 100 can be integrated into the navigation module 500.

[0195] Based on the previous embodiment, in one embodiment, establishing a simulated connection between the endoscope 300 and the operating lever 420 includes:

[0196] Obtain the command to establish a simulation connection;

[0197] Determine the initial pose of the endoscope in the navigation coordinate system;

[0198] Determine the initial pose of the control lever in the simulation coordinate system;

[0199] A simulation connection is established based on the initial pose of the endoscope in the navigation coordinate system and the initial pose of the control lever in the simulation coordinate system.

[0200] Specifically, the command to establish the simulation connection can be input by the user in any form, such as via a button on the joystick 420, a keyboard, mouse, or foot switch, voice input and recognition, or gesture recognition. The initial pose of the endoscope 300 when the simulation connection is established can be determined by the navigation module 500. This pose of the endoscope 300 is used as the basis for the simulation mapping, and a simulation mapping is established with the initial pose of the joystick 420 of the simulation module 400. In subsequent processes, changes in the pose of the joystick 420 will cause changes in the pose of the endoscope 300, and the poses of the two have a one-to-one correspondence.

[0201] In this embodiment, after obtaining the instruction to establish a simulation mapping, a simulation connection is established based on the initial posture of the endoscope in the navigation coordinate system and the initial posture of the control stick in the simulation coordinate system. The initial state of the simulation mapping is recorded, which facilitates the subsequent mapping of the position changes of the control stick to the endoscope.

[0202] Based on the previous embodiment, in one embodiment, establishing a simulation connection based on the initial pose of the endoscope 300 in the navigation coordinate system and the initial pose of the control stick in the simulation coordinate system includes:

[0203] Based on the initial pose of the endoscope in the navigation coordinate system and the first mapping reference point in the endoscope, determine the initial pose of the first mapping reference point in the navigation coordinate system.

[0204] Based on the initial pose of the joystick in the simulation coordinate system and the second mapping reference point in the joystick, determine the initial pose of the second mapping reference point in the simulation coordinate system.

[0205] The initial pose of the first mapping reference point in the navigation coordinate system and the initial pose of the second mapping reference point in the simulation coordinate system serve as the initial state of the simulation connection.

[0206] Specifically, a first mapping reference point is determined in the endoscope, and a second mapping reference point is determined in the control lever. Since the initial pose of the endoscope in the navigation coordinate system is known, the pose (including orientation) of the first mapping reference point in the endoscope can also be determined. Therefore, the initial pose of the first mapping reference point in the navigation coordinate system can be determined, and its actual pose can describe the subsequent pose changes of the endoscope. Similarly, the initial pose (including orientation) of the second mapping reference point in the simulation coordinate system can also be determined, and its actual pose can describe the subsequent pose changes of the control lever. Using the initial poses of these two mapping reference points as the initial state of the simulation connection facilitates the subsequent mapping of the control lever's pose changes to the endoscope.

[0207] This embodiment defines a first mapping reference point in the endoscope and a second mapping reference point in the control lever, which facilitates the accurate mapping of the control lever's pose changes to the endoscope.

[0208] Based on the previous embodiment, in one embodiment, the transformation relationship between the robotic arm coordinate system and the navigation coordinate system is pre-defined, and the method executed by the processing module further includes:

[0209] Based on the transformation relationship between the robotic arm coordinate system and the navigation coordinate system, the initial pose of the first mapping reference point in the navigation coordinate system is transformed to the robotic arm coordinate system;

[0210] The initial pose of the first mapping reference point in the robot arm coordinate system is also used as the initial state of the simulation connection.

[0211] Specifically, the method of "pre-calibrating the transformation relationship between the robotic arm coordinate system and the navigation coordinate system" can be flexibly selected. For example, a navigation probe can be installed at the end of the robotic arm to collect three or more feature points in the surgical space. The transformation relationship between the two coordinate systems can be calculated based on the spatial coordinates of these feature points in the robotic arm coordinate system and the navigation coordinate system, respectively. Another example is to install a reference frame of known shape at the end of the robotic arm. The connection relationship between the reference frame and the end of the robotic arm is known, that is, the position of the reference frame in the robotic arm coordinate system is known. The reference frame can also be tracked by the navigation module, and the transformation relationship between the two coordinate systems can be calculated accordingly.

[0212] Due to the pre-defined transformation relationship between the robotic arm coordinate system and the navigation coordinate system, the initial pose of the first mapping reference point in the navigation coordinate system can also be transformed to the robotic arm coordinate system. The initial poses of the second mapping reference point in the simulation coordinate system, the first mapping reference point in the navigation coordinate system, and the first mapping reference point in the robotic arm coordinate system are recorded as the initial state of the simulation connection. Subsequently, changes in the pose of the control lever (i.e., changes in the pose of the first mapping reference point) will cause changes in the pose of the second mapping reference point. Specifically, the change in the pose of the second mapping reference point in the robotic arm coordinate system is used to calculate the target pose of the robotic arm and plan its motion trajectory, while the change in the pose of the second mapping reference point in the navigation coordinate system is used to verify whether the endoscope has moved to the required position.

[0213] In another embodiment, the fixed connection between the robotic arm end effector and the endoscope is pre-calibrated, and the method executed by the processing module further includes:

[0214] The fixed connection between the robotic arm's end effector and the endoscope, along with the initial pose of the first mapping reference point in the navigation coordinate system, yields the initial pose of the first mapping reference point in the robotic arm's coordinate system.

[0215] The initial pose of the first mapping reference point in the robot arm coordinate system is also used as the initial state of the simulation connection.

[0216] Similar to the previous embodiment, the initial pose of the first mapping reference point in the navigation coordinate system is transformed to the robot arm coordinate system as part of the "initial state of the simulation connection". The first mapping precision point in the robot arm coordinate system can guide the robot arm movement faster and more accurately, and adjust the endoscope to the position indicated by the control lever.

[0217] Based on the first two embodiments, motion boundaries can also be determined for the endoscope. Specifically, motion boundaries can be set in medical images containing a 3D model of the patient, or they can be determined based on a navigation coordinate system. When the distance between the endoscope and the motion boundary is less than a preset threshold, feedback resistance can be output through a joystick, or through a robotic arm (i.e., the robotic arm will no longer execute the adjustment action indicated by the joystick, preventing the endoscope at the end of the robotic arm from reaching the motion boundary), or information can be output through a display module. Of course, the above-mentioned motion boundaries can also be converted to the robotic arm coordinate system, so that the endoscope can be adjusted to reach or exceed the motion boundary during the robotic arm motion planning stage, thus improving safety.

[0218] In another embodiment, it is not necessary to calibrate the transformation relationship between the robotic arm coordinate system and the navigation coordinate system, nor is it necessary to calibrate the connection relationship between the robotic arm end effector and the endoscope (if the reference frame is set at the end effector, this fixed connection relationship still needs to be calibrated). In this case, the navigation module is used to obtain the current position and attitude of the endoscope. The user judges the difference between the endoscope and the desired attitude based on this, and continuously controls the robotic arm to adjust the position and attitude of the endoscope through the simulation module until the endoscope is adjusted to the desired attitude. Furthermore, the motion boundary of the endoscope in the navigation coordinate system is also determined. If it is determined that the distance between the endoscope and the motion boundary in the navigation coordinate system is less than a preset threshold, an information prompt is output, or feedback resistance is output through the control stick.

[0219] Further embodiments of the endoscopic robot provided by this invention can be found in the above description of the endoscopic robot, and will not be repeated here.

[0220] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0221] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0222] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pose adjustment method of an endoscopic robot, characterized by, The application relates to a method for establishing a simulation connection between a mechanical arm connected with an endoscope and a simulation module. The method comprises the following steps: establishing the simulation connection between the mechanical arm connected with the endoscope and the simulation module in initial postures of the mechanical arm connected with the endoscope and the simulation module; wherein the simulation module comprises a base and an operating rod, and the position and posture of the operating rod have a corresponding relationship with the position and posture of the endoscope in the simulation connection state; 2. The pose adjustment method of an endoscopic robot according to claim 1, characterized in that, controlling the movement of the mechanical arm according to the posture change of the operating rod to adjust the position and posture of the endoscope. The method for establishing the simulation connection between the mechanical arm connected with the endoscope and the simulation module in initial postures of the mechanical arm connected with the endoscope and the simulation module comprises the following steps: obtaining an instruction for establishing the simulation connection; determining the initial posture of the endoscope in the mechanical arm coordinate system according to the initial posture of the mechanical arm and the fixed connection relationship between the end of the mechanical arm and the endoscope; determining the initial posture of the operating rod in the simulation coordinate system; 3. The pose adjustment method of an endoscopic robot according to claim 2, characterized in that, establishing the simulation connection according to the initial posture of the endoscope in the mechanical arm coordinate system and the initial posture of the operating rod in the simulation coordinate system. The method for establishing the simulation connection according to the initial posture of the endoscope in the mechanical arm coordinate system and the initial posture of the operating rod in the simulation coordinate system comprises the following steps: determining the initial posture of a first mapping reference point in the mechanical arm coordinate system according to the initial posture of the endoscope in the mechanical arm coordinate system and the first mapping reference point in the endoscope; determining the initial posture of a second mapping reference point in the simulation coordinate system according to the initial posture of the operating rod in the simulation coordinate system and the second mapping reference point in the operating rod; 4. The pose adjustment method of an endoscopic robot according to claim 1, characterized by, the initial posture of the first mapping reference point in the mechanical arm coordinate system and the initial posture of the second mapping reference point in the simulation coordinate system are used as the initial state of the simulation connection. The method for controlling the movement of the mechanical arm according to the posture change of the operating rod to adjust the position and posture of the endoscope comprises the following steps: obtaining the current posture of the operating rod in the simulation coordinate system; obtaining the posture change of the operating rod in the simulation coordinate system according to the current posture and the initial posture of the operating rod in the simulation coordinate system; scaling the posture change of the operating rod according to a scaling coefficient to obtain the posture change of the endoscope in the mechanical arm coordinate system; 5. The pose adjustment method of an endoscopic robot according to claim 4, characterized in that, determining the target posture of the end of the mechanical arm according to the posture change of the endoscope in the mechanical arm coordinate system and controlling the movement of the mechanical arm to the target posture. The method for scaling the posture change of the operating rod according to a scaling coefficient to obtain the posture change of the endoscope in the mechanical arm coordinate system comprises the following steps: decomposing the posture change of the operating rod into a displacement change amount and an angle change amount; scaling the displacement change amount according to a displacement scaling coefficient and scaling the angle change amount according to an angle scaling coefficient; obtaining the posture change of the endoscope in the mechanical arm coordinate system according to the scaled displacement change amount and the angle change amount.

6. The pose adjustment method of an endoscopic robot according to claim 5, characterized in that, In the case that the user selects the translation mode, the angle variation of the operating rod is shielded; in the case that the user selects the rotation mode, the displacement variation of the operating rod is shielded.

7. The pose adjustment method of an endoscopic robot according to claim 4, characterized in that, The scaling coefficient is adjustable.

8. The pose adjustment method of an endoscopic robot according to claim 7, characterized in that, The method further comprises: According to the operation of the user on the operating rod, the endoscope is moved to the vicinity of the target position with a first scaling coefficient; According to the treatment input by the user, the first scaling coefficient is adjusted to a second scaling coefficient; According to the operation of the user on the operating rod, the endoscope is moved to the target position with the second scaling coefficient.

9. The pose adjustment method of an endoscopic robot according to claim 1, characterized in that, The movement boundary of the endoscope is predetermined, and the method further comprises: According to the relative position of the endoscope and the movement boundary, it is determined whether to output a feedback resistance through the operating rod.

10. The pose adjustment method of an endoscopic robot according to claim 9, characterized in that, The method further comprises: According to the simulation connection, the movement boundary in the mechanical arm coordinate system is mapped to the simulation coordinate system to obtain a mapped boundary; According to the operating rod in the simulation coordinate system and the mapped boundary, it is determined whether to output a feedback resistance.

11. The pose adjustment method of an endoscopic robot according to claim 10, characterized in that, The method further comprises: According to the movement boundary in the mechanical arm coordinate system and the pose of the endoscope in the mechanical arm coordinate system, the movement boundary of the endoscope in a first mapped reference point coordinate system in the endoscope is obtained; According to the movement boundary in the first mapped reference point coordinate system and a scaling coefficient, a mapped boundary in the operating rod coordinate system is obtained; According to the pose of the operating rod in the simulation coordinate system and the mapped boundary in the operating rod coordinate system, a mapped boundary in the simulation coordinate system is obtained.

12. The pose adjustment method of an endoscopic robot according to any one of claims 9-11, characterized in that, The closer the endoscope is to the movement boundary, the greater the feedback resistance output by the operating rod; or, a segmented interval is set for the distance between the endoscope and the movement boundary, and the feedback resistance output by the operating rod is different in different segmented intervals.

13. The pose adjustment method of an endoscopic robot according to claim 9, wherein, When the endoscope reaches the movement boundary, an alarm information is further provided through the output module.

14. The pose adjustment method of an endoscopic robot according to claim 1, characterized by, The method further comprises: According to the simulation connection, a mapping relationship between the medical image coordinate system and the simulation coordinate system is obtained, the movement boundary of the medical image coordinate system is mapped to the simulation coordinate system to obtain a mapped boundary, and a corresponding force feedback is output according to the distance between the operating rod and the mapped boundary.

15. An endoscopic robot, comprising: The method further comprises: A processing module, a mechanical arm, and a simulation module; The mechanical arm is connected to the endoscope at the end thereof, and is used to adjust the pose of the endoscope; The simulation module comprises a base and an operating rod, and the base is capable of detecting the pose of the operating rod; The processing module comprises a memory and a processor, and the processor implements the pose adjustment method of the endoscope robot according to any one of claims 1 to 13 when executing a computer program stored in the memory, controls the movement of the mechanical arm according to the pose of the operating rod, and adjusts the pose of the endoscope.

16. The endoscope robot according to claim 15, wherein The base comprises a wireless detection module, and the operating rod is provided with a wireless positioning module; the wireless detection module determines the spatial pose of the operating rod through the wireless positioning module; or, The simulation module comprises a base and a telescopic operating rod; the orientation of the operating rod on the base is adjustable; the base determines the spatial pose of the operating rod by detecting the orientation of the operating rod and the telescopic amount of the operating rod; or, The simulation module comprises a connecting rod connected to the base; the operating rod is the last rod body of the connecting rod; the base determines the spatial pose of the operating rod by detecting the angle of each rod body.

17. The endoscopic robotic system of claim 15, wherein, Further comprising a navigation module; the mechanical arm terminal segment or the endoscope is provided with a positioning marker; the navigation module tracks the positioning marker to determine the position of the endoscope in a navigation coordinate system.

18. An endoscopic robot, comprising: Comprise: a processing module, a mechanical arm, a simulation module, and a navigation module; The simulation module comprises a base and an operating rod; the mechanical arm terminal is connected to an endoscope; the mechanical arm terminal segment or the endoscope is provided with a positioning marker; the navigation module tracks the positioning marker to determine the position and posture of the endoscope in a navigation coordinate system; The processing module can execute the following method: Establish a simulation connection between the connected endoscope and the operating rod, so that the position and posture of the operating rod have a corresponding relationship with the position and posture of the endoscope; According to the change of the position and posture of the operating rod, control the movement of the mechanical arm to adjust the position and posture of the endoscope.

19. The endoscopic robotic system of claim 18, wherein, The establishment of the simulation connection between the connected endoscope and the operating rod comprises: Obtain an instruction for establishing a simulation connection; Determine the initial position and posture of the endoscope in a navigation coordinate system; Determine the initial position and posture of the operating rod in a simulation coordinate system; According to the initial position and posture of the endoscope in the navigation coordinate system and the initial position and posture of the operating rod in the simulation coordinate system, establish a simulation connection.

20. The endoscopic robotic system of claim 19, wherein, According to the initial position and posture of the endoscope in the navigation coordinate system and the initial position and posture of the operating rod in the simulation coordinate system, establish a simulation connection comprises: According to the initial position and posture of the endoscope in the navigation coordinate system and a first mapping reference point in the endoscope, determine the initial position and posture of the first mapping reference point in the navigation coordinate system; According to the initial position and posture of the operating rod in the simulation coordinate system and a second mapping reference point in the operating rod, determine the initial position and posture of the second mapping reference point in the simulation coordinate system; The initial position and posture of the first mapping reference point in the navigation coordinate system and the initial position and posture of the second mapping reference point in the simulation coordinate system are used as the initial state of the simulation connection.

Images