Endoscope robot

Abstract

The invention provides an endoscope robot. The endoscope robot comprises a mechanical arm, an endoscope and a processing module. The tail section of the mechanical arm is connected with the endoscope; the processing module judges whether to output a corresponding prompt signal or not according to the distance between the position of the endoscope and a preset boundary. The endoscope is connected to the tail end of the mechanical arm, the position and angle of the endoscope are maintained and adjusted through the mechanical arm, the requirement for the posture of the endoscope at the position for a long time and the requirement for adjusting the posture of the endoscope are met, and when the distance between the endoscope and the preset boundary is smaller than the distance threshold value, the processing module controls and outputs a prompt signal. And the safety in the process of adjusting the pose of the endoscope is improved.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to an endoscopic robot. Background Technology

[0002] Neurosurgical endoscopy is a minimally invasive surgical technique that uses an endoscope to access the brain or spinal cord through a small incision to treat various neurological diseases. Endoscopic surgery requires the cooperation of two surgeons: one performs the surgical procedure, while the other holds the endoscope and adjusts its angle. Both surgeons need extensive training to achieve seamless coordination. The instability of the endoscope, especially during prolonged periods, places a significant workload on the surgeon and further degrades its stability.

[0003] Currently, existing technologies offer several methods for fixing endoscopes, such as securing the endoscope to the end of an articulated arm or robotic arm. While these methods can stably fix the endoscope for extended periods, adjusting its position / angle typically requires resetting the articulated arm or robotic arm and reinstalling the endoscope. This adjustment process is complex and time-consuming. Furthermore, directly adjusting the robotic arm's position without disassembling the endoscope can easily cause the endoscope to compress or collide with human tissue, posing a risk of injury.

[0004] To address the shortcomings of existing technologies where adjusting the position / angle of the endoscope is complex and can easily cause accidental injury to the patient, this invention provides an endoscopic robot. Summary of the Invention

[0005] This invention provides an endoscopic robot to address the shortcomings of existing technologies where the process of adjusting the position / angle of the endoscope is complex and can easily cause accidental injury to the patient.

[0006] This invention provides an endoscope robot, comprising: a robotic arm, an endoscope, and a processing module;

[0007] The end of the robotic arm is connected to the endoscope;

[0008] The processing module determines whether to output a corresponding prompt signal based on the distance between the position of the endoscope and the preset boundary.

[0009] Optionally, the processing module determines the position of the endoscope in the robotic arm coordinate system based on the posture of the robotic arm and the fixed connection between the end of the robotic arm and the endoscope.

[0010] Optionally, the processing module transforms the preset boundary in the medical image coordinate system to the robotic arm coordinate system based on the registration relationship, thereby obtaining the preset boundary in the robotic arm coordinate system.

[0011] Optionally, the robotic arm provides a follow-up working mode, which can detect the force applied by the user and adjust the posture of the robotic arm according to the force applied by the user.

[0012] Furthermore, the robotic arm executes the following process in the follow-up working mode:

[0013] The force applied by the user is obtained by using a force sensor installed on the robotic arm;

[0014] The user's control intention is determined based on the force applied by the user;

[0015] The admittance control parameters of the robotic arm are determined based on the user's control objectives;

[0016] The movement of the robotic arm is controlled based on the admittance control parameters of the robotic arm and the force applied by the user.

[0017] Optionally, the force sensor acquires signals at a preset frequency to obtain the force applied by the user in sequence.

[0018] Furthermore, determining the user's control intention based on the force applied by the user includes:

[0019] The user-applied forces in the sequence form are preprocessed and feature extracted; wherein the features include at least two of the following metrics: average force, force variation, force standard deviation, and force variance.

[0020] Based on the extracted features, analyze the user's control purpose.

[0021] Furthermore, the step of analyzing the user's control intent based on the extracted features includes:

[0022] The features are input into a pre-trained control purpose recognition model to obtain the user's control purpose.

[0023] Optionally, analyzing the user's control intent based on the extracted features includes:

[0024] If the average force is greater than the first average force threshold and less than the second average force threshold, the standard deviation of the force is greater than the first standard deviation threshold, and the change in force is less than the first change threshold, then the user's control purpose is determined to be a pre-adjustment.

[0025] If the average force is greater than the first average force threshold and less than the second average force threshold, the standard deviation of the force is greater than the first standard deviation threshold, and the change in force is greater than the first change threshold, or if the average force is greater than the second average force threshold, the standard deviation of the force is less than the second standard deviation threshold, and the change in force is less than the second change threshold, then the user's control purpose is determined to be a coarse adjustment.

[0026] If the direction of the force changes, the average force value is greater than the second average force threshold, and the standard deviation of the force is greater than the second standard deviation threshold; or, if the direction of the force changes, the average force value is greater than the second average force threshold, and the change in force is greater than the second change threshold, then the user's control objective is determined to be direction correction.

[0027] If the average force is greater than the first average force threshold, less than the second average force threshold, and the standard deviation of the force is less than the first standard deviation threshold, then the user's control objective is determined to be fine adjustment.

[0028] If the average force is less than the first average force threshold, then the user's control objective is determined to be the end of the adjustment.

[0029] Optionally, the robotic arm provides resistance feedback when the distance between the endoscope and the preset boundary is less than a distance threshold.

[0030] Furthermore, the resistance feedback provided by the robotic arm is also combined with the movement direction setting of the endoscope.

[0031] Optionally, it also includes a simulation module that detects the attitude changes of its end effector lever and, in conjunction with simulation mapping, controls the movement of the robotic arm to adjust the attitude of the endoscope.

[0032] Furthermore, when the processing module detects that the distance between the endoscope and the preset boundary is less than a distance threshold, it controls the simulation module to output resistance feedback.

[0033] Optionally, it also includes a navigation module, wherein the end segment of the robotic arm or the endoscope is provided with a positioning mark, and the navigation module tracks the positioning mark to determine the position of the endoscope in the navigation coordinate system, for determining whether the distance between the endoscope and the preset boundary is less than a distance threshold.

[0034] Furthermore, it also includes a navigation probe, the navigation module detects the pose of the navigation probe and sends it to the processing module;

[0035] The processing module controls the movement of the robotic arm based on the pose of the navigation probe in order to adjust the pose of the endoscope.

[0036] Optionally, the endoscopic robot also includes a display module for displaying preset boundaries in the patient model and the current position of the endoscope, and / or, the display module for outputting prompt information.

[0037] The endoscopic robot provided by this invention has at least the following beneficial effects:

[0038] 1. The endoscope is connected to the end of the robotic arm. The robotic arm maintains and adjusts the position and angle of the endoscope, which meets the needs of long-term endoscope posture and the need to adjust the endoscope position. The processing module also controls the output of a prompt signal when the distance between the endoscope and the preset boundary is less than the distance threshold, which improves the safety of the endoscope position adjustment process.

[0039] 3. Users can adjust the endoscope's position by dragging the robotic arm. If the distance between the endoscope and the preset boundary is less than the distance threshold during the adjustment process, resistance feedback will be provided, giving an intuitive warning signal and improving the user's operating experience.

[0040] 4. By using force sensors installed on the robotic arm to collect the force applied by the user and analyzing the user's control purpose, the robotic arm admittance control parameters are adjusted adaptively, making the user's control process of dragging and adjusting the robotic arm more convenient.

[0041] 5. It provides a mode of "adjusting endoscope position through simulation module", which makes operation more flexible and can reduce errors and risks caused by hand tremors. The simulation module can provide force feedback to improve safety.

[0042] 6. It provides a mode of "adjusting endoscope position using navigation probes", which can reduce errors caused by human hand tremors. In addition, the endoscopic robot can reuse the navigation module and navigation probe of the "surgical navigation system", which expands the system functions with a simple structure and reduces the overall cost. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is one of the structural schematic diagrams of the endoscopic robot provided by the present invention;

[0045] Figure 2 This is the second structural schematic diagram of the endoscopic robot provided by the present invention;

[0046] Figure 3This is the third structural schematic diagram of the endoscopic robot provided by the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] The following is combined with Figures 1-3 The endoscopic robot of the present invention is described. Figure 1 This is one of the structural schematic diagrams of an endoscopic robot provided by the present invention, such as... Figure 1 As shown, the endoscopic robot includes: a processing module 100, a robotic arm 200, and an endoscope 300;

[0049] Specifically, the robotic arm 200 can be fixed to a surgical cart, the ground, the wall of a hospital bed, the ceiling, or other locations. The robotic arm 200 comprises multiple articulated arms, with its distal articulated arm connected to the endoscope 300. Since the connection between the distal articulated arm and the endoscope 300 is fixed, the position and orientation of the endoscope 300 connected to the distal articulated arm can be calculated for any given robotic arm posture. The endoscope 300 is a rigid endoscope used for medical testing and treatment, primarily entering sterile tissues and organs or, through surgical incisions, into sterile cavities such as the abdominal cavity and thoracic cavity.

[0050] The processing module 100 adopts a general computer architecture, which may include a memory and a processor. The processor executes the computer program stored in the memory to process data. Specific processing content includes, for example: receiving the posture data of the robotic arm 100 transmitted by the robotic arm 100, and processing it in combination with the installation depth of the endoscope 300 and the data model of the endoscope 300 to obtain the spatial position data of the endoscope 300; or, directly receiving the spatial position data of the endoscope 300 obtained by the robotic arm 100. Then, it determines whether to output a prompt signal based on the spatial position of the endoscope 300 and a preset boundary. If the distance between the endoscope 300 and the preset boundary is less than a distance threshold, a prompt signal is output.

[0051] In a specific endoscopic scenario, the surgeon places an observation sheath at a location on the skull. The endoscope 300 penetrates the sheath to image the target area. The endoscope 300 can move within the sheath, adjusting its position and angle to observe different target areas. In this scenario, the area within the observation sheath constitutes the safe operating range of the endoscope 300, and corresponding movement boundaries are set based on the inner wall of the sheath. More specifically, the position of the movement boundaries can be determined based on the surgical plan. For example, the position of the observation sheath is planned in a patient model. Before the endoscopic surgery, the observation sheath is placed in the corresponding position according to the surgical plan, and the movement boundaries can be set directly based on the observation sheath in the patient model. Alternatively, the position of the movement boundaries can be determined by intraoperative navigation. For example, before the endoscopic surgery, the observation sheath is placed directly at a location on the patient, and its position in the surgical space is determined through 3D imaging, optical navigation, etc., and the movement boundaries are set accordingly. Based on this, by setting a distance threshold, a prompt signal is output when the distance between the endoscope and the preset boundary is less than the threshold, which provides a certain safety margin for the operation process. Of course, the above-mentioned distance threshold can also be set to zero, that is, the prompt signal is only output when the endoscope reaches the preset boundary. In the actual adjustment of the endoscope 300, multiple distance thresholds can also be set. For example, when the distance between the endoscope 300 and the preset boundary is less than the first distance threshold, the display is controlled to output a character prompt and the speaker is controlled to output a sound prompt. When the distance between the endoscope 300 and the preset boundary is less than the second distance threshold, the robotic arm 200 is controlled to output movement resistance or the robotic arm is directly locked (that is, the movement resistance output by the robotic arm is greater than the force that the user can apply).

[0052] This embodiment uses the endoscope fixed by the end of the robotic arm to maintain the endoscope's position and posture for a long time, saving doctors' physical strength. Furthermore, during the adjustment of the endoscope's position and posture, it can detect in real time whether the distance between the endoscope and the preset boundary is less than a distance threshold. If the distance is less than the distance threshold, a prompt will be given to avoid the endoscope colliding with dangerous areas, which greatly improves the safety of the use process.

[0053] Based on any embodiment, in one embodiment, the processing module 100 determines the position of the endoscope 300 in the robotic arm coordinate system according to the posture of the robotic arm 200 and the fixed connection relationship between the end of the robotic arm and the endoscope 300.

[0054] Specifically, the position of the distal articulated arm of the robotic arm 200 can be determined based on its posture. The installation position and depth of the endoscope 300 at the distal end of the robotic arm are known, and the data model of the endoscope 300 is also known. Therefore, the position of the endoscope 300 in the robotic arm coordinate system can be determined. Furthermore, in some embodiments, the endoscope robot can be adapted to various models of endoscopes. For a specific model of endoscope, its installation depth at the distal end of the robotic arm is known, and the data model of the endoscope is also known. That is, the spatial position of that model of endoscope in the robotic arm coordinate system can be calculated.

[0055] Based on any embodiment, in one embodiment, the processing module 100 transforms the preset boundary in the medical image coordinate system to the robotic arm coordinate system based on the registration relationship, thereby obtaining the preset boundary in the robotic arm coordinate system.

[0056] Specifically, the transformation relationship between the robotic arm coordinate system and the medical image coordinate system has been obtained during the preoperative registration stage. For example, a registration probe is installed at the end of the robotic arm, and the registration probe is used to collect registration markers on the patient by adjusting the posture of the robotic arm. These markers are then used for registration in the medical image space to obtain the transformation relationship (i.e., the registration relationship) between the medical image coordinate system and the robotic arm coordinate system. Another example is that a "point cloud acquisition module" is installed on the robotic arm 200 to collect point clouds on the patient's surface. These point clouds are then registered with the patient model in the medical image space to obtain the aforementioned registration relationship.

[0057] In some endoscopic surgical scenarios, when using an endoscope to observe internal human tissues, an observation sheath leading to the target area needs to be implanted first. The endoscope is then inserted into the observation sheath. The observation sheath provides space for the endoscope to move, and the observation port at the distal end of the endoscope can image the target area. Preset boundaries can be set according to the position of the observation sheath. Specifically, the inner wall of the observation sheath can be used as the preset boundary, or a preset boundary can be set with a certain margin. Furthermore, the length of the endoscope is typically 50–500 mm. The endoscope can move not only laterally but also along its length; therefore, boundaries can be set not only based on the inner wall of the observation sheath but also at the distal end of the observation sheath. There are several ways to obtain the position of the observation sheath. For example, the position of the observation sheath can be planned during the preoperative planning stage. During the operation, the position of the observation sheath can be mapped to the surgical space based on the registration relationship. Then, with the help of the navigation system, the observation sheath can be accurately placed in the planned position. Similarly, preset boundaries can also be mapped to the surgical space. Another example is that the surgeon manually places the observation sheath during the operation, and after implantation, medical images are acquired from the patient to determine the position of the observation sheath in the medical image space. Then, the boundaries are manually set based on the observation sheath in the medical image space. Yet another example is that a deep learning model can be used to process the patient's medical image data to extract the observation sheath and set corresponding preset boundaries. Yet another example is that positioning markers are set in the observation sheath. After the surgeon manually places the observation sheath, the navigation system tracks these positioning markers to determine the position of the observation sheath. Alternatively, the observation sheath may have feature points, which can be acquired with the help of the navigation system to determine the position of the observation sheath. After determining the position of the preset boundaries, it can be converted to the robotic arm coordinate system.

[0058] In other endoscopic surgical scenarios, the endoscope is inserted directly into the target area to acquire images, requiring the setting of preset boundaries based on the patient's medical images. For example, in transnasal endoscopic surgery, the nasal cavity is the operating space of the endoscope. A patient model is reconstructed based on the patient's medical images, and the aforementioned preset boundaries are set in the patient model according to the nasal cavity.

[0059] This embodiment transforms the preset boundary in the medical image coordinate system to the robotic arm coordinate system based on the registration relationship, which facilitates the real-time determination of whether the endoscope exceeds the preset boundary in the robotic arm coordinate system based on its position, thereby improving data processing efficiency.

[0060] Based on any embodiment, in one embodiment, the robotic arm 200 provides a follow-up working mode, which can detect the force applied by the user and adjust the posture of the robotic arm 200 according to the force applied by the user.

[0061] Specifically, in the robotic arm follow-up working mode, the user (doctor, nurse, etc.) can drag the robotic arm 200. After detecting the force applied by the user, the robotic arm 200 adjusts its pose according to the force applied by the user (different forces represent different pose adjustment intentions), thus realizing the adjustment of the pose of the endoscope 300. Furthermore, the magnitude of the applied force can be divided into multiple ranges. For example, if the magnitude of the applied force is in the first range, it is determined to be an unexpected disturbance, and the pose of the robotic arm is not adjusted. If the magnitude of the applied force is in the second range, the movement of the robotic arm 200 is adjusted at a "slow speed." If the magnitude of the applied force is in the third range, the movement of the robotic arm 200 is adjusted at a "medium speed." It is understood that the above "slow speed" and "medium speed" are only used to exemplify the relative speed of the robotic arm 200's movement and do not represent a limitation on specific speed values. In addition, it is preferable to control the force-bearing section of the robotic arm to move along the force direction. When it is detected that it is about to encounter an obstacle area, the movement direction of the robotic arm can be determined based on the force direction and obstacle avoidance requirements to avoid encountering the obstacle area (for example, to prevent the endoscope 300 at the end of the robotic arm 200 from crossing the preset boundary).

[0062] This embodiment provides a follow-up working mode for the robotic arm, allowing users to easily adjust the endoscope to the desired position by dragging the robotic arm.

[0063] Based on the previous embodiment, in one embodiment, the robotic arm 200 executes the following process in the follow-up working mode:

[0064] S1. The force applied by the user is obtained by using a force sensor installed on the robotic arm;

[0065] S2. Determine the user's control objective based on the force applied by the user;

[0066] S3. Determine the admittance control parameters of the robotic arm according to the user's control objectives;

[0067] S4. Control the movement of the robotic arm according to the admittance control parameters of the robotic arm and the force applied by the user.

[0068] Specifically, in this embodiment, the force sensor can be a force sensor installed at the end of the robotic arm. The user can change the posture of the robotic arm by applying force to the end of the robotic arm, thereby setting the tool connected to the end of the robotic arm to the desired position and posture. The force sensor can also include force sensors installed at each joint. The user can apply force to the robotic arm at any segment to change the posture of the robotic arm. Of course, both types of force sensors can be installed simultaneously to detect the force applied by the user. We obtain the force applied by the user by installing force sensors on the robotic arm and analyze the user's control purpose. Different control purposes correspond to different robotic arm admittance control parameters. The robotic arm admittance control parameters include the mass parameter m and the damping parameter c. When the set simulated mass m is too large, the robotic arm has a large "inertia", its motion state is not easy to change, and it has a certain shielding effect on the noise in the force. The damping c affects the steady-state speed output by the robotic arm when the user inputs a certain force. The greater the damping, the greater the force that the user needs to input to make the robotic arm reach a certain steady-state speed. In other words, this embodiment can flexibly adjust the admittance control parameters of the robotic arm according to the identified "manipulation purpose", so that the response characteristics of the robotic arm are more in line with the user's current usage needs, and the process of dragging the robotic arm to adjust the endoscope is more convenient.

[0069] Based on the previous embodiment, in some embodiments, the force sensor acquires signals according to a preset frequency, and in S1, it obtains the force applied by the user in a sequential form.

[0070] Specifically, the forces applied by the user are collected in sequence, which facilitates data processing by the computer. Moreover, this embodiment analyzes the user's control intention based on the sequence of forces, rather than the force at a single moment, thus more accurately analyzing the user's control intention.

[0071] Based on the previous embodiment, in some embodiments, S2 includes:

[0072] S21. Preprocess the user-applied forces in sequence form and extract features; wherein the features include at least two of the following indicators: average force, force variation, force standard deviation, and force variance;

[0073] S22. Analyze the user's control purpose based on the extracted features.

[0074] Specifically, the force sequence applied by the user is first preprocessed to improve data quality and enhance the accuracy of the analysis of the user's control intent. Preprocessing may include one or more of the following: outlier removal, missing data imputation, and moving average. The principle of moving average is to select a window of a certain size, calculate the arithmetic mean of the data within the window, and then use the calculated average as the data at the center point of the window. The window is moved according to the point or line spacing, and this averaging method is repeated until the entire graph has undergone the above process. Moving average can smooth the data and reduce high-frequency noise. After preprocessing, feature extraction is further performed. These features characterize the properties of the forces applied by the user over a certain period of time, and these features reflect the user's control intent.

[0075] Based on the previous embodiment, in some embodiments, S22 includes:

[0076] The features are input into a pre-trained control intention recognition model to obtain the user's control intention.

[0077] Specifically, in this embodiment, the features extracted from the force sequence are input into the control purpose recognition model to identify the user's control purpose. The pre-trained model can improve the efficiency and accuracy of the recognition process. Using the above features as training samples can accelerate the convergence speed of the network model during training and improve the accuracy of the prediction results. The control purpose recognition model can be a traditional machine learning model, such as a support vector machine (SVM) or a long short-term memory network (LSTM), or a deep learning model, such as a recurrent neural network (RNN). There are no restrictions here.

[0078] Based on the foregoing embodiments, in some embodiments, S22 includes:

[0079] If the average force is greater than the first average force threshold and less than the second average force threshold, the standard deviation of the force is greater than the first standard deviation threshold, and the change in force is less than the first change threshold, then the user's control purpose is determined to be a pre-adjustment.

[0080] If the average force is greater than the first average force threshold and less than the second average force threshold, the standard deviation of the force is greater than the first standard deviation threshold, and the change in force is greater than the first change threshold, or if the average force is greater than the second average force threshold, the standard deviation of the force is less than the second standard deviation threshold, and the change in force is less than the second change threshold, then the user's control purpose is determined to be a coarse adjustment.

[0081] If the direction of the force changes, the average force value is greater than the second average force threshold, and the standard deviation of the force is greater than the second standard deviation threshold; or, if the direction of the force changes, the average force value is greater than the second average force threshold, and the change in force is greater than the second change threshold, then the user's control objective is determined to be direction correction.

[0082] If the average force is greater than the first average force threshold, less than the second average force threshold, and the standard deviation of the force is less than the first standard deviation threshold, then the user's control objective is determined to be fine adjustment.

[0083] If the average force is less than the first average force threshold, then the user's control objective is determined to be the end of the adjustment.

[0084] Specifically, in the pre-adjustment phase, the user holds the robotic arm or the tool at its end, and the force sensor detects a small and continuously changing force, requiring control of the robotic arm's output speed. In the coarse adjustment phase, the user applies a larger force to pull the robotic arm, which accelerates and maintains a high speed. The force sensor initially detects a rapidly increasing force value and then stabilizes. The admittance control parameters need to be adjusted appropriately based on the magnitude of the force to control the robotic arm to move at a higher average speed by the corresponding offset. Sometimes, the user needs to apply a reverse force to the moving machine or the tool at its end to adjust its posture or change its current motion state, i.e., the direction adjustment phase. In this phase, the force sensor detects a large instantaneous reverse force, and correspondingly, the parameters need to be adjusted to make the robotic arm decelerate and move in the opposite direction in time. In the fine adjustment phase, the tool at the end of the robotic arm is already near the target position, and the user applies a small force to fine-tune the tool's position and posture. Force sensors can collect a sequence of forces that decrease and fluctuate around a small value. It is necessary to control the speed of the robotic arm to gradually decrease and stabilize at a small value. The robotic arm moves at a low speed and exhibits certain damping characteristics to avoid over-adjustment. At the end of the adjustment phase, the tool at the end of the robotic arm reaches the target position, the user's hand is removed from the robotic arm or the tool at the end of the robotic arm, and the contact force disappears. At this time, it is desirable for the robotic arm to maintain its position and posture.

[0085] This embodiment accurately judges the user's control purpose by setting specific analysis strategies and combining them with thresholds that conform to user habits based on experience. This enables the robotic arm to select appropriate admittance control parameters, and the response characteristics of the robotic arm are more in line with the user's real-time usage needs.

[0086] Based on the foregoing embodiments, in some embodiments, S4 includes:

[0087] Based on the admittance control parameters and the force applied by the user, determine the expected motion speed of the robotic arm end effector at the next moment, as well as the expected position and orientation.

[0088] Control the robotic arm to move to the desired position and posture according to the movement speed.

[0089] Specifically, based on the identified "user control purpose," the admittance control parameters of the robotic arm can be determined. Correspondingly, the "force-velocity" functional relationship can be established. Combining this with the force applied by the user to obtain the velocity change, and then with the current velocity of the robotic arm's end effector, the desired motion velocity of the end effector can be determined. Similarly, the "force-displacement" functional relationship can be determined to obtain the displacement change. Combining this with the current pose of the robotic arm's end effector, the desired pose of the end effector can be determined. Based on this, the robotic arm can be controlled to move at the desired velocity to reach that desired pose.

[0090] Based on the previous embodiment, in one embodiment, the robotic arm 200 provides resistance feedback when the distance between the endoscope 300 and a preset boundary is less than a distance threshold.

[0091] Specifically, the resistance provided by the robotic arm 200 can prevent the endoscope 300 from reaching or crossing a preset boundary. In one implementation, the resistance can be set to "provide resistance feedback when the distance between the endoscope and the preset boundary is less than a distance threshold, and the closer the endoscope is to the preset boundary, the greater the resistance provided by the robotic arm 200." In one embodiment, multiple distance thresholds are set. When the distance between the endoscope and the preset boundary is less than a first distance threshold, a "smaller" feedback resistance is provided; when the distance between the endoscope and the preset boundary is less than a second distance threshold, a "larger" feedback resistance is provided; or when the distance is less than the second distance threshold, the robotic arm 200 is directly locked (i.e., the output resistance exceeds the dragging force range that the user can apply to the robotic arm; for example, the motion stop control command provided by the robotic arm itself can be directly invoked to achieve the locking effect).

[0092] Based on the previous embodiment, in one embodiment, the resistance feedback provided by the robotic arm 200 also incorporates the direction of endoscope movement. The feedback resistance can be set to "the closer the endoscope 300 is to the preset boundary, the greater the resistance provided by the robotic arm 200." Alternatively, the resistance can be segmented according to the distance between the endoscope 300 and the preset boundary. For the same position, the resistance provided by the robotic arm 200 when the endoscope 300 is close to the preset boundary is greater than the resistance provided when the endoscope 300 is far from the preset boundary. By combining distance and the direction of endoscope movement to set the resistance feedback, the convenience of adjusting the endoscope 300 can be improved while ensuring safety.

[0093] Reference Figure 2 Based on the previous embodiment, in one embodiment, the endoscope robot further includes a simulation module 400. The simulation module 400 detects the posture changes of its operating lever 420 and, in conjunction with simulation mapping, controls the movement of the robotic arm 200 to adjust the posture of the endoscope 300.

[0094] Specifically, the simulation module 400 includes 410 and a joystick 420. A "simulation mapping" between the simulation module 400 and the endoscope 300 has been established before the operation. The pose change of the joystick 420 will be mapped to the endoscope 300. The specific execution process is as follows: The simulation module 400 obtains the current pose of the joystick 420 and sends the pose to the processing module 100. The processing module 100 calculates the pose change of the joystick 420, or the simulation module 400 directly calculates the pose change of the joystick 420 and sends it to the processing module 100. Then, the processing module 100 maps this pose change to the robotic arm coordinate system according to the simulation mapping, and controls the movement of the robotic arm 200 to adjust the pose of the endoscope 300 so that the pose of the endoscope 300 achieves the "pose change", that is, the pose of the endoscope 300 and the joystick 420 are synchronized.

[0095] Preferably, in the "simulation mapping" state, the joystick 420 is parallel to the endoscope 300, allowing the user to control the joystick 420 more intuitively and reducing the difficulty of adjusting the endoscope 300. Specifically, the "simulation mapping" can be established as follows: After adjusting the joystick 420 to approximately the same orientation as the endoscope 300, the user inputs the command "establish simulation mapping." Upon receiving this command, the processing module 100 establishes the simulation mapping and acquires the position of the joystick 420 in real time, adjusting the position of the endoscope 300 accordingly. Alternatively, the user can input the command "establish simulation mapping" from any orientation of the joystick 420. Upon receiving this command, the processing module 100 controls the robotic arm 200 to adjust the orientation of the endoscope 300 to be the same as the joystick 420, serving as the initial state for the "simulation mapping," after which the simulation control of the endoscope 300 can be executed.

[0096] This embodiment supports guiding the movement of the robotic arm through a simulation module to adjust the posture of the endoscope. The simulation module improves the ease of operation and also filters out the instability of endoscope imaging caused by hand tremors.

[0097] Based on the previous embodiment, in one embodiment, when the processing module 100 detects that the distance between the endoscope 300 and the preset boundary is less than a distance threshold, it controls the simulation module 400 to output resistance.

[0098] Specifically, still refer to Figure 2The simulation module 400 includes a base and multiple operating levers. The final operating lever 420 establishes a simulation mapping with the endoscope 300. When the distance between the endoscope 300 and a preset boundary is detected to be less than a distance threshold, the multiple operating levers output movement resistance to the final operating lever. In another embodiment, the simulation module 400 includes a base and a retractable operating lever connected to the base. The posture (angle) of the operating lever on the base is also adjustable. When the distance between the endoscope 300 and the preset boundary is detected to be less than a distance threshold, resistance is output in the extension and / or angle adjustment direction of the operating lever.

[0099] Reference Figure 2 Based on any embodiment, in one embodiment, a navigation module 500 is further included. The end segment of the robotic arm 200 or the endoscope 300 is provided with a positioning mark 310. The navigation module 500 tracks the positioning mark 310 to determine the position of the endoscope 300 in the navigation coordinate system, and is used to determine whether the distance between the endoscope 300 and the preset boundary is less than a distance threshold.

[0100] Specifically, the navigation module 500 can be an electromagnetic navigation module, in which case the positioning mark 310 is an electromagnetic positioning mark. Alternatively, the navigation module 500 can be an optical navigation module, in which case the positioning mark 310 is an active or passive optical mark. The positioning mark 310 can be placed on the endoscope 300, and the navigation module 500 can determine the spatial position of the endoscope 300 by tracking the positioning mark 310. The positioning mark 310 can also be placed at the end of the robotic arm; since there is a fixed connection between the end of the robotic arm and the endoscope 300, the spatial position of the endoscope 300 can be calculated accordingly.

[0101] In one feasible implementation, a conversion relationship between the navigation coordinate system and the medical imaging coordinate system is established beforehand (i.e., registration is completed). During the actual adjustment of the endoscope 300, the preset boundary in the medical imaging coordinate system can be converted to the navigation coordinate system according to this conversion relationship. In the navigation coordinate system, it is determined whether the distance between the endoscope 300 and the preset boundary is less than a distance threshold. Alternatively, the spatial position of the endoscope 300 can be converted to the medical imaging coordinate system according to this conversion relationship. In the medical imaging coordinate system, it is determined whether the distance between the endoscope 300 and the preset boundary is less than a distance threshold. In another feasible implementation, a conversion relationship between the navigation coordinate system and the robotic arm coordinate system, as well as a conversion relationship between the medical imaging coordinate system, are established beforehand. During the actual adjustment of the endoscope 300, the position of the endoscope 300, determined by the tracking of the navigation module 500, is converted to the robotic arm coordinate system, and the preset boundary in the medical imaging coordinate system is converted to the robotic arm coordinate system. Then, in the robotic arm coordinate system, it is determined whether the distance between the endoscope 300 and the preset boundary is less than a distance threshold. Furthermore, this embodiment can also be combined with the embodiment described above, which is used to apply force to adjust the posture of the robotic arm in the follow-up working mode of the robotic arm. During the process of the user manually adjusting the robotic arm, the navigation module 500 can track the spatial position of the robotic arm end / endoscope in real time.

[0102] In this embodiment, the navigation module tracks the spatial position of the endoscope to determine whether the distance between the endoscope and the preset boundary is less than a distance threshold, thereby reducing the error caused by the robotic arm and improving the adjustment accuracy of the endoscope.

[0103] Based on the previous embodiment, in one embodiment, the endoscopy robot further includes a navigation probe, and the navigation module 500 detects the pose of the navigation probe and sends it to the processing module 100.

[0104] The processing module 100 controls the movement of the robotic arm 200 according to the pose of the navigation probe in order to adjust the pose of the endoscope 300.

[0105] Specifically, in this embodiment, a navigation probe is used to control the position and orientation of the endoscope 300. In one implementation, the navigation probe is placed in the target orientation, the navigation module 500 records the orientation of the navigation probe at this time, and then the navigation probe is removed. The processing module 100 issues a command to control the movement of the robotic arm 200 to adjust the endoscope 300 to the recorded target orientation. In another implementation, a "pose mapping" between the navigation probe and the endoscope is established. In this state, the pose change of the navigation probe will be "mapped" to the endoscope 300. The specific execution process is as follows: the navigation module 500 obtains the pose of the navigation probe and sends the pose to the processing module 100. The processing module 100 controls the movement of the robotic arm 200 accordingly to adjust the pose of the endoscope 300, so that the pose of the endoscope 300 is synchronized with the pose of the navigation probe. As an equivalent variation, after the navigation module 500 obtains the pose of the navigation probe, it further calculates the "pose change" of the navigation probe and sends the "pose change" to the processing module 100. The processing module 100 controls the robotic arm 200 to move and adjust the endoscope 300 to realize the "pose change".

[0106] Preferably, in the mapped state, the navigation probe is parallel to the endoscope 300, allowing the user to adjust the navigation probe more intuitively and reducing the difficulty of adjustment. Specifically, after adjusting the orientation of the navigation probe to be approximately the same as that of the endoscope 300, the user can input the command "Establish pose mapping". Upon receiving the command, the processing module 100 can acquire the pose of the navigation probe in real time and adjust the pose of the endoscope 300 accordingly. Alternatively, the user can input the command "Establish pose mapping" in any orientation of the navigation probe. Upon receiving the command, the processing module 100 controls the robotic arm 200 to move and adjust the orientation of the endoscope 300 to be the same as that of the endoscope 300, serving as the initial state for "pose mapping".

[0107] This embodiment supports adjusting the endoscope's posture by guiding the robotic arm's movement with a probe, which improves operational convenience and filters out endoscopic imaging instability caused by human hand tremors.

[0108] Based on any embodiment, in one embodiment, the endoscopic robot further includes a display module for displaying preset boundaries in the patient model and the current position of the endoscope 300, and / or, the display module for outputting prompt information.

[0109] Specifically, the display module can be a projector, monitor, touchscreen, etc., capable of simultaneously displaying the preset boundary and the current position of the endoscope 300 in the patient model. Users can intuitively understand the relative positions of the two and more accurately adjust the pose of the endoscope 300. Further, in one embodiment, the display module displays the preset boundary and endoscope 300 in the patient model from multiple perspectives; alternatively, the display module includes multiple screens, each displaying the preset boundary and endoscope 300 from a different perspective. The display module can also output prompts, such as displaying text indicating that the boundary has been reached, or color-coding the contact area between the endoscope 300 and the preset boundary to enhance the user's attention to that area.

[0110] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An endoscopic robot, characterized by, The endoscope robot comprises: a mechanical arm, an endoscope, and a processing module; a tail end of the mechanical arm is connected to the endoscope; the processing module determines whether to output a corresponding prompt signal according to a distance between a position of the endoscope and a preset boundary.

2. The endoscope robot according to claim 1, wherein the processing module determines the position of the endoscope in a mechanical arm coordinate system according to a posture of the mechanical arm and a fixed connection relationship between the tail end of the mechanical arm and the endoscope.

3. The endoscopic robot of claim 2, wherein, the processing module converts the preset boundary in a medical image coordinate system to the mechanical arm coordinate system based on a registration relationship to obtain the preset boundary in the mechanical arm coordinate system.

4. The endoscopic robot of claim 1, wherein, the mechanical arm provides a follow-up working mode, can detect a force applied by a user, and adjusts a pose of the mechanical arm according to the force applied by the user.

5. The endoscopic robot of claim 4, wherein, the mechanical arm executes the following process in the follow-up working mode: a force sensor arranged on the mechanical arm is used to acquire the force applied by the user; a control purpose of the user is determined according to the force applied by the user; a mobility control parameter of the mechanical arm is determined according to the control purpose of the user; the mechanical arm is controlled to move according to the mobility control parameter of the mechanical arm and the force applied by the user.

6. The method of claim 5, wherein, the force sensor acquires the force applied by the user in a sequence form according to a preset frequency.

7. The method of claim 6, wherein, the determination of the control purpose of the user according to the force applied by the user comprises: preprocessing and feature extraction are performed on the force applied by the user in the sequence form; wherein the features include at least two of the following indexes: average force, force variation, force standard deviation, and force variance; the control purpose of the user is analyzed according to the extracted features.

8. The method of claim 7, wherein, the analysis of the control purpose of the user according to the extracted features comprises: the features are input into a pre-trained control purpose recognition model to obtain the control purpose of the user.

9. The method of claim 7, wherein, the analysis of the control purpose of the user according to the extracted features comprises: if the average force is greater than a first average force threshold, smaller than a second average force threshold, the force standard deviation is greater than a first standard deviation threshold, and the force variation is smaller than a first variation threshold, it is determined that the control purpose of the user is preliminary adjustment; if the average force is greater than the first average force threshold, smaller than the second average force threshold, the force standard deviation is greater than the first standard deviation threshold, and the force variation is greater than the first variation threshold, or if the average force is greater than the second average force threshold, the force standard deviation is smaller than a second standard deviation threshold, and the force variation is smaller than a second variation threshold, it is determined that the control purpose of the user is rough adjustment; if the direction of the force changes, the average force is greater than the second average force threshold, the force standard deviation is greater than the second standard deviation threshold, or if the direction of the force changes, the average force is greater than the second average force threshold, and the force variation is greater than the second variation threshold, it is determined that the control purpose of the user is direction correction; if the average force is greater than the first average force threshold, smaller than the second average force threshold, and the force standard deviation is smaller than the first standard deviation threshold, it is determined that the control purpose of the user is fine adjustment; if the average force is smaller than the first average force threshold, it is determined that the control purpose of the user is adjustment end.

10. The endoscopic robot of claim 4, wherein, The mechanical arm provides resistance feedback when the distance between the endoscope and the preset boundary is less than a distance threshold.

11. The endoscopic robot of claim 10, wherein, The resistance feedback provided by the mechanical arm is also set in combination with the motion direction of the endoscope.

12. The endoscopic robot of claim 2, wherein, An emulation module is further included, which detects the change in the pose of the end effector thereof and controls the motion of the mechanical arm in combination with an emulation mapping to adjust the pose of the endoscope.

13. The endoscopic robot of claim 12, wherein, The processing module controls the emulation module to output resistance feedback when it detects that the distance between the endoscope and the preset boundary is less than a distance threshold.

14. The endoscopic robot of claim 1, wherein, A navigation module is further included, and the distal end of the mechanical arm or the endoscope is provided with a positioning marker, and the navigation module tracks the positioning marker to determine the position of the endoscope in a navigation coordinate system, which is used to determine whether the distance between the endoscope and the preset boundary is less than a distance threshold.

15. The endoscopic robot of claim 14, wherein, A navigation probe is further included, and the navigation module detects the pose of the navigation probe and sends it to the processing module. The processing module controls the motion of the mechanical arm according to the pose of the navigation probe to adjust the pose of the endoscope.

16. The endoscopic robot of claim 1, wherein, The endoscopic robot further includes a display module, which is used to display the preset boundary and the current position of the endoscope in a patient model, and / or the display module is used to output prompt information.

Images