Endoscopic adjustment methods, devices and electronic equipment

Abstract

This manual provides an endoscope adjustment method, device, and electronic equipment. The endoscope tube includes: a first straight segment at the distal end and a second straight segment within the trocar; a first adjustment unit for adjusting the relative position of the first and second straight segments while maintaining the skew angle between them; a second adjustment unit for adjusting the skew angle between them; and a third adjustment unit for controlling the extension and retraction of the second straight segment within the trocar. This method acquires a set of positions of feature points on each target instrument. When a point in the set is outside the endoscope's field of view, at least one of the first, second, and third adjustment units is automatically adjusted so that all points in the set are within the endoscope's field of view. During surgery, the surgeon does not need to manually adjust the endoscope, thus simplifying the procedure and shortening the operation time.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to endoscope adjustment methods, devices and electronic equipment. Background Technology

[0002] During robotic surgery, the movement of the endoscope or surgical instruments can easily cause them to fall out of the endoscopic field of view, leading to surgical complications. To avoid these complications, surgeons need to frequently switch between endoscopic control mode and instrument control mode, adjusting the endoscope's position in a timely manner to prevent instruments from falling out of the endoscopic field of view. This makes the surgical procedure more complex, longer, and still more prone to complications. Summary of the Invention

[0003] The purpose of this application is to provide an endoscope adjustment method, device, and electronic device to solve the problems of complex surgical procedures, long operation time, and difficulty in avoiding accidents.

[0004] To solve the above-mentioned technical problems, this specification provides a first aspect of an endoscope adjustment method, wherein the endoscope tube extends into the cavity through a hole on a trocar; the endoscope tube includes: a first straight segment located at the end of the tube; a second straight segment fixed in the hole of the trocar; a first adjustment part for adjusting the relative position of the first and second straight segments of the endoscope while keeping the skew angle between the first and second straight segments constant; a second adjustment part for adjusting the skew angle between the first and second straight segments; and a third adjustment part for controlling the telescopic movement of the second straight segment within the trocar; the method includes: acquiring a set of positions of feature points on each target instrument; determining whether each position point in the set is within the field of view of the endoscope; and when there are position points in the set that are not within the field of view, automatically adjusting at least one of the first adjustment part, the second adjustment part, and the third adjustment part so that each position point in the set is within the field of view of the endoscope.

[0005] In some embodiments, when at least one of the first adjustment unit, the second adjustment unit, and the third adjustment unit is automatically adjusted so that all position points in the set are within the field of view of the endoscope, the first adjustment unit, the third adjustment unit, and the second adjustment unit are adjusted sequentially.

[0006] In some embodiments, when adjusting the first adjustment part, the distance between the first projection point and the second projection point is maximized. The first projection point is the projection point of the end of the first straight segment of the endoscope on the target plane, and the second projection point is the projection point of the center point of each position point in the set on the target plane. The target plane is a plane perpendicular to the axis of the second straight segment of the endoscope.

[0007] In some embodiments, the center point of each location point in the set is determined by the following method: when there are two target devices, the midpoint of the line connecting the end positions of the target devices is taken as the center point of each location point in the set; when there are three target devices, the incenter of the triangle formed by the end positions of the target devices is taken as the center point of each location point in the set.

[0008] In some embodiments, when adjusting the third adjustment section, the extension and retraction distance of the third adjustment section is minimized, and each position point in the set is within the swing range of the second adjustment section of the endoscope.

[0009] In some embodiments, when adjusting the second adjustment section, the first straight segment of the endoscope is directed to the center point of each location point in the set.

[0010] In some embodiments, before obtaining the set of locations of feature points on each target instrument, the method further includes: obtaining the target instrument specified by the surgeon through a UI interface.

[0011] In some embodiments, the method further includes: predicting the state parameters of each feature point of the target instrument at the next moment based on images acquired by the endoscope at the current moment and at multiple moments prior to the current moment; the state parameters include position data and motion parameters; inputting the predicted state parameters of the target instrument at the next moment into a pre-trained decision model to obtain output endoscope control parameters; the decision model is used to determine the control parameters of the endoscope based on the state parameters of the target instrument; and adjusting the endoscope according to the endoscope control parameters.

[0012] In some embodiments, the decision model is trained by the following method: acquiring sample trajectory data of the target instrument; the sample trajectory data includes state parameters of multiple location points arranged in chronological order; for each location point in the sample trajectory data, the following operations are performed iteratively: inputting the state data of the current location point into the decision model to obtain the output endoscope control parameters; determining the expected image to be acquired by the endoscope at the next moment based on the endoscope's field of view model; determining the reward value for model training based on the expected image; if the reward value reaches a predetermined condition, taking the next moment after the current moment as the current moment; if the reward value does not reach the predetermined condition, adjusting the parameters of the decision model and re-performing the iterative operation on each location point in the sample trajectory data.

[0013] In some embodiments, determining the reward value for model training based on the predicted image includes: identifying a set of feature point locations of the target device from the predicted image, identifying the set of feature points and determining the size of the anchor frame used for identification; calculating the ratio of the area of ​​the anchor frame to the area of ​​the predicted image, and using the ratio of the areas as the reward value for model training.

[0014] In some embodiments, predicting the state parameters of each feature point of the target instrument at the next moment based on images acquired by the endoscope at the current moment and at multiple moments prior to the current moment includes: acquiring images acquired by the endoscope at the current moment and at multiple moments prior to the current moment; identifying the motion trajectory of the target instrument in each image; and determining the state parameters of the target instrument at the next moment based on the motion trajectory of the target instrument.

[0015] In some embodiments, identifying the motion trajectory of the target device based on each image includes: identifying each device in a first image and a second image at adjacent time points, and determining the size and position of the anchor frame used for identification; determining the appearance feature values ​​of each device in the first image and the second image; calculating the similarity between the appearance feature values ​​of each device in the first image and the appearance feature values ​​of each device in the second image, and using the similarity as a first matching confidence level; calculating the positional deviation values ​​between the anchor frames of each device in the first image and the anchor frames of each device in the second image; determining a second matching confidence level based on the positional deviation values; acquiring the motion parameters of each device at the time of acquisition of the first image and the motion parameters of each device at the time of acquisition of the second image; determining a third matching confidence level between each device in the first image and each device in the second image based on the motion parameters; fusing the first matching confidence level, the second matching confidence level, and the third matching confidence level according to a predetermined ratio to obtain a fourth matching confidence level; matching the device with the highest fourth matching degree in the first image and the second image; and determining the motion trajectory of the target device from the matching results.

[0016] In some embodiments, after matching the device with the highest fourth matching degree in the first image and the second image, the method further includes: if the fourth matching confidence level reaches a predetermined matching degree, treating the matching position in the second image and the matching position in the first image as the same trajectory and assigning them the same trajectory number; if the fourth matching confidence level does not reach the predetermined matching degree, determining that the matching position in the second image and the matching position in the first image are not the same trajectory, and assigning a new trajectory number to the matching position in the second image, wherein the new trajectory number is different from the trajectory number of the matching position in the first image.

[0017] The second aspect of this specification provides an endoscope adjustment device, wherein the endoscope tube extends into a cavity through a hole on a trocar; the endoscope tube includes: a first straight segment located at the end of the tube; a second straight segment fixed in the hole of the trocar; a first adjustment unit for adjusting the relative position of the first and second straight segments of the endoscope while keeping the skew angle between the first and second straight segments constant; a second adjustment unit for adjusting the skew angle between the first and second straight segments; and a third adjustment unit for controlling the telescopic movement of the second straight segment within the trocar; the device includes: a first acquisition unit for acquiring a set of positions of feature points on each target instrument; a judgment unit for judging whether each position point in the set is within the field of view of the endoscope; and a first adjustment unit for automatically adjusting at least one of the first, second, and third adjustment units when there are position points in the set that are not within the field of view, so that each position point in the set is within the field of view of the endoscope.

[0018] In some embodiments, when at least one of the first adjustment unit, the second adjustment unit, and the third adjustment unit is automatically adjusted so that all position points in the set are within the field of view of the endoscope, the first adjustment unit, the third adjustment unit, and the second adjustment unit are adjusted sequentially.

[0019] In some embodiments, when adjusting the first adjustment part, the distance between the first projection point and the second projection point is maximized. The first projection point is the projection point of the end of the first straight segment of the endoscope on the target plane, and the second projection point is the projection point of the center point of each position point in the set on the target plane. The target plane is a plane perpendicular to the axis of the second straight segment of the endoscope.

[0020] In some embodiments, the center point of each location point in the set is determined by the following method: when there are two target devices, the midpoint of the line connecting the end positions of the target devices is taken as the center point of each location point in the set; when there are three target devices, the incenter of the triangle formed by the end positions of the target devices is taken as the center point of each location point in the set.

[0021] In some embodiments, when adjusting the third adjustment section, the extension and retraction distance of the third adjustment section is minimized, and each position point in the set is within the swing range of the second adjustment section of the endoscope.

[0022] In some embodiments, when adjusting the second adjustment section, the first straight segment of the endoscope is directed to the center point of each location point in the set.

[0023] In some embodiments, the apparatus further includes a second acquisition unit for acquiring a target instrument specified by the surgeon through a UI interface.

[0024] In some embodiments, the apparatus further includes: a prediction unit, configured to predict the state parameters of each feature point of the target instrument at the next moment based on images acquired by the endoscope at the current moment and at multiple moments prior to the current moment; the state parameters include position data and motion parameters; a first processing unit, configured to input the predicted values ​​of the state parameters of the target instrument at the next moment into a pre-trained decision model to obtain output endoscope control parameters; the decision model is used to determine the control parameters of the endoscope based on the state parameters of the target instrument; and a second adjustment unit, configured to adjust the endoscope according to the endoscope control parameters.

[0025] In some embodiments, the apparatus further includes a training unit for training the decision model; the training unit includes: a first acquisition subunit for acquiring sample trajectory data of the target instrument; the sample trajectory data includes state parameters of a plurality of position points arranged in chronological order; a first processing subunit, a first determination subunit, a second determination subunit, a third determination subunit, and an adjustment subunit for cyclically performing operations on each position point in the sample trajectory data; wherein, the first processing subunit is used to input the state data of the position point at the current moment into the decision model to obtain output endoscope control parameters; the first determination subunit is used to determine the expected image to be acquired by the endoscope at the next moment based on the endoscope's field of view model; the second determination subunit is used to determine the reward value for model training based on the expected image; the third determination subunit is used to take the next moment of the current moment as the current moment if the reward value reaches a predetermined condition; the adjustment subunit is used to adjust the parameters of the decision model and re-perform the cyclic operation on each position point in the sample trajectory data if the reward value does not reach the predetermined condition.

[0026] In some embodiments, the second determining subunit includes: a first identification subunit, configured to identify a set of feature point locations of the target device from the projected image, identify the set of feature points and determine the size of the anchor frame used for identification; and a first calculation subunit, configured to calculate the ratio of the area of ​​the anchor frame to the area of ​​the projected image, and use the ratio of the areas as a reward value for model training.

[0027] In some embodiments, the prediction unit includes: a second acquisition subunit, configured to acquire images acquired by the endoscope at the current time and at multiple times prior to the current time; a second identification subunit, configured to identify the motion trajectory of the target instrument based on each image; and a fourth determination subunit, configured to determine the state parameters of the target instrument at the next time based on the motion trajectory of the target instrument.

[0028] In some embodiments, the second identification subunit includes: a third identification subunit, configured to identify each device in a first image and a second image at adjacent time points, and determine the size and position of the anchor frame used for identification; a fifth determination subunit, configured to determine the appearance feature values ​​of each device in the first image and the second image; a second calculation subunit, configured to calculate the similarity between the appearance feature values ​​of each device in the first image and the appearance feature values ​​of each device in the second image, and use the similarity as a first matching confidence level; a third calculation subunit, configured to calculate the positional deviation values ​​between the anchor frames of each device in the first image and the anchor frames of each device in the second image; and a sixth determination subunit, configured to determine the positional deviation values ​​based on the positional deviation values. The system comprises the following subunits: a third acquisition subunit, used to acquire the motion parameters of each device at the time of the first image acquisition and the time of the second image acquisition; a seventh determination subunit, used to determine the third matching confidence of each device in the first image and each device in the second image based on the motion parameters; a fusion subunit, used to fuse the first matching confidence, the second matching confidence, and the third matching confidence according to a predetermined ratio to obtain a fourth matching confidence; a matching subunit, used to match the device with the highest fourth matching confidence in the first image and the second image; and an eighth determination subunit, used to determine the motion trajectory of the target device from the matching results.

[0029] In some embodiments, after matching the device with the highest fourth matching degree in the first image and the second image, the system further includes: a second processing subunit, configured to, when the fourth matching confidence degree reaches a predetermined matching degree, treat the matching position in the second image and the matching position in the first image as the same trajectory and assign the same trajectory number; and a third processing subunit, configured to, when the fourth matching confidence degree does not reach the predetermined matching degree, determine that the matching position in the second image and the matching position in the first image are not the same trajectory, and assign a new trajectory number to the matching position in the second image, wherein the new trajectory number is different from the trajectory number of the matching position in the first image.

[0030] A third aspect of this specification provides a surgical robot, in which an endoscope and instruments are mounted on a robotic arm. The endoscope is a tubular structure, and the tubular body of the endoscope and the tubular bodies at the ends of each instrument extend into a cavity through holes on a trocar. The tubular body of the endoscope includes: a first straight segment located at the end of the tubular body; a second straight segment fixed in the hole of the trocar; a first adjustment unit for adjusting the relative position of the first and second straight segments of the endoscope while maintaining a constant skew angle between them; and a second adjustment unit. The surgical robot includes: a first adjustment unit for adjusting the skew angle between the first and second straight line segments; a third adjustment unit for controlling the extension and retraction of the second straight line segment within the trocar; and a controller for acquiring a set of positions of feature points on each target instrument; determining whether each position point in the set is within the field of view of the endoscope; and automatically adjusting at least one of the first, second, and third adjustment units when a position point in the set is not within the field of view of the endoscope, so that each position point in the set is within the field of view of the endoscope.

[0031] A fourth aspect of this specification provides an electronic device, comprising: a memory and a processor, wherein the processor and the memory are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to implement the steps of the method described in any of the first aspects.

[0032] A fifth aspect of this specification provides a computer storage medium storing computer program instructions that, when executed, implement the steps of the method described in any of the first aspects.

[0033] The endoscopic adjustment method, device, and electronic equipment provided in this manual, for use with a flexible snake-bone endoscope, acquire a set of positions of feature points on each target instrument. When a position point in the set is not within the endoscope's field of view, at least one of the first, second, and third adjustment parts on the endoscope tube is automatically adjusted so that all position points in the set are within the endoscope's field of view. This solution can automatically adjust the flexible snake-bone endoscope during surgery to keep surgical instruments within the endoscope's field of view, eliminating the need for manual adjustment by the surgeon, thereby simplifying the surgical procedure and shortening the operation time. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0035] Figure 1 A three-dimensional structural diagram of the surgical robot system is shown;

[0036] Figure 2 A schematic diagram of the assembly of the endoscope, instruments, and trocar is shown.

[0037] Figure 3 A schematic diagram of the interior of an endoscope is shown;

[0038] Figure 4 A schematic diagram of the first adjustment section and the second adjustment section of the endoscope is shown;

[0039] Figure 5 A schematic diagram of the third adjustment section of the endoscope is shown;

[0040] Figure 6 and Figure 7 Schematic diagrams of endoscopes and instruments are shown for cases with 2 and 3 instruments, respectively;

[0041] Figures 8 to 11 The diagrams show four states during the endoscope adjustment process.

[0042] Figure 12 A flowchart of an endoscope adjustment method provided in this specification is shown;

[0043] Figure 13 A flowchart of an automatic adjustment method for the first, second, and third adjustment units is shown.

[0044] Figure 14 A flowchart of another endoscope adjustment method provided in this specification is shown;

[0045] Figure 15 A flowchart of a training method for a decision model is shown;

[0046] Figure 16 A schematic diagram of the electronic device provided in this specification is shown. Detailed Implementation

[0047] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0048] like Figure 1 As shown, a surgical robot system typically consists of a control unit 100, an execution unit 200, and an imaging unit 300. The control unit 100, usually called the doctor's console, is located outside the sterile area of ​​the operating room and is used by the surgeon to send control commands to the execution unit 200. The execution unit 200, i.e., the surgical robot device (referred to as the surgical robot in this specification), is used to execute specific surgical operations on the patient using surgical instruments based on the control commands sent by the control unit 100. An endoscope can also be mounted on the surgical robot. The imaging unit 300, usually called an imaging cart, is used to process the information acquired by the endoscope to form a three-dimensional high-definition image and feed it back to the control unit 100, etc.

[0049] The endoscope adjustment methods provided in this manual are applicable to... Figure 2 , Figure 3 , Figure 4 and Figure 5 The endoscope shown. Figure 2 A schematic diagram of the assembly of an endoscope, instruments, and a trocar is shown, where A represents the trocar, B represents the endoscope, and three instruments are positioned near the endoscope B. The endoscope B is a flexible tubular structure with a lens located at the end of the tubular structure. Figure 3 A schematic diagram of the endoscope's internal structure is shown, revealing that its internal structure is bamboo-like. Figure 4 A schematic diagram of the first adjustment section B1 and the second adjustment section B2 of the endoscope is shown. Figure 5 A schematic diagram of the third adjustment section B3 of the endoscope is shown.

[0050] The trocar is used to puncture abdominal tissues, deliver gas into the abdominal cavity, and create a passage for endoscopes and instruments to enter and exit the abdominal cavity from the outside. In other words, the endoscope tube and the tubes at the ends of the instruments extend into the cavity through holes on the trocar, and the endoscope tube and the tubes at the ends of the instruments are confined within the trocar.

[0051] like Figure 4 and Figure 5 As shown, the endoscope B includes a first adjustment section B1, a second adjustment section B2, and a third adjustment section B3 on its tube body. The first adjustment section B1 and the second adjustment section B2 are located on the side of the trocar facing the cavity, and the third adjustment section B3 is located on the side of the trocar away from the cavity.

[0052] Figure 6 and Figure 7 Schematic diagrams of the endoscopes and instruments are shown for cases with 2 and 3 instruments, respectively. Figure 6 As shown, the endoscope B also includes a first straight segment B4 and a second straight segment B5 on its tube. Figure 6 (Not shown in the image). Both the first straight segment B4 and the second straight segment B5 are rigid, straight tubes, meaning they cannot be bent. For example... Figures 8 to 11 As shown, the flexible section B6 is located between the first straight segment B4 and the second straight segment B5. Multiple points on the tube body of the flexible section B6 can be bent. In some embodiments, the tube body of the flexible section B6 can be bent into any shape.

[0053] The first adjustment unit B1 is used to adjust the relative position of the first straight segment and the second straight segment of the endoscope while keeping the skew angle between the first straight segment and the second straight segment constant. Figures 8 to 9 A schematic diagram showing the adjustment of the first adjustment part B1 is shown, namely... Figure 8 The endoscope posture shown is obtained by adjusting the first adjustment unit B1. Figure 9 The endoscope is shown in the following position.

[0054] In some embodiments, the first adjustment part B1 may be disposed between the first straight segment B4 and the flexible part B6, or between the second straight segment B5 and the flexible part B6, or on the flexible part B6. In some embodiments, the first adjustment part B1 is not a component disposed at a certain position on the endoscope tube, but a collection of multiple components distributed at multiple positions on the endoscope tube, and the components in the collection work together to realize the adjustment function of the first adjustment part B1.

[0055] The second adjustment part B2 is used to adjust the skew angle between the first straight line segment and the second straight line segment. Figures 10 to 11 A schematic diagram of adjusting the second adjustment unit B2 is shown, namely... Figure 10The endoscope posture shown is obtained by adjusting the second adjustment unit B2. Figure 11 The endoscope posture is shown. In some embodiments, the second adjustment unit B2 can adjust the skew angle between the first straight segment B4 and the second straight segment B5 of the endoscope while keeping the relative position of the predetermined end point of the first straight segment B4 and the second straight segment B5 unchanged.

[0056] The third adjustment unit B3 is used to control the extension and retraction of the second straight segment within the puncture device, that is, to adjust the length of the second straight segment extending into the cavity. Figures 9 to 10 A schematic diagram of adjusting the third adjustment unit B3 is shown, namely... Figure 9 The endoscope posture shown is obtained by adjusting the third adjustment unit B3. Figure 10 The endoscope is shown in the following position. Figure 5 A schematic diagram of the third adjustment unit B3 is shown. The extension / retraction distance of the second straight segment can be converted to... Figure 5 The change in length of the straight segment between the end of the puncture device and the third adjustment section B3, as shown in H.

[0057] like Figure 12 As shown, the endoscope adjustment method provided in this manual includes the following steps:

[0058] S10: Obtain the set of positions of feature points on each target instrument.

[0059] There are usually two, three, or even more target instruments. In order to observe the operation of each target instrument, the field of view of the endoscope should include the key point positions, i.e., the feature point positions, of each target instrument. Therefore, S10 needs to acquire the positions of each feature point on each target instrument to obtain a set of position points.

[0060] In some embodiments, the target instrument may be specified by the surgeon through a UI interface. Therefore, prior to S10, the procedure may further include: acquiring the target instrument specified by the surgeon through the UI interface.

[0061] The feature points of the target instrument may include the distal end of the instrument, as well as other key points. These other key points can be automatically calibrated based on the type of instrument, for example... Figure 6 M1 and M2 in the middle, Figure 7 The M3, M4, and M5 in the diagram can also be designated by the surgeon through the UI interface.

[0062] S20: Determine whether each location point in the set is within the field of view of the endoscope.

[0063] Before step S20, a field of view model of the endoscope can be established in advance. Based on the position of the endoscope, the field of view model, and the position coordinates of each position point in the set, it can be determined whether each position point is within the field of view of the endoscope.

[0064] S30: When a location point in the set is not within the field of view, automatically adjust at least one of the first adjustment unit, the second adjustment unit, and the third adjustment unit so that each location point in the set is within the field of view of the endoscope.

[0065] In some cases, S30 can adjust only one or more of the first adjustment unit, the second adjustment unit, and the third adjustment unit based on the relative positional relationship between the endoscope's orientation and the regions formed by the various position points in the set, or based on the relative positions of the regions formed by the various position points in the set within the image acquired by the endoscope. Which adjustment unit to adjust and how to adjust it can be determined according to the specific circumstances; that is, different adjustment methods are used for different situations.

[0066] When all locations in the set are within the endoscopic field of view, no adjustment is needed, or other adjustment methods can be used. Other adjustment methods include... Figure 14 The adjustment method is shown.

[0067] The endoscopic adjustment method, device, and electronic equipment provided in this manual, for use with a flexible snake-bone endoscope, acquire a set of positions of feature points on each target instrument. When a position point in the set is not within the endoscope's field of view, at least one of the first, second, and third adjustment parts on the endoscope tube is automatically adjusted so that all position points in the set are within the endoscope's field of view. This solution can automatically adjust the flexible snake-bone endoscope during surgery to keep surgical instruments within the endoscope's field of view, eliminating the need for manual adjustment by the surgeon, thereby simplifying the surgical procedure and shortening the operation time.

[0068] This specification provides an automatic adjustment method applicable to any situation. This adjustment method does not require pre-determining the specific situation, nor does it require pre-determining different adjustment methods for different situations. Specifically, as... Figure 13 As shown, the automatic adjustment method sequentially adjusts the first adjustment unit B1, the second adjustment unit B2, and the third adjustment unit B3.

[0069] Step S31: When adjusting the first adjustment part B1, the first projection point is the projection point of the end of the first straight segment of the endoscope on the target plane, and the second projection point is the projection point of the center point of each position point in the set on the target plane. The target plane is a plane perpendicular to the axis of the second straight segment of the endoscope. Figure 8 and Figure 9As shown, P represents the target plane, Q1 represents the first projection point, Q2 represents the second projection point, O represents the center point of each location point in the set, and L represents the distance between the first projection point and the second projection point.

[0070] Since the center point O of each position point in the set remains unchanged when adjusting the first adjustment part B1, it is only necessary to adjust the first projection point in a direction away from the center point O.

[0071] In some embodiments, such as Figure 6 As shown, when there are two target instruments, the midpoint of the line connecting the ends of the target instruments can be used as the center point of each position in the set. Figure 6 In the middle, O represents the center point; for example... Figure 7 As shown, when there are three target devices, the incenter of the triangle formed by the end positions of the target devices is taken as the center point of each position in the set. Figure 7 In this context, O represents the center point. In some embodiments, when determining the center point of each location point in the set, the centroid of the region determined by each location point can also be used as the center point of each location point in the set. Of course, other methods can also be used to determine the center of each location point in the set, which will not be listed here.

[0072] Step S32: Adjust the third adjustment part so that the extension and retraction distance of the third adjustment part is minimized, and all position points in the set are within the swing range of the second adjustment part of the endoscope.

[0073] The swing range of the second adjustment unit can refer to the set of positions that the endoscope's field of view axis can reach during the arbitrary adjustment of the second adjustment unit; or it can refer to the set of various field of view ranges that the endoscope can reach during the arbitrary adjustment of the second adjustment unit.

[0074] Step S33: Adjust the second adjustment unit so that the first straight segment of the endoscope points to the center point of each location point in the set.

[0075] In some embodiments, after step S33, step S34 is further included: adjusting the third adjustment part again so that the extension distance of the third adjustment part is minimized and all position points in the set are within the swing range of the second adjustment part of the endoscope.

[0076] As Figure 13In one modified embodiment, the automatic adjustment method may also involve sequentially adjusting the first adjustment unit, the second adjustment unit, and the third adjustment unit, wherein the adjustment of the first adjustment unit refers to step S31. When adjusting the second adjustment unit, the first straight line segment is made to point towards the center point of each location point in the set, while maintaining the skew angle between the first and second straight line segments within a predetermined range. If the skew angle between the first and second straight line segments is within the predetermined range, but the first straight line segment cannot point towards the center point of each location point in the set, then the second adjustment unit is adjusted so that the first straight line segment moves towards the center point of each location point in the set, while maintaining the skew angle between the first and second straight line segments at the endpoint value of the predetermined range, or a deviation from the endpoint value less than a preset threshold. When adjusting the third adjustment unit, the first straight line segment is made to point towards the center point of each location point in the set. If adjusting the second adjustment unit has already made the first straight line segment point towards the center point of each location point in the set, then adjusting the third adjustment unit is unnecessary.

[0077] This manual also provides an endoscope adjustment method that can be used for Figure 2 , Figure 3 , Figure 4 and Figure 5 The endoscope shown is the same as Figure 12 The endoscopic adjustment methods shown can be used in combination and can also be applied to any other type of endoscope. For example... Figure 14 As shown, it includes the following steps:

[0078] S41: Based on the images acquired by the endoscope at the current moment and at multiple moments prior to the current moment, predict the state parameters of each feature point of the target instrument at the next moment; the state parameters include position data and motion parameters.

[0079] The characteristic points of the target instrument may include the distal end of the instrument, as well as other key points. These other key points may be automatically calibrated based on the type of instrument, or they may be calibrated by the surgeon through a UI interface.

[0080] S42: Input the predicted state parameters of the target instrument at the next moment into the pre-trained decision model to obtain the output endoscope control parameters; the decision model is used to determine the control parameters of the endoscope based on the state parameters of the target instrument.

[0081] S43: Adjust the endoscope according to the endoscope control parameters.

[0082] In some embodiments, such as Figure 15 As shown, the decision model is trained through the following steps S441 to S446.

[0083] S441: Acquire sample trajectory data of the target instrument; the sample trajectory data includes state parameters of multiple location points arranged in chronological order.

[0084] S442: Input the state data of the current position point into the decision model to obtain the output endoscope control parameters.

[0085] S443: Based on the endoscope's field of view model, determine the expected image that the endoscope will acquire at the next moment.

[0086] S444: Determine the reward value for model training based on the predicted image.

[0087] In some embodiments, S444 can identify a set of feature points of the target device from the projected image, identify the set of feature points and determine the size of the anchor frame used for identification; calculate the ratio of the area of ​​the anchor frame to the area of ​​the projected image, and use the ratio of the area as the reward value for model training.

[0088] S445: Determine whether the reward value has met the predetermined conditions. If the reward value has met the predetermined conditions, execute S446; if the reward value has not met the predetermined conditions, execute S447.

[0089] S446: Set the next time step of the current time step as the current time step, and jump to S442 to continue execution.

[0090] S447: Adjust the parameters of the decision model and retrain it using each location point in the sample trajectory data, that is, re-execute S442 to S447.

[0091] In some embodiments, S41 includes the following steps S411 to S413.

[0092] S411: Acquire images obtained by the endoscope at the current time and at multiple times prior to the current time.

[0093] S412: Identify the motion trajectory of the target device based on each image.

[0094] S413: Determine the state parameters of the target device at the next moment based on the motion trajectory of the target device.

[0095] In some embodiments, S412 includes matching the instruments in the images at each adjacent time point using the steps S4121 to S41210.

[0096] S4121: Identify each device in the first and second images at adjacent time points, and determine the size and position of the anchor frame used for identification.

[0097] S4122: Determine the appearance feature values ​​of each instrument in the first and second images.

[0098] S4123: Calculate the similarity between the appearance feature values ​​of each instrument in the first image and the appearance feature values ​​of each instrument in the second image, and use the similarity as the first matching confidence level.

[0099] S4124: Calculate the positional deviation between the anchor frame of each instrument in the first image and the anchor frame of each instrument in the second image.

[0100] S4125: Determine the second matching confidence level based on the position deviation value.

[0101] S4126: Obtain the motion parameters of each instrument at the first image acquisition time and the motion parameters of each instrument at the second image acquisition time.

[0102] S4127: Based on the motion parameters, determine the third matching confidence level between each device in the first image and each device in the second image.

[0103] S4128: The first matching confidence, the second matching confidence, and the third matching confidence are fused according to a predetermined ratio to obtain a fourth matching confidence.

[0104] S4129: Match the device with the highest matching degree in the first image and the second image.

[0105] S41210: Determine the motion trajectory of the target device from the matching results.

[0106] In some embodiments, after S4129, the following steps S41211, S41212, and S41213 are further included.

[0107] S41211: Determine whether the fourth match confidence level has reached the predetermined match level. If the fourth match confidence level has reached the predetermined match level, execute S41212; otherwise, execute S41213.

[0108] S41212: Treat the matching position in the second image and the matching position in the first image as the same trajectory and assign them the same trajectory number.

[0109] S41213: Determine that the matching position in the second image is not the same trajectory as the matching position in the first image, and assign a new trajectory number to the matching position in the second image, wherein the new trajectory number is different from the trajectory number of the matching position in the first image.

[0110] This manual provides an endoscope adjustment device that can be used to achieve... Figure 12The endoscopic adjustment method shown is illustrated. The device includes a first acquisition unit, a judgment unit, and a first adjustment unit.

[0111] The first acquisition unit is used to acquire a set of locations of feature points on each target instrument. The judgment unit is used to determine whether each location point in the set is within the field of view of the endoscope. The first adjustment unit is used to automatically adjust at least one of the first adjustment unit, the second adjustment unit, and the third adjustment unit when there are location points in the set that are not within the field of view, so that all location points in the set are within the field of view of the endoscope.

[0112] In some embodiments, when at least one of the first adjustment unit, the second adjustment unit, and the third adjustment unit is automatically adjusted so that all position points in the set are within the field of view of the endoscope, the first adjustment unit, the third adjustment unit, and the second adjustment unit are adjusted sequentially.

[0113] In some embodiments, when adjusting the first adjustment part, the distance between the first projection point and the second projection point is maximized. The first projection point is the projection point of the end of the first straight segment of the endoscope on the target plane, and the second projection point is the projection point of the center point of each position point in the set on the target plane. The target plane is a plane perpendicular to the axis of the second straight segment of the endoscope.

[0114] In some embodiments, the center point of each location point in the set is determined by the following method: when there are two target devices, the midpoint of the line connecting the end positions of the target devices is taken as the center point of each location point in the set; when there are three target devices, the incenter of the triangle formed by the end positions of the target devices is taken as the center point of each location point in the set.

[0115] In some embodiments, when adjusting the third adjustment section, the extension and retraction distance of the third adjustment section is minimized, and each position point in the set is within the swing range of the second adjustment section of the endoscope.

[0116] In some embodiments, when adjusting the second adjustment section, the first straight segment of the endoscope is directed to the center point of each location point in the set.

[0117] In some embodiments, the apparatus further includes a second acquisition unit for acquiring a target instrument specified by the surgeon through a UI interface.

[0118] In some embodiments, the apparatus further includes: a prediction unit, configured to predict the state parameters of each feature point of the target instrument at the next moment based on images acquired by the endoscope at the current moment and at multiple moments prior to the current moment; the state parameters include position data and motion parameters; a first processing unit, configured to input the predicted values ​​of the state parameters of the target instrument at the next moment into a pre-trained decision model to obtain output endoscope control parameters; the decision model is used to determine the control parameters of the endoscope based on the state parameters of the target instrument; and a second adjustment unit, configured to adjust the endoscope according to the endoscope control parameters.

[0119] In some embodiments, the apparatus further includes a training unit for training the decision model; the training unit includes: a first acquisition subunit for acquiring sample trajectory data of the target instrument; the sample trajectory data includes state parameters of a plurality of position points arranged in chronological order; a first processing subunit, a first determination subunit, a second determination subunit, a third determination subunit, and an adjustment subunit for cyclically performing operations on each position point in the sample trajectory data; wherein, the first processing subunit is used to input the state data of the position point at the current moment into the decision model to obtain output endoscope control parameters; the first determination subunit is used to determine the expected image to be acquired by the endoscope at the next moment based on the endoscope's field of view model; the second determination subunit is used to determine the reward value for model training based on the expected image; the third determination subunit is used to take the next moment of the current moment as the current moment if the reward value reaches a predetermined condition; the adjustment subunit is used to adjust the parameters of the decision model and re-perform the cyclic operation on each position point in the sample trajectory data if the reward value does not reach the predetermined condition.

[0120] In some embodiments, the second determining subunit includes: a first identification subunit, configured to identify a set of feature point locations of the target device from the projected image, identify the set of feature points and determine the size of the anchor frame used for identification; and a first calculation subunit, configured to calculate the ratio of the area of ​​the anchor frame to the area of ​​the projected image, and use the ratio of the areas as a reward value for model training.

[0121] In some embodiments, the prediction unit includes: a second acquisition subunit, configured to acquire images acquired by the endoscope at the current time and at multiple times prior to the current time; a second identification subunit, configured to identify the motion trajectory of the target instrument based on each image; and a fourth determination subunit, configured to determine the state parameters of the target instrument at the next time based on the motion trajectory of the target instrument.

[0122] In some embodiments, the second identification subunit includes: a third identification subunit, configured to identify each device in a first image and a second image at adjacent time points, and determine the size and position of the anchor frame used for identification; a fifth determination subunit, configured to determine the appearance feature values ​​of each device in the first image and the second image; a second calculation subunit, configured to calculate the similarity between the appearance feature values ​​of each device in the first image and the appearance feature values ​​of each device in the second image, and use the similarity as a first matching confidence level; a third calculation subunit, configured to calculate the positional deviation values ​​between the anchor frames of each device in the first image and the anchor frames of each device in the second image; and a sixth determination subunit, configured to determine the positional deviation values ​​based on the positional deviation values. The system comprises the following subunits: a third acquisition subunit, used to acquire the motion parameters of each device at the time of the first image acquisition and the time of the second image acquisition; a seventh determination subunit, used to determine the third matching confidence of each device in the first image and each device in the second image based on the motion parameters; a fusion subunit, used to fuse the first matching confidence, the second matching confidence, and the third matching confidence according to a predetermined ratio to obtain a fourth matching confidence; a matching subunit, used to match the device with the highest fourth matching confidence in the first image and the second image; and an eighth determination subunit, used to determine the motion trajectory of the target device from the matching results.

[0123] In some embodiments, after matching the device with the highest fourth matching degree in the first image and the second image, the system further includes: a second processing subunit, configured to, when the fourth matching confidence degree reaches a predetermined matching degree, treat the matching position in the second image and the matching position in the first image as the same trajectory and assign the same trajectory number; and a third processing subunit, configured to, when the fourth matching confidence degree does not reach the predetermined matching degree, determine that the matching position in the second image and the matching position in the first image are not the same trajectory, and assign a new trajectory number to the matching position in the second image, wherein the new trajectory number is different from the trajectory number of the matching position in the first image.

[0124] For details regarding the endoscopic adjustment device described above, please refer to the relevant descriptions and effects in the above method embodiments; they will not be repeated here.

[0125] This specification provides a surgical robot with an endoscope and instruments mounted on a robotic arm. The endoscope has a tubular structure, and the tubular body of the endoscope and the tubular bodies at the ends of each instrument extend into the cavity through holes on a trocar. Furthermore, the tubular body of the endoscope includes a first straight section, a second straight section, a first adjustment section, a second adjustment section, and a third adjustment section.

[0126] A first straight segment is located at the end of the endoscope tube. A second straight segment is located within the trocar. A first adjustment unit is used to adjust the relative position of the first and second straight segments of the endoscope while maintaining a constant skew angle between them. A second adjustment unit is used to adjust the skew angle between the first and second straight segments. A third adjustment unit is used to control the telescopic movement of the second straight segment within the trocar.

[0127] The surgical robot further includes: a controller, used to acquire a set of positions of feature points on each target instrument; determine whether each position point in the set is within the field of view of the endoscope; when there is a position point in the set that is not within the field of view, automatically adjust at least one of the first adjustment unit, the second adjustment unit, and the third adjustment unit so that each position point in the set is within the field of view of the endoscope.

[0128] This invention also provides an electronic device, such as... Figure 16 As shown, the electronic device may include a processor 1601 and a memory 1602, wherein the processor 1601 and the memory 1602 may be connected via a bus or other means. Figure 16 Taking the example of a connection between China and Israel via a bus.

[0129] Processor 1601 can be a Central Processing Unit (CPU). Processor 1601 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0130] The memory 1602, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the endoscope adjustment method in the embodiments of the present invention (e.g., the first acquisition unit, the judgment unit, and the first adjustment unit mentioned above). The processor 1601 executes various functional applications and data classification of the processor by running the non-transitory software programs, instructions, and modules stored in the memory 1602, thereby realizing the endoscope adjustment method in the above method embodiments.

[0131] The memory 1602 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor 1601, etc. Furthermore, the memory 1602 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1602 may optionally include memory remotely located relative to the processor 1601, and these remote memories may be connected to the processor 1601 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0132] The one or more modules are stored in the memory 1602, and when executed by the processor 1601, the endoscope adjustment method in the above method embodiment is executed.

[0133] For details regarding the electronic device described above, please refer to the relevant descriptions and effects in the above method embodiments; further details will not be repeated here.

[0134] This specification also provides a computer storage medium storing computer program instructions that, when executed, implement the steps of the above-described method embodiments.

[0135] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0136] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0137] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0138] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0139] The above description is merely an embodiment of the present specification and is not intended to limit the embodiments of the present specification. For those skilled in the art, various modifications and variations can be made to the embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present specification should be included within the scope of the claims of the embodiments of the present specification.

Claims

1. An endoscope adjustment device, characterized in that, The endoscope tube extends into the cavity through a hole in the trocar; the endoscope tube includes: The first straight section is located at the end of the pipe body; The second straight section is fixed in the hole of the puncture device; A flexible section is disposed between the first straight segment and the second straight segment; A first adjustment section is provided in the flexible section for adjusting the relative position of the first straight section and the second straight section of the endoscope while keeping the skew angle between the first straight section and the second straight section constant. The second adjustment section is used to adjust the skew angle between the first straight line segment and the second straight line segment. The third adjustment section is used to control the extension and retraction of the second linear segment within the puncture device; The device includes: The first acquisition unit is used to acquire a set of locations of feature points on each target instrument; The judgment unit is used to determine whether each location point in the set is within the field of view of the endoscope; The first adjustment unit is used to automatically adjust the first adjustment section, the second adjustment section, and the third adjustment section in a predetermined order when there are position points in the set that are not within the field of view, so that each position point in the set is within the field of view of the endoscope; wherein, the first adjustment unit adjusts the first adjustment section in a direction away from the center point of each position point in the set towards the first projection point, and adjusts the second adjustment section in a direction closer to the center point of each position point in the set towards the first straight line segment.

2. The apparatus according to claim 1, characterized in that, When at least one of the first adjustment unit, the second adjustment unit, and the third adjustment unit is automatically adjusted so that all position points in the set are within the field of view of the endoscope, the first adjustment unit, the third adjustment unit, and the second adjustment unit are adjusted in sequence.

3. The apparatus according to claim 2, characterized in that, When adjusting the first adjustment part, the distance between the first projection point and the second projection point is maximized. The first projection point is the projection point of the end of the first straight segment of the endoscope on the target plane. The second projection point is the projection point of the center point of each position point in the set on the target plane. The target plane is a plane perpendicular to the axis of the second straight segment of the endoscope.

4. The apparatus according to claim 3, characterized in that, The center point of each location point in the set is determined by the following method: When there are two target instruments, the midpoint of the line connecting the end positions of the target instruments is taken as the center point of each position point in the set; When there are three target instruments, the incenter of the triangle formed by the end positions of the target instruments is taken as the center point of each position point in the set.

5. The apparatus according to claim 2, characterized in that, When adjusting the third adjustment section, the extension and retraction distance of the third adjustment section is minimized, and all position points in the set are within the swing range of the second adjustment section of the endoscope.

6. The apparatus according to claim 2, characterized in that, When adjusting the second adjustment section, the first straight segment of the endoscope is directed to the center point of each location point in the set.

7. The apparatus according to claim 1, characterized in that, The device further includes: The second acquisition unit is used to acquire the target instrument specified by the surgeon through the UI interface.

8. The apparatus according to claim 1, characterized in that, The device further includes: The prediction unit is used to predict the state parameters of each feature point of the target instrument at the next moment based on the images acquired by the endoscope at the current moment and at multiple moments before the current moment; the state parameters include position data and motion parameters. The first processing unit is used to input the predicted state parameters of the target instrument at the next moment into a pre-trained decision model to obtain the output endoscope control parameters; the decision model is used to determine the control parameters of the endoscope based on the state parameters of the target instrument. The second adjustment unit is used to adjust the endoscope according to the endoscope control parameters.

9. The apparatus according to claim 8, characterized in that, The device further includes a training unit for training the decision model; the training unit includes: The first acquisition subunit is used to acquire sample trajectory data of the target instrument; the sample trajectory data includes state parameters of multiple location points arranged in chronological order. A first processing subunit, a first determining subunit, a second determining subunit, a third determining subunit, and an adjustment subunit are used to cyclically perform operations on each location point in the sample trajectory data; wherein... The first processing subunit is used to input the state data of the current position point into the decision model to obtain the output endoscope control parameters; The first determining subunit is used to determine the expected image to be acquired by the endoscope at the next moment based on the endoscope's field of view model; The second determining subunit is used to determine the reward value for model training based on the predicted image; The third determining subunit is used to take the next moment after the current moment as the current moment when the reward value reaches a predetermined condition; The adjustment subunit is used to adjust the parameters of the decision model and re-execute the cyclic operation on each location point in the sample trajectory data when the reward value does not meet the predetermined conditions.

10. The apparatus according to claim 9, characterized in that, The second determining subunit includes: a first identification subunit, used to identify a set of feature point locations of the target device from the expected image, and to identify the set of feature points and determine the size of the anchor frame used for identification; and a first calculation subunit, used to calculate the ratio of the area of ​​the anchor frame to the area of ​​the expected image, and to use the ratio of the area as the reward value for model training.

11. The apparatus according to claim 8, characterized in that, The prediction unit includes: The second acquisition subunit is used to acquire images obtained by the endoscope at the current time and at multiple times prior to the current time; The second recognition subunit is used to identify the motion trajectory of the target device based on each image. The fourth determining subunit is used to determine the state parameters of the target device at the next moment based on the motion trajectory of the target device.

12. The apparatus according to claim 11, characterized in that, The second identification subunit includes: The third identification subunit is used to identify each device in the first and second images at adjacent time points, and to determine the size and position of the anchor frame used for identification. The fifth determining subunit is used to determine the appearance feature values ​​of each instrument in the first and second images; The second calculation subunit is used to calculate the similarity between the appearance feature values ​​of each instrument in the first image and the appearance feature values ​​of each instrument in the second image, and to use the similarity as the first matching confidence level. The third calculation subunit is used to calculate the positional deviation between the anchor frame of each instrument in the first image and the anchor frame of each instrument in the second image. The sixth determining subunit is used to determine the second matching confidence level based on the position deviation value; The third acquisition subunit is used to acquire the motion parameters of each instrument at the first image acquisition time and the motion parameters of each instrument at the second image acquisition time. The seventh determining subunit is used to determine the third matching confidence level between each instrument in the first image and each instrument in the second image based on the motion parameters. The fusion subunit is used to fuse the first matching confidence, the second matching confidence, and the third matching confidence according to a predetermined ratio to obtain a fourth matching confidence. The matching subunit is used to match the fourth device with the highest matching degree in the first image and the second image; The eighth determining subunit is used to determine the motion trajectory of the target device from the matching results.

13. The apparatus according to claim 12, characterized in that, After matching the device with the highest matching degree in the first image and the second image, the process further includes: The second processing subunit is used to treat the matching position in the second image and the matching position in the first image as the same trajectory and assign the same trajectory number when the fourth matching confidence reaches a predetermined matching degree. The third processing subunit is used to determine that the matching position in the second image is not the same trajectory as the matching position in the first image when the fourth matching confidence level does not reach the predetermined matching level, and to assign a new trajectory number to the matching position in the second image, wherein the new trajectory number is different from the trajectory number of the matching position in the first image.

14. A surgical robot, characterized in that, The robotic arm is equipped with an endoscope and instruments. The endoscope has a tubular structure, and the tubular body of the endoscope and the tubular bodies at the ends of each instrument extend into the cavity through holes on the trocar. Furthermore, the tubular body of the endoscope includes: The first straight section is located at the end of the pipe body; The second straight section is fixed in the hole of the puncture device; A flexible section is disposed between the first straight segment and the second straight segment; A first adjustment section is provided in the flexible section for adjusting the relative position of the first straight section and the second straight section of the endoscope while keeping the skew angle between the first straight section and the second straight section constant. The second adjustment section is used to adjust the skew angle between the first straight line segment and the second straight line segment. The third adjustment section is used to control the extension and retraction of the second linear segment within the puncture device; The surgical robot also includes: The controller is used to acquire a set of locations of feature points on each target instrument; determine whether each location point in the set is within the field of view of the endoscope; when there are location points in the set that are not within the field of view, automatically adjust the first adjustment unit, the second adjustment unit, and the third adjustment unit in a predetermined order so that each location point in the set is within the field of view of the endoscope; wherein, the first adjustment unit is adjusted in a direction away from the center point of each location point in the set towards the first projection point, and the second adjustment unit is adjusted in a direction closer to the center point of each location point in the set towards the first straight line segment.

15. An electronic device, characterized in that, include: The system includes a memory and a processor, which are communicatively connected. The memory stores computer instructions, and the processor executes these instructions to implement an endoscope adjustment method. The endoscope tube extends into the cavity through a hole in a trocar. The endoscope tube includes: The first straight section is located at the end of the pipe body; The second straight section is used to fix it in the hole of the puncture device; A flexible section is disposed between the first straight segment and the second straight segment; A first adjustment section is provided in the flexible section for adjusting the relative position of the first straight section and the second straight section of the endoscope while keeping the skew angle between the first straight section and the second straight section constant. The second adjustment section is used to adjust the skew angle between the first straight line segment and the second straight line segment. The third adjustment section is used to control the extension and retraction of the second linear segment within the puncture device; The endoscopic adjustment method includes: Obtain the set of locations of feature points on each target instrument; Determine whether each location point in the set is within the field of view of the endoscope; When a location point in the set is not within the field of view, the first adjustment unit, the second adjustment unit, and the third adjustment unit are automatically adjusted in a predetermined order so that each location point in the set is within the field of view of the endoscope; wherein, the first adjustment unit is adjusted in a direction away from the center point of each location point in the set towards the first projection point, and the second adjustment unit is adjusted in a direction closer to the center point of each location point in the set towards the first straight line segment.

16. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed, implement an endoscope adjustment method. The endoscope tube extends into the cavity through a hole in the trocar. The endoscope tube includes: The first straight section is located at the end of the pipe body; The second straight section is used to fix it in the hole of the puncture device; A flexible section is disposed between the first straight segment and the second straight segment; A first adjustment section is provided in the flexible section for adjusting the relative position of the first straight section and the second straight section of the endoscope while keeping the skew angle between the first straight section and the second straight section constant. The second adjustment section is used to adjust the skew angle between the first straight line segment and the second straight line segment. The third adjustment section is used to control the extension and retraction of the second linear segment within the puncture device; The endoscopic adjustment method includes: Obtain the set of locations of feature points on each target instrument; Determine whether each location point in the set is within the field of view of the endoscope; When a location point in the set is outside the field of view, the first adjustment unit, the second adjustment unit, and the third adjustment unit are automatically adjusted in a predetermined order so that each location point in the set is within the field of view of the endoscope. Specifically, the first adjustment part is adjusted in the direction away from the center point of each position point in the set, and the second adjustment part is adjusted in the direction closer to the center point of each position point in the set, while the first straight line segment part is adjusted in the direction closer to the center point of each position point in the set.

Images