Instrument pose control method and device, computer device, and storage medium

Abstract

The application relates to a control method and device of an instrument pose, a computer device, a storage medium and a computer program product. The method comprises the following steps: when a target instrument is determined to be delivered to a target position through an endoscope, determining a real-time pose of the target instrument through at least one first sensor arranged at a hard end part of the target instrument. If the real-time pose deviates from a preset pose, the target instrument is controlled to be adjusted to the preset pose. In this way, the efficiency of the control of the instrument pose is greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical instruments, and in particular to a method and device for controlling the pose of an instrument, a computer device, a storage medium, and a computer program product. BACKGROUND

[0002] With the development of medical instrument technology, in the process of repairing an abnormal region in a target object, an endoscope is often used to determine the abnormality of the target region. When the abnormality of the target region is determined, the target region can be repaired by an instrument.

[0003] To ensure that no impact is caused on regions other than the target region, the pose of the instrument needs to be constantly adjusted or determined. In the conventional technology, the pose of the instrument is often determined and adjusted by human judgment.

[0004] However, the pose of the instrument is difficult to determine at one time by human adjustment, and it often needs to be operated multiple times to complete the adjustment of the pose of the instrument, which greatly increases the time for repairing the abnormality. Therefore, by manually controlling the instrument, it is difficult to ensure accurate control of the instrument, and a large amount of time is consumed, and there is a problem of low efficiency in controlling the pose of the instrument. SUMMARY

[0005] Therefore, it is necessary to provide a method and device for controlling the pose of an instrument, a computer device, a computer readable storage medium, and a computer program product, which can improve the efficiency of controlling the pose of the instrument.

[0006] In a first aspect, the present application provides a method for controlling the pose of an instrument. The method comprises:

[0007] When a target instrument is determined to be delivered to a target position by an endoscope, determining a real-time pose of the target instrument by at least one first sensor arranged at a hard end portion of the target instrument;

[0008] If the real-time pose deviates from a preset pose, controlling the target instrument to adjust to the preset pose; the preset pose is determined by the position of an object to be detected and the position of an instrument hole of the endoscope in which the target instrument is located.

[0009] In a second aspect, the present application also provides a device for controlling the pose of an instrument. The device comprises:

[0010] A determining module configured to, when a target instrument is determined to be delivered to a target position by an endoscope, determine a real-time pose of the target instrument by at least one first sensor arranged at a hard end portion of the target instrument;

[0011] The control module is configured to control the target instrument to adjust to a preset pose if the real-time pose deviates from the preset pose, wherein the preset pose is determined based on a position of the object to be detected and a position of an instrument hole of the endoscope in which the target instrument is located.

[0012] In a third aspect, the present application provides a computer device. The computer device comprises a memory and a processor. The memory stores a computer program. The processor implements the following steps when executing the computer program:

[0013] When it is determined that the target instrument is delivered to the target position through the endoscope, a real-time pose of the target instrument is determined by at least one first sensor arranged at a hard end portion of the target instrument.

[0014] If the real-time pose deviates from a preset pose, the target instrument is controlled to adjust to the preset pose, wherein the preset pose is determined based on a position of the object to be detected and a position of an instrument hole of the endoscope in which the target instrument is located.

[0015] In a fourth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the following steps:

[0016] When it is determined that the target instrument is delivered to the target position through the endoscope, a real-time pose of the target instrument is determined by at least one first sensor arranged at a hard end portion of the target instrument.

[0017] If the real-time pose deviates from a preset pose, the target instrument is controlled to adjust to the preset pose, wherein the preset pose is determined based on a position of the object to be detected and a position of an instrument hole of the endoscope in which the target instrument is located.

[0018] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program. The computer program is executed by a processor to implement the following steps:

[0019] When it is determined that the target instrument is delivered to the target position through the endoscope, a real-time pose of the target instrument is determined by at least one first sensor arranged at a hard end portion of the target instrument.

[0020] If the real-time pose deviates from a preset pose, the target instrument is controlled to adjust to the preset pose, wherein the preset pose is determined based on a position of the object to be detected and a position of an instrument hole of the endoscope in which the target instrument is located.

[0021] The control method, device, computer device, storage medium and computer program product of the instrument pose described above, by pre-determining that the target instrument is delivered to the target position through the endoscope, avoid the need for additional adjustment of the position of the target instrument after subsequent determination of the pose. In this way, based on the determination that the target instrument reaches the target position, effective pose determination can be performed. Therefore, when the target instrument reaches the target position, the real-time pose of the target instrument can be automatically and timely determined through the at least one first sensor arranged at the hard end of the target instrument, avoiding errors caused by manual adjustment of the instrument, and greatly improving the efficiency of control of the instrument pose. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 An application environment diagram of the control method of the instrument pose in one embodiment;

[0023] Figure 2 A flowchart of the control method of the instrument pose in one embodiment;

[0024] Figure 3 A schematic diagram of the endoscope structure in one embodiment;

[0025] Figure 4 A schematic diagram of the target instrument structure in one embodiment;

[0026] Figure 5 A schematic diagram of the endoscope pose in one embodiment;

[0027] Figure 6 A schematic diagram of the target instrument delivery in one embodiment;

[0028] Figure 7 A schematic diagram of the target instrument pose and the preset pose in one embodiment;

[0029] Figure 8 A schematic diagram of the first sensor in one embodiment;

[0030] Figure 9 A schematic diagram of the second sensor in one embodiment;

[0031] Figure 10 A schematic diagram of the first sensor in another embodiment;

[0032] Figure 11 A schematic diagram of the second sensor in another embodiment;

[0033] Figure 12 A flowchart of the step of determining the target position in one embodiment;

[0034] Figure 13 A flowchart of the step of determining the target position in another embodiment;

[0035] Figure 14 Fig. 13 is a schematic diagram of determining target position in one embodiment;

[0036] Figure 15 Fig. 14 is a schematic diagram of the lens end of an endoscope in one embodiment;

[0037] Figure 16 Fig. 15 is a schematic diagram of the lens end of an endoscope in another embodiment;

[0038] Figure 17 Fig. 16 is a schematic diagram of the flow of adjusting the pose in one embodiment;

[0039] Figure 18 Fig. 17 is a schematic diagram of the interactive interface in one embodiment;

[0040] Figure 19 Fig. 18 is a schematic diagram of the various platforms in one embodiment;

[0041] Figure 20 Fig. 19 is a schematic diagram of the flow of the control method of the instrument pose in another embodiment;

[0042] Figure 21 Fig. 20 is a structural block diagram of the control device of the instrument pose in one embodiment;

[0043] Figure 22 Fig. 21 is an internal structural diagram of the computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0045] The control method of the instrument pose provided by the embodiments of the present application can be applied to, for example, Figure 1The application environment is shown. Among them, the operation platform 102 communicates with the computer equipment 104 through the network, and the operation platform is used to load the endoscope and the instrument. The data storage system can store the data required to be processed by the computer equipment 104. The data storage system can be integrated on the computer equipment 104, or placed on the cloud or other network computer equipment. When the computer equipment 104 determines that the target instrument on the operation platform 102 is delivered to the target position through the endoscope on the operation platform 102, it determines the real-time pose of the target instrument through at least one first sensor arranged at the hard end of the target instrument. If the real-time pose deviates from the preset pose, the computer equipment 104 controls the target instrument to adjust to the preset pose. Among them, the computer equipment can be a terminal or a server, and the terminal can be but not limited to various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices and the like. The server can be realized by an independent server or a server cluster composed of multiple servers.

[0046] In one embodiment, as shown in Figure 2 , a control method of instrument pose is provided, which is applied to the computer equipment 104 in Figure 1 for example, including the following steps:

[0047] Step S202, when determining that the target instrument is delivered to the target position through the endoscope, the real-time pose of the target instrument is determined through at least one first sensor arranged at the hard end of the target instrument.

[0048] Among them, the endoscope is a medical imaging device that can directly observe the target region in the target object. The structure of the endoscope is as shown in Figure 3 , wherein the docking device can be connected with the image platform, and the endoscope head image acquisition device is started to transmit the collected picture to the display device in the image platform and the computer equipment in real time. The image platform is as shown in Figure 2 , the image platform and the computer equipment communicate through the network, and the image platform is used to acquire the endoscope image.

[0049] Among them, the target instrument is a device that can repair the abnormal target region in the target object. The structure of the target instrument is as shown in Figure 4As shown, the target instrument includes a flexible snake bone with four rotating mechanisms, a driving wire, and an electric hook. Among them, the rotating mechanisms in the flexible snake bone cooperate with each other to realize the change of the pose of the head end of the instrument. At the same time, the number of rotating mechanisms can be increased according to actual needs, so as to realize the adjustment of the flexibility and the maximum bending angle of the instrument. Among them, the driving wire passes through the guide wire hole inside the snake bone and is finally fixed at the head end of the instrument, realizing the driving of the snake bone. Among them, the electric hook is installed at the head end of the instrument, and the electric hook can be replaced with a duckbill clamp or the like according to actual needs. Among them, the hard end part of the target instrument is composed of the flexible snake bone, the driving wire, and the electric hook.

[0050] It should be noted that the endoscope structure and the target instrument structure involved in the instrument pose control method in the present application can also be other structures, which are not limited in particular.

[0051] Among them, the first sensor is used to sense and collect the position data of the target instrument.

[0052] Among them, the real-time pose of the target instrument represents the pose of the head end of the hard end part of the target instrument.

[0053] Specifically, the computer device receives the delivery instruction generated by the triggering operation of the operator, and determines the delivery mode of the endoscope and the target instrument according to the delivery instruction. In the case that the delivery mode of the endoscope is manual delivery mode, the computer device receives the image information collected by the endoscope, and determines whether the endoscope reaches the abnormal area. Or, in the case that the delivery mode of the endoscope is automatic delivery mode, the computer device sends a first driving instruction to the endoscope to control the endoscope to deliver, and determines whether the endoscope reaches the abnormal area through the image information collected by the endoscope.

[0054] After the endoscope reaches the abnormal area, in the case that the delivery mode of the target instrument is manual delivery mode, the computer device determines whether the target instrument is delivered to the target position of the abnormal area through the endoscope based on the position information collected by at least one first sensor arranged at the hard end part of the target instrument, and the position information collected by at least one second sensor arranged at the lens end of the endoscope.

[0055] When the computer device determines that the target instrument is delivered to the target position through the endoscope, the computer device determines the real-time pose of the target instrument at the target position through the first sensor.

[0056] For example, in the case that the delivery mode of the endoscope is manual delivery mode, as shown in the figure, Figure 5 The operator manually sends the endoscope into the cavity of the target object, and slowly delivers forward until the operator determines that the abnormal area containing the object to be detected through the image information collected by the endoscope, the operator stops delivery, and the endoscope keeps the current pose.

[0057] Once the operator confirms via the computer display that the endoscope has reached the abnormal area, and the target instrument is being transported manually, if... Figure 6 As shown, the operator begins inputting the target instrument. The computer equipment or the operator activates at least one first sensor located at the hard end of the target instrument and at least one second sensor located at the lens end of the endoscope to sense and feedback position information in real time. The operator manually inserts the target instrument into the cavity through the endoscope channel and, based on the position information collected by at least one first sensor and at least one second sensor, determines whether the target instrument has been transported to the target position in the abnormal area via the endoscope. When the computer equipment determines that the target instrument has been transported to the target position via the endoscope, the computer equipment determines the real-time pose of the target instrument at the target position using the position information collected by the first sensor. Figure 6 An optional embodiment of the distribution of the first and second sensors is provided, wherein B1 and B2 are the first sensors, and A1, A2, and A3 are the second sensors. It should be noted that there can be multiple first sensors and multiple second sensors, and there is no specific limitation.

[0058] Alternatively, in the case of automated endoscope delivery, the endoscope is fixed to the operating platform, with its lens positioned at the entrance of the target cavity. Based on confirmation commands triggered by the operator, the computer system controls the operating platform to deliver the endoscope until an abnormal area containing the object to be examined is observed, at which point the endoscope stops and maintains its current position. During automated delivery, the operator can change the endoscope's position by manipulating the main control arm within the computer system.

[0059] When the target instrument is transported automatically, the operator mounts the instrument on the operating platform and places it into the instrument channel of the endoscope. Based on a confirmation command triggered by the operator, the computer system activates at least one first sensor and at least one second sensor to sense and provide real-time position information. Based on the position information collected by the at least one first sensor and the at least one second sensor, the computer system determines whether the target instrument has been transported to the target location in the abnormal area via the endoscope. When the computer system determines that the target instrument has been transported to the target location via the endoscope, it determines the real-time pose of the target instrument at the target location using the position information collected by the first sensor.

[0060] It should be noted that the endoscope and the delivery method of the target instrument involved in the present application can be manually delivered or automatically delivered, and the specific delivery method is not limited. However, the real-time pose of the target instrument and the pose adjustment process of the target instrument are automatically controlled to ensure accurate determination and timely adjustment of the pose of the target instrument and improve the efficiency of the pose control of the target instrument.

[0061] In step S204, if the real-time pose deviates from the preset pose, the target instrument is controlled to adjust to the preset pose. The preset pose is determined based on the position of the object to be detected and the position of the instrument hole of the endoscope where the target instrument is located.

[0062] The preset pose is determined based on the position of the object to be detected and the position of the instrument hole of the endoscope where the target instrument is located. For example, if the position of the object to be detected is m point and the position of the instrument hole of the endoscope where the target instrument is located is n point, the preset pose can be that the hard end of the target instrument delivered out of the endoscope is on the straight line determined by the m point and the n point. The specific implementation is not limited. Figure 7 As shown in the figure, the real-time pose deviates from the preset pose.

[0063] Specifically, the computer device determines the error between the real-time pose and the preset pose. If the error is not within the preset error range, it is determined that the real-time pose deviates from the preset pose. The computer device iteratively calculates the real-time pose based on the preset pose at least once until the error between the pose of the current iteration and the preset pose is within the preset error range.

[0064] For example, as shown in the structure of the target instrument, Figure 4 The length of the driving wire in the target instrument affects the real-time pose of the target instrument, and the change in the length of the driving wire is caused by the change in the driving mechanism of the target instrument. Therefore, in the case where the real-time pose deviates from the preset pose, the computer device takes the real-time pose as the previous to-be-corrected pose of the first iteration and iterates. For each iteration, if the previous to-be-corrected pose deviates from the preset pose, the driving wire is adjusted to obtain the current length of the driving wire of the current iteration, and the current to-be-corrected pose is determined through the current length. The computer device determines whether to enter the next iteration based on the error between the current to-be-corrected pose and the preset pose. If not, the computer device determines that the current to-be-corrected pose of the current iteration is adjusted.

[0065] The control method of the above instrument pose avoids the need for additional adjustment of the position of the target instrument after the pose is determined. In this way, effective pose determination can be performed based on the target instrument reaching the target position. Therefore, when the target instrument reaches the target position, the real-time pose of the target instrument can be automatically and timely determined by the at least one first sensor provided on the hard end of the target instrument, avoiding errors caused by manual adjustment of the instrument, and greatly improving the efficiency of control of the instrument pose.

[0066] In one embodiment, the manner of determining that the target instrument is delivered to the target position through the endoscope includes: determining a first distance between an end point of the hard end of the target instrument and a lens end of the endoscope by at least one first sensor provided on the hard end of the target instrument and at least one second sensor provided on the lens end of the endoscope, and determining whether the target instrument is delivered to the target position through the endoscope according to the first distance.

[0067] The first sensor and the second sensor can be non-shape sensors (i.e., each non-shape sensor can be regarded as a single spatial point in space, such as an image sensor, etc.), or can be shape sensors. Each non-shape sensor is arranged on a certain point of the hard end of the target instrument. For example, as shown in Figure 8 two non-shape sensors are arranged on the two end points of the hard end of the target instrument. Sensor B2 is arranged on the leading end of the hard end of the target instrument, and sensor B1 is arranged on the trailing end of the hard end of the target instrument. For the second sensor, as shown in Figure 9 three non-shape sensors are arranged on the lens end of the endoscope, wherein A1, A2, and A3 are the second sensors (belonging to non-shape sensors), and O1 is the center of the lens end of the endoscope. L12 is a line connecting the centers of the two instrument holes, and L13 is a line passing through the center O1 of the lens end of the endoscope and perpendicular to L2. A11 and A12 are distributed on the two sides of L13, and the line connecting the center of sensor A1 and the center of sensor A2 passes through the center O1 and is parallel to L12. Sensor A3 is located at the intersection of the straight line L13 and the edge (i.e., the outer diameter) of the lens end of the endoscope. The sensors A1, A2, and A3 can feed back three-dimensional coordinates in real time.

[0068] Each shape sensor is arranged on a certain section of the hard end of the target instrument. The shape sensor can be an optical fiber. For the first sensor, as shown in Figure 10As shown, a shape sensor is disposed on a first preset length of the hard end of the target instrument, and the first preset length is a preset proportion of the total length, and the total length is the length of the entire hard end of the target instrument. For example, when the preset proportion is 1, the shape sensor is disposed on the entire hard end of the target instrument, and for example, when the preset proportion is 1 / 2, the shape sensor can be disposed on the upper half or the lower half of the hard end of the target instrument. The shape sensor is fixed inside the target instrument and passes through the central axis inside the target instrument. The second sensor is a shape sensor, which can change shape with the bending of the target instrument. Q1 is the tail end of the hard end of the target instrument, and Q2 is the head end of the hard end of the target instrument. The data obtained by the shape sensor is used for three-dimensional reconstruction of the shape of the target instrument, so that the three-dimensional coordinates of Q1 and Q2 and the bending shape of the target instrument can be obtained. For the second sensor, such as Figure 11 As shown, the shape sensor can be nested in the second preset length of the lens end of the endoscope, and the second preset length is a preset proportion of the circumference, such as one turn of the lens end of the endoscope, or 1 / 4 of the circumference, or 1 / 3 of the circumference, which is not limited. Wherein each point on the shape sensor is a feature point on the target instrument, such as E1, E2, E3, and E4. The line L21 connecting E1 and E2 is parallel to the line L22 connecting the center of the instrument hole, and the line L23 connecting E2 and E3 is perpendicular to L21. The shape sensor can obtain the three-dimensional coordinates of the feature points.

[0069] It should be noted that in order to ensure that the calculated target position is accurate and effective, when determining the target position, if the first sensor is a non-shape sensor, correspondingly, the second sensor is also a non-shape sensor. If the second sensor is a shape sensor, correspondingly, the second sensor is also a shape sensor. That is, the type of the first sensor and the type of the second sensor are matched.

[0070] Wherein, if the first sensor and the second sensor are non-shape sensors, then the number of the first sensor and the number of the second sensor are both multiple, which ensures that the target position can be accurately determined based on the data of multiple sensors, and avoids that the target position obtained from the data of one sensor has a large error.

[0071] Wherein, the end points of the hard end of the target instrument are two, which are the tail end and the head end of the hard end of the target instrument.

[0072] Specifically, in the case that the first sensor and the second sensor are non-shape sensors, the computer device determines the first distance between the end point and the lens end of the endoscope through the first sensor on the end point and each second sensor disposed on the lens end of the endoscope. The computer device determines whether the target instrument is delivered to the target position through the endoscope according to the first distance between the end point and the lens end of the endoscope.

[0073] Alternatively, the computer device determines the first distance between the leading end and the lens end of the endoscope through the first sensor on the leading end and the second sensor arranged at the lens end of the endoscope, and determines the first distance between the trailing end and the lens end of the endoscope through the first sensor on the trailing end and the second sensor arranged at the lens end of the endoscope. The computer device determines whether the target instrument is delivered to the target position through the endoscope according to the first distance between the leading end and the lens end of the endoscope and the first distance between the trailing end and the lens end of the endoscope.

[0074] In the case where the first sensor and the second sensor are shape sensors, the computer device determines the position of the end point in the first sensor, and determines the position information of each point in the second preset length through the second sensor. The computer device determines a plurality of line segments through the position information of any two points. The computer device takes the distance between the end point and each line segment as the first distance. The computer device determines whether the target instrument is delivered to the target position through the endoscope according to each first distance.

[0075] In the embodiment, the first distance between the end point of the hard end of the target instrument and the endoscope can be accurately reflected through the at least one first sensor arranged at the hard end of the target instrument and the at least one second sensor arranged at the lens end of the endoscope. In this way, whether the target instrument reaches the target position can be known in time according to the first distance, and the position of the target instrument for subsequent pose determination is ensured to be accurate without manual determination, thereby avoiding the error of pose determination caused by the inaccurate position of the target instrument.

[0076] In one embodiment, as shown in Figure 12 At least one first sensor is arranged at the leading end or the trailing end of the hard end of the target instrument. The first distance between the end point of the hard end of the target instrument and the lens end of the endoscope is determined through the at least one first sensor arranged at the hard end of the target instrument and the at least one second sensor arranged at the lens end of the endoscope, and whether the target instrument is delivered to the target position through the endoscope is determined according to the first distance.

[0077] In step S1202, the position information of the end point of the hard end of the target instrument is determined based on the first sensing data of the first sensor.

[0078] Specifically, in the case where the first sensor is arranged at the leading end or the trailing end of the hard end of the target instrument, the computer device directly obtains the leading end position information or the trailing end position information of the leading end from the first sensing data. Alternatively, in the case where one first sensor is arranged at the leading end and the trailing end of the hard end of the target instrument respectively, the computer device obtains the leading end position information based on the first sensor arranged at the leading end, and obtains the trailing end position information based on the first sensor arranged at the trailing end.

[0079] In step S1204, the computer device determines the plane position information of the target plane where the lens end of the endoscope is located based on the second sensing data of each second sensor.

[0080] The second sensing data represents the position of the second sensor. The plane position information is the position information of the normal vector of the target plane.

[0081] Specifically, the computer device constructs a plane function of the target plane where the lens end of the endoscope is located, and determines the plane position information of the target plane according to the second sensing data of each second sensor and the plane function.

[0082] For example, the plane where the second sensors A1, A2, and A3 are located is the target plane. Let the plane function be:

[0083] S1=M·x+N·y+Q·z+D

[0084] Substituting each second sensing data into the plane function can determine the parameters M, N, Q, and D, that is, the plane position information (M, N, Q) is determined.

[0085] For example, the second sensing data obtained by each second sensor is respectively the three-dimensional coordinates (x1, y1, z1) of point a1, the three-dimensional coordinates (x2, y2, z2) of point a2, and the three-dimensional coordinates (x3, y3, z3) of point a3. The point a1 can be regarded as the second sensor A1, the point a2 can be regarded as the second sensor A2, and the point a3 can be regarded as the second sensor A3. Specifically, the plane function is the following formula (formula 1):

[0086] M·(x-x1)+N·(y-y1)+Q·(z-z1)=0

[0087] where D=-M·x1-N·y1-Q·z1.

[0088] At this time, substituting the coordinates of points a1, a2, and a3 into formula 1 respectively, the following expressions about M parameter, N parameter, and Q parameter are obtained respectively:

[0089] M=(y2-y1)·(z3-z1)-(z2-z1)·(y3-y1)

[0090] N=(x3-x1)·(z2-z1)-(x2-x1)·(z3-z1)

[0091] Q=(x2-x1)·(y3-y1)-(x3-x1)·(y2-y1)

[0092] Thus, the parameters M, N, Q and D are obtained, i.e., the plane position information (M, N, Q) is obtained.

[0093] In step S1206, based on the position information of the end point of the target instrument hard end portion and the plane position information, a first distance between the end point of the target instrument hard end portion and the lens end of the endoscope is determined.

[0094] Specifically, based on the position information of the end point and the plane position information, the computer device determines the first distance between the end point of the target instrument hard end portion and the lens end of the endoscope through vector calculation.

[0095] For example, taking the determination of the first distance from the head end to the target plane as an example, the plane position information is the normal vector of the target plane The head end position information of the head end B2 is (x B1 , y B1 , z B1 ), and the first distance d1 from the head end to the lens end of the endoscope is:

[0096]

[0097] In step S1208, if the first distance is within a preset distance range, it is determined that the target instrument is delivered to the target position through the endoscope.

[0098] Specifically, in the case where the first sensor is disposed at the tail end of the target instrument hard end portion, the computer device obtains the first distance corresponding to the tail end and determines the first preset distance range corresponding to the tail end. If the first distance corresponding to the tail end is within the first preset distance range corresponding to the tail end, it is determined that the target instrument is delivered to the target position through the endoscope.

[0099] In the case where the first sensor is disposed at the head end of the target instrument hard end portion, the computer device obtains the first distance corresponding to the head end and determines the first preset distance range corresponding to the head end. If the first distance corresponding to the head end is within the second preset distance range corresponding to the head end, it is determined that the target instrument is delivered to the target position through the endoscope.

[0100] In the case where the first sensor is disposed at both the tail end and the head end of the target instrument hard end portion, if the first distance corresponding to the head end is within the first preset distance range and the first distance corresponding to the tail end is within the first preset distance range, it is determined that the target instrument is delivered to the target position through the endoscope.

[0101] The difference between the maximum value in the second preset range and the maximum value in the first preset range is the total length of the target instrument hard end portion. Similarly, the difference between the minimum value in the second preset range and the minimum value in the first preset range is the total length of the target instrument hard end portion.

[0102] It should be noted that the first sensor and the second sensor in the embodiment are both non-shape sensors. Since the non-shape sensors are less affected by temperature or friction and have better robustness, they are not easy to be damaged and are not easy to be affected by the external environment.

[0103] In the embodiment, in the case where the target instrument is delivered through the endoscope, since the target plane where the lens end of the endoscope is located does not change, in the case where the first sensor and the second sensor are non-shape sensors, the plane position information of the target plane can be directly determined accurately and timely through the plurality of second sensor data. In this way, taking the target plane as the reference plane, the first distance from at least one end point to the target plane can be quickly determined, so that the position determination of whether the target instrument reaches the target position can be efficiently completed. In addition, the position determination of the target instrument through the non-shape sensor can effectively avoid the influence of the external environment on the sensor data, so that the effectiveness of the sensor data is ensured.

[0104] In one embodiment, as shown in Figure 13 the first sensor is a shape sensor, the first sensor is arranged at least at the tail end of the hard end of the target instrument, and the second sensor is a shape sensor; the first distance between the end point of the hard end of the target instrument and the lens end of the endoscope is determined through at least one first sensor arranged at the hard end of the target instrument and at least one second sensor arranged at the lens end of the endoscope, and whether the target instrument is delivered to the target position through the endoscope is determined according to the first distance, including:

[0105] In step S1302, the position information of the end point of the hard end of the target instrument is determined based on the plurality of first sensor data of the first sensor.

[0106] In the case where the first sensor and the second sensor are shape sensors, the first sensor is arranged at the tail end of the hard end of the target instrument and extends towards the head end, or the first sensor is arranged at the head end of the hard end of the target instrument and extends towards the tail end. The length of the first sensor is not greater than the distance from the tail end to the head end of the hard end of the target instrument. The second sensor is arranged along the outer periphery of the lens end of the endoscope, and the length of the second sensor is not greater than the circumference of the lens end of the endoscope.

[0107] Specifically, the computer device determines the position information of at least one end point from the plurality of first sensor data.

[0108] It should be noted that when the first sensor is a shape sensor, since each point on the shape sensor can be regarded as a sensing point to obtain the sensing data corresponding to each sensing point, it is not necessary to acquire multiple first sensing data by deploying multiple first sensors on the hard end of the target instrument.

[0109] Step S1304: Based on multiple second sensing data from the second sensor, determine the position information of at least two line segments at the lens end of the endoscope, wherein the line segments at the lens end of the endoscope are determined by any two points on the second sensor.

[0110] The positional information of a line segment refers to the positional information of the two points that make up the line segment.

[0111] Specifically, the computer device acquires multiple second sensor data to determine the position information of at least two line segments on the target plane where the endoscope lens is located.

[0112] Step S1306: Based on the position information of the endpoints of the target instrument's hard end and the position information of two line segments of the lens end of the endoscope, determine the first distance between the endpoints of at least two of the target instrument's hard end and the lens end of the endoscope.

[0113] Specifically, for each line segment, the computer device determines a first distance from the beginning of the target instrument's hard end to the corresponding line segment based on the position information of the beginning of the target instrument's hard end and the position information of two points corresponding to the line segment. Alternatively, for each line segment, the computer device determines a first distance from the end of the target instrument's hard end to the corresponding line segment based on the position information of the end of the target instrument's hard end and the position information of two points corresponding to the line segment.

[0114] For example, such as Figure 14 As shown, the length of the second sensor is the circumference of the endoscope lens. Two line segments are arbitrarily defined: line segment 1 is composed of E1 and E3, and line segment 2 is composed of E2 and E4. For line segment 1, the computer calculates the distance from the beginning Q2 to line segment 1, i.e.:

[0115]

[0116] The computer device calculates the distance from the beginning Q2 to line segment 2, that is:

[0117]

[0118] It should be noted that when determining the line segment, priority should be given to selecting the line segment passing through the center of the endoscope lens end, which facilitates the establishment of a coordinate system and allows for rapid location of points on the line segment. If the line segment is selected arbitrarily, it can be selected in different quadrants first. This way, even when the target plane is uncertain, the distance between the target instrument and the endoscope can be accurately reflected by calculating the distance from the endpoint to the line segment in different quadrants.

[0119] It should be noted that the first and second sensors are shape sensors, which can extract more sensing points (i.e., feature points). In this way, when the delivered target instrument has a large bending angle or multiple bending segments, the position information of multiple sensing points can accurately determine the pose of the target instrument.

[0120] Step S1308: If each of the first distances is within the preset distance range, then it is determined that the target instrument is delivered to the target position through the endoscope.

[0121] Specifically, the computer device compares each first distance with a preset distance range. If each first distance is within the preset distance range, it determines that the target instrument is delivered to the target location via the endoscope.

[0122] In this embodiment, when the first and second sensors are shape sensors, more positional information of the sensing points can still be obtained without deploying multiple sensors. Therefore, by calculating multiple line segments from the endpoint to the lens end of the endoscope, multiple first distances can be obtained. Thus, based on these multiple first distances, it is possible to more accurately determine whether the target instrument has reached the target position.

[0123] In one embodiment, determining the real-time pose of the target device using at least one first sensor disposed at the hard end of the target device includes: determining the head position information and the tail position information of the target device's hard end based on multiple first sensing data from the first sensor; determining the degree of bending of the target device based on the head position information; constructing a reference coordinate system based on the tail position information, the reference coordinate system being centered on the tail end of the target device; and determining the real-time pose of the target device based on the degree of bending and the reference coordinate system.

[0124] In particular, when determining that the target instrument is delivered to the target location via endoscopy, such as Figure 15 As shown. The second sensing data fed back by the second sensors A1, A2, and A3 allows the establishment of a coordinate system T centered on the center of the target plane at the endoscope's lens end. E Z B The axis is perpendicular to the target plane and points in front of the endoscope's field of view, XE The axis is parallel to the line connecting the centers of the two instrument holes and points towards the second sensor A1. E The axis is determined according to the right-hand rule. Taking the left instrument hole as an example, the axis is determined by the center O of the instrument hole. L Establish coordinate system T with the coordinate axes as the center. L Each coordinate axis and T E Parallel. Subsequently, the coordinate system T can be determined based on the geometric relationship between the center of the instrument port and the center of the target plane at the endoscope lens tip. E To T L The transformation relationship is as follows. When the target instrument reaches the target position, it can be assumed that the center point of the tail end coincides with the center point of the instrument hole; that is, the pose of the tail end is consistent with the target plane. Therefore, the reference coordinate system established based on the tail end position information is T. L Therefore, the real-time pose of the target instrument can be directly regarded as the pose of its head relative to the target plane.

[0125] Specifically, when the computer equipment determines that the target instrument has been delivered to the target location via the endoscope, it uses multiple first sensor data to determine the position information of the head end and the tail end of the target instrument's rigid end. The computer equipment then determines the degree of curvature of the target instrument using a preset spatial transformation function and the head end position information. The computer equipment establishes a reference coordinate system centered on the tail end of the target instrument and based on the tail end coordinate information. The computer equipment determines the transformation matrix of the coordinate system using the degree of curvature and the reference coordinate system. Based on this transformation matrix, the computer equipment determines the position information of the head end in the reference coordinate system; this position information of the head end in the reference coordinate system is the real-time pose of the target instrument.

[0126] The preset spatial transformation function can be determined by the position information of multiple sample ends and the degree of curvature of multiple samples. For example, the degree of curvature of the sample ends at different positions can be measured in real time, that is, each sample end position corresponds to a sample curvature. Through data fitting, the mapping relationship between the end position information and the degree of curvature can be determined.

[0127] For example, such as Figure 16 As shown, the target instrument is simplified as an arc. A coordinate system Tee is established with the tip (i.e., the first sensor B2) as the center. When the target instrument is in a flat initial state, the directions of each coordinate axis are consistent with the reference coordinate system. The projection point of the tip onto the target plane where the endoscope lens is located is P. L O L P L With Y L The included angle of the axis is That is, the beginning is in the reference coordinate system T L The bending direction within. The tip of the target instrument's rigid end faces the Zee axis and Z.L The angle between the axis is θ, which is the bending angle of the head end in the reference coordinate system T L The computer device determines the bending direction and the bending angle according to the position information (Xee, Yee, Zee) of the head end and the preset space conversion function F1, and determines the bending degree, that is:

[0128]

[0129] The computer device takes the tail end of the target instrument as the center and establishes a reference coordinate system according to the tail end coordinate information. The computer device determines the conversion matrix of the coordinate system Thus, the position information T ee→L of the head end in the reference coordinate system can be determined, that is, the real-time pose:

[0130]

[0131] The computer device obtains the error matrix F error , and determines the error err between the preset pose and the real-time position according to the error matrix, that is:

[0132] err = F error (T Target , T ee→L )

[0133] Wherein, T Target represents the preset pose.

[0134] In this embodiment, therefore, when the target instrument reaches the target position, the real-time pose of the target instrument can be automatically and timely determined through the at least one first sensor arranged at the hard end of the target instrument, and the error caused by manual adjustment of the instrument is avoided.

[0135] In one embodiment, as Figure 17 shown, if the real-time pose deviates from the preset pose, the target instrument is controlled to adjust to the preset pose, including:

[0136] Step S1702, updating the previous mechanism position of each rotating mechanism in the target instrument based on the previous to-be-corrected pose and the preset pose to determine the current mechanism position; wherein the previous to-be-corrected pose corresponding to the first iteration is the real-time pose.

[0137] Wherein, the adjusted pose after each iteration is taken as the to-be-corrected pose. The target instrument includes a flexible snake bone with multiple rotating mechanisms. The mechanism can adjust the flexibility and the maximum bending angle of the target instrument.

[0138] Specifically, when the computer device determines the real-time pose and the preset pose offset, an iteration process is entered. The computer device determines a previous to-be-corrected pose corresponding to a previous iteration, and updates the previous mechanism position through the previous to-be-corrected pose and the preset pose by Jacobian matrix calculation to determine a current mechanism position.

[0139] In the first iteration, the previous to-be-corrected pose is the real-time pose.

[0140] In the first iteration, the previous to-be-corrected pose is the real-time pose.

[0141] Specifically, the computer device determines the current length of each drive wire based on the position relationship of each rotating mechanism and the geometric relationship between the rotating mechanism and the drive wire. For example, the current length of each drive wire is as follows:

[0142] L1~Ln=F2(q1~qn)

[0143] In the first iteration, the previous to-be-corrected pose is the real-time pose.

[0144] In the first iteration, the previous to-be-corrected pose is the real-time pose.

[0145] Specifically, the computer device adjusts the previous to-be-corrected pose through the current length of each drive wire to obtain a current to-be-corrected pose.

[0146] Specifically, the computer device adjusts the previous to-be-corrected pose through the current length of each drive wire to obtain a current to-be-corrected pose.

[0147] Specifically, the computer device determines the error between the current pose to be corrected and the preset pose. If the error is not within the preset error range, the computer device uses the current pose to be corrected as the previous pose to be corrected in the next iteration and the current pose to be corrected as the previous mechanism position in the next iteration. It then returns to update the previous mechanism position of each rotating mechanism in the target device based on the previous pose to be corrected and the preset pose. The step of determining the current mechanism position continues to be executed until the error is within the preset error range, at which point it stops and determines that the target device has been adjusted to the preset pose.

[0148] In this embodiment, if the real-time pose and the preset pose offset are determined, the current length of the drive wire for restoring the target instrument to the preset pose is determined through iterative calculations. This allows for accurate adjustment of the target instrument to the preset pose, avoiding multiple operations by the operator and improving the efficiency of pose adjustment.

[0149] In one embodiment, updating the previous positions of each rotating mechanism in the target device based on the previous pose to be corrected and the preset pose, and determining the current mechanism position, includes: calculating the current joint velocity of the target device using the Jacobian matrix based on the previous pose to be corrected and the preset pose; and determining the current mechanism position based on the current joint velocity and the previous mechanism position.

[0150] Specifically, the computer equipment performs Jacobi matrix calculations on the previous pose to be corrected and the preset pose to obtain the current joint velocity of the target device. The computer equipment integrates the current joint velocity over time to obtain the integral result, and then superimposes the integral result with the previous mechanism position to obtain the current mechanism position for the current iteration.

[0151] For example, the computer device performs Jacobian matrix calculations on the previous pose to be corrected and the preset pose to obtain the end-effector velocity v of the hard end of the target instrument. e Computer equipment operates based on the speed v of the hardware's end. e And the previous institutional position q(t) n-1 The position q(t) of the mechanism in the current iteration is obtained by Euler integration. n ), as shown below:

[0152] q(t n )=q(t n-1 )+J -1 (q(t n-1 ))v e (t n-1 )Δt

[0153] Wherein, the time corresponding to the current iteration is t. n The time corresponding to the previous iteration is t. n-1 Where Δt represents the time integral, J-1 (q(t n-1 )) represents the inverse matrix of the Jacobian matrix at the previous time (the time corresponding to the previous iteration), J -1 (q(t n-1 ))v e (t n-1 ) represents the joint velocity at the current time, which is determined by J -1 (q(t n-1 ))v e (t n-1 )Δt determines the increased joint position, and thus, the current mechanism position q(t n-1 ) is obtained by adding the increased joint position to the joint position q(t n ) at the previous time. That is, the current mechanism position at the current time is determined by Euler integration.

[0154] In the embodiment, the Jacobian matrix is performed on the previous to-be-corrected pose and the preset pose, so that the joint velocity at the current iteration can be accurately determined. In this way, the increment of the rotating mechanism can be obtained by integrating the joint velocity at the current iteration, so that the mechanism position at the current iteration of the current iteration can be accurately obtained, which is beneficial to subsequent determination of the current length of the driving wire, so as to accurately adjust the target instrument to the preset pose.

[0155] In one embodiment, the method further comprises at least one of the following: first, outputting the first distance between the end point of the hard end of the target instrument and the lens end of the endoscope to a display device for display; second, determining the delivery length of the hard end of the target instrument delivered out of the lens end of the endoscope according to the first distance between the end point of the hard end of the target instrument and the lens end of the endoscope. The delivery length is output to a display device for display.

[0156] It should be noted that the computer device and the image platform can display the first distance and the delivery length.

[0157] For the first, after determining the first distance, the display device in the computer device displays an interactive interface containing the first distance to realize the display of the first distance. At the same time, the image platform also displays an interactive interface containing the first distance to realize the display of the first distance. For the second, after determining the first distance, the computer device determines the delivery length according to the first distance, and the display device in the computer device displays an interactive interface containing the delivery length to realize the display of the delivery length. At the same time, the image platform also displays an interactive interface containing the delivery length to realize the display of the delivery length.

[0158] For example, the interactive interface can be a pop-up window containing the first distance or the delivery length. Figure 18As shown, the instrument information module is used to display the instrument type, instrument delivery distance (i.e., corresponding to the delivery length), the first distance, etc. The endoscope image display module is used to display the image observed by the lens end of the endoscope in real time. The warning information module is used to prompt the operator of various emergency information, such as the endoscope and the instrument reaching the limit position, etc. The endoscope information module is used to display the self-rotation angle of the endoscope, the delivery length, and other information. The object to be detected information module is used to display the physical information of the target object.

[0159] In this embodiment, by outputting the first distance to the display device for display, the movement process of the target instrument can be fed back in real time. At the same time, after determining the delivery length of the hard end of the target instrument out of the lens end of the endoscope through the first distance, the delivery length is displayed on the display device, so that the operator can intuitively observe whether the target instrument has moved to the target position, and can timely remind the operator to repair the target instrument through the target position.

[0160] In one embodiment, in order to facilitate a more clear understanding of the technical solutions of the present application, a more detailed embodiment is provided for description. The present embodiment realizes the control of the instrument pose through a computer device, an operation platform, and an image platform. Figure 19 As shown, the computer device includes a main console, a processing unit, and a storage unit. The operation platform includes an endoscope, an instrument, a sensing unit, an instrument driving unit, an endoscope driving unit, a self-rotation unit, and an endoscope delivery unit. The image module includes an image acquisition unit and a pose display unit.

[0161] The computer device further includes a display device, a main operation arm, and a clutch adjustment member. The display device is used to display the 2D (two dimensional) or 3D (3-Dimensions) image observed by the endoscope in real time. The main operation arm and the execution end slave arm are configured in a master-slave control relationship, so as to control the movement of the surgical instrument or the auxiliary device such as the endoscope mounted on the slave arm through the main operation arm. The clutch adjustment member is used to establish or disconnect the master-slave control relationship, and control the instrument or the endoscope at the execution end to perform other actions.

[0162] The instrument driving unit can independently carry and control one instrument, and can realize the bending, advancing and retreating, self-rotation, clamping, and other actions of the instrument in any direction. The endoscope driving unit is used to carry the endoscope and realize the bending of the endoscope in any direction. The endoscope self-rotation unit is used to control the self-rotation of the endoscope. The endoscope delivery unit is used to deliver the endoscope into the natural cavity.

[0163] Specifically, the endoscope provides an image of a target object in a body, and an image acquisition unit acquires the endoscope image. A sensing unit senses and acquires position data of the endoscope and the target instrument, and sends the position data to a processing unit, which performs real-time calculation on the endoscope and instrument pose and transmits the pose to a pose display unit, so that the pose display unit displays the pose of the endoscope and the target instrument in real time. The processing unit processes the data fed back by the driving unit and performs error control, and sends a motion instruction to the driving unit, so that the driving unit drives the endoscope and the target instrument to reach a preset pose. Then, the target instrument performs biopsy operation, abnormality repair operation, etc. after reaching the preset pose. The storage unit is used to store the data of each unit in real time. The processing unit is also used to process the instructions sent by the master control hand and perform trajectory planning on the execution unit.

[0164] As shown in Figure 20 , the control method of the instrument pose is as follows:

[0165] Step 1: The computer device receives a delivery instruction generated by a trigger operation of an operator, and determines a delivery mode of the endoscope and the target instrument according to the delivery instruction. In the case that the delivery mode of the endoscope is a manual delivery mode, the computer device receives image information collected by the endoscope, and determines whether the endoscope reaches an abnormal area. Alternatively, in the case that the delivery mode of the endoscope is an automatic delivery mode, the computer device sends a first driving instruction to the endoscope to control the endoscope to deliver, and determines whether the endoscope reaches the abnormal area through the image information collected by the endoscope. If not, continue to deliver the endoscope, if yes, deliver the instrument.

[0166] Step 2: In the case that the first sensor and the second sensor are non-shape sensors, based on the first sensing data of the first sensor, the computer device determines the position information of the end point of the hard end of the target instrument. Based on the second sensing data of each second sensor, the computer device determines the plane position information of the target plane where the lens end of the endoscope is located. Based on the position information of the end point of the hard end of the target instrument and the plane position information, the computer device determines the first distance between the end point of the hard end of the target instrument and the lens end of the endoscope. If the first distance is within a preset distance range, the computer device determines that the target instrument is delivered to the target position through the endoscope (i.e., the target instrument is completely delivered). If the first distance is not within the preset distance range, the target instrument is not completely delivered, and the delivery of the target instrument continues.

[0167] Or, in the case that the first sensor and the second sensor are shape sensors, the first sensor is disposed at least at the tip or tail end of the target instrument rigid end portion. Based on the plurality of first sensing data of the first sensor, position information of the end point of the target instrument rigid end portion is determined. Based on the plurality of second sensing data of the second sensor, position information of at least two line segments of the lens end of the endoscope is determined, the line segments of the lens end of the endoscope being determined by any two points on the second sensor. Based on the position information of the end point of the target instrument rigid end portion and the position information of the at least two line segments of the lens end of the endoscope, at least two first distances between the end point of the target instrument rigid end portion and the lens end of the endoscope are determined. If each of the first distances is within a preset distance range, it is determined that the target instrument is delivered to the target position through the endoscope (i.e., the target instrument is completely delivered). If there is at least a first distance that is not within the preset distance range, the target instrument is not completely delivered, and the delivery of the target instrument continues.

[0168] Step 3: Based on the plurality of first sensing data of the first sensor, head end position information of the head end of the target instrument rigid end portion and tail end position information of the tail end of the target instrument rigid end portion are determined. Based on the head end position information, the bending degree of the target instrument is determined. Based on the tail end position information, a reference coordinate system is constructed, which is a coordinate system determined according to the tail end of the target instrument as the center. Based on the bending degree and the reference coordinate system, the real-time pose of the target instrument is determined. If the real-time pose deviates from the preset pose, an automatic path is planned according to the real-time pose to automatically correct the target instrument, specifically, based on the previous to-be-corrected pose and the preset pose, the previous mechanism position of each rotating mechanism in the target instrument is updated to determine the current mechanism position; wherein the previous to-be-corrected pose corresponding to the first iteration is the real-time pose. Based on the current mechanism position, the current length of each driving wire in the target instrument is determined; wherein the previous mechanism position corresponding to the first iteration is determined by the real-time pose. Based on each current length, the previous to-be-corrected pose is updated to obtain the current to-be-corrected pose. In the case that the error between the current to-be-corrected pose and the preset pose is not within a preset error range, the current to-be-corrected pose is taken as the previous to-be-corrected pose of the next iteration, and the current to-be-corrected pose is taken as the previous mechanism position of the next iteration, and the step of updating the previous mechanism position of each rotating mechanism in the target instrument based on the previous to-be-corrected pose and the preset pose to determine the current mechanism position is continued to be executed (i.e., the target instrument is automatically corrected), until the error is within the preset error range, and it is determined that the target instrument is adjusted to the preset pose. At this time, it is determined that the pose of the target instrument is corrected, and the abnormality repair operation is waited to start.

[0169] In the embodiment, by determining in advance that the target instrument is delivered to the target position through the endoscope, it is avoided that the position of the target instrument needs to be additionally adjusted after the pose is determined. In this way, the pose determination can be effectively performed based on the target instrument reaching the target position. Therefore, when the target instrument reaches the target position, the real-time pose of the target instrument can be automatically and timely determined through the at least one first sensor arranged at the hard end of the target instrument, errors caused by manual adjustment of the instrument are avoided, the efficiency of the control of the pose of the instrument is greatly improved, and the preparation time before the abnormal repair operation is significantly shortened. In addition, errors in pose adjustment caused by dirt blocking and the like are also avoided. In addition, the reset path and the delivery distance of the target instrument are automatically sensed and planned by the system, the safety and convenience of the operation are improved, and the service life of the target instrument is greatly improved by the fast automatic reset of the target instrument, avoiding the target instrument being kept at the maximum bending angle for a long time and the internal driving wire being kept at a too large tension caused by the bending coupling of the endoscope and friction.

[0170] It should be understood that, although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0171] Based on the same inventive concept, the embodiment of the present application also provides an instrument pose control device for implementing the above-mentioned instrument pose control method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more instrument pose control device embodiments provided below can refer to the limitations of the instrument pose control method in the foregoing, which will not be described here again.

[0172] In one embodiment, as shown in Figure 21 An instrument pose control device is provided, comprising: a determination module 2102 and a control module 2104, wherein:

[0173] The determination module 2102 is configured to determine the real-time pose of the target instrument through at least one first sensor arranged at the hard end of the target instrument when it is determined that the target instrument is delivered to the target position through the endoscope.

[0174] The control module 2104 is configured to control the target instrument to adjust to the preset pose if the real-time pose deviates from the preset pose, and the preset pose is determined based on a position of the object to be detected and a position of a scope hole of the endoscope in which the target instrument is located.

[0175] In one embodiment, the determination module 2102 is configured to determine a first distance between the end point of the hard end of the target instrument and the lens end of the endoscope by using at least one first sensor arranged at the hard end of the target instrument and at least one second sensor arranged at the lens end of the endoscope, and determine whether the target instrument is delivered to the target position through the endoscope according to the first distance.

[0176] In one embodiment, the determination module 2102 is configured to determine position information of the end point of the hard end of the target instrument based on first sensing data of the first sensor, determine plane position information of a target plane in which the lens end of the endoscope is located based on second sensing data of the second sensor, and determine the first distance between the end point of the hard end of the target instrument and the lens end of the endoscope based on the position information of the end point of the hard end of the target instrument and the plane position information. If the first distance is within a preset distance range, it is determined that the target instrument is delivered to the target position through the endoscope.

[0177] In one embodiment, the determination module 2102 is configured to determine position information of the end point of the hard end of the target instrument based on a plurality of first sensing data of the first sensor, determine position information of at least two line segments of the lens end of the endoscope based on a plurality of second sensing data of the second sensor, and determine at least two first distances between the end point of the hard end of the target instrument and the lens end of the endoscope based on the position information of the end point of the hard end of the target instrument and the position information of the at least two line segments of the lens end of the endoscope. If each of the first distances is within a preset distance range, it is determined that the target instrument is delivered to the target position through the endoscope.

[0178] In one embodiment, the control module 2104 is configured to determine head position information of a head end of the hard end of the target instrument and tail position information of a tail end of the hard end of the target instrument based on a plurality of first sensing data of the first sensor, determine a bending degree of the target instrument based on the head position information, construct a reference coordinate system based on the tail position information, and determine a real-time pose of the target instrument based on the bending degree and the reference coordinate system.

[0179] In an embodiment, the control module 2104 is configured to update a previous mechanism position of each rotating mechanism in the target instrument based on a previous to-be-corrected pose and a preset pose, and determine a current mechanism position; wherein the previous to-be-corrected pose corresponding to the first iteration is the real-time pose. Based on the current mechanism position, a current length of each driving wire in the target instrument is determined; wherein the previous mechanism position corresponding to the first iteration is determined by the real-time pose. Based on each current length, the previous to-be-corrected pose is updated to obtain a current to-be-corrected pose. In a case where an error between the current to-be-corrected pose and the preset pose is not within a preset error range, the current to-be-corrected pose is taken as the previous to-be-corrected pose of the next iteration, and the current to-be-corrected pose is taken as the previous mechanism position of the next iteration, and the step of updating the previous mechanism position of each rotating mechanism in the target instrument based on the previous to-be-corrected pose and the preset pose to determine the current mechanism position is continued to execute until the error is within the preset error range, and it is determined that the target instrument is adjusted to the preset pose.

[0180] In an embodiment, the control module 2104 is configured to obtain a current joint speed of the target instrument by Jacobian matrix calculation based on a previous to-be-corrected pose and a preset pose. Based on the current joint speed and the previous mechanism position, a current mechanism position is determined.

[0181] In an embodiment, the control module 2104 is configured to output a first distance between an end point of a hard end of the target instrument and a lens end of the endoscope to a display device for display. The control module 2104 is configured to determine a delivery length of the hard end of the target instrument delivered out of the lens end of the endoscope according to the first distance between the end point of the hard end of the target instrument and the lens end of the endoscope. The delivery length is output to the display device for display.

[0182] Each module in the above apparatus for controlling the pose of an instrument can be implemented wholly or partially by software, hardware, and combinations thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform operations corresponding to each module.

[0183] In an embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in FIG. 8. Figure 22As shown in the figure. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the control data of the instrument pose. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with the terminal outside through network connection. The computer program is executed by the processor to realize an instrument pose control method.

[0184] Those skilled in the art can understand that, Figure 22 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0185] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to realize the steps in each of the above method embodiments.

[0186] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by the processor to realize the steps in each of the above method embodiments.

[0187] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by the processor to realize the steps in each of the above method embodiments.

[0188] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0189] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0190] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0191] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A device for controlling the posture of an instrument, characterized in that, The device includes: The determination module is used to determine the real-time pose of the target instrument by using at least one first sensor disposed at the hard end of the target instrument when the target instrument is determined to be delivered to the target position through the endoscope; The control module is used to control the target instrument to adjust to the preset pose if the real-time pose deviates from the preset pose; the preset pose is determined by the position of the object to be detected and the position of the instrument port of the endoscope where the target instrument is located. The control module is also used to perform at least one iterative calculation on the real-time pose based on the preset pose until the error between the pose of the current iteration and the preset pose is within the preset error range. The control module is further configured to perform at least one iterative calculation on the real-time pose based on a preset pose, including: using the real-time pose as the previous pose to be corrected in the first iteration, and performing iteration; for each iteration, if the previous pose to be corrected deviates from the preset pose, adjusting the drive wire to obtain the current length of the drive wire in the current iteration, and determining the pose to be corrected in the current iteration through the current length; wherein, the length of the drive wire affects the real-time pose of the target instrument, and the change in the length of the drive wire is caused by the change in the drive mechanism of the target instrument.

2. The apparatus according to claim 1, characterized in that, The determining module is further configured to determine a first distance between the endpoint of the hard end of the target instrument and the lens end of the endoscope by using at least one first sensor disposed on the hard end of the target instrument and at least one second sensor disposed on the lens end of the endoscope, and determine whether the target instrument is delivered to the target position by the endoscope based on the first distance.

3. The apparatus according to claim 2, characterized in that, At least one first sensor is provided at the end or head of the hard end of the target instrument; The determining module is further configured to determine the position information of the endpoint of the hard end of the target instrument based on the first sensing data of the first sensor; Based on the second sensing data of each second sensor, the planar position information of the target plane where the lens end of the endoscope is located is determined; Based on the position information of the endpoint of the target instrument's hard end and the planar position information, a first distance between the endpoint of the target instrument's hard end and the lens end of the endoscope is determined; if the first distance is within a preset distance range, it is determined that the target instrument is transported to the target position through the endoscope.

4. The apparatus according to claim 2, characterized in that, The first sensor is a shape sensor, and the first sensor is at least disposed at the end or tail end of the hard end of the target instrument; the second sensor is a shape sensor. The determining module is further configured to determine the position information of the endpoint of the target instrument's hard end based on multiple first sensing data from the first sensor. Based on multiple second sensor data from the second sensor, the position information of at least two line segments at the lens end of the endoscope is determined, and the line segments at the lens end of the endoscope are determined by any two points on the second sensor; based on the position information of the endpoint of the target instrument's hard end and the position information of the two line segments at the lens end of the endoscope, a first distance between at least two endpoints of the target instrument's hard end and the lens end of the endoscope is determined; if each of the first distances is within a preset distance range, it is determined that the target instrument is transported to the target position through the endoscope.

5. The apparatus according to any one of claims 1 to 4, characterized in that, The control module is further configured to determine the head position information of the head end of the target instrument and the tail position information of the tail end of the target instrument based on multiple first sensing data from the first sensor; determine the degree of bending of the target instrument based on the head position information; construct a reference coordinate system based on the tail position information, the reference coordinate system being determined with the tail end of the target instrument as the center; and determine the real-time pose of the target instrument based on the degree of bending and the reference coordinate system.

6. The apparatus according to claim 1, characterized in that, The control module is further configured to update the previous mechanism position of each rotating mechanism in the target device based on the previous pose to be corrected and the preset pose, and determine the current mechanism position; wherein, the previous pose to be corrected corresponding to the first iteration is the real-time pose; based on the current mechanism position, determine the current length of each drive wire in the target device; wherein, the previous mechanism position corresponding to the first iteration is determined by the real-time pose; based on each current length, update the previous pose to be corrected to obtain the current pose to be corrected; if it is determined that the error between the current pose to be corrected and the preset pose is not within the preset error range, use the current pose to be corrected as the previous pose to be corrected for the next iteration, and use the current pose to be corrected as the previous mechanism position for the next iteration, return to the step of updating the previous mechanism position of each rotating mechanism in the target device based on the previous pose to be corrected and the preset pose, and continue to execute until the error is within the preset error range, and stop, determining that the target device has been adjusted to the preset pose.

7. The apparatus according to claim 6, characterized in that, The control module is also used to calculate the current joint velocity of the target device using a Jacobian matrix based on the previous pose to be corrected and the preset pose; and to determine the current mechanism position based on the current joint velocity and the previous mechanism position.

8. The apparatus according to claim 3 or 4, characterized in that, The control module is further configured to output a first distance between the endpoint of the target instrument's hard end and the lens end of the endoscope to a display device for display; the control module is further configured to determine the delivery length of the target instrument's hard end from the lens end of the endoscope based on the first distance between the endpoint of the target instrument's hard end and the lens end of the endoscope; and output the delivery length to a display device for display.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, The steps implemented by the processor when executing the computer program include: When it is determined that the target instrument has been delivered to the target position through the endoscope, the real-time pose of the target instrument is determined by at least one first sensor installed at the hard end of the target instrument. If the real-time pose deviates from the preset pose, the target instrument is controlled to adjust to the preset pose; the preset pose is determined by the position of the object to be detected and the position of the instrument port of the endoscope where the target instrument is located. The method of controlling the target device to adjust to the preset pose includes: performing at least one iterative calculation on the real-time pose based on the preset pose until the error between the pose of the current iteration and the preset pose is within the preset error range. The calculation of the real-time pose based on the preset pose is performed at least once, including: taking the real-time pose as the previous pose to be corrected in the first iteration and performing iteration; for each iteration, when the previous pose to be corrected deviates from the preset pose, adjusting the drive wire to obtain the current length of the drive wire in the current iteration, and determining the pose to be corrected in the current iteration through the current length; wherein, the length of the drive wire affects the real-time pose of the target instrument, and the change in the length of the drive wire is caused by the change in the drive mechanism of the target instrument.

10. The computer device according to claim 9, characterized in that, The method of delivering the target instrument to the target location via an endoscope includes: By using at least one first sensor disposed at the hard end of the target instrument and at least one second sensor disposed at the lens end of the endoscope, a first distance between the endpoint of the hard end of the target instrument and the lens end of the endoscope is determined, and the target instrument is determined to be delivered to the target position by the endoscope based on the first distance.

11. The computer device according to claim 10, characterized in that, At least one first sensor is provided at the end or head of the hard end of the target instrument; The step of determining a first distance between the endpoint of the target instrument's rigid end and the end of the endoscope's lens end using at least one first sensor disposed on the rigid end of the target instrument and at least one second sensor disposed on the lens end of the endoscope, and determining whether the target instrument has been delivered to the target position via the endoscope based on the first distance, includes: Based on the first sensing data from the first sensor, the position information of the endpoint of the target instrument's rigid end is determined; Based on the second sensing data of each second sensor, the planar position information of the target plane where the lens end of the endoscope is located is determined; Based on the position information of the endpoint of the target instrument's hard end and the planar position information, a first distance between the endpoint of the target instrument's hard end and the lens end of the endoscope is determined; If the first distance is within the preset distance range, then it is determined that the target instrument is delivered to the target location via the endoscope.

12. The computer device according to claim 10, characterized in that, The first sensor is a shape sensor, and the first sensor is at least disposed at the end or tail end of the hard end of the target instrument; the second sensor is a shape sensor. The step of determining a first distance between the endpoint of the target instrument's rigid end and the end of the endoscope's lens end using at least one first sensor disposed on the rigid end of the target instrument and at least one second sensor disposed on the lens end of the endoscope, and determining whether the target instrument has been delivered to the target position via the endoscope based on the first distance, includes: Based on multiple first sensing data from the first sensor, the position information of the endpoint of the target instrument's hard end is determined; Based on multiple second sensing data from the second sensor, the position information of at least two line segments at the lens end of the endoscope is determined, wherein the line segments at the lens end of the endoscope are determined by any two points on the second sensor. Based on the position information of the endpoints of the target instrument's hard end and the position information of two line segments of the lens end of at least the endoscope, a first distance is determined between the endpoints of at least two of the target instrument's hard end and the lens end of the endoscope. If each of the first distances is within the preset distance range, then it is determined that the target instrument is delivered to the target position via the endoscope.

13. The computer device according to any one of claims 9 to 12, characterized in that, The step of determining the real-time pose of the target instrument by means of at least one first sensor disposed at the hard end of the target instrument includes: Based on multiple first sensing data from the first sensor, the head position information of the head end of the target instrument and the tail position information of the tail end of the target instrument are determined. Based on the aforementioned end position information, the degree of bending of the target instrument is determined; A reference coordinate system is constructed based on the tail end position information. The reference coordinate system is determined with the tail end of the target instrument as the center. Based on the degree of curvature and the reference coordinate system, the real-time pose of the target instrument is determined.

14. The computer device according to claim 9, characterized in that, If the real-time pose deviates from the preset pose, the step of controlling the target device to adjust to the preset pose includes: Based on the previous pose to be corrected and the preset pose, the previous positions of each rotating mechanism in the target device are updated to determine the current position of the mechanism; wherein, the previous pose to be corrected corresponding to the first iteration is the real-time pose. Based on the current mechanism position, the current length of each drive wire in the target device is determined; wherein, the previous mechanism position corresponding to the first iteration is determined by the real-time pose. Based on each current length, the previous pose to be corrected is updated to obtain the current pose to be corrected. If it is determined that the error between the current pose to be corrected and the preset pose is not within the preset error range, the current pose to be corrected is used as the previous pose to be corrected in the next iteration, and the current pose to be corrected is used as the previous mechanism position in the next iteration. Based on the previous pose to be corrected and the preset pose, the previous mechanism position of each rotating mechanism in the target device is updated. The step of determining the current mechanism position continues to be executed until the error is within the preset error range, and then stops, determining that the target device has been adjusted to the preset pose.

15. The computer device according to claim 14, characterized in that, The step of updating the previous mechanism position of each rotating mechanism in the target device based on the previous pose to be corrected and the preset pose, and determining the current mechanism position, includes: Based on the previous pose to be corrected and the preset pose, the current joint velocity of the target device is obtained by calculating using the Jacobian matrix. The current mechanism position is determined based on the current joint velocity and the previous mechanism position.

16. The computer device according to claim 11 or 12, characterized in that, The steps performed by the processor when executing the computer program also include at least one of the following: The first method involves outputting the first distance between the end point of the target instrument's rigid end and the lens end of the endoscope to a display device for display. The second method: Determine the delivery length of the target instrument's hard end from the end of the endoscope's lens end based on the first distance between the endpoint of the target instrument's hard end and the end of the endoscope's lens end; The output length is then sent to a display device for display.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, The steps implemented when the computer program is executed by the processor include: When it is determined that the target instrument has been delivered to the target position through the endoscope, the real-time pose of the target instrument is determined by at least one first sensor installed at the hard end of the target instrument. If the real-time pose deviates from the preset pose, the target instrument is controlled to adjust to the preset pose; the preset pose is determined by the position of the object to be detected and the position of the instrument port of the endoscope where the target instrument is located. The method of controlling the target device to adjust to the preset pose includes: performing at least one iterative calculation on the real-time pose based on the preset pose until the error between the pose of the current iteration and the preset pose is within the preset error range. The calculation of the real-time pose based on the preset pose is performed at least once, including: taking the real-time pose as the previous pose to be corrected in the first iteration and performing iteration; for each iteration, when the previous pose to be corrected deviates from the preset pose, adjusting the drive wire to obtain the current length of the drive wire in the current iteration, and determining the pose to be corrected in the current iteration through the current length; wherein, the length of the drive wire affects the real-time pose of the target instrument, and the change in the length of the drive wire is caused by the change in the drive mechanism of the target instrument.

18. The computer-readable storage medium according to claim 17, characterized in that, The method of delivering the target instrument to the target location via an endoscope includes: By using at least one first sensor disposed at the hard end of the target instrument and at least one second sensor disposed at the lens end of the endoscope, a first distance between the endpoint of the hard end of the target instrument and the lens end of the endoscope is determined, and the target instrument is determined to be delivered to the target position by the endoscope based on the first distance.

19. The computer-readable storage medium according to claim 18, characterized in that, At least one first sensor is provided at the end or head of the hard end of the target instrument; The step of determining a first distance between the endpoint of the target instrument's rigid end and the end of the endoscope's lens end using at least one first sensor disposed on the rigid end of the target instrument and at least one second sensor disposed on the lens end of the endoscope, and determining whether the target instrument has been delivered to the target position via the endoscope based on the first distance, includes: Based on the first sensing data from the first sensor, the position information of the endpoint of the target instrument's rigid end is determined; Based on the second sensing data of each second sensor, the planar position information of the target plane where the lens end of the endoscope is located is determined; Based on the position information of the endpoint of the target instrument's hard end and the planar position information, a first distance between the endpoint of the target instrument's hard end and the lens end of the endoscope is determined; If the first distance is within the preset distance range, then it is determined that the target instrument is delivered to the target location via the endoscope.

20. The computer-readable storage medium according to claim 18, characterized in that, The first sensor is a shape sensor, and the first sensor is at least disposed at the end or tail end of the hard end of the target instrument; the second sensor is a shape sensor. The step of determining a first distance between the endpoint of the target instrument's rigid end and the end of the endoscope's lens end using at least one first sensor disposed on the rigid end of the target instrument and at least one second sensor disposed on the lens end of the endoscope, and determining whether the target instrument has been delivered to the target position via the endoscope based on the first distance, includes: Based on multiple first sensing data from the first sensor, the position information of the endpoint of the target instrument's hard end is determined; Based on multiple second sensing data from the second sensor, the position information of at least two line segments at the lens end of the endoscope is determined, wherein the line segments at the lens end of the endoscope are determined by any two points on the second sensor. Based on the position information of the endpoints of the target instrument's hard end and the position information of two line segments of the lens end of at least the endoscope, a first distance between the endpoints of at least two of the target instrument's hard end and the lens end of the endoscope is determined. If each of the first distances is within the preset distance range, then it is determined that the target instrument is delivered to the target position via the endoscope.

21. The computer-readable storage medium according to any one of claims 17 to 20, characterized in that, The step of determining the real-time pose of the target instrument by means of at least one first sensor disposed at the hard end of the target instrument includes: Based on multiple first sensing data from the first sensor, the head position information of the head end of the target instrument and the tail position information of the tail end of the target instrument are determined. Based on the aforementioned end position information, the degree of bending of the target instrument is determined; A reference coordinate system is constructed based on the tail end position information. The reference coordinate system is determined with the tail end of the target instrument as the center. Based on the degree of curvature and the reference coordinate system, the real-time pose of the target instrument is determined.

22. The computer-readable storage medium according to claim 17, characterized in that, If the real-time pose deviates from the preset pose, the step of controlling the target device to adjust to the preset pose includes: Based on the previous pose to be corrected and the preset pose, the previous positions of each rotating mechanism in the target device are updated to determine the current position of the mechanism; wherein, the previous pose to be corrected corresponding to the first iteration is the real-time pose. Based on the current mechanism position, the current length of each drive wire in the target device is determined; wherein, the previous mechanism position corresponding to the first iteration is determined by the real-time pose. Based on each current length, the previous pose to be corrected is updated to obtain the current pose to be corrected. If it is determined that the error between the current pose to be corrected and the preset pose is not within the preset error range, the current pose to be corrected is used as the previous pose to be corrected in the next iteration, and the current pose to be corrected is used as the previous mechanism position in the next iteration. Based on the previous pose to be corrected and the preset pose, the previous mechanism position of each rotating mechanism in the target device is updated. The step of determining the current mechanism position continues to be executed until the error is within the preset error range, and then stops, determining that the target device has been adjusted to the preset pose.

23. The computer-readable storage medium according to claim 22, characterized in that, The step of updating the previous mechanism position of each rotating mechanism in the target device based on the previous pose to be corrected and the preset pose, and determining the current mechanism position, includes: Based on the previous pose to be corrected and the preset pose, the current joint velocity of the target device is obtained by calculating using the Jacobian matrix. The current mechanism position is determined based on the current joint velocity and the previous mechanism position.

24. The computer-readable storage medium according to claim 19 or 20, characterized in that, The steps implemented when the computer program is executed by the processor also include at least one of the following: The first method involves outputting the first distance between the end point of the target instrument's rigid end and the lens end of the endoscope to a display device for display. The second method: Determine the delivery length of the target instrument's hard end from the end of the endoscope's lens end based on the first distance between the endpoint of the target instrument's hard end and the end of the endoscope's lens end; The output length is then sent to a display device for display.

25. A computer program product, comprising a computer program, characterized in that, The steps implemented when the computer program is executed by the processor include: When it is determined that the target instrument has been delivered to the target position through the endoscope, the real-time pose of the target instrument is determined by at least one first sensor installed at the hard end of the target instrument. If the real-time pose deviates from the preset pose, the target instrument is controlled to adjust to the preset pose; the preset pose is determined by the position of the object to be detected and the position of the instrument port of the endoscope where the target instrument is located. The method of controlling the target device to adjust to the preset pose includes: performing at least one iterative calculation on the real-time pose based on the preset pose until the error between the pose of the current iteration and the preset pose is within the preset error range. The calculation of the real-time pose based on the preset pose is performed at least once, including: taking the real-time pose as the previous pose to be corrected in the first iteration and performing iteration; for each iteration, when the previous pose to be corrected deviates from the preset pose, adjusting the drive wire to obtain the current length of the drive wire in the current iteration, and determining the pose to be corrected in the current iteration through the current length; wherein, the length of the drive wire affects the real-time pose of the target instrument, and the change in the length of the drive wire is caused by the change in the drive mechanism of the target instrument.

26. The computer program product according to claim 25, characterized in that, The method of delivering the target instrument to the target location via an endoscope includes: By using at least one first sensor disposed at the hard end of the target instrument and at least one second sensor disposed at the lens end of the endoscope, a first distance between the endpoint of the hard end of the target instrument and the lens end of the endoscope is determined, and the target instrument is determined to be delivered to the target position by the endoscope based on the first distance.

27. The computer program product according to claim 26, characterized in that, At least one first sensor is provided at the end or head of the hard end of the target instrument; The step of determining a first distance between the endpoint of the target instrument's rigid end and the end of the endoscope's lens end using at least one first sensor disposed on the rigid end of the target instrument and at least one second sensor disposed on the lens end of the endoscope, and determining whether the target instrument has been delivered to the target position via the endoscope based on the first distance, includes: Based on the first sensing data from the first sensor, the position information of the endpoint of the hard end of the target instrument is determined; Based on the second sensing data of each second sensor, the planar position information of the target plane where the lens end of the endoscope is located is determined; Based on the position information of the endpoint of the target instrument's hard end and the planar position information, a first distance between the endpoint of the target instrument's hard end and the lens end of the endoscope is determined; If the first distance is within the preset distance range, then it is determined that the target instrument is delivered to the target location via the endoscope.

28. The computer program product according to claim 26, characterized in that, The first sensor is a shape sensor, and the first sensor is at least disposed at the end or tail end of the hard end of the target instrument; the second sensor is a shape sensor. The step of determining a first distance between the endpoint of the target instrument's rigid end and the end of the endoscope's lens end using at least one first sensor disposed on the rigid end of the target instrument and at least one second sensor disposed on the lens end of the endoscope, and determining whether the target instrument has been delivered to the target position via the endoscope based on the first distance, includes: Based on multiple first sensing data from the first sensor, the position information of the endpoint of the target instrument's hard end is determined; Based on multiple second sensing data from the second sensor, the position information of at least two line segments at the lens end of the endoscope is determined, wherein the line segments at the lens end of the endoscope are determined by any two points on the second sensor. Based on the position information of the endpoints of the target instrument's hard end and the position information of two line segments of the lens end of at least the endoscope, a first distance is determined between the endpoints of at least two of the target instrument's hard end and the lens end of the endoscope. If each of the first distances is within the preset distance range, then it is determined that the target instrument is delivered to the target position via the endoscope.

29. The computer program product according to any one of claims 25 to 28, characterized in that, The step of determining the real-time pose of the target instrument by means of at least one first sensor disposed at the hard end of the target instrument includes: Based on multiple first sensing data from the first sensor, the head position information of the head end of the target instrument and the tail position information of the tail end of the target instrument are determined. Based on the aforementioned end position information, the degree of bending of the target instrument is determined; A reference coordinate system is constructed based on the tail end position information. The reference coordinate system is determined with the tail end of the target instrument as the center. Based on the degree of curvature and the reference coordinate system, the real-time pose of the target instrument is determined.

30. The computer program product according to claim 25, characterized in that, If the real-time pose deviates from the preset pose, the step of controlling the target device to adjust to the preset pose includes: Based on the previous pose to be corrected and the preset pose, the previous positions of each rotating mechanism in the target device are updated to determine the current position of the mechanism; wherein, the previous pose to be corrected corresponding to the first iteration is the real-time pose. Based on the current mechanism position, the current length of each drive wire in the target device is determined; wherein, the previous mechanism position corresponding to the first iteration is determined by the real-time pose. Based on each current length, the previous pose to be corrected is updated to obtain the current pose to be corrected. If it is determined that the error between the current pose to be corrected and the preset pose is not within the preset error range, the current pose to be corrected is used as the previous pose to be corrected in the next iteration, and the current pose to be corrected is used as the previous mechanism position in the next iteration. Based on the previous pose to be corrected and the preset pose, the previous mechanism position of each rotating mechanism in the target device is updated. The step of determining the current mechanism position continues to be executed until the error is within the preset error range, and then stops, determining that the target device has been adjusted to the preset pose.

31. The computer program product according to claim 30, characterized in that, The step of updating the previous mechanism position of each rotating mechanism in the target device based on the previous pose to be corrected and the preset pose, and determining the current mechanism position, includes: Based on the previous pose to be corrected and the preset pose, the current joint velocity of the target device is obtained by calculating using the Jacobian matrix. The current mechanism position is determined based on the current joint velocity and the previous mechanism position.

32. The computer program product according to claim 27 or 28, characterized in that, The steps implemented when the computer program is executed by the processor also include at least one of the following: The first method involves outputting the first distance between the end point of the target instrument's rigid end and the lens end of the endoscope to a display device for display. The second method: Determine the delivery length of the target instrument's hard end from the end of the endoscope's lens end based on the first distance between the endpoint of the target instrument's hard end and the end of the endoscope's lens end; The output length is then sent to a display device for display.

Images