Method for estimating position of surgical instrument in visual field of endoscope

Abstract

The invention provides a method for estimating the position of a surgical instrument in the visual field of an endoscope. The method comprises the following steps that 1, a transformation matrix between a surgical instrument coordinate system and an endoscope coordinate system is calculated; 2, selecting feature points on the surgical instrument, and constructing a position matrix of the feature points on the surgical instrument in an endoscope coordinate system by using a transformation matrix between the surgical instrument coordinate system and the endoscope coordinate system; 3, calculating the positions of the feature points on the surgical instrument under the visual field of the endoscope; and 4, judging whether the surgical instrument is in the visual field of the endoscope or not. According to the method for estimating the position of the surgical instrument in the visual field of the endoscope, high-precision control over positioning of the tail end of the surgical instrument can be kept, so that it is guaranteed that an operation with the surgical instrument has enough safe and comfortable experience.

Description

Technical Field

[0001] The present invention relates to the technical field of medical instruments, and in particular to a method for estimating the position of a surgical instrument within the field of view of an endoscope. Background Art

[0002] At present, surgical robots have inherent characteristics of multiple joints and modular features. Figure 1 As shown, the surgical robot generally includes: a base 1, a central column 2, a suspension rod 3, a scope holding arm 4, a tool holding arm 5 and other structures, wherein the instrument carried on the scope holding arm 4 is an endoscope, and the instrument carried on the tool holding arm 5 is a surgical instrument.

[0003] As the requirements for surgical robots become higher and higher, such as the expectation that surgical robots can meet the requirements of high-precision and high-safety surgical specifications, it is first necessary to improve some auxiliary surgical functions of surgical robots, such as calibration of key positions of surgical robots, and accurate, rapid and safe judgment of whether the end of the surgical instrument moves within the endoscope field of view through observation of the endoscope field of view. Because in the actual operation, due to the limitations of the endoscope field of view, when the operator controls the main hand of the surgical robot and then moves the surgical instrument, there is a risk that the surgical instrument will exceed the endoscope field of view. If the surgical instrument cannot be seen, it will have a certain impact on the safety and efficiency of the operation. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present application proposes a method for estimating the position of a surgical instrument within the field of view of an endoscope.

[0005] In order to achieve the above object, the present application proposes a method for estimating the position of a surgical instrument within the field of view of an endoscope, comprising the following steps:

[0006] Step 1, calculating the transformation matrix between the surgical instrument coordinate system and the endoscope coordinate system;

[0007] Step 2: Select the feature points on the surgical instrument, and then use the transformation matrix between the surgical instrument coordinate system and the endoscope coordinate system to construct the position matrix of the feature points on the surgical instrument in the endoscope coordinate system;

[0008] Step 3, calculating the position of the feature points on the surgical instrument under the endoscope field of view;

[0009] Step 4: Determine whether the surgical instrument is within the field of view of the endoscope.

[0010] In some embodiments, the specific process of step 1 is as follows:

[0011] Step 101, calculating the transformation matrix of the endoscope coordinate system relative to the surgical robot base coordinate system according to the DH parameters of each joint of the endoscope;

[0012] Step 102: Calculate the homogeneous transformation matrix of the surgical instrument coordinate system relative to the surgical robot base coordinate system according to the DH parameters of the surgical instrument;

[0013] Step 103: Calculate the homogeneous transformation matrix of the surgical instrument coordinate system relative to the endoscope coordinate system through the principle of homogeneous coordinate transformation.

[0014] In some embodiments, in step 2, the position matrix of the feature points on the surgical instrument in the endoscope coordinate system is represented by T t1 =[p xt1 p yt1 p zt1 1] T , where p xt1 , p yt1 , p zt1 is the feature point P on the surgical instrument t1 Coordinates in the endoscope coordinate system.

[0015] In some embodiments, the feature point P t1 The end point of the surgical instrument.

[0016] In some embodiments, in step 3, the endoscope parameters are defined as follows: focal length f len 、Pixel size len x and len y , center pixel position (lcen x , lcen y ), where len x and len y Respectively represent the pixel size of the endoscope field of view in the x and y directions; the endoscope mapping matrix J is calculated len ,

[0017]

[0018] Among them, the endoscope mapping matrix J len Represents: the transformation matrix that transforms the Cartesian spatial position of a point relative to the endoscope to the spatial coordinates under the endoscope's field of view;

[0019] Combined formula (1) and T t1 =[p xt1 p yt1 p zt1 1] T , calculate the position matrix of the feature points on the surgical instrument in the endoscope field of view: Q len =J len ·T t1 =[q lx1 ,q ly1 ,qlz1 ] T , map the position matrix to the imaging plane, and obtain the pixel position coordinates of the feature points on the surgical instrument on the imaging plane:

[0020] In some embodiments, in step 4, the following steps are also included:

[0021] Step 401: determine whether the position of the feature point on the surgical instrument under the endoscope field of view meets the depth of field range of the endoscope. The determination standard is: whether the position of the feature point on the surgical instrument under the endoscope field of view satisfies formula (2). If so, determine that the position of the feature point on the surgical instrument under the endoscope field of view meets the depth of field range of the endoscope, and then jump to step 402; if not, determine that the position of the feature point on the surgical instrument under the endoscope field of view does not meet the depth of field range of the endoscope, and perform warning processing;

[0022] depl min <ql z1 <depl max (2)

[0023] Among them, [depl min depl max ] is the depth of field of the endoscope;

[0024] Step 402: determine whether the pixel position coordinates of the feature point on the surgical instrument on the imaging plane satisfy equation (3); if so, determine that the feature point on the surgical instrument is within the resolution range of the endoscope field of view, indicating that the surgical instrument is within the endoscope field of view; if not, determine that the feature point on the surgical instrument is not within the resolution range of the endoscope field of view, and perform a warning process;

[0025]

[0026] Among them, [res lx res ly ] is the resolution in the endoscope parameters.

[0027] The beneficial effect of the solution of the present application is that the above-mentioned method for estimating the position of the surgical instrument within the field of view of the endoscope has the following advantages:

[0028] 1) High computational efficiency: This application directly uses endoscope parameters to calculate the projection position and determine whether the surgical instrument is beyond the field of view, which is more efficient;

[0029] 2) Strong versatility: It adopts the common imaging principle of cameras and has stronger versatility.

[0030] The method for estimating the position of a surgical instrument within the field of view of an endoscope involved in the present application can maintain high-precision control over the positioning of the distal end of the surgical instrument so as to ensure a sufficiently safe and comfortable experience during surgery with the surgical instrument; the method can be widely used in high-tech fields such as industrial robots, medical robots, and space robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 The schematic diagram of the structure of the surgical robot in the prior art is shown.

[0032] Figure 2 A flow chart of a method for estimating the position of a surgical instrument within the field of view of an endoscope in an embodiment is shown.

[0033] Figure 3 A schematic diagram of the kinematic coordinate system of the mirror holding arm instrument is shown.

[0034] Figure 4 A schematic diagram of the kinematic coordinate system of the robotic arm instrument is shown.

[0035] Figure 5 A schematic diagram of the closed-loop transformation sequence between the endoscope coordinate system and the surgical instrument coordinate system is shown.

[0036] Figure 6 A schematic diagram of the working space of the end movement of surgical instruments under the endoscopic field of view is shown using the Monte Carlo method to simulate the end movement of the surgical instrument.

[0037] Figure 7 A schematic diagram of the position estimation of the end of the robotic arm under angle error compensation PID control is shown.

[0038] Figure 8 A schematic diagram of the end position estimation of the robotic arm based on the Monte Carlo method is shown. DETAILED DESCRIPTION

[0039] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0040] like Figure 2 As shown, the method for estimating the position of a surgical instrument in the field of view of an endoscope involved in the present application comprises the following steps:

[0041] Step 1: Calculate the transformation matrix between the surgical instrument coordinate system and the endoscope coordinate system.

[0042] Step 2: Select the feature points on the surgical instrument, and then use the transformation matrix between the surgical instrument coordinate system and the endoscope coordinate system to construct the position matrix of the feature points on the surgical instrument in the endoscope coordinate system.

[0043] Specifically, in step 2, the position matrix of the feature points on the surgical instrument in the endoscope coordinate system is represented by Tt1 =[p xt1 p yt1 p zt1 1] T , where p xt1 , p yt1 , p zt1 is the feature point P on the surgical instrument t1 Coordinates in the endoscope coordinate system. Optimized feature point P t1 The end point of the surgical instrument.

[0044] Step 3: Calculate the position of the feature points on the surgical instrument under the endoscope field of view.

[0045] Specifically, in step 3, the endoscope parameters are defined as follows: focal length f len 、Pixel size len x and len y , center pixel position (lcen x , lcen y ), where len x and len y Respectively represent the pixel size of the endoscope field of view in the x and y directions; then the endoscope mapping matrix J is calculated len , as shown in formula (1),

[0046]

[0047] Among them, the endoscope mapping matrix J len Represents: The transformation matrix that transforms the Cartesian spatial position of a point relative to the endoscope to the spatial coordinates under the endoscope's field of view.

[0048] Combined formula (1) and T t1 =[p xt1 p yt1 p zt1 1] T , calculate the position matrix of the feature points on the surgical instrument in the endoscope field of view: Q len =J len ·T t1 =[q lx1 ,q ly1 ,q lz1 ] T , map the position matrix to the imaging plane, and obtain the pixel position coordinates of the feature points on the surgical instrument on the imaging plane:

[0049] Step 4: Determine whether the surgical instrument is within the field of view of the endoscope.

[0050] Specifically, in step 4, the following steps are also included:

[0051] Step 401: determine whether the position of the feature point on the surgical instrument under the endoscope field of view meets the depth of field range of the endoscope. The determination standard is: whether the position of the feature point on the surgical instrument under the endoscope field of view satisfies formula (2). If so, determine that the position of the feature point on the surgical instrument under the endoscope field of view meets the depth of field range of the endoscope, and then jump to step 402; if not, determine that the position of the feature point on the surgical instrument under the endoscope field of view does not meet the depth of field range of the endoscope, and perform warning processing;

[0052] depl min <ql z1 <depl max (2)

[0053] Among them, [depl min depl max ] is the depth of field of the endoscope.

[0054] Step 402: determine whether the pixel position coordinates of the feature point on the surgical instrument on the imaging plane satisfy equation (3); if so, determine that the feature point on the surgical instrument is within the resolution range of the endoscope field of view, indicating that the surgical instrument is within the endoscope field of view; if not, determine that the feature point on the surgical instrument is not within the resolution range of the endoscope field of view, and perform a warning process;

[0055]

[0056] Among them, [res lx res ly ] is the resolution in the endoscope parameters.

[0057] Specifically, in step 1, the transformation principle between the surgical instrument coordinate system and the endoscope coordinate system is as follows: the transformation matrix of the surgical instrument coordinate system and the endoscope coordinate system relative to the surgical robot base coordinate system is obtained by calculating the coordinate transformation principle of the surgical robot and the DH parameter, and the transformation matrix between the surgical instrument coordinate system and the endoscope coordinate system is calculated accordingly. Then, given the origin coordinate system and the target coordinate system, the DH parameter transformation matrix is ​​expressed as:

[0058]

[0059] Among them, the parameter (α i-1 , a i-1 , d i ,θ i) is defined in the following journal (“Introduction to Robotics-Mechanics and Control”, John J.Craig, China Machine Press, pages 44-50, February 1, 2018).

[0060] Then, the transformation matrix from the surgical robot base coordinate system to the instrument end coordinate system can be expressed as:

[0061]

[0062] In the above formula, R: 3×3 rotation matrix; P: 3×1 position vector.

[0063] Specifically, in order to obtain the homogeneous transformation matrix from the surgical instrument coordinate system to the endoscope coordinate system, the specific process of step 1 is as follows:

[0064] Step 101: Calculate the transformation matrix of the endoscope coordinate system relative to the surgical robot base coordinate system according to the DH parameters of each joint of the endoscope.

[0065] Step 102: Calculate the homogeneous transformation matrix of the surgical instrument coordinate system relative to the surgical robot base coordinate system according to the DH parameters of the surgical instrument.

[0066] Step 103: Calculate the homogeneous transformation matrix of the surgical instrument coordinate system relative to the endoscope coordinate system through the principle of homogeneous coordinate transformation.

[0067] The step 101 is a kinematic analysis of the endoscope holding arm instrument, wherein the endoscope holding arm instrument refers to an endoscope, and the DH method proposed by John J. Craig is used to establish the kinematic coordinate system of the endoscope holding arm instrument. Figure 3 , this method uses four parameters (α i-1 , a i-1 , d i ,θ i ) is used to describe the geometric relationship between the i-1th rod and the i-th rod, where {x0, y0, z0} represents the endoscope base coordinate system, {x 14 ,y 14 , z 14} represents the endoscope end coordinate system, and describes the endoscope end coordinate point P in the endoscope coordinate system len The position and posture R len For details, please refer to Table 1 DH parameter table of the mirror holding arm instrument.

[0068] Table 1 DH parameter table of mirror holding arm instrument

[0069] DH parameters of mirror arm

[0070]

[0071] According to the DH parameter table of the endoscope arm shown in Table 1 and formulas (4)-(5), the homogeneous transformation matrix of the endoscope coordinate system relative to the base coordinate system of the surgical robot is:

[0072]

[0073] The attitude matrix and position vector of the endoscope tip center are:

[0074]

[0075] Among them, s(·)=sin(·), c(·)=cos(·), s 2l，41 = sin(θ 2l +θ 4l ), c 2l，4l =cos(θ 2l +θ 4l ), where the endoscope DH degree of freedom parameter is defined as: θ len =[θ 2l ,θ 4l ,θ 8l ,θ 9l , d 11l ,θ 12l ] T , select d len =10 3 [2.0950, 1.1300, 0.6368, 0.2291, 0.5740, 0.0690, 0.0550] T , here, the initial parameters of endoscope DH are defined: d len =[d1, d3, d5, d7, d9, a5, a 12 ] T .

[0076] The step 102 is a kinematic analysis of the robotic arm instrument, wherein the robotic arm instrument refers to a surgical instrument, see Figure 4 The DH method proposed by John J.Craig is used to establish the kinematic coordinate system of the robotic arm. This method uses four parameters (α i-1 , a i-1 , d i ,θ i ) is used to describe the geometric relationship between the i-1th rod and the i-th rod, {x0, y0, z0} represents the surgical instrument base coordinate system, {x 14 ,y 14 , z 14} represents the surgical instrument end coordinate system. The surgical instrument end coordinate point P is described in the surgical instrument coordinate systemmec The position and orientation R mec , refer to Table 2, DH parameter table of robotic arm equipment.

[0077] Table 2 DH parameter table of robotic arm equipment

[0078] DH parameters of robot arm

[0079]

[0080] According to the DH parameter table of the robotic arm shown in Table 2 and formulas (4)-(5), the homogeneous transformation matrix of the surgical instrument coordinate system relative to the surgical robot base coordinate system is:

[0081]

[0082] The posture matrix and position vector of the surgical instrument end center are:

[0083]

[0084] Among them, s(·)=sin(·), c(·)=cos(·), s 2，4 = sin(θ2+θ4), c 2，4 =cos(θ2+θ4),s 13，14 = sin(θ 13 +θ 14 ), c 13，14 =cos(θ 13 +θ 14 ), here, define the freedom parameter of the robot arm DH: θ mec =[θ 2m ,θ 4m ,θ 8m ,θ 9m , d 11m ,θ 12m ,θ 13m ,θ 14m ] T , select d mec =10 3 [2.0950, 1.1300, 0.6368, 0.2291, 0.5740, 0.1146, 0.0690, 0.0057] T Here, the initial parameters of the robot arm DH are defined: mec =[d1, d3, d5, d7, d9, a4, a5, a 13 ] T .

[0085] In step 103, refer to Figure 5, the two DH transformation expressions of the endoscope coordinate system and the surgical instrument coordinate system form a closed-loop transformation sequence. Among them, {D} represents the endoscope / surgical instrument base coordinates; {A}, {C} represent the fifth coordinate system {5} of the endoscope and the surgical instrument, respectively, because the DH parameter configurations of the endoscope and the surgical instrument are different starting from the {5}th coordinate system; {U}, {B} represent the end tool coordinate systems of the endoscope and the surgical instrument, respectively.

[0086] Therefore, different coordinate systems can be expressed in two different ways as matrix transformation multiplication:

[0087]

[0088] Combining equations (12-1) and (12-2), we get the transformation equation:

[0089]

[0090] All transformations except All the outside are known, and we can apply the transformation equation (13) to solve:

[0091]

[0092] Furthermore, the homogeneous transformation matrix of the surgical instrument coordinate system relative to the endoscope coordinate system is solved:

[0093]

[0094] See also Figure 3-5 , represents the homogeneous transformation matrix of the mirror holding arm from coordinate system {4} to coordinate system {5{; represents the homogeneous transformation matrix of the mirror holding arm from coordinate system {5} to coordinate system {14}; represents the homogeneous transformation matrix of the robotic arm from coordinate system {4} to coordinate system {5}; Represents the homogeneous transformation matrix of the robotic arm from coordinate system {5} to coordinate system {14}.

[0095] definition: because The associated parameters are obtained:

[0096]

[0097] s x，y =sin(x+y),c x，y =cos(x+y)

[0098] in,

[0099]

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106] In step 3, the formula (1) and T t1 =[p xt1 p yt1 p zt1 1] T , calculate the position matrix of the feature points on the surgical instrument in the endoscope field of view:

[0107] Q len =J len ·T t1 =[q lx1 ,q ly1 ,q lz1 ] T (16)

[0108] The position matrix is ​​mapped to the imaging plane to obtain the pixel position coordinates of the feature points on the surgical instrument on the imaging plane:

[0109]

[0110] The above steps 1 to 4 are the method for estimating the position of the surgical instrument within the endoscope field of view. Through numerical calculation, it is easy to determine whether the surgical instrument is included in the endoscope field of view. Here, a numerical example is given to illustrate.

[0111] Assume θ len =[1.2189, 1.6871, -0.2436, -2.7911, 70.2000.0.9223] T ,θ mec =[1.2189, 1.6871, -0.2436, -2.7911, 70.2000, 0.9223, 1.0359, -1.2783] T , where the focal length f len =2mm, pixel size len x =1.2μm and len y =1.2μm, center pixel position (lcenx , lcen y ) = [2048, 1080];

[0112] According to formulas (15)-(17), we can calculate:

[0113]

[0114] After calculation by formula (2)-(3), the numerical results show that the feature point on the surgical instrument is within the depth of field of the endoscope (depl min , depl max ) = [3, 200] data points, and are within the endoscope field of view resolution range (res lx ,res ly )=[4096,2160] data points.

[0115] The Monte Carlo method is applied below to solve the working space of the end position of the surgical instrument under the endoscopic field of view to verify the feasibility and significance of the method involved in this application.

[0116] Within the range of the joint motion space, set the range of joint variables and randomly select joint angles:

[0117]

[0118] Among them, N J represents the number of degrees of freedom of the surgical robot, q j represents the DH degree of freedom parameter.

[0119] Here is the definition: Where j = 1, ..., N J , k = 1, ..., N t , N J =l0,N t The number of samples, q = [q 8_end ,q 9_end , d 11_end ,q 12_end ,q 8_left ,q 9_left , d 11_left ,q 12_left ,q 13_left ,q 14_left ] T represents the DH degree of freedom parameter, ζ jk : The kth discrete random variable of the jjth joint, and is arbitrarily selected in the interval [0, 1].

[0120] See also Figure 3-4 , select the initial parameters of endoscope / surgical instrument DH,

[0121] dlen =10 3 [2.0950, 1.1300, 0.6368, 0.2291, 0.5740, 0.0690, 0.0550] T ,

[0122] d mec =10 3 [2.0950, 1.1300, 0.6368, 0.2291, 0.5740, 0.1146, 0.0690, 0.0057] T ,

[0123] d mec =[d1, d3, d5, d7, d9, a4, a5, a 13 ] T , d len =[d1, d3, d5, d7, d9, a5, a 12 ] T .

[0124] Substituting the above sample points into the kinematic equation, we get:

[0125]

[0126] See also Figure 6 , when the number of samples N t When the motion geometry space of surgical instrument position estimation is large enough It is the actual motion workspace U of the surgical instrument position Act .

[0127] like Figures 7-8 As shown, simulation verification can also be carried out through the physical simulation platform of the surgical robot. By comparing the numerical simulation results, it is shown that the surgical instrument end position estimation effect obtained by the physical simulation platform of the surgical robot under the angle error compensation PID control is better, thereby verifying that the method involved in the present application is effective and feasible.

[0128] In the context of the demand for precision medical surgical manipulation, the method for estimating the position of surgical instruments within the endoscopic field of view involved in this application not only establishes the analytical form of the kinematic model of the robotic arm instrument under the endoscopic field of view, and provides judgment criteria for the position estimation of surgical instruments under the endoscopic field of view, but also solves the working space of the end position of the surgical instrument under the endoscopic field of view through the Monte Carlo method, and verifies the feasibility of the method through the surgical robot physical simulation platform.

[0129] The method for estimating the position of a surgical instrument within the field of view of an endoscope involved in the present application can maintain high-precision control over the positioning of the distal end of the surgical instrument so as to ensure a sufficiently safe and comfortable experience during surgery with the surgical instrument; the method can be widely used in high-tech fields such as industrial robots, medical robots, and space robots.

[0130] The above is only a preferred specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can make equivalent substitutions or changes according to the technical solution and concept of the present application within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A method for estimating the position of a surgical instrument within an endoscope field of view, characterized in that: The following steps are involved: Step 1, calculating the transformation matrix between the surgical instrument coordinate system and the endoscope coordinate system; Step 2: Select the feature points on the surgical instrument, and then use the transformation matrix between the surgical instrument coordinate system and the endoscope coordinate system to construct the position matrix of the feature points on the surgical instrument in the endoscope coordinate system; Step 3, calculating the position of the feature points on the surgical instrument under the endoscope field of view; Step 4: Determine whether the surgical instrument is within the field of view of the endoscope.

2. The method for estimating the position of a surgical instrument within the field of view of an endoscope according to claim 1, characterized in that: The specific process of step 1 is as follows: Step 101, calculating the transformation matrix of the endoscope coordinate system relative to the surgical robot base coordinate system according to the DH parameters of each joint of the endoscope; Step 102: Calculate the homogeneous transformation matrix of the surgical instrument coordinate system relative to the surgical robot base coordinate system according to the DH parameters of the surgical instrument; Step 103: Calculate the homogeneous transformation matrix of the surgical instrument coordinate system relative to the endoscope coordinate system through the principle of homogeneous coordinate transformation.

3. The method for estimating the position of a surgical instrument within the field of view of an endoscope according to claim 1, characterized in that: In step 2, the position matrix of the feature points on the surgical instrument in the endoscope coordinate system is represented by T t1 =[p xt1 p yt1 p zt1 1] T , where p xt1 , p yt1 , p zt1 is the feature point P on the surgical instrument t1 Coordinates in the endoscope coordinate system.

4. The method for estimating the position of a surgical instrument within the field of view of an endoscope according to claim 3, characterized in that: The feature point P t1 The end point of the surgical instrument.

5. The method for estimating the position of a surgical instrument within the field of view of an endoscope according to claim 3, characterized in that: In step 3, the endoscope parameters are defined as follows: focal length f len 、Pixel size len x and len y , center pixel position (lcen x , lcen y ), where len x and len y Respectively represent the pixel size of the endoscope field of view in the x and y directions; the endoscope mapping matrix J is calculated len , Among them, the endoscope mapping matrix J len Represents: the transformation matrix that transforms the Cartesian spatial position of a point relative to the endoscope to the spatial coordinates under the endoscope's field of view; Combined formula (1) and T t1 =[p xt1 p yt1 p zt1 1] T , calculate the position matrix of the feature points on the surgical instrument in the endoscope field of view: Q len =J len ·T t1 =[q lx1 ,q ly1 ,q lz1 ] T , map the position matrix to the imaging plane, and obtain the pixel position coordinates of the feature points on the surgical instrument on the imaging plane:

6. The method for estimating the position of a surgical instrument within the field of view of an endoscope according to claim 5, characterized in that: In step 4, the following steps are also included: Step 401: determine whether the position of the feature point on the surgical instrument under the endoscope field of view meets the depth of field range of the endoscope. The determination standard is: whether the position of the feature point on the surgical instrument under the endoscope field of view satisfies formula (2). If so, determine that the position of the feature point on the surgical instrument under the endoscope field of view meets the depth of field range of the endoscope, and then jump to step 402; if not, determine that the position of the feature point on the surgical instrument under the endoscope field of view does not meet the depth of field range of the endoscope, and perform warning processing; depl min <q lz1 <depl max (2) Among them, [depl min depl max ] is the depth of field of the endoscope; Step 402: determine whether the pixel position coordinates of the feature point on the surgical instrument on the imaging plane satisfy equation (3); if so, determine that the feature point on the surgical instrument is within the resolution range of the endoscope field of view, indicating that the surgical instrument is within the endoscope field of view; if not, determine that the feature point on the surgical instrument is not within the resolution range of the endoscope field of view, and perform a warning process; Among them, [res lx res ly ] is the resolution in the endoscope parameters.

Images