Endoscope displacement control system and method

Abstract

The present application provides an endoscope displacement control system and method, the system includes: an endoscope, an endoscope driver, a control device and a water jet driver; the control device includes: a first displacement control module, used to control the endoscope to retreat, save a first endoscope image at a first position Z1 where the endoscope can see the sensitive tissue, record the position Z1, and mark or automatically identify the first coordinate information of pixels on both sides of the sensitive tissue on the image; a second displacement control module, used to continue to adjust the endoscope to retreat, save a second endoscope image at a second position Z2 where the endoscope can see the sensitive tissue, record the position Z2, and mark or automatically identify the second coordinate information of pixels on both sides of the sensitive tissue on the image; a third displacement control module, used to determine that the sensitive tissue is located at the Z0 position of the endoscope coordinate system based on the first coordinate information and the second coordinate information, and control the endoscope to move forward to the Z0 position again.

Description

Technical Field

[0001] The present application relates to the field of medical devices, and in particular to an endoscope displacement control system and method. Background Art

[0002] In recent years, imaging technology has been increasingly used in the field of medical devices. Using real-time imaging technology to accurately guide surgical execution is an important development direction of imaging technology, especially in the field of automated surgery.

[0003] Taking urology as an example, the use of transrectal dual-plane ultrasound probes to image the surgical area has become a common means of surgical planning due to its high ultrasound frequency and good real-time performance. For example, the planning of water jet minimally invasive surgery is based on the doctor's spatial position recognition, manual annotation, or manual input of parameters by reading the transverse and sagittal ultrasound images respectively, and converting the doctor's input information into the cutting path planning during the operation to form a surgical planning plan.

[0004] However, ultrasound images generally have low resolution, large image noise, and unclear imaging of tissues. These problems are very likely to cause deviations in the identification of sensitive parts. In urological surgery, it is very necessary to accurately identify sensitive parts (such as the colliculus) and implement more accurate planning around the sensitive parts. The processing methods in the prior art include: raising the protection threshold, that is, once it is identified that it may reach the colliculus or it is believed based on experience that it may be close to the colliculus, the planning is stopped. This conservative measure can better ensure safety, but because the cutoff position of the planned area is set too conservatively, the actual planned area is greatly reduced, and the volume of the hyperplastic tissue to be removed in the planned area is also greatly reduced, so that the expected surgical effect cannot be achieved; however, if conservative measures are not taken, due to the limitations of ultrasound images in identifying the surgical area, the safety measures for sensitive parts are insufficient, and unnecessary damage to patients may be caused. Summary of the invention

[0005] In order to solve the above problems, the present application provides an endoscope displacement control system and method for a water jet surgical system, so as to help accurately determine the planning area based on the identified sensitive parts and achieve precise planning.

[0006] The present application provides an endoscope displacement control system, characterized in that it includes: an endoscope, an endoscope driver, a control device and a water jet driver; the control device is connected to the endoscope through the endoscope driver, and is connected to the water jet through the water jet driver; the endoscope driver is also used to read the position information of the endoscope, and the initial position of the endoscope is located at the zero position of the endoscope coordinate system;

[0007] The control device comprises:

[0008] a first displacement control module, used to control the endoscope to move to a first position Z1 where sensitive tissue can be seen in the endoscope field of view, save a first endoscopic image of the position, obtain first pixel coordinates A1 (x_A1, y_A1) and second pixel coordinates B1 (x_B1, y_B1) on the first endoscopic image, respectively representing left and right sides of the sensitive tissue, and record position information of the first position Z1;

[0009] a second displacement control module, used for controlling the endoscope to move to a second position Z2 where the sensitive tissue can be seen in the endoscope field of view, saving a second endoscope image at the position, obtaining a third pixel coordinate A2 (x_A2, y_A2) and a fourth pixel coordinate B2 (x_B2, y_B2) on the second endoscope image respectively representing the left and right sides of the sensitive tissue, and recording the position information of the second position Z2;

[0010] The third displacement control module is used to determine that the sensitive tissue is located at the Z0 position of the endoscope coordinate system according to the first pixel coordinate A1 (x_A1, y_A1) and the second pixel coordinate B1 (x_B1, y_B1), and the third pixel coordinate A2 (x_A2, y_A2) and the fourth pixel coordinate B2 (x_B2, y_B2), and control the endoscope to move to a third position determined based on the Z0 position.

[0011] Preferably, the third displacement control module also includes: a pixel length calculation submodule, used to calculate the first pixel length L1 according to the first pixel coordinate A1 (x_A1, y_A1) and the second pixel coordinate B1 (x_B1, y_B1), and calculate the second pixel length L2 according to the third pixel coordinate A2 (x_A2, y_A2) and the fourth pixel coordinate B2 (x_B2, y_B2); wherein the first pixel length L1 represents the pixel length of the sensitive tissue in the first endoscopic image obtained by the endoscope at the first position Z1; the second pixel length L2 represents the pixel length of the sensitive tissue in the second endoscopic image obtained by the endoscope at the second position Z2; the first pixel length L1 and the second pixel length L2 can be obtained by the following formulas:

[0012] .

[0013] Preferably, the third displacement control module also includes: a distance calculation submodule, used to determine a first distance s1 between the endoscope tip and the sensitive tissue when the endoscope is in the first position Z1, and a second distance s2 between the endoscope tip and the sensitive tissue when the endoscope is in the second position Z2, based on the first pixel length L1 and the second pixel length L2.

[0014] Preferably, the distance calculation submodule determines the first distance s1 according to the following formula: s1=s0 L2 / (L1-L2); and determine the second distance s2 according to the following formula: s2=s0+s1, where s0 represents the distance between the first position Z1 and the second position Z2.

[0015] Preferably, the third displacement control module further includes: an endoscope coordinate system position calculation submodule, configured to determine, based on the first distance s1 or the second distance s2, that the sensitive tissue is located at position Z0 of the endoscope coordinate system.

[0016] Preferably, the endoscope displacement control system further comprises: a first surgery planning sector reserved angle determination module, configured to determine the surgery planning sector reserved angle of the position according to the first endoscopic image acquired at the first position Z1.

[0017] Preferably, the first displacement control module is also used to determine the first upper edge pixel coordinate E1 (x_E1, y_E1) in the first endoscopic image at the first position Z1, and the first surgical planning fan-shaped reserved angle determination module calculates the pixel coordinate O1 (x_O1, y_O1) of the center of the field of view in the first endoscopic image at the first position Z1 and the pixel coordinate D1 (x_D1, y_D1) of the water jet nozzle at the first position Z1 according to the first upper edge pixel coordinate E1 (x_E1, y_E1).

[0018] Preferably, the first surgical planning fan-shaped reserved angle determination module also calculates the fan-shaped reserved angles ∠A1D1O1 and ∠B1D1O1 on both sides in the cross-sectional view at the first position Z1 based on the pixel coordinates O1 (x_O1, y_O1) of the center of the field of view and the pixel coordinates D1 (x_D1, y_D1) of the water jet nozzle at the first position Z1.

[0019] Preferably, the endoscope displacement control system further comprises: a second surgery planning sector reserved angle determination module, configured to determine the surgery planning sector reserved angle of the position according to the second endoscope image acquired at the second position Z2.

[0020] The present application also provides an endoscope displacement control method, characterized by an endoscope, an endoscope driver, a control device and a water jet driver; the control device is connected to the endoscope through the endoscope driver, and is used to connect to the water jet through the water jet driver; the endoscope driver is used to read the position information of the endoscope, and the initial position of the endoscope is located at the zero position of the endoscope coordinate system; the method comprises the following steps:

[0021] After the endoscope is installed in place, the endoscope is controlled to move back until the first position Z1 of the sensitive tissue can be seen in the endoscope field of view, and the first endoscopic image of the position is saved, and the first pixel coordinates A1 (x_A1, y_A1) and the second pixel coordinates B1 (x_B1, y_B1) representing the left and right sides of the sensitive tissue on the first endoscopic image are obtained, and the position information of the first position Z1 is recorded;

[0022] Control the endoscope to continue to retreat until a second position Z2 of the sensitive tissue is visible in the endoscope field of view, save a second endoscopic image of the position, obtain third pixel coordinates A2 (x_A2, y_A2) and fourth pixel coordinates B2 (x_B2, y_B2) on the second endoscopic image, respectively representing the left and right sides of the sensitive tissue, and record position information of the second position Z2; and

[0023] Control the endoscope to move to a third position determined based on the Z0 position, where the Z0 position is the position of the sensitive tissue in the endoscope coordinate system, determined according to the first pixel coordinate A1 (x_A1, y_A1) and the second pixel coordinate B1 (x_B1, y_B1), and the third pixel coordinate A2 (x_A2, y_A2) and the fourth pixel coordinate B2 (x_B2, y_B2).

[0024] Compared with the prior art, the endoscope displacement control system of the present application can more accurately calculate the Z0 position of the sensitive tissue in the endoscope coordinate system through the endoscopic image, and can analyze the cross-sectional view of the location of the sensitive tissue to determine the recommended resection angle at that location, so as to obtain better surgical resection effect while avoiding damage to the tissue area to be protected as much as possible in actual clinical surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the present application, the drawings of the specification of the present application will be described and illustrated below. Obviously, the drawings in the following description only illustrate certain aspects of some exemplary embodiments of the present application, and it is possible for a person of ordinary skill in the art to obtain other drawings based on these drawings without creative work.

[0026] Figure 1is a schematic diagram of the system structure consisting of a water jet and an endoscope according to an embodiment of the present application;

[0027] Figure 2-1 and Figure 2-2 It is the first schematic diagram of the operation process of the existing endoscope displacement control system;

[0028] Figure 3 It is the second schematic diagram of the operation process of the existing endoscope displacement control system;

[0029] Figure 4 is a schematic structural diagram of an endoscope displacement control system according to an embodiment of the present application;

[0030] Figure 5 is a schematic diagram of the endoscopic imaging principle according to the present application;

[0031] Figure 6-1 and Figure 6-2 is a cross-sectional view of the endoscope position of the present application;

[0032] Figure 7 is a schematic structural diagram of an endoscope displacement control system according to an embodiment of the present application;

[0033] Figure 8 It is a schematic diagram of an endoscope image area in an endoscope displacement control system according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] Various exemplary embodiments of the present application are described in detail below with reference to the accompanying drawings. The description of the exemplary embodiments is merely illustrative and is by no means intended to limit the present application and its application or use. The present application can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to make the present application thorough and complete and to fully express the scope of the present application to those skilled in the art. It should be noted that unless otherwise stated, the relative arrangement of the components and steps, numerical expressions, and numerical values, etc., described in these embodiments should be interpreted as being merely exemplary, rather than as limitations.

[0035] The words “include” or “comprising” and the like used in this application mean that the elements preceding the words include the elements listed after the words, and do not exclude the possibility of also including other elements.

[0036] All terms (including technical terms or scientific terms) used in this application have the same meaning as those understood by ordinary technicians in the field to which this application belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be understood to have the same meaning as their meanings in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined in this article.

[0037] Components not described in detail in this section, parameters such as specific models of components, relationships between components, and control circuits may be considered as technologies, methods, and equipment known to ordinary technicians in the relevant fields, but in appropriate circumstances, such technologies, methods, and equipment should be considered as part of the specification.

[0038] In order to better understand the present application, its specific implementation method will be described in detail below, including hardware design, software design and functional design.

[0039] like Figure 1 This is a schematic diagram of the system structure consisting of a water jet and an endoscope provided according to an embodiment of the present application.

[0040] As shown in the figure, the water jet surgical system includes: a water jet, a water jet driver, an endoscope, an endoscope driver, and a control device. The control device sends control signals to the water jet driver and the endoscope driver respectively to drive the movement of the water jet and the endoscope respectively. The water jet is used to perform water jet resection surgery, and the endoscope is usually used for intraoperative observation, for example, during the operation, the image of the endoscope is used to provide the doctor with real-time and clear information of a certain part that he wants to observe. Usually, the water jet and the endoscope are sent to the part to be operated or observed through the same or adjacent and parallel channels. The water jet has a water jet main body, and a nozzle part is provided at the front end of the main body. Through the high-pressure water jet channel provided in the main body, the high-pressure water jet is sprayed through the nozzle part to the target to be resected to achieve resection. The endoscope has an endoscope main body, and a lens is provided at the front end of its main body. The lens provided at the front end can capture images of information within the field of view.

[0041] In practical applications, the zero point of the water jet position is marked by the nozzle position, but the nozzle position of the water jet is not necessarily located at the front end. For example, it can be a sideways water jet, and the nozzle position is some distance away from the front end of the water jet body. In practical applications, the center of the field of view of the endoscope is not at the front end vertex of the endoscope. The center point of the objective lens of the endoscope is the true center of the field of view (generally within 1 mm from the front end). Even if the water jet and the endoscope are both located at the front end of the channel, the nozzle position of the water jet and the front end of the endoscope are not on the same cross section, and there is a position difference. It can be converted by a fixed difference (constant). For the convenience of description, the front end of the endoscope is used as the center of its field of view in the scheme of this article. However, it should be understood that the factors such as the position deviation described above can also be obtained by converting the position relationship such as the available or measurable physical distance.

[0042] like Figure 1As shown, in the prostate surgery scenario of urology, the water jet and the endoscope enter the prostate and bladder through the urethra through a specific rigid channel to perform prostate hyperplasia tissue resection. The water jet and the endoscope are controlled by separate drive devices. The positional relationship and coordinate transformation of the water jet and the endoscope can be converted by reading the position by the drive device. The path of water jet surgical resection, such as the sagittal and cross-sectional images generated by transrectal biplane ultrasound, is planned separately and / or jointly on the two-dimensional images. For example, the length and depth of the water jet resection range are planned on the ultrasound sagittal image, and the angle of the water jet resection range is planned on the ultrasound cross-sectional image. This type of surgical planning can be performed immediately before the operation is performed. Compared with the plan based on preoperative images (such as magnetic resonance images), it is more real-time and can minimize the impact of factors such as patient movement and body changes. However, as mentioned above, the planning is performed based on ultrasound images, which are limited by the clarity of the ultrasound images themselves. It is not always possible to clearly and accurately identify sensitive tissue sites, making it impossible to clearly determine the exact location of sensitive tissue sites during the operation, posing challenges to the safety and accuracy of surgical planning.

[0043] The solution of the prior art includes determining the planned end point position by means of endoscopic observation. For example, the starting point of the surgical plan can be determined by ultrasound images or preoperative anatomical images (which can be ultrasound images or MRI images or similar images), for example, the starting point position is selected at the bladder neck position (the starting point of the hyperplastic tissue) determined based on the image. However, when determining the end point of the surgical plan, since the tissue position is not easy to identify, the operator usually controls the endoscope while observing and stopping according to the observed information, and the front end position of the endoscope when it stops is used as the end point.

[0044] As mentioned above, the water jet and the endoscope are first inserted into the patient's body together. At this time, the water jet and the endoscope are both located at the front end of the channel. The channel is adjusted and fixed, and then the endoscope is moved back to the end point of the operation. In this way, in order to avoid damage to the colliculus seminalis, the following plan is currently adopted: the end point of the endoscope is adjusted to above the colliculus seminalis position, and it is set as the end point of the operation. The colliculus seminalis area is no longer planned, and conservative treatment is adopted. Since objects within the field of view of the endoscope are generally located in front of the endoscope, as the position of the endoscope continues to move backward, the colliculus seminalis will appear in the field of view of the endoscope, such as Figure 2-1 As shown in Figure 1, at this time, the seminal colliculus tissue begins to appear in the endoscope field of view; as the endoscope moves backward, Figure 2-2 As shown, more and more seminal colliculus tissue will appear in the endoscopic field of view.

[0045] The solution proposed in this application is to reasonably determine the surgical planning area, mainly the end point of the surgical planning area. That is, it is hoped that by reasonably determining the end point of the surgical planning area, it can achieve the maximum range of resection (other tissues other than sensitive tissues, such as hyperplastic tissue) while taking into account protection and avoiding sensitive tissues (taking the seminal fossa as an example, but it should be understood that it is also applicable to other sensitive tissues or tissues that want to be protected). The ideal end point of the surgical planning area is as follows: Figure 2-1 , Figure 2-2 The most anterior position of the sensitive tissue shown; i.e. Figure 2-2 The endoscope shown, after observing the sensitive tissue in the endoscope field of view, is already behind the sensitive tissue. Therefore, it is necessary to drive the endoscope to move forward a distance d so that the front end of the endoscope is flush with the front end of the sensitive tissue.

[0046] Therefore, if the endoscope is to be adjusted to be just above the verumontanum region, it is necessary to adjust the endoscope forward by a distance d after the verumontanum region appears, such as Figure 3 It should be noted that Figure 2-1 and Figure 2-2 That is, the fossa vermiformis area is in the endoscope field of view. Even if the endoscope is moved forward to a position where the fossa vermiformis is just out of sight, the fossa vermiformis area is still in front of the endoscope. Therefore, the actual required distance d cannot be determined directly by relying on the observation image of the endoscope. In today's clinical applications, the distance d can only be estimated based on the doctor's experience, which makes the operation risky. If the estimated distance d is too long, the operation effect may be unsatisfactory. If the estimated distance d is too short, the fossa vermiformis area may not be protected.

[0047] The present application provides an endoscope displacement control system and method. The control device is connected to the endoscope through the endoscope driver and is connected to the water jet through the water jet driver; the endoscope driver is also used to read the position information of the endoscope, and the initial position of the endoscope is located at the zero position of the endoscope coordinate system.

[0048] The control device comprises:

[0049] The first displacement control module 401 is used to control the endoscope to retreat, save the first endoscope image at the first position Z1 where the endoscope can see the sensitive tissue, and record the position Z1.

[0050] The first endoscopic image acquired and saved at position Z1 is identified or processed to determine the pixel position points on the left and right sides of the sensitive tissue in the image and record them; the identification or processing can be done by the doctor or operator marking in the image through observation, or it can be done automatically through the image contour extraction and processing algorithm; the determined pixel position points on the left and right sides are determined based on the contour of the sensitive tissue, and the leftmost or rightmost position of the tissue image contour can be selected, or other specific position points on the contour. After identification or processing, the first pixel coordinate information A1 (x_A1, y_A1) and the second pixel coordinate information B1 (x_B1, y_B1) on the left and right sides of the sensitive tissue can be obtained;

[0051] The second displacement control module 402 is used to continue adjusting the endoscope to retreat, save the second endoscope image when the endoscope can see the second position Z2 of the sensitive tissue, record the position Z2, and mark or automatically identify the third pixel coordinate information A2 (x_A2, y_A2) and the fourth pixel coordinate information B2 (x_B2, y_B2) on the left and right sides of the sensitive tissue on the image;

[0052] The third displacement control module 403 is used to determine that the sensitive tissue is located at the Z0 position of the endoscope coordinate system based on the first pixel coordinate information A1 (x_A1, y_A1) and the second pixel coordinate information B1 (x_B1, y_B1), as well as the third pixel coordinate information A2 (x_A2, y_A2) and the fourth pixel coordinate information B2 (x_B2, y_B2), and control the endoscope to move forward to the Z0 position.

[0053] like Figure 5 As shown, a schematic diagram of the endoscope image when the sensitive tissue is at different distances from the endoscope (indicated by s1 and s2). L represents the actual length of the sensitive tissue, f represents the focal length of the endoscope imaging; L1 represents the length of the endoscope in the endoscope image when the endoscope is at a distance of s1 from the sensitive tissue; L2 represents the length of the endoscope in the endoscope image when the endoscope is at a distance of s2 from the sensitive tissue.

[0054] According to the imaging principle of the endoscope, what is observed in the fiber endoscope is the inverted image of the object. The imaging principle of the fiber endoscope can be compared to the imaging of the lens. The focal length f of the endoscope remains unchanged, and the overall size of the picture displayed on the monitor remains unchanged. The farther the observed object is from the endoscope, the smaller the projection of the object presented on the monitor screen.

[0055] The above parameters satisfy the following formulas: L / L1=s1 / f, and L / L2=s2 / f.

[0056] The lengths L1 and L2 of the sensitive tissues in the endoscopic image can be obtained by manual annotation or automatically identified by related algorithms such as computer vision or contour extraction. The plane where the sensitive tissue is located and the plane of the endoscopic imaging are two parallel planes, located on both sides of the vertex of the end of the endoscope (in this application, the direction in which the endoscope is inserted forward is the distal end, also called the front end or the end, both of which indicate the direction relatively farther away from the operator). Among them, the distance from the plane where the endoscope end is located to the plane where the sensitive tissue is located is the observation distance, and the distance from the plane where the endoscope end is located to the endoscope imaging plane is the focal length f of the endoscope.

[0057] like Figure 6-1 and Figure 6-2 As shown: a schematic diagram of endoscope imaging when the distance between the observed sensitive tissue and the endoscope is s1 and s2 respectively. As a schematic diagram, the black area represents the display image range, and the white area represents the imaging area of ​​the endoscope image in the display image range.

[0058] A1 and A2 represent the left end points of the sensitive tissue contour in the two endoscopic images, respectively, represented by the first pixel coordinate A1 (x_A1, y_A1) and the third pixel coordinate A2 (x_A2, y_A2);

[0059] B1 and B2 represent the right end points of the sensitive tissue contour in the two endoscopic images, respectively, represented by the second pixel coordinates B1 (x_B1, y_B1) and the fourth pixel coordinates B2 (x_B2, y_B2);

[0060] D1 and D2 represent the position of the water jet nozzle in the two endoscopic images, pixel coordinates D1 (x_D1, y_D1) and D2 (x_D2, y_D2), respectively;

[0061] E1 and E2 represent the upper edge of the endoscope field of view in the two endoscopic images, pixel coordinates E1 (x_E1, y_E1) and E2 (x_E2, y_E2), respectively;

[0062] O1 and O2 represent the center of the endoscope field of view in the two endoscopic images, pixel coordinates O1(x_O1,y_O1) and O2(x_O2,y_O2), respectively;

[0063] As mentioned above, in the endoscope-guided surgical operation scenario, the initial positions of the endoscope and the water jet are both at the front end of the channel, and the position information of the endoscope can be read by the endoscope driver. When the channel is fixed and ready, the endoscope is at the zero position of its coordinates; at this time, it is assumed that the seminal fossa tissue is at the Z0 position of the endoscope coordinate system, but for the operator, the position of Z0 is not known for sure. As mentioned above, this uncertainty is undesirable;

[0064] According to an embodiment of the present application, after the endoscope and the water jet are fixed in the channel, a first control instruction is sent through the first displacement control module 401, so that the endoscope can perform a retreat action under the control of the first control instruction, and the endoscope retreats a first distance; the position corresponding to the first distance is the first position Z1 where sensitive tissue (such as the seminal fossa) can be seen; the position Z1 is recorded, and the endoscopic image obtained at the position is saved, and the first pixel coordinates A1 (x_A1, y_A1) and the second pixel coordinates B1 (x_B1, y_B1) on the left and right sides of the sensitive tissue are obtained; at this time, the distance from the end of the endoscope to the sensitive tissue is recorded as s1, and the actual distance of s1 is unknown;

[0065] The second control instruction is sent through the second displacement control module 402, so that the endoscope can continue to perform the retreat action under the control of the second control instruction, and the endoscope retreats a second distance, and the position corresponding to the second distance is the second position Z2 where the sensitive tissue (spermia) can be seen; the position Z2 is recorded, and the endoscopic image obtained at the position is saved, and the third pixel coordinates A2 (x_A2, y_A2) and the fourth pixel coordinates B2 (x_B2, y_B2) on the left and right sides of the sensitive tissue are obtained; at this time, the distance from the end of the endoscope to the sensitive tissue is recorded as s2, and the actual distance of s2 is unknown;

[0066] After obtaining the distance s1 of the first position Z1 and the distance s2 of the second position Z2, although the values ​​of s1 and s2 are unknown, the distance between the two positions Z1 and Z2 can be determined according to the difference between the two: s0 = Z2 - Z1 = s2 - s1.

[0067] The third displacement control module 403 is used to determine that the sensitive tissue is located at the Z0 position of the endoscope coordinate system according to the first endoscope image obtained at the first position Z1 and the second endoscope image obtained at the second position Z2, and control the endoscope to move to the Z0 position.

[0068] Specifically, the third displacement control module 403 includes: a pixel length calculation submodule 403a, which is used to calculate the first pixel length L1 and the second pixel length L2 according to the first pixel coordinate A1 (x_A1, y_A1) and the second pixel coordinate B1 (x_B1, y_B1), the third pixel coordinate A2 (x_A2, y_A2) and the fourth pixel coordinate B2 (x_B2, y_B2); wherein the first pixel length L1 represents the pixel length of the sensitive tissue in the first endoscopic image obtained by the endoscope at the position Z1 away from the tissue s1; the second pixel length L2 represents the pixel length of the sensitive tissue in the second endoscopic image obtained by the endoscope at the position Z2 away from the tissue s2. The first pixel length L1 and the second pixel length L2 can be obtained by the following formulas:

[0069]

[0070] The third displacement control module 403 further includes:

[0071] The distance calculation submodule 403b is used to calculate a first distance s1 from the endoscope tip to the sensitive tissue when the endoscope tip is at the position Z1 and a second distance s2 from the endoscope tip to the sensitive tissue when the endoscope tip is at the position Z2 according to the first pixel length L1 and the second pixel length L2;

[0072] Specifically, based on the recorded position information of the first position Z1 and the second position Z2, the distance between Z1 and Z2 can be determined, which is also the difference between the first distance s1 and the second distance s2, that is, s0=s2-s1=Z2-Z1;

[0073] According to the principle of endoscopic imaging, the first distance s1 can be determined according to the following formula using the pixel length information of the endoscopic images at two positions: s1=s0 L2 / (L1-L2), and further, the second distance s2 is calculated and determined according to the following formula: s2=s0+s1.

[0074] The third displacement control module 403 further includes:

[0075] The endoscope coordinate system position calculation submodule 403c is used to calculate the Z0 position of the sensitive tissue in the endoscope coordinate system according to the first distance s1 or the second distance s2.

[0076] Since s1 = Z1 - Z0, s2 = Z2 - Z0

[0077] As mentioned above, the specific values ​​of the first distance s1 and the second distance s2 can be determined by the distance calculation submodule 403b. Further, the sensitive tissue can be determined to be located at the Z0 position of the endoscope coordinate system according to Z0=Z1-s1; or, the sensitive tissue can be determined to be located at the Z0 position of the endoscope coordinate system according to Z0=Z2-s2.

[0078] The third displacement control module 403 also sends a third control instruction to enable the endoscope to move to a third position under the control of the third control instruction, and the third position is a position determined based on the Z0 position. Specifically, the third position can be the Z0 position; however, more preferably, considering the radial length of sensitive tissues such as the seminal fossa tissue (generally not more than 3mm), it is preferable to set the endoscope to move to a position slightly forward of the Z0 position for safety reasons. It is preferred. Regardless of the setting, at least ensure that the position to which the endoscope moves (i.e., the position as the end point of the surgical planning range) does not exceed the Z0 position.

[0079] Through the above embodiments, the present invention aims at the problem of surgical planning based on ultrasonic images, and proposes a scheme such as endoscopic image recognition for auxiliary planning. The control device can be simply used to drive the monocular endoscope to move and obtain its position and image information. The size of the sensitive tissue area on the image is estimated, and the position of the sensitive tissue is determined by calculating the image information, so as to move the endoscope to the relevant position to assist in the planning of the surgical path on the ultrasonic image. The surgical planning area can be set more accurately, and in actual clinical operations, it is possible to avoid planning operations on protected tissue areas as much as possible, and the maximum range of planned resection can be achieved.

[0080] like Figure 7 Shown is a schematic structural diagram of an endoscope displacement control system according to another embodiment of the present disclosure.

[0081] Compared to Figure 4 In the illustrated embodiment, the first displacement control module 401 in the endoscope displacement control system is further used to mark or automatically identify the first upper edge coordinate information E1 (x_E1, y_E1) in the endoscope image at the first position Z1; the second displacement control module 402 is further used to mark or automatically identify the second upper edge information E2 (x_E2, y_E2) in the endoscope image at the second position Z2.

[0082] The endoscope displacement control system further includes: a first surgical planning fan-shaped reserved angle determination module 701, which is used to calculate the fan-shaped reserved angles ∠A1D1O1 and ∠B1D1O1 on both sides of the cross section at the Z1 position of the endoscope coordinate system according to the first upper edge coordinate information E1 (x_E1, y_E1).

[0083] Preferably, the first surgical planning fan-shaped reserved angle determination module 701 is specifically used to calculate the pixel coordinates O1 (x_O1, y_O1) of the center of the endoscopic image field of the cross-sectional view at the Z1 position of the endoscopic coordinate system and the coordinates D1 (x_D1, y_D1) of the water jet nozzle at the Z1 position of the endoscopic coordinate system based on the first upper edge coordinate information E1 (x_E1, y_E1).

[0084] Preferably, the first surgical planning fan-shaped reserved angle determination module 701 is further specifically used to calculate the fan-shaped reserved angles ∠A1D1O1 and ∠B1D1O1 on both sides of the cross-section at the Z1 position of the endoscope coordinate system according to the pixel coordinates O1 (x_O1, y_O1) of the center of the endoscopic image field of the cross-sectional view and the coordinates D1 (x_D1, y_D1) of the water jet nozzle at the Z1 position of the endoscope coordinate system, such as Figure 6-1 shown.

[0085] Preferably, the endoscope displacement control system further includes a second surgical planning fan-shaped reserved angle determination module 702, which is used to calculate the fan-shaped reserved angles on both sides in the cross-sectional view at the Z2 position of the endoscope coordinate system according to the second upper edge coordinate information E2 (x_E2, y_E2).

[0086] As mentioned above, when cutting or removing around sensitive tissue (still taking the collenchyma as an example), the area near the collenchyma needs to be protected in the planning. This planning is mainly based on ultrasonic cross-sectional image planning, and the endoscopic image essentially obtains information of the cross-sectional view. Therefore, the endoscopic image information can be used to assist the planning scheme, reduce the impact of unclear ultrasonic images, and make the planning scheme more perfect. In this embodiment, how to use the information of the endoscopic image to provide key data (sector angle) is important. Specifically, Figure 8 As shown in the figure, a "butterfly"-shaped resection method is used when planning the cross section: the 6 o'clock direction of the cross section is planned to retain a certain fan angle to avoid the verumontanum area. The starting position of the "butterfly"-shaped resection area shall not be earlier than the verumontanum position Z0 (calculated in Plan 1); the end position shall not exceed the front end position ZL of the endoscope (the doctor determines the final position of the endoscope based on experience or the endoscope through other landmarks). The surgical area is planned for the left and right sectors respectively, and the area near the verumontanum is protected.

[0087] According to one embodiment of the present invention, the initial positions of the endoscope and the water jet are both at the front end of the channel according to the surgical operation. Fix the channel and place the endoscope at the zero position of its coordinates; assume that the seminal fossa tissue is located at the Z0 position of the endoscope coordinate system; adjust the endoscope backward until the first position Z1 of the seminal fossa can be seen; record position Z1 and save the endoscopic image. In addition to marking (or AI identifying) the pixel coordinates A1 (x_A1, y_A1) and B1 (x_B1, y_B1) on both sides of the seminal fossa tissue on the image, it is also necessary to mark (or AI identify) the pixel coordinates E1 (x_E1, y_E1) of the upper edge of the endoscopic image field of view; continue to adjust the endoscope backward until the second position Z2 of the seminal fossa can be seen; record position Z2 and save the endoscopic image. In addition to marking (or AI identifying) the pixel coordinates A2 (x_A2, y_A2) and B2 (x_B2, y_B2) on both sides of the seminal fossa tissue on the image, it is also necessary to mark (or AI identify) the pixel coordinates E2 (x_E2, y_E2) of the upper edge of the endoscopic image field of view.

[0088] Taking the cross-sectional view at the Z1 position of the endoscope coordinate system as an example, it is necessary to calculate the fan angles ∠A1D1O1 and ∠B1D1O1 reserved at the 6 o'clock direction when planning the left and right sides of the area near the vermis. Calculate the pixel coordinates D1 (x_D1, y_D1) of the water jet nozzle position in the cross-sectional screenshot at the Z1 position of the endoscope coordinate system and the pixel coordinates O1 (x_O1, y_O1) of the center of the endoscope image field of view in the cross-sectional view.

[0089] Specifically, for a 0° viewing angle endoscope, only the front-to-back distance is changed, and the pixel coordinates of the center of the image field of view will not change, that is, O1(x_O1,y_O1)= O2(x_O2,y_O2). According to A1O1 / A2O2=B1O1 / B2O2, the horizontal coordinate conforms to the relationship: (x_A1-x_O1) / (x_A2-x_O2)=(x_B1-x_O1) / (x_B2-x_O2), and the vertical coordinate conforms to the relationship: (y_A1-y_O1) / (y_A2-y_O2)=(y_B1-y_O1) / (y_B2-y_O2).

[0090] According to the above relationship, the pixel coordinates of the center of the image field of view can be determined: O1 (x_O1, y_O1) and O2 (x_O2, y_O2); according to the hardware structure design of the surgical channel, it can be known that the distance between the nozzle of the knife head and the center of the endoscope (the center of the field of view) in the same cross section is expressed as c; according to the factory design parameters of the endoscope, it can be known that the field of view angle of the endoscope is 2W;

[0091] 1) At the Z1 position of the endoscope coordinate system, the radius of the cross-sectional field of view: r1 = s1 tanW

[0092] 2) At the Z1 position of the endoscope coordinate system, the coordinate D1 (x_D1, y_D1) of the water jet nozzle satisfies the following conditions

[0093]

[0094] We can obtain D1(x_D1,y_D1) and find the reserved sector angles ∠A1D1O1 and ∠B1D1O1

[0095] a) Further, according to the pixel coordinates of A1, D1, and O1, the left cut protection angle ∠A1D1O1 can be calculated; specifically, when the pixel coordinates of A1, D1, and O1 are determined, the side lengths of A1D1, A1O1, and D1O1 can be determined, and the left cut protection angle can be calculated according to the following formula:

[0096]

[0097] b) Further, according to the pixel coordinates of B1, D1, and O1, the right cut protection angle ∠B1D1O1 can be calculated; specifically, when the pixel coordinates of B1, D1, and O1 are determined, the side lengths of B1D1, B1O1, and D1O1 can be determined, and the right cut protection angle can be calculated according to the following formula:

[0098]

[0099] In specific operations, the actual size of the seminal tissue (or other tissue) can also be calculated as the transverse diameter L = r A1B1 / O1E1, where, when the pixel coordinates of O1, E1, A1, and B1 are determined, the lengths of A1B1 and O1E1 are also determined. Combined with the endoscope field radius r1, the transverse diameter of the seminal follicle tissue can be determined.

[0100] This embodiment further utilizes the design parameters of the surgical instrument channel and the endoscope itself to further calculate and determine the cross-sectional information near the sensitive tissue area, providing reliable and accurate information for the cross-sectional planning of the surgical protection tissue area, helping to assist in the completion of comprehensive and accurate planning, and being able to more accurately determine the location of sensitive tissues, helping to form a safe and detailed surgical planning plan.

[0101] According to another embodiment of the present invention, there is provided an endoscope displacement control method, characterized in that it comprises an endoscope, an endoscope driver, a control device and a water jet driver; the control device is connected to the endoscope via the endoscope driver, and is used to connect to the water jet via the water jet driver; the endoscope driver is used to read the position information of the endoscope, and the initial position of the endoscope is located at the zero position of the endoscope coordinate system;

[0102] The method comprises:

[0103] After the endoscope is installed in place, the endoscope is controlled to move back until the first position Z1 of the sensitive tissue can be seen in the endoscope field of view, and the first endoscopic image of the position is saved, and the first pixel coordinates A1 (x_A1, y_A1) and the second pixel coordinates B1 (x_B1, y_B1) representing the left and right sides of the sensitive tissue on the first endoscopic image are obtained, and the position information of the first position Z1 is recorded;

[0104] Control the endoscope to continue to retreat until a second position Z2 of the sensitive tissue is visible in the endoscope field of view, save a second endoscopic image of the position, obtain third pixel coordinates A2 (x_A2, y_A2) and fourth pixel coordinates B2 (x_B2, y_B2) on the second endoscopic image, respectively representing the left and right sides of the sensitive tissue, and record position information of the second position Z2; and

[0105] Control the endoscope to move to a third position determined based on the Z0 position, where the Z0 position is the position of the sensitive tissue in the endoscope coordinate system, determined according to the first pixel coordinate A1 (x_A1, y_A1) and the second pixel coordinate B1 (x_B1, y_B1), and the third pixel coordinate A2 (x_A2, y_A2) and the fourth pixel coordinate B2 (x_B2, y_B2).

[0106] It should be understood that the specific embodiments described above are only used to explain the present application, and the protection scope of the present application is not limited thereto. Any technical personnel familiar with the technical field can make changes, substitutions, and combinations within the technical scope disclosed in the present application according to the technical solution and concept of the present application, which should be covered by the protection scope of the present application.

Claims

1. An endoscope displacement control system, characterized in that: include: Endoscopes, endoscope drivers, control devices, and water jet drivers; The control device is connected to the endoscope via the endoscope driver and is connected to the water jet via the water jet driver; The endoscope driver is also used to read the position information of the endoscope, and the initial position of the endoscope is located at the zero position of the endoscope coordinate system; The control device comprises: a first displacement control module, used to control the endoscope to move to a first position Z1 where sensitive tissue can be seen in the endoscope field of view, save a first endoscopic image of the position, obtain first pixel coordinates A1 (x_A1, y_A1) and second pixel coordinates B1 (x_B1, y_B1) on the first endoscopic image, respectively representing left and right sides of the sensitive tissue, and record position information of the first position Z1; a second displacement control module, used for controlling the endoscope to move to a second position Z2 where the sensitive tissue can be seen in the endoscope field of view, saving a second endoscope image at the position, obtaining a third pixel coordinate A2 (x_A2, y_A2) and a fourth pixel coordinate B2 (x_B2, y_B2) on the second endoscope image respectively representing the left and right sides of the sensitive tissue, and recording the position information of the second position Z2; The third displacement control module is used to determine that the sensitive tissue is located at the Z0 position of the endoscope coordinate system according to the first pixel coordinate A1 (x_A1, y_A1) and the second pixel coordinate B1 (x_B1, y_B1), and the third pixel coordinate A2 (x_A2, y_A2) and the fourth pixel coordinate B2 (x_B2, y_B2), and control the endoscope to move to a third position determined based on the Z0 position.

2. The endoscope displacement control system according to claim 1, characterized in that: The third displacement control module also includes: a pixel length calculation submodule, which is used to calculate a first pixel length L1 according to the first pixel coordinate A1 (x_A1, y_A1) and the second pixel coordinate B1 (x_B1, y_B1), and calculate a second pixel length L2 according to the third pixel coordinate A2 (x_A2, y_A2) and the fourth pixel coordinate B2 (x_B2, y_B2); wherein the first pixel length L1 represents the pixel length of the sensitive tissue in the first endoscopic image obtained by the endoscope at the first position Z1; the second pixel length L2 represents the pixel length of the sensitive tissue in the second endoscopic image obtained by the endoscope at the second position Z2; the first pixel length L1 and the second pixel length L2 can be obtained by the following formulas: 。 3. The endoscope displacement control system according to claim 2, characterized in that: The third displacement control module also includes: a distance calculation submodule, which is used to determine a first distance s1 between the endoscope tip and the sensitive tissue when the endoscope is in the first position Z1, and a second distance s2 between the endoscope tip and the sensitive tissue when the endoscope is in the second position Z2, based on the first pixel length L1 and the second pixel length L2.

4. The endoscope displacement control system according to claim 3, characterized in that: The distance calculation submodule determines the first distance s1 according to the following formula: s1=s0 L2 / (L1-L2); and determine the second distance s2 according to the following formula: s2=s0+s1, where s0 represents the distance between the first position Z1 and the second position Z2.

5. The endoscope displacement control system according to claim 3, characterized in that: The third displacement control module further includes: an endoscope coordinate system position calculation submodule, which is used to determine that the sensitive tissue is located at the Z0 position of the endoscope coordinate system according to the first distance s1 or the second distance s2.

6. The endoscope displacement control system according to claim 1, characterized in that: Also includes: The first surgery planning sector reserved angle determination module is used to determine the surgery planning sector reserved angle of the position according to the first endoscopic image acquired at the first position Z1.

7. The endoscope displacement control system according to claim 6, characterized in that: The first displacement control module is also used to determine the first upper edge pixel coordinate E1 (x_E1, y_E1) in the first endoscopic image at the first position Z1, and the first surgical planning fan-shaped reserved angle determination module calculates the pixel coordinate O1 (x_O1, y_O1) of the center of the field of view in the first endoscopic image at the first position Z1 and the pixel coordinate D1 (x_D1, y_D1) of the water jet nozzle at the first position Z1 according to the first upper edge pixel coordinate E1 (x_E1, y_E1).

8. The endoscope displacement control system according to claim 7, characterized in that: The first surgical planning fan-shaped reserved angle determination module also calculates the fan-shaped reserved angles ∠A1D1O1 and ∠B1D1O1 on both sides in the cross-sectional view at the first position Z1 based on the pixel coordinates O1 (x_O1, y_O1) of the center of the field of view and the pixel coordinates D1 (x_D1, y_D1) of the water jet nozzle at the first position Z1.

9. The endoscope displacement control system according to claim 6, characterized in that: Also includes: The second surgery planning sector reserved angle determination module is used to determine the surgery planning sector reserved angle of the position according to the second endoscopic image acquired at the second position Z2.

Images