Endoscope state detection system and method

Abstract

The invention discloses an endoscope state detection system and method. The endoscope state detection system comprises a first form detection unit used for detecting the bending form of a bending part, a second form detection unit used for detecting the form of a flexible part of the endoscope, and a detection unit used for performing form integration and alignment on a first form and a second form to obtain shape data of an insertion part, the integration unit is used for acquiring the shape data of the bent part and the flexible part and identifying the overall shape of the insertion part inserted into the lumen by using the shape data, the overall shape of the insertion part is acquired by respectively acquiring the postures of the bent part and the flexible part and performing segmented reconstruction, and a display picture is generated according to the shape data synthesized by the integration unit. According to the scheme, an extra shape obtaining related sensor does not need to be placed at the bending part of the endoscope, the overall shape of the whole endoscope insertion part is provided for a doctor while the diameter of the end part of the endoscope head is not increased to influence the bending performance and the instrument channel size, the operability of the endoscope is improved, and the burden of the doctor and a patient is reduced.

Description

Technical Field

[0001] The present application relates to the field of medical equipment technology, and in particular to an endoscope status detection system and a detection method. Background Art

[0002] At present, endoscopy is an effective way to diagnose and treat digestive tract diseases and early cancer. However, due to the complex structure of the digestive tract system, poor lighting environment and constant peristalsis, the information that an endoscope can provide to doctors is limited. In particular, the shape and position of the endoscope in the human body can easily form loops in the digestive tract, causing discomfort or even life-threatening to the patient. Therefore, endoscopy requires extremely high experience from doctors. At the same time, the endoscope head is prone to contact with the inner wall of the digestive tract in the body, making insertion and withdrawal difficult, and the doctor cannot know the size of the contact force by operating the endoscope, which further increases the requirements for the doctor.

[0003] The detection method of the contact force between the endoscope and the human tissue can be divided into direct detection at the head end of the endoscope and indirect detection in the operating part according to the installation position. Direct detection at the head end requires the placement of a pressure sensor at the head end, but this method will inevitably increase the diameter of the insertion part, and the head end will be deformed during installation, affecting its detection of the contact force on the subject. At the same time, common commercial thin film pressure sensors increase the difficulty of endoscope disinfection. If indirect detection is adopted by placing a tension sensor in the operating part, the contact force of the front end is deduced by detecting the tension on the traction wire. However, due to the limited space in the operating part and the elasticity, friction, and other external factors of the traction wire, the accuracy of the tension may be affected, introducing additional errors, thereby affecting the calculation of the contact force.

[0004] To solve the above problems, the present application proposes a method for deriving the front contact force according to the shape of the insertion part, which requires obtaining the overall shape of the insertion part. CN112294236 A discloses a method for deriving the bending angle of the front bending part using the displacement information of the traction wire, but this method cannot obtain the shape of the flexible part of the insertion part other than the bending part. Although CN104755007A discloses a method for placing an optical fiber sensor in the insertion part to obtain the shape of the entire insertion part, and CN104755005 A discloses a method for generating the shape of the entire insertion part using electromagnetic positioning, the above two methods require placing sensors in the flexible part of the bending part, and the bendable section of the bending part is provided with a node ring. The node ring is a magnetic conductive material, which is easy to interfere with the electromagnetic positioning, and the frequent bending of the node ring will also damage the life of the optical fiber. Furthermore, the bending part of the endoscope integrates various structures necessary for endoscopic imaging and instrument channels through which surgical instruments can pass. The entire pipeline space is limited. Arranging sensors or optical fiber channels in it will inevitably reduce the space of the instrument channel or increase the diameter of the bending part of the endoscope, greatly reducing the efficiency of the operation. Summary of the invention

[0005] In order to accurately obtain the overall insertion shape information of the insertion part without making the structure of the device larger or more complicated, so as to make the endoscopic operation more precise and improve the efficiency of inspection and treatment, the present application proposes an endoscope state detection system and method, which can respectively obtain the postures of the bending part and the flexible part and obtain the overall shape of the insertion part by means of partial reconstruction.

[0006] The endoscope state detection system of the present application includes: an insertion portion, including a bending portion and a flexible portion, the bending portion being bent by the operation of the operating portion; a first morphology detection unit, used to detect displacement change information of the traction wire in the insertion portion, and estimate the first morphology of the bending portion based on the displacement change information; a second morphology detection unit, used to detect morphology data of the flexible portion in a current morphology, and estimate the second morphology of the flexible portion based on the morphology data; an integration unit, configured to morphologically integrate and align the first morphology and the second morphology to obtain shape data of the insertion portion, the shape data being used to identify the overall shape of the insertion portion inserted into the lumen; and a display unit, used to generate a display screen based on the shape data synthesized by the integration unit.

[0007] Preferably, the first form detection unit comprises: a displacement detection unit for detecting a displacement contact amount generated by a displacement change of the traction wire; and a posture calculation unit for calculating the first form of the bending portion according to the displacement contact amount and a preset analytical model.

[0008] Preferably, the second morphology detection unit comprises a morphology sensor arranged in the flexible portion along the length direction of the insertion portion, and the morphology sensor is used to implement distributed multi-point measurement or continuous measurement.

[0009] Preferably, a first posture sensor is provided on one side of the bending portion close to the flexible portion, which is arranged parallel to the bending axis direction of the bending portion and is used to obtain a first alignment posture corresponding to the first form;

[0010] A second posture sensor is provided inside the operating portion of the endoscope and / or outside the endoscope, for acquiring a second alignment posture corresponding to the second form;

[0011] The integration unit is used to splice the first alignment posture and the second alignment posture to obtain shape data of the insertion portion.

[0012] Preferably, the endoscope state detection system further comprises: an auxiliary unit, which assists in correcting the overall shape of the insertion portion by acquiring the base posture and insertion state of the insertion portion.

[0013] Preferably, the auxiliary unit includes: a base posture correction unit, which is used to obtain the base point coordinates and relative position orientation of the insertion unit relative to the subject through a second posture sensor to correct the base posture of the insertion unit; and an insertion state correction unit, which is used to obtain the motion state information of the insertion unit to correct the insertion state of the insertion unit.

[0014] Preferably, the insertion state correction part can obtain the movement state information of the insertion part through a visual sensor arranged on the insertion part, or a visual sensor arranged at the junction of the insertion part outside the lumen and inside the lumen, or a visual sensor arranged at the insertion part aimed at the insertion part, so as to correct the insertion state of the insertion part.

[0015] Preferably, the endoscope state detection system also includes: a predicted posture acquisition unit, which is used to measure the relative position information of the flexible part in the respective measurement areas and / or the real-time alignment posture in the respective measurement areas according to a real-time posture measurement unit arranged in multiple measurement areas along the length direction of the insertion part, so as to obtain the predicted posture of the second form; and a posture updating unit, which is used to update the second form of the flexible part using the predicted posture during the solution process of the second form detection unit.

[0016] Preferably, the endoscope state detection system also includes: a contact force estimation unit, which is used to estimate the magnitude and direction of the contact force applied by the insertion part to the object based on the change in the bending angle of the bending part and the change in the deflection of the flexible part when the insertion part contacts the inside of the object.

[0017] Preferably, the contact force estimation unit further comprises: a deflection change calculation unit for obtaining a deflection angle change value according to a relative difference between a real-time deflection angle and an initial angle formed between the passive deflection section and the non-deflection section after the flexible section contacts the object.

[0018] Another aspect of the present application provides an endoscope state detection method, which is applied to the aforementioned endoscope state detection system, and the method includes: detecting displacement change information of the traction wire in the insertion part, and estimating the first form of the bending part based on the displacement change information; detecting the morphological data of the flexible part in the current form, and estimating the second form of the flexible part based on the morphological data; morphologically integrating and aligning the first form and the second form to obtain shape data of the insertion part, and the shape data is used to identify the overall shape of the insertion part inserted into the lumen; generating a display screen based on the shape data synthesized by the integration unit.

[0019] Preferably, the magnitude and direction of the contact force are estimated based on the change in the bending angle of the bending portion and the change in the swing angle of the flexible portion; wherein the change in the swing angle is the relative difference between the real-time swing angle and the initial angle formed between the passive swing segment and the non-swing segment after the flexible portion contacts the object.

[0020] The bending portion and the flexible portion provided in the present application obtain the overall shape of the insertion portion through partial reconstruction, and at the same time, the displacement change information of the traction wire is measured by the sensor in the operating portion, and the morphology sensor placed in the flexible section respectively obtains the first shape of the bending portion and the second shape of the flexible portion, and the first shape and the second shape are integrated to obtain the overall shape of the entire insertion portion. There is no need to place additional shape-acquisition related sensors in the bending portion. The overall shape of the insertion portion can be obtained without increasing the diameter of the endoscope head end and without affecting the bending performance and the diameter of the instrument channel. In addition, the posture-related data of the bending portion and the flexible portion are processed and fused in a morphologically integrated manner, which helps to improve the accuracy of the morphological reconstruction and make the displayed shape of the insertion portion closer to the actual situation. Furthermore, when doctors perform endoscopic diagnosis and treatment, they can obtain the overall shape of the endoscope insertion part in the subject, which can effectively avoid or deal with the situation of looping when the endoscope insertion part is in the lumen of the subject, or identify and solve the looping problem at an early stage; further, they can also obtain more reliable perception based on the force information of the curved part and / or the flexible part, better grasp the contact force information between the head end of the endoscope head and the tissue, and further accurately estimate the change trend of the contact force. The movement path of the insertion part can be adjusted to reduce damage to the subject, and the state of the subject's inspected part can be accurately evaluated, reducing the burden on the subject, and quickly inserting and withdrawing. By reducing the reliance on the doctor's experience, the surgical risk is reduced. Even for less experienced doctors, these systems can provide necessary guidance and feedback to ensure the safe conduct of the operation, thereby reducing the patient's discomfort during the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of the structure of an endoscope status detection system provided by an embodiment of the present application;

[0023] Figure 2 A schematic diagram of a structure in which a bending portion is bent by displacement of a traction wire provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the structure of a thin film sensor provided in an embodiment of the present application

[0025] Figure 4 A schematic diagram of the structure of placing the applied thin film sensor in the operating part provided in the embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of the configuration sensor provided in the embodiment of the present application placed in the flexible portion;

[0027] Figure 6 A schematic diagram of a morphology integration method using optical fiber as a morphology sensor provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of a morphology integration method using optical fiber as a morphology sensor provided in an embodiment of the present application;

[0029] Figure 8 A schematic diagram of a morphological integration method using an electromagnetic sensor as a morphological sensor provided in an embodiment of the present application;

[0030] Fig. 9 A schematic diagram of a method for realizing posture integration of a bending portion and a flexible portion by using an electromagnetic transmitting coil provided in an embodiment of the present application;

[0031] Fig.10 A schematic diagram of a method for realizing posture integration of a bending portion and a flexible portion using a strain gauge provided in an embodiment of the present application;

[0032] Fig.11 A schematic diagram of a method for realizing posture integration of a bending portion and a flexible portion using a micro camera provided in an embodiment of the present application;

[0033] Fig.12 A system block diagram of an auxiliary unit provided in an embodiment of the present application;

[0034] Fig.13 A schematic diagram of insert information detection provided in an embodiment of the present application;

[0035] Fig.14 A system block diagram of morphology optimization provided in an embodiment of the present application;

[0036] Fig.15 A schematic diagram of a structure of morphological optimization provided in an embodiment of the present application;

[0037] Fig.16 A schematic diagram of the colon structure;

[0038] Fig.17 A schematic diagram of the variation of the extension of the head end and the contact with the tissue provided in the embodiment of the present application;

[0039] Fig.18 A schematic diagram of a change in the insertion portion of an endoscope provided in an embodiment of the present application when it contacts the underlying tissue just after passing the curved section;

[0040] Fig.19 A schematic diagram of a change in the insertion portion of an endoscope provided in an embodiment of the present application when the insertion portion just passes the curved section and contacts the tissue above;

[0041] Fig. 20 A schematic diagram of analyzing a bending portion as an elastic beam provided in an embodiment of the present application;

[0042] Fig.21 A schematic diagram of a pseudo-rigid body analysis in which a bending portion is modeled as a rigid link and a torque spring provided in an embodiment of the present application;

[0043] Fig. 22 A schematic diagram of a pseudo-rigid body model analysis after adding a passive deflection section and a non-deflection section provided in an embodiment of the present application;

[0044] Fig.23 A schematic diagram of changes in contact between the end of the endoscope head and tissue provided in an embodiment of the present application;

[0045] Fig.24 For the general Fig.23 Schematic diagram of pseudo-rigid body analysis for contact situations;

[0046] Fig.25a It is a schematic diagram of the change of the contact between the flexible section of the insertion part of the endoscope and the tip end and the tissue;

[0047] Fig.25b It is a schematic diagram of the change of the contact between the flexible section of the insertion part of the endoscope and the tip end and the tissue;

[0048] Fig.26a For the general Fig.25a Schematic diagram of pseudo-rigid body analysis for contact situations;

[0049] Figure 26b For the general Fig.25b Schematic diagram of pseudo-rigid body analysis for contact situations

[0050] Fig. 27 A schematic flow chart of an endoscope system status detection method provided in an embodiment of the present application.

[0051] Description of Reference Numerals

[0052] 1. Endoscope state detection system; 10. Endoscope; 11. Insertion part; 12. Operation part; 14. Head end; 15. Operation knob; 16. Bending part; 17. Traction wire; 18. Flexible part; 181. Passive deflection section; 182. Non-deflection section; 161. Both end node rings; 121. Side partition; 20. First form detection unit; 30. Second form detection unit; 40. Integration unit; 50. Display unit; 60. Auxiliary unit; 70. Prediction posture acquisition unit; 72. Posture update unit; 80. Contact force estimation unit; 90. Display unit; 21. Thin film sensor; 22. Sliding body; 23. Insertion portion through hole; 24. Visual sensor; 211. Detection area; 212. Circuit area; 222. Protrusion; 31. Shape sensor; 311. Electromagnetic sensor; 312. Electromagnetic transmitting coil; 314. Electromagnetic receiving coil; 315. Micro camera; 401. First posture sensor; 402. Second posture sensor; 601. Base posture correction unit; 602. Insertion state correction unit. DETAILED DESCRIPTION

[0053] In the description of the embodiments of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application. In addition, the terms "first", "second", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0054] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0055] In the embodiments of the present application, unless otherwise clearly specified and limited, the first feature being "above" or "below" the second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature being "above", "above" and "above" the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. The first feature being "below", "below" and "below" the second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature. In the description of this specification, unless otherwise clearly specified and limited, "forward and backward" is equivalent to "advance and retreat".

[0056] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0057] The endoscope state detection system of each embodiment of the present application is described below. The endoscope state detection system can be widely applied to insertion instruments with flexible components such as medical endoscopes, and the endoscope state detection system can be applied to medical endoscopes such as upper gastrointestinal endoscopes, colon endoscopes, ultrasonic endoscopes, cystoscopes, and pyeloscopes.

[0058] In this embodiment, the subject is mainly assumed to be a patient or an organ of a patient who is undergoing diagnosis or treatment. The subject may not be a patient or an organ, but a patient model or an organ model for simulation. Alternatively, the subject may be an animal such as mammals, reptiles, or birds. In addition, not limited to the medical field, the subject may also be an instrument or workpiece having a lumen or space inside.

[0059] The target of the endoscope state detection system of this embodiment, that is, the lumen of the subject, is mainly the digestive organs, bronchial tubes, urinary organs, etc. In addition to the lumen, organs after opening the abdomen during surgery, etc. can also be targeted.

[0060] In this embodiment, the earth reference coordinate system generally refers to a coordinate system based on the earth, and the earth reference system determines the absolute position and posture of the insertion portion in the lumen of the subject.

[0061] like Figure 1 , a schematic diagram of the structure of an endoscope state detection system is shown. An endoscope 10 of the endoscope state detection system 1 includes an insertion portion 11 for inserting into a lumen and an operation portion 12 provided at the base end side of the insertion portion 11. The insertion portion 11 includes: an endoscope head end 14 provided at the front end; a bending portion 16 provided at the base end side of the endoscope head end 14; and an elongated flexible portion 18 provided at the base end side of the bending portion 16.

[0062] Furthermore, one end of four traction wires 17 (not shown) as a bending operation transmission member is connected to the front end of the bending portion 16 of the insertion portion 11. The traction wires 17 pass through the inside of the flexible portion 18, and the other end is connected to the operation knob 15 as a bending operation portion provided on the operation portion 12. The traction wires are moved in the longitudinal direction by the operation of the operation knob 15. The bending portion 16 is bent in the vertical direction and the left-right direction of the endoscope 10 by the movement of the traction wires 17.

[0063] Furthermore, the bending portion 16 is provided with two end node rings 161 consisting of a front node ring 161A and a rear node ring 161B, the front node ring 161A (not shown in the figure) is connected to the end of the endoscope head 14, and the rear node ring 161B (not shown in the figure) is connected to the front end of the flexible portion 18. The traction wire 17 is connected to the front node ring 161A through a through hole formed inside the front node ring 161A. When the traction wire 17 is displaced, the front node ring 161A of the insertion portion 11 is driven to bend, and the rear node ring 161B is bent by extrusion. The displacement of the traction wire 17 directly acts on the bending portion 16, causing the bending angle or shape thereof to change.

[0064] like Figure 1 and Figure 2As shown, the present application provides an endoscope state detection system including: a first form detection unit 20, which is used to detect the displacement change information of the traction wire 17 in the insertion part 11, and estimate the first form of the bending form of the bending part 16 according to the displacement change information; a second form detection unit 30, which includes a form sensor 31 installed in the flexible part 18, and a plurality of form sensors 31 can be placed in the flexible part 18 at a certain predetermined interval along the length direction of the insertion part 11 or formed continuously in the flexible part 18, and is used to detect the form data at the corresponding position of the flexible part 18, and estimate the second form of the flexible part 18 at each position or the entire flexible part including the position and direction according to the form data; an integration unit 40, which is configured to integrate and align the first form and the second form, obtain and integrate the characteristic information such as the bending angle change, shape change, bending direction and spatial position of the first form and the second form, so as to obtain the shape data of the insertion part, and the shape data is used to identify the overall shape of the insertion part inserted into the lumen, and the display unit 50 is used to generate a display screen according to the shape data synthesized by the integration unit 40.

[0065] The first form detection unit 20 includes: a displacement detection unit 201 for detecting the displacement contact amount generated by the displacement change of the traction wire; and a posture estimation unit 202 for estimating the first form of the bending portion according to the displacement contact amount and a preset analytical model.

[0066] For example, Figure 2 As shown, a schematic diagram of detecting the bending angle of the bending portion using a thin film sensor is shown. The first form detection unit 20 specifically includes a thin film sensor 21 provided on the side partition inside the operating portion 12, and a sliding body 22. The sliding body 22 is connected to one end of the traction wire 17. When the operating portion 12 is operated, the traction wire 17 is displaced by pulling the sliding body 22. When the traction wire 17 is displaced, the thin film sensor 21 is in direct contact with the sliding body 22. The displacement detection unit 201 detects the corresponding displacement contact amount during contact and movement, and obtains displacement change information based on the linear correspondence between the displacement contact amount generated by the contact and the displacement change amount. The posture estimation unit 203 obtains the relationship between the displacement change information of the traction wire 17 and the bending angle of the bending portion 16 according to the displacement change information combined with high-order fitting, neural network and other methods, and combines the preset kinematic model such as exponential product or DH parameter method to perform modeling and analytical fitting to obtain the first form of the bending portion 16.

[0067] Specifically, the bending angle of the bending portion 16 can be obtained in an end-to-end manner, and the two end node rings 161 are arranged at basically the same intervals and shapes in the length direction of the insertion portion 11. Therefore, after obtaining the bending angle of the bending portion 16 of the endoscope in one direction, the angle can be averaged according to the number of the two end node rings 161 in this direction as the bending angle of each node ring in the bending portion 16 in this direction.

[0068] like Figure 3 As shown, a schematic diagram of a film sensor 21 is shown. The film sensor 21 arranged on the side partition 121 of the operating part has a detection area 211, in which multiple layers of resistors and circuits are arranged. When it is in contact, it is connected to the circuit. According to the different corresponding positions, resistors with different resistance values ​​will be introduced into the circuit, thereby generating different voltages. The size of the resistance (voltage) is linearly related to the displacement contact amount formed by the contact position. Therefore, when the traction wire 17 changes in displacement, the direct contact position of the sliding body 22 and the film sensor 21 changes accordingly. The film sensor 21 collects the change in the displacement contact amount between the sliding body 22 and the film sensor 21 due to the displacement change of the traction wire 17. The change in the displacement contact amount causes the film sensor 21 to produce a resistance change; the circuit area 212 is used to power the detection area 211 and process and transmit the displacement change, so as to determine the displacement contact amount of the traction wire 17 and then estimate the first form of the bending portion.

[0069] like Figure 4 As shown, it is a schematic diagram of the placement of a thin film sensor in the operating part. The thin film sensor 21 is attached to the side partition 121 of the operating part 12 to realize the displacement change of the traction wire. The sliding body 22 is provided with a protrusion 222 (not shown in the figure). The thin film sensor 21 is in direct contact with the sliding body 22 through the protrusion 222. When the sliding body 22 moves, the protrusion 222 moves on the thin film sensor 21, which changes the contact point inside the thin film sensor 21, so that the resistance change introduced into the circuit by the thin film sensor 21 is changed, and then the displacement change information of the traction wire 17 is calculated. Specifically, when the bending hand wheel on the operating part 12 of the operating endoscope 10 is rotated, the sliding body 22 is driven to generate displacement through the chain mechanism in the insertion part 11, thereby driving the traction wire 17 to generate displacement, that is, the change in the contact position corresponding to the direct contact between the thin film sensor 21 and the protrusion 222 on the sliding body 22 is equivalent to the displacement amount of the traction wire 17, and the thin film sensor 21 collects the contact position information and then detects the displacement change information of the traction wire 17. By experimentally measuring the displacement of the traction wire and the bending angle of the front end bending portion, the relationship between the bending angle and the displacement can be established, and then the bending shape of the bending portion 16 at the front end of the endoscope can be estimated after the displacement change of the traction wire is obtained during diagnosis and treatment.

[0070] According to one embodiment of the present application, the second form detection unit 30 can be through the form sensor 31 of the flexible portion 18, and the form sensor 31 is placed along the length direction of the insertion portion 11. The form sensor 31 can be arranged in a column or continuously at a predetermined interval to realize distributed multi-point measurement or continuous measurement to obtain the second form of the flexible portion 18.

[0071] Specifically, the morphology sensor 31 can use a flexible strain sensor to detect changes in the flexible part 18 when it is bent. The flexibility of the flexible strain sensor enables it to fit tightly on the flexible part 18 without affecting its normal bending and operation, accurately detect morphology changes, and provide precise information about the bending posture of the flexible part 18.

[0072] According to an embodiment of the present application, the morphology sensor 31 may also be an electromagnetic transmitting device placed in the flexible portion 18 of the endoscope, wherein a plurality of electromagnetic transmitting devices are placed in series in the flexible portion 18 along the length direction of the insertion portion 11 at fixed intervals to form an electromagnetic transmitting device, and a plurality of corresponding electromagnetic receivers are installed at fixed positions in a certain space outside the endoscope 10, and are placed in series to form an electromagnetic receiving device. The electromagnetic transmitter may be an electromagnetic transmitting coil, which generates an alternating electromagnetic field when an alternating current is passed through the coil; the electromagnetic receiving device is composed of an electromagnetic receiving coil, which generates an induced voltage on the coil when receiving the alternating electromagnetic field, and different postures of the electromagnetic transmitter will cause the receiving device to generate different induced voltages, which may be the morphology data of the current shape of the flexible portion 18. By pre-calibrating the electromagnetic receiving device, the different induced voltages generated by different postures can be solved, thereby obtaining the position and posture of each electromagnetic transmitter in the flexible portion 18, and then the second shape of the flexible portion 18 can be obtained by fitting the data points.

[0073] Among them, the fitting algorithm used when fitting the data points can adopt second-order Bezier curve fitting and neural network, etc.

[0074] Among them, the accuracy of the calculated second form can be improved by powering the electromagnetic transmitting device at different times or by supplying alternating current of different frequencies.

[0075] According to an embodiment of the present application, the acquisition of the second form of the flexible portion 18 can also be carried out in the form of optical fiber tracking positioning. In this case, the form sensor 31 adopts a FBG (Fiber Bragg Grating) sensor. The FBG sensor carves a Bragg grating on the optical fiber at intervals of a small section, and uses its reflection and transmission characteristics to detect the curvature under the influence of strain and temperature. The FBG sensors are placed in a certain order along the length direction of the insertion portion 11. When the posture of the flexible portion 18 of the endoscope 10 changes, the reflection characteristics of the FBG sensor change. Through spectrometer analysis and comparison with the standard reflection spectrum, the current curvature at the specific position of the flexible portion 18 where each FBG sensor is located can be obtained. The information of multiple FBG sensors is summarized to obtain the current curvature at multiple specific positions on the flexible portion 18 as form data. Through multiple curvature fitting, the form of the flexible segment 6 of the endoscope can be detected.

[0076] like Figure 5 , which is a schematic diagram showing a structure of placing a morphology sensor in a flexible portion. Specifically, the morphology sensor 31 can be placed at a certain position between two traction wires 17 .

[0077] like Figure 6 As shown, a method of realizing posture integration of the bending part and the flexible part by using a multi-core optical fiber is shown. The multi-core optical fiber is used as the morphological sensor 31, and the integration unit 40 integrates the fixed relative position of the traction wire 17 and the multi-core optical fiber. The multi-core optical fiber is composed of multiple optical fibers placed in the insertion part 11 at a certain angle. Each optical fiber in the multi-core optical fiber can independently detect the different bending degrees and directions of the flexible part 18 at its corresponding position. The shape fitting can be performed using the information provided by the symmetrically arranged optical fibers in the multi-core optical fiber. The displacement change of the traction wire 17 corresponds to the different bending directions of the bending part 16. Each optical fiber in the multi-core optical fiber has a fixed relative position. The fixed relative position is used for pre-calibration, and each possible bending angle of the bending part 16 is bound to each specific optical fiber, so as to integrate the overall shape of the insertion part.

[0078] like Figure 7 , a method of using multi-axis optical fiber to realize the posture integration of the bending part and the flexible part is shown. Using the multi-axis optical fiber as the shape sensor 31, the integration unit 40 integrates the information of the first posture sensor 401 placed on the base end side of the bending part 16.

[0079] like Figure 8 , a method of integrating the posture of the bending part and the flexible part using electromagnetic sensors is shown. The electromagnetic sensor 311 is used as the shape sensor 31 for detecting the posture of the flexible part 18 , and the integration unit 40 integrates the information of the first posture sensor 401 placed on the base end side of the bending part 16 .

[0080] Furthermore, in order to better integrate the overall shape of the insertion part, the initial reference systems of the first form and the second form are converted to the earth reference system, and the rotation matrix and the translation vector are fused through operations such as rotation and translation of the posture information to obtain the first alignment posture and the second alignment posture of the bending portion 16 and the flexible portion 18 of the endoscope 10 in the earth reference system, thereby obtaining the first and second alignment postures to fuse the first and second forms to obtain the overall shape of the insertion part 11.

[0081] At Figure 1 , Figure 7 and Figure 8 The specific integration method shown in the figure includes defining a reference coordinate system as s, which may be a ground coordinate system in the present application, and the second posture sensor 402 in the operating portion 12 is capable of converting the second form into a second alignment posture R of the flexible portion 18 relative to the reference coordinate system. sf The first posture sensor 401 placed on the base end side of the bending portion 16 can convert the first form into a first alignment posture R of the bending portion 16 relative to the reference coordinate system. sc , the first alignment posture R sc and the second alignment posture R sf The first position sensor 401 is placed parallel to the bending axis of the bending portion 16. Or directly use R sc The first state of the bending portion 16 is transformed and aligned with the second alignment posture R of the flexible portion 18 when displayed. sf Perform end splicing.

[0082] like Fig. 9 FIG. 1 is a schematic diagram showing a method of realizing the posture integration of the bending part and the flexible part by using an electromagnetic transmitting coil. An electromagnetic transmitting coil 312 parallel to the axial direction of the flexible part 18 is placed at the base of the bending part 16 close to the flexible part 18, and an electromagnetic receiving coil 313 (not shown in the figure) is placed at a certain position outside the endoscope. When an alternating current is passed through the electromagnetic transmitting coil, a voltage change can be generated in the electromagnetic receiving coil, and the posture information R of the electromagnetic transmitting coil can be obtained by calibrating the voltage. st , using this information and the second alignment posture R sf , integration can be achieved, and the integration method is similar to the above and will not be repeated here.

[0083] like Fig.10As shown, a schematic diagram of a method of realizing posture integration of the bending portion and the flexible portion using strain gauges is shown. A plurality of strain gauges 314 are arranged circumferentially at the connection between the bending portion 16 and the flexible portion 18. For example, four strain gauges 314 are placed at 90-degree intervals or three strain gauges 314 are placed at 120-degree intervals. When the strain gauge is bent, the resistance changes, thereby causing the voltage to change. Thus, the deflection direction of the bending portion 16 relative to the flexible portion 18 can be calibrated. For example, when the bending portion 16 bends to the right, the strain gauge attached to the right side will produce a voltage change, so that the bending direction of the bending portion can be aligned with the right axis, thereby achieving morphological integration.

[0084] like Fig.11 , a schematic diagram of a method of realizing posture integration of the bending portion and the flexible portion using a micro camera is shown. A micro camera 315 is placed at a position of the flexible portion 18 close to the bending portion 16, and the relative position of the camera relative to the flexible portion 18 can be obtained through calibration. The camera captures the bending direction of the bending portion 16 inside the endoscope and aligns the bending direction to the second alignment posture coordinate system, so that integration can be achieved.

[0085] According to one embodiment of the present application, linear interpolation, spline function interpolation, transformation matrix combination, etc. can also be used between the first form and the second form to perform morphological integration to ensure a smooth transition between the two postures, and the complete shape data can be spliced ​​together by using methods such as geometric alignment and feature alignment.

[0086] According to one embodiment of the present application, the morphology sensor 31 can be an optical displacement sensor, a shape memory alloy sensor, an optical bending sensor, etc., which can be integrated into the flexible portion 18 to realize morphology measurement, monitor the bending angle change and rotation change of the flexible portion 18, and use the local posture data of the base of the endoscope insertion portion 11, including position and direction, to provide more comprehensive motion information; of course, as long as it can meet the functions, size, etc., it can be any sensor.

[0087] like Fig.12, showing a system block diagram based on an auxiliary unit. The auxiliary unit 60 includes: a base posture correction unit 601, which is used to correct the base posture of the insertion unit by acquiring the base point coordinates and relative orientation of the insertion unit relative to the subject; an insertion state correction unit 602, which is used to obtain the motion state information of the insertion unit 11 to correct the insertion state of the insertion unit 11. Specifically, it can include obtaining the posture information of the base end of the insertion unit 11 close to the operating unit 14 through the base posture detection of the insertion unit and the detection of the insertion state, and the insertion information including the movement, rotation, insertion start, insertion depth, etc. of the insertion unit 11, and fusing them to obtain the insertion unit optimization morphological information, so that the system can automatically optimize and correct the displayed endoscope model when displaying, including the relative posture of the insertion unit 11 and only displaying the posture of the part of the insertion unit 11 that enters the human body, so that the doctor can more clearly observe the real-time posture of the current insertion unit 11 in the body, such as the bending and position.

[0088] In some embodiments, the base point coordinates and relative orientation of the insertion portion 11 relative to the subject can be obtained through a signal transmitter placed in the operating portion 14, and a receiving position in the space where the signal can be received during the endoscopic examination operation. The base posture of the base side of the insertion portion provides a reference point for the entire insertion portion 11 in space, which is the basis for determining the overall shape of the entire insertion portion 11.

[0089] In some embodiments, the acquisition of the motion state information of the insertion portion 11 can be achieved by a posture sensor, an electromagnetic sensor, etc., which is arranged at the base of the bending portion 16 close to the flexible portion 18. The motion state information includes, for example, the rotation information, movement information, initial posture at the beginning of insertion, insertion depth information, etc. of the insertion portion 11.

[0090] Specifically, by providing an electromagnetic sensor or posture sensor on the base of the bending portion 16, the posture changes of the insertion portion 11 during the insertion process can be effectively tracked, and the posture changes of the insertion portion 11 during the insertion process can be cumulatively calculated. By combining the initial posture and real-time posture change data, the current posture of the insertion portion 11 is continuously updated, and the posture change curve of the entire insertion process is obtained through cumulative calculation.

[0091] like Fig.13As shown, a schematic diagram of insertion status detection is shown, which includes an insertion portion through hole 23, which is a channel for the insertion portion 11 to be inserted, and the insertion portion 11 enters the lumen of the subject through the channel; a visual sensor 24, which can be placed on one side of the insertion portion through hole 23, so that the visual sensor 24 can more accurately capture the posture and position of the insertion portion 11 at the through hole 23, and calculate the movement and rotation information of the insertion portion by photographing the insertion portion, thereby obtaining the motion trajectory, speed and other motion status information of the insertion portion 11, and can determine the display starting point and display length of the insertion portion 11 before displaying.

[0092] Specifically, the visual sensor 24 can detect the displacement of the insertion portion 11 by collecting and analyzing the position changes of the insertion portion 11 in continuous image frames, for example, by using feature point matching, optical flow method, etc.; the rotation angle of the insertion portion 11 around a certain axis can be obtained by tracking the feature points on the insertion portion 11 or using a pattern recognition algorithm, and the action of the insertion portion starting to enter the field of view or start to contact the tissue in the lumen can be detected as the "insertion start" point by detecting changes in the edge of the object or the appearance of specific marks; the position changes of the insertion portion can also be monitored through image recognition and tracking algorithms based on the images collected by the visual sensor 24.

[0093] In some embodiments, the visual sensor 24 disposed on the insertion portion 11, or the visual sensor 24 disposed at the junction of the lumen outside and inside the lumen of the insertion portion 11, or the visual sensor 24 disposed and aligned with the insertion portion 11, etc., can obtain the corresponding motion state information during insertion to correct the insertion state of the insertion portion 11. Specifically, the visual sensor 24 can be implemented by using a detection camera or the like to achieve image capture.

[0094] In some embodiments, the base point coordinates and relative orientation of the insertion part 11 relative to the subject can be obtained by the second posture sensor 402, and the motion state information of the insertion part 11 can be obtained by the first posture sensor 401 to obtain the motion posture during the insertion process and accumulate the posture change curve during the entire insertion process.

[0095] According to another embodiment of the present application, the shape of the flexible portion 18 needs to take up a certain amount of computing time when acquiring and solving the posture and shape information, and the data update frequency of the second form will inevitably decrease when the shape of the flexible portion 18 changes. When the data update frequency affects the refresh rate of the shape display of the entire insertion portion 11, the system is delayed or delayed when capturing and processing the information of the shape change of the flexible portion 18, and cannot reflect the latest state of the flexible portion 18 in real time. In particular, when the computational complexity is high or the sensor data processing efficiency is not high enough, the real-time performance of the overall shape display of the insertion portion 11 will be greatly affected, causing the overall shape display of the insertion portion 11 to lag, reduce accuracy, delay control, and poor real-time observation experience, which may affect the doctor's real-time adjustment of the bending angle or position of the insertion portion 11, the insertion depth and other operations timeliness, and the deviation of the insertion process, external interference or the instability of the device itself, resulting in the shape and posture of the insertion portion 11 of the endoscope 10 deviating from the expected posture. At this time, the shape optimization such as data acquisition, posture solution, error detection and feedback control is used to achieve accurate control of the overall shape and posture of the insertion portion 11 and shape correction.

[0096] Therefore, if Fig.14 , shows a system block diagram of endoscope posture detection morphology optimization, which can estimate the shape change trend of the flexible part 18 through posture prediction, update the shape display in advance, and reduce the lag. The endoscope posture detection system 1 also includes: a predicted posture acquisition unit 70, which is used to measure the relative position information of the flexible part in each measurement area according to multiple measurement areas set along the length direction of the insertion part and / or obtain the real-time alignment posture in each measurement area to optimize the second form and obtain the predicted posture; a posture update unit 72, which is used to update the second form of the flexible part using the predicted posture during the solution process of the second form detection unit 30. The predicted posture acquisition unit 70 sets multiple measurement areas along the length direction of the insertion part 11, and these measurement areas can be equidistant or non-equidistant as needed; the collected position information and the real-time alignment posture data measured in multiple measurement areas are analyzed to determine the dynamic changes of the flexible part 18 during the insertion process to realize the update of the second form when the second form detection unit 30 solves. This method can optimize the display of the overall shape of the insertion part 11, reduce the frame rate drop caused by the solution time of the second form of the endoscope, and the morphological distortion caused by a single sensor. It can display in the best way and help doctors avoid shapes such as loops, ensure real-time transmission of more frames of images, greatly reduce problems such as frame rate drop caused by shape solution and distortion of partially reconstructed shape display, ensure the accuracy and reliability of real-time display, and ensure that accurate morphological information display can be provided in various operating environments.

[0097] like Fig.15A structural schematic diagram of morphology optimization is shown. The predicted posture acquisition unit 70 includes a real-time posture measurement unit placed in the flexible part 18. The nine-axis or six-axis gyroscopes are arranged at a certain interval. The adjacent nine-axis or six-axis gyroscopes are deflected and placed at a certain angle to capture the posture changes of the flexible part 18 in different directions. The angle deflection information of the multiple nine-axis or six-axis gyroscopes placed in a deflected manner is used to obtain the current real-time posture of the flexible part 18 as the predicted posture of the flexible part 18. The posture update unit 72 uses the predicted posture to optimize the second form of the flexible part 18; when the second form detection unit 30 is solving the information of the form sensor 31, the posture update unit 72 can use the acceleration information of the nine-axis or six-axis gyroscope to temporarily replace the second form detection unit 30 to update the real-time form information of the flexible part 18 in real time. By using the predicted posture and acceleration information for alternative updates, the accuracy of the endoscope operation and the high quality of the display are ensured, the generation frame rate of the form corresponding to the second form of the flexible portion 18 is improved, distortion is reduced, and the image delay and image display quality degradation caused by the decrease in frame rate due to the time required for acquiring and solving the second form data are eliminated, thereby enhancing the real-time and accuracy of the endoscope insertion operation.

[0098] Among them, the predicted posture acquisition unit 70 can also use additional additional posture sensors arranged at certain intervals in the flexible part 18 as a real-time posture measurement unit 701, which is used to obtain the real-time alignment posture of the flexible part 18 in the respective measurement area estimated by the second form detection unit 30, and resolve the real-time alignment posture into the ground posture of the flexible part 18 with the ground as the reference coordinate system. The ground posture resolved by the real-time alignment posture temporarily replaces the detection result of the second form detection unit 30 so as to update the real-time form information of the flexible part 18 in real time.

[0099] Furthermore, the reconstruction of the shape of the flexible portion 18 by the predicted posture acquisition unit 70 and the second form detection unit 30 involves information fusion of multiple sensors. Commonly used algorithms include Kalman filtering, neural networks, etc. For example, Kalman filtering can use the data of the predicted posture acquisition unit 70 as the prediction result and the data of the second form detection unit 30 as the real data, and substitute them into the algorithm process to achieve data fusion for better reconstruction.

[0100] According to one embodiment of the present application, the endoscope state detection system 1 also includes: a contact force estimation unit 80, which is used to estimate the magnitude and direction of the contact force applied by the insertion portion to the object based on the change in the bending angle of the bending portion 16 when the insertion portion 11 contacts the inside of the object, or the change in the bending angle of the bending portion 16 and the change in the deflection of the flexible portion 18.

[0101] The magnitude and direction of the contact force applied by the insertion portion to the subject are estimated according to the change in the bending angle of the bending portion 16 , and the contact force can be estimated by a suitable contact force model.

[0102] The contact force estimation unit 80 estimates the magnitude and direction of the contact force using the bending shape of the bending portion 16 or according to the bending angle change of the bending portion 16 and the swing angle change of the flexible portion 18. The shape of the bending portion 16 obtained by the first shape detection unit 20 and the flexible segment shape of the flexible portion 18 obtained by the second shape detection unit 30, including information or shape information, are used to estimate the force contact information generated when the subject contacts the end of the endoscope head 14 or the information necessary for calculating the contact force. Specifically, the bending portion 16 can be virtually segmented to obtain the position, direction, bending amount, etc. of each segment, and then the force conditions of the entire bending portion 16 and the flexible portion 18 are fitted to estimate the contact force. The specific estimation method will be described in detail below.

[0103] The change in the deflection angle is a relative difference between a real-time deflection angle formed between the passive deflection section 181 and the non-deflection section 182 after the flexible portion 18 contacts the object and an initial angle.

[0104] Among them, considering that when the subject tissue contacts the head end 14 of the endoscope, the flexible portion 18 is deformed under the action of the contact force, the flexible portion 18 is divided into a passive deflection segment 181 and a non-deflection segment 182 under the action of the contact force. The passive deflection segment 181 produces passive bending under the action of the contact force, and the non-deflection segment 182 refers to the normal state of the flexible portion 18 when it contacts the subject, is not affected by the contact force, maintains the original posture or angle, and does not produce passive bending.

[0105] During the insertion, diagnosis, and treatment processes performed by an insertion machine such as the endoscope 10, the insertion portion 11 and the subject move roughly slowly so that the bending portion 16 bends under the action of the contact force, the endoscope head end 14 contacts the subject, and the two end node rings 161 corresponding to the bending portion 16 are deformed. Based on the specific deformation, it is possible to estimate the torque generated by the force on each joint on the two end node rings 161 when the endoscope head end 14 contacts the subject and is subjected to a force load applied by the subject, thereby achieving contact force estimation to obtain the magnitude and direction of the contact force.

[0106] In the following description, the large intestine is used as an example of the lumen of the subject. The large intestine is an organ whose shape and configuration vary from person to person, and whose shape may change due to the passage of time, the insertion of instruments, etc. For example, there are movable intestines and fixed intestines in the large intestine. The rectum, descending colon, and ascending colon are movable intestines that can move freely, and the sigmoid colon, transverse colon, and cecum are fixed intestines.

[0107] For example, Fig.16 A diagram showing the physiological structure of the colon is shown. The colon is located in the abdominal cavity and pelvic cavity, and its overall shape is "M". It includes four parts: ascending colon, transverse colon, descending colon and sigmoid colon. There are colon bags on the colon wall. During operation, you should follow these bag-like structures to avoid excessive stretching or compression.

[0108] like Figures 17 to 19 The figure shows the morphology of the endoscope head end 14 when it contacts the tissue during the colon examination. When the endoscope is in the curved or narrow part of the colon, it is very important to better judge the appropriate force of the endoscope 10 in contact with the tissue to ensure the smooth progress of the examination process and improve the comfort and safety of the patient during the examination. When the endoscope 10 contacts the transverse colon area, it is considered to be in contact with the upper tissue of the colon, and when it contacts the ascending colon, descending colon, and sigmoid colon, it can be considered to be in contact with the lower tissue.

[0109] When not in contact with the tissue of the subject, the curved portion 16 can bend by the traction of the traction wire. In the absence of a torque generated by an external force, the curved portion 16 at the front end of the endoscope 10 can bend in all directions under the displacement traction of the traction wire 17, based on the state where no external force is applied. The bending of the curved portion 16 is due to the displacement change of the traction wire 17, which pulls the front end node ring 161 of the curved portion 17, and when it bends around the axis, it squeezes the rear end node ring, causing the curved portion 16 to bend as a whole. Therefore, the traction of the traction wire on the tip of the curved portion can be equivalent to a bending moment M acting on the tip.

[0110] like Fig. 20 As shown, the calculation method of the bending moment M can be obtained by the Bernoulli-Euler beam theory. When the front end of the endoscope 10 is deflected, its deflection angle θ0 can be calculated from the displacement of the traction wire 17 detected by the first morphological detection unit 20. E is the elastic modulus of the bending part, I is the moment of inertia, then the bending stiffness is EI, the length of the bending part is L, the horizontal coordinate of the load position acting on the beam is a, and the vertical coordinate is b:

[0111]

[0112]

[0113] According to the Bernoulli-Euler equation, the formula for the bending moment M at the corresponding position is:

[0114]

[0115] Therefore, the magnitude of the bending moment M generated by the traction wire 17 on the two end node rings 161 of the bending portion 16 when there is no contact can be derived.

[0116] Furthermore, based on the Bernoulli-Euler beam theory, a pseudo-rigid body model can be used to model and analyze the curved part, and after the modeling is completed, dynamics can be used for analysis. Fig.21 As shown, according to the mechanical property information of each node ring 161 at both ends of the bending part 16, it can be approximated as a plurality of rigid links connected by torsion spring pivots, and the length of each joint and the spring represent the deformation and stiffness of the material respectively. For example, when the bending part 16 is modeled as a pseudo-rigid body consisting of 5 rigid rods, 4 torsion springs, and a pin joint, θ i is the rotation angle of the connecting rod (i=1, 2, 3, 4), k i is the spring stiffness (i=1, 2, 3, 4). When the connecting rod rotates, the torsion spring will generate a bending moment τ i . 5 The ratio of the length of each rigid rod to the length of the beam (bend 16) L is the characteristic radius coefficient r i (i=0, 1, 2, 3, 4), and θ0 can be derived by the endoscope first form detection unit 20 according to the displacement of the traction wire 17, θ i It is calculated based on the ratio of the characteristic radius coefficient, that is, and

[0117] When the bending portion 16 is bent by the operation unit and is not in contact with the tissue of the subject, the moment balance formula can be obtained:

[0118]

[0119] like Fig. 22 As shown, during the endoscopic examination, when the tip 14 contacts the subject, the force applied will cause the joint angle of the bending portion 16 to be different from the angle when there is no contact. When the bending portion 16 contacts the subject, the force F applied by the subject to the bending portion 16 changes the bending angle of each joint, and the angle between the force F and the horizontal direction is φ, φ∈[0°,180°]. According to the moment balance formula, when the tip 14 of the endoscope contacts the subject, there is a formula:

[0120]

[0121] [J T ] is the transpose of the Jacobian matrix of the dimensionless form of the kinematic equations:

[0122]

[0123] However, since the flexible section of the endoscope is bendable and deformable as a whole, when the tip of the bending section contacts the object, there will be no obvious angle change, but the entire flexible section will deflect under the action of force F. Therefore, the estimation of the flexible section 18 is added to the pseudo-rigid body model of the bending section 16, so that the front end contact force can be predicted. Fig.18 As shown, a passive deflection section 181 and a non-deflection section 182 are added to the original pseudo-rigid body model of the bending part, θ1 is the deflection angle of the passive deflection section 181 relative to the non-deflection section 182, and this angle can be measured by the second morphology detection unit 30; other angles θ i (i=2,3,4,5) can be obtained from the displacement of the traction wire and the characteristic radius coefficient r i (i=2,3,4,5) is obtained. The other analysis processes are the same as above, and the force balance formula at this time is:

[0124]

[0125] The transpose of the Jacobian matrix at this time is:

[0126]

[0127] After obtaining the specific angle value, [J T ] to solve the contact force F and bending moment M. The formula is:

[0128]

[0129] When the endoscope 10 enters the subject and contacts the tissue, Figure 23 to 2 As shown in FIG. 6 , during the contact between the head end portion 14 and the object, the flexible portion 18, under the action of the contact force, will convert the action of this part of the force into a deflection angle change θ1 of the flexible portion 18, and the sign of θ1 can be determined by the change direction. θ1 is obtained by the second form detection unit 30, θ i (i=2,3,4,5) is determined by the wire displacement and characteristic radius coefficient r i (i=2,3,4,5) is obtained, and the contact force F and the bending moment M of the traction wire can be calculated by substituting them into the above formula. At the same time, the angle of the force relative to the original horizontal state can be calculated by the following formula:

[0130]

[0131] For example, Fig.23 and Fig.24As shown, when the insertion portion 11 of the endoscope 10 is inserted into the descending colon segment, the bending portion 16 comes into contact with the intestinal wall when bending. Assuming that the bending angle of the endoscope to the left is positive and to the right is negative, since the head portion 14 will deform the intestinal wall and wrap a part of the head portion 14 after coming into contact with the intestinal wall, the angle of the contact force F generated by the subject tissue relative to the horizontal direction is θ1 is obtained by the second form sensor, θ i (i=2,3,4,5) is determined by the wire displacement and characteristic radius coefficient r i (i=2,3,4,5) is obtained and substituted into the above formula to calculate the size and direction of the contact force F and the bending moment M.

[0132] For example, Fig.25a , Fig.25b and Fig.26a , Figure 26b As shown, since the entire insertion portion of the endoscope 10 is soft and bendable, it may not be in an undeflected state. Therefore, the first form detected by the first form detection unit 20 or the displacement change information of the traction wire 17 can be combined to obtain the relative deflection angle θ1 of the passive deflection section 181 of the flexible portion 18. The specific method is: when the traction wire 17 is displaced and the front end bending portion 16 is bent, if at this time, the angle θ1 between the passive deflection section 181 and the non-deflection section 182 changes, it is determined that the end of the endoscope head 14 is in contact with the tissue of the subject. At this time, the difference between the real-time deflection angle θ1' of the passive deflection section 181 of the flexible portion 18 and the non-deflection section 182 of the flexible portion 18 after contact and the initial angle of the passive deflection section 181 and the non-deflection section 182 of the flexible portion 18 at the initial moment (when the end of the endoscope head 14 is not in contact with the subject) of the flexible portion 18 can be calculated and used as θ1 in the formula. The acquisition of other angles is the same as described above. Substituting them into the above formulas, we can obtain the contact force and bending moment M.

[0133]

[0134] Since the endoscope head will be wrapped by human tissue when it contacts with tissue, in order to simplify the calculation, it can be assumed that after the endoscope head contacts with human tissue, the force exerted by human tissue on the endoscope head is at a 90-degree angle to the direction of the endoscope head. In order to distinguish the direction, the angle between the force and the horizontal direction can be defined as The angle symbols are the same as defined above and will not be repeated here.

[0135] Furthermore, for the same end position, under the action of force, due to the characteristic radius coefficient r of the curved portion 16 i When different, the pseudo-rigid body angle θ i are also different, then the spring stiffness coefficient k iwill also be different. Through experiments and using algorithms such as ant colony algorithm, the optimal characteristic radius coefficient r can be solved. i , thereby improving the prediction accuracy of calculating the contact force between the endoscope head end and the tissue.

[0136] By estimating the first form and the second form, the contact force generated when the subject's tissue contacts the tip of the endoscope can be calculated, thereby avoiding excessive pressure on the tissue, reducing damage to surrounding tissue during surgery, and improving the safety of the surgery; further, during endoscopic surgery, based on the bending angle change rate and / or the yaw change rate of the insertion portion at different time points, the changing trend of the contact force can also be further accurately estimated, thereby adjusting the movement path of the insertion portion to reduce damage to the subject, and accurate contact force estimation helps doctors control surgical instruments more accurately, especially when performing delicate operations such as cutting, suturing or tissue processing, thereby improving the accuracy of the surgery.

[0137] In some implementations of the application, the display unit is used to display the overall shape of the endoscope insertion portion obtained by integrating the first form with the second form; and / or display the corrected form after the auxiliary unit 60 uses the information of the auxiliary information acquisition unit to assist in correcting the form of the endoscope insertion portion, and / or display the estimated result of the contact force estimated by the contact force estimation unit. The overall form of the insertion portion, the corrected form, and the force conditions at different parts can be intuitively displayed in a visual manner. For example, the size and direction of the contact force can also be indicated by color, arrows or other graphic elements during display. The operator can more clearly perceive the interaction between the endoscope and the surrounding tissues, so that the operating skills and strength can be flexibly adjusted according to the actual situation. This not only improves the safety of the operation, but also reduces the risks during surgery and is more conducive to flexible adjustment of the operating skills and strength.

[0138] like Fig. 27 As shown, the second aspect of the present application provides a working method of an endoscope state detection system, which is applied to the aforementioned endoscope state detection system, and includes step S100: detecting the displacement change information of the traction wire in the insertion part, and estimating the first form of the bending part according to the displacement change information; step S102: detecting the morphological data of the flexible part in the current form, and estimating the second form according to the morphological data; step S104: morphologically integrating and aligning the first form and the second form to obtain the shape data of the insertion part, and the shape data is used to identify the overall shape of the insertion part inserted into the lumen; step S106: generating a display screen according to the shape data synthesized by the integration unit.

[0139] Furthermore, the working method of the detection system also includes: estimating the magnitude and direction of the contact force between the insertion portion and the subject according to the change in the bending angle of the bending portion and / or the change in the deflection of the flexible portion when the insertion portion contacts the subject. Specifically, by comparing the change in the displacement of the traction wire 17 / the bending angle of the bending portion 16 when the endoscope 10 is not in contact with the subject and when the endoscope is in contact with the subject, the magnitude and direction of the contact force between the insertion portion and the subject is estimated by the bending change of the bending portion.

[0140] Specifically, based on the change in the bending angle of the bending portion 16 and the displacement / deflection angle relationship of the passive deflection section 181 of the flexible section relative to the non-deflection section 182 of the flexible section when the subject tissue contacts the head end 14 of the endoscope 10, the torque generated by the fused bending portion 2 and the flexible section 18 is used to estimate the magnitude and direction of the contact force between the insertion portion and the subject.

[0141] According to one embodiment of the present application, before the integration unit 40 displays through the display unit 90, it also includes: obtaining the base posture and insertion state of the insertion part to assist in correcting the overall shape of the insertion part. Specifically, the base point coordinates and relative position orientation of the insertion part relative to the subject are obtained by a second posture sensor to correct the base posture of the insertion part; the movement state information of the insertion part is obtained by a visual sensor set on the insertion part or a visual sensor set at the junction of the insertion part outside the lumen and inside the lumen, or a visual sensor set to align with the insertion part, to correct the insertion state of the insertion part.

[0142] According to one embodiment of the present application, in step S106, the specific display screen includes: displaying the overall shape of the endoscope insertion portion obtained by integrating the first form and the second form; and / or displaying the auxiliary correction result after correcting the endoscope form using the information of the auxiliary information acquisition unit, and / or displaying the contact force estimation result.

[0143] In this way, when performing endoscopic diagnosis and treatment, the doctor can clearly know the shape of the endoscope in the human body, and can effectively avoid or deal with the situation of the endoscope insertion part getting tangled in the human body. At the same time, the doctor can also better grasp the contact force information between the head end of the endoscope and the tissue based on the force information of the bending part and / or the flexible part, so that the insertion and withdrawal can be carried out quickly, thereby reducing the risk of the doctor operating the endoscope, the experience requirement of the doctor, and the discomfort of the patient during the operation.

[0144] The present application is not limited to the above-mentioned embodiments, and various modifications can be made in the implementation stage without departing from the scope of its main purpose. Each embodiment can be appropriately combined and implemented within the possible scope, in which case the effect of the combination can be obtained. Furthermore, the above-mentioned embodiments include inventions at various stages, and various inventions can be refined by appropriate combinations of the disclosed multiple constituent elements.

Claims

1. An endoscope state detection system, the endoscope comprising an operating unit, characterized in that: include: an insertion portion, comprising a bending portion and a flexible portion, wherein the bending portion is bent and operated by operating the operating portion; a first shape detection unit, configured to detect displacement change information of the traction wire in the insertion portion, and estimate the first shape of the bending portion according to the displacement change information; a second form detection unit, configured to detect form data of the flexible portion in a current form, and estimate a second form of the flexible portion according to the form data; an integration unit configured to perform shape integration and alignment on the first form and the second form to obtain shape data of the insertion portion, wherein the shape data is used to identify an overall shape of the insertion portion inserted into the lumen; The display unit is used to generate a display screen according to the shape data synthesized by the integration unit.

2. The endoscope status detection system according to claim 1, characterized in that: The first morphology detection unit comprises: A displacement detection unit, used to detect the displacement contact amount generated by the displacement change of the traction wire; The posture calculation unit is used to calculate the first shape of the bending portion according to the displacement contact amount and a preset analytical model.

3. The endoscope status detection system according to claim 1, characterized in that: The second morphology detection unit includes a morphology sensor arranged in the flexible portion along the length direction of the insertion portion, and the morphology sensor is used to implement distributed multi-point measurement or continuous measurement.

4. The endoscope status detection system according to claim 1, characterized in that: A first posture sensor is provided on one side of the bending portion close to the flexible portion, and is arranged parallel to the bending axis direction of the bending portion, and is used to obtain a first alignment posture corresponding to the first form; A second posture sensor is provided inside the operating portion of the endoscope and / or outside the endoscope, for acquiring a second alignment posture corresponding to the second form; The integration unit uses the first alignment posture and the second alignment posture to splice the first form and the second form to obtain shape data of the insertion portion.

5. The endoscope status detection system according to claim 1, characterized in that: The system further comprises: An auxiliary unit acquires the base posture and insertion state of the insertion portion to assist in correcting the display of the overall shape of the insertion portion.

6. The endoscope status detection system according to claim 5, characterized in that: The auxiliary unit comprises: A base posture correction unit, used for obtaining the base point coordinates and relative position orientation of the insertion unit relative to the subject, so as to correct the base posture of the insertion unit; The insertion state correction unit is used to obtain the motion state information of the insertion unit to correct the insertion state of the insertion unit.

7. The endoscope status detection system according to claim 1, characterized in that: The system further comprises: a predicted posture acquisition unit, configured to measure the relative position information of the flexible portion in each measurement area and / or the real-time alignment posture in each measurement area according to the real-time posture measurement units arranged in the plurality of measurement areas along the length direction of the insertion portion, so as to obtain the predicted posture of the second form; A posture updating unit is used to update the second shape of the flexible part using the predicted posture during the calculation process of the second shape detection unit.

8. The endoscope status detection system according to claim 1, characterized in that: The system further comprises: The contact force estimation unit is used to estimate the size and direction of the contact force applied by the insertion part to the object according to the change of the bending angle of the bending part and the change of the deflection of the flexible part when the insertion part contacts the inside of the object.

9. An endoscope state detection method, applied to the endoscope state detection system according to any one of claims 1 to 8, characterized in that: include: Detecting displacement change information of the traction wire in the insertion portion, and estimating the first shape of the bending portion according to the displacement change information; detecting shape data of the flexible portion in a current shape, and estimating a second shape of the flexible portion based on the shape data; Integrating and aligning the first form and the second form to obtain shape data of the insertion portion, wherein the shape data is used to identify the overall shape of the insertion portion inserted into the lumen; A display screen is generated based on the shape data synthesized by the integration unit.

10. The endoscope state detection method according to claim 9, characterized in that: The method further comprises: estimating the magnitude and direction of the contact force according to the change in the bending angle of the bending portion and the change in the yaw angle of the flexible portion; The change in the deflection angle is a relative difference between a real-time deflection angle formed between a passive deflection section and a non-deflection section after the flexible portion contacts the object and an initial angle.

Images