Real-time pipeline shape sensing module and endoscope shape sensing method

Abstract

The invention discloses a real-time sensing module of a pipeline shape, which comprises a guide wire and a control module, the guide wire comprises an elastic rod and a sensor array, the sensor array is arranged in the elastic rod and is arranged along the central axis direction of the elastic rod, and each group of sensor array comprises a plurality of sensors for measuring shape strain. And the control module is in communication connection with the sensor and is used for reconstructing the three-dimensional shape of the guide wire according to the shape strain information measured by the sensor. When the real-time sensing module is applied to the endoscope, positioning errors caused by cavity deformation in the endoscope examination and treatment process can be effectively compensated, and the positioning precision and the operation safety are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscopes, and in particular to a real-time sensing module for a pipeline shape and a sensing method for an endoscope shape. Background Art

[0002] Endoscopic technology has become an important means of clinical minimally invasive diagnosis and treatment, which can effectively reduce patient damage, improve diagnostic and treatment efficiency, and accelerate postoperative recovery. However, due to the complex structure of human cavities such as bronchi, urethra, and intestines, doctors cannot grasp the shape of the endoscope inside the human body in real time during the operation of the endoscope. They can only rely on personal experience to operate the endoscope, perform inspections, and treat, which is prone to problems such as entering the wrong branch and endoscopic loops.

[0003] Shape sensing technology can provide information about the shape of the endoscope inside the human body, helping doctors to operate the endoscope more accurately to complete diagnostic and treatment operations. Although the optical fiber-based shape sensing method can sense the shape information of the endoscope in real time, it has disadvantages such as expensive equipment, the need for additional registration methods to establish a spatial relationship with the patient, and inconvenience in use. The shape sensing method based on magnetic positioning is extremely susceptible to interference, and multiple position and posture sensors need to be buried at different positions of the shape sensing guidewire body to reconstruct the shape of the guidewire, which will inevitably introduce more wires, posing great challenges to the miniaturized design and processing and assembly of the shape sensing guidewire. Summary of the invention

[0004] In view of some or all of the problems in the prior art, the present invention provides a real-time sensing module for pipeline shape in a first aspect, comprising:

[0005] A guidewire comprising an elastic rod and a plurality of sensor arrays, wherein the sensor arrays are disposed in the elastic rod and arranged along a central axis of the elastic rod, wherein each sensor array comprises a plurality of sensors, and the sensors are configured to measure shape strain; and

[0006] A control module is communicatively connected to the sensor and is used to reconstruct the position and three-dimensional shape of the guide wire according to the shape strain information measured by the sensor.

[0007] Furthermore, the plurality of groups of sensor arrays are evenly distributed.

[0008] Furthermore, the spacing between two adjacent groups of sensor arrays is determined according to the minimum bending radius of the elastic rod axis.

[0009] Further, each sensor array includes at least three sensors.

[0010] Furthermore, the at least three sensors are evenly arranged circumferentially within the cross section of the elastic rod.

[0011] Furthermore, the sensor is a stretchable flexible sensor (Stretchable Sensor).

[0012] Furthermore, the control module includes:

[0013] A decoder, which is connected to each sensor through a wire, and is used to decode the shape strain information measured by the sensor and upload it to the reconstruction submodule; and

[0014] The reconstruction submodule is communicatively connected to the decoder and is used to perform shape reconstruction according to the decoding information of the decoder.

[0015] Furthermore, the reconstruction submodule includes an industrial computer.

[0016] A second aspect of the present invention provides a method for sensing the shape of an endoscope, comprising:

[0017] Inserting an endoscope into a human body cavity and controlling the endoscope to move in the human body cavity, wherein a real-time sensing module as described above is arranged in a working channel of the endoscope;

[0018] Obtaining shape strain information measured by each sensor;

[0019] reconstructing the shape of the guidewire based on the shape strain information;

[0020] determining the shape of the working channel of the endoscope based on the shape of the guidewire; and

[0021] In combination with a preset three-dimensional model of the endoscope, the shape of the endoscope is determined according to the shape of the working channel.

[0022] Furthermore, a reconstruction method based on the Frenet-Serret equation is used to reconstruct the shape of the working channel of the endoscope based on the shape strain information.

[0023] Furthermore, the reconstruction method comprises:

[0024] Based on the shape strain information measured by each sensor, the discrete curvature and bending direction of each group of sensor arrays are determined;

[0025] Fitting the explicit function of the discrete curvature and bending direction with respect to the length distance by an interpolation method;

[0026] Differentiate the explicit function of the bending direction with respect to the length distance with respect to the length to obtain the explicit torsion function;

[0027] Determining the unit tangent vector at each location based on an explicit function of the discrete curvature with respect to the length distance and an explicit torsion function; and

[0028] Based on the unit tangent vector, the position coordinates of each point are determined to obtain the three-dimensional shape of the guide wire. The shape of the endoscope working channel is the same as the three-dimensional shape of the guide wire.

[0029] Furthermore, a data-driven deep learning method is used to reconstruct the shape of the working channel of the endoscope based on the shape strain information.

[0030] Furthermore, the deep learning method comprises:

[0031] The shape strain information measured by each sensor is input into the trained deep learning model to obtain the three-dimensional shape of the endoscope working channel.

[0032] Furthermore, the training of the deep learning model includes:

[0033] Two square calibration plates are vertically combined to form a camera imaging space, where black metal balls are evenly arranged on the square calibration plates for calibrating the two-dimensional camera.

[0034] The guide wire is set to different shapes and positions within the camera imaging field of view, and the horizontal plane and the vertical plane are photographed respectively by two cameras, and the shape strain information of each sensor array in the guide wire is obtained by a control module;

[0035] Extracting the guidewire based on the image captured by the camera, reconstructing the complete guidewire shape, and constructing a data set of the guidewire in different positions and shapes; and

[0036] The data set is divided into training samples and verification samples to train the deep learning model to obtain model parameters that meet the set iteration requirements and construct a deep learning model.

[0037] Furthermore, the deep learning model adopts BP neural network or convolutional neural network.

[0038] Furthermore, the shape sensing method further includes:

[0039] In combination with a preset human cavity model, the position of the endoscope in the human body is determined according to the shape of the endoscope.

[0040] Furthermore, the shape sensing method further includes:

[0041] The position and shape of the endoscope are displayed by a display device.

[0042] Furthermore, the shape sensing method further includes:

[0043] The position and shape of the endoscope are synchronously displayed with the human body cavity model through a display device.

[0044] The present invention provides a real-time sensing module for the shape of a pipe. The module is based on an elastic rod, and multiple sensor arrays capable of measuring shape strain are embedded at different positions of the elastic rod to construct a guidewire that can be placed in the working channel of an endoscope. The curvature information of different positions of the elastic rod is calculated in real time by measuring the strain of the sensor. The complete three-dimensional shape of the sensing guidewire is reconstructed in real time by combining the geometric parameters of the elastic rod and the position information of the sensor array using a numerical calculation method. The real-time sensing module is applied to the working channel of an endoscope, which can effectively compensate for the positioning error introduced by the deformation of the cavity during endoscopic examination and treatment, and improve the positioning accuracy and operation safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] To further illustrate the above and other advantages and features of various embodiments of the present invention, a more specific description of various embodiments of the present invention will be presented with reference to the accompanying drawings. It will be understood that these drawings only depict typical embodiments of the present invention and are therefore not to be considered as limiting the scope thereof. In the accompanying drawings, for clarity, identical or corresponding parts will be represented by identical or similar reference numerals.

[0046] Figure 1 A schematic structural diagram showing a real-time sensing module for pipeline shape according to an embodiment of the present invention;

[0047] Figure 2 A schematic diagram showing the arrangement of a sensor array of a guidewire according to an embodiment of the present invention;

[0048] Figure 3 A schematic diagram showing the arrangement of sensors in a sensor array according to an embodiment of the present invention;

[0049] Figure 4 A schematic diagram showing the bending deformation of a guide wire according to an embodiment of the present invention is shown;

[0050] Figure 5 A schematic structural diagram showing an endoscope capable of sensing the shape of a working channel in real time according to an embodiment of the present invention;

[0051] Figure 6 A schematic diagram showing the use of an endoscope capable of sensing the shape of a working channel in real time according to one embodiment of the present invention;

[0052] Figure 7 A schematic flow chart showing a method for sensing the shape of an endoscope according to an embodiment of the present invention;

[0053] Figure 8 A schematic diagram showing the principle of reconstructing the three-dimensional shape of a guide wire based on the Frenet-Serret equation according to an embodiment of the present invention;

[0054] Fig. 9A schematic diagram showing a process of a sensing guidewire curve fitting method based on data-driven deep learning according to an embodiment of the present invention; and

[0055] Fig.10 A schematic structural diagram of a guidewire shape sensing calibration platform based on a two-dimensional high-precision camera according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0056] In the following description, the present invention is described with reference to various embodiments. However, those skilled in the art will recognize that various embodiments can be implemented without one or more specific details or with other replacement and / or additional methods, materials or components. In other cases, well-known structures, materials or operations are not shown or described in detail to avoid blurring the inventive point of the present invention. Similarly, for the purpose of explanation, specific quantities, materials and configurations are set forth to provide a comprehensive understanding of embodiments of the present invention. However, the present invention is not limited to these specific details. In addition, it should be understood that the various embodiments shown in the drawings are illustrative representations and are not necessarily drawn in correct proportions.

[0057] In this specification, reference to "one embodiment" or "the embodiment" means that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present invention. The phrase "in one embodiment" appearing in various places in this specification does not necessarily all refer to the same embodiment.

[0058] It should be noted that the embodiments of the present invention describe the method steps in a specific order, but this is only for the purpose of illustrating the specific embodiment, rather than limiting the order of the steps. On the contrary, in different embodiments of the present invention, the order of the steps can be adjusted according to actual needs.

[0059] In order to conveniently realize the real-time position tracking and shape sensing of the flexible endoscope, the present invention provides a guide wire that can be placed in the working channel of the flexible endoscope, which is based on an elastic rod and is constructed by embedding multiple groups of retractable flexible sensor arrays at different positions of the elastic rod. The curvature information of different positions of the elastic rod is calculated in real time by measuring the strain of the retractable flexible sensor, and the complete three-dimensional shape of the guide wire is reconstructed in real time by combining the geometric parameters of the elastic rod and the position information of the sensor array, thereby effectively compensating for the positioning error introduced by the deformation of the cavity during endoscopic examination and treatment, and improving the positioning accuracy and operation safety.

[0060] The technical solution of the present invention is further described below in conjunction with the accompanying drawings of the embodiments.

[0061] Figure 1 FIG. 2 is a schematic diagram showing the structure of a real-time sensing module for pipeline shape according to an embodiment of the present invention. Figure 1As shown, a real-time sensing module for pipeline shape includes a guide wire and a control module 103, wherein the guide wire includes an elastic rod 101 and a sensor array 102. The sensor array 102 is arranged in the elastic rod 101 and arranged along the central axis direction of the elastic rod 101, wherein each group of sensor arrays includes a plurality of sensors, and the sensors can be used to measure shape strain in real time. The control module 103 is communicatively connected to the sensor, and is used to reconstruct the position and three-dimensional shape of the guide wire according to the shape strain information measured by the sensor.

[0062] When the guide wire is applied to a flexible endoscope, it is preferred to select a guide wire with an outer diameter that matches the inner diameter of the working channel of the flexible endoscope, insert it into the working channel of the endoscope, and its distal end is flush with the end of the hard part of the front end of the endoscope. When the endoscope body is operated to travel in the human body cavity, the bending part of the endoscope can be controlled by the endoscope control end to achieve advance and retreat and steering in the human body cavity by bending at different angles, and the flexible part of the endoscope is constrained by the shape of the human body cavity structure and deforms accordingly under the driving force of the interaction between the two. Since the guide wire fits well with the endoscope and changes in shape accordingly, the position and shape information of the guide wire can be directly used to represent the position and shape of the flexible endoscope. In one embodiment of the present invention, the elastic rod can be made of plastic materials such as PFA, PEEK, TEFE, PTFE, etc. It should be understood that in other embodiments of the present invention, the elastic rod can also be made of other materials with better elasticity.

[0063] like Figure 1 As shown, in one embodiment of the present invention, multiple sensor arrays are arranged along the central axis direction of the elastic rod 101. In order to adapt to the working channel of the endoscope, in one embodiment of the present invention, there are certain differences in the arrangement of the sensor arrays at the curved portion and the flexible portion corresponding to the working channel of the endoscope. Specifically, Figure 1 As shown, at the distal end point corresponding to the endoscope working channel, at the connection between the curved portion and the flexible portion, that is, at both ends of the curved portion, a group of sensor arrays are arranged respectively, and at the flexible portion corresponding to the endoscope working channel, multiple groups of sensor arrays are arranged, wherein the length of the curved portion is recorded as L C , the length of the flexible portion is L S , wherein L S >L C . Figure 2 A schematic diagram showing the arrangement of a sensor array of a guide wire capable of real-time shape sensing according to an embodiment of the present invention is shown. Figure 2 As shown, in one embodiment of the present invention, the sensor arrays of the flexible portion are evenly distributed, that is, at a certain interval L Δ Arrange a set of sensor arrays. In one embodiment of the present invention, L ΔAn appropriate value is selected according to the minimum bending radius of the elastic rod to ensure the reconstruction accuracy. It should be understood that in other embodiments of the present invention, the spacing between any two adjacent sensor arrays may also be unequal.

[0064] As mentioned above, each sensor array includes at least two sensors, but adding sensors can effectively improve the accuracy of the results and reduce errors. Therefore, in one embodiment of the present invention, each sensor array preferably includes at least three sensors. Each sensor array is numbered S i =(S i,1 ,…S i,j ,…,S i,J ), i = 0, 1, ..., n; j = 1, 2, ..., J, and J ≥ 3, where n is the total number of sensor arrays, i is the sensor array number, and j is the sensor unit number in each array. Figure 3 A schematic diagram showing the arrangement of sensors in a sensor array of a guidewire capable of sensing shape in real time according to an embodiment of the present invention is shown. Figure 3 As shown, in one embodiment of the present invention, the sensors in the same sensor array are evenly arranged in the circumferential direction within the cross section of the elastic rod, that is, the center point of each sensor is located on the cross section of the elastic rod, and the sensors are arranged around the center of the cross section at a preset angle θ. j ,j=1,2,…,J;J≥3. The distance from the center of each sensor to the center of the cross section is recorded as r j ,j=1,2,…,J;J≥3.

[0065] Based on the structure described above, Figure 4 A schematic diagram of bending deformation of a guide wire capable of real-time shape sensing according to an embodiment of the present invention is shown as follows: Figure 4 As shown, when the guide wire is bent and deformed, the sensors not on the bending neutral plane N will produce strain responses, wherein the sensors inside the neutral plane produce compression responses, and the sensors outside the neutral plane produce tension responses. The plane where the guidewire is bent is perpendicular to the neutral plane N, and the angle between it and the x-axis of the local reference coordinate system is θ b At this time, the distance from each sensor to the neutral plane is recorded as D j ,j=1,2,…,J;J≥3.

[0066] In one embodiment of the present invention, the sensor is a stretchable flexible sensor (StretchableSensor), which can detect the shape strain of a specified position of the elastic rod in real time.

[0067] like Figure 1As shown, in one embodiment of the present invention, the control module 103 mainly includes a decoder 131 and a reconstruction submodule 132, wherein the decoder 131 is connected to each sensor through a wire 104, and is used to decode the shape strain information measured by the sensor and upload it to the reconstruction submodule 132, and the reconstruction submodule 132 is communicatively connected to the decoder 131, and is used to reconstruct the shape according to the decoded information of the decoder. In one embodiment of the present invention, for example, an industrial computer can be used as a reconstruction submodule, and the decoder parses the sensor signal into an industrial computer readable signal, such as time t, sensor array S i The strain information is ε i,t The industrial computer is provided with software for converting the strain information into three-dimensional shape information of the guide wire, and the software can be implemented based on a curve reconstruction algorithm, for example, a reconstruction method based on the Frenet-Serret equation, a data-driven deep learning method, etc.

[0068] As mentioned above, the guide wire of the real-time sensing module can be set in a flexible endoscope, specifically, can be placed in the working channel of the endoscope, or built into the interior of the endoscope, or built into other instruments that use the working channel of the endoscope. Figure 5 FIG. 1 is a schematic diagram showing the structure of an endoscope capable of sensing the shape of a working channel in real time according to an embodiment of the present invention. Figure 5 As shown, the endoscope includes a control end 501 and a working channel 502. When in operation, the guide wire 503 is arranged in the working channel 502 of the endoscope, and the outer surface of the guide wire 503 is attached to the inner surface of the working channel 502, and the distal end of the guide wire 503 is flush with the end point of the hard part of the front end 502 of the working channel. Figure 6 FIG. 1 is a schematic diagram showing the use of an endoscope capable of sensing the shape of a working channel in real time according to an embodiment of the present invention. Figure 6 As shown, the endoscope can be used for inspection and treatment of human cavities such as bronchi, urethra, and intestines. When the endoscope is operated to move in the human cavity, the guide wire deforms accordingly with the deformation of the endoscope working channel. Since the sensing guide wire fits well with the endoscope working channel, the position and shape information of the guide wire can be used to indicate the position and shape of the endoscope working channel.

[0069] Figure 7 FIG. 2 is a flow chart showing a method for sensing the shape of an endoscope according to an embodiment of the present invention. Figure 7 As shown, a method for sensing the shape of an endoscope includes:

[0070] First, in step 701, the system is initialized. First, a real-time sensing module that matches the inner diameter of the working channel of the endoscope is selected, and the guide wire of the real-time sensing module is inserted into the working channel of the endoscope. Then, the endoscope is operated to be inserted into a human cavity, such as the trachea, intestine, urethra, etc., and the shape sensing system is registered with the patient, thereby unifying the coordinate space of the real-time sensing module and the patient's cavity model. Here, the real-time sensing module that matches the inner diameter of the working channel of the endoscope means that after insertion, the outer surface of the guide wire of the real-time sensing module can be attached to the inner surface of the working channel of the endoscope. In order to more intuitively display the real-time position and shape of the endoscope, in one embodiment of the present invention, the patient's personalized cavity three-dimensional model reconstructed from the patient's CT or other images, and the three-dimensional model of the endoscope can also be imported into the industrial computer of the control center to facilitate subsequent positioning and display;

[0071] Next, in step 702, the strain is measured. When the endoscope is operated to move forward, backward, turn, etc. in the human body cavity, the shape strain information measured by each sensor in the guide wire is tracked in real time by the control module;

[0072] Next, in step 703, the shape of the guidewire is reconstructed. The shape of the guidewire is reconstructed based on the shape strain information. As mentioned above, different reconstruction methods can be used to achieve shape reconstruction.

[0073] In one embodiment of the present invention, since the distal end of the guidewire may be subjected to different loads and produce different degrees of deformation, the constant curvature assumption is no longer valid. Therefore, the curvature can be measured at different discrete positions along the guidewire for dense sampling, and the shape of the guidewire curve can be reconstructed based on the Frenet-Serret equation. Specifically, at time t, the sensor array S i =(S i,1 ,…S i,j ,…,S i,J ) of the jth sensor strain ε i-j，t The curvature κ of the elastic rod i,t The relationship is:

[0074]

[0075] For simplicity, the time symbol t is omitted. For a sensor array containing three sensors, the strain-curvature equations of the three sensors are:

[0076]

[0077] Among them, r i-j ,θ i-j is a known design parameter and can be calibrated. The strain rate ε i-jMeasured directly by each sensor. By explicitly defining the local apparent curvature vector The closed-form analytical solution of the above equations can be obtained:

[0078]

[0079] where and are the unit vectors of the x and y axes in the cross-sectional coordinate system respectively. Based on this, the curvature and bending direction are calculated for all sensor arrays, and a set of discrete curvatures briefly denoted as κ i and the bending direction

[0080] Let l represent the length distance along the elastic rod and 0 < l < L. The length position where each sensor array is located is l i , i = 0, 1, …, n. According to the corresponding discrete curvature and bending direction The explicit functions κ(l) and θ b (l) of the curvature and bending direction with respect to the length distance l are fitted by the interpolation method. Differentiate the bending direction function θ b (l) with respect to the length l to obtain the explicit torsion function:

[0081] τ(l) = θ' b (l);

[0082] The mathematical basis for the three-dimensional configuration reconstruction of the guide wire is the curve theory in differential geometry. Generally, the shape and position parameters of the shape-sensing guide wire can be obtained by solving the Frenet-Serret equations. In a three-dimensional Euclidean space, the three-dimensional shape of the shape-sensing guide wire is represented by a curve. According to the fundamental theorem of curve theory, using the three geometric invariants of curve arc length, curvature, and torsion, the three-dimensional shape of the curve can be characterized in three-dimensional space. Figure 8 The schematic diagram of the principle of reconstructing the three-dimensional shape of the guide wire based on the Frenet-Serret equations according to an embodiment of the present invention is shown, as Figure 8 shown. A Frenet coordinate frame is set at each point on the curve. When the point on the curve moves, the Frenet coordinate frame moves synchronously, and the motion parameters of the Frenet coordinate frame are used to reconstruct the three-dimensional shape of the curve. The specific process is as follows:

[0083] First, in the Frenet coordinate frame, the parametric equation of the regular curve C is used to represent the three-dimensional shape of the guide wire. The direction vector at any point l on the guide wire is which is also the unit tangent vector of the curve at point l. The principal normal vector at point l is Where κ(l)≠0. The subnormal vector at point l is Therefore, any point on the guide wire has a completely determined unit orthogonal coordinate frame, namely the Frenet coordinate frame

[0084] Next, in the Frenet coordinate system, the curvature at any point l on the guide wire can be expressed as The torsion can be expressed as Based on the curvature κ(l) and torsion τ(l) of the guidewire, the unit tangent vector at each location can be obtained based on the following formula:

[0085] as well as

[0086] Finally, according to the following formula, the position coordinates {x(l), y(l), z(l)} of each point of r(l) can be solved to reconstruct the complete three-dimensional shape and position of the sensing guidewire:

[0087]

[0088] in, is the initial position of a point on the sensing guidewire, and Δl is the distance between two adjacent points.

[0089] In yet another embodiment of the present invention, a data-driven deep learning method is used to reconstruct the shape of the guidewire based on the shape strain information. Fig. 9 A schematic diagram of a process of a sensing guidewire curve fitting method based on data-driven deep learning according to an embodiment of the present invention is shown. Fig. 9 As shown in the figure, based on the data-driven deep learning method, we first need to build a high-precision 2D camera design guidewire shape sensing calibration platform. Fig.10 FIG. 1 is a schematic diagram showing a structure of a guidewire shape sensing calibration platform based on a two-dimensional high-precision camera according to an embodiment of the present invention. Fig.10 As shown in FIG. 1 , two square calibration plates are vertically combined into a camera imaging space. Black metal balls are evenly arranged on the calibration plates for calibrating the two-dimensional camera. Next, the guide wire is set to different shapes and positions within the camera imaging field of view. The two cameras respectively shoot the horizontal plane (camera B) and the vertical plane (camera A). At the same time, the real-time strain information ε of each sensor array in the guide wire is tracked in real time through the control module. i,t The guidewire structure is extracted from the images of cameras A and B by image segmentation and other methods, and the complete guidewire shape C is reconstructed. t Based on this, a dataset of different guidewire positions and shapes {ε i,t :C t}, where i = 0, 1, ..., n; t = 1, 2, ..., T, that is, each sensor has n sensor arrays and collects T data pairs with different positions and postures. Finally, the sensor position and shape data obtained by the calibration platform are divided into training samples and verification samples, which are used to train the deep learning model to obtain model parameters that meet the set iteration requirements, thereby constructing a data-driven deep learning curve fitting model. In one embodiment of the present invention, the deep learning model can adopt a BP neural network, a convolutional neural network or other structures;

[0090] Next, at step 704, the shape of the working channel is determined. Since the guide wire fits tightly with the endoscope working channel, the shape of the guide wire can be used to represent the shape of the endoscope working channel; and

[0091] Next, in step 705, the shape and position of the endoscope are determined. In combination with the relationship between the working channel and the overall endoscope body in the three-dimensional model of the endoscope, the shape of the endoscope can be determined according to the shape of the working channel. In one embodiment of the present invention, since the coordinate space of the real-time sensing module and the patient cavity model has been unified in the initialization stage, the position of the endoscope in the human body can be further confirmed in combination with the human cavity model, and then the position and shape of the endoscope in the human body can be finally confirmed; and

[0092] Finally, in step 706, the shape is displayed. The position and shape of the endoscope are displayed in real time through a display device. In one embodiment of the present invention, in order to be more intuitive, the endoscope model can also be transformed in real time in position and shape, and displayed on the display of the industrial computer simultaneously with the human cavity model, so as to provide a reference for the doctor.

[0093] Navigation technology can significantly improve the accuracy of endoscopy and treatment through the human body, but the current electromagnetic navigation method can only track the position of the end point of the endoscope through the positioning guidewire, and cannot sense the complete shape information of the endoscope in real time, and cannot compensate for the interference of the deformation of the human cavity structure, which is prone to problems such as cavity navigation errors and endoscope loops. Although the fiber optic shape sensing method can sense the shape information of the endoscope in real time, it has the disadvantages of expensive equipment, the need for additional registration methods to establish a spatial relationship with the patient, and inconvenience in use.

[0094] The setting and method proposed in the present invention are based on an elastic rod and embedded with multiple retractable flexible sensor arrays to construct a guidewire that can be placed in the working channel of an endoscope. The curvature information of different positions of the elastic rod is calculated in real time by measuring the strain information of each sensor in real time. The complete three-dimensional shape of the guidewire is reconstructed in real time by a curve fitting method based on the Frenet-Serret equation or a curve fitting method based on deep learning, combined with the geometric parameters of the elastic rod and the position information of the sensor array. The real-time spatial matching with the patient's personalized three-dimensional cavity model and the dynamic display of the mutual position relationship between the endoscope and the cavity to the doctor can effectively compensate for the navigation error introduced by the cavity deformation during endoscopic examination and treatment, reduce the risk of cavity path errors and endoscopic looping, and improve the positioning accuracy and safety of inspection and treatment of bronchi, intestines, urethra, etc.

[0095] Although various embodiments of the present invention are described above, it should be understood that they are presented as examples only and not as limitations. It is obvious to those skilled in the relevant art that various combinations, modifications and changes can be made thereto without departing from the spirit and scope of the present invention. Therefore, the breadth and scope of the present invention disclosed herein should not be limited by the exemplary embodiments disclosed above, but should only be defined according to the attached claims and their equivalents.

Claims

1. A real-time sensing module for pipeline shape, characterized in that: include: A guidewire, comprising an elastic rod and a plurality of sensor arrays, wherein the sensor arrays are disposed in the elastic rod and arranged along the central axis of the elastic rod, wherein each sensor array comprises a plurality of sensors, and the sensors are configured to measure shape strain; as well as A control module is communicatively connected to the sensor and is configured to reconstruct the three-dimensional shape of the guidewire according to the shape strain information measured by the sensor.

2. The real-time sensing module according to claim 1, characterized in that: The plurality of sensor array groups are evenly distributed along the central axis direction of the elastic rod.

3. The real-time sensing module according to claim 1, characterized in that: The distance between two adjacent sensor array groups is determined according to the minimum bending radius of the elastic rod axis.

4. The real-time sensing module according to claim 1, characterized in that: Each sensor array includes at least three sensors.

5. The real-time sensing module according to claim 4, characterized in that: The at least three sensors are evenly arranged in the circumferential direction within the cross section of the elastic rod.

6. The real-time sensing module according to claim 1, characterized in that: The sensor is a stretchable flexible sensor.

7. The real-time sensing module according to claim 1, characterized in that: The control module comprises: A decoder, which is connected to each sensor through a wire and is configured to decode the shape strain information measured by the sensor and upload it to the reconstruction submodule; and A reconstruction submodule is communicatively connected to the decoder and is configured to reconstruct the shape of the guidewire according to the decoded information of the decoder.

8. The real-time sensing module according to claim 7, characterized in that: The reconstruction submodule includes an industrial computer.

9. A method for sensing the shape of an endoscope, characterized in that: Includes steps: Inserting a guide wire of the real-time sensing module according to any one of claims 1 to 8 into a working channel of an endoscope, wherein the outer diameter of the guide wire of the real-time sensing module matches the inner diameter of the working channel of the endoscope; Inserting an endoscope into a human body cavity and controlling the movement of the endoscope in the human body cavity; Obtaining shape strain information measured by each sensor in the guidewire; reconstructing the shape of the guidewire based on the shape strain information; Determining the shape of the working channel of the endoscope based on the shape of the guidewire; as well as In combination with a preset three-dimensional model of the endoscope, the shape of the endoscope is determined according to the shape of the working channel.

10. The sensing method according to claim 9, characterized in that: A reconstruction method based on the Frenet-Ser ret equation is used to reconstruct the shape of the working channel of the endoscope based on the shape strain information.

11. The sensing method according to claim 10, characterized in that: The reconstruction method comprises the steps of: Based on the shape strain information measured by each sensor, the discrete curvature and bending direction of each group of sensor arrays are determined; Fitting the explicit function of the discrete curvature and bending direction with respect to the length distance by an interpolation method; Differentiate the explicit function of the bending direction with respect to the length distance with respect to the length to obtain the explicit torsion function; Determine the unit tangent vector at each location based on the explicit function of the discrete curvature with respect to the length distance and the explicit torsion function; as well as Based on the unit tangent vector, the position coordinates of each point are determined to obtain the three-dimensional shape of the guide wire. The shape of the endoscope working channel is the same as the three-dimensional shape of the guide wire.

12. The sensing method according to claim 9, characterized in that: A data-driven deep learning method is used to reconstruct the shape of the working channel of the endoscope based on the shape strain information.

13. The sensing method according to claim 12, characterized in that: The deep learning method comprises the steps of: The shape strain information measured by each sensor is input into the trained deep learning model to obtain the three-dimensional shape of the endoscope working channel.

14. The sensing method according to claim 13, characterized in that: The training of the deep learning model includes the steps of: Two square calibration plates are vertically combined to form a camera imaging space, wherein black metal balls are evenly arranged on the square calibration plates, and the black metal balls are configured to calibrate a two-dimensional camera; The guide wire is set to different shapes and positions within the camera imaging field of view, and the horizontal plane and the vertical plane are photographed respectively by two cameras, and the shape strain information of each sensor array in the guide wire is obtained by a control module; Extracting the guidewire based on the image captured by the camera, reconstructing the complete guidewire shape, and constructing a data set of the guidewire in different positions and shapes; and The data set is divided into training samples and verification samples to train the deep learning model to obtain model parameters that meet the set iteration requirements and construct a deep learning model.

15. The sensing method according to claim 12, characterized in that: The deep learning model uses BP neural network or convolutional neural network.

16. The sensing method according to claim 9, characterized in that: Also includes the steps: In combination with a preset human cavity model, the position of the endoscope in the human body is determined according to the shape of the endoscope.

17. The sensing method according to claim 9, characterized in that: Also includes the steps: The position and shape of the endoscope are displayed by a display device.

18. The sensing method according to claim 9, characterized in that: Also includes the steps: The position and shape of the endoscope are synchronously displayed with the human body cavity model through a display device.

Images