Shape sensing system and reconstruction method

Abstract

The present invention describes a modular shape sensing system and method designed for applications in continuum and soft robots, enabling the reconstruction of the centerline of modular structures within a computational model. Using modules equipped with pressure or strain sensors, the system measures the three-dimensional curvature of the structure, enabling the precise reproduction of its shape. The innovation lies in the flexibility of using different types of sensors, reducing costs compared to conventional technologies that rely on fiber optics and Fiber Bragg Grating (FBG) and Optical Frequency Domain Reflectometry (OFDR) interrogators.

Description

[0001] SHAPE SENSING SYSTEM AND SHAPE RECONSTRUCTION METHOD FIELD OF APPLICATION

[0002]

[0001] The present invention applies to the field of sensing. More specifically, it relates to a shape sensing system and method based on a modular structure.

[0003] FUNDAMENTALS OF THE INVENTION

[0004]

[0002] Currently, in the state of the art, it is possible to find shape sensors using optical fiber with different shapes and technologies. There are sensors with Fiber Bragg Grating (FBG) arrays in conventional single-mode fibers. These fibers are integrated into different materials in order to read the deformations and associate them with the 3D curvature of the materials. There is also the use of FBG fibers in epoxy material; the fibers are placed in a mold so that the cross-section contains FBGs with a 120° angle difference between them, managing to read the deformation plane in each cross-section and converting it into curvature through a deformation-curvature model.

[0005]

[0003] Another existing shape sensing implementation is using multi-core fiber. A multi-core fiber, unlike conventional fibers, has more than one core through which light can propagate. In shape sensing, these cores are interrogated by FBG interrogators and / or OFDR (Optical Frequency Domain Reflectometry) interrogators; in this way, it is possible to measure deformation in a quasi-distributed manner, in the case of FBGs, and in a distributed manner, in the case of OFDR. These interrogators can also measure temperature for subsequent crosstalk compensation. The measured deformations are converted into curvature by a mathematical model, and the shape can be reconstructed.

[0006]

[0004] The present invention proposes the use of a modular structure for shape measurement. This structure has modules containing pressure or deformation sensors; each module provides a 3D curvature. By reproducing the curve for each module, it is possible to reproduce the centerline of the structure and, therefore, its shape. The innovation lies in the non-mandatory use of fiber optics and FBG and OFDR interrogators, allowing the use of various other less expensive types of sensors.

[0007] STATE OF THE ART

[0008]

[0005] US7781724 document describes a fiber optic position and shape detection device and its method of use. The device comprises a fiber optic medium. The fiber optic medium comprises at least two single-core optical fibers or one multicore optical fiber with at least two fiber cores. In either case, the fiber cores are spaced to minimize mode coupling between the cores. A fiber Bragg grating array is arranged within each fiber core, and a frequency domain reflectometer is positioned in an operable relationship with the fiber optic medium. During use, the device is attached to an object. The strain in the fiber optic is measured, and the strain measurements are correlated with local curvature measurements. The local curvature measurements are integrated to determine the position and / or shape of the object.

[0006] However, in the present invention, instead of one or more optical fibers composing the sensor body, a device with a modular mechanical structure is described in which it is possible to calculate the local curvature in each module using at least two measurement points, for the 2D case, and at least three points, for the 3D case. The quantity measured at these points can be of a diverse nature such as deformation, pressure, elongation, magnetic field, among others. Several different sensors can be used at these measurement points, such as optical fiber with OFDR, FBG, sensors based on macrobend losses in optical fibers, piezoelectric strain gauge, resistive or capacitive, Hall effect sensor. To cause the quantity to be measured, different techniques can be used, such as magnets, springs, pressurized chambers, optical fiber coils, among others.These quantities are directly related to the local 3D curvature; integrating this curvature over all the structure's modules using the parallel transport / Bishop equations results in the structure's shape as a parameterized curve with respect to arc length in three-dimensional space.

[0009]

[0007] Document EP3014214 describes an optical shape detection system and method with at least two optical fibers (OSF1, OSF2), both comprising optical shape detection elements. A processor (P) is configured to record a coordinate system indicating the position of one of the optical fibers (OSF1) in space, and to record the position (R2) of the other optical fiber (OSF2) relative to that coordinate system. An optical console system (C, Cl, C2) serves to interrogate the optical shape detection elements in both optical fibers (OSF1, OSF2) and, consequently, determine a three-dimensional shape measurement (I) of both optical fibers (OSF1, OSF2), based on the recorded position (R2) of the second optical fiber (OSF2) relative to the coordinate system.This enables 3D optical shape detection along the length of both optical fibers (OSF1, OSF2), thus allowing the 3D shape reconstruction of, for example, long medical devices with lengths of several meters. More than two shape-detecting optical fibers, for example, incorporated into separate devices, can be registered in this way in a hierarchical data structure, thus enabling the shape detection of very long instruments.

[0010]

[0008] However, this document differs from the present invention in that instead of one or more optical fibers composing the sensor body, a device with a modular mechanical structure is described in which it is possible to calculate the local curvature in each module using at least two measurement points, for the 2D case, and at least 3 points, for the 3D case. The quantity measured at these points can be of a diverse nature, such as deformation, pressure, elongation, magnetic field, among others. Several different sensors can be used at these measurement points, such as optical fiber with OFDR, FBG, sensors based on macrobend losses in optical fibers, piezoelectric strain gauge, resistive or capacitive, Hall effect sensor. To cause the quantity to be measured, different techniques can be used, such as magnets, springs, pressurized chambers, optical fiber coils, among others.These quantities are directly related to the local 3D curvature; integrating this curvature over all the structure's modules using the parallel transport / Bishop equations results in the structure's shape as a parameterized curve with respect to arc length in three-dimensional space.

[0011]

[0009] Document CN110243301 describes a multicore optical fiber shape sensor of the core-to-core scanning type based on dynamic BOTDA (Brillouin Optical Time Domain Analysis). The core-to-core scanning type multicore optical fiber shape sensor consists of a multicore optical fiber, a multicore optical fiber fan-in device, a multicore optical fiber fan-out device, two optical switches, and single-mode optical fibers to connect each part. Both ends of the multicore optical fiber are sequentially connected with multicore optical fiber fan-in / out devices and optical switches; and, by controlling the two optical switches, an optical wave can scan core-to-core in each fiber core of the multicore optical fiber in order to acquire respective strain information from each fiber core.The multicore fiber optic shape sensor of the core-to-core scanning type can be used in a shape detection device of a BOTDA dynamic detection system, can be widely used for health monitoring of an intelligent structure, and can also be used in a tensioned skin structure of a robot or aircraft wing to detect changes in the shape of the tensioned skin structure in real time.

[0012]

[0010] However, this document differs from the present invention in that, instead of one or more optical fibers composing the sensor body, a device with a modular mechanical structure is described in which it is possible to calculate the local curvature in each module using at least two measurement points for the 2D case, and at least three points for the 3D case. The quantity measured at these points can be of a diverse nature, such as deformation, pressure, elongation, magnetic field, among others. Several different sensors can be used at these measurement points, such as optical fiber with OFDR, FBG, sensors based on macrobend losses in optical fibers, piezoelectric strain gauge, resistive or capacitive, Hall effect sensor. To cause the quantity to be measured, different techniques can be used, such as magnets, springs, pressurized chambers, optical fiber coils, among others.These quantities are directly related to the local 3D curvature; integrating this curvature over all the structure's modules using the parallel transport / Bishop equations results in the structure's shape as a parameterized curve with respect to arc length in three-dimensional space.

[0013]

[0011] Document CN110243305 describes a multicore serial cycle fiber shape sensor based on dynamic BOTDA. The fiber shape sensor comprises a multicore fiber, a single-mode fiber and a multicore fiber Fan-in device, a single-mode and multicore fiber Fan-out device, and a plurality of single-mode fibers, where the plurality of single-mode fibers are connected between the Fan-in device and the Fan-out device. By sequentially and circularly connecting a plurality of fiber cores from a multicore fiber in series, the function of expanding the plurality of fiber cores to form a one-dimensional topological optical path is achieved.A fiber shape sensor can be used in a shape detection device of a BOTDA dynamic detection system, and can be widely used for integrity monitoring of an intelligent structure. It can also be used in a tensioned skin structure of a robot or an airplane wing, with shape changes detected in real time.

[0014]

[0012] However, this document differs from the present invention in that instead of one or more optical fibers composing the sensor body, a device with a modular mechanical structure is described in which it is possible to calculate the local curvature in each module using at least two measurement points, for the 2D case, and at least three points, for the 3D case. The quantity measured at these points can be of a diverse nature, such as deformation, pressure, elongation, magnetic field, among others. Several different sensors can be used at these measurement points, such as optical fiber with OFDR, FBG, sensors based on macrobend losses in optical fibers, piezoelectric strain gauge, resistive or capacitive, Hall effect sensor. To cause the quantity to be measured, different techniques can be used, such as magnets, springs, pressurized chambers, optical fiber coils, among others.These quantities are directly related to the local 3D curvature; integrating this curvature over all the structure's modules using the parallel transport / Bishop equations results in the structure's shape as a parameterized curve with respect to arc length in three-dimensional space.

[0013] Document WO2008 / 115375 discloses a position and / or shape sensing device using optical fiber, including an optical fiber with two or more single-core optical fibers or a multicore optical fiber with two or more fiber cores. In both cases, the fiber cores are spaced so that the mode coupling between the fiber cores is reduced and preferably minimized. The optical fiber is physically associated with an object. The voltage in at least one part of the optical fiber, where it is associated with the object, is determined by an OFDR using one or more Rayleigh scattering patterns for that part of the optical fiber.The determined tension is used to determine the position and / or shape of the object.

[0015]

[0014] However, this document differs from the present invention in that instead of one or more optical fibers composing the sensor body, a device with a modular mechanical structure is described in which it is possible to calculate the local curvature in each module using at least two measurement points, for the 2D case, and at least 3 points, for the 3D case. The quantity measured at these points can be of a diverse nature such as deformation, pressure, elongation, magnetic field, among others. Several different sensors can be used at these measurement points such as optical fiber with OFDR, FBG, sensors based on macrobend losses in optical fibers, piezoelectric strain gauge, resistive or capacitive, Hall effect sensor. To cause the quantity to be measured, different techniques can be used such as magnets, springs, pressurized chambers, optical fiber coils, among others.These quantities are directly related to the local 3D curvature; integrating this curvature over all the structure's modules using the parallel transport / Bishop equations results in the structure's shape as a parameterized curve with respect to arc length in three-dimensional space.

[0016]

[0015] Document CN106610273 describes a shape detection device and method based on a spiral FBG sensor array. The device comprises the spiral FBG sensor array, an FBG demodulator, data acquisition and shape reconstruction equipment, and display equipment. The device does not require other peripheral auxiliary equipment, is not affected by electromagnetic interference, has good medical compatibility, does not emit radiation, and can achieve long-distance remote monitoring. Through the spiral configuration of the FBG sensor array, the device can perform deformation detection at multiple points of a single optical fiber, forming a quasi-distributed detection system. With the encapsulation of the optical fiber in a nickel-titanium alloy filament, the device protects the grid points from wear, has strong anti-interference capability, and extends the detection range.The device can obtain the shape of a flexible robot in real time, and has high real-time responsiveness. Furthermore, the device detects curvature and torsion, is simple in its detection method, can be used for shape detection in gastroscopic and colonoscopes and in flexible / soft robots, and plays an important role in flexible robots in the areas of disaster relief, medical rehabilitation, and national security.

[0016] Therefore, it can be concluded that the present invention differs from the prior art documents presented here, since instead of one or more optical fibers with a spiral FBG matrix composing the sensor body, a device with a modular mechanical structure is described in which it is possible to calculate the local curvature in each module using at least two measurement points, for the 2D case, and at least 3 points, for the 3D case.The quantities measured at these points can be of diverse nature, such as deformation, pressure, elongation, magnetic field, among others. Various different sensors can be used at these measurement points, such as fiber optics with OFDR, FBG, sensors based on macrobend losses in optical fibers, piezoelectric strain gauges, resistive or capacitive strain gauges, and Hall effect sensors. To induce the quantity to be measured, different techniques can be used, such as magnets, springs, pressurized chambers, fiber optic coils, among others. These quantities are directly related to the local 3D curvature; the integration of this curvature over all the modules of the structure using the parallel transport / Bishop equations results in the shape of the structure as a parameterized curve with respect to the arc length in three-dimensional space.

[0017] SUMMARY OF THE INVENTION

[0018]

[0017] The present invention describes a modular shape sensing system and method designed for applications in continuous and soft robots, allowing the reconstruction of the centerline of modular structures in a computational model. Using modules with pressure or deformation sensors, the system measures the three-dimensional curvature of the structure, enabling the precise reproduction of its shape. The innovation lies in the flexibility of using different types of sensors, reducing costs compared to traditional technologies that rely on fiber optics and Fiber Bragg Grating (FBG) and Optical Frequency Domain Reflectometry (OFDR) interrogators.

[0019] BRIEF DESCRIPTION OF THE FIGURES

[0020]

[0018] The invention can be better understood through the brief description in the following figures:

[0021] Figure 1 shows the module of the structure containing three cylindrical cavities.

[0022] Figure 2 shows a cylindrical cavity with a washer, rubber, spring, and tendon responsible for measuring the curvature in each module.

[0023] Figure 3 shows the sensor in operation; on the left, the curve reconstructed by the mathematical model can be seen, which is done by integrating the tangent vector. On the right, the modular structure containing the curvature sensors in each module can be seen.

[0024] Figure 4 illustrates the cross-section of a module of the modular structure.

[0025] DETAILED DESCRIPTION OF THE INVENTION

[0026]

[0019] The invention can be better understood through the following detailed description, in accordance with the attached figures.

[0027]

[0020] The present invention comprises a shape sensing system composed of a modular mechanical structure, in which it is possible to calculate the local curvature in each module using at least two measurement points for the two-dimensional case, and at least three non-aligned points for the three-dimensional case. The quantity measured at these points can be of a diverse nature such as deformation, pressure, elongation, magnetic field, among others. To cause the quantity to be measured, different techniques can be used, such as magnets, springs, pressurized chambers, fiber optic coils, among others. These quantities are directly related to the local three-dimensional curvature (of each module). Several different sensors can be used at these measurement points, such as fiber optics with OFDR, FBG, sensors based on macrobend losses in optical fibers, piezoelectric, resistive or capacitive strain gauges, Hall effect sensors, among others.These quantities are directly related to the local three-dimensional curvature by a mathematical model that relates the quantity at each measurement point to the magnitude and direction of the curvature in each module. Using the magnitude and direction of the curvature in each module, it is possible to reconstruct the centerline of the structure using the parallel transport / Bishop equations, as illustrated in Figure 3. Mathematically, the centerline is a parameterized curve with respect to the arc length in three-dimensional space, describing the spatial position of the points on the line passing through the center of the structure relative to a reference frame at the sensor base.

[0028]

[0021] To exemplify the concept, a possible implementation will be presented with the application of the system to a structure, called here a modular structure (1). Figure 1 shows a module (8) of the modular structure (1), each module (8) has 3 cylindrical cavities (2), which are the measurement points (9), in which there is a structure with rubber rings (3), washers (4), sensor (5) (in this application example, optical fiber is used), spring (6) and tendon (7), illustrated in Figure 2. The tendon (7) serves to move the flexible modular structure (1).

[0029]

[0022] The operating principle is as follows: the tendons (7) move the modular structure (1), which bends and deforms, and as the tendons (7) move the modular structure (1), the generated curvatures exert pressure on the springs (6) which, in turn, press the washers (4) and the sensor (5) present in the cavities. The sensor (5) undergoes deformations in the cylindrical cavities (2) and locally increases its length, while a properly programmed microcontroller (in this application an OFDR interrogator) is able to measure these deformations in a distributed manner over the sensor (5). It is important to note that the use of optical fiber in the device is not mandatory; the springs could be pressing other types of sensors, such as piezoresistive sensors, pressure sensors, among others. Readings from these other sensors can be taken using properly programmed microcontrollers.It is also important to note that springs can be replaced by other devices such as pressurized chambers, magnets, and others. Depending on the implementation, springs may not be necessary; for example, implementations using fiber optic coils employ the macrobend loss principle.

[0030]

[0023] Using these measures, a shape reconstruction method through a mathematical model implemented in a computer-readable medium reconstructs the curve. The computer program is the Python implementation of a shape reconstruction algorithm. This algorithm uses the following vector differential equations:

[0031] Dr.

[0032] ds

[0033] = k1M1 + k2M2

[0034] ds

[0035] dM1_

[0036] ds

[0037] dM2_

[0038]

[0039] ds

[0040]

[0024] Equations 2, 3, and 4 are known as Bishop's equations or parallel transport equations. Equation 1 results from integrating the tangent vector to obtain the coordinates that define the central curve of the device.

[0041]

[0025] The equations relate the vectors associated with a parameterized spatial curve with respect to the length of its own arc. The vectors are: the position vector r, which defines the curve, the unit tangent vector T, which points in the direction tangent to the curve at each point, and the unit vectors

[0042]

[0043] M2, together with the tangent vector, forms an orthonormal basis known as a Bishop basis or parallel transport basis. This vector basis is determined by solving equations 2, 3, and 4. For this, it is necessary that the scalar functions k1 and k2 be previously determined, as well as the initial conditions of the problem. The initial conditions are determined to correspond with the coordinate axis determined in the laboratory or in an application environment. The functions k1 and k2 are the first and second Bishop curvatures; they determine the geometric shape of the curve. These functions are determined by the curvature sensors present in each module. In the case presented, the set of cavities with rubber, washers, springs, optical fiber, and the OFDR interrogator are these sensors.

[0044]

[0026] The general model that transforms the curvature sensor data into ke k2se is given as follows:

[0045] k1= K cos(0)

[0046] k2= Ksin(0)

[0047]

[0027] The functions ke k2 are determined by the magnitude K and the angle θ for each modulus. These quantities are determined by a phasor curvature. K phasor = /

[0048]

[0049] (e1,e2,e3..., u1, u2, u3... ) = a + bi which is a function of both the value of the measured quantity (e1,e2, e3... ) at each measurement point ( 9 ) and the position of these measurement points ( 9 ), defined by the phasors (

[0050]

[0051] u 1; u2, u3... ), which is present in the cross-section of each module, K = |

[0052]

[0053] |κ fasor | e 0 = angle(Kf asor The measurement points (9) can be seen in Figure 4, which represents the cross-section of a module of the modular structure (1). It is possible to see the measurement points (9) MP i , the body of the structure, the distance di between the measurement point (9) MP i and the neutral line, the angle θ i from the measurement point (9) MPi measured with respect to the y-axis, the angle 0b of Kf asor measured with respect to the ye axis, the distance r between the origin and the measurement point (9) MPi.

[0054]

[0028] With the determination of k1 and k2 in each module, equations 1, 2, 3, and 4 can be solved iteratively, considering k1 and k2 constant in the length between each module. Figure 3 shows the sensor in operation.

[0055]

[0029] Another mathematical model can be used, this model is characterized by the Frenet-Serret equations. In this model we also have vector differential equations, but involving the tangent (T), normal (N) and binormal (B) vectors as shown below:

[0056] dT

[0057] = KN

[0058] ds

[0059] dN

[0060] = −κ + τ

[0061] ds

[0062] dB

[0063] = −τ

[0064] ds

[0065]

[0030] The curvature K is the same as in the previous model and the torsion T is defined as the derivative of the angle function with respect to s as below:

[0066] T =

[0067]

[0068] ds

[0069]

[0031] Solving the system, the solution for all three vectors is obtained and, from the integration of the tangent vector, the shape of the sensor can be reconstructed.

[0070]

[0032] The main innovation of this system lies in its modular structure, which eliminates the need for exclusive optical fibers and sophisticated interrogators, such as FBGs and OFDRs. This provides a more accessible and flexible alternative, capable of using a variety of pressure or strain sensors, without compromising measurement accuracy and shape reconstruction. The modular design allows each segment of the structure to be measured individually, and the data to be integrated to produce a complete and faithful representation of the shape of the flexible structure.

[0071]

[0033] The present invention has been disclosed in this descriptive report in terms of its preferred embodiment. However, other modifications and variations are possible from the present description, and are still within the scope of the invention disclosed herein.

[0072] REFERENCE SIGNS

[0073] 1. Modular structure

[0074] 2. Cylindrical cavities

[0075] 3. Rubber rims

[0076] 4. Washers

[0077] 5. Sensor

[0078] 6. Spring

[0079] 7. Tendon

[0080] 8. Module

[0081] 9. Measurement Point

Claims

CLAIMS 1. A shape sensing system CHARACTERIZED by comprising a modular structure (1), composed of a plurality of modules (8), each containing measurement points (9) with sensors (5) capable of detecting physical quantities such as deformation, pressure, elongation and magnetic field caused by elements at the measurement points (9), wherein the detected quantities are read by microcontrollers and related, from a mathematical model, to the local two-dimensional or three-dimensional curvature in each module, allowing the computational reconstruction and reproduction of the two-dimensional or three-dimensional shape of the modular structure (1).

2. System, according to claim 1, CHARACTERIZED in that in the modules (8) at least two measurement points (9) are used for the reproduction of the two-dimensional curve and at least three measurement points (9) for the reproduction of the three-dimensional curve.

3. System, according to claim 1, CHARACTERIZED in that the elements generating the measured quantity at the measuring points (9) can be magnets, springs (6), pressurized chambers and optical fiber coils.

4. System, according to claim 1, CHARACTERIZED in that the sensors (5) present at the measurement points (9) can be of different types, such as optical fiber, macrobend loss-based sensors, piezoelectric strain gauges, resistive or capacitive sensors, and Hall effect sensors, depending on the type of quantity to be measured.

5. System, according to claim 1, CHARACTERIZED in that the mathematical model solves a set of vector differential equations, using initial conditions and the respective scalar functions as input, to calculate the direction and orientation vectors of the two-dimensional or three-dimensional curvature of the modular structure ( 1 ).

6. System, according to claim 5, CHARACTERIZED in that the differential equations of the mathematical model are preferably the Bishop / Parallel Transport equations or Frenet-Serret equations, in which the equations relate the vectors associated with a spatial curve parameterized with respect to the length of its own arc with its derivatives, the tangent vector being integrated with respect to the arc length to define the two-dimensional or three-dimensional curve that represents the shape.

7. System, according to claim 6, CHARACTERIZED in that the mathematical model can be used for any type of modular structures, with the determination of scalar functions for solving differential equations.

8. System according to claim 6, CHARACTERIZED in that the scalar functions of the differential equations are determined by a phasor curvature, with magnitude and angle, which is a function of the value of the quantities measured at each measurement point (9) as well as the position of these measurement points in the cross-section, where these quantities are detected by the sensors (5) whose position coincides with that of the measurement points (9) present in the cross section of each module (8).

9. System, according to claims 6 or 7, CHARACTERIZED in that the mathematical model obtained is implemented in a computer-readable medium that reproduces the central line of the modular structure (1) in two or three dimensions.

10. Shape reconstruction method CHARACTERIZED by solving a set of vector differential equations, using initial conditions and the respective scalar functions as input, to calculate the direction and orientation vectors of the two-dimensional or three-dimensional curvature of a modular structure ( 1 ).

11. Method, according to claim 10, CHARACTERIZED in that the differential equations of the mathematical model are preferably the Bishop / Parallel Transport equations or Frenet-Serret equations, in which the equations relate the vectors associated with a spatial curve parameterized with respect to the length of its own arc with its derivatives, the tangent vector being integrated with respect to the arc length to define the two-dimensional or three-dimensional curve that represents the shape.

12. Method, according to claim 11, CHARACTERIZED in that the mathematical model can be used for any type of modular structures, with the determination of scalar functions for solving the differential equations.

13. Method according to claim 11, characterized in that the scalar functions of differential equations are determined by a phasor curvature, with magnitude and angle, which is a function of the value of the quantities measured at each measurement point (9) as well as the position of these measurement points in the cross section, where these quantities are detected by the sensors (5) whose position coincides with that of the measurement points (9) present in the cross section of each module (8).

14. Method, according to claims 11 or 12, CHARACTERIZED in that the mathematical model obtained is implemented in a computer-readable medium that reproduces the central line of the modular structure (1) in two or three dimensions.

Images