Fiber optic shape sensing management

By embedding nanomagnet and Bragg grating in optical fibers, combining spectral analysis and machine learning, high-precision tracking of fiber shape and position is achieved, solving the problem of positioning accuracy and multi-degree of freedom application of electromagnetic tracking systems in medical care. Improves the safety and efficiency of medical operations.

CN118948439APending Publication Date: 2024-11-15NORTHERN DIGITAL
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410601519.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-15
Filing Date
2024-05-15
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing electromagnetic tracking systems are difficult to accurately track the shape and position of optical fibers during medical treatment, especially in areas with high metal distortion or applications requiring multiple degrees of freedom, which affects the positioning and operational accuracy of medical equipment.

Method used

By embedding a nanomagnet and Bragg grating in the optical fiber, the frequency and phase shift of reflected light is analyzed using a tunable laser and spectrometer, and combined with machine learning models, the shape, position and other characteristics of the optical fiber are determined to achieve 6DOF fiber tracking.

Benefits of technology

It improves the positioning accuracy and stability of optical fibers in medical environments, reduces dependence on imaging modes, reduces the risk of radiation exposure to patients and medical staff, and enhances the safety and efficiency of the surgery.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118948439A_ABST
    Figure CN118948439A_ABST
Patent Text Reader

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for tracking fiber shaping. In some embodiments, a server obtains an optical signal reflected through an optical fiber that includes one or more embedded sensors and a nanomagnetometer embedded at a distal location of the optical fiber. The server determines a frequency shift for each of the reflected optical signals that is applied by the one or more embedded sensors on the reflected optical signals. The server determines a phase shift for each of the reflected optical signals that is applied by the nano-magnetometer on the reflected optical signals. The server uses the determined frequency and phase shift of each of the reflected optical signals to determine characteristics of the optical fiber, including the shape of the optical fiber and the position of the optical fiber relative to an external reference.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 502,240, filed on May 15, 2023, which is incorporated herein by reference. Technical Field

[0003] This patent specification relates to tracking optical fibers in electromagnetic fields. Background Art

[0004] Electromagnetic tracking (EMT) systems are used to help locate instruments and patient anatomy during medical procedures. These systems utilize a magnetic transmitter close to one or more magnetic sensors. The one or more sensors can be spatially positioned relative to the transmitter and sense the magnetic field generated by the transmitter. Summary of the invention

[0005] Some tracking systems can provide characteristics of optical fibers used in medical procedures. For example, these characteristics can include the posture, shape, and position of the optical fiber. In more detail, the tracking system can use the provided optical fiber characteristics to locate the instrument relative to the patient's anatomy and make measurements. These medical procedures can span many fields and can include, for example, surgical interventions, diagnostic procedures, imaging procedures, radiation therapy, etc. In some cases, the optical fiber can be attached to the instrument during the medical procedure to provide the characteristics of the instrument, such as, for example, position, orientation, and shape. In some examples, the characteristics of the instrument may include one or more of the above characteristics. Although many methods can be used to provide the characteristics of the optical fiber, artificial intelligence technology (such as machine learning) can be trained and evaluated using the characteristics of the optical fiber. By developing such technology to determine multiple characteristics of the optical fiber, applications and computing processes used to track instruments during medical procedures can be improved.

[0006] In some embodiments, the tracking system can use electromagnetic tracking technology to determine characteristic information of the optical fiber, such as attitude information. Specifically, the electromagnetic tracking technology can use a sensor small enough to approach the diameter of the optical fiber to measure the attitude while maintaining sensitivity to the electromagnetic field required for tracking. In addition, by using electromagnetic tracking technology, the tracking system can use light in the optical fiber to interpret the sensed magnetic signal to obtain or measure the attitude.

[0007] In some embodiments, the tracking system can utilize various components of electromagnetic tracking technology for shape sensing. In some examples, the tracking system can include a nanomagnetometer, an electromagnetic tracking system, and a fiber optic shape sensing system. In more detail, the nanomagnetometer can be embedded at the far end of the optical fiber. In some examples, the optical fiber can be equipped with one or more Bragg gratings. The tracking system can utilize a tunable laser covering a certain wavelength range, which transmits light along the optical fiber. The tracking system can utilize a spectrometer and a computer system to interpret the reflected light.

[0008] The reflected light can carry information about the magnetic field sensed by the nanomagnetometer. Specifically, the nanomagnetometer can sense the magnetic field and apply the sensed magnetic field to the transmitted light through a physical process that changes the phase of the light after the light is reflected from the sensor. In addition, the electromagnetic tracking system can observe the wavelength shift in the reflected light, which may be caused by the bending and twisting of the optical fiber, which can indicate a measure of the strain introduced into the optical fiber. The electromagnetic tracking system can analyze the phase shift applied by the nanomagnetometer and the wavelength shift applied by the optical fiber to estimate the characteristics of the optical fiber (e.g., the strain of the optical fiber). In response, the tracking system can estimate the curvature, angle of curvature, and shape of the optical fiber. In some examples, one of the wavelengths emitted from the laser is used in conjunction with the nanomagnetometer to sense the magnetic field and track the position of the sensor. In some examples, the tracking system can use other wavelengths emitted from the laser for shape estimation. The process will be further described below.

[0009] In one general aspect, a method performed by one or more computing devices includes obtaining optical signals reflected through an optical fiber, the optical fiber comprising one or more embedded sensors and a nanomagnetometer embedded at a distal location of the optical fiber; determining a frequency shift of each of the reflected optical signals, the frequency shift imposed on the reflected optical signal by the one or more embedded sensors; determining a phase shift of each of the reflected optical signals, the phase shift imposed on the reflected optical signal by the nanomagnetometer; and determining one or more characteristics of the optical fiber using the determined frequency shift and phase shift of each of the reflected optical signals, wherein the one or more characteristics of the optical fiber include a shape of the optical fiber and a position of the optical fiber relative to an external reference.

[0010] Other embodiments of this and other aspects of the present disclosure include corresponding systems, devices, and computer programs, which are configured to perform the actions of the method, encoded on a computer storage device. A system consisting of one or more computers can be so configured by software, firmware, hardware, or a combination thereof installed on the system, which software, firmware, and hardware cause the system to perform the actions in operation. One or more computer programs can be so configured by having instructions, which, when executed by a data processing device, cause the device to perform the actions.

[0011] The foregoing and other embodiments may each optionally include one or more of the following features alone or in combination. For example, one embodiment includes all of the following features in combination.

[0012] In some embodiments, determining one or more characteristics of the optical fiber includes determining a position of the optical fiber using at least the determined phase shift of each of the reflected optical signals.

[0013] In some embodiments, determining one or more characteristics of the optical fiber includes determining a shape of the optical fiber using at least the determined frequency shift of each of the reflected optical signals.

[0014] In some embodiments, the method includes commanding a tunable laser to transmit an optical signal through an optical fiber.

[0015] In some embodiments, the optical signal comprises light having a range of wavelengths.

[0016] In some embodiments, determining the phase shift of each of the reflected optical signals also includes: determining one or more magnetic fields sensed by the nanomagnetometer based on the determined phase shift of each of the reflected optical signals; and using the determined one or more magnetic fields to identify the position of the nanomagnetometer located at a distal location of the optical fiber.

[0017] In some embodiments, determining the frequency shift of each of the reflected optical signals further comprises using the determined frequency shift of each of the reflected optical signals to determine measures of bending at a plurality of locations along the optical fiber.

[0018] In some embodiments, determining one or more characteristics of an optical fiber using the determined frequency shift and the determined phase shift of each of the reflected optical signals also includes: determining the strain of the optical fiber using the determined measurement of the bend, which is determined at multiple locations along the optical fiber using the determined frequency shift, wherein the determined frequency shift of each of the reflected optical signals represents: (i) the location of the strain of the optical fiber and (ii) the location of each of one or more embedded sensors in the optical fiber; wherein the one or more characteristics of the optical fiber include (i) the curvature of the optical fiber at the location of the strain, (ii) the angle of the bend at the location of the strain, (iii) the shape of the optical fiber, and (iv) the location of the optical fiber.

[0019] In some embodiments, the method includes: estimating the shape of the optical fiber by combining the curvature of the optical fiber and the angle of the curvature at each location of strain; and determining the position of the optical fiber using (i) the estimated shape of the optical fiber and (ii) the identified position of the nanomagnetometer, wherein the external reference represents the identified position of the nanomagnetometer.

[0020] In some embodiments, the estimated shape of the optical fiber indicates the shape of the optical fiber in three-dimensional spatial coordinates, and wherein the estimated shape of the optical fiber includes one or more of an orientation of a center of the optical fiber, a first radial axis of the optical fiber, and a second radial axis of the optical fiber.

[0021] In some embodiments, the one or more embedded sensors include one or more fiber Bragg gratings (FBGs) embedded at multiple locations along the optical fiber.

[0022] In some embodiments, the optical fiber includes multiple cores, wherein the nanomagnetometer reflects a portion of the optical signal through the cores of a first subset of the multiple cores, and the one or more embedded sensors reflect a portion of the optical signal through the cores of a second subset, wherein the cores of the first subset are different from the cores of the second subset.

[0023] In some embodiments, the method includes: training a machine learning model to predict one or more characteristics of the optical fiber using the determined frequency and phase shift of each of the reflected optical signals; and applying the trained machine learning model.

[0024] In some embodiments, the method includes moving the wavelength of the nanomagnetometer within the operating range of the interrogator.

[0025] In some embodiments, the optical fiber is positioned in the surgical environment.

[0026] In one general aspect, a system includes: an optical fiber; one or more sensors and a nanomagnetometer embedded in the optical fiber; and a demodulator device configured to: transmit an optical signal through the optical fiber; receive reflected optical signals from the optical fiber; determine a frequency shift of each of the reflected optical signals, the frequency shift imposed on the reflected optical signal by the one or more embedded sensors; determine a phase shift of each of the reflected optical signals, the phase shift imposed on the reflected optical signal by the nanomagnetometer; and use the determined frequency shift and phase shift of each of the reflected optical signals to determine one or more characteristics of the optical fiber, wherein the one or more characteristics of the optical fiber include a shape of the optical fiber and a position of the optical fiber relative to an external reference.

[0027] The subject matter described in this patent specification can be implemented in a variety of embodiments and may result in one or more of the following advantages. By embedding a nanomagnetometer in the distal end of an optical fiber, the position of the optical fiber (for example, within an operating room) can be identified using a minimum number of components. Typically, one or more sensors may be embedded in the patient's body to track the position of the optical fiber. However, these sensors can be eliminated from the system, and the nanomagnetometer can provide enhanced functionality in its location while reducing the total footprint of the components in the patient's body. The combination of one or more sensors embedded in the patient's body results in an increase in the number of wires embedded in the patient's body. However, by embedding the nanomagnetometer into the optical fiber, the number of wires can be reduced due to the removal of the sensor, which reduces the amount of hardware required to monitor and track the position of the optical fiber in the patient's body.

[0028] The details of one or more embodiments of the subject matter of this patent specification are set forth in the following drawings and description. Other features, aspects, and advantages of the subject matter will become apparent from this patent specification, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a schematic diagram of an exemplary system for providing characteristics of a tracked optical fiber.

[0030] Figure 2 is a block diagram of an exemplary system for representing one or more components of an optical fiber.

[0031] Figure 3 An exemplary cross section of an optical fiber is shown.

[0032] Figure 4A-4B The convention used to represent the shape of optical fibers is shown.

[0033] Figure 5 A data flow diagram graphically representing the collection characteristics of an optical fiber is shown.

[0034] Figure 6 A computer system executing a training data generator that collects training data to train a fiber property sensing machine learning system.

[0035] Figure 7 is a flow chart of the operation of a data generator for computationally generating training data for training an optical fiber characteristic sensing machine learning system.

[0036] Figure 8 is another flow diagram of the operation of a training data generator for physically generating training data to train an optical fiber characteristic sensing machine learning system.

[0037] Fig. 9 It is a computing system that determines the characteristic information of optical fiber.

[0038] Fig.10 It is the data flow of the operation of the optical fiber characteristic learning machine.

[0039] Fig.11 is a flow chart for determining the characteristics of an optical fiber.

[0040] Fig.12 A block diagram of a computing system that may be used in conjunction with the methods described in this document is shown.

[0041] The same reference numbers and names in different drawings represent the same elements.The components shown here, their connections and relationships, and their functions are exemplary only, and are not meant to limit the embodiments described and / or claimed in this document. DETAILED DESCRIPTION

[0042] Fiber optic shape sensing (FOSS) may include applications used in spatial navigation of medical procedures and other applications. FOSS may be implemented by utilizing a tracking system that determines the characteristics of an optical fiber. Specifically, a tracking system may transmit a light beam through an optical fiber and measure the reflected light to analyze the characteristics of the optical fiber. In more detail, the tracking system may identify the characteristics of the optical fiber because the reflected light may carry useful information encoded in the wavelength shift (caused by the bending and twisting of the optical fiber). In some cases, the wavelength shift may be analyzed by a spectrometer and a computer system to infer the measured value of the strain in the optical fiber. The strain measurements may be analyzed to estimate the curvature and angle of curvature of the optical fiber. The estimated curvature, angle of curvature and other characteristics may be used to estimate the shape of the optical fiber.

[0043] In some embodiments, the tracking system can utilize a variety of methods to relate strain to wavelength shift in the FOSS field. In some examples, implementers of a system including an optical fiber can inscribe a Bragg grating on the optical fiber. The Bragg grating can be designed and / or configured to reflect a specific wavelength called the Bragg wavelength. The Bragg wavelength can be a function of the strain of the optical fiber. In some examples, implementers of the system can add microscopic defects to the optical fiber and can utilize a natural process called Rayleigh backscattering. Therefore, the wavelength of the backscattered light (e.g., transmitted light reflected from a microscopic defect on the optical fiber) will also change with strain.

[0044] In some embodiments, the tracking system can generate an estimate of the shape of the optical fiber. The shape estimate can be a collection of point coordinates in three dimensions, which can be expressed relative to a common origin. The shape origin can be arbitrarily selected to be any point (position) along a measurable section of the optical fiber. In some embodiments, the shape of the optical fiber can exist in its own reference frame. This reference frame can be independent of other components and therefore is not limited to a larger surgical environment. Therefore, it is beneficial to locate the origin of the optical fiber shape relative to an external reference.

[0045] In some embodiments, the system can utilize an electromagnetic tracking system and a fiber optic shape sensing system. The electromagnetic tracking system can include a nanomagnetometer embedded at the distal end of the optical fiber. The nanomagnetometer can sense the magnetic field radiated within the system and can apply a phase shift to the transmitted light, the magnitude of the phase shift being proportional to the magnitude of the sensed magnetic field. The nanomagnetometer can transmit the reflected phase-shifted light along the optical fiber.

[0046] In some examples, the optical fiber can be equipped with one or more Bragg gratings. In some examples, the optical fiber can incorporate microscopic defects and utilize backscattered light backscattered from the defects using the Rayleigh method.

[0047] In some embodiments, the system can utilize a tunable laser to transmit light through an optical fiber. For example, the light can cover a range of wavelengths. The system can utilize a spectrometer and a computer system that can interpret the light reflected from the optical fiber. The reflected light can carry information about the magnetic field sensed by the nanomagnetometer through a physical process that changes the phase of the light after the light is reflected from the sensor. In addition, the system can observe a wavelength shift or frequency shift in the reflected light caused by the bending and twisting of the optical fiber. The bending and twisting of the optical fiber can provide a measurement of the strain in the optical fiber. The strain in the optical fiber can be used to estimate the curvature, curvature angle (angle of curvature) and other characteristics of the optical fiber. The system can then determine the shape of the optical fiber using the estimated curvature, curvature angle and other characteristics of the optical fiber. In some examples, one of the transmission wavelengths emitted from the tunable laser is used in conjunction with a nanomagnetometer to sense the magnetic field and track the position of the sensor (e.g., the position of the nanomagnetometer). The shape of the optical fiber and the position of the nanomagnetometer can be used to provide an indication of the characteristics of the optical fiber. Other wavelengths emitted from the transmitted laser can be used for shape estimation of the optical fiber.

[0048] Figure 1 1 is a schematic diagram of an exemplary electromagnetic tracking system 100 implemented in a surgical environment (e.g., an operating room). The tracking system 100 can collect information from a variety of components to determine the location of one or more medical devices (e.g., a scalpel, a probe, a guidewire, etc.) and equipment, etc. The tracking system 100 can be attached to a guidewire 112 or a nanomagnetometer 122 can be embedded in the guidewire 112 for tracking the guidewire 112 during a variety of medical procedures involving a patient. The guidewire 112 can include an optical fiber. Tracking the guidewire 112 can be used to determine the location of the guidewire 112 within the patient (e.g., the patient's vascular system). Once the shape, position, and other characteristics of the guidewire 112 are determined, the guidewire 112 can be used to guide other medical devices (e.g., a catheter) through the patient (e.g., the patient's vascular system).

[0049] In some embodiments, the tracking system 100 can employ tracking technology that provides six degrees of freedom (6DOF). 6DOF tracking can be used in medical applications (e.g., tracking medical devices in an operating room) to track one or more objects (e.g., medical devices such as scalpels, one or more robotic arms, etc.) to determine and identify the corresponding three-dimensional position, orientation, etc. of the one or more objects for a medical professional (e.g., a surgeon). Such tracking can be used for a variety of applications, such as providing guidance to professionals, and in some cases, can reduce reliance on other imaging modalities (e.g., fluoroscopy) that may expose patients, healthcare providers, and others to ionizing radiation, which can create health risks.

[0050] In some embodiments, the nanomagnetometer can be a device for measuring one or more magnetic fields. The nanomagnetometer can be configured to measure the direction, intensity, or relative change of the magnetic field at a specific location (e.g., the distal end of the optical fiber). For example, the nanomagnetometer can be constructed small enough to approach the diameter of the optical fiber used for shape sensing. Despite the small size of the nanomagnetometer, the nanomagnetometer still has functions and can detect multiple sensitivity levels in the electromagnetic (EM) field required for tracking the optical fiber. In response to measuring multiple sensitivity levels in the electromagnetic field, the nanomagnetometer can generate a signal reflecting the measured characteristics. The measured characteristics of the EM field depend on the position and orientation of the nanomagnetometer relative to the guidewire 112. The nanomagnetometer can measure the characteristics of the EM field and provide measurement information to a computing device such as a computer system, for example, data representing the EM measurement information is assigned to the reflected light transmitted by the demodulator 126. Based on the measurement information provided, the computing device can determine the position, shape, orientation, and other characteristics of the optical fiber. In some examples, a single nanomagnetometer can provide a position and orientation in 5DOF. In some examples, multiple nanomagnetometers can provide a position and orientation in 6DOF. By using this technology, the position, orientation, shape, etc. of a medical device can be determined and processed by a computing device, for example, the computing device recognizes the positioning, position and shape of the medical device, and can graphically represent the medical device (e.g., optical fiber, etc.) in an image such as a registered medical image.

[0051] In order to generate understandable tracking data, a reference coordinate system is established by the tracking system 100. The relative posture (e.g., positioning, position) of the nanomagnetometer 122 at the distal end of the optical fiber can be determined relative to the reference coordinate system 108 associated with the field generator 120. Specifically, the nanomagnetometer 122 can define the reference coordinate system 108 relative to the patient 106 because the nanomagnetometer 122 is attached to the patient 106. In some embodiments, the tracking system 100 can use a variety of electromagnetic sensors in conjunction with the nanomagnetometer 122 to enhance the detection of the coordinate system 108. In some embodiments, by using the nanomagnetometer 122 to establish the reference coordinate system 108, the position and orientation of other electromagnetic sensors or the position and orientation of the guidewire 112 can be determined relative to the reference coordinate system 108. Defining a reference coordinate system allows viewing data (e.g., position data associated with the nanomagnetometer, position data associated with the patient, etc.) from a reference system, all of which are placed on the same reference system, so that the data is easier to understand. In some embodiments, catheter 114 is inserted over guidewire 112 after guidewire 112 is inserted into the patient.

[0052] In this specific embodiment, the control unit 116 and the sensor interface unit 118 are configured to parse the signal generated by the reflected light transmitted by the demodulator 126. For example, the control unit 116 and the sensor interface unit 118 can receive the signal generated by the reflected light, for example, through a wired or wireless connection. The control unit 116 and the sensor interface unit 118 can use electromagnetic tracking methods to determine the attitude, position and other characteristics of the nanomagnetometer 122. As described above, the attitude of the nanomagnetometer 122 can define the reference coordinate system 108. The attitude of the nanomagnetometer 122 can provide information about the leading section of the guidewire 112. The optical-based tracking system 100 can adopt a technique for tracking and identifying the corresponding three-dimensional position, orientation, shape, etc. of an object for medical professionals (for example, using an optical tracking method).

[0053] In some embodiments, the geometry and size of guidewire 112 can be changed. In some examples, guidewire 112 can have a maximum diameter of about 0.8 millimeters (mm). In some examples, the diameter of guidewire 112 can be greater than or less than about 0.8 mm. Guidewire 112 can be used to guide a medical device or a measuring device through a patient (e.g., through the patient's vascular system). In some examples, guidewire 112 can guide an optical fiber through a patient. In some examples, an optical fiber can guide guidewire 112 through a patient according to a navigation context. Although guidewire 112 can have a cylindrical geometry, one or more other types of geometries (e.g., a geometry with a rectangular cross section) can be used. In some embodiments, guidewire 112 can include a wire bundle.

[0054] In some embodiments, field generator 120 may be located below patient 106, e.g., below the surface on which the patient is located, embedded in the surface on which the patient is lying (e.g., a table top, etc.), to emit an electromagnetic field sensed by nanomagnetometer 122. In some examples, field generator 120 is an Aurora tabletop field generator (TTFG), although other field generator technologies and / or designs may be employed.

[0055] In some embodiments, the nanomagnetometer 122 can use one or more electromagnetic fields emitted by the field generator 120 to generate position information of the nanomagnetometer 122. Specifically, the nanomagnetometer 122 can sense the electromagnetic field emitted by the field generator 120 and obtain the sensed electromagnetic field. Based on the obtained electromagnetic field, for example, the nanomagnetometer 122 can apply the sensed electromagnetic field to the reflected transmitted light, and the demodulator 126 and the computing system 128 can use the information applied to the reflected transmitted light to determine the position information of the nanomagnetometer 122. The position information can be, for example, three-dimensional coordinates, spatial coordinates, geographic coordinates, or other types of coordinates. As will be further described below, the nanomagnetometer 122 can apply the generated sensed electromagnetic field to the reflected light by affecting the phase of the reflected light.

[0056] The position information (e.g., position, orientation, and other information) of the tracked nanomagnetometer 122 can refer to the direction that the tracked sensor faces relative to a global reference point (e.g., reference coordinate system 108), and can also be expressed using a coordinate system, and for example, expressed as a vector of orientation coordinates (e.g., azimuth (ψ) and altitude (θ)) or Cartesian coordinates (e.g., x, y, and z). In some examples, the tracked nanomagnetometer 122 can provide position information in 5DOF. The tracking system 100 can operate to determine the characteristics of the optical fiber, as described below. In some examples, the tracking system 100 can characterize the optical fiber in terms of 6DOF characteristics using information provided by the nanomagnetometer 122 and information provided by other examples within the optical fiber. The 6DOF characteristics can allow the magnetometer position and orientation information of the tracked nanomagnetometer 122 to be measured in relation to front / back position, up / down position, left / right position, azimuth, altitude, and flip, among other characteristics.

[0057] In some embodiments, the optical fiber may form one or more shapes as it extends into and through the body of the patient 106. The interrogator 126 may transmit light along the optical fiber, and the reflected light may be used to track, for example, the position, orientation, shape, and other characteristics of the measured section of the optical fiber. In response, the tracking system 100 (e.g., the computing system 128) may determine information indicating characteristics of the optical fiber.

[0058] exist Figure 1In the example shown, the hook shape 107 is generated by the guidewire 112, which can be resolved by the system 100. For example, the demodulator 126 can transmit a fiber optic signal (e.g., light) along the optical fiber. In response, the demodulator 126 can receive the reflected fiber optic signal (e.g., one or more waveforms reflected back) passing through the optical fiber. The received fiber optic signal can be used to determine, for example, the mechanical strain along the length of the optical fiber, and the determined mechanical strain can ultimately be used to determine the shape of the optical fiber. For example, the shape of the optical fiber segment can be determined based on the fiber optic signal transmitted to and from the demodulator 126.

[0059] In some embodiments, the interrogator 126 is an electro-optical data acquisition system that provides measurements of light reflected through the optical fiber. The interrogator 126 provides these measurements to a computing device (eg, computer system 128) to further determine the characteristics of the optical fiber.

[0060] Typically, a fiber Bragg grating (FBG) is a distributed Bragg reflector that is built into a relatively short fiber section that reflects light of a specific wavelength and transmits light of other wavelengths. FBGs can be produced by producing periodic changes in the refractive index of the fiber core, which produces a wavelength-specific dielectric reflector. By adopting this technology, FBGs can be used as in-line optical fibers for sensing applications. Due to the shape of the optical fiber, a small amount of light is reflected at each periodic refractive change. The reflected optical signals are coherently combined to produce a relatively large reflection at a specific wavelength (for example, when the grating period is about half the wavelength of the input light). For example, the reflection point can be set along the optical fiber, for example, at a point (position) corresponding to half the wavelength of the input light. This is called the Bragg condition, and the wavelength at which the reflection occurs is called the Bragg wavelength. Optical signals at non-phase-matched wavelengths other than the Bragg wavelength are substantially transparent.

[0061] Therefore, light propagates through the grating with negligible attenuation or signal change. Only those wavelengths that satisfy the Bragg condition are affected and strongly reflected back. The ability to accurately preset and maintain the grating wavelength is a major feature and advantage of fiber Bragg gratings.

[0062] The central wavelength of the reflected component satisfies the Bragg relation: Bragg =2nΛ, where n is the refractive index and Λ is the period of the refractive index variation of the FBG. Due to the temperature and strain dependence of the parameters n and Λ, the wavelength of the reflected component will also vary as a function of temperature and / or strain. This dependence can be used to determine the temperature or strain based on the reflected FBG wavelength.

[0063] In some embodiments, the nanomagnetometer 122 can provide the system 100 with a reference coordinate system 108, which can be aligned with the patient 106, as described above in conjunction with the field generator 120. The position, orientation, shape, posture, etc. of the guidewire 112 can be defined within the reference coordinate system 108. In this way, the optical fiber can be tracked relative to the patient's anatomy.

[0064] In some cardiology applications, the shape of the fiber optic segment can be used to support medical procedures. For example, the shape of the fiber optic segment can provide information about transseptal puncture in the case of mitral valve repair / replacement or catheterization through the atrial septal wall for atrial fibrillation treatment. In addition, the shape of the fiber optic segment can be used to cannulate the blood vessels entering the kidney from the aorta for stent placement.

[0065] Tracking systems are often accompanied by a computing device (e.g., computer system 128) that can process and present the measurement data. For example, in a surgical intervention, a surgical tool measured by the tracking system can be visualized relative to anatomy marked with annotations from a preoperative plan. Another such example can include an X-ray image annotated with real-time updates from a tracked guidewire.

[0066] Medical procedures supported by tracking systems often measure reference coordinate systems attached to the patient. This allows medical professionals to visualize the patient's anatomy, make measurements relative to it, and correct for the patient's overall movement or motion. In practice, this can be achieved by affixing a nanomagnetometer 122 to the distal end of the optical fiber and sensing the shape of the guidewire 112.

[0067] The described tracking systems may be advantageous because they do not require line of sight to the object being tracked. That is, they do not require a direct, unobstructed line of sight between the tool being tracked and the camera for light to pass through. In some embodiments, the described systems have improved metal immunity and electrical interference resistance. That is, they do not require minimal metal and electrical noise sources to be present in their vicinity to provide consistent tracking performance.

[0068] In medical procedures where surgical or endoscopic tools can improve patient outcomes, additional intraoperative imaging modalities such as ultrasound, MRI, or X-ray can be used. The tracking system described (e.g., Figure 1Another advantage of the system 100) is that the shape of the surgical / endoscopic tool is not distorted by imaging artifacts. In addition, the EMT system does not require direct manual control of the imaging probe by a skilled practitioner to maintain imaging quality. In addition, the system does not expose patients and medical staff to ionizing radiation. Therefore, the range of workflows that benefit from this technology is very wide, covering a variety of endovascular surgeries, electrophysiology, structural heart interventions, peripheral vascular interventions, bronchoscopic interventions, endoscopic surgery, neurosurgical interventions, biopsy needle guidance, percutaneous coronary interventions, transcatheter embolization procedures, pain management procedures, urological interventions, robotic laparoscopic interventions, etc.

[0069] There are a variety of techniques with which optical transducers built into optical fibers can produce measurements (e.g., wavelengths) that can be used to estimate characteristic information along the length of the fiber, such as attitude, shape, position, and the like. In some embodiments, the optical fiber can be equipped with a series of FBGs that correspond to periodic variations in the refractive index manufactured in the optical fiber. In some embodiments, the optical fiber may rely on Rayleigh scattering, a natural process caused by microscopic defects in the optical fiber. For example, techniques using FBGs, Rayleigh scattering, both, and the like have the ability to reflect light with specific wavelengths that can correspond to strain within the optical fiber. Deformations in the optical fiber cause these wavelengths to shift, and the wavelength shift can be measured by a system component called a demodulator 126 that measures the wavelength shift using wavelength division multiplexing (WDM), optical frequency domain reflectometry (OFDR), and the like.

[0070] Using Rayleigh scattering, the reflected light in the optical fiber is returned to the reference light, where beat frequency interference occurs. The magnitude of the signal frequency is linearly related to the position of the reflected light, and the shift in the signal spectrum is related to the changes in strain and temperature at that position. The light from the light source coherently interferes with the light that was previously emitted a short time ago and scattered back from a certain distance along the sensing fiber. If the sweep is linear, the frequency difference between the two optical signals is proportional to the propagation time delay along the sensing fiber and therefore proportional to the distance. This means that the instantaneous beat frequency measured at the detector is mapped to a specific position along the fiber. OFDR operates in the frequency domain (or Fourier domain): the OFDR sensor system tunes the frequency range and receives the frequency response from the optical fiber, which is converted into the time / space domain by Fourier transform.

[0071] In this way, the characteristics of the optical fiber can be estimated by using one or more artificial intelligence techniques (such as a trained machine learning system). In some examples, a FOSS system can be used to estimate the characteristics of the optical fiber. By fixing the optical fiber in this way, a sensing / measurement paradigm for a 6DOF tracking system can be implemented, thereby enabling attitude, shape, and position measurements along the optical fiber in the coordinate space of the 6DOF tracking system. In addition, in the process of supporting optical tracking, this can allow one to make attitude measurements outside the measurement volume or line of sight of the optical tracking system. In the context of the process of supporting electromagnetic tracking, this can allow one to make various fiber optic measurements in areas with high metal distortion where electromagnetic sensors typically perform poorly, or one can use fiber optic measurements to correct for electromagnetic / metal distortion.

[0072] Although Figure 1 Mainly referring to the system 100 including the nanomagnetometer components, it should be understood that the system 100 may include an optical tracking system including, for example, one or more cameras, one or more sensors (e.g., 6DOF sensors), and one or more other components.

[0073] As described above, the operation of the system 100 may be controlled by the computer system 128. Specifically, the computer system 128 may be used to interface with the system 100 and cause the position information of the nanomagnetometer 122 and the section of the optical fiber within the guidewire 112 to be determined to determine various characteristics of the optical fiber.

[0074] In some embodiments, the system 100 can use the data provided by the nanomagnetometer 122 to provide the characteristics of the optical fiber. The demodulator 126 and the computing system 128 can measure the phase on the reflected light applied by the nanomagnetometer 122, and infer the intensity of the magnetic field based on the measured phase. The computing system 128 can provide information indicating the intensity of the measured field to the trained machine learning model to generate the characteristics of the optical fiber. The trained machine learning model can output, for example, a confidence or statistical probability, which can infer the position, location and shape information of the optical fiber. The trained machine learning model can identify the intensity of the magnetic field measured by the nanomagnetometer 122 as an indication of the position of the nanomagnetometer 122, and can infer other characteristics (e.g., posture, shape and position) of the optical fiber.

[0075] Figure 2is a block diagram of an exemplary system 200 for representing the shape of an optical fiber. System 200 includes a demodulator, an optical fiber, and a magneto-optical (MO) sensor. As shown in system 200, the demodulator may include a tunable laser, a detector, a beam splitter, a polarizer, and a 45° splicer. The optical fiber may include a variety of components that receive transmitted light from the demodulator and transmit reflected light back to the demodulator. Specifically, the optical fiber may include a collimator, a λ / 4 plate, an MO material, and a reflector. The MO sensor may include a FBG or a Rayleigh core and a polarization-maintaining fiber core.

[0076] As shown in example 200, various components are texture-encoded to indicate grouping based on different physical components. Likewise, dashed boxes (e.g., common components 202, fiber optic shape sensing (FOSS) components 204, optical fiber 208, and nanomagnetometer components 206) indicate associations with different measurement technologies. These components of system 200 are similar to those shown in system 100.

[0077] In some embodiments, the demodulator can transmit light along the optical fiber and can receive light reflected from the optical fiber. The transmitted light can reflect the Bragg grating, defects in the optical fiber and the nanomagnetometer, and return to the demodulator. Specifically, the return light reflected from the nanomagnetometer can be phase-shifted by a certain amount, which is proportional to the size of the magnetic field sensed by the nanomagnetometer. Here, the nanomagnetometer can apply a phase shift proportional to the size of the magnetic field sensed by the nanomagnetometer on the light. The demodulator can be configured to measure the phase shift in the return light reflected from the nanomagnetometer to estimate and / or measure the size of the magnetic field.

[0078] In some embodiments, the demodulator can measure the phase shift of the received light in various ways. In some examples, the demodulator can utilize a phase-locked loop (PLL) that can measure the phase angle between (i) the transmitted light and (ii) the received light. In some examples, the demodulator can utilize a variety of demodulation techniques to measure and identify the phase shift. In some examples, the demodulator can use phase contrast microscopy techniques to calculate or estimate the phase shift of the received light. Other techniques for measuring the phase shift of the received light are also possible.

[0079] In some embodiments, the demodulator can use the measured phase shift to estimate the position of the nanomagnetometer. In more detail, the demodulator can determine the strength of the magnetic field sensed by the nanomagnetometer 122 based on the size of the estimated phase shift measured from the optical light. The strength of the magnetic field can be related to the size of the estimated phase shift of the optical light. For example, the demodulator can identify the strength of the magnetic field and perform triangulation, or use the strength of the magnetic field to locate the position of the nanomagnetometer.

[0080] like Figure 1As shown, the field generator 120 can emit one or more magnetic fields sensed by the nanomagnetometer 122. Each of the one or more magnetic fields can be separated, for example, by time (e.g., time division multiplexing) and / or frequency (e.g., frequency division multiplexing). The nanomagnetometer 122 can measure the intensity of the emitted magnetic field at its current position in the patient 106. The intensity of the emitted magnetic field can change as the nanomagnetometer 122 moves throughout the patient 106. Based on the measured value of the emitted magnetic field by the nanomagnetometer 122, the nanomagnetometer 122 can apply a phase shift proportional to the intensity of the measured emitted magnetic field to the optical light (once the optical light is received / detected). For example, the nanomagnetometer 122 can apply a phase shift to the optical light by offsetting the polarization channel before and after the polarization channel passes through the magneto-optical material and causing interference when these channels are recombined.

[0081] The demodulator can determine the strength of the estimated magnetic field based on the measured phase shift. In some examples, based on the measured phase shift, the demodulator can compare (i) the magnitude of the power of the magnetic field emitted by the field generator 120 and (ii) the difference between the strength of the magnetic field determined based on the measured phase shift. The demodulator can use the compared difference to estimate the attitude of, for example, the nanomagnetometer 122 based on the position of the field generator 120. The demodulator can determine that a lower difference indicates that the nanomagnetometer is closer to the field generator 120, and similarly, a larger difference (e.g., a difference that satisfies a threshold) is farther away from the field generator 120. In response, the demodulator can identify the position of the nanomagnetometer 122 in the patient 106. The position of the nanomagnetometer can be identified, for example, with position coordinates (X, Y, Z) or other coordinates.

[0082] In some cases, the demodulator can use the strength of the magnetic field determined according to the measured phase shift to determine the position of the nanomagnetometer. The strength of the magnetic field can correspond to a specific position above the field generator 120, which indicates the X, Y, Z position of the nanomagnetometer to the demodulator. In some cases, the compared difference between the strength of the magnetic field emitted by the field generator 120 and the strength of the magnetic field determined according to the measured phase shift can indicate the X, Y, Z position of the nanomagnetometer to the demodulator. In some examples, the demodulator can access data that associates the strength of the magnetic field from the field generator 120 with the three-dimensional position above the field generator 120. In this way, the demodulator can refer to the stored data when interpreting the position of the nanomagnetometer based on the specific strength of the magnetic field from the field generator 120 or the difference between the strength of the magnetic field from the field generator 120 and the strength of the magnetic field from the measured phase shift.

[0083] In some embodiments, the system 100 may include a plurality of field generators 120. Each field generator may be located in a different position below the patient 106. In some cases, each field generator may emit a magnetic field having corresponding characteristics. The corresponding characteristics may include a specific amplitude, phase, frequency, and polarization. Each field generator may emit a magnetic field having different characteristics to distinguish the field generators from each other. The nanomagnetometer 122 may apply a phase shift on the reflected light according to the magnetic field emitted in different ways, and send the corresponding reflected light back to the demodulator 126. The demodulator 126 may interpret each of the different phase shifts from the reflected light to identify the corresponding emitted magnetic field from the corresponding field generator. In response, the demodulator 126 may triangulate the position of the nanomagnetometer 122 using a plurality of phase shifts of the reflected light. In some cases, for example, the demodulator 126 may triangulate the position of the nanomagnetometer 122 using the intensity of the corresponding emitted magnetic field, the transformation matrix of the X, Y, and Z positions, and other calculated data.

[0084] In some examples, the interrogator can utilize a model of the magnetic field strength and the emitted magnetic field. Using the model of the emitted magnetic field, the interrogator can utilize a nonlinear system to determine the 5DOF of the nanomagnetometer, which describes the position and orientation of the nanomagnetometer.

[0085] As shown in system 200, common component 202 may include a beam splitter, and nanomagnetometer component 206 may include multiple components. When the reflected phase-shifted light is transmitted along optical fiber 208, nanomagnetometer 206 may use a beam splitter and a polarizer. A polarizer is a filter that allows light waves of a specific polarization to pass through while blocking light waves of other polarizations. A beam splitter may be a device that can divide an incident light beam into two or more light beams, such as an optical power distribution device. Each split light beam may pass through a specific core of an optical fiber. The core is the physical medium of an optical fiber, which carries the light signal received from the attached light source and delivers it to a receiving device. For example, an optical fiber core is a continuous, thin strand of hair made of silica glass or plastic. A protective layer may surround the optical fiber core and may be a thin glass layer that forms a single solid glass fiber for light transmission. The protective layer forms a boundary that prevents light waves from spreading, thereby causing refraction. The coating is intended to absorb impact, provide protection to prevent excessive bending of the cable, and strengthen the optical fiber core. The primary cladding is essentially a layer of plastic that does not interfere with the optical transmission of the core or the protective layer.Nanomagnetometers can utilize a beam splitter in transmitting the reflected phase-shifted light to an interrogator.

[0086] In some embodiments, the demodulator can be configured to transmit multiple wavelengths of light and measure reflected light of multiple wavelengths. Specifically, the demodulator can be modified to extend its dynamic wavelength range, which covers the typical operating range of FOSS systems in addition to the operating wavelength of the nanomagnetometer. For example, the demodulator can use a light wavelength of 1310nm.

[0087] As shown in example 200, a tunable laser can transmit light along an optical fiber to a beam splitter. The tunable laser can be, for example, a 1310 nanometer (nm) laser or another laser with a different wavelength. The beam splitter can be, for example, a 2x2 beam splitter. The transmitted light can be delivered to a balanced detector in a common component 202 for measurement, to a FBG or Rayleigh core, and to a polarizer of a nanomagnetometer 206. After the transmitted light passes through the beam splitter, the transmitted light can pass through a polarizer and a 45-degree splicer in the nanomagnetometer component 206. The 45-degree splicer can equally divide the transmitted light into the horizontal axis and the vertical axis of the polarization-maintaining fiber core. The horizontal light and the vertical light are then delivered to a collimator. The collimator can narrow a particle beam or a beam. For example, the collimator can be a gradient refractive index (GRIN) lens. After the horizontal light component and the vertical light component pass through the collimator, the light is delivered to a quarter wave plate (QWP or λ / 4 wave plate). The QWP can convert two orthogonal polarizations into right-hand circularly polarized (RHCP) light and left-hand circularly polarized (LHCP) light.

[0088] RHCP light and LHCP light are then delivered to magneto-optical (MO) material. The back side of MO film can act as a reflector, which changes (flips) the polarization of RHCP light and LHCP light by introducing π phase shift in light. Therefore, the reflected RHCP light and the reflected LHCP light then pass through QWP, and the polarization traveling along the fast axis of the polarization-maintaining fiber core now moves along the slow axis, and vice versa. When the reflected RHCP light and the reflected LHCP light arrive at the 45 degree splicer, the reflected light can be reassociated and combined. Their addition can allow interference between two orthogonal polarizations. At this point, the Faraday rotation inside the nanocomposite material produces a phase shift between the orthogonal polarizations of the two reflections. Due to the design of the nanomagnetometer component 206 and the double pass through the MO material (that is, once in the forward direction and once when reflected), therefore, the phase difference caused by the Faraday rotation can be four times that of the single-pass phase change.

[0089] The reflected light then passes through a beam splitter and is provided to a balanced detector of the common component 202. The balanced detector can detect an interference pattern caused by a phase shift caused by the magnetic field between the two polarizations. The balanced detector can determine a phase shift reflecting the interference pattern. The determined phase shift can be provided to a demodulator and a computing system to determine the position coordinates of the nanomagnetometer based on the phase shift.

[0090] In some embodiments, the nanomagnetometer can be configured using a variety of configurations. In some examples, the nanomagnetometer can be a single component. In some examples, the nanomagnetometer can be implemented as a group of two or more miniaturized nanomagnetometers. These miniaturized nanomagnetometers can communicate with each of the three unused cores of the optical fiber respectively. In this example, by using three miniaturized nanomagnetometers, any possible crosstalk at the coupling point (position) between the nanomagnetometer and the optical fiber can be eliminated. In some examples, the three unused cores used by the miniaturized nanomagnetometer do not include Bragg gratings. The remaining four used cores may include embedded Bragg gratings. In some examples, the nanomagnetometer can be configured to significantly reduce or eliminate the overall crosstalk at the coupling point (position) using a specific core of the optical fiber.

[0091] In some examples, the nanomagnetometer can be implemented as a group of four miniaturized nanomagnetometers. These miniaturized nanomagnetometers can communicate with each of the four cores of the optical fiber in a one-to-one matching manner. Specifically, the four miniaturized nanomagnetometers can communicate with the three unused cores and the center core of the optical fiber. Similar to the above example with three miniaturized nanomagnetometers, in the example with four miniaturized nanomagnetometers, crosstalk at the coupling position can be eliminated.

[0092] In some examples, two additional nanomagnetometers can be used to provide a 6DOF electromagnetic tracking solution. In this case, since the MFOS core is spirally wound, multiple nanomagnetometers can be oriented differently from each other. They can have very similar positions within the MFOD, but have different orientations due to the spiral winding. Two or more orientations can be used to construct, for example, azimuth information, altitude, and roll information, while also providing multiple position solutions.

[0093] In some embodiments, the nanomagnetometer can be designed so that its diameter matches the diameter of its adjacent components. In more detail, the nanomagnetometer can be designed so that its diameter is suitable for the diameter of a variety of catheters and guidewires (such as the catheters and guidewires shown in system 100). In addition, the nanomagnetometer can be designed so that its diameter is suitable for the diameter of the optical fiber. For example, the diameter of the optical fiber can be between 195 microns (μm) and 460 μm.

[0094] In some examples, the nanomagnetometer can operate at a wavelength of 1310 nanometers (nm). In order for the nanomagnetometer to work with a FOSS component, the wavelength of the nanomagnetometer can be changed to adapt it to the operating range of the demodulator of the FOSS component. For example, the wavelength of the nanomagnetometer can be increased to 1510nm to fit the range of 1510nm to 1590nm of the FOSS component. In addition, by changing the wavelength of the nanomagnetometer to the lower limit of the range of the demodulator, the operation of the nanomagnetometer can match the wavelength distribution of the FOSS component, wherein the lower wavelength corresponds to the far end of the optical fiber. Similarly, by changing the wavelength of the nanomagnetometer to the upper limit of the range of the demodulator, the operation of the nanomagnetometer can match the wavelength distribution of the FOSS component, wherein the higher wavelength corresponds to the near end of the optical fiber.

[0095] The section of the optical fiber located within the guidewire 112 may include a number of components, structures, etc. Figure 3 An exemplary cross section of a multi-fiber sensor (MFOS) 300 is shown. The MFOS 300 can be, for example, a multi-core fiber sensor or a plurality of mechanically bundled fiber sensors. For example, the MFOS 300 can include a plurality of fiber cores 302, 304, 306, 308, 310, 312, and 314 (hereinafter referred to as "cores 302-314"). A subset of these fiber cores (e.g., three fiber cores) can perform three-dimensional shape sensing. Fewer degrees of freedom can be determined with fewer optical fibers. The MFOS 300 can include: a plurality of optical fiber cores, each surrounded by a protective layer; and a cladding for protection surrounding the entire MFOS 300. The plurality of fiber cores 302-314 is the physical medium of the optical fiber, which carries the optical signal received from the attached light source (e.g., the demodulator 126) and transmits it to the receiving device. For example, the fiber cores 302-314 can be continuous, hair-thin strands composed of silica glass or plastic. The protective layer is a thin layer of glass surrounding each of the cores 302-314, thereby forming a solid glass optical fiber for light transmission. The protective layer forms a boundary that prevents light waves from spreading and causing refraction, which enables data to be transmitted over the length of the optical fiber. The cladding is designed to absorb shock, provide protection from excessive bending of the cable, and strengthen the optical fiber cores 302-314. The cladding can be a layer of plastic that does not interfere with the light transmission or protective layer of the cores 302-314.

[0096] In some embodiments, the optical fiber may include more or fewer components than those shown in MFOS 300. For example, MFOS 300 may include multiple optical fibers with additional strength / position elements. The position elements may maintain a specific geometry and organization of the sensing optical fiber and provide additional support to the optical fiber. In some examples, MFOS 300 may be further encased in epoxy for strength. For example, MFOS 300 may be twisted during epoxy bonding to create a helical structure. When additional strength / position elements are added, the overall diameter of MFOS 300 may be larger.

[0097] In some embodiments, the MFOS 300 may be manufactured using a variety of configurations. As described above, the MFOS 300 may be combined with a Bragg grating and may include multiple cores. In some examples, seven is a common number for multiple cores. An estimate of the fiber shape requires at least three cores to be evenly distributed around the center of the fiber. For example, in the case of three cores, each core is positioned at an angle of 120 degrees. In some examples, four cores may be combined in the MFOS 300, wherein each core is positioned at an angle of 90 degrees. However, in some cases, when four cores are used, a fourth core may be positioned in the center of the fiber. The location of the fourth core in the center of the fiber enables the fourth core to be affected only by temperature, shear strain, and tensile strain, and not by the effects caused by bending. As a result, the core at the center of the fiber is used to distinguish between strains caused by bending and strains caused by temperature. This assumes that there are no effects caused by tensile strain and shear strain. In addition, the outer core of the MFOS 300 may be added to provide structural stability to the fiber. Thus, in the seven-core configuration of the MFOS 300, the three outer cores remain unused.

[0098] In some embodiments, the nanomagnetometer 122 can utilize multiple cores in the optical fiber core. The nanomagnetometer 122 can transmit the phase-shifted reflected light through three unused cores of the optical fiber. The three unused cores can be three of the outer cores, such as cores 304, 308, and 312. In some examples, these cores can be designed so that they are equipped with Bragg gratings. In some examples, these cores can be designed without Bragg gratings and can have microscopic defects instead.

[0099] In some embodiments, the nanomagnetometer 122 can transmit the phase-shifted reflected light through the central core of the optical fiber. The central core can be the core 314 of the optical fiber. By transmitting through the central core, the phase-shifted reflected light can be protected from the effects of temperature, shear strain, and tensile strain applied to the optical fiber.

[0100] In some embodiments, one or more optical adapters and one or more fan-out splicers may be incorporated into the optical fiber. Optical adapters and fan-out splicers may be used to facilitate coupling between the optical fiber and the nanomagnetometer 122. In more detail, the cores of the optical fibers are separated and maintained separately to manage physical connections. Optical adapters help manage the physical connection between the optical fiber and the nanomagnetometer 122. For example, in the case where the nanomagnetometer 122 transmits phase-shifted reflected light through the central core of the optical fiber, the cores of the optical fiber need to be combined at the contact point (position) where they contact the nanomagnetometer 122.

[0101] The attitude information (eg, position, orientation, shape) of the optical fiber may be defined, for example, by a variety of functions. Figure 4A and Figure 4B A randomly generated shape 400 of an MFOS is shown. The random 3-D shape is generated as a function of x(s), y(s), and z(s), which together represent the overall shape 400 (e.g., shape) of the MFOS. The random 3-D shape can be used, for example, to train a machine learning model. Once the machine learning model is fully trained, the trained machine learning model can be used to calculate the pose, shape, and / or position information of the optical fiber. The generated shape can have constraints such that the shape represents a typical usage scenario, length, curvature, etc. When generating the shape 400, initial conditions (e.g., initial position, orientation, etc.) can be set. In the example shown, the initial condition is the initial position and orientation of the first end of the MFOS. Once the shape is fully generated, T(s), N(s), and B(s) are calculated at discrete points s throughout the shape 400. T(s) defines the orientation of the center of the MFOS at each point s. N(s) defines the radial axis of the MFOS, which is perpendicular to T(s) at each point s. B(s) defines another radial axis of the MFOS that is perpendicular to T(s) and N(s) at each point s. T(s), N(s), and B(s) each define a vector and together define the orientation of the MFOS at each point s along the shape 400. The number of discrete points s throughout the shape 400 can vary based on the requirements for smoothness and are not necessarily equally spaced. Additional fiber cores in the MFOS are generated around the curve of the shape 400. In the embodiment shown, there are three cores 402, 404, 406. In other embodiments, there may be more or fewer fiber cores. By having a core at the center of the shape 400, the system can distinguish multiple properties from each other, for example, distinguishing strain from temperature. In some embodiments, there may be a core at the center of the shape 400.

[0102] Additional cores are generated by forming core curves as functions of s and t, where s parameterizes the length of the core and t parameterizes the orientation of the core. Each additional core has a corresponding curve. For example, for the case of three cores 402, 404, 406, there are three additional curves. Assuming that the cores are evenly spaced around a circle of radius r, where r is the radial distance of each core from the center of the shape, the position of each core relative to the center of shape 400 can be determined by equation (1). In this example, t can be three values ​​(e.g., 0, 2π / 3, 4π / 3) for evenly spacing the cores. As described above, shape 400 consists of three components x(s), y(s), and z(s), and there are also three components, namely each of T(s), N(s), and B(s).

[0103] Curve(s, t) = Shape(s) + N(s) r cos(t) + B(s) r sin(t) (1)

[0104] In the case where a core is generated around the center of shape 400, the angle and distance of the core relative to the center of shape 400 can be determined at each point s along shape 400. Figure 4B , the angle θ1 and the distances r1 and d1 can be calculated for the angle and distance of the first core 402 relative to the center of the MFOS. The angle θ1 and the radius r1 can be used as polar coordinates to define the position of the core relative to the center of the MFOS. b The direction of the bend in the MFOS can be defined. The bend axis 408 is relative to the bend axis angle θ b The distance d1 defines the distance between the first fiber core 402 and the bending axis 408 .

[0105] Once the angle and distance of each core relative to shape 400 is determined, for example at each point s along shape 400, the curvature κ of each individual core 402, 404, 406 can be determined at each point s along shape 400. The curvature κ is determined by performing a calculation for each core curve.

[0106] κ i (s) = ‖T′ i (s)‖ (2)

[0107] Once information about κ and d is determined, for example with respect to shape 400, the strain ε on each core, for example due to twisting and bending, can be determined. For example, this information can be used as training data. Figure 5 and Figure 6 As further defined, the training data can be used to train a machine learning model.

[0108] Figure 5 A block diagram for collecting properties in an optical fiber is shown, which graphically represents a fiber shape sensing (FOSS) system 500 using optical frequency domain reflectometry (OFDR) and MFOS 502. A tunable laser source 504 is linearly scanned over a limited wavelength range (e.g., about 1-1.5 μm). Light 506 emitted by the laser source 504 is split into two beams 508 and 510 by an optical coupler 512. Beam 508 follows a reference optical path 514 and is fixed by design. The reference optical path 514 receives beam 508 and returns beam 516. The second beam 510 is sent into the MFOS 502, and the MFOS returns beam 518, which carries information about the strain in the MFOS 502. The return beams 516 and 518 cause interference patterns to appear in the coupler 512. These interference patterns are represented in beam 520.

[0109] The beam 520 containing the interference pattern may be analyzed by a variety of methods. For example, the beam 520 may be analyzed for amplitude information, which is not phase sensitive. In this case, the interference represented in the beam 520 is measured by a photodetector 522. The photodetector 522 converts the light into electrical signals, which are then processed by a data acquisition system 524. The data acquisition system 524 may include, for example, one or more analog-to-digital (A / D) converters. One or more other signal processing techniques (e.g., filtering, etc.) may also be employed in the data acquisition system 524. The data processed by the data acquisition system 524 may be provided (e.g., uploaded) to a computer system 526. The computer system 526 may be trained by a machine learning (ML) process, wherein the data from the data acquisition system 524 may be converted into the shape of the MFOS 502. In some embodiments, the computer system 526 may determine the shape of the MFOS 502 using other computational processes in which the data is processed by the data acquisition system 524.

[0110] Another technique (e.g., phase sensitive technique) for analyzing the beam 520 containing the interference pattern is to analyze the amplitude information and the phase information. For example, the beam 520 can be separated into, for example, orthogonal polarizations by the photodetector 528. For example, the polarization component 530 refers to the component of the beam 520 perpendicular to the plane of incidence, while the polarization component 532 refers to the component of the beam 520 that is in the plane. Each component is measured by a corresponding photodetector 534, 536, which converts the corresponding light into electrical signals, which are then processed by the data acquisition system 538. The data acquisition system 538 can include, for example, one or more A / D converters and possible components to perform one or more other signal processing techniques (e.g., filtering). The converted signals from the data acquisition system 538 can be provided (e.g., uploaded) to the computer system 540. The computer system 540 can be trained by a machine learning (ML) process, wherein the data from the data acquisition system 538 can be converted into the shape of the MFOS 502.

[0111] The described computer systems (eg, computers 526, 540) may execute a training data generator that utilizes the captured data to determine the position and orientation of a surgical tool (or other object). Figure 6 , the computer system 610 executes the training data generator 600. The computer system 610 may be similar to Figure 1 The computer system 128. The training data generator 600 (e.g., program, software, software application, or code) includes machine instructions for a programmable processor and may be implemented in a high-level procedural programming language and / or an object-oriented programming language and / or in assembly / machine language. As used herein, the terms machine-readable medium and computer-readable medium refer to a computer program product, device, and / or apparatus (e.g., a disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions.

[0112] Figure 7 7 is a flow chart of a method 700 for computationally generating data to train a machine learning system to determine the shape of an MFOS. For example, a training data generator (e.g., Figure 6 The operation of the training data generator 600 of the method 700 may follow. First, a random 3-D shape is generated (702). For example, the random 3-D shape may be generated as a function of x(s), y(s), and z(s) to represent the overall shape generated. For example, reference Figure 4A and Figure 4BThe random 3-D shape is generated by the process described above. As described above, the generated shape can have constraints so that the shape represents a typical usage scenario, length, curvature, etc.

[0113] Next, initial conditions for the generated shape may be set 704. For example, the initial conditions may include an initial position and orientation of a first end of the shape. Other initial conditions may also be set (e.g., N(0), B(0)).

[0114] T(s), N(s), and B(s) can then be calculated at discrete points s throughout the generated shape (706). T(s) defines the orientation of the center of the shape at each point s. N(s) defines a radial axis of the shape that is perpendicular to T(s) at each point s. B(s) defines another radial axis of the shape that is perpendicular to T(s) and N(s) at each point s. T(s), N(s), and B(s) define the area of ​​the shape at each point s along the generated shape. The number of discrete points s for the entire shape can vary based on the requirements for smoothness and are not necessarily equally spaced.

[0115] Next, additional fiber cores are generated around the curve of the shape (708). For example, there may be three fiber cores. In other embodiments, there may be more or fewer fiber cores. In some embodiments, there may be a core at the center of the shape. Additional cores may be generated, for example, by forming a curve of the core as a function of s and t, where s parameterizes the length of the core and t parameterizes the orientation of the core. Each additional core may have its own curve. For example, where three cores are utilized, there may be three additional curves. Assuming that the cores are evenly spaced around a circle of radius r, where r is the radial distance of each core from the center of the shape, the position of each core relative to the center of the shape may be determined as described above.

[0116] The angle and distance of the core relative to the center of the shape can then be determined at discrete points s along the shape (710). For example, as described above, the angle θ and the distances r and d can be calculated for the angle and distance of the core relative to the center of the shape. The angle θ and the radius r can be used as polar coordinates to define the position of the core relative to the center of the shape. b The direction of the bend in the shape at point s can be defined.

[0117] Next, the curvature κ of each individual core may be determined (712). For example, as described above, the curvature of each individual core may be determined for each point s along the shape.

[0118] Then, the strain ε on each core may be determined (714). For example, the strain may be due to twisting and bending. The strain ε may be determined at each point s along the shape. The strain data may be represented as a vector or matrix, for example.

[0119] The strain ε on each core and the randomly generated shape can then be used as ML training data. For example, the random shape and strain data can be paired (716), and the ML training data can be used to train a machine learning system to determine the randomly generated shape based on the strain data. For example, method 700 can be repeated as needed to generate enough data to train the machine learning system. The machine learning system can be trained to receive the core strain as input and output the shape of the MFOS.

[0120] To implement the machine learning system, one or more machine learning techniques may be employed. For example, supervised learning techniques may be implemented, in which training is based on expected outputs known for inputs. Supervised learning may be viewed as an attempt to map inputs to outputs, and then estimate outputs for previously unseen inputs (newly introduced inputs). Unsupervised learning techniques may also be employed, in which training is provided based on known inputs but unknown outputs. Reinforcement learning techniques may also be used, in which the system may be viewed as learning from the consequences of the actions taken (e.g., the input values ​​are known and feedback provides a performance metric). In some arrangements, the techniques implemented may employ two or more of these approaches.

[0121] In some embodiments, a neural network technique can be implemented using data representing strain (e.g., a numerical matrix representing strain values ​​at each point s along a shape) to call a training algorithm for automatically learning the shape and related information. Such a neural network typically uses many layers. Once the number of units and layers of each layer are defined, the weights and thresholds of the neural network are typically set to minimize the prediction error by training the network. This technique of minimizing errors can be viewed as fitting the model (represented by the network) to the training data. By using strain data (e.g., a vector or matrix), a function of quantization error (e.g., a square error function used in regression techniques) can be defined. By minimizing the error, a neural network capable of determining the attributes of an input image can be developed. Other factors can also be considered during the neural network development process. For example, the model may try to fit the data too closely (e.g., the degree to which the modeling of the curve fitting to the overall function is degraded). This overfitting of the neural network may occur during model training, and one or more techniques may be implemented to reduce its impact.

[0122] A type of machine learning called deep learning can be used, in which a set of algorithms attempts to model high-level abstractions in the data by using a model architecture with a complex structure or other form composed of multiple nonlinear transformations. This deep learning technique can be regarded as a learning representation based on data. Generally, deep learning techniques can be regarded as using a cascade of multi-layer nonlinear processing units for feature extraction and transformation. The next layer uses the output of the previous layer as input. The algorithm can be supervised, unsupervised, a combination of supervised and unsupervised, etc. These techniques are based on learning multiple levels of features or representations of data (e.g., strain data). Therefore, multi-layer nonlinear processing units and supervised or unsupervised learning of representations can be used in each layer, wherein these layers form a hierarchy from low-level features to high-level features. By adopting this layer, many parameterized transformations are used as data is transferred from the input layer to the output layer. In one arrangement structure, the machine learning system uses a fifty-layer deep neural network architecture (e.g., ResNet50 architecture).

[0123] The machine learning system can be, for example, a neural network. In addition, multiple smaller neural networks can be put together sequentially to implement the functionality of a single large neural network. This allows the neural network functionality, strain measurements, bending and twisting calculations, shape calculations, and position calculations to be divided along the main FOSS technology blocks. For example, a neural network can act as a Fourier transformer. In some embodiments, it may be more effective to train a smaller network. For example, determining a regression model on a smaller data block may be more effective than determining a model on a larger data set.

[0124] Figure 8 8 is a flow chart of a method 800 for physically generating data to train a machine learning system to determine characteristics of an MFOS. These characteristics may include, for example, shape, location, positioning, and other characteristics. For example, a training data generator (e.g., Figure 6 The operation of the training data generator 600 of FIG. 8 may perform the method 800. In the method 800, rather than using a simulated shape, the training MFOS is set to a specific shape and the resulting strain is measured using, for example, the system 500 as described above.

[0125] First, a random shape is generated (802). For example, a random 3-D shape can be generated as a function of x(s), y(s), and z(s) to represent the overall shape generated. This can be used, for example, to refer to Figure 4A and Figure 4B As described above, the generated shape can have constraints so that the shape represents a typical usage scenario, length, curvature, etc.

[0126] Next, the physical MFOS is positioned in the randomly generated shape (804). For example, the robotic MFOS may position itself into the randomly generated shape. In another example, the user may position the MFOS into the generated shape.

[0127] Then, the shape of the MFOS can be measured (806). For example, the shape of the MFOS can be measured, for example, using a calibration system. Using a hardware device to measure the shape of the MFOS can improve the training data. For example, the shape of the MFOS may not be accurate for a randomly simulated shape. In addition, the measurement of the shape may not be accurate, for example, due to manufacturing tolerances. In addition to collecting training data, method 800 can also be used to calibrate a system (e.g., similar to system 500). In some examples, depending on the context of training the ML training data, the position of the MFOS, the positioning of the MFOS, and / or other data related to the MFOS can be measured and used for the ML training data.

[0128] Next, strains in the MFOS may be measured (808). For example, a system similar to system 500 may be used to measure strains in the MFOS. Strain ε may be measured at discrete points s along the shape. The strains and measured shapes may then be used as ML training data. For example, a random shape may be paired with strain data (810), and the ML training data may be used to train a machine learning system to determine the randomly generated shape from the strain data. Method 800 may be repeated as needed to generate enough data to train the machine learning system. The machine learning system may be trained to receive strain data as input and output the shape of the MFOS. For example, the machine learning system may be trained similar to machine learning system 610, as described above. The position, shape, and / or location of the MFOS may also be measured and used as ML training data.

[0129] The training data collected by the above methods (e.g., by training data generator 600) can be used to train a machine learning system. For example, one or more techniques can be implemented to determine shape information based on strain provided to a computer system (e.g., computer system 128). For such techniques, information from one or more data sources can be used. For example, data representing strain throughout the fiber (e.g., strain data) can be generated. For one type of data collection method, training data can be generated using simulated shapes (e.g., similar to Figure 7 Method 700).

[0130] In addition to modeling shape, other techniques may be used in conjunction to determine shape information. One or more forms of artificial intelligence (e.g., machine learning) may be employed so that a computing process or device can learn to determine shape information based on training data without being explicitly programmed for that task. Using this training data, machine learning may estimate shape information using techniques such as regression. To produce such an estimate, one or more quantities may be defined as measures of shape information. For example, strain levels in two locations may be defined. Such strains may be defined using one or more conventions. Once trained, the learning machine may be able to output a numerical value representing the shape between the two locations. The input to a trained learning machine may take one or more forms. For example, a representation of strain data may be provided to a trained learning machine. One type of representation may be a phase-sensitive representation of the strain data (e.g., containing amplitude information and phase information, similar to Figure 5 ). Another type of representation may be a phase-insensitive representation of the strain data (e.g., containing only amplitude information). In some arrangements, the machine learning system may be able to render an image based on the input provided. Once rendered, the image may be used to determine the shape of the fiber. The machine learning system may also be able to render one or more components (e.g., x(s), y(s), and z(s)) based on the input provided. Once rendered, these components may define the shape of the fiber, for example, along an axis.

[0131] In order to implement this environment, one or more machine learning techniques can be used. For example, supervised learning techniques can be implemented, wherein the training is based on the expected output known for the input. Supervised learning can be regarded as an attempt to map the input to the output and then estimate the output for the previously unused input. Unsupervised learning techniques can also be used, wherein the training is provided based on known inputs but unknown outputs. Reinforcement learning techniques can also be used, wherein the system can be regarded as learning from the consequences of the actions taken (for example, the input values ​​are known and the feedback provides a performance metric). In some arrangements, the implemented techniques can adopt two or more of these methods. For example, the applied learning can be regarded as not strictly supervised learning because the shape can be considered unknown before the calculation is performed. Although the shape is unknown, the implemented techniques can check the strain data in accordance with the collected shape data (for example, wherein the simulated shape is connected to certain strain data). By using two sources of information about shape information, it can be considered that reinforcement learning techniques are being implemented.

[0132] In some arrangements, neural network techniques can be implemented using training data as well as shape data (e.g., a numerical vector representing the shape) to call a training algorithm to automatically learn the shape and related information, such as strain data. Such neural networks typically employ many layers. Once the number of units and layers of each layer are defined, the weights and thresholds of the neural network are typically set to minimize the prediction error by training the network. This technique of minimizing the error can be viewed as fitting the model (represented by the network) to the training data. Using shape data and strain data, a function that quantifies the error (e.g., a squared error function used in regression techniques) can be defined. By minimizing the error, a neural network capable of estimating shape information can be developed. Other factors can also be considered during the neural network development process. For example, the model may attempt to fit the data too closely (e.g., the degree to which the modeling of the curve fitting to the overall function is degraded). This overfitting of the neural network may occur during model training, and one or more techniques may be implemented to reduce its impact.

[0133] like Fig. 9 As shown, shape manager 900 (which includes multiple modules) is executed by server 906 present at computing environment 902. In this arrangement, shape manager 900 includes data generator 904, which can collect training data. In this arrangement, such data can be pre-stored (e.g., in a strain database) and retrieved from storage device 908. Data representing such shape information can also be retrieved from one or more sources external to computing environment 902; for example, such information can be obtained from one or more storage devices of the shape manager (e.g., an entity separate from computing environment 902). Along with the strain data, storage device 908 (or other storage device at computing environment 902) can include a shape database. For example, storage device 908 includes simulated shape database 910 and physical shape database 912, the simulated shape database 910 including data generated computationally (e.g., by Figure 7 The physical shape database 912 contains shape data that is physically generated (e.g., by Figure 8 The shape database 910, 912 may include information about a number of previously determined shapes, recently determined shapes, and the like. For example, based on the information stored in the shape database 910, 912, data may be retrieved for training and use by a learning machine to determine shape information (e.g., the shape of an optical fiber, etc.). For example, the shape database 910, 912 may include data representing multiple types of shape information (e.g., the shape presented, components on the x-axis, y-axis, and z-axis, etc.).

[0134] To train a learning machine (e.g., implemented as a neural network), the shape manager 900 includes a shape learning machine trainer 914 that uses simulated shapes and physical shapes to perform training operations. In some arrangements, the trainer 914 can calculate a numerical representation (e.g., in vector form) of the strain data used for machine training.

[0135] like Fig. 9 As shown in , the shape learning machine trainer 914 may also provide other types of functionality. For example, the shape learning machine trainer 914 may store shape features (e.g., calculated feature vectors) in the shape feature database 916 for later retrieval and use. Such shape feature data may be obtained from sources other than the shape learning machine trainer 914. For example, the shape learning machine 918 may also store data representing shape features in the shape feature database 916. In some arrangements, such shape features may be provided directly to the shape learning machine trainer 914, shape learning machine 918, etc., and stored accordingly in the shape feature database 916. In other arrangements, calculations may be performed by the shape learning machine trainer 914, shape learning machine 918, etc. to generate shape features, which are then stored in the shape feature database 916. For example, a numerical value representing one or more shape features (e.g., feature vectors) may be calculated by the shape learning machine trainer 914, shape learning machine 918, etc. based on strain data. As shown in the figure, such stored shape feature data can exist in storage device 908 (e.g., in shape feature database 916). Such shape feature data can be provided to or received from other locations within or outside of computing environment 902. For example, the data can be provided to other systems that are remotely located relative to computing environment 902 for further analysis, storage, etc.

[0136] Typically, the shape learning machine trainer 914 can use one or more techniques to generate the shape learning machine 918 (e.g., a neural network). For example, strain data for each shape in the shape database can be used to define a function. The shape learning machine 918 can be trained by determining the shape based on the provided strain data.

[0137] Once trained, the shape learning machine 918 can be used to determine the shape of the optical fiber based on strain data (not used to train the machine). For example, the strain data can be provided to the shape learning machine 918. For example, a numerical representation of the strain data (e.g., a vector) can be input, and the shape learning machine 918 can calculate the shape components of the optical fiber (e.g., the components x(s), y(s), and z(s) of the shape). Based on the calculated shape components, the shape learning machine 918 can render a 3D representation of the shape.

[0138] exist Fig. 9 In the example shown in , the functionality of the data generator 904, the shape learning machine trainer 914, and the shape learning machine 918 are represented as being included in the shape manager 900. However, in some arrangements, the functionality of one or more of these modules may be provided outside the computing environment 902. Likewise, in this example, the simulated shape database 910, the physical shape database 912, and the shape feature database 916 are stored in the storage device 908. However, one or more of these databases may be stored outside the storage device 908, and in some arrangements, one or more of these databases may be stored outside the computing environment 902. In some arrangements, the shape manager 900 may be implemented in software, hardware, or a combination of hardware and software. Likewise, the modules included in the shape manager 900 may be implemented separately in hardware and / or software. One or more database technologies (e.g., structural representation, etc.) may be used to store the databases 910, 912, 916.

[0139] Fig.10 is a data flow representing the operation of the shape learning machine after initial training. As described above, training data from a variety of sources may be used; for example, simulated shapes (e.g., such as using Figure 7 ), physical shapes (e.g., such as those generated using methods such as method 700 shown in Figure 8 800 shown in ), simulated shapes, and physical MFOS can all be used to generate training data. The strain data 1002 can be provided to a shape learning machine 1004 running in a computer system 1008. The shape learning machine 1004 can determine the shape based on the strain data. Fig.10 The representative strain data 1002 is input into a shape learning machine 1004 and generates an output 1006 representing how the MFOS is shaped. In some arrangements, additional information (e.g., initial conditions) may also be input into the shape learning machine 1004. The output 1006 may include functions of x(s), y(s), and z(s). In addition, a value representing a confidence level (e.g., a level varying from 0.0 to 1.0) may be output for each function. In some arrangements, the output 1006 may be a 3D rendering of the MFOS. In some examples, the shape learning machine 1004 may be configured to generate position and / or pose information of the MFOS.

[0140] Fig.111 is a flowchart 1100 for determining an optical fiber shape. Flowchart 1100 may generally be performed by a single computing device (e.g., computing system 128). However, multiple computing devices (e.g., demodulator 126, control unit 116, sensor interface unit 118, computing system 128, etc.) may perform the process illustrated by flowchart 1100. In addition to being performed at a single site, the execution of flowchart 1100 may also be distributed between two or more locations on a network.

[0141] A computing device may obtain an optical signal reflected through an optical fiber, which may include one or more embedded sensors and a nanomagnetometer (1102) embedded at a distal position of the optical fiber. The computing device may command a tunable laser to transmit an optical signal through the optical fiber, the optical signal including light having a certain wavelength range. The one or more embedded sensors may include one or more fiber Bragg gratings embedded at multiple positions along the optical fiber. The Bragg gratings may be designed and / or configured to reflect a specific wavelength called a Bragg wavelength, such as a frequency shift. The Bragg wavelength may be a function of optical fiber strain. The nanomagnetometer may be embedded at the distal end of the optical fiber. The optical fiber may include multiple cores.

[0142] The nanomagnetometer may reflect a portion of the optical signal through the cores of the first subset, and the Bragg grating may reflect a portion of the optical signal through the cores of the second subset, the cores of the first subset being different from the cores of the second subset. For example, cores 314, 304, 308, and 312 may be used for shape sensing, while cores 306, 302, and 310 may be used for communications caused by the nanomagnetometer (e.g., magnetic tracking).

[0143] The computing device may determine a frequency shift of each of the reflected optical signals, the frequency shift imposed on the reflected optical signals by one or more embedded sensors (1104). Specifically, the computing device may use the determined frequency shift of each of the reflected optical signals to determine measurements of bending at multiple locations along the optical fiber. As described above, the Bragg grating is configured to reflect a specific wavelength when transmitted light is reflected from the Bragg grating. The specific wavelength is a function of optical fiber strain. A demodulator may receive the optical signals reflected from the optical fiber, and the demodulator may measure the frequency shift of each of the reflected optical signals by comparing a specified wavelength (e.g., a Bragg wavelength) with the wavelength of the transmitted light.

[0144] The computing device may determine a phase shift of each of the reflected optical signals, which is imposed on the reflected optical signals by the nanomagnetometer (1106). In response to the demodulator receiving the optical signal reflected from the optical fiber, the demodulator may measure the phase shift of each of the reflected optical signals. The demodulator may provide an estimated phase shift of each of the reflected optical signals to the computing system. The computing system may determine one or more magnetic fields sensed by the nanomagnetometer based on the determined phase shift of each of the reflected optical signals. Based on the determined one or more magnetic fields, the computing system may identify the position of the nanomagnetometer located at a distal position of the optical fiber in the patient's body. This position may be used as an external reference for the position of the optical fiber.

[0145] The computing device may use the determined frequency and phase shift of each of the reflected optical signals to determine one or more characteristics of the optical fiber (1108). The one or more characteristics of the optical fiber include the shape of the optical fiber and the position of the optical fiber relative to an external reference. In response to measuring the estimated frequency shift and the estimated phase shift of the reflected optical signal, the computing device may determine a variety of characteristics of the optical fiber. For example, the estimated frequency shift may indicate to the computing device a measurement of bending at multiple locations along the optical fiber. Specifically, the determined frequency shift may indicate the location of the strain of the optical fiber and the location of each of the one or more embedded sensors in the optical fiber. The computing device may determine the location of each of the measured optical fiber strains by associating the determined frequency shift with the determined location of each of the one or more embedded sensors.

[0146] In addition, based on the strain determined at each of its corresponding positions on the optical fiber, the computing system can estimate the shape of the optical fiber. The shape of the optical fiber can be estimated by combining the curvature of the optical fiber and the angle of the curvature at each position of the strain. For each position, the curvature of the optical fiber and the angle of the curvature can be determined based on the strain determined at each position indicated by the Bragg grating. The estimated shape of the optical fiber can indicate the shape of the optical fiber in three-dimensional spatial coordinates. And, the estimated shape of the optical fiber can include the orientation of the center of the optical fiber, the first radial axis of the optical fiber, and the second radial axis of the optical fiber. Other characteristics of the optical fiber can also be determined.

[0147] For example, in response to estimating the shape of the optical fiber, the computing device can determine the position of the optical fiber within the patient. The position of the optical fiber can be determined using the estimated shape of the optical fiber and the identified position of the nanomagnetometer, which represents an external reference and contributes to the overall position identification of the optical fiber.

[0148] In some embodiments, a machine learning model can be trained to predict one or more characteristics of an optical fiber. The machine learning model can be trained using a variety of data types (e.g., simulation data, physical data, estimated frequency shifts, estimated phase shifts, strain estimates, curvature of the optical fiber, and other characteristics). Once the model is fully trained, the trained machine learning model can be deployed and used at a computing system to help determine the location of the optical fiber.

[0149] Fig.12 12 is a block diagram of computing devices 1200, 1250 that can be used as a client or server or multiple servers to implement the systems and methods described in this document. Computing devices 1200 and 1250 are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown here, their connections and relationships, and their functions are exemplary only and are not intended to limit the embodiments described and / or claimed in this document.

[0150] The computing device 1200 includes a processor 1202, a memory 1204, a storage device 1206, a high-speed interface 1208 connected to the memory 1204 and a high-speed expansion port 1210, and a low-speed interface 1212 connected to a low-speed bus 1214 and the storage device 1206. Each of the components 1202, 1204, 1206, 1208, 1210, and 1212 is interconnected using a variety of buses and can be installed on a general motherboard or installed in other ways as appropriate. The processor 1202 can process instructions for execution within the computing device 1200, including instructions stored in the memory 1204 or on the storage device 1206 to display graphical information of a GUI on an external input / output device (e.g., a display 1216 coupled to the high-speed interface 1208). In other embodiments, multiple processors and / or multiple buses and multiple memories and multiple types of memories can be used as appropriate. Additionally, multiple computing devices 1200 may be connected, with each device providing portions of the necessary operations, for example, as a server bank, a group of blade servers, or a multi-processor system.

[0151] Memory 1204 stores information within computing device 1200. In one embodiment, memory 1204 is a computer-readable medium. In one embodiment, memory 1204 is one or more volatile memory units. In another embodiment, memory 1204 is one or more non-volatile memory units.

[0152] Storage device 1206 is capable of providing mass storage for computing device 1200. In one embodiment, storage device 1206 is a computer readable medium. In various different embodiments, storage device 1206 may be a floppy disk device, a hard disk device, an optical disk device or a magnetic tape device, a flash memory or other similar solid state memory device or device array, including a device in a storage area network or other configuration. In one embodiment, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, implement one or more methods, such as the above-described methods. The information carrier is a computer or machine readable medium, such as memory 1204, storage device 1206, or memory on processor 1202.

[0153] The high-speed controller 1208 manages bandwidth-intensive operations of the computing device 1200, while the low-speed controller 1212 manages less bandwidth-intensive operations. This division of responsibilities is exemplary only. In one embodiment, the high-speed controller 1208 is coupled to the memory 1204, the display 1216, for example, through a graphics processor or accelerator, and is coupled to the high-speed expansion ports 1210, which can accept a variety of expansion cards (not shown). In this embodiment, the low-speed controller 1212 is coupled to the storage device 1206 and the low-speed expansion ports 1214. The low-speed expansion ports, which can include a variety of communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices (e.g., keyboards, pointing devices, scanners) or network devices (e.g., switches or routers), for example, through a network adapter.

[0154] The computing device 1200 can be implemented in a variety of different forms, as shown. For example, it can be implemented as a standard server 1220, or multiple times in a group of such servers. It can also be implemented as part of a rack server system 1224. In addition, it can be implemented in a personal computer such as a laptop computer 1222. Alternatively, components from the computing device 1200 can be combined with other components in a mobile device (not shown) such as device 1250. Each of such devices can contain one or more of the computing devices 1200, 1250, and the entire system can be composed of multiple computing devices 1200 and 1250 that communicate with each other.

[0155] The computing device 1250 includes a processor 1252, a memory 1264, an input / output device such as a display 1254, a communication interface 1266, and a transceiver 1268, among other components. The device 1250 may also be provided with a storage device (e.g., a microdrive) or other device to provide additional storage. Each of the components 1250, 1252, 1264, 1254, 1266, and 1268 are interconnected using various buses, and several components may be mounted on a common motherboard or otherwise as appropriate.

[0156] The processor 1252 can process instructions for execution within the computing device 1250, including instructions stored in the memory 1264. The processor can also include separate analog processors and digital processors. The processor can provide, for example, coordination of other components of the device 1250, such as control of a user interface, applications run by the device 1250, and wireless communications performed by the device 1250.

[0157] The processor 1252 can communicate with the user through a control interface 1258 and a display interface 1256 coupled to a display 1254. The display 1254 can be, for example, a TFT LCD display or an OLED display or other suitable display technology. The display interface 1256 may include appropriate circuits for driving the display 1254 to present graphics and other information to the user. The control interface 1258 can receive commands from the user and convert them for submission to the processor 1252. In addition, an external interface 1262 in communication with the processor 1252 can be provided to enable near-field communication of the device 1250 with other devices. For example, the external interface 1262 can provide wired communication, for example via a docking process, or wireless communication, for example via Bluetooth or other such technologies.

[0158] Memory 1264 stores information in computing device 1250. In one embodiment, memory 1264 is a computer-readable medium. In one embodiment, memory 1264 is one or more volatile memory units. In another embodiment, memory 1264 is one or more non-volatile memory units. An extended memory 1274 may also be provided and connected to device 1250 via an extended interface 1272, which may include, for example, a SIMM card interface. Such extended memory 1274 may provide additional storage space for device 1250, or may also store applications or other information for device 1250. Specifically, extended memory 1274 may include instructions for executing or supplementing the above-mentioned processes, and may also include security information. Therefore, for example, extended memory 1274 may be provided as a security module for device 1250, and may be programmed with instructions that allow secure use of device 1250. In addition, secure applications and additional information may be provided via a SIMM card, such as placing identification information on a SIMM card in an unbreakable manner.

[0159] The memory may include, for example, flash memory and / or MRAM memory, as described below. In one embodiment, a computer program product is tangibly embodied in an information carrier. The computer program product comprises instructions that, when executed, implement one or more methods, such as the above methods. The information carrier is a computer or machine readable medium, such as the memory 1264, the expansion memory 1274, or a memory on the processor 1252.

[0160] The device 1250 can communicate wirelessly via a communication interface 1266, which can include digital signal processing circuitry, where necessary. The communication interface 1266 can provide communication in a variety of modes or protocols, such as GSM voice calls, SMS, EMS or MMS messages, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, etc. Such communication can be performed, for example, via a radio frequency transceiver 1268. In addition, short-range communications can be performed, for example, using Bluetooth, WiFi, or other such transceivers (not shown). In addition, a GPS receiver module 1270 can provide additional wireless data to the device 1250, which can be used as appropriate by applications running on the device 1250.

[0161] Device 1250 may also communicate audibly using audio codec 1260, which may receive voice information from a user and convert it into usable digital information. Audio codec 1260 may also generate audible sounds for the user, such as through a speaker, such as in a handset of device 1250. Such sounds may include sounds from voice phone calls, may include recorded sounds, such as voice messages, music files, etc., and may also include sounds generated by applications running on device 1250.

[0162] The computing device 1250 can be implemented in many different forms, as shown in the figure. For example, it can be implemented as a cellular phone 1280. It can also be implemented as part of a smart phone 1282, a personal digital assistant or other similar mobile device.

[0163] Various embodiments of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs, computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be special purpose or general purpose, coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0164] These computer programs (also referred to as programs, software, software applications or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or in assembly / machine languages. As used herein, the terms "machine-readable medium", "computer-readable medium" refer to any computer program product, device and / or means for providing machine instructions and / or data to a programmable processor, such as a disk, an optical disk, a memory, a programmable logic device (PLD), the programmable processor including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0165] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointer device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic input, voice input, or tactile input.

[0166] The systems and techniques described herein may be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0167] A computing system may include clients and servers. Clients and servers are generally remote from each other and generally interact through a communication network. The relationship of client and server arises through computer programs running on the respective computers and having a client-server relationship with each other.

[0168] In addition, certain data may be processed in one or more ways prior to storage or use so that personally identifiable information is removed. For example, in some embodiments, the identity of a user may be processed so that personally identifiable information about the user cannot be determined, or the geographic location of a user may be summarized where location information (e.g., city, zip code, or state level) is obtained so that the specific location of the user cannot be determined. Thus, a user may have control over what information is collected about the user, how that information is used, and what information is provided to the user.

[0169] Many embodiments have been described. However, it should be understood that various modifications may be made without departing from the spirit and scope of the invention. Therefore, other embodiments are within the scope of the appended claims. Although this patent specification contains details of many specific implementations, these should not be construed as limitations on the scope of what may be claimed, but rather as descriptions of features that may be unique to specific embodiments. Certain features described in this patent specification in the context of separate embodiments may also be implemented in combination in a single embodiment.

[0170] Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any applicable subcombination. Furthermore, although features may be described above as functioning in certain combinations, or even initially claimed, in some cases one or more features from a claimed combination may be deleted from that combination, and a claimed combination may involve subcombinations or variations of subcombinations.

[0171] Equally, although the operation is described in a specific order in the figure, this should not be understood as requiring to perform these operations in the specific order shown or in a sequential order, or requiring to perform all the operations shown to obtain the desired result. In some cases, multitasking and parallel processing can be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring this separation in all embodiments, and it should be understood that the described program components and systems can be integrated into a single software product or packaged into multiple software products.

[0172] Specific embodiments of the subject matter have been described. Other embodiments are also within the scope of the appended claims. For example, the actions described in the claims can be performed in a different order and still achieve the desired results. As an example, some of the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results.

Claims

1. A method implemented by a computer device, comprising: obtaining an optical signal reflected through an optical fiber, the optical fiber comprising one or more embedded sensors and a nanomagnetometer embedded at a distal location of the optical fiber; determining a frequency shift of each of the reflected optical signals, the frequency shift imposed on the reflected optical signals by the one or more embedded sensors; determining a phase shift of each of the reflected optical signals, the phase shift imposed on the reflected optical signals by the nanomagnetometer; as well as One or more characteristics of the optical fiber are determined using the determined frequency shift and the phase shift of each of the reflected optical signals, wherein the one or more characteristics of the optical fiber include a shape of the optical fiber and a position of the optical fiber relative to an external reference.

2. The computer-implemented method of claim 1, wherein: Determining the one or more characteristics of the optical fiber includes determining the position of the optical fiber using at least the determined phase shift of each of the reflected optical signals.

3. The computer-implemented method of claim 1, wherein: Determining the one or more characteristics of the optical fiber includes determining the shape of the optical fiber using at least the determined frequency shift of each of the reflected optical signals.

4. The computer-implemented method of claim 1, wherein: The method also includes commanding a tunable laser to transmit the optical signal through the optical fiber.

5. The computer-implemented method of claim 4, wherein: The optical signal includes light having a range of wavelengths.

6. The computer-implemented method of claim 1, wherein: Determining the phase shift of each of the reflected optical signals further comprises: determining one or more magnetic fields sensed by the nanomagnetometer based on the determined phase shift of each of the reflected optical signals; and The determined one or more magnetic fields are used to identify a position of the nanomagnetometer at the distal location of the optical fiber.

7. The computer-implemented method of claim 6, wherein: Determining the frequency shift of each of the reflected optical signals further includes determining measurements of bending at a plurality of locations along the optical fiber using the determined frequency shift of each of the reflected optical signals.

8. The computer-implemented method of claim 7, wherein: Determining the one or more characteristics of the optical fiber using the determined frequency shift and the determined phase shift of each of the reflected optical signals further comprises: determining a strain in the optical fiber using the determined measurements of the bend, the measurements of the bend being determined at the plurality of locations along the optical fiber using the determined frequency shifts, wherein the determined frequency shift of each of the reflected optical signals is indicative of: (i) the location of the strain in the optical fiber and (ii) the location of each of the one or more embedded sensors in the optical fiber; The one or more characteristics of the optical fiber include (i) the curvature of the optical fiber at the location of the strain, (ii) the angle of the curvature at the location of the strain, (iii) the shape of the optical fiber, and (iv) the location of the optical fiber.

9. The computer-implemented method of claim 8, wherein: The method further comprises: estimating the shape of the optical fiber by combining the curvature of the optical fiber and the angle of the curvature at each location of the strain; and The position of the optical fiber is determined using (i) the estimated shape of the optical fiber and (ii) the identified position of the nanomagnetometer, wherein the external reference represents the identified position of the nanomagnetometer.

10. The computer-implemented method of claim 9, wherein The estimated shape of the optical fiber indicates the shape of the optical fiber in three-dimensional space coordinates, and in, The estimated shape of the optical fiber includes one or more of an orientation of a center of the optical fiber, a first radial axis of the optical fiber, and a second radial axis of the optical fiber.

11. The computer-implemented method of claim 1, wherein: The one or more embedded sensors include one or more fiber Bragg gratings (FBGs) embedded at a plurality of locations along the optical fiber.

12. The computer-implemented method of claim 1, wherein: The optical fiber includes a plurality of cores, the nanomagnetometer reflects a portion of the optical signal through a core of a first subset of the plurality of cores, and the one or more embedded sensors reflect a portion of the optical signal through a core of a second subset, wherein the core of the first subset is different from the core of the second subset.

13. The computer-implemented method of claim 1, wherein: The method further comprises: training a machine learning model to predict the one or more characteristics of the optical fiber using the determined frequency shift and the phase shift of each of the reflected optical signals; and Apply the trained machine learning model.

14. The computer-implemented method of claim 1, wherein: The method further comprises: The wavelength of the nanomagnetometer is varied within the operating range of the interrogator.

15. The computer-implemented method of claim 1, wherein: The optical fiber is positioned in a surgical environment.

16. A system comprising: One or more computers and one or more storage devices storing instructions, which, when executed by the one or more computers, are operable to cause the one or more computers to perform operations including: obtaining an optical signal reflected through an optical fiber, the optical fiber comprising one or more embedded sensors and a nanomagnetometer embedded at a distal location of the optical fiber; determining a frequency shift of each of the reflected optical signals, the frequency shift imposed on the reflected optical signals by the one or more embedded sensors; determining a phase shift of each of the reflected optical signals, the phase shift imposed on the reflected optical signals by the nanomagnetometer; as well as One or more characteristics of the optical fiber are determined using the determined frequency shift and the phase shift of each of the reflected optical signals, wherein the one or more characteristics of the optical fiber include a shape of the optical fiber and a position of the optical fiber relative to an external reference.

17. The system of claim 16, wherein: Determining the one or more characteristics of the optical fiber includes determining the position of the optical fiber using at least the determined phase shift of each of the reflected optical signals.

18. The system of claim 16, wherein: Determining the one or more characteristics of the optical fiber includes determining a shape of the optical fiber using at least the determined frequency shift of each of the reflected optical signals.

19. The system of claim 16, wherein: The system also includes commanding a tunable laser to transmit the optical signal through the optical fiber.

20. The system of claim 19, wherein: The optical signal includes light having a range of wavelengths.

21. The system of claim 16, wherein: Determining the phase shift of each of the reflected optical signals further comprises: determining one or more magnetic fields sensed by the nanomagnetometer based on the determined phase shift of each of the reflected optical signals; and The determined one or more magnetic fields are used to identify a position of the nanomagnetometer at the distal location of the optical fiber.

22. The system of claim 21, wherein: Determining the frequency shift of each of the reflected optical signals further includes determining measurements of bending at a plurality of locations along the optical fiber using the determined frequency shift of each of the reflected optical signals.

23. The system of claim 22, wherein: Determining the one or more characteristics of the optical fiber using the determined frequency shift and the determined phase shift of each of the reflected optical signals further comprises: determining strain in the optical fiber using the determined measurements of the bend, the measurements of the bend being determined at the plurality of locations along the optical fiber using the determined frequency shifts, wherein the determined frequency shift of each of the reflected optical signals is indicative of: (i) the location of the strain in the optical fiber and (ii) the location of each of the one or more embedded sensors in the optical fiber, The one or more characteristics of the optical fiber include (i) the curvature of the optical fiber at the location of the strain, (ii) the angle of the curvature at the location of the strain, (iii) the shape of the optical fiber, and (iv) the location of the optical fiber.

24. The system of claim 23, wherein: The system further comprises: estimating the shape of the optical fiber by combining the curvature of the optical fiber and the angle of the curvature at each location of the strain; and The position of the optical fiber is determined using (i) the estimated shape of the optical fiber and (ii) the identified position of the nanomagnetometer, wherein the external reference represents the identified position of the nanomagnetometer.

25. The system of claim 24, wherein The estimated shape of the optical fiber indicates the shape of the optical fiber in three-dimensional spatial coordinates, and in, The estimated shape of the optical fiber includes one or more of an orientation of a center of the optical fiber, a first radial axis of the optical fiber, and a second radial axis of the optical fiber.

26. The system of claim 16, wherein: The one or more embedded sensors include one or more fiber Bragg gratings (FBGs) embedded at a plurality of locations along the optical fiber.

27. The system of claim 16, wherein: The optical fiber includes a plurality of cores, the nanomagnetometer reflects a portion of the optical signal through a core of a first subset of the plurality of cores, and the one or more embedded sensors reflect a portion of the optical signal through a core of a second subset, wherein the core of the first subset is different from the core of the second subset.

28. The system of claim 16, wherein: The system further comprises: training a machine learning model to predict the one or more characteristics of the optical fiber using the determined frequency shift and the phase shift of each of the reflected optical signals; and Apply the trained machine learning model.

29. The system of claim 16, wherein: The system further comprises: The wavelength of the nanomagnetometer is varied within the operating range of the interrogator.

30. A non-transitory computer-readable medium storing software, the software comprising instructions executable by one or more computers, the instructions, upon execution, causing the one or more computers to perform operations comprising: obtaining an optical signal reflected through an optical fiber, the optical fiber comprising one or more embedded sensors and a nanomagnetometer embedded at a distal location of the optical fiber; determining a frequency shift of each of the reflected optical signals, the frequency shift imposed on the reflected optical signals by the one or more embedded sensors; determining a phase shift of each of the reflected optical signals, the phase shift imposed on the reflected optical signals by the nanomagnetometer; and one or more characteristics of the optical fiber are determined using the determined frequency shift and the phase shift of each of the reflected optical signals, wherein, The one or more characteristics of the optical fiber include a shape of the optical fiber and a position of the optical fiber relative to an external reference.