Visual perception system based on optical fiber

Abstract

The invention relates to a visual perception system based on optical fibers. The visual perception system comprises a plurality of perception units and a photosensitive chip, each perception unit comprises a micro lens and a multimode conical optical fiber, the micro lens is arranged on the optical fiber end face of the multimode conical optical fiber, an evanescent field part of the multimode conical optical fiber is opposite to the photosensitive chip in the vertical direction, and the distance between the evanescent field part and the photosensitive chip is 0 mm. The multimode tapered fiber includes at least two tapered regions. The visual perception system has the advantages of being simple in structure, small in size, wide in view field, capable of simultaneously obtaining light information of a high-dimensional light field in a short time and achieving high-precision light information reconstruction and the like.

Description

Technical Field

[0001] This invention relates to the field of visual perception technology, and more specifically to a fiber optic-based visual perception system. Background Technology

[0002] Wide field-of-view visual perception has important application prospects in fields such as autonomous driving, intelligent navigation of drones, precision environmental monitoring, and biomedical imaging.

[0003] However, in traditional visual perception systems, light intensity is typically integrated only within a broadband integral, without independent sampling of the electric field direction and specific wavelength. To achieve multi-dimensional optical information perception, a combination of discrete optical modules is usually required, such as using gratings or interference structures for spectral resolution and waveplates and polarizers for polarization measurement. Consequently, existing systems generally suffer from complex structures, large size, high alignment requirements, and time-consuming measurements. They also struggle to achieve simultaneous acquisition of high-dimensional information, typically only characterizing a single cross-section in a two-dimensional parameter space, such as polarization at a fixed wavelength or the spectrum under a fixed polarization state. Furthermore, limited by the planar or curved design of optical devices and sensors, the maximum field of view of existing systems is usually confined to a hemispherical range (~180°). Therefore, these limitations make it difficult for existing systems to meet the detection requirements of light fields with mixed polarization states and broadband continuous spectra, and to satisfy the real-time perception of dynamic, high-dimensional light fields in complex scenes. Summary of the Invention

[0004] Therefore, the purpose of this invention is to provide a fiber-optic-based visual perception system, which has multiple advantages such as simple structure, small size, and the ability to simultaneously acquire optical information of high-dimensional light fields in a short time.

[0005] A fiber-optic-based visual perception system includes multiple sensing units and a photosensitive chip. Each sensing unit includes a microlens and a multimode tapered optical fiber. The microlens is disposed on the end face of the multimode tapered optical fiber. The evanescent field portion of the multimode tapered optical fiber is perpendicular to the photosensitive chip and the distance between them is 0 mm. The multimode tapered optical fiber includes at least two tapered regions.

[0006] The visual perception system described in this invention can simultaneously acquire optical information of a high-dimensional light field in a short time, and has a simple structure and small size.

[0007] Furthermore, the combination of the diameter and height of the microlens of each sensing unit can be selected from one of the following three: a diameter of 75 μm and a height of 50 μm, a diameter of 90 μm and a height of 50 μm, or a diameter of 125 μm and a height of 50 μm.

[0008] Furthermore, within a light incident angle range of 60°±5°, the microlens has a diameter of 75μm and a height of 50μm.

[0009] Furthermore, within a light incident angle range of 75°±5°, the microlens has a diameter of 90μm and a height of 50μm.

[0010] Furthermore, the multimode tapered optical fiber includes two tapered regions with waist diameters of 9 μm and 4 μm, respectively.

[0011] Furthermore, the multimode tapered optical fiber has a core diameter and cladding diameter of 105 / 125 μm, a numerical aperture of 0.22, and a tapered region length of 1 cm.

[0012] Furthermore, the visual perception system also includes an analysis and reconstruction unit, which includes a deep learning algorithm model for reconstructing the light information output by the photosensitive chip.

[0013] Furthermore, the deep learning algorithm model is a ResNet model based on TensorFlow.

[0014] Furthermore, the microlenses corresponding to the plurality of sensing units are geometrically arranged on at least one horizontal plane.

[0015] Furthermore, the microlenses of the plurality of sensing units are geometrically arranged on multiple horizontal planes in the vertical direction.

[0016] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the fiber-optic-based visual perception system of the present invention;

[0018] Figure 2 This is a schematic diagram of the sensing unit structure of the fiber-optic-based visual perception system of the present invention. Figure 3 The coupling efficiency between optical fiber and bare optical fiber under different microlens parameters according to the present invention; Figure 4 The transmittance of multimode tapered optical fibers with different waist diameters for the sensing unit of the present invention; Figure 5 This is an interference speckle pattern corresponding to different optical information collected by the fiber-optic-based visual perception system of the present invention; Figure 6 This is a physical diagram of a fiber-optic-based visual perception system according to an embodiment of the present invention. Figure 7 for Figure 6The image shown is a diagram illustrating the spectral information reconstruction results of a fiber-optic-based visual perception system. Figure 8 for Figure 6 The image shown is a high-dimensional optical information reconstruction result of a fiber-optic-based visual perception system. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are merely illustrative of the technical solutions of this invention and should not be construed as limiting the scope of protection of this invention.

[0020] This invention provides a fiber optic-based visual perception system, such as... Figure 1-2 As shown, the visual perception system includes multiple sensing units and at least one photosensitive chip 3; each sensing unit includes a microlens 1 and an optical fiber 2, wherein the microlens 1 is disposed on the optical fiber end face of the optical fiber 2; the microlens 1, the optical fiber 2, and the photosensitive chip 3 sequentially form an optical path.

[0021] Specifically, microlens 1 is a hemisphere, and the combination of its diameter and height can be selected from one or more of the following three types: a diameter of 75 μm and a height of 50 μm, a diameter of 90 μm and a height of 50 μm, and a diameter of 125 μm and a height of 50 μm. To further improve coupling efficiency within a light incident angle range of 60°±5°, a microlens with a diameter of 75 μm and a height of 50 μm is preferred; to further improve coupling efficiency within a light incident angle range of 75°±5°, a microlens with a diameter of 90 μm and a height of 50 μm is preferred.

[0022] refer to Figure 3 The figure illustrates the optical coupling effect between a bare optical fiber and optical fibers with microlenses having diameters of 75 μm and heights of 50 μm, 90 μm and heights of 50 μm, and 125 μm and heights of 50 μm, respectively. The data in the figure are half-angle data. Both the bare optical fiber and the microlensed optical fiber have a numerical aperture of 0.22, corresponding to an incident angle of ~12.7° where light can propagate by total internal reflection within the fiber. Figure 3 In diagram a, perpendicular rays have the highest coupling efficiency within the numerical aperture of the bare fiber. Light with an incident angle exceeding the maximum incident angle corresponding to the fiber's numerical aperture is propagated by the cladding mode. The coupling efficiency gradually decreases as the incident angle increases, with the limiting coupling angle for the bare fiber being ~40°, at which point the coupling efficiency is >10. -5 Furthermore, it is clearly limited by the minimum measurement accuracy of a 1nW optical power meter and the minimum photosensitivity of a 1nW photosensitive chip. However, in Figure 3In b-3D, using microlenses with different parameters on the fiber end face can achieve coupling close to 90°, exhibiting high coupling sensing capability and improving effective information sensing under high incident angle conditions. Furthermore, within the light incident angle range of 55°-80°, microlenses with specific parameters achieve increased coupling efficiency at large incident angles. For example, a microlens with a diameter of 75μm and a height of 50μm is used within the light incident angle range of 60°±5°, and a microlens with a diameter of 90μm and a height of 50μm is used within the same range, thus achieving increased coupling efficiency. Simultaneously, with the fiber having a limiting coupling angle of ~90° using microlenses, the coupling efficiency is >10. -3 This is two orders of magnitude higher than that of bare optical fiber, resulting in a significant improvement in coupling efficiency.

[0023] Therefore, in this invention, by using a microlens with the aforementioned diameter and height on the end face of the optical fiber, optical coupling can be achieved within a large angle incident light range of 180°, with a numerical aperture close to ~1. Furthermore, strong optical coupling at a specific angle can be achieved, further improving the coupling efficiency.

[0024] In this invention, the microlens 1 is preferably fabricated using two-photon lithography, which includes, in sequence: fiber end-face pretreatment, two-photon lithography processing, sample post-processing, and optical performance testing. The specific steps are as follows: Fiber end face pretreatment: The fiber end face needs to undergo cleaning, plasma activation, silanization, and drying pretreatment steps in sequence. Specifically, the fiber end face is ultrasonically treated with acetone, isopropanol, and deionized water for 15 minutes each, dried with nitrogen, activated with oxygen plasma at 150W power for 1 minute, then immersed in a 1% 3-propyl methacrylate solution dissolved in ethanol for silanization treatment, and finally dried with nitrogen.

[0025] Two-photon lithography: The pre-treated fiber end face is fixed on a three-dimensional micro-motion platform, and then photoresist, including photosensitive resin and photoinitiator, is coated on the fiber end face. Then, the processing objective lens is aligned with the fiber end face with an alignment error of <50nm. After setting the parameters of laser power, direct writing speed, processing path and interface position, the femtosecond laser is controlled to scan along a preset trajectory. The femtosecond laser has a wavelength of 515nm, a pulse width of 100-150 fs, a total power of 1W, and a maximum direct writing speed of up to 20W, that is, 200,000 points are processed per second.

[0026] Post-processing and optical performance testing of samples: The photolithographically processed fiber end face was first developed and the photoresist removed. Then, it was immersed in acetone, isopropanol, and n-hexane solutions for 5 min, 3 min, and 2 min respectively to clean the uncured photoresist, and then dried. Next, the size, incident light coupling angle, optical transmittance, mechanical strength, surface roughness, and environmental stability of the microlens were measured to obtain microlenses that met the requirements. The dimensional deviation of the microlens was <0.05 μm.

[0027] This invention uses two-photon lithography to fabricate microlenses. Because two-photon lithography is based on the principle of two-photon polymerization, it can directly realize true three-dimensional nanostructures. The manufacturing process does not require masks, is fast, highly reliable, and has low loss.

[0028] In a preferred embodiment, the optical fiber 2 is a multimode tapered optical fiber 2, which includes at least two tapered sections, the diameter of the waist cone of the tapered section decreasing sequentially in the optical path.

[0029] In this invention, a collimated light source is incident from a random direction, passes through a microlens 1, and enters a multimode tapered fiber 2. After receiving the light, the multimode tapered fiber 2 excites a large number of intrinsic modes, which then propagate within the fiber. When the light reaches the tapered region within the fiber, the tapering of the multimode fiber causes total internal reflection, resulting in an evanescent field effect and interference speckle. This interference speckle can be collected by a photosensitive chip 3. The multimode tapered fiber in the multi-tapered region exhibits multi-channel characteristics, facilitating the simultaneous acquisition of different optical information. Furthermore, controllable mode leakage depletes the input power, enhancing interference and making the interference speckle characteristics more pronounced. The photosensitive chip's ability to collect light and extract features further improves the quality of the reconstructed optical information.

[0030] refer to Figure 4 , Figure 4 The transmittance of multimode tapered optical fibers with different waist diameters is shown. Figure 4 In multimode tapered optical fibers, as the waist diameter decreases, the mode leakage increases because light does not satisfy the law of total internal reflection when propagating in the fiber, resulting in a decrease in transmittance and a corresponding increase in evanescent field effect.

[0031] In a more preferred embodiment, the multimode tapered optical fiber 2 includes two tapered sections, and the core diameter and cladding diameter of the multimode tapered optical fiber 2 are 105 / 125 μm, the numerical aperture is 0.22, the tapered section length is 1 cm, and the tapered waist diameter is 9 μm and 4 μm along the optical path, respectively.

[0032] The multimode tapered optical fiber 2 of the present invention is preferably prepared by a fused taper method. The specific steps are as follows: The coating layer of the optical fiber in the target area is removed, and then ultrasonically cleaned for 15 minutes with a mixture of anhydrous ethanol and isopropanol in a volume ratio of 1:1. Then it is cleaned with deionized water and dried with nitrogen to remove impurities from the surface of the target optical fiber. Next, the target optical fiber is fixed on the fiber clamp of the tapering system. The coaxiality of the clamp is ≤3μm. Tapered optical fiber is prepared by the fused taper method. In the fused taper method, an adjustable beam current oxyhydrogen flame is used as the heating source. After ignition, wait 5 minutes to allow the flame head of the heating source to stabilize and reset the tapering system. The following tapering parameters are set: hydrogen flow rate of 120 SCCM, oxygen flow rate of 15 SCCM, flame head movement speed of 60 mm / min, flame head movement length of 0 mm / min, clamp movement speed of 30 mm / min, and preset tapering length. Then, the fiber is tapered to form a tapered region with a waist diameter of 9μm and 4μm. Then, the fused taper-treated optical fiber is fixed onto a micro-fiber fixture using a micro-displacement platform. The fixing method can be UV photosensitive adhesive, and the Z-axis of the micro-displacement stage is adjusted to make the fixture contact the tapered optical fiber. The evanescent field effect of the tapered optical fiber is tested by introducing a laser with a wavelength of 700nm, and the transmittance of the tapered optical fiber is tested by introducing a laser with a power of 3mW.

[0033] Compared to other methods, the melt taper method used in this invention is low in cost, has controllable dimensions, good uniformity, and is simple and efficient to operate.

[0034] To achieve better acquisition of the interference speckle generated by the multimode tapered fiber 2 by the photosensitive chip 3, please refer to... Figure 1-2 The multimode tapered optical fibers 2 are sequentially arranged on the photosensitive chip 3. The evanescent field effect portion, i.e., the leaky mode portion, of the multimode tapered optical fibers 2 is aligned with the photosensitive chip 3, meaning that the evanescent field portion of the multimode tapered optical fibers 2 and the photosensitive chip 3 are vertically opposite and completely attached, with a vertical distance of 0 mm. In this invention, the photosensitive chip 3 collects optical information, including: wavelength speckle with a step size of 0.1 nm in the wavelength range of 400 nm-1100 nm, speckle at different power levels, speckle under full Stokes polarization, light intensity, and incident angle. In a specific embodiment, the photosensitive chip 3 may be a CMOS photosensitive chip.

[0035] Furthermore, the fiber-optic-based visual perception system of the present invention also includes an analysis and reconstruction unit and a display unit. The analysis and reconstruction unit is used to analyze the light information collected by the photosensitive chip 3 to reconstruct the light information, and the display unit is used to display the results of the analysis and reconstruction unit. The analysis and reconstruction unit includes a deep learning algorithm model, and the collected light information is input into this model for analysis and reconstruction. In a specific embodiment, the analysis and reconstruction unit may be placed inside the photosensitive chip 3.

[0036] In a preferred embodiment, the deep learning algorithm model is a ResNet model based on TensorFlow. Compared with other algorithm models, the ResNet model based on TensorFlow can achieve a leap from "predicting only the position" to "accurately reconstructing the waveform". It can effectively capture the high-frequency details of the cone interference fringes by utilizing residual connections, avoid the gradient vanishing problem of deep networks, significantly improve the robustness of the model under noise or uneven illumination, and have better generalization ability in non-ideal environments than other models. It further improves the perception ability of multi-dimensional information and the accuracy of optical information reconstruction.

[0037] To improve the quality of reconstructed optical information, the aforementioned deep learning algorithm model can be trained. Specifically, the collected optical information is divided into a training set, a validation set, and a test set. The training set serves as the foundation of the deep learning algorithm, the validation set verifies the training effect, and the test set serves as the basis for reconstructed information. When the regression loss function of the validation set no longer decreases with the increase of training epochs, and the automatic learning rate reduction mechanism has been triggered to the lowest level, the model's potential has been fully explored and no overfitting has occurred, thus determining that training has ended. The regression loss function is preferably a composite loss combining MSE (Mean Squared Error) with physical constraints to simultaneously ensure waveform fitting accuracy and physical plausibility.

[0038] Furthermore, to prevent damage to the devices, the high-dimensional visual perception system of this invention also includes a housing, within which the plurality of sensing units are housed. The housing is designed using 3D modeling software and can be manufactured using a 3D printer with millimeter-level precision.

[0039] In a preferred embodiment, the microlenses corresponding to the plurality of sensing units are geometrically arranged on a horizontal plane. More preferably, the microlenses corresponding to the plurality of sensing units are geometrically arranged on multiple horizontal planes in the vertical direction. For example, the plurality of microlenses are arranged sequentially on a horizontal plane in geometric shapes such as circles, triangles, quadrilaterals, pentagons, and teardrops, thereby creating multiple fields of view that achieve 360° coupling on the horizontal and vertical planes, further improving the light information acquisition effect at large angles, and realizing wide field of view and high-dimensional light information visual perception at any angle.

[0040] refer to Figure 6 , 8 It should be noted that, in order to facilitate the demonstration of the light information sensed by the microlenses at different positions on the spherical shell, in Figure 8 The microlenses, previously invisible from the viewpoint shown in the image, were adjusted to be observable. In reality, the microlenses are uniformly distributed across the spherical shell within the same plane. Figure 6 , 8In the illustrated embodiment, the visual perception system includes a housing and multiple sensing units disposed within the housing. Each sensing unit includes a microlens 1 with a diameter of 125 μm and a height of 50 μm, and a multimode tapered optical fiber 2 containing two tapered regions. The core diameter and cladding diameter of the multimode tapered optical fiber 2 are 105 / 125 μm, the numerical aperture is 0.22, the tapered region length is 1 cm, and the tapered waist diameter is 9 μm and 4 μm, respectively. The evanescent field effect portions of the multimode tapered optical fiber 2 are vertically opposite to and attached to the CMOS photosensitive chip. The microlenses of the photosensitive unit are disposed on four horizontal planes arranged vertically along the spherical housing, with four microlenses arranged circularly on the spherical housing on each plane.

[0041] refer to Figure 7 , Figure 7 It shows Figure 6 A schematic diagram illustrating the reconstruction results of the spectral and light intensity information of the visual perception system. (See diagram for example.) Figure 7 As shown, Figure 6 The visual perception system in this invention can reconstruct the received light information in a way that perfectly matches the real data. The visual perception system of this invention has high accuracy and reliability.

[0042] refer to Figure 8 , Figure 8 In a, wavelength information and 0° linear polarization information can be detected simultaneously in the (-45°, 45°) direction, with a peak wavelength of 690nm; Figure 8 In b, wavelength information and left-handed circular polarization information can be detected simultaneously in the (45°, 45°) direction, with a peak wavelength of 475nm. Figure 8 In c, wavelength information and 90° linear polarization information can be detected simultaneously in the (-45°, -45°) direction, with a peak wavelength of 532nm. Figure 8 In step d, wavelength information and right-handed circular polarization information can be detected simultaneously in the (0°, 90°) direction, with a peak wavelength of 400nm. Therefore, the visual perception system of this invention can perceive different optical information at different locations while achieving high-precision optical information reconstruction.

[0043] Compared to existing technologies, the fiber-optic-based visual perception system of this invention utilizes microlenses positioned on the end face of the fiber to increase the incident angle of the incident light, overcoming the limitation of the numerical aperture of the fiber and achieving efficient coupling of incident light over a wide range of angles. The combination of microlenses and multimode tapered fibers results in more stable and controllable interference speckle characteristics. Furthermore, the multimode tapered fiber has at least two tapered regions, providing more dimensional optical information without introducing additional splicing losses. This multidimensional optical information improves the accuracy of optical information reconstruction. Moreover, by utilizing deep learning algorithms, high-precision multidimensional optical information can be acquired with a single input, enabling the processing of high-dimensional optical information in the visual perception system. The processing results are highly accurate, and the optical information reconstruction effect is excellent. This visual perception system is suitable for detecting optical fields with mixed polarization states and broadband continuous spectra, as well as for real-time perception of dynamic, high-dimensional optical fields in complex scenes. It can be widely applied in fiber optic imaging, fiber optic communication, and other fields, and can achieve long-term, large-scale optical information data acquisition exceeding 8 hours, greatly increasing the detector's operating time. It also boasts advantages of miniaturization and ultra-compactness.

[0044] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and the present invention also intends to include these modifications and variations.

Claims

1. A fiber optic-based visual perception system, characterized in that, It includes multiple sensing units and a photosensitive chip. Each sensing unit includes a microlens and a multimode tapered optical fiber. The microlens is disposed on the fiber end face of the multimode tapered optical fiber. The evanescent field portion of the multimode tapered optical fiber is perpendicular to the photosensitive chip and the distance is 0 mm. The multimode tapered optical fiber includes at least two tapered regions.

2. The visual perception system according to claim 1, characterized in that, The combination of diameter and height of the microlens of each sensing unit can be selected from one of the following three: 75 μm diameter and 50 μm height, 90 μm diameter and 50 μm height, or 125 μm diameter and 50 μm height.

3. The visual perception system according to claim 1, characterized in that, The microlens has a diameter of 75 μm and a height of 50 μm within a light incident angle range of 60°±5°.

4. The visual perception system according to claim 1, characterized in that, Within a light incident angle range of 75°±5°, the microlens has a diameter of 90μm and a height of 50μm.

5. The visual perception system according to any one of claims 1-4, characterized in that, The multimode tapered optical fiber includes two tapered regions with waist diameters of 9 μm and 4 μm, respectively.

6. The visual perception system according to claim 5, characterized in that, The multimode tapered optical fiber has a core diameter and cladding diameter of 105 / 125 μm, a numerical aperture of 0.22, and a tapered region length of 1 cm.

7. The visual perception system according to any one of claims 1-6, characterized in that, The visual perception system also includes an analysis and reconstruction unit, which includes a deep learning algorithm model for reconstructing the light information output by the photosensitive chip.

8. The visual perception system according to any one of claims 7, characterized in that, The deep learning algorithm model is a ResNet model based on TensorFlow.

9. The visual perception system according to any one of claims 1-8, characterized in that, The microlenses of the plurality of sensing units are geometrically arranged on at least one horizontal plane.

10. The visual perception system according to any one of claims 9, characterized in that, The microlenses of the multiple sensing units are geometrically arranged on multiple horizontal planes in the vertical direction.

Images