Optical fiber coupling real-time correction method in single-mode and multi-mode optical fiber endoscopic imaging system

Abstract

The invention discloses an optical fiber coupling real-time correction method in a single-mode and multi-mode optical fiber endoscopic imaging system, which comprises the following steps of: firstly, manually adjusting reflectors M1 and M2 to an optimal multi-mode optical fiber coupling state, acquiring images of light spots at the near end and the far end of a laser beam in the state by using a CCD (charge coupled device) 1 and a CCD 2, and calculating the position of a central coordinate of the images; and collecting a speckle image formed by the light spot through the scattering medium by using a CCD3 (charge coupled device 3). The system monitors the center coordinates of near-end and far-end light spots at the current moment and the correlation coefficient of the current speckle image and the collimation state image in real time. The piezoelectric mirror bracket is fed back and adjusted in real time through the center coordinates of the two spot light spots and the correlation coefficient of the speckles, so that the correlation coefficient of the speckles is kept above a set threshold value, and automatic calibration of light beams is achieved. On the basis of the traditional thinking that two points determine one straight line, the correlation coefficient of using speckles is increased, the alignment precision is improved, and the method has certain practicability and application prospects.

Description

Technical Field

[0001] This invention relates to the field of single-mode and multi-mode fiber imaging, and more specifically to a real-time fiber coupling correction method in a single-mode and multi-mode fiber endoscopic imaging system. Background Technology

[0002] Single-mode and multimode fiber imaging is a method that uses multimode fiber to transmit optical signals and reconstructs images through computational techniques. Compared to single-mode fiber, multimode fiber has a larger core diameter (typically 100 to several hundred micrometers), allowing multiple optical modes to be transmitted simultaneously. Multimode fiber is flexible and suitable for operation in narrow or curved environments, making it applicable to the field of medical endoscopy. Single-mode and multimode fiber endoscopic imaging systems are generally as follows: Figure 1 As shown, it generally consists of three parts: The optical field modulation part, which typically refers to the laser being incident on a spatial light modulator (SLM or DMD), and the optical field being modulated by loading a mask onto the SLM. The modulated optical field is then spatially filtered through lenses L1 and L2 and a pinhole Iris, ensuring that only the laser generated by the SLM is incident into the multimode fiber. The laser-to-multimode fiber coupling part typically uses a collimating lens or objective lens to incident the modulated optical field into the multimode fiber. To achieve efficient and accurate coupling, the collimating lens is usually placed on a displacement platform, and the laser is incident on the central region of the fiber end face by moving the displacement platform. The imaging part typically uses the modulation of the incident light field to achieve a focal point scanning of the object to be imaged at the fiber output end. The laser reflected from the object or the fluorescence excited by the focal point is then collected by the PMT after passing through the multimode fiber again, and the object is reconstructed using algorithms.

[0003] Achieving focused scanning behind a multimode fiber is a crucial aspect of imaging. To achieve this, the transmission matrix of the multimode fiber is typically measured. The input light field distribution is then calculated using the transmission matrix, which is used to adjust the mask pattern applied to the spatial light modulator, thus enabling focused scanning behind the fiber. However, in this system, even a sub-micron change in the position of the modulated light field incident on the multimode fiber end face will alter the transmission matrix. In other words, even a slight change in the fiber coupling state can cause the measured transmission matrix to fail, making focused scanning imaging impossible. Summary of the Invention

[0004] To address the aforementioned shortcomings in the prior art, this invention provides a real-time fiber coupling correction method for single-mode and multi-mode fiber optic endoscopic imaging systems.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0006] A real-time fiber coupling correction method in a single-mode or multi-mode fiber optic endoscopic imaging system includes the following steps:

[0007] S1. In a single-mode multimode fiber optic endoscopic imaging system, an optical electron microscope frame based on piezoelectric ceramic electro-tuning is introduced and equipped with mirrors M1 and M2, non-polarizing beam splitters NPBS1, NPBS2 and NPBS3, scattering medium and charge-coupled devices CCD1, CCD2 and CCD3.

[0008] S2. Manually adjust reflectors M1 and M2 to the optimal multimode fiber coupling state. Use CCD1 and CCD2 to acquire images of the laser beam spot at the near and far ends of the laser beam under this state and calculate the position of its center coordinates. Use CCD3 to acquire the speckle image formed by the spot through the scattering medium.

[0009] S3. Monitor the center coordinates of the near and far light spots at the current moment and the correlation coefficient between the current speckle image and the collimation state image;

[0010] S4. The piezoelectric mirror frame is adjusted in real time by using the center coordinates of the two spot lights and the correlation coefficient of the speckle to keep the correlation coefficient of the speckle above the set threshold, thereby achieving automatic beam calibration.

[0011] Furthermore, the specific calculation method for the correlation coefficient in S3 is as follows:

[0012]

[0013] In the formula, and ρ and g are the average values ​​of the speckle images f(x,y) and g(x,y) in the speckle region S, respectively, where N is the number of pixels in region S and ρ is the correlation coefficient.

[0014] Furthermore, in S2, the correlation coefficient is greater than 0.99 if and only if the center of the corrected spot completely coincides with the center of the spot before correction.

[0015] Furthermore, step S4 specifically includes the following steps:

[0016] S41. Perform coarse alignment on the multimode fiber and record the pixel position (x10, y10) of the centroid of the incident laser spot on CCD1; record the pixel position (x20, y20) of the centroid of the incident laser spot on CCD2; record the first speckle pattern on CCD3 after the incident laser passes through the scattering medium.

[0017] S42. Due to changes in the external environment, the frames M1 and M2 drift, so the pixel position of the light spot centroid on CCD1 becomes (x1, y1); the pixel position of the light spot centroid on CCD2 becomes (x2, y2), and the speckle pattern on CCD3 becomes the second speckle pattern.

[0018] S43. Calculate the correlation coefficient based on the second speckle plot and the first speckle plot;

[0019] S44. Determine the relationship between (x1,y1) and (x10,y10). If (x1,y1) = (x10,y10), keep M1 unchanged; if they are not equal, move M1 until (x1,y1) = (x10,y10).

[0020] If (x1,y1) = (x10,y10), determine the relationship between (x2,y2) and (x20,y20). If (x2,y2) = (x20,y20), keep M2 unchanged; if they are not equal, move M2 until (x2,y2) = (x20,y20).

[0021] S45. Calculate the correlation between the first and second speckle patterns after the movement in S44. If the correlation is less than 0.99, use a random collision algorithm to correct M2 until the correlation is greater than 0.99, completing the multimode fiber collimation correction. Specifically, first randomly move the knob of M2 in the horizontal direction, record the third speckle pattern of CCD3 at this time, and calculate its correlation coefficient with the first speckle pattern. If this correlation coefficient is greater than the correlation coefficient between the second and first speckle patterns, it proves that the horizontal movement direction of M2 is correct. Continue moving in this direction until the correlation coefficient increases to an extreme value. If the correlation coefficient is less than the correlation coefficient between the second and first speckle patterns, it proves that the horizontal movement direction of M2 is incorrect, and it needs to be moved in the opposite direction until the correlation coefficient increases to an extreme value. When the horizontal movement of M2 reaches an extreme value, move the knob of M2 in the vertical direction. Record the fourth speckle pattern of CCD3 at this time and calculate its correlation coefficient with the first speckle pattern. If the correlation coefficient is greater than the correlation coefficient between the second speckle plot and the first speckle plot, it proves that the vertical direction of M2 is correct. Continue moving in this direction until the correlation coefficient increases to an extreme value. If the correlation coefficient is less than the correlation coefficient between the second speckle plot and the first speckle plot, it proves that the vertical direction of M2 is incorrect, and it needs to be moved in the opposite direction until the correlation coefficient increases to an extreme value. Repeat this back-and-forth movement of the knob in the horizontal and vertical directions until the correlation coefficient is greater than 0.99, then stop moving.

[0022] The present invention has the following beneficial effects:

[0023] This invention provides a real-time fiber coupling correction method for single-mode and multi-mode fiber optic endoscopic imaging systems. This method enables high-precision automatic calibration of the laser beam, preventing performance degradation due to laser collimation drift in long-term laser applications. Building upon the traditional approach of determining a straight line from two points, this method incorporates a speckle correlation coefficient to improve alignment accuracy, demonstrating practicality and promising application prospects. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of a traditional single-mode or multimode fiber optic endoscopic imaging system in the prior art.

[0025] Figure 2 This is a flowchart illustrating the real-time correction method for fiber optic coupling in a single-mode or multimode fiber optic endoscopic imaging system according to an embodiment of the present invention.

[0026] Figure 3 The embodiments of the present invention are equipped with a single-mode or multimode fiber optic endoscopic imaging system based on a real-time fiber optic coupling correction method using optical speckle.

[0027] Figure 4 Images of light spots and speckle patterns obtained during collimation coupling in an embodiment of the present invention.

[0028] Figure 5 Images of speckle and speckle patterns in misaligned coupling according to an embodiment of the present invention.

[0029] Figure 6 The image shows the spot and speckle after automatic calibration according to an embodiment of the present invention. Detailed Implementation

[0030] Example

[0031] A real-time fiber coupling correction method in a single-mode or multimode fiber optic endoscopic imaging system, such as Figure 2 As shown, it includes the following steps:

[0032] S1. In a single-mode multimode fiber optic endoscopic imaging system, an optical electron microscope frame based on piezoelectric ceramic electro-tuning is introduced and equipped with mirrors M1 and M2, non-polarizing beam splitters NPBS1, NPBS2 and NPBS3, scattering medium and charge-coupled devices CCD1, CCD2 and CCD3.

[0033] This embodiment is based on a single-mode / multimode fiber optic endoscopic imaging system using a fiber-optic coupling real-time correction method based on optical speckle. Figure 3As shown. The system incorporates an optical adjustment frame based on piezoelectric ceramics (or stepper motors) to mount reflectors M1 and M2, non-polarizing beam splitters NPBS1, 2, and 3, a scattering medium, and CCDs 1, 2, and 3. The specific steps are as follows: First, reflectors M1 and M2 are manually adjusted to the optimal multimode fiber coupling state. CCDs 1 and 2 are used to acquire images of the laser beam at the near and far ends in this state and calculate their center coordinates. CCD 3 is used to acquire the speckle image formed by the beam passing through the scattering medium. The system monitors in real time the center coordinates of the near and far ends of the beam and the correlation coefficient between the current speckle image and the collimated image. The piezoelectric frame is adjusted in real time using the center coordinates of the two beam points and the correlation coefficient of the speckle to keep the correlation coefficient above a set threshold, thus achieving automatic beam calibration. Specifically:

[0034] S2. Manually adjust reflectors M1 and M2 to the optimal multimode fiber coupling state. Use CCD1 and CCD2 to acquire images of the laser beam spot at the near and far ends of the laser beam under this state and calculate the position of its center coordinates. Use CCD3 to acquire the speckle image formed by the spot through the scattering medium.

[0035] This embodiment is based on the principle of two points determining a straight line. CCD1 and CCD2 are used to acquire the near and far ends of the beam along the straight line and calculate their center positions, recording the speckle image at this time. The center coordinates of the two spots and the correlation coefficient between the current speckle image and the collimated speckle image are monitored in real time. If the correlation coefficient is detected to be lower than a set threshold, the reflector is moved to bring the beam back to the laser incident coupling state. The process of coarse beam adjustment based on the spot images acquired by the two CCDs is as follows: If the beam deviates significantly from the collimated state, the correlation coefficient of the speckle changes very little when the reflector is moved, requiring coarse adjustment using the center coordinates of the spots. First, the near-end reflector is adjusted to bring the center of the near-end spot back to the collimated state, then the far-end reflector is adjusted to bring the far-end spot back to the collimated state. During this process, if the center of the near-end spot deviates from the collimated state, the near-end reflector is adjusted first.

[0036] S3. Monitor the center coordinates of the near-end and far-end light spots at the current moment and the correlation coefficient between the current speckle image and the collimation state image. The specific calculation method is as follows:

[0037]

[0038] In the formula, and ρ and g are the average values ​​of the speckle images f(x,y) and g(x,y) in the speckle region S, respectively, where N is the number of pixels in region S and ρ is the correlation coefficient.

[0039] S4. The piezoelectric mirror frame is adjusted in real time using the center coordinates of the two spot beams and the correlation coefficient of the speckle, so that the correlation coefficient of the speckle remains above the set threshold, thereby achieving automatic beam calibration.

[0040] A flowchart illustrating the real-time fiber coupling correction method in a single-mode / multimode fiber optic endoscopic imaging system is shown below. Figure 2 As shown. First, artificial beam collimation is employed. Then, under artificial collimation coupling, CCD1 and CCD2 acquire spot images of the near and far ends of the beam, respectively, and calculate the center coordinates of the spots. CCD3 acquires the speckle image formed after the beam passes through the frosted glass. These three data sets are stored in the system. The changes in the center coordinates of the spots and the correlation coefficient of the speckle are monitored in real-time using the three CCD cameras. When the correlation coefficient of the speckle changes and exceeds a set threshold, coupling failure is determined. First, the calculated center coordinates of the spots are read using CCD1 and CCD2 to coarsely adjust M1 and M2 until the speckle correlation coefficient is less than the set threshold. Then, the speckle correlation coefficient is used to finely adjust M1 and M2, as follows:

[0041] S41. Perform coarse alignment on the multimode fiber and record the pixel position (x10, y10) of the centroid of the incident laser spot on CCD1; record the pixel position (x20, y20) of the centroid of the incident laser spot on CCD2; record the first speckle pattern on CCD3 after the incident laser passes through the scattering medium.

[0042] When the coarse-adjusted mirror brings the beam back to the relevant range (i.e., the correlation coefficient of the speckle changes significantly), the center coordinates of the speckle are no longer used; only the correlation coefficient is considered. Randomly move one of the mirrors. If the speckle correlation coefficient increases, the adjustment direction is correct, and you can continue moving in that direction. To avoid environmental disturbances, the mirror is set to switch to the opposite direction after two consecutive decreases in the speckle correlation coefficient. If you switch directions twice within the same channel, the mirror is considered to have reached its optimal position in that direction, and you can switch to another channel for that mirror, following the same steps. If you switch between channels twice, switch to another mirror and perform the same steps.

[0043] S42. Due to changes in the external environment, the frames M1 and M2 drift, so the pixel position of the light spot centroid on CCD1 becomes (x1, y1); the pixel position of the light spot centroid on CCD2 becomes (x2, y2), and the speckle pattern on CCD3 becomes the second speckle pattern.

[0044] S43. Calculate the correlation coefficient based on the second speckle plot and the first speckle plot;

[0045] S44. Determine the relationship between (x1,y1) and (x10,y10). If (x1,y1) = (x10,y10), keep M1 unchanged; if they are not equal, move M1 until (x1,y1) = (x10,y10).

[0046] If (x1,y1) = (x10,y10), determine the relationship between (x2,y2) and (x20,y20). If (x2,y2) = (x20,y20), keep M2 unchanged; if they are not equal, move M2 until (x2,y2) = (x20,y20).

[0047] S45. Calculate the correlation between the first speckle pattern and the second speckle pattern after the movement in S44. If the correlation is less than 0.99, use the random collision algorithm to correct M2 until the correlation is greater than 0.99, and complete the multimode fiber collimation correction.

[0048] In this embodiment, a similar function can also be achieved using a neural network algorithm. First, the multimode fiber is manually collimated, and the light spot patterns 10 and 20 on CCD1 and CCD2, and the speckle pattern 10 on CCD3 are recorded. Then, by running the three CCDs for an extended period, that is, by repeatedly recording the light spot pattern 10, light spot pattern 20, and speckle pattern under this state... Figure 1 Expand the dataset to represent the aligned state. Then, after moving M1, record the first and second spot images on CCD1 and CCD2, and the third spot image on CCD3 again. Expand the dataset again by recording the first, second, and third spot images in this state multiple times; this dataset represents the misaligned state of M1. Return M1 to its original position, move M2, and record spot images 12 and 22 on CCD1 and CCD2, and speckle image 12 on CCD3 again. Expand the dataset again by recording the first, second, and third spot images in this state multiple times; this dataset represents the misaligned state of M2. Simultaneously move M1 and M2, and record the first and second spot images on CCD1 and CCD2, and the third spot image on CCD3 again. Expand the dataset again by recording the first, second, and third spot images in this state multiple times; this dataset represents the misaligned state of both M1 and M2. By employing a neural network classification method, the system can classify states as aligned, misaligned (M1), misaligned (M2), and both misaligned (M1 and M2). During actual operation, by acquiring data and a third-party light spot image, the trained network identifies the corresponding state. If the state is aligned, no adjustment is needed; if M1 is misaligned, M1 is adjusted until it is aligned; if M2 is misaligned, M2 is adjusted until it is aligned; if both M1 and M2 are misaligned, M2 is first adjusted until it becomes misaligned (M1), then M1 is adjusted until it is aligned, thus completing the automatic tuning and alignment process.

[0049] According to the fine-tuning method described in this invention, the speckle correlation coefficient can be adjusted back to above 0.95. If the optical system experiences a large disturbance, causing the speckle correlation coefficient to drop instantly below 0.1, it is suitable to first use the center coordinates of the speckle to perform coarse adjustment of the beam position, adjusting the speckle correlation coefficient to a position with relatively high correlation, and then use the change in the speckle correlation coefficient for fine-tuning. The system calibration results are as follows: Figure 4 , Figure 5 , Figure 6 As shown, where Figure 4 , Figure 5 , Figure 6 These are images of the spot and speckle when collimated, images of the spot and speckle when misaligned, and images of the spot and speckle after automatic calibration.

[0050] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0051] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0052] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

Claims

1. A real-time fiber coupling correction method in a single-mode or multi-mode fiber optic endoscopic imaging system, characterized in that, Includes the following steps: S1. In a single-mode multimode fiber optic endoscopic imaging system, an optical electron microscope frame based on piezoelectric ceramic electro-tuning is introduced and equipped with mirrors M1 and M2, non-polarizing beam splitters NPBS1, NPBS2 and NPBS3, scattering medium and charge-coupled devices CCD1, CCD2 and CCD3. S2. Manually adjust reflectors M1 and M2 to the optimal multimode fiber coupling state. Use CCD1 and CCD2 to acquire images of the laser beam spot at the near and far ends of the laser beam under this state and calculate the position of its center coordinates. Use CCD3 to acquire the speckle image formed by the spot through the scattering medium. S3. Monitor the center coordinates of the near and far light spots at the current moment and the correlation coefficient between the current speckle image and the collimation state image; S4. The piezoelectric mirror frame is adjusted in real time by using the center coordinates of the two spot lights and the correlation coefficient of the speckle to keep the correlation coefficient of the speckle above the set threshold, thereby achieving automatic beam calibration.

2. The real-time fiber coupling correction method in a single-mode / multimode fiber optic endoscopic imaging system according to claim 1, characterized in that, The specific calculation method for the correlation coefficient in S3 is as follows: In the formula, and ρ and g are the average values ​​of the speckle images f(x,y) and g(x,y) in the speckle region S, respectively, where N is the number of pixels in region S and ρ is the correlation coefficient.

3. The real-time fiber coupling correction method in a single-mode / multimode fiber optic endoscopic imaging system according to claim 2, characterized in that, In S2, the correlation coefficient is greater than 0.99 if and only if the center of the corrected spot completely coincides with the center of the spot before correction.

4. The real-time fiber coupling correction method in a single-mode / multimode fiber optic endoscopic imaging system according to claim 1, characterized in that, S4 specifically includes the following steps: S41. Perform coarse alignment on the multimode fiber and record the pixel position (x10, y10) of the centroid of the incident laser spot on CCD1; record the pixel position (x20, y20) of the centroid of the incident laser spot on CCD2; record the first speckle pattern on CCD3 after the incident laser passes through the scattering medium. S42. Due to changes in the external environment, the frames M1 and M2 drift, and the pixel position of the light spot centroid on CCD1 becomes (x1, y1). The pixel position (x2, y2) of the light spot centroid on CCD2 is transformed into the second speckle pattern on CCD3; S43. Calculate the correlation coefficient based on the second speckle plot and the first speckle plot; S44. Determine the relationship between (x1,y1) and (x10,y10). If (x1,y1) = (x10,y10), then keep M1 unchanged. If they are not equal, move M1 until (x1,y1) = (x10,y10); If (x1,y1) = (x10,y10), determine the relationship between (x2,y2) and (x20,y20). If (x2,y2) = (x20,y20), keep M2 unchanged; if they are not equal, move M2 until (x2,y2) = (x20,y20). S45. Calculate the correlation between the first speckle pattern and the second speckle pattern after the movement in S44. If the correlation is less than 0.99, use the random collision algorithm to correct M2 until the correlation is greater than 0.99, and complete the multimode fiber collimation correction.

Images