A Method and System for Fiber Bundle Parameter Calibration and Image Super-Resolution Based on Beam Deflection Control

Abstract

This invention discloses a fiber bundle parameter calibration and image super-resolution method and system based on beam deflection control, belonging to the field of optical calibration technology. The method combines the ideal image point coordinates and the actual image point coordinates, optimizes and solves the initial state parameters of the system, including the initial rotation angle of the single rotating wedge, based on the optimization method, and calculates the optimal rotation angle of the single rotating wedge and the corresponding image micro-displacement based on the model, thus completing the parameter calibration of the single rotating wedge fiber bundle optical system. Based on the known displacement, it implements multi-frame micro-displacement image super-resolution of the fiber bundle optical system.

Description

Technical Field

[0001] This invention relates to the field of optical calibration technology, and more specifically to a method and system for optical fiber bundle parameter calibration and image super-resolution based on beam deflection control. Background Technology

[0002] Structurally, a fiber optic bundle optical system can be divided into four parts: a front imaging objective, a fiber bundle, a rear microscope objective, and an image sensor. By extending a flexible fiber bundle from the external space into the inside of a narrow opening, the fiber optic bundle optical system can achieve image transmission in blind zones. It has significant application value in industrial, disaster relief, and archaeological fields such as engine blade defect detection, life search in building wreckage, and exploration of the interior of seabed shipwrecks.

[0003] Currently, limited by the current level of fiber bundle processing and manufacturing, the core size of fiber bundles is difficult to reduce, typically ranging from tens of micrometers, far exceeding the pixel size of mainstream CMOS image sensors (generally 3-5 micrometers), resulting in low imaging resolution for fiber bundle systems. Multi-frame micro-displacement image super-resolution technology acquires image sequences with relative displacement through various mechanical methods such as rotating optical wedges, piezoelectric ceramics, and mirrors. It utilizes sub-pixel sampling information to eliminate the honeycomb grid effect in fiber bundle optical system images, reconstructing high-resolution images with prominent details.

[0004] Chinese patent application No. 202111571233 discloses an image super-resolution reconstruction method based on fiber bundle combined with micro-scanning technology. It uses piezoelectric ceramics to control the sampling surface of the fiber bundle to achieve high-precision two-dimensional lateral micro-displacement scanning. However, piezoelectric ceramics are expensive and require high-voltage drive. Chinese patent application No. 202110510008 discloses a fiber bundle multi-frame image super-resolution reconstruction method and apparatus, proposing an efficient multi-frame micro-displacement image super-resolution method without triangulation. However, registration errors between images significantly affect the image super-resolution effect. Chinese patent application No. 202410454989 discloses an endoscopic imaging image reconstruction method, apparatus, device, and medium. It combines a constructed fiber bundle temporal domain model and a fiber bundle spatial domain model to implement image super-resolution. However, since the training of the fiber bundle temporal domain model and the fiber bundle spatial domain model depends on existing target high-definition video training sets, it has limitations in generalization ability.

[0005] Therefore, how to improve the estimation accuracy of the initial state parameters of the fiber bundle optical system, while ensuring the smooth implementation of the fiber bundle optical system parameter calibration and image super-resolution, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] In view of the above problems, the present invention is proposed to provide a fiber bundle parameter calibration and image super-resolution method and system based on beam deflection control to overcome or at least partially solve the above problems.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, embodiments of the present invention provide a fiber bundle parameter calibration and image super-resolution method based on beam deflection control, comprising: Step 1: Construct a fiber optic bundle imaging system based on a single rotating optical wedge as the core component. This includes a display screen, collimator, single rotating optical wedge, imaging objective, fiber optic bundle, microscope, and image sensor, sequentially connected. The rotating optical wedge, imaging objective, and the front surface of the fiber optic bundle are combined into a rigid mechanical body sharing a common optical axis and mounted on a two-dimensional rotating base. The rear surface of the fiber optic bundle is connected to a fiber optic rotating support. The single rotating optical wedge is also connected to the optical wedge rotating support. Step 2: Based on the constructed fiber optic bundle imaging system, by combining the ideal image point coordinates obtained from ray tracing calculations with the actual image point coordinates obtained from actual image acquisition, the initial state parameters of the fiber optic bundle imaging system are optimized using an optimization method. Then, based on the solution results, the optimal rotation angle of the single rotating wedge and the corresponding image micro-displacement are calculated, completing the system parameter calibration. Step 3: Based on the optimal approximate rotation angle set of the single rotating wedge obtained by calibration, multiple frames of micro-displacement images are acquired sequentially, and super-resolution reconstruction is achieved based on the multiple frames of micro-displacement images to obtain a high-resolution image that surpasses the resolution of the original image.

[0008] Furthermore, in step two, the initial state parameters of the fiber optic bundle imaging system include the incident ray vector of the incident parallel light, the initial rotation angle of the single rotating wedge, the initial rotation angle of the rigid geometric transformation between the front and rear surfaces of the fiber optic bundle, the X-axis lateral translation, and the Y-axis vector translation.

[0009] Furthermore, in step two, the coordinates of the ideal image point are calculated based on ray tracing, specifically including the following steps: Based on the horizontal rotation angle θ and pitch tilt angle of the two-dimensional rotating base of the fiber optic imaging system Construct the rotation transformation matrix The incident ray vector of the fiber optic imaging system is transformed into a three-dimensional spatial vector v in a new coordinate system. rot ; Using the wedge angle α, refractive index n, and controllable rotation angle ψ of a single rotating optical wedge, along with the initial rotation angle... The sum of these values ​​is used to calculate the vector of the outgoing ray after passing through the single rotating optical wedge, based on the vector form of the law of refraction: Based on the obtained single-rotation wedge outgoing ray vector, combined with the focal length of the imaging objective and the pixel size at the front surface of the fiber bundle, the coordinates of the ideal image point are calculated based on the camera imaging model.

[0010] Furthermore, the vector of the outgoing ray after passing through the single rotating optical wedge is calculated according to the vector form of the law of refraction, specifically including the following formula:

[0011] Among them, v out This represents the vector of the outgoing light rays after passing through the single rotating optical wedge.

[0012] Furthermore, based on the obtained single-rotation wedge outgoing ray vector, combined with the focal length f of the imaging objective lens and the pixel size d at the front surface of the fiber bundle, the coordinates of the ideal image point are calculated based on the camera imaging model, specifically including the following formulas:

[0013] In the formula, Represents the coordinates of the ideal image point of the light ray on the front surface of the fiber bundle; The pixel size at the front surface of the fiber bundle; Indicates the focal length of an image objective lens; This represents the cosine of the angle between the incident light direction and the positive X-axis in the ray vector emitted from a single rotating optical wedge. This represents the cosine of the angle between the incident light direction and the positive y-axis in the ray vector emitted from a single rotating optical wedge. This represents the cosine of the angle between the incident light direction and the positive z-axis in the ray vector emitted from a single rotating wedge. and These are the coordinates of the intersection point between the system's optical axis and the image plane.

[0014] Furthermore, by controlling the two-dimensional rotating base, the image point converged by the imaging objective lens is precisely aligned with the core center of a specific fiber in the fiber bundle. An image sensor acquires an image of the back surface of the fiber bundle, and template matching is used to locate the core center coordinates (u) of the luminescent fiber. r ,v r ); By introducing an initial rotation angle β0, an X-axis lateral translation Δx, and a Y-axis vector translation Δy to characterize the rigid geometric transformation between the front and rear surfaces of the fiber bundle, and by adding an additional controllable rotation angle β to characterize the rigid geometric transformation between the front and rear surfaces of the fiber bundle, the actual image point coordinates of the front surface of the fiber bundle are calculated.

[0015] Furthermore, the actual image point coordinates on the front surface of the fiber bundle are calculated, specifically using the following formula:

[0016] in, This represents the actual image point coordinates obtained.

[0017] Furthermore, in step two, the initial state parameters of the fiber optic bundle imaging system are optimized using an optimization method. The optimization method employs the Levenberg-Marquardt method or the trust region method, minimizing the objective function as the sum of the absolute errors between the processed imagined point coordinates and the actual image point coordinates at each sampling point, thereby obtaining the optimal estimate of the initial state parameters of the system.

[0018] Furthermore, step two, the specific process of calculating the optimal rotation angle of the single rotating optical wedge, includes: Set the resolution of the super-resolution image to K×K times the resolution of the original image, and determine the ideal set of sub-pixel displacement sampling coordinates for each coordinate dimension; Using the first frame image acquired when the controllable rotation angle β=0 as the reference image, the coordinates of the reference image point and the corresponding sub-pixel micro-displacement (0,0) are calculated using the obtained optimal estimation parameters; Using the rotation adjustment accuracy of the optical wedge rotating bracket as the step size, the values ​​of β in the range of [0°, 360°) are traversed, and the coordinates of the target image point under each controllable rotation angle are calculated in turn. The integer displacement and sub-pixel micro displacement of each target image point relative to the reference image point are calculated respectively, and the actual coordinate set of sub-pixel displacement sampling is obtained. For each 2D subpixel sampled ideal coordinates, the sum of the absolute errors between these coordinates and the actual coordinates is calculated. The controllable rotation angle corresponding to the minimum error is selected as the optimal approximate rotation angle under that ideal coordinate, ultimately forming a system containing K... 2 The set of optimal rotation angles β opt Complete the parameter calibration.

[0019] Furthermore, step three, which involves super-resolution reconstruction based on the multi-frame micro-displacement images, specifically includes: β in the optimal rotation angle set (1) The image corresponding to 0° is used as the reference image. The integer displacements of the remaining images and the reference image are calculated sequentially, and the images are translated to achieve image registration. By utilizing the subpixel micro-displacement of each frame of the registered image, combined with the Delaunay triangulation method or the bicubic spline interpolation method, super-resolution reconstruction of multi-frame micro-displacement images can be achieved, resulting in high-resolution images.

[0020] Secondly, embodiments of the present invention provide a fiber bundle parameter calibration and image super-resolution system based on beam deflection control, comprising: The display screen, collimator, single rotating optical wedge, imaging objective, fiber optic bundle, microscope, and image sensor are arranged sequentially. The rotating optical wedge, imaging objective lens, and front surface of the fiber bundle are combined into a rigid mechanical body with a common optical axis and mounted on a two-dimensional rotating base; the rear surface of the fiber bundle is connected to the fiber rotating bracket; and the single rotating optical wedge is also connected to the optical wedge rotating bracket. The parameter calibration and super-resolution imaging method described in any of the first aspects of this invention is realized by using the horizontal rotation and pitch tilting motion of the two-dimensional rotating base and the angle adjustment of the rotating support.

[0021] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method and apparatus for parameter calibration and image super-resolution of a single rotating wedge fiber bundle optical system based on beam deflection control, which has the following beneficial effects: The fiber bundle imaging system based on a single rotating optical wedge core device of the present invention has a simple structure and can effectively generate multi-frame micro-displacement image sequences. On this basis, by introducing a triple motion control mechanism of "two-dimensional rotating base + controllable rotation of single rotating optical wedge + controllable rotation of fiber bundle rear surface", the estimation accuracy of the initial state parameters of the fiber bundle optical system can be effectively improved.

[0022] The method for calculating the optimal rotation angle of a single rotating optical wedge provided by this invention can guide the setting of the rotation angle of a single rotating optical wedge in a practical system, and help to effectively implement multi-frame micro-displacement image super-resolution technology.

[0023] The super-resolution imaging technology based on multi-frame micro-displacement images designed in this invention has a clear physical process and does not require building a complex dataset to train a neural network. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0025] Figure 1 This is a structural diagram of the optical system provided in an embodiment of the present invention; Figure 2 This is an overall flowchart provided in the embodiments of the present invention; Figure 3 This is a schematic diagram illustrating the calculation results of the optimal rotation angle of a single rotating optical wedge provided in an embodiment of the present invention; wherein, Figure 3 (a) shows the set of target image point coordinates calculated for all candidate values ​​of controllable rotation angles, with the controllable rotation angle interval between two adjacent image point coordinates being... ; Figure 3(b) shows the distribution of target image point coordinates corresponding to the optimal set of rotation angles found, which are not uniformly distributed. Figure 3 (c) represents the integer pixel offset corresponding to the optimal rotation angle set, which is used to guide the subsequent image registration between the target image and the reference image; Figure 3 (d) represents the subpixel micro-displacement corresponding to the optimal rotation angle set, used to guide the implementation of super-resolution of subsequent multi-frame micro-displacement images. Figure 4 This is a schematic diagram of the registration results of a partially magnified image provided in an embodiment of the present invention; Figure 5 The images shown in the embodiments of the present invention are the original high-resolution image, the single-frame fiber bundle back surface image, the single-frame interpolated image, and the multi-frame micro-displacement super-resolution image. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] This invention constructs a system parameter calibration and super-resolution imaging structural model for a single-rotating wedge fiber bundle optical system, which consists of a "display screen-parallel light tube-single rotating wedge-imaging objective-fiber bundle-microscope-image sensor" link. By combining the ideal image point coordinates and the actual image point coordinates, the initial state parameters of the system, including the initial rotation angle of the single rotating wedge, are optimized and solved based on the optimization method. The optimal rotation angle of the single rotating wedge and the corresponding image micro-displacement are calculated based on the model, thus completing the parameter calibration of the single-rotating wedge fiber bundle optical system. Based on the known displacement, multi-frame micro-displacement super-resolution imaging of the fiber bundle optical system is then implemented.

[0028] The system parameter calibration and super-resolution imaging structural model combine a "single rotating optical wedge + imaging objective lens + fiber bundle front surface" into a rigid mechanical body, maintaining a common optical axis, and then mounts it on a two-dimensional rotating base. The horizontal rotation and pitch tilting motions of the two-dimensional rotating base drive the rigid mechanical body on the fiber bundle front surface to rotate, causing incident parallel light in a fixed direction to image at different image point positions on the fiber bundle front surface.

[0029] For the ideal image point propagation path, let the three-dimensional space vector of the ray incident on the front surface of the single rotating optical wedge in the initial condition be: .in, This represents the cosine of the initial angle between the incident light direction and the positive X-axis. This represents the cosine of the initial angle between the incident light direction and the positive y-axis. This represents the cosine of the initial angle between the incident light direction and the positive z-axis; when the two-dimensional rotating base performs horizontal rotation and pitch tilting motions, the three-dimensional spatial vector of the light ray incident on the single rotating wedge in the transformed new coordinate system. Represented as: (1) In the formula, For the horizontal rotation angle with the two-dimensional rotating base and pitch angle The relevant rotation transformation matrix. Assume the wedge angle of the single rotating optical wedge is... The refractive index is n. The initial rotation angle of the unknown optical wedge is... and known controllable rotation angle Under combined regulation, using the vector form of the law of refraction, the vector of light rays passing through the rear surface of the single rotating optical wedge... The calculation formula is: (2) If the focal length of the imaging objective is According to the camera imaging model, the ideal image point coordinates of the light rays on the front surface of the fiber bundle The calculation formula is as follows: (3) In the formula, The pixel size at the front surface of the fiber bundle; and These are the coordinates of the intersection point between the system's optical axis and the image plane.

[0030] To determine the actual image point propagation path, the horizontal rotation and pitch tilt of the two-dimensional rotating base are controlled, ensuring that the image point converged by the imaging objective is precisely aligned with the center of a specific fiber core within the fiber bundle. Then, an image sensor acquires an image of the back surface of the fiber bundle. The core center coordinates of the light-emitting optical fiber are located by template matching. Eccentricity and other assembly errors in the optical system affect the rigid geometric transformation between the front and rear surface images of the fiber bundle, which includes image rotation and lateral translation, introducing an initial rotation angle. Lateral translation along the x and y axes and This characterizes the geometric transformation between the front and rear surfaces of the fiber bundle. Simultaneously, it aims to improve the incident light vector. With lateral translation and The system identification between them, and the additional introduction of a controllable rotation angle to the back surface of the fiber bundle. The lateral rotational motion is characterized. Then the image point coordinates on the front surface of the fiber bundle... It can be calculated based on the rigid geometric transformation of the front and rear surfaces of the fiber bundle: (4) To reduce the impact of imaging distortion in the optical system, the image point coordinates should be kept as close as possible to the image center during the actual calibration process.

[0031] To accurately calculate the initial state parameters of the system The system utilizes three motion control components—a two-dimensional rotating base, an optical wedge rotating bracket, and an optical fiber rotating bracket—to rotate at different horizontal angles. and pitch angle Controllable rotation angle of the optical wedge Controllable rotation angle of the rotating bracket Obtain under the condition Ideal image point coordinate set of the front surface of the fiber bundle and the actual image point coordinate set Then, the optimal estimate of the parameters to be solved is obtained by minimizing the following error function using the Levenberg-Marquardt (LM) method or the trust region method (dogleg method). : (5) After the initial state parameters of the system are solved, the optimal rotation angle of the single rotating wedge that satisfies the requirement of uniform sub-pixel micro-displacement across multiple frames is further calculated. It is assumed that the super-resolution image resolution is [a certain percentage] of the original image resolution. If the coordinate dimension is multiplied by 1, then each coordinate dimension needs to be interpolated between the integer sampled coordinate points of the original image. Subpixel displacement sampling points can be achieved by adjusting the controllable rotation angle of a single rotating wedge. For a given incident parallel light, when the single rotating wedge rotates at equal intervals around its central axis, the converging image point of the outgoing light after passing through the imaging objective will undergo non-uniform displacement on the front surface of the fiber bundle. To obtain a multi-frame image sequence with approximately uniform subpixel displacement, the controllable rotation angle of the single rotating wedge is first set. The first frame image acquired at that time is used as the reference image. The coordinates of the reference image point are calculated using equations (1)-(3) combined with the estimated values ​​of the initial state parameters of the system. The corresponding subpixel micro-displacement Then set the rotation adjustment accuracy of the optical wedge rotating bracket. For the rotation step size, from Calculate the target image point coordinate set under different controllable rotation angles in sequence. Then, calculate the coordinates of each point in the target image set. Relative to reference image point coordinates integer displacement and subpixel micro-displacement : (6) (7) In the formula, The function represents rounding to positive infinity. Equation (7) provides the actual coordinate set of sub-pixel displacement sampling under different controllable rotation angles. When the super-resolution image resolution is 1 / 3 of the original image resolution. The ideal coordinates for sub-pixel displacement sampling in a certain coordinate dimension are times the original coordinates. For example, when the super-resolution image resolution is 1 / 3 the resolution of the original image... When the displacement is multiplied by 1, the sub-pixel micro-displacement amounts on the X and Y axes are respectively and After determining the sub-pixel micro-displacement, the sub-pixel micro-displacement values ​​along the X and Y axes are sequentially extracted to construct a two-dimensional sub-pixel sampling ideal coordinate set. (For example, the ideal coordinates of a certain two-dimensional sub-pixel sample) Then, it is compared with the actual coordinates of the subpixel displacement sampling. Calculate the sum of the absolute values ​​of the errors together. : (8) Find the controllable rotation angle of the single rotating optical wedge corresponding to the sum of the minimum absolute values ​​of errors. This serves as the optimal approximate rotation angle under the specified two-dimensional sub-pixel sampling ideal coordinates. The optimal rotation angle is found by repeating the above steps. The optimal approximate rotation angle set corresponding to the ideal coordinate set of two-dimensional sub-pixel sampling under the condition of super-resolution image 100x super-resolution. (in (corresponding to the reference image), thereby completing the parameter calibration of the single rotating wedge fiber bundle optical system.

[0032] Finally, based on the calibrated set of optimal approximate rotation angles for a single rotating wedge... , obtain Frame micro-displacement images, and pre-completion of system imaging distortion correction. Let the rotation angle... The corresponding image is used as the reference image. The integer displacement between the remaining images and the reference image is calculated according to Equation (6). Then, the micro-displacement image is translated to complete the image registration between the reference image and the micro-displacement image. Furthermore, the micro-displacement calculated by Equation (7) is combined with the Delaunay Triangulation method or the Cubic Spline Interpolation method to achieve super-resolution of multi-frame micro-displacement images, and finally a high-resolution image that surpasses the original image resolution of the fiber bundle optical system is obtained.

[0033] In one specific embodiment, the present invention first discloses a method for parameter calibration and image super-resolution of a single rotating wedge fiber bundle optical system based on beam deflection control, comprising the following steps: Step 1: Constructing the system link and structure model: An optical link is constructed consisting of a display screen, a collimator, a single rotating optical wedge, an imaging objective, a fiber optic bundle, a microscope, and an image sensor. A system structure model is established in which the "single rotating optical wedge + imaging objective + front surface of fiber optic bundle" are combined into a rigid mechanical body with a common optical axis. The rigid mechanical body is then mounted on a two-dimensional rotating base. Step 2: System initial state parameter calibration: By combining the ideal image point coordinates obtained from ray tracing calculations with the actual image point coordinates obtained from actual image acquisition, the initial state parameters of the system, including the initial rotation angle of the single rotating wedge, are optimized using optimization methods. Then, based on the solution results, the optimal rotation angle of the single rotating wedge and the corresponding image micro-displacement are calculated to complete the system parameter calibration. Step 3: Super-resolution reconstruction of multi-frame micro-displacement images: Based on the optimal approximate rotation angle set of the single rotating wedge obtained by calibration, multiple frames of micro-displacement images are acquired sequentially. After registering the multiple frames of images, super-resolution reconstruction is achieved by interpolation or triangulation methods to obtain high-resolution images that exceed the resolution of the original images.

[0034] The implementation structure for calibration and image super-resolution of the single rotating wedge fiber bundle optical system in this invention is shown in the attached figure. Figure 1 As shown, light emitted from a tiny light-emitting point on the display screen is collimated into a parallel beam after passing through a collimator. This beam is then deflected by a single rotating wedge and converged onto the front surface of the fiber bundle by an imaging objective. The image of the front surface is transmitted to the rear surface via an optical fiber array within the flexible fiber bundle, subsequently imaged by a microscope onto an image sensor, and finally stored in a computer. The system parameter calibration and super-resolution imaging structure model combine the "single rotating wedge + imaging objective + front surface of the fiber bundle" into a rigid structure, mounted on a two-dimensional rotating base. Changes in the image point position are achieved through precise computer control of the horizontal rotation angle and pitch angle of the two-dimensional rotating base, ensuring the image point is accurately positioned at the center of a specific fiber in the fiber bundle. Simultaneously, the rear surface of the fiber bundle is mounted within a rotating support. This support provides high-precision lateral rotation around a central axis, introducing controllable rotational changes to the rigid geometric transformation of the front and rear surfaces of the fiber bundle, further improving the estimation accuracy of the geometric transformation parameters.

[0035] This embodiment is based on the appendix. Figure 2 The flowchart shown is divided into two parts: the system parameter calibration stage and the image super-resolution stage. According to the attached... Figure 1The structural model shown is used to build an optical system, and system calibration and image super-resolution of the single rotating wedge fiber bundle optical system are carried out.

[0036] First, system parameter calibration is performed. For the ideal image point calculation stage, the three-dimensional vector of the incident parallel light collimated from the collimator and output to the single rotating wedge is pre-estimated as follows: The horizontal rotation angle of the two-dimensional rotating base. and pitch angle Under the given conditions, the three-dimensional vector of the incident parallel light in the new coordinate system after the two-dimensional rotating base has undergone horizontal rotation and pitch tilting motions can be calculated according to equation (1). .

[0037] The three-dimensional vector of the incident parallel light in the new coordinate system is obtained. Then, the wedge angle for a single rotating optical wedge is... The refractive index is Controllable rotation angle The vector of light rays transmitted through the surface of the single rotating optical wedge is calculated using equation (2). .

[0038] Then consider the focal length of the imaging objective lens. Pixel size at the front surface of the fiber bundle The coordinates of the ideal image point on the front surface of the fiber bundle are calculated using equation (3). .

[0039] To ensure that the actual image point on the front surface of the fiber bundle accurately falls into the center of the photosensitive area of ​​a certain fiber, the horizontal rotation angle of the two-dimensional rotating base needs to be determined in advance. and pitch angle The angle calibration of the two-dimensional rotating base takes into account the controllable rotation angle of the single rotating optical wedge. And the scanning range and number of scanning points of the two-dimensional rotating base. In this embodiment, during the system parameter calibration stage, the controllable rotation angle range of the single rotating optical wedge is set to... Considering that system imaging distortion will affect the accuracy of solving the initial state parameters, the horizontal pitch scanning range of the two-dimensional rotating base is limited to... Within. Because the system state parameters to be solved... It contains 7 variables to be solved, and is a controllable rotation angle for a specific single rotating optical wedge. Set the horizontal rotation angle of the two-dimensional rotating base. and pitch angle The number of scanning points is 5 (a total of 25 scanning points) to ensure accurate solution of the variables. Therefore, by combining the four sets of controllable rotation angles of the single rotating wedge and the 25 horizontal pitch scanning points of the two-dimensional rotating base for each controllable rotation angle, the coordinates of 100 ideal image points on the front surface of the fiber bundle, aligned with the center of the photosensitive area of ​​the fiber bundle, can be calculated. .

[0040] In the actual image point calculation stage, 100 different horizontal rotation and pitch tilt scanning points of the two-dimensional rotating base were divided into 4 groups, and the controllable rotation angle of the rotating support on the rear surface of the fiber bundle was set to 100°. The rigid geometric transformation relationship between the front and rear surfaces of the fiber bundle was adjusted. Images of the rear surface of the fiber bundle were acquired using an image sensor at 100 scanning points. Based on the acquired images of the back surface of the fiber bundle, template matching was used to locate the core center coordinates of the luminescent fiber. This embodiment sets the standard deviation based on the single-fiber spot characteristics of the fiber bundle. A two-dimensional Gaussian distribution function template is used for template matching with the image of the back surface of the fiber bundle. The location of the brightest point in the image is calculated by detecting the template matching and used as the coordinates of the fiber core center. The rigid geometric transformation parameters of the front and back surfaces of the fiber bundle are pre-estimated as follows: Using equation (4) combined with the controllable rotation angle of the rotating support Calculate the actual image point coordinates on the front surface of the fiber bundle The above calculations were performed sequentially on 100 images of the back surface of the fiber bundle to obtain the actual image point coordinates of 100 fiber bundle front surfaces. .

[0041] For the coordinates of the ideal image point and actual image point coordinates The optimal estimate of the parameter to be solved is obtained by minimizing the error function shown in equation (5) using the Levonberg-Marquardt method (LM method). This embodiment addresses the incident parallel light vector. The estimation error is The initial rotation angle of the single rotating optical wedge and the initial rotation angle of the front and rear surfaces of the fiber bundle The estimation error is Translation of the front and rear surfaces of the fiber bundle The estimation error is It has high estimation accuracy.

[0042] Subsequently, based on the estimation results of the system's initial state parameters, this embodiment searches for the optimal rotation angle of a single rotating optical wedge that satisfies multi-frame uniform sub-pixel micro-displacement. The resolution of the super-resolution image is set to a fraction of the original image resolution. The subpixel displacements on the X and Y axes are both times that of the previous generation. It can construct a set of ideal coordinates containing 16 sub-pixel samples. Therefore, it is necessary to search for the corresponding... The optimal rotation angles for different single-rotating optical wedges are determined. The rotation adjustment accuracy of the single-rotating optical wedge mechanical structure used in this embodiment is [insert accuracy here]. Then it is necessary to calculate the candidate values ​​of the controllable rotation angle of a single rotating optical wedge. The coordinates of a total of 1800 ideal image points under the given conditions. Under the given conditions, the coordinates of the reference image point are calculated using equations (1)-(3) combined with the estimated initial state parameters of the system. Then, the target image point coordinate set corresponding to the remaining 1799 controllable rotation angle candidate values ​​is calculated. The integer displacement relative to the reference image point coordinates is calculated using equations (6) and (7), respectively. and subpixel micro-displacement Furthermore, equation (8) is used to search for the optimal rotation angle set corresponding to the ideal coordinate set of the 16 sub-pixel samples. Complete the system parameter calibration. (Attached) Figure 3 The calculation results of the optimal rotation angle of the single rotating optical wedge in this embodiment are shown. Figure 3 (a) shows the set of target image point coordinates calculated for all candidate values ​​of controllable rotation angles, with the controllable rotation angle interval between two adjacent image point coordinates being... ; Figure 3 (b) shows the distribution of target image point coordinates corresponding to the optimal set of rotation angles found, which are not uniformly distributed. Figure 3 (c) represents the integer pixel offset corresponding to the optimal rotation angle set, which is used to guide the subsequent image registration between the target image and the reference image; Figure 3 (d) represents the subpixel micro-displacement corresponding to the optimal rotation angle set, which is used to guide the implementation of super-resolution of subsequent multi-frame micro-displacement images.

[0043] Based on the calibrated set of optimal rotation angles for a single rotating optical wedge Adjust the controllable rotation angle of the single rotating optical wedge and collect the corresponding data. The image of the back surface of the fiber bundle is generated, and system imaging distortion correction is performed simultaneously. Let the rotation angle... The corresponding image is used as the reference image, and the remaining images are the target images. Image registration between the target and reference images is guided by previously calculated integer displacements. (Appendix) Figure 4 The results of partial local magnified image registration are shown, among which... This represents the actual coordinates of the subpixel displacement sampling in the current image. For the fiber bundle back surface image sequence and the set of actual subpixel displacement sampling coordinates, the Delaunay triangulation method is used to achieve super-resolution of multi-frame micro-displacement images. (See appendix) Figure 5 The images showcased original high-resolution images, single-frame back surface images of fiber bundles, single-frame interpolated images, and multi-frame micro-displacement super-resolution images. Compared to single-frame back surface images of fiber bundles, multi-frame micro-displacement super-resolution images effectively eliminated honeycomb artifacts and enhanced the browsing experience. At the same time, compared to single-frame interpolated images, they had higher resolution, no jagged edges, and improved detail resolution. The overall image quality was close to that of the original high-resolution image.

[0044] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0045] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for fiber bundle parameter calibration and image super-resolution based on beam deflection control, characterized in that, include: Step 1: Construct a fiber optic bundle imaging system based on a single rotating optical wedge as the core component. This includes a display screen, collimator, single rotating optical wedge, imaging objective, fiber optic bundle, microscope, and image sensor, sequentially connected. The single rotating optical wedge, imaging objective, and the front surface of the fiber optic bundle are combined into a rigid mechanical body sharing a common optical axis and mounted on a two-dimensional rotating base. The rear surface of the fiber optic bundle is connected to a fiber optic rotating support. The single rotating optical wedge is also connected to the optical wedge rotating support. Step 2: Based on the constructed fiber optic bundle imaging system, by combining the ideal image point coordinates obtained from ray tracing calculations with the actual image point coordinates obtained from actual image acquisition, the initial state parameters of the fiber optic bundle imaging system are optimized using an optimization method. Then, based on the solution results, the optimal rotation angle of the single rotating wedge and the corresponding image micro-displacement are calculated, completing the system parameter calibration. Step 3: Based on the optimal approximate rotation angle set of the single rotating wedge obtained by calibration, multiple frames of micro-displacement images are acquired sequentially, and super-resolution reconstruction is achieved based on the multiple frames of micro-displacement images to obtain a high-resolution image that surpasses the resolution of the original image.

2. The method as described in claim 1, characterized in that, In step two, the initial state parameters of the fiber optic bundle imaging system include the incident ray vector of the incident parallel light, the initial rotation angle of the single rotating wedge, the initial rotation angle of the rigid geometric transformation between the front and rear surfaces of the fiber optic bundle, the X-axis lateral translation, and the Y-axis vector translation.

3. The method as described in claim 1, characterized in that, Step two involves calculating the coordinates of the ideal image point based on ray tracing, specifically including the following steps: Based on the horizontal rotation angle θ and pitch tilt angle of the two-dimensional rotating base of the fiber optic imaging system Construct the rotation transformation matrix The incident ray vector of the fiber optic imaging system is transformed into a three-dimensional spatial vector v in a new coordinate system. rot ; Using the wedge angle α, refractive index n, and controllable rotation angle ψ of a single rotating optical wedge, along with the initial rotation angle... The sum of these values ​​is used to calculate the vector of the outgoing ray after passing through the single rotating optical wedge, based on the vector form of the law of refraction: Based on the obtained single-rotation wedge outgoing ray vector, combined with the focal length of the imaging objective and the pixel size at the front surface of the fiber bundle, the coordinates of the ideal image point are calculated based on the camera imaging model.

4. The method according to claim 3, characterized in that, The vector vector of the outgoing ray after passing through the single rotating optical wedge is calculated according to the vector form of the law of refraction, specifically including the following formula: Among them, v out This represents the vector of the outgoing light rays after passing through the single rotating optical wedge.

5. The method according to claim 3, characterized in that, Based on the obtained single-rotation wedge outgoing ray vector, combined with the focal length f of the imaging objective lens and the pixel size d at the front surface of the fiber bundle, the coordinates of the ideal image point are calculated based on the camera imaging model, specifically including the following formulas: In the formula, Represents the coordinates of the ideal image point of the light ray on the front surface of the fiber bundle; The pixel size at the front surface of the fiber bundle; Indicates the focal length of an image objective lens; This represents the cosine of the angle between the incident light direction and the positive X-axis in the ray vector emitted from a single rotating optical wedge. This represents the cosine of the angle between the incident light direction and the positive y-axis in the ray vector emitted from a single rotating optical wedge. This represents the cosine of the angle between the incident light direction and the positive z-axis in the ray vector emitted from a single rotating wedge. and These are the coordinates of the intersection point between the system's optical axis and the image plane.

6. The method according to claim 1, characterized in that, Step two, based on the actual image point coordinates obtained from the actual image acquisition, specifically includes: The two-dimensional rotating base is controlled to precisely align the image point converged by the imaging objective with the core center of a specific fiber in the fiber bundle. An image sensor acquires an image of the back surface of the fiber bundle, and template matching is used to locate the core center coordinates (u) of the luminescent fiber. r ,v r ); By introducing an initial rotation angle β0, an X-axis lateral translation Δx, and a Y-axis vector translation Δy to characterize the rigid geometric transformation between the front and rear surfaces of the fiber bundle, and by adding an additional controllable rotation angle β to characterize the rigid geometric transformation between the front and rear surfaces of the fiber bundle, the actual image point coordinates of the front surface of the fiber bundle are calculated.

7. The method according to claim 6, characterized in that, The actual image point coordinates on the front surface of the fiber bundle are calculated using the following formulas: in, This represents the actual image point coordinates obtained.

8. The method according to claim 1, characterized in that, Step two involves using optimization methods to solve for the initial state parameters of the fiber optic bundle imaging system, and then calculating the optimal rotation angle of the single rotating wedge and the corresponding image micro-displacement based on the solution results. Specifically, this includes: The sum of the absolute errors between the imagined point coordinates and the actual image point coordinates at each sampling point is used as the objective function to minimize the initial state parameters of the fiber bundle imaging system using the Levenberg-Marquardt method or the trust region method, thereby obtaining the optimal estimate of the initial state parameters of the system. Set the resolution of the super-resolution image to K×K times the resolution of the original image, and determine the ideal set of sub-pixel displacement sampling coordinates for each coordinate dimension; Using the first frame image acquired when the controllable rotation angle β=0 as the reference image, the coordinates of the reference image point and the corresponding sub-pixel micro-displacement (0,0) are calculated using the obtained optimal estimation parameters; Using the rotation adjustment accuracy of the optical wedge rotating bracket as the step size, the values ​​of β in the range of [0°, 360°) are traversed, and the coordinates of the target image point under each controllable rotation angle are calculated in turn. The integer displacement and sub-pixel micro displacement of each target image point relative to the reference image point are calculated respectively, and the actual coordinate set of sub-pixel displacement sampling is obtained. For each 2D subpixel sampled ideal coordinates, the sum of the absolute errors between these coordinates and the actual coordinates is calculated. The controllable rotation angle corresponding to the minimum error is selected as the optimal approximate rotation angle under that ideal coordinate, ultimately forming a system containing K... 2 The set of optimal rotation angles β opt Complete the parameter calibration.

9. The method according to claim 1, characterized in that, Step three involves super-resolution reconstruction based on the multi-frame micro-displacement images, specifically including: β in the optimal rotation angle set (1) The image corresponding to 0° is used as the reference image. The integer displacements of the remaining images and the reference image are calculated sequentially, and the images are translated to achieve image registration. By utilizing the subpixel micro-displacement of each frame of the registered image, combined with the Delaunay triangulation method or the bicubic spline interpolation method, super-resolution reconstruction of multi-frame micro-displacement images can be achieved, resulting in high-resolution images.

10. A single rotating wedge fiber bundle optical system, characterized in that, include: The display screen, collimator, single rotating optical wedge, imaging objective, fiber optic bundle, microscope, and image sensor are arranged sequentially. The rotating optical wedge, imaging objective lens, and front surface of the fiber bundle are combined into a rigid mechanical body with a common optical axis and mounted on a two-dimensional rotating base; the rear surface of the fiber bundle is connected to the fiber rotating bracket; and the single rotating optical wedge is also connected to the optical wedge rotating bracket. The parameter calibration and super-resolution imaging method described in any one of claims 1 to 9 is achieved by using the horizontal rotation and pitch tilting motion of the two-dimensional rotating base and the angle adjustment of the rotating support.

Images