A confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer

Abstract

The present invention discloses a method for correcting confocal microscopic image distortion based on a two-axis MEMS scanning galvanometer. The method comprises: acquiring an original image using a confocal microscope equipped with a two-axis MEMS scanning galvanometer; calculating the galvanometer normal vector; calculating the incident light unit vector; calculating the reflected light unit vector; calculating the coordinates projected onto the scanned end face; and correlating the coordinates with pixels for image correction. The method not only removes the distortion introduced by the MEMS galvanometer but also corrects it when the scanned end face's posture is unknown, thereby obtaining a distortion-free confocal image.

Description

Technical Field

[0001] The present invention belongs to the technical field of microscopes, and in particular to a confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer. Background Art

[0002] Laser scanning confocal microscopy is currently the most commonly used fluorescence imaging tool. By filtering out-of-focus light through a conjugate pinhole, signal quality can be effectively enhanced and confocal microscopy can be equipped with optical sectioning capabilities. Confocal microscopy is a point-scanning imaging mode. Laser scanning is currently commonly implemented using a galvanometer mirror assembly (galvanometer mirror-resonant galvanometer mirror) (e.g., Wang, Jiafu, et al. "A confocal endoscope for cellular imaging." Engineering 1.3 (2015): 351-360; Xu, Baoteng, et al. "Real-time correction of transverse dislocation and distortion in probe-based confocal laserendomicroscopy imaging." Optics Communications 522 (2022): 128653). However, the use of a galvanometer mirror assembly has the disadvantages of being bulky and difficult to assemble and adjust. The use of MEMS galvanometer mirrors can effectively overcome these drawbacks. Currently, there are invention patents for using MEMS galvanometers to achieve confocal microscopy (such as: a MEMS-based handheld laser scanning confocal microscope. Publication number: CN213715590U, publication date: 2021.7.16 Application number: CN202022648855.9). Although MEMS galvanometers can effectively reduce the system volume and the difficulty of installation and adjustment, it is well known that MEMS galvanometers, like galvanometer groups, will introduce unique image distortion. However, the existing solutions do not perform any processing or correction for the image distortion introduced by MEMS galvanometers. In addition to the distortion introduced by the MEMS galvanometer, if the scanning end face of the MEMS reflected light has pitch or yaw, it will further introduce distortion. Therefore, it is necessary to propose a MEMS confocal image distortion correction method based on a standard imaging test target, so that the image can be optimally corrected under the condition of unknown scanned end face posture. Summary of the Invention

[0003] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and provide a method for correcting confocal microscopic image distortion based on a two-axis MEMS scanning galvanometer, which can realize confocal microscopic imaging using a two-axis MEMS scanning galvanometer and can effectively correct the distortion introduced by the MEMS scanning galvanometer and the pitch and yaw of the scanned end surface.

[0004] The technical solution to achieve the purpose of the present invention is: a confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer, the method comprising:

[0005] Step 1: Use a confocal microscope system based on a two-axis MEMS scanning galvanometer to collect the original distorted image of the standard imaging test target;

[0006] Step 2: Determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer;

[0007] Step 3: Based on the parameters determined in step 2, the image is subjected to distortion correction using a distortion correction method. Specifically, the actual scanning coordinates corresponding to each pixel in the image are calculated, associated with the original pixel, and the positions are redistributed to obtain a distortion-corrected image.

[0008] Step 4: Detect straight line segments in the distortion-corrected image and calculate the horizontality and verticality of the horizontal and vertical line segments;

[0009] Step 5, determining whether the horizontality and verticality meet the corresponding preset requirements. If any one of the horizontality and verticality does not meet the requirements, proceed to step 6; otherwise, proceed to step 7;

[0010] Step 6, change the α preset in step 2 xij , α yij , γ x , γ y , repeat steps 3 to 5 to correct the image, and then proceed to step 7;

[0011] Step 7: Determine whether the corrected image meets the preset correction requirements. If so, the correction is completed, and the correction result and the pitch and yaw angles of the corresponding scanned sample plane are obtained, as well as the one-to-one mapping relationship between each original pixel point in the image and the actual scanning coordinates are obtained; if not, return to step 6;

[0012] Step 8: Using the mapping relationship obtained in step 7, perform distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

[0013] Furthermore, the line connecting the center of the scanned sample plane and the center of the MEMS scanning galvanometer in step 2 should be perpendicular to the γ x =γ y =0 when the scanned sample plane.

[0014] Furthermore, step 3, based on the parameters determined in step 2, uses a distortion correction method to correct the image distortion, specifically including:

[0015] Based on γ x =γ y = 0, the normal vector of the scanned sample plane is the Z axis, the sample plane is the X axis and the Y axis, and the XYZ coordinate system is established;

[0016] Step 3-1: Determine the unit normal vector of the MEMS scanning galvanometer based on the parameters determined in step 2. ;

[0017] Step 3-2, determine the incident light unit vector , and the reflected light unit vector ;

[0018] Step 3-3, determining the coordinates of the intersection point O of the scanned sample surface and a perpendicular line passing through the center point of the MEMS scanning galvanometer and perpendicular to the scanned sample surface in the XYZ coordinate system;

[0019] Step 3-4: Based on the data information obtained in the above steps, the coordinates (x, y, z) of the reflected light incident on the scanned sample plane are calculated, thereby obtaining the corresponding relationship between the spatial position, i.e., the actual scanning coordinates, and the angle of the pixel point in the i-th row and j-th column;

[0020] Step 3-5: By calculating the actual scanning coordinates corresponding to each pixel point, the pixel points are redistributed to obtain a distortion-corrected image.

[0021] Furthermore, the unit normal vector of the MEMS scanning galvanometer in step 3-1 is for:

[0022]

[0023] If the scanned sample plane has pitch and yaw angles γ x , γ y , then the unit normal vector of the MEMS scanning galvanometer is Changes to:

[0024] .

[0025] Furthermore, in step 3-2, the laser incident position is set to the center of the MEMS scanning galvanometer and reflection occurs there, and the incident light unit vector is derived by spatial geometry. , reflected light unit vector They are:

[0026]

[0027]

[0028] Where, are the unit normal vectors Components along the X, Y, and Z axes.

[0029] Furthermore, the coordinates of the intersection point O in step 3-3 in the XYZ coordinate system are:

[0030]

[0031] The calculation formula for the coordinates (x, y, z) of the reflected light incident on the scanned sample plane in steps 3-4 is:

[0032]

[0033] in, 、 、 They are Components along the X, Y, and Z axes.

[0034] Furthermore, in step 4, the horizontality and verticality of the horizontal and vertical line segments are calculated, specifically including:

[0035] If a total of M horizontal line segments are detected, there are M-1 pairs of adjacent horizontal line segments; if a total of N vertical line segments are detected, there are N-1 pairs of adjacent vertical line segments;

[0036] For each pair of adjacent horizontal line segments, calculate the distance between the left endpoints and the distance between the right endpoints of the two line segments;

[0037] For each pair of adjacent vertical line segments, calculate the distance between the upper endpoints and the distance between the lower endpoints of the two segments;

[0038] Based on all the spacings obtained above, the horizontality ΔHor and verticality ΔVer of the horizontal and vertical line segments after distortion correction are calculated respectively:

[0039]

[0040] Where Δd left_m , Δdright_m The left and right endpoints of the two segments in the mth pair of adjacent horizontal segments are respectively, m=1,2,...,M-1; Δd up_n , Δd down_n They are the upper endpoint distance and lower endpoint distance of the two segments in the nth pair of adjacent vertical segments, n=1,2,...,N-1.

[0041] Furthermore, the confocal microscope system based on the two-axis MEMS scanning galvanometer in step 1 includes:

[0042] a confocal microscope for confocal detection and imaging of samples;

[0043] A signal processor is used to analyze and process the sample information obtained by reciprocating scanning.

[0044] Furthermore, the confocal microscope includes a laser, a MEMS scanning galvanometer, a dichroic mirror and a photodetector, and the photodetector is one of the following: a photomultiplier tube or an avalanche photodiode.

[0045] Furthermore, the signal processor performs analysis and processing, specifically including:

[0046] Controlling the scanning angle of the MEMS scanning galvanometer;

[0047] receiving a photodetector signal;

[0048] Image reconstruction;

[0049] Perform distortion correction on distorted images.

[0050] Compared with the prior art, the present invention has the following significant advantages:

[0051] (1) By establishing the connection between the actual scanning position of the MEMS scanning mirror at any time and the image pixel point, the distortion introduced by the MEMS scanning mirror can be effectively corrected.

[0052] (2) The method of the present invention is the first to achieve the elimination of distortion introduced by the pitch and yaw of the scanned end face without knowing the pitch and yaw conditions of the scanned end face.

[0053] (3) The present invention can effectively correct the distortion of a two-axis MEMS confocal microscope and improve the image quality.

[0054] The present invention is further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 The flowchart of a method for correcting MEMS confocal microscopy image distortion in one embodiment is shown.

[0056] Figure 2 The figure is a structural diagram of a confocal microscope system based on a two-axis MEMS scanning galvanometer in one embodiment.

[0057] Figure 3 FIG. 1 is a schematic diagram of the MEMS confocal microscopy image distortion correction result of the method of the present invention in one embodiment, wherein Figure 3 (a) in the figure is the original raster image captured by the system. Figure 3 (b) in the figure is the result of grid distortion correction. Figure 3 (c) in the figure is the original image of the resolution plate collected by the system. Figure 3 (d) is the mapping relationship obtained by the present invention. Figure 3 (c) is the result of distortion correction. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0059] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), such directional indications are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0060] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features specified as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0061] In one embodiment, combined Figure 1 , provides a confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer, the method comprising:

[0062] S1, using a confocal microscope system based on a two-axis MEMS scanning galvanometer to collect the original distorted image of the standard imaging test target;

[0063] S2, determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer;

[0064] S3, based on the parameters determined in S2, uses a distortion correction method to correct the image distortion. Specifically, the actual scanning coordinates corresponding to each pixel in the image are calculated, associated with the original pixel, and the positions are redistributed to obtain a distortion-corrected image.

[0065] S4, detecting straight line segments in the distortion-corrected image, and calculating the horizontality and verticality of the horizontal and vertical line segments;

[0066] S5, determining whether the horizontality and verticality meet corresponding preset requirements. If either the horizontality or verticality does not meet the requirements, executing S6; otherwise, executing S7;

[0067] S6, change the α preset in S2 xij , α yij , γ x , γ y , repeat S3 to S5 to correct the image, and then execute S7;

[0068] S7, determining whether the corrected image meets the preset correction requirements. If so, the correction is completed, and the correction result and the pitch and yaw angles of the corresponding scanned sample plane are obtained, as well as the one-to-one mapping relationship between each original pixel point of the image and the actual scanning coordinate are obtained; if not, returning to S6;

[0069] S8, using the mapping relationship obtained in S7, performing distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

[0070] Further preferably, in one embodiment, the line connecting the center of the scanned sample plane and the center of the MEMS scanning galvanometer in S2 should be perpendicular to γ x =γ y =0 when the scanned sample plane.

[0071] Further preferably, in one embodiment, the step of performing distortion correction on the image based on the parameters determined in step S2 using a distortion correction method in step S3 specifically includes:

[0072] Since a pixel point corresponds to a temporary position of the galvanometer during the overall scanning process, its X-axis scanning angle is α xij , the Y-axis scanning angle is α yij For the galvanometer plane, its normal vector is constantly changing.

[0073] Based on γ x =γ y = 0, the normal vector of the scanned sample plane is the Z axis, the sample plane is the X axis and the Y axis, and the XYZ coordinate system is established;

[0074] S3-1, based on the parameters determined in S2, determine the unit normal vector of the MEMS scanning galvanometer ;

[0075] Here, the unit normal vector of the MEMS scanning galvanometer at a specific moment is for:

[0076]

[0077] If the scanned sample plane has pitch and yaw angles γ x , γ y , then the unit normal vector of the MEMS scanning galvanometer is Changes to:

[0078] .

[0079] S3-2, determine the incident light unit vector , and the reflected light unit vector ;

[0080] Here, the laser incident position is assumed to be the center of the MEMS scanning galvanometer and reflection occurs there, and the incident light unit vector is derived by spatial geometry. , reflected light unit vector They are:

[0081]

[0082]

[0083] Where, are the unit normal vectors Components along the X, Y, and Z axes.

[0084] S3-3, determining the coordinates of the intersection O of a perpendicular line passing through the center point of the MEMS scanning galvanometer and perpendicular to the surface of the scanned sample and the surface of the scanned sample in the XYZ coordinate system;

[0085] Here, according to the spatial geometry deduction, the coordinates are:

[0086]

[0087] S3-4, based on the data information obtained in the above steps, the coordinates (x, y, z) of the reflected light incident on the scanned sample plane are calculated, thereby obtaining the corresponding relationship between the spatial position, i.e., the actual scanning coordinates, and the angle of the pixel point in the i-th row and j-th column;

[0088] Here, the coordinates (x, y, z) are calculated as:

[0089]

[0090] in, 、 、 They are Components along the X, Y, and Z axes.

[0091] S3-5, by calculating the actual scanning coordinates corresponding to each pixel point, the pixel points are reallocated to obtain a distortion-corrected image.

[0092] Preferably, in some embodiments, in S4 , straight line segments in the corrected image are detected by Hough transform.

[0093] Furthermore, in one embodiment, calculating the horizontality and verticality of the horizontal line segment and the vertical line segment in S4 specifically includes:

[0094] If a total of M horizontal line segments are detected, there are M-1 pairs of adjacent horizontal line segments; if a total of N vertical line segments are detected, there are N-1 pairs of adjacent vertical line segments;

[0095] For each pair of adjacent horizontal line segments, calculate the distance between the left endpoints and the distance between the right endpoints of the two line segments;

[0096] For each pair of adjacent vertical line segments, calculate the distance between the upper endpoints and the distance between the lower endpoints of the two segments;

[0097] Based on all the spacings obtained above, the horizontality ΔHor and verticality ΔVer of the horizontal and vertical line segments after distortion correction are calculated respectively:

[0098]

[0099] Where Δd left_m , Δd right_m The left and right endpoints of the two segments in the mth pair of adjacent horizontal segments are respectively, m=1,2,...,M-1; Δd up_n , Δd down_nare the upper and lower endpoints of the nth pair of adjacent vertical line segments, respectively, where n = 1, 2, ..., N-1. It should be noted that the units of the above calculation results are all pixels.

[0100] Preferably, in some embodiments, the preset requirement in S5 is: whether the horizontality or verticality is less than 5 pixels.

[0101] Furthermore, in one embodiment, the confocal microscope system based on the two-axis MEMS scanning galvanometer described in S1 includes:

[0102] a confocal microscope for confocal detection and imaging of samples;

[0103] A signal processor is used to analyze and process the sample information obtained by reciprocating scanning.

[0104] Furthermore, in one embodiment, the confocal microscope includes a laser, a MEMS scanning galvanometer, a dichroic mirror and a photodetector, and the photodetector is one of the following: a photomultiplier tube or an avalanche photodiode.

[0105] Furthermore, in one embodiment, the signal processor performs analysis and processing, specifically including:

[0106] Controlling the scanning angle of the MEMS scanning galvanometer;

[0107] receiving a photodetector signal;

[0108] Image reconstruction;

[0109] Perform distortion correction on distorted images.

[0110] For example, a confocal microscopy system based on a two-axis MEMS scanning galvanometer is Figure 2 As shown, light emitted by laser 1 is first reflected by dichroic mirror 3 and then incident on two-axis MEMS scanning galvanometer 4. Through two-dimensional scanning by the two-axis MEMS scanning galvanometer 4, the laser scans the sample in the objective lens-sample module 5. Light reflected from the sample passes through dichroic mirror 3 along the original path and is detected by photodetector 2. Signal processor 6 implements MEMS scanning galvanometer control, photodetector signal reception, image reconstruction, and image distortion correction.

[0111] In one embodiment, a confocal microscopy image distortion correction device based on a two-axis MEMS scanning galvanometer is provided, the device comprising the following steps performed in sequence:

[0112] The first module is used to acquire the original distorted image of the standard imaging test target using a confocal microscope system based on a two-axis MEMS scanning galvanometer;

[0113] The second module is used to determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer;

[0114] The third module is used to perform distortion correction on the image using a distortion correction method based on the parameters determined by the second module. Specifically, the actual scanning coordinates corresponding to each pixel in the image are calculated, associated with the original pixel points, and the positions are redistributed to obtain a distortion-corrected image.

[0115] The fourth module is used to detect straight line segments in the distortion-corrected image and calculate the horizontality and verticality of horizontal and vertical line segments;

[0116] The fifth module is used to determine whether the horizontality and verticality meet the corresponding preset requirements. If either the horizontality or the verticality does not meet the preset requirements, the sixth module is executed; otherwise, the seventh module is executed;

[0117] The sixth module is used to change the α preset in the second module. xij , α yij , γ x , γ y , repeat the third module to the fifth module to correct the image, and then execute the seventh module;

[0118] The seventh module is used to determine whether the corrected image meets the preset correction requirements. If so, the correction is completed, and the correction results and the pitch and yaw angles of the corresponding scanned sample plane are obtained, as well as the one-to-one mapping relationship between each original pixel point of the image and the actual scanning coordinates are obtained; if not, the process returns to the sixth module for execution;

[0119] The eighth module is used to use the mapping relationship obtained in the seventh module to perform distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

[0120] Regarding the specific definition of the confocal microscopy image distortion correction system based on a two-axis MEMS scanning galvanometer, please refer to the definition of the confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer above, which will not be repeated here. The various modules in the above-mentioned confocal microscopy image distortion correction system based on a two-axis MEMS scanning galvanometer can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0121] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following is achieved:

[0122] S1, using a confocal microscope system based on a two-axis MEMS scanning galvanometer to collect the original distorted image of the standard imaging test target;

[0123] S2, determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer;

[0124] S3, based on the parameters determined in S2, uses a distortion correction method to correct the image distortion. Specifically, the actual scanning coordinates corresponding to each pixel in the image are calculated, associated with the original pixel, and the positions are redistributed to obtain a distortion-corrected image.

[0125] S4, detecting straight line segments in the distortion-corrected image, and calculating the horizontality and verticality of the horizontal and vertical line segments;

[0126] S5, determining whether the horizontality and verticality meet corresponding preset requirements. If either the horizontality or verticality does not meet the requirements, executing S6; otherwise, executing S7;

[0127] S6, change the α preset in S2 xij , α yij , γ x , γ y , repeat S3 to S5 to correct the image, and then execute S7;

[0128] S7, determining whether the corrected image meets the preset correction requirements. If so, the correction is completed, and the correction result and the pitch and yaw angles of the corresponding scanned sample plane are obtained, as well as the one-to-one mapping relationship between each original pixel point of the image and the actual scanning coordinate are obtained; if not, returning to S6;

[0129] S8, using the mapping relationship obtained in S7, performing distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

[0130] The specific limitations of each step can be found in the above-mentioned limitations of the confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer, which will not be repeated here.

[0131] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements:

[0132] S1, using a confocal microscope system based on a two-axis MEMS scanning galvanometer to collect the original distorted image of the standard imaging test target;

[0133] S2, determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer;

[0134] S3, based on the parameters determined in S2, uses a distortion correction method to correct the image distortion. Specifically, the actual scanning coordinates corresponding to each pixel in the image are calculated, associated with the original pixel, and the positions are redistributed to obtain a distortion-corrected image.

[0135] S4, detecting straight line segments in the distortion-corrected image, and calculating the horizontality and verticality of the horizontal and vertical line segments;

[0136] S5, determining whether the horizontality and verticality meet corresponding preset requirements. If either the horizontality or verticality does not meet the requirements, executing S6; otherwise, executing S7;

[0137] S6, change the α preset in S2 xij , α yij , γ x , γ y , repeat S3 to S5 to correct the image, and then execute S7;

[0138] S7, determining whether the corrected image meets the preset correction requirements. If so, the correction is completed, and the correction result and the pitch and yaw angles of the corresponding scanned sample plane are obtained, as well as the one-to-one mapping relationship between each original pixel point of the image and the actual scanning coordinate are obtained; if not, returning to S6;

[0139] S8, using the mapping relationship obtained in S7, performing distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

[0140] The specific limitations of each step can be found in the above-mentioned limitations of the confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer, which will not be repeated here.

[0141] As a specific example, the present invention is further verified and explained.

[0142] By the method of the present invention, the correction result is as follows Figure 3 As shown, Figure 3 (a) shows the original image of the grid captured by a confocal microscope system based on a two-axis MEMS scanning galvanometer. The original image has obvious distortion, with the left side appearing contracted and the right side appearing stretched. The scanned end face has an obvious unknown pitch angle. Figure 3 (b) is the image after distortion correction. It can be seen that the parallelism and verticality of the grid stripes are at a good level. After obtaining the mapping relationship between the actual coordinates and each pixel of the image by shooting the grid, the resolution board is photographed in real time. Figure 3 As shown in (c) in the figure, the distortion correction is performed and the result is as follows: Figure 3 As shown in (d), it can be seen that the distortion in the original resolution plate image is effectively controlled.

[0143] In summary, the method of the present invention can not only remove the distortion caused by MEMS micromirror scanning and the pitch and yaw of the scanned end surface, but also correct the distortion introduced when the posture of the scanned end surface is unknown, thereby obtaining a distortion-free confocal image.

[0144] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only illustrative of the principles of the present invention. Without departing from the spirit and scope of the present invention, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer, characterized in that: The method comprises: Step 1: Use a confocal microscope system based on a two-axis MEMS scanning galvanometer to collect the original distorted image of the standard imaging test target; Step 2: Determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer; Step 3: Based on the parameters determined in step 2, the image is subjected to distortion correction using a distortion correction method. Specifically, the actual scanning coordinates corresponding to each pixel in the image are calculated, associated with the original pixel, and the positions are redistributed to obtain a distortion-corrected image. Step 4: Detect straight line segments in the distortion-corrected image and calculate the horizontality and verticality of the horizontal and vertical line segments; Step 5, determining whether the horizontality and verticality meet the corresponding preset requirements. If any one of the horizontality and verticality does not meet the preset requirements, proceed to step 6; otherwise, proceed to step 7; Step 6, change the α preset in step 2 xij , α yij , γ x , γ y , repeat steps 3 to 5 to correct the image, and then proceed to step 7; Step 7: Determine whether the corrected image meets the preset correction requirements. If so, the correction is completed, and the correction result and the pitch and yaw angles of the corresponding scanned sample plane are obtained, as well as the one-to-one mapping relationship between each original pixel point in the image and the actual scanning coordinates are obtained; if not, return to step 6; Step 8: Using the mapping relationship obtained in step 7, perform distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

2. The confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 1, characterized in that: The line connecting the center of the sample plane being scanned and the center of the MEMS scanning galvanometer in step 2 should be perpendicular to the γ x =γ y =0 when the scanned sample plane.

3. The confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 1, characterized in that: Step 3, based on the parameters determined in step 2, uses a distortion correction method to correct the image distortion, specifically including: Based on γ x =γ y = 0, the normal vector of the scanned sample plane is the Z axis, the sample plane is the X axis and the Y axis, and the XYZ coordinate system is established; Step 3-1: Determine the unit normal vector of the MEMS scanning galvanometer based on the parameters determined in step 2. ; Step 3-2, determine the incident light unit vector , and the reflected light unit vector ; Step 3-3, determining the coordinates of the intersection point O of the scanned sample surface and a perpendicular line passing through the center point of the MEMS scanning galvanometer and perpendicular to the scanned sample surface in the XYZ coordinate system; Step 3-4: Based on the data information obtained in the above steps, the coordinates (x, y, z) of the reflected light incident on the scanned sample plane are calculated, thereby obtaining the corresponding relationship between the spatial position, i.e., the actual scanning coordinates, and the angle of the pixel point in the i-th row and j-th column; Step 3-5: By calculating the actual scanning coordinates corresponding to each pixel point, the pixel points are redistributed to obtain a distortion-corrected image.

4. The confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 3, characterized in that: The unit normal vector of the MEMS scanning galvanometer in step 3-1 for: ； If the scanned sample plane has pitch and yaw angles γ x , γ y , then the unit normal vector of the MEMS scanning galvanometer is Changes to: 。 5. The confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 4, characterized in that: In step 3-2, the laser incident position is assumed to be the center of the MEMS scanning galvanometer and reflection occurs there. The unit vector of the incident light is derived by spatial geometry. , reflected light unit vector They are: ； ； Where, are the unit normal vectors Components along the X, Y, and Z axes.

6. The confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 5, characterized in that: The coordinates of the intersection point O in step 3-3 in the XYZ coordinate system are: ； The calculation formula for the coordinates (x, y, z) of the reflected light incident on the scanned sample plane in steps 3-4 is: ； in, 、 、 They are Components along the X, Y, and Z axes.

7. The confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 1, characterized in that: In step 4, the horizontality and verticality of the horizontal and vertical line segments are calculated, specifically including: If a total of M horizontal line segments are detected, there are M-1 pairs of adjacent horizontal line segments; if a total of N vertical line segments are detected, there are N-1 pairs of adjacent vertical line segments; For each pair of adjacent horizontal line segments, calculate the distance between the left endpoints and the distance between the right endpoints of the two line segments; For each pair of adjacent vertical line segments, calculate the distance between the upper endpoints and the distance between the lower endpoints of the two segments; Based on all the spacings obtained above, the horizontality ΔHor and verticality ΔVer of the horizontal and vertical line segments after distortion correction are calculated respectively: ； Where Δd left_m , Δd right_m The left and right endpoints of the two segments in the mth pair of adjacent horizontal segments are respectively, m=1,2,...,M-1; Δd up_n , Δd down_n They are the upper endpoint distance and lower endpoint distance of the two segments in the nth pair of adjacent vertical segments, n=1,2,...,N-1.

8. The confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 1, characterized in that: The confocal microscope system based on the two-axis MEMS scanning galvanometer described in step 1 includes: a confocal microscope for confocal detection and imaging of samples; A signal processor is used to analyze and process the sample information obtained by reciprocating scanning.

9. The method for correcting confocal microscopic image distortion based on a two-axis MEMS scanning galvanometer according to claim 8, characterized in that: The confocal microscope includes a laser, a MEMS scanning galvanometer, a dichroic mirror and a photodetector. The photodetector is one of the following: a photomultiplier tube or an avalanche photodiode.

10. The confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 9, characterized in that: The signal processor performs analysis and processing, specifically including: Controlling the scanning angle of the MEMS scanning galvanometer; receiving a photodetector signal; Image reconstruction; Perform distortion correction on distorted images.

Images