Confocal microscopic image distortion correction method based on two-axis MEMS scanning galvanometer

Abstract

The invention discloses a confocal microscopic image distortion correction method based on a two-axis MEMS (Micro Electro Mechanical System) scanning galvanometer. The method comprises the following steps: acquiring an original image by using a confocal microscope of a two-axis MEMS scanning galvanometer; calculating to obtain a galvanometer normal vector; calculating to obtain an incident light unit vector; calculating to obtain a reflected light unit vector; according to the method, distortion introduced by the MEMS galvanometer can be removed, distortion introduced by the MEMS galvanometer can be corrected under the condition that the posture of the scanned end face is unknown, and then a distortionless confocal image is obtained.

Description

Technical Field

[0001] The invention belongs to the technical field of microscopes, and in particular to a confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer. Background Art

[0002] Laser scanning confocal microscopy is currently the most commonly used fluorescence imaging tool. By filtering out-of-focus light through a conjugate pinhole, the signal quality can be effectively enhanced, and the confocal microscope has an optical sectioning function. Confocal microscopy imaging is a point scanning imaging mode. The laser scanning implementation method is currently generally based on a galvanometer mirror group (galvanometer galvanometer-resonance galvanometer) (such as Wang, Jiafu, et al. "A confocal endoscope for cellular imaging." Engineering 1.3 (2015): 351-360; Xu, Baoteng, et al. "Real-time correction of transverse dislocation and distortion in probe-based confocal laserendomicroscopy imaging." Optics Communications 522 (2022): 128653). However, the use of a galvanometer mirror group has the disadvantages of large size and difficulty in assembly and adjustment, while the use of a MEMS galvanometer mirror can effectively overcome these disadvantages. At present, there are invention patents that use MEMS galvanometers to achieve confocal microscopy (such as: a handheld laser scanning confocal microscope based on MEMS. Publication number: CN213715590U, publication date: 2021.7.16 Application number: CN202022648855.9). Although MEMS galvanometers can effectively reduce the system volume and installation difficulty, it is well known that MEMS galvanometers, like galvanometer groups, will introduce unique image distortions. However, the existing solutions do not perform any processing or correction for the image distortion introduced by MEMS galvanometers. In addition to the distortion introduced by the MEMS galvanometer, if there is pitch or yaw on the scanning end face of the MEMS reflected light, further distortion will be introduced. Therefore, it is necessary to propose a MEMS confocal image distortion correction method based on a standard imaging test target, so as to achieve optimal distortion correction of the image under the condition of unknown scanned end face posture. Summary of the invention

[0003] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and to provide a confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer, which can realize confocal microscopy imaging using a two-axis MEMS scanning galvanometer and can effectively correct the distortion introduced by the MEMS scanning galvanometer and the pitch and yaw of the scanned end surface.

[0004] The technical solution to achieve the purpose of the present invention is: a confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer, the method comprising:

[0005] Step 1, using a confocal microscope system based on a two-axis MEMS scanning galvanometer to collect an original distorted image of a standard imaging test target;

[0006] Step 2: Determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer;

[0007] Step 3: Based on the parameters determined in step 2, the image is subjected to distortion correction using a distortion correction method. Specifically, the actual scanning coordinates corresponding to each pixel in the image are calculated, associated with the original pixel, and the positions are redistributed to obtain a distortion-corrected image.

[0008] Step 4, detecting straight line segments in the distortion-corrected image, and calculating the horizontality and verticality of the horizontal and vertical line segments;

[0009] Step 5, judging whether the horizontality and verticality meet the corresponding preset requirements, if any one of the horizontality and verticality does not meet the requirements, executing step 6, otherwise executing step 7;

[0010] Step 6, change the alpha preset in step 2 xij , α yij , γ x , γ y , repeat steps 3 to 5 to correct the image, and then proceed to step 7;

[0011] Step 7, determine whether the corrected image meets the preset correction requirements. If yes, complete the correction, obtain the correction result and the pitch and yaw angles of the corresponding scanned sample plane, and the one-to-one mapping relationship between each original pixel point of the image and the actual scanning coordinates; if no, return to step 6;

[0012] Step 8: Using the mapping relationship obtained in step 7, perform distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

[0013] Furthermore, the line connecting the center of the scanned sample plane and the center of the MEMS scanning galvanometer in step 2 should be perpendicular to the γ x =γ y =0 when the scanned sample plane.

[0014] Furthermore, in step 3, based on the parameters determined in step 2, the image is subjected to distortion correction using a distortion correction method, which specifically includes:

[0015] Based on γ x =γ y =0, the normal vector of the scanned sample plane is the Z axis, the sample plane is the X axis and the Y axis, and the XYZ coordinate system is established;

[0016] Step 3-1: Based on the parameters determined in step 2, determine the unit normal vector of the MEMS scanning galvanometer. ;

[0017] Step 3-2, determine the incident light unit vector , and the reflected light unit vector ;

[0018] Step 3-3, determining the coordinates of the intersection point O of the vertical line passing through the center point of the mirror surface of the MEMS scanning galvanometer and perpendicular to the surface of the scanned sample and the surface of the scanned sample in the XYZ coordinate system;

[0019] Step 3-4, based on the data information obtained in the above steps, the coordinates (x, y, z) of the reflected light incident on the scanned sample plane are calculated, thereby obtaining the corresponding relationship between the spatial position, i.e., the actual scanning coordinates, and the angle of the pixel point in the i-th row and j-th column;

[0020] Step 3-5, by calculating the actual scanning coordinates corresponding to each pixel point, the positions of the pixels are redistributed to obtain a distortion-corrected image.

[0021] Furthermore, the unit normal vector of the MEMS scanning galvanometer in step 3-1 is for:

[0022]

[0023] If the scanned sample plane has pitch and yaw angles γ x , γ y , then the unit normal vector of the MEMS scanning galvanometer is Changes to:

[0024] .

[0025] Furthermore, in step 3-2, the laser incident position is set to the center of the MEMS scanning galvanometer and reflection occurs there, and the incident light unit vector is derived by spatial geometry. , reflected light unit vector They are:

[0026]

[0027]

[0028] In the formula, are the unit normal vectors Components along the X, Y, and Z axes.

[0029] Furthermore, the coordinates of the intersection point O in step 3-3 in the XYZ coordinate system are:

[0030]

[0031] The calculation formula for the coordinates (x, y, z) of the reflected light incident on the scanned sample plane in step 3-4 is:

[0032]

[0033] in, , , They are Components along the X, Y, and Z axes.

[0034] Furthermore, in step 4, the horizontality and verticality of the horizontal line segment and the vertical line segment are calculated, specifically including:

[0035] If M horizontal line segments are detected, there are M-1 pairs of adjacent horizontal line segments; if N vertical line segments are detected, there are N-1 pairs of adjacent vertical line segments;

[0036] For each pair of adjacent horizontal line segments, calculate the distance between the left endpoints and the distance between the right endpoints of the two line segments;

[0037] For each pair of adjacent vertical line segments, calculate the distance between the upper endpoints and the distance between the lower endpoints of the two line segments;

[0038] According to all the spacings obtained above, the horizontality ΔHor and verticality ΔVer of the horizontal and vertical line segments after distortion correction are calculated respectively:

[0039]

[0040] In the formula, Δd left_m , Δdright_m are the distances between the left and right endpoints of the two segments in the mth pair of adjacent horizontal segments, m=1,2,...,M-1; Δd up_n , Δd down_n They are respectively the distance between the upper endpoints and the distance between the lower endpoints of the two segments in the nth pair of adjacent vertical line segments, n=1,2,...,N-1.

[0041] Furthermore, the confocal microscope system based on the two-axis MEMS scanning galvanometer described in step 1 includes:

[0042] a confocal microscope for confocal probing and imaging of the sample;

[0043] A signal processor is used to analyze and process the sample information obtained by reciprocating scanning.

[0044] Furthermore, the confocal microscope comprises a laser, a MEMS scanning galvanometer, a dichroic mirror and a photodetector, and the photodetector is one of the following: a photomultiplier tube or an avalanche photodiode.

[0045] Furthermore, the signal processor performs analysis and processing, specifically including:

[0046] Controlling the scanning angle of the MEMS scanning galvanometer;

[0047] receiving a photodetector signal;

[0048] Image reconstruction;

[0049] Perform distortion correction on distorted images.

[0050] Compared with the prior art, the present invention has the following significant advantages:

[0051] (1) By establishing the connection between the actual scanning position of the MEMS scanning mirror at any time and the image pixel point, the distortion introduced by the MEMS scanning mirror can be effectively corrected.

[0052] (2) The method of the present invention is the first to eliminate the distortion introduced by the pitch and yaw of the scanned end surface without knowing the pitch and yaw conditions of the scanned end surface.

[0053] (3) The present invention can effectively correct the distortion of a two-axis MEMS confocal microscope and improve the image quality.

[0054] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a flow chart of a method for correcting MEMS confocal microscopy image distortion in an embodiment.

[0056] Figure 2 The figure is a structural diagram of a confocal microscope system based on a two-axis MEMS scanning galvanometer in one embodiment.

[0057] Figure 3 FIG. 1 is a schematic diagram of the MEMS confocal microscopy image distortion correction result of the method of the present invention in one embodiment, wherein Figure 3 (a) in the figure is the original raster image captured by the system. Figure 3 (b) in the figure is the grid distortion correction result. Figure 3 (c) in the figure is the original image of the resolution plate collected by the system. Figure 3 (d) is the mapping relationship obtained by the present invention. Figure 3 (c) is the result of distortion correction. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0060] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0061] In one embodiment, in combination Figure 1 , provides a confocal microscopy image distortion correction method based on a two-axis MEMS scanning galvanometer, the method comprising:

[0062] S1, using a confocal microscope system based on a two-axis MEMS scanning galvanometer to collect the original distorted image of the standard imaging test target;

[0063] S2, determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer;

[0064] S3, based on the parameters determined in S2, using a distortion correction method to correct the image distortion, specifically: calculating the actual scanning coordinates corresponding to each pixel point in the image, associating them with the original pixel points, and redistributing the positions to obtain a distortion-corrected image;

[0065] S4, detecting straight line segments in the distortion-corrected image, and calculating the horizontality and verticality of the horizontal line segments and the vertical line segments;

[0066] S5, judging whether the horizontality and verticality meet the corresponding preset requirements, if any one of the horizontality and verticality does not meet the requirements, executing S6, otherwise executing S7;

[0067] S6, change the α preset in S2 xij , α yij , γ x , γ y , repeat S3 to S5 to correct the image, and then execute S7;

[0068] S7, judging whether the corrected image meets the preset correction requirements, if yes, completing the correction, obtaining the correction result and the pitch and yaw angles of the corresponding scanned sample plane, and the one-to-one mapping relationship between each original pixel point of the image and the actual scanning coordinate; if no, returning to S6;

[0069] S8, using the mapping relationship obtained in S7, performing distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

[0070] Further preferably, in one embodiment, the line connecting the center of the scanned sample plane and the center of the MEMS scanning galvanometer in S2 should be perpendicular to γ x =γ y =0 when the scanned sample plane.

[0071] Further preferably, in one embodiment, the step S3 of performing distortion correction on the image based on the parameters determined in S2 by using a distortion correction method specifically includes:

[0072] Since a pixel point corresponds to a temporary position of the galvanometer during the overall scanning process, its X-axis scanning angle is α xij , the Y-axis scanning angle is α yij For the galvanometer plane, its normal vector is constantly changing.

[0073] Based on γ x =γ y =0, the normal vector of the scanned sample plane is the Z axis, the sample plane is the X axis and the Y axis, and the XYZ coordinate system is established;

[0074] S3-1, based on the parameters determined in S2, determine the unit normal vector of the MEMS scanning galvanometer ;

[0075] Here, the unit normal vector of the MEMS scanning galvanometer at a specific moment is for:

[0076]

[0077] If the scanned sample plane has pitch and yaw angles γ x , γ y , then the unit normal vector of the MEMS scanning galvanometer is Changes to:

[0078] .

[0079] S3-2, determine the incident light unit vector , and the reflected light unit vector ;

[0080] Here, the laser incident position is assumed to be the center of the MEMS scanning galvanometer and reflection occurs here. The unit vector of the incident light is derived through spatial geometry: , reflected light unit vector They are:

[0081]

[0082]

[0083] In the formula, are the unit normal vectors Components along the X, Y, and Z axes.

[0084] S3-3, determining the coordinates of the intersection point O of the vertical line passing through the center point of the mirror surface of the MEMS scanning galvanometer and perpendicular to the surface of the scanned sample and the surface of the scanned sample in the XYZ coordinate system;

[0085] Here, according to the spatial geometry derivation, the coordinates are:

[0086]

[0087] S3-4, based on the data information obtained in the above steps, the coordinates (x, y, z) of the reflected light incident on the scanned sample plane are calculated, thereby obtaining the corresponding relationship between the spatial position, i.e., the actual scanning coordinates and the angle of the pixel point in the i-th row and j-th column;

[0088] Here, the coordinates (x, y, z) are calculated as:

[0089]

[0090] in, , , They are Components along the X, Y, and Z axes.

[0091] S3-5, by calculating the actual scanning coordinates corresponding to each pixel point, the positions of the pixels are reallocated to obtain a distortion-corrected image.

[0092] Preferably, in some embodiments, in S4, straight line segments in the corrected image are detected by Hough transform.

[0093] Further, in one embodiment, calculating the horizontality and verticality of the horizontal line segment and the vertical line segment in S4 specifically includes:

[0094] If M horizontal line segments are detected, there are M-1 pairs of adjacent horizontal line segments; if N vertical line segments are detected, there are N-1 pairs of adjacent vertical line segments;

[0095] For each pair of adjacent horizontal line segments, calculate the distance between the left endpoints and the distance between the right endpoints of the two line segments;

[0096] For each pair of adjacent vertical line segments, calculate the distance between the upper endpoints and the distance between the lower endpoints of the two line segments;

[0097] According to all the spacings obtained above, the horizontality ΔHor and verticality ΔVer of the horizontal and vertical line segments after distortion correction are calculated respectively:

[0098]

[0099] In the formula, Δd left_m , Δd right_m are the distances between the left and right endpoints of the two segments in the mth pair of adjacent horizontal segments, m=1,2,...,M-1; Δd up_n , Δd down_nare the upper endpoint distance and the lower endpoint distance of the two line segments in the nth pair of adjacent vertical line segments, n=1,2,...,N-1. It should be noted that the units of the above calculation results are all pixels.

[0100] Preferably, in some embodiments, the preset requirement in S5 is: whether the horizontality or verticality is less than 5 pixels.

[0101] Furthermore, in one of the embodiments, the confocal microscope system based on the two-axis MEMS scanning galvanometer described in S1 includes:

[0102] a confocal microscope for confocal probing and imaging of the sample;

[0103] A signal processor is used to analyze and process the sample information obtained by reciprocating scanning.

[0104] Furthermore, in one of the embodiments, the confocal microscope includes a laser, a MEMS scanning galvanometer, a dichroic mirror and a photodetector, and the photodetector is one of the following: a photomultiplier tube or an avalanche photodiode.

[0105] Furthermore, in one embodiment, the signal processor performs analysis processing, specifically including:

[0106] Controlling the scanning angle of the MEMS scanning galvanometer;

[0107] receiving a photodetector signal;

[0108] Image reconstruction;

[0109] Perform distortion correction on distorted images.

[0110] For example, a confocal microscopy system based on a two-axis MEMS scanning galvanometer is Figure 2 As shown, the light emitted by the laser 1 is first reflected by the dichroic mirror 3 and then incident on the two-axis MEMS scanning galvanometer 4. Through the two-dimensional scanning of the two-axis MEMS scanning galvanometer 4, the laser can scan the sample in the objective lens-sample module 5. The sample reflected light passes through the dichroic mirror 3 along the original path and is detected by the photodetector 2. The signal processor 6 realizes the MEMS scanning galvanometer control, the reception of the photodetector signal, the image reconstruction and the image distortion correction functions.

[0111] In one embodiment, a confocal microscopic image distortion correction device based on a two-axis MEMS scanning galvanometer is provided, the device comprising the following steps performed in sequence:

[0112] The first module is used to collect the original distorted image of the standard imaging test target using a confocal microscope system based on a two-axis MEMS scanning galvanometer;

[0113] The second module is used to determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer;

[0114] The third module is used to perform distortion correction on the image using a distortion correction method based on the parameters determined by the second module, specifically: calculating the actual scanning coordinates corresponding to each pixel point in the image, associating them with the original pixel points, and reallocating the positions to obtain a distortion-corrected image;

[0115] The fourth module is used to detect straight line segments in the distortion-corrected image and calculate the horizontality and verticality of the horizontal and vertical line segments;

[0116] The fifth module is used to determine whether the horizontality and verticality meet the corresponding preset requirements. If any one of the horizontality and verticality does not meet the preset requirements, the sixth module is executed, otherwise the seventh module is executed;

[0117] The sixth module is used to change the α preset in the second module. xij , α yij , γ x , γ y , repeat the third module to the fifth module to correct the image, and then execute the seventh module;

[0118] The seventh module is used to determine whether the corrected image meets the preset correction requirements. If so, the correction is completed to obtain the correction result and the pitch and yaw angles of the corresponding scanned sample plane, as well as the one-to-one mapping relationship between each original pixel point of the image and the actual scanning coordinates; if not, return to execute the sixth module;

[0119] The eighth module is used to use the mapping relationship obtained in the seventh module to perform distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

[0120] For the specific definition of the confocal microscopic image distortion correction system based on a two-axis MEMS scanning galvanometer, please refer to the definition of the confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer in the above text, which will not be repeated here. Each module in the above-mentioned confocal microscopic image distortion correction system based on a two-axis MEMS scanning galvanometer can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0121] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following is achieved:

[0122] S1, using a confocal microscope system based on a two-axis MEMS scanning galvanometer to collect the original distorted image of the standard imaging test target;

[0123] S2, determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer;

[0124] S3, based on the parameters determined in S2, using a distortion correction method to correct the image distortion, specifically: calculating the actual scanning coordinates corresponding to each pixel point in the image, associating them with the original pixel points, and redistributing the positions to obtain a distortion-corrected image;

[0125] S4, detecting straight line segments in the distortion-corrected image, and calculating the horizontality and verticality of the horizontal line segments and the vertical line segments;

[0126] S5, judging whether the horizontality and verticality meet the corresponding preset requirements, if any one of the horizontality and verticality does not meet the requirements, executing S6, otherwise executing S7;

[0127] S6, change the α preset in S2 xij , α yij , γ x , γ y , repeat S3 to S5 to correct the image, and then execute S7;

[0128] S7, judging whether the corrected image meets the preset correction requirements, if yes, completing the correction, obtaining the correction result and the pitch and yaw angles of the corresponding scanned sample plane, and the one-to-one mapping relationship between each original pixel point of the image and the actual scanning coordinate; if no, returning to S6;

[0129] S8, using the mapping relationship obtained in S7, performing distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

[0130] For the specific definition of each step, please refer to the definition of the confocal microscopy image distortion correction method based on the two-axis MEMS scanning galvanometer in the above text, which will not be repeated here.

[0131] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements:

[0132] S1, using a confocal microscope system based on a two-axis MEMS scanning galvanometer to collect the original distorted image of the standard imaging test target;

[0133] S2, determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer;

[0134] S3, based on the parameters determined in S2, using a distortion correction method to correct the image distortion, specifically: calculating the actual scanning coordinates corresponding to each pixel point in the image, associating them with the original pixel points, and redistributing the positions to obtain a distortion-corrected image;

[0135] S4, detecting straight line segments in the distortion-corrected image, and calculating the horizontality and verticality of the horizontal line segments and the vertical line segments;

[0136] S5, judging whether the horizontality and verticality meet the corresponding preset requirements, if any one of the horizontality and verticality does not meet the requirements, executing S6, otherwise executing S7;

[0137] S6, change the α preset in S2 xij , α yij , γ x , γ y , repeat S3 to S5 to correct the image, and then execute S7;

[0138] S7, judging whether the corrected image meets the preset correction requirements, if yes, completing the correction, obtaining the correction result and the pitch and yaw angles of the corresponding scanned sample plane, and the one-to-one mapping relationship between each original pixel point of the image and the actual scanning coordinate; if no, returning to S6;

[0139] S8, using the mapping relationship obtained in S7, performing distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

[0140] For the specific definition of each step, please refer to the definition of the confocal microscopy image distortion correction method based on the two-axis MEMS scanning galvanometer in the above text, which will not be repeated here.

[0141] As a specific example, the present invention is further verified and explained.

[0142] By the method of the present invention, the correction result is as follows Figure 3 As shown, Figure 3 (a) is the original image of the grid taken by the confocal microscope system based on the two-axis MEMS scanning galvanometer. The original image has obvious distortion, with the left side showing a contraction state and the right side showing a stretching state. The scanned end face has an obvious unknown pitch angle. Figure 3 (b) is the image after distortion correction. It can be seen that the parallelism and verticality of the grid stripes are at a good level. After obtaining the mapping relationship between the actual coordinates and each pixel of the image by shooting the grid, the resolution board is photographed in real time. Figure 3 As shown in (c) of the figure, the distortion is corrected and the result is as follows Figure 3 As shown in (d) in the figure, it can be seen that the distortion in the original resolution plate image is effectively controlled.

[0143] In summary, the method of the present invention can not only remove the distortion caused by MEMS micromirror scanning and the pitch and roll of the scanned end surface, but also correct the distortion introduced when the posture of the scanned end surface is unknown, thereby obtaining a distortion-free confocal image.

[0144] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer, characterized in that: The method comprises: Step 1, using a confocal microscope system based on a two-axis MEMS scanning galvanometer to collect an original distorted image of a standard imaging test target; Step 2: Determine the following parameters: the incident angle β of the laser incident on the MEMS scanning galvanometer, the X-axis scanning angle α corresponding to the pixel point in the i-th row and j-th column in the image xij and Y-axis scanning angle α yij , the initial values ​​of pitch and yaw angles required by the distortion correction algorithm γ x , γ y , and the distance L between the center of the scanned sample plane and the center of the MEMS scanning galvanometer; Step 3: Based on the parameters determined in step 2, the image is subjected to distortion correction using a distortion correction method. Specifically, the actual scanning coordinates corresponding to each pixel in the image are calculated, associated with the original pixel, and the positions are redistributed to obtain a distortion-corrected image. Step 4, detecting straight line segments in the distortion-corrected image, and calculating the horizontality and verticality of the horizontal and vertical line segments; Step 5, judging whether the horizontality and verticality meet the corresponding preset requirements, if any one of the horizontality and verticality does not meet the preset requirements, executing step 6, otherwise executing step 7; Step 6, change the alpha preset in step 2 xij , α yij , γ x , γ y , repeat steps 3 to 5 to correct the image, and then proceed to step 7; Step 7, determine whether the corrected image meets the preset correction requirements. If yes, complete the correction, obtain the correction result and the pitch and yaw angles of the corresponding scanned sample plane, and the one-to-one mapping relationship between each original pixel point of the image and the actual scanning coordinates; if no, return to step 6; Step 8: Using the mapping relationship obtained in step 7, perform distortion correction on the microscopic image captured in real time by the confocal microscope system based on the two-axis MEMS scanning galvanometer.

2. The confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 1, characterized in that: The line connecting the center of the sample plane to be scanned and the center of the MEMS scanning galvanometer in step 2 should be perpendicular to the γ x =γ y =0 when the scanned sample plane.

3. The confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 1, characterized in that: Step 3, based on the parameters determined in step 2, uses a distortion correction method to correct the image distortion, specifically including: Based on γ x =γ y =0, the normal vector of the scanned sample plane is the Z axis, the sample plane is the X axis and the Y axis, and the XYZ coordinate system is established; Step 3-1: Based on the parameters determined in step 2, determine the unit normal vector of the MEMS scanning galvanometer. ; Step 3-2, determine the incident light unit vector , and the reflected light unit vector ; Step 3-3, determining the coordinates of the intersection point O of the vertical line passing through the center point of the mirror surface of the MEMS scanning galvanometer and perpendicular to the surface of the scanned sample and the surface of the scanned sample in the XYZ coordinate system; Step 3-4, based on the data information obtained in the above steps, the coordinates (x, y, z) of the reflected light incident on the scanned sample plane are calculated, thereby obtaining the corresponding relationship between the spatial position, i.e., the actual scanning coordinates, and the angle of the pixel point in the i-th row and j-th column; Step 3-5, by calculating the actual scanning coordinates corresponding to each pixel point, the positions of the pixels are redistributed to obtain a distortion-corrected image.

4. The confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 3, characterized in that: The unit normal vector of the MEMS scanning galvanometer in step 3-1 for: ； If the scanned sample plane has pitch and yaw angles γ x , γ y , then the unit normal vector of the MEMS scanning galvanometer is Changes to: 。 5. The confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 4, characterized in that: In step 3-2, the laser incident position is assumed to be the center of the MEMS scanning galvanometer and reflection occurs there. The unit vector of the incident light is derived by spatial geometry. , reflected light unit vector They are: ； ； In the formula, are the unit normal vectors Components along the X, Y, and Z axes.

6. The confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 5, characterized in that: The coordinates of the intersection point O in step 3-3 in the XYZ coordinate system are: ； The calculation formula for the coordinates (x, y, z) of the reflected light incident on the scanned sample plane in step 3-4 is: ； in, , , They are Components along the X, Y, and Z axes.

7. The confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 1, characterized in that: In step 4, the horizontality and verticality of the horizontal line segment and the vertical line segment are calculated, specifically including: If M horizontal line segments are detected, there are M-1 pairs of adjacent horizontal line segments; if N vertical line segments are detected, there are N-1 pairs of adjacent vertical line segments; For each pair of adjacent horizontal line segments, calculate the distance between the left endpoints and the distance between the right endpoints of the two line segments; For each pair of adjacent vertical line segments, calculate the distance between the upper endpoints and the distance between the lower endpoints of the two line segments; According to all the spacings obtained above, the horizontality ΔHor and verticality ΔVer of the horizontal and vertical line segments after distortion correction are calculated respectively: ； In the formula, Δd left_m , Δd right_m are the distances between the left and right endpoints of the two segments in the mth pair of adjacent horizontal segments, m=1,2,...,M-1; Δd up_n , Δd down_n They are respectively the distance between the upper endpoints and the distance between the lower endpoints of the two segments in the nth pair of adjacent vertical line segments, n=1,2,...,N-1.

8. The confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 1, characterized in that: The confocal microscope system based on the two-axis MEMS scanning galvanometer described in step 1 includes: a confocal microscope for confocal probing and imaging of the sample; A signal processor is used to analyze and process the sample information obtained by reciprocating scanning.

9. The confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 8, characterized in that: The confocal microscope comprises a laser, a MEMS scanning galvanometer, a dichroic mirror and a photodetector. The photodetector is one of the following: a photomultiplier tube or an avalanche photodiode.

10. The confocal microscopic image distortion correction method based on a two-axis MEMS scanning galvanometer according to claim 9, characterized in that: The signal processor performs analysis and processing, specifically including: Controlling the scanning angle of the MEMS scanning galvanometer; receiving a photodetector signal; Image reconstruction; Perform distortion correction on distorted images.

Images