Laser confocal scanning microscope, distortion correction method and acquisition card

Abstract

The invention provides a laser confocal scanning microscope, a distortion correction method and an acquisition card, and the laser confocal scanning microscope comprises a transflective mirror which is fixedly installed between a resonance galvanometer and a galvanometer galvanometer, and a monitoring light path and a second detector which are arranged in the transmission direction of the transflective mirror. The acquisition card is respectively connected with the first detector and the second detector; the confocal signal and the grating signal which are synchronously acquired by the acquisition card in an isochronous manner are twin images in time and space, and the grating image formed by the grating signal depicts the movement of the resonance galvanometer in the scanning process, so that distortion correction can be performed on the twin object plane image of the grating image through distortion analysis on the grating image, and the distortion correction accuracy is improved. Therefore, the correction accuracy is effectively improved, and the method is not easily influenced by factors such as environment, service life and optical path deviation.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a laser confocal scanning microscope, a distortion correction method, and an acquisition card. Background Art

[0002] As a high-precision, high-value-added measuring instrument, industrial confocal microscopes are powerful tools for measuring surface topography and play a vital role in the research, development, production, and testing of materials such as ceramics, metals, and semiconductors. Currently, confocal laser scanning has developed a variety of scanning imaging methods, including galvanometer / resonance-based dual-mirror scanning, MEMS (Micro-Electro-Mechanical Systems)-based galvanometer scanning, Nipkow disk scanning, and line scanning. Of these, galvanometer / resonance-based dual-mirror scanning has become the mainstream due to its cost and technological maturity.

[0003] In theory, resonant scanning is designed to perform simple harmonic motion at an inherent resonant frequency, with its angular velocity being maximum at the center and zero at the ends. This causes distortion in the captured image when isochronous sampling is used. Methods for eliminating this distortion can be categorized as hardware correction and software correction. Hardware correction generally involves non-isochronous sampling using a pixel clock. However, the scanning curve of the resonant galvanometer changes with environmental factors such as temperature and humidity, while a fixed pixel clock cannot be changed. Furthermore, a fixed pixel clock limits the image size and pixel count, thereby reducing the scalability of the system. Software correction involves correcting the distorted image after obtaining it using standard isochronous sampling. One method involves sinusoidal correction based on a standard sine curve. This uses a sinusoidal curve to represent the motion state of the galvanometer. However, since it assumes the galvanometer is in an ideal operating state, in actual use, the galvanometer is often affected by factors such as the environment, lifespan, and optical path deviations, causing it to deviate from the standard sinusoidal curve. Therefore, the image corrected by this method still exhibits significant distortion.

[0004] In summary, the hardware correction method or software correction method used in the prior art has low correction accuracy and is easily affected by factors such as the environment, lifespan, and optical path deviation. Summary of the Invention

[0005] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defects that the hardware correction method or software correction method used in the prior art has low correction accuracy and is easily affected by factors such as environment, lifespan and optical path deviation.

[0006] The present application provides a laser confocal scanning microscope, comprising a laser light source, a resonant galvanometer, a galvanometer, a spectroscope, an excitation optical path, a confocal optical path, and a first detector. The microscope further comprises: a transflective mirror fixedly mounted between the resonant galvanometer and the galvanometer, a monitoring optical path and a second detector arranged in the transmission direction of the transflective mirror, and an acquisition card connected to the first and second detectors, respectively.

[0007] When the microscope scans the sample, the light beam emitted by the laser light source is reflected by the beam splitter to the resonant galvanometer mirror. The resonant galvanometer mirror deflects the incident light beam for the first time and then emits it to the transflective mirror. The transflective mirror reflects most of the light beam to the galvanometer galvanometer mirror. The galvanometer galvanometer mirror deflects the incident light beam for the second time and then focuses the deflected light beam onto the sample surface through the excitation optical path. The reflected light excited on the sample surface returns along the original path, is transmitted through the beam splitter to the confocal optical path, and reaches the first detector through the confocal optical path. The transflective mirror transmits a small part of the light beam to the monitoring optical path, and the monitoring optical path converts the real-time deflection angle of the resonant galvanometer mirror into a grating signal and sends it to the second detector.

[0008] The acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, and generates an object plane image corresponding to the confocal signal and a grating image corresponding to the grating signal, and then uses the grating image to perform distortion correction on the object plane image to obtain a target image.

[0009] Optionally, the monitoring optical path includes an attenuation plate and a Ronchi grating;

[0010] The attenuation plate attenuates the transmitted light beam and then transmits it to the Ronchi grating. The Ronchi grating converts the real-time deflection angle of the resonant galvanometer into a grating signal according to the attenuated light beam.

[0011] The present application also provides a distortion correction method, which is applied to an acquisition card in a laser confocal scanning microscope according to any one of the above embodiments, and the method comprises:

[0012] synchronously and isochronously collecting the confocal signal output by the first detector and the grating signal output by the second detector, and generating an object plane image corresponding to the confocal signal and a grating image corresponding to the grating signal;

[0013] The object plane image is subjected to distortion correction using the grating image to obtain a target image.

[0014] Optionally, the performing distortion correction on the object plane image using the grating image to obtain a target image includes:

[0015] determining a position of at least one grid in the raster image at each pixel row;

[0016] The object plane image is corrected row by row according to the position of the at least one grid in each pixel row to obtain a target image.

[0017] Optionally, determining the position of at least one grid in the raster image in each pixel row includes:

[0018] If the current scene is a high-precision scene, determining the positions of all grids in the raster image in each pixel row;

[0019] If the current scene is a real-time scene and the resonant galvanometer scans in one direction, the position of the grid of the region of interest in the grating image in each pixel row is determined.

[0020] Optionally, determining the positions of all grids in the raster image in respective pixel rows includes:

[0021] Determining a row brightness curve corresponding to brightness values ​​of each column in each pixel row of the raster image;

[0022] Smoothing the row brightness curve of each pixel row, and performing peak fitting on the smoothed row brightness curve until the number of peaks in each pixel row is equal, thereby obtaining a first peak sequence for each pixel row;

[0023] The positions of all grids in the raster image in each pixel row are determined according to the first peak sequence of each pixel row.

[0024] Optionally, the smoothing of the row brightness curves of each pixel row and peak fitting of the smoothed row brightness curves until the number of peaks of each pixel row is equal to obtain a first peak sequence of each pixel row includes:

[0025] Set the current value of the smoothing radius;

[0026] Smoothing the row brightness curve of each pixel row based on the current value, and performing peak fitting on the smoothed row brightness curve to obtain a preliminary peak position sequence for each pixel row;

[0027] Comparing the preliminary peak position sequences of each pixel row, if the number of peaks of at least one pixel row is not equal to the number of peaks of the other pixel rows, updating the current value, and continuing to perform the smoothing process on the row brightness curve of each pixel row based on the current value and subsequent steps until the number of peaks of each pixel row is equal;

[0028] If the number of peaks in each pixel row is equal, the preliminary peak position sequence of each pixel row is used as the final first peak position sequence.

[0029] Optionally, determining the position of the grid of the region of interest in the raster image in each pixel row includes:

[0030] Determining a column mean brightness curve corresponding to the brightness value of each pixel row in each column of the grating image, and performing peak fitting on the column mean brightness curve of each column to obtain a second peak sequence;

[0031] The region of interest of the grating image is cropped according to the second peak sequence, and the peak position of each pixel row is extracted from the cropped grating image as the position of the grid of the region of interest in the grating image in each pixel row.

[0032] Optionally, the clipping of the region of interest of the grating image according to the second peak sequence includes:

[0033] determining the number of peaks in the second peak sequence;

[0034] determining a starting position and an ending position when clipping the grating image according to the number of peaks in the second peak sequence;

[0035] The region of interest of the raster image is cropped according to the starting position and the ending position.

[0036] Optionally, performing row-by-row correction on the object plane image according to the position of each pixel row of the at least one grid to obtain a target image includes:

[0037] Acquire a first blank image to be filled, wherein the image height of the first blank image is the same as the image height of the object plane image;

[0038] Based on the positions of all grids in the grating image in each pixel row, the brightness values ​​of the corresponding pixel rows in the object surface image are segmented and sampled, and the sampled brightness values ​​are sequentially filled into the first blank image to obtain a target image.

[0039] Optionally, performing row-by-row correction on the object plane image according to the position of each pixel row of the at least one grid to obtain a target image includes:

[0040] Aligning the grating image row by row according to the positions of the grids of the region of interest in each pixel row in the grating image to obtain an aligned grating image;

[0041] According to the position of the grid of the region of interest in the grating image in each pixel row, the object plane image is aligned row by row to obtain an aligned object plane image;

[0042] The aligned object plane image is corrected line by line according to the aligned grating image to obtain a target image.

[0043] Optionally, aligning the grating image row by row according to the positions of the grids of the region of interest in each pixel row in the grating image to obtain the aligned grating image includes:

[0044] Determining the maximum and minimum values ​​of positions in all pixel rows according to the positions of the grids of the region of interest in the raster image in each pixel row, and determining the original image width of the raster image;

[0045] determining a sampling interval for resampling the raster image according to the maximum value, the minimum value, and the original image width;

[0046] Acquire a second blank image to be filled, wherein the image height of the second blank image is the same as the image height of the grating image;

[0047] The grating image is resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the second blank image to obtain an aligned grating image.

[0048] Optionally, aligning the object plane image row by row according to the position of the grid of the region of interest in each pixel row in the grating image to obtain the aligned object plane image includes:

[0049] Acquire a third blank image to be filled, wherein the image height of the third blank image is the same as the image height of the object plane image;

[0050] The object plane image is resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the third blank image to obtain an aligned object plane image.

[0051] Optionally, performing line-by-line correction on the aligned object plane image according to the aligned grating image to obtain a target image includes:

[0052] Determining a column mean curve corresponding to the brightness value of each pixel row in each column of the aligned grating image, and performing peak fitting on the column mean curve of each column to obtain a third peak sequence;

[0053] Acquire a fourth blank image to be filled, wherein the image height of the fourth blank image is the same as the image height of the aligned object plane image;

[0054] Based on the position of the third peak sequence, the brightness values ​​of the corresponding pixel rows in the aligned object plane image are segmented and sampled, and the sampled brightness values ​​are sequentially filled into the fourth blank image to obtain a target image.

[0055] The present application also provides an acquisition card, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the distortion correction method as described in any one of the above embodiments.

[0056] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0057] The present application provides a laser confocal scanning microscope, a distortion correction method and an acquisition card, which includes not only a laser light source, a resonant galvanometer, a galvanometer, a spectroscope, an excitation light path, a confocal light path and a first detector, but also a transflective mirror fixedly installed between the resonant galvanometer and the galvanometer, a monitoring light path and a second detector arranged in the transmission direction of the transflective mirror, and an acquisition card connected to the first detector and the second detector respectively; wherein, when the microscope of the present application scans a sample, the light beam emitted by the laser light source is reflected to the resonant galvanometer through the spectroscope, the resonant galvanometer deflects the incident light beam for the first time and then emits it to the transflective mirror, the transflective mirror reflects most of the light beam to the galvanometer After the galvanometer galvanometer deflects the incident light beam twice, it focuses the deflected light beam onto the sample surface through the excitation light path. The reflected light excited on the sample surface returns along the original path, is transmitted to the confocal light path through the spectroscope, and reaches the first detector from the confocal light path. The reflector transmits a small part of the light beam to the monitoring light path, and the monitoring light path converts the real-time deflection angle of the resonant galvanometer into a grating signal, which is then sent to the second detector; the acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, and generates an object surface image corresponding to the confocal signal and a grating image corresponding to the grating signal. The grating image is used to perform distortion correction on the object surface image to obtain the target image. In this process, the confocal signal and grating signal synchronously and isochronously collected by the acquisition card are twin images in time and space, wherein the grating image formed by the grating signal depicts the movement of the resonant galvanometer during the scanning process. Therefore, the present application can perform distortion correction on the twin object surface image through distortion analysis of the grating image, thereby effectively improving the correction accuracy, and is not easily affected by factors such as environment, life span, and optical path deviation. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0059] Figure 1 A schematic structural diagram of a laser confocal scanning microscope provided in an embodiment of the present application;

[0060] Figure 2 A diagram showing the optical path for light splitting using a reflective mirror provided in an embodiment of the present application;

[0061] Figure 3 A diagram showing a grating image provided in an embodiment of the present application;

[0062] Figure 4 A display diagram of an object plane image provided in an embodiment of the present application;

[0063] Figure 5 A flowchart of a distortion correction method provided in an embodiment of the present application;

[0064] Figure 6 This is a diagram showing a deviation in the brightest position between two adjacent rows of the same grating provided in an embodiment of the present application;

[0065] Figure 7 A schematic diagram of a row brightness curve corresponding to a pixel row in a grating image provided in an embodiment of the present application;

[0066] Figure 8 A schematic diagram of the peak fitting results of one pixel row provided in an embodiment of the present application;

[0067] Figure 9 A diagram showing the region of interest at the center of a grating image provided in an embodiment of the present application;

[0068] Figure 10 A target image display diagram after distortion correction of an object plane image using a full grating alignment distortion correction algorithm provided in an embodiment of the present application;

[0069] Figure 11 A partial enlarged comparison diagram of the grating image before and after alignment provided in an embodiment of the present application;

[0070] Figure 12 A partial magnified comparison diagram of the object plane image before and after alignment provided in the embodiment of the present application;

[0071] Figure 13This is a target image display diagram after the distortion of the object surface image is corrected using the local grating alignment correction algorithm provided in the embodiment of the present application. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0073] In one embodiment, Figure 1 As shown, Figure 1 A structural schematic diagram of a laser confocal scanning microscope provided in an embodiment of the present application; the present application provides a laser confocal scanning microscope, including a laser light source, a resonant galvanometer, a galvanometer, a spectrometer, an excitation light path, a confocal light path and a first detector, the microscope also including: a transflective mirror fixedly mounted between the resonant galvanometer and the galvanometer, a monitoring light path and a second detector arranged in the transmission direction of the transflective mirror, and an acquisition card respectively connected to the first detector and the second detector.

[0074] In which, when the microscope scans the sample, the light beam emitted by the laser light source is reflected to the resonant galvanometer mirror through the beam splitter, the resonant galvanometer mirror deflects the incident light beam for the first time and then emits it to the transflective mirror, the transflective mirror reflects most of the light beam to the galvanometer galvanometer mirror, the galvanometer galvanometer mirror deflects the incident light beam for the second time, and then focuses the deflected light beam to the sample surface through the excitation light path, the reflected light excited on the sample surface returns along the original path, is transmitted to the confocal light path through the beam splitter, and reaches the first detector from the confocal light path, the transflective mirror transmits a small part of the light beam to the monitoring light path, and the monitoring light path converts the real-time deflection angle of the resonant galvanometer mirror into a grating signal and sends it to the second detector.

[0075] The acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, and generates an object plane image corresponding to the confocal signal and a grating image corresponding to the grating signal, and then uses the grating image to perform distortion correction on the object plane image to obtain a target image.

[0076] In this embodiment, Figure 1As shown, the present application adds a motion process collection optical path of a resonant galvanometer to the original laser confocal scanning microscope equipped with a two-dimensional galvanometer. The collection optical path is composed of a reflective mirror arranged between the resonant galvanometer and the galvanometer, a monitoring optical path and a second detector arranged in the transmission direction of the reflective mirror, and an acquisition card connected to the first detector and the second detector respectively. Since the reflective mirror can selectively transmit or reflect light according to the wavelength of light, the present application can reflect most of the excitation light to the sample through the reflective mirror, while transmitting a small part of the excitation light for monitoring the motion of the resonant galvanometer. In this way, the nonlinear motion (such as a sinusoidal waveform) of the resonant galvanometer can be recorded in real time through the monitoring optical path and the second detector. The acquisition card of the present application can record the two signals with the same timestamp, which can ensure pixel-level alignment and use the galvanometer motion data collected by the second detector to remap the pixel position of the object surface image collected by the first detector, thereby eliminating sinusoidal scanning distortion.

[0077] The transflective mirror of the present application is fixedly mounted in the optical path between the resonant galvanometer and the galvanometer. This ensures a constant splitting ratio and direction while simplifying system calibration. Furthermore, the present application can set the splitting ratio of the transflective mirror. For example, the present application can use 95% of the energy in the light beam received by the transflective mirror for sample scanning and 5% for feedback on the galvanometer status. The specific ratio can be set based on actual conditions and is not limited here. This ensures that a high-resolution object surface image is obtained while eliminating scanning distortion of the object surface image through the grating image collected by the second detector.

[0078] Furthermore, the laser light source of the present application can provide monochromatic, high-brightness excitation light (such as Argon ion laser, semiconductor laser, etc.); the resonant galvanometer of the present application can adopt a high-frequency resonant galvanometer (such as 4-12 kHz), and the galvanometer can adopt a low-speed galvanometer (such as 1-30 Hz); the excitation light path of the present application may include a scanning lens and an objective lens, which can focus the light beam deflected by the galvanometer onto the sample surface and collect the reflection / fluorescence signal; the confocal light path of the present application is located in front of the first detector to block the out-of-focus light signal and improve the axial resolution; the first detector and the second detector of the present application can use a photomultiplier tube (PMT) or an avalanche photodiode (APD) to convert the light signal into an electrical signal.

[0079] For example, when the present application is scanning a sample, the resonant galvanometer oscillates at high frequency in the form of a sine wave, driving the light beam to scan back and forth rapidly on the sample surface. This process only uses the linear interval of the sine wave (such as near the peak or trough) to collect signals, and the laser is usually turned off or the data is discarded during the retracement phase (non-linear interval). During this process, the galvanometer galvanometer slowly steps in the form of a sawtooth wave, moving one step (corresponding to one row of pixels in the image) after completing each row of X-axis scanning, and quickly resets to the starting position at the end of the frame. After being deflected by the two-dimensional galvanometer mirror and reflected by the transflective mirror, the laser beam is focused onto the sample surface through the scanning lens and objective lens. The excited fluorescence or reflected light of the sample returns along the original path, is separated by the spectroscope, and reaches the first detector through the confocal optical path. The first detector converts the optical signal into an electrical signal and synchronizes it with the position of the galvanometer mirror to construct a two-dimensional image pixel by pixel, which is the object surface image in the present application. In addition, the transflective mirror of the present application can also transmit a small portion of the laser beam, which is collected by the monitoring optical path to convert the real-time deflection angle of the resonant galvanometer into a grating signal. The monitoring optical path is connected to the second detector, so that the grating signal can be sent to the acquisition card through the second detector. After the acquisition card performs image processing on the grating signal, a grating image can be obtained. Then, the present application can use the grating image to perform distortion correction on the object surface image, so that the target image can be obtained. For example, when the present application determines the location of each grid in the grating image (the location of each local brightness maximum), it can determine the distortion level between two adjacent locations, and then complete the distortion correction based on the distortion level.

[0080] In the above embodiment, the laser confocal scanning microscope includes not only a laser light source, a resonant galvanometer, a galvanometer, a spectroscope, an excitation light path, a confocal light path and a first detector, but also includes a reflective mirror fixedly installed between the resonant galvanometer and the galvanometer, a monitoring light path and a second detector arranged in the transmission direction of the reflective mirror, and an acquisition card connected to the first detector and the second detector respectively; wherein, when the microscope of the present application scans the sample, the light beam emitted by the laser light source is reflected to the resonant galvanometer through the spectroscope, the resonant galvanometer deflects the incident light beam for the first time and then emits it to the reflective mirror, the reflective mirror reflects most of the light beam to the galvanometer, and the galvanometer After the light beam is deflected twice, the deflected light beam is focused onto the sample surface through the excitation light path. The reflected light excited on the sample surface returns along the original path, is transmitted to the confocal light path through the spectroscope, and reaches the first detector through the confocal light path. The reflector transmits a small part of the light beam to the monitoring light path, and the monitoring light path converts the real-time deflection angle of the resonant galvanometer into a grating signal, which is then sent to the second detector. The acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, and generates an object surface image corresponding to the confocal signal and a grating image corresponding to the grating signal. The grating image is used to perform distortion correction on the object surface image to obtain the target image. In this process, the confocal signal and grating signal synchronously and isochronously collected by the acquisition card are twin images in time and space, wherein the grating image formed by the grating signal depicts the movement of the resonant galvanometer during the scanning process. Therefore, the present application can perform distortion correction on the twin object surface image through distortion analysis of the grating image, thereby effectively improving the correction accuracy, and is not easily affected by factors such as environment, life span, and optical path deviation.

[0081] In one embodiment, Figure 2 As shown, Figure 2 A diagram illustrating an optical path for light splitting using a reflective mirror provided in an embodiment of the present application; the monitoring optical path may include an attenuation plate and a Ronchi grating.

[0082] The attenuation plate attenuates the transmitted light beam and then transmits it to the Ronchi grating. The Ronchi grating converts the real-time deflection angle of the resonant galvanometer into a grating signal according to the attenuated light beam.

[0083] In this embodiment, the monitoring optical path provided in the transmission direction of the transflective mirror may include an attenuation plate and a Ronchi grating. The attenuation plate can attenuate the transmitted light beam before transmitting it to the Ronchi grating, thereby preventing the high-power laser (transmitted light beam) from damaging the second detector. The Ronchi grating can convert the real-time deflection angle of the resonant galvanometer into a grating signal based on the attenuated light beam. For example, the Ronchi grating of the present application can utilize the deflection of the resonant galvanometer to move the light beam on the grating, thereby generating a periodic light intensity signal. This periodic light intensity signal is the grating signal of the present application. This grating signal directly maps the real-time deflection angle of the resonant galvanometer. After processing by an acquisition card, the signal can be obtained as follows: Figure 3 The grating image shown has a horizontal direction corresponding to the movement direction of the resonant galvanometer, each image row corresponds to a movement cycle of the resonant galvanometer, and the distance between the grids reflects the movement of the resonant galvanometer, that is, the distortion level at each position.

[0084] At the same time, the acquisition card of the present application can also process the collected confocal signal, which reflects the fluorescence / reflected light intensity distribution of the sample and is time-space aligned with the grating signal. Therefore, the present application can generate the following image after image processing of the confocal signal: Figure 4 The object surface image shown is then used to perform distortion correction on the object surface image using the galvanometer motion data reflected by the grating image, thereby effectively improving the accuracy of distortion correction while avoiding the influence of factors such as environment, lifespan, and optical path deviation.

[0085] In one embodiment, Figure 5 As shown, Figure 5 This is a flow chart of a distortion correction method provided in an embodiment of the present application. This application also provides a distortion correction method applied to an acquisition card in a laser confocal scanning microscope according to any of the above embodiments. The method may include:

[0086] S110: synchronously and isochronously collecting the confocal signal output by the first detector and the grating signal output by the second detector, and generating an object plane image corresponding to the confocal signal and a grating image corresponding to the grating signal;

[0087] S120: Performing distortion correction on the object surface image using the grating image to obtain a target image.

[0088] In this embodiment, Figure 1As shown, the present application adds a motion process collection optical path of a resonant galvanometer to the original laser confocal scanning microscope equipped with a two-dimensional galvanometer. The collection optical path is composed of a reflective mirror arranged between the resonant galvanometer and the galvanometer, a monitoring optical path and a second detector arranged in the transmission direction of the reflective mirror, and an acquisition card connected to the first detector and the second detector respectively. Since the reflective mirror can selectively transmit or reflect light according to the wavelength of light, the present application can reflect most of the excitation light to the sample through the reflective mirror, while transmitting a small part of the excitation light for monitoring the motion of the resonant galvanometer. In this way, the nonlinear motion (such as a sinusoidal waveform) of the resonant galvanometer can be recorded in real time through the monitoring optical path and the second detector. The acquisition card of the present application can record the two signals with the same timestamp, which can ensure pixel-level alignment and use the galvanometer motion data collected by the second detector to remap the pixel position of the object surface image collected by the first detector, thereby eliminating sinusoidal scanning distortion.

[0089] In a specific implementation, when the present application is scanning a sample, the resonant galvanometer oscillates at high frequency in the form of a sine wave, driving the light beam to scan back and forth rapidly on the sample surface. This process only uses the linear interval of the sine wave (such as near the peak or trough) to collect signals, and the laser is usually turned off or data is discarded during the retracement phase (non-linear interval). During this process, the galvanometer galvanometer slowly steps in the form of a sawtooth wave, moving one step (corresponding to one row of pixels in the image) after completing each row of X-axis scanning, and quickly resets to the starting position at the end of the frame. After being deflected by the two-dimensional galvanometer mirror and reflected by the transflective mirror, the laser beam is focused onto the sample surface through the scanning lens and objective lens. The fluorescence or reflected light excited by the sample returns along the original path, is separated by the spectroscope, and reaches the first detector through the confocal optical path. The first detector converts the optical signal into an electrical signal and synchronizes it with the galvanometer position to construct a two-dimensional image pixel by pixel, that is, the object surface image in the present application. In addition, the transflective mirror of the present application can also transmit a small portion of the laser beam, which is collected by the monitoring optical path to convert the real-time deflection angle of the resonant galvanometer into a grating signal. The monitoring optical path is connected to the second detector, so that the grating signal can be sent to the acquisition card through the second detector. After the acquisition card performs image processing on the grating signal, a grating image can be obtained. Then, the present application can use the grating image to perform distortion correction on the object surface image, so that the target image can be obtained. For example, when the present application determines the location of each grid in the grating image (the location of each local brightness maximum), it can determine the distortion level between two adjacent locations, and then complete the distortion correction based on the distortion level.

[0090] In the above embodiment, the confocal signal and grating signal synchronously and isochronously collected by the acquisition card are twin images in time and space, wherein the grating image formed by the grating signal depicts the movement of the resonant galvanometer during the scanning process. Therefore, the present application can perform distortion correction on the twin object surface image through distortion analysis of the grating image, thereby effectively improving the correction accuracy and being less susceptible to factors such as environment, lifespan, and optical path deviation.

[0091] In one embodiment, performing distortion correction on the object plane image using the grating image to obtain a target image in S120 may include:

[0092] S121: Determine the position of at least one grid in the raster image in each pixel row.

[0093] S122: Correct the object plane image row by row according to the position of the at least one grid in each pixel row to obtain a target image.

[0094] In this embodiment, when using a grating image to correct the distortion of an object surface image, the position of at least one grid in the grating image in each pixel row can be determined first. In this way, the object surface image can be corrected according to the position of at least one grid in each pixel row to obtain a target image.

[0095] When determining the position of at least one grid in a grating image in each pixel row, the present application can determine the number of grids to be determined based on the current scene. For example, if the current scene requires high precision, the object surface image can be corrected by the positions of all grids in the grating image in each pixel row. If the current scene requires relatively low precision but high real-time performance, the object surface image can be corrected by screening local grids, thereby meeting the real-time performance requirement.

[0096] In one embodiment, determining the position of at least one grid in the raster image in each pixel row in S121 may include:

[0097] S1211: If the current scene is a high-precision scene, determine the positions of all grids in the raster image in each pixel row.

[0098] S1212: If the current scene is a real-time scene and the resonant galvanometer scans in one direction, the position of the grid of the region of interest in the grating image in each pixel row is determined.

[0099] In this embodiment, when determining the position of at least one grid in a grating image in each pixel row, the scene characteristics of the current scene can be first determined. If the current scene is a high-precision scene, that is, a scene that requires the accuracy of image distortion correction to be higher than a preset accuracy threshold, such as in applications such as micro-nano processing, super-resolution imaging, and precision measurement, the preset accuracy threshold can be a threshold of a single dimension, such as error or resolution, or a threshold of multiple dimensions. The specific setting can be based on actual conditions and is not limited here. When the current scene is determined to be a high-precision scene, the present application can correct the object surface image by using the positions of all grids in the grating image in each pixel row, thereby achieving accurate distortion correction of the object surface image.

[0100] Furthermore, if the current scene is a real-time scene, and the resonant galvanometer is scanning in one direction, that is, the real-time requirement for image distortion correction is higher than the preset time threshold, and the adjacent motion cycles of the resonant galvanometer are deviated only at the starting point, such as laser processing, high-speed imaging, dynamic display and other application scenarios, the preset time threshold in these scenes can be set according to the actual situation, and no restrictions are made here. When the present application determines that the current scene is a real-time scene, the grid of the area of ​​interest in the grating image can be determined by screening the local grid. The grid of the area of ​​interest can be the grid at the center position of the grating image, or the grid at a non-center position. The specific selection can be made according to the actual situation and no restrictions are made here. The present application corrects the object surface image through the grid of the area of ​​interest, which can greatly shorten the processing time of the distortion correction and thus meet the real-time requirements.

[0101] In one embodiment, determining the positions of all grids in the raster image in each pixel row in S1211 may include:

[0102] S12111: Determine a row brightness curve corresponding to the brightness values ​​of each column in each pixel row of the raster image.

[0103] S12112: Smoothing the row brightness curves of each pixel row, and performing peak fitting on the smoothed row brightness curves until the number of peaks of each pixel row is equal, thereby obtaining a first peak sequence of each pixel row.

[0104] S12113: Determine the positions of all grids in the grating image in each pixel row according to the first peak sequence of each pixel row.

[0105] In this embodiment, when determining the positions of all grids in each pixel row in the grating image, due to mechanical motion errors and electronic noise, there is a certain degree of misalignment between the brightest positions of the same grid in adjacent rows of the grating image (corresponding to two adjacent motion cycles of the resonant galvanometer). Figure 6 As shown, Figure 6 This is a diagram showing a deviation in the brightest position between two adjacent rows of the same grating provided in an embodiment of the present application; Figure 6 The position deviation can be attributed to the fact that there is a certain deviation between the starting point of adjacent motion cycles of the resonant galvanometer and the time of reaching the same grid, which means that it is impossible to simply treat one grid as a position for correction.

[0106] Based on this, when determining the position of each grid in the grating image, the present application can first determine the brightness value of each column in each pixel row of the grating image, and form a row brightness curve corresponding to each pixel row. = , c=1, ..., C (C is the image width of the raster image, that is, the number of pixel columns, is the brightness value of the pixel at row r and column c in the raster image. The brightness value represents the brightness or color intensity of the position and can be directly determined based on the raster image. Schematically, Figure 7 As shown, Figure 7 This is a schematic diagram of a row brightness curve corresponding to a pixel row in a grating image provided in an embodiment of the present application. Figure 7 In , each peak corresponds to a grid.

[0107] Next, the present application can smooth the row brightness curve of each pixel row to highlight the main trend or structure of the row brightness curve by suppressing noise or irrelevant details, thereby improving the readability, analyzability or visual effect of the row brightness curve and providing a more accurate data basis for subsequent peak fitting.

[0108] After the present application smoothes the row brightness curve of each pixel row, the peak value fitting can be performed on the smoothed row brightness curve, so that the number of peak values ​​of each pixel row can be determined. When the number of peak values ​​of each pixel row is equal, the first peak value sequence of each pixel row can be obtained. , i=1,……, ,in is the number of peaks obtained by fitting the row brightness curve of row r under the current smoothing radius. Schematically, Figure 8 As shown, Figure 8 A schematic diagram of the peak fitting results of one pixel row provided in an embodiment of the present application; Figure 8 The red line in the figure represents the pixel position with the maximum brightness in the grid area, and the green line represents the fitting position. Obviously, the fitting position is more reasonable.

[0109] Based on the above processing, the present application can obtain the first peak sequence corresponding to each pixel row. Since the first peak sequence indicates the peak position of multiple peaks obtained by fitting each pixel row under the current smoothing radius, the present application can determine the position of all grids in the raster image in each pixel row based on the first peak sequence of each pixel row and form the corresponding grid position matrix , where r=1, ..., R, i=1, ..., N, R is the image height of the raster image, that is, the number of pixel rows, and N is the number of grids in each row. is the position of the i-th grid in the r-th row in the raster image (i.e., the horizontal coordinate).

[0110] In one embodiment, in S12112, the row brightness curves of each pixel row are smoothed, and peak fitting is performed on the smoothed row brightness curves until the number of peaks of each pixel row is equal, thereby obtaining a first peak sequence of each pixel row, which may include:

[0111] S121121: Set the current value of the smoothing radius.

[0112] S121122: Smoothing the row brightness curve of each pixel row based on the current value, and performing peak fitting on the smoothed row brightness curve to obtain a preliminary peak position sequence of each pixel row.

[0113] S121123: Compare the preliminary peak position sequences of each pixel row. If the number of peaks of at least one pixel row is not equal to the number of peaks of other pixel rows, update the current value and continue to perform the smoothing of the row brightness curve of each pixel row based on the current value and subsequent steps until the number of peaks of each pixel row is equal.

[0114] S121124: If the number of peaks in each pixel row is equal, the preliminary peak position sequence of each pixel row is used as the final first peak sequence.

[0115] In this embodiment, when smoothing the row brightness curve of each pixel row, the selection of the smoothing radius (or the smoothing window size) will directly affect the quality of the processing effect and the degree of feature retention. Therefore, the present application can first set the current value of the smoothing radius, which can be an initial value, and then smooth the row brightness curve of each pixel row based on the current value, and then perform peak fitting. After obtaining the preliminary peak position sequence of each pixel row, the number of peaks in the preliminary peak position sequence of each pixel row can be compared. If the number of peaks of at least one pixel row is not equal to the number of peaks of other pixel rows, the current value of the smoothing radius is updated. For example, when the initial value s=1, the updated current value can be s=s+1. Then, the present application can continue to smooth the row brightness curve of each pixel row based on the updated current value, and perform peak fitting on the smoothed row brightness curve to obtain the preliminary peak position sequence of each pixel row, and compare the number of peaks in the preliminary peak position sequence of each pixel row until the number of peaks of each pixel row is equal. At this time, the preliminary peak position sequence of each pixel row can be used as the final first peak sequence. The present application can determine the positions of all grids in the grating image in each pixel row according to the first peak sequence of each pixel row, and perform distortion correction on the object surface image according to the positions of each grid in each pixel row, so as to obtain accurate correction results.

[0116] In one embodiment, determining the position of the grid of the region of interest in the raster image in each pixel row in S1212 may include:

[0117] S12121: Determine a column mean brightness curve corresponding to the brightness value of each pixel row in each column of the grating image, and perform peak fitting on the column mean brightness curve of each column to obtain a second peak sequence.

[0118] S12122: Crop the region of interest of the grating image according to the second peak sequence, and extract the peak position of each pixel row from the cropped grating image as the position of the grid of the region of interest in the grating image in each pixel row.

[0119] In this embodiment, if the current scene is a real-time scene, and the resonant galvanometer is scanning in one direction, that is, the real-time requirement for image distortion correction is higher than the preset time threshold, and the adjacent motion cycles of the resonant galvanometer are deviated only at the starting point, such as in application scenarios such as laser processing, high-speed imaging, and dynamic display, the grid of the area of ​​interest in the grating image can be determined by screening the local grid. The grid of the area of ​​interest can be the grid at the center position of the grating image or the grid at a non-center position. The specific selection can be made according to the actual situation and is not limited here. The present application corrects the object surface image through the grid of the area of ​​interest, which can greatly shorten the processing time of the distortion correction and thus meet the real-time requirements.

[0120] Specifically, when determining the position of the grid of the area of ​​interest in the grating image in each pixel row, the present application can first determine the column mean brightness curve corresponding to the brightness value of each pixel row in each column of the grating image, and perform peak fitting on the column mean brightness curve of each column to obtain a second peak sequence. Then, the present application can crop the area of ​​interest of the grating image according to the second peak sequence, and extract the peak position of each pixel row from the cropped grating image. The peak position is the position of the grid of the area of ​​interest in the grating image in each pixel row.

[0121] For example, when the present application calculates the column mean brightness curve corresponding to the brightness values ​​of each pixel row in each column of the raster image, After that, you can Perform peak fitting to obtain the second peak sequence , k=1, ..., K, where K is the number of peaks; since the second peak sequence indicates the number of grids in the grating image, and the resonant galvanometer of the present application scans unidirectionally, the present application can select a region of the grating image with a low or lowest degree of distortion as a region of interest based on the scanning characteristics of the resonant galvanometer and the number of grids in the grating image. After cropping the region of interest, the peak position of each pixel row is extracted from the cropped grating image. The peak position is the position of the grid in the region of interest in each pixel row of the grating image. In this way, during distortion correction, only one region of interest in the grating image needs to be peak detected row by row, thereby greatly shortening the correction time and meeting the real-time requirements.

[0122] In one embodiment, the step of clipping the region of interest of the grating image according to the second peak sequence in S12122 may include:

[0123] S121221: Determine the number of peaks in the second peak sequence.

[0124] S121222: Determine the starting position and the ending position when cropping the grating image according to the number of peaks in the second peak sequence.

[0125] S121223: Crop the region of interest of the grating image according to the starting position and the end position.

[0126] In this embodiment, when the region of interest of the grating image is cropped according to the second peak sequence, the region of interest can be the central region of the grating image or a non-central region. The specific selection can be made according to actual conditions and is not limited here.

[0127] In a specific implementation, when the present application selects the central area of ​​a grating image as the area of ​​interest, the number of peaks in the second peak sequence can be determined first, and then the starting position and the end position when cropping the grating image can be determined based on the number of peaks in the second peak sequence. In this way, the area of ​​interest of the grating image can be cropped based on the starting position and the end position.

[0128] For example, when the second peak sequence of the present application is , k=1, ..., K, where K is the number of peaks, the application can set the starting position and the end position as follows:

[0129]

[0130]

[0131] in, , which is the midpoint of the second peak sequence, is the starting position, is the end position, is the midpoint peak of the second peak sequence, is the peak before the midpoint of the second peak sequence, is a peak after the midpoint of the second peak sequence. After clipping the grating image using the above starting position and end position, we can get the following: Figure 9 The region of interest at the center of the grating image shown contains only the brightness data of one grid. By using the region of interest to correct the object surface image line by line, the final target image can be obtained, thereby meeting the real-time requirements.

[0132] In one embodiment, performing row-by-row correction on the object plane image according to the position of the at least one grid at each pixel row to obtain a target image in S122 may include:

[0133] S1221: Acquire a first blank image to be filled, where the image height of the first blank image is the same as the image height of the object plane image.

[0134] S1222: Based on the positions of all grids in the grating image in each pixel row, segmentally sample the brightness values ​​of the corresponding pixel rows in the object surface image, and fill the sampled brightness values ​​into the first blank image in sequence to obtain a target image.

[0135] In this embodiment, if the current scene is a high-precision scene, that is, a scene where the accuracy requirement for image distortion correction is higher than a preset accuracy threshold, such as application scenarios such as micro-nano processing, super-resolution imaging, and precision measurement, the present application can correct the object surface image through the positions of all grids in the grating image in each pixel row, thereby achieving accurate distortion correction of the object surface image.

[0136] Specifically, the present application can obtain the first blank image to be filled , the image height of the first blank image It can be consistent with the image height of the object surface image, and the image width W can be a pre-specified value, such as W=1024. Then, the present application can sequentially extract the rth row image of the object surface image. = , r=1, ..., R, c=1, ..., C, R is the maximum number of rows of the object surface image, C is the maximum number of columns of the object surface image, is the brightness value of the object surface image in row r and column c, and then the application can perform the following mapping:

[0137]

[0138] Where T can be a linear mapping or a nonlinear mapping, and J = W / (N-1). This mapping is equivalent to lie in The brightness curve segment in is sampled linearly or nonlinearly, the number of sampling points is J, and the number of sampling points can be set according to the actual situation. Then the sampling data is filled into the corresponding position of the first blank image to obtain the final target image. The target image is as follows: Figure 10 As shown by Figure 10 It can be seen that after the distortion correction of the object plane image is performed by the above-mentioned distortion correction method, the present application can achieve accurate distortion correction of the object plane image.

[0139] In one embodiment, performing row-by-row correction on the object plane image according to the position of the at least one grid at each pixel row to obtain a target image in S122 may include:

[0140] S221: aligning the grating image row by row according to the positions of the grids of the region of interest in the grating image in each pixel row to obtain an aligned grating image.

[0141] S222: aligning the object plane image row by row according to the position of the grid of the region of interest in each pixel row in the grating image to obtain an aligned object plane image.

[0142] S223: Correcting the aligned object plane image line by line according to the aligned grating image to obtain a target image.

[0143] In this embodiment, if the current scene is a real-time scene and the resonant galvanometer is scanning in one direction, that is, the real-time requirement for image distortion correction is higher than the preset time threshold, and the adjacent motion cycles of the resonant galvanometer are deviated only at the starting point, such as in application scenarios such as laser processing, high-speed imaging, and dynamic display, the grid of the area of ​​interest in the grating image can be determined by screening the local grid, and the object surface image can be corrected by the grid of the area of ​​interest. This can greatly shorten the processing time of the distortion correction and thus meet the real-time requirements.

[0144] Specifically, when the present application performs row-by-row correction on the object surface image according to the positions of the grids in the region of interest in the grating image at each pixel row, the grating image can be first aligned row-by-row according to the positions of the grids in the region of interest at each pixel row to obtain an aligned grating image, and then the object surface image is aligned row-by-row according to the positions of the grids in the region of interest at each pixel row to obtain an aligned object surface image, and then the aligned grating image is used to perform row-by-row correction on the aligned object surface image, so that the target image can be obtained. The time for the present application to obtain the target image in this way is significantly shorter than the time for obtaining the target image after performing row-by-row correction on the object surface image by all the grids in the grating image at each pixel row, and therefore can meet the real-time requirements to a great extent.

[0145] In one embodiment, the step S221 of aligning the grating image row by row based on the positions of the grids of the region of interest in each pixel row to obtain the aligned grating image may include:

[0146] S2211: Determine the maximum and minimum values ​​of positions in all pixel rows according to the positions of the grids of the region of interest in the raster image in each pixel row, and determine the original image width of the raster image.

[0147] S2212: Determine a sampling interval for resampling the raster image according to the maximum value, the minimum value, and the original image width.

[0148] S2213: Acquire a second blank image to be filled, where the image height of the second blank image is the same as the image height of the grating image.

[0149] S2214: resampling the grating image within the sampling interval, and sequentially filling the resampled brightness values ​​into the second blank image to obtain an aligned grating image.

[0150] In this embodiment, when aligning the grating image row by row according to the position of the grid of the region of interest in the grating image in each pixel row, the present application can first determine the maximum and minimum values ​​of the positions in all pixel rows according to the position of the grid of the region of interest in the grating image in each pixel row, and determine the original image width of the grating image. In this way, the sampling interval for resampling the grating image can be determined according to the maximum and minimum values ​​of the positions in all pixel rows and the original image width. Then, the present application can obtain a second blank image to be filled, and the image height of the second blank image is the same as the image height of the grating image. In this way, the grating image can be resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the second blank image to obtain an aligned grating image. The image height of the aligned grating image is consistent with the image height before alignment, and the image width is changed.

[0151] In a specific implementation, the present application can extract the peak position of each pixel row in the region of interest of the grating image. , r=1,…,R, R is the image height, that is, the number of pixel rows. Then the raster image is aligned row by row through the following mapping to obtain the aligned raster image, which is recorded as A:

[0152]

[0153] in, and are the r-th row images of the grating image I before alignment and the grating image A after alignment, It can be selected as linear mapping or nonlinear mapping, which is equivalent to In the interval Resampling is performed in is the original image width of the raster image, L is the pixel width of A, is the maximum value of the peak positions of all rows, is the minimum value of the peak position of all rows, and:

[0154] +

[0155]

[0156]

[0157] After the above mapping, we can get Figure 11 The enlarged comparison of the grating images before and after alignment is shown. Figure 11 The left side of is the local grating image before alignment, and the right side is the local grating image after alignment. Figure 11 It can be seen that the misalignment between the brightest positions of the same grid in adjacent rows of the aligned grating image is small. By using the aligned grating image to correct the object surface image row by row, a relatively accurate correction result can be obtained.

[0158] In one embodiment, in S222, aligning the object plane image row by row according to the position of the grid of the region of interest in each pixel row in the grating image to obtain the aligned object plane image may include:

[0159] S2221: Acquire a third blank image to be filled, where the image height of the third blank image is the same as the image height of the object plane image.

[0160] S2222: resampling the object plane image within the sampling interval, and sequentially filling the resampled brightness values ​​into the third blank image to obtain an aligned object plane image.

[0161] In this embodiment, when aligning the object surface image row by row according to the position of the grid of the region of interest in the grating image in each pixel row, the present application can first obtain a third blank image to be filled, and the image height of the third blank image is the same as the image height of the object surface image. Then, the present application can align the object surface image row by row by using the above-mentioned method of aligning the grating image row by row to obtain the aligned object surface image, which is recorded as B:

[0162]

[0163] in, and are the r-th row images of the object surface image G before alignment and the object surface image B after alignment, It can be selected as linear mapping or nonlinear mapping, which is equivalent to In the interval Resampling is performed in , and the local magnification comparison of the object surface image before and after alignment is shown in the figure below. Figure 12 As shown, Figure 12 The left side is the partial enlarged image before alignment, and the right side is the partial enlarged image after alignment. Figure 12 It can be seen that the object plane image after alignment in this application is clearer and some distortion has been eliminated.

[0164] In one embodiment, performing line-by-line correction on the aligned object plane image according to the aligned grating image to obtain a target image in S223 may include:

[0165] S2231: Determine a column mean curve corresponding to the brightness value of each pixel row in each column of the aligned grating image, and perform peak fitting on the column mean curve of each column to obtain a third peak sequence.

[0166] S2232: Acquire a fourth blank image to be filled, where the image height of the fourth blank image is the same as the image height of the aligned object plane image.

[0167] S2233: Based on the position of the third peak sequence, segmentally sample the brightness values ​​of the corresponding pixel rows in the aligned object plane image, and fill the sampled brightness values ​​into the fourth blank image in sequence to obtain a target image.

[0168] In this embodiment, after obtaining the aligned grating image and the aligned object plane image, the present application may perform line-by-line correction on the aligned object plane image according to the aligned grating image to obtain a target image.

[0169] Specifically, the present application can first determine the column mean curve corresponding to the brightness value of each pixel row in each column of the aligned grating image, and perform peak fitting on the column mean curve of each column to obtain a third peak sequence, and then obtain a fourth blank image to be filled. The image height of the fourth blank image is the same as the image height of the aligned object surface image. In this way, based on the position of the third peak sequence, the brightness values ​​of the corresponding pixel rows in the aligned object surface image can be segmented and sampled, and the sampled brightness values ​​can be sequentially filled into the fourth blank image to obtain the target image.

[0170] For example, the present application may first calculate the column mean curve corresponding to the brightness value of each pixel row in each column of the aligned grating image A. Then, Perform peak fitting to obtain the third peak sequence , k=1, ..., K, K is the number of peaks, then, the present application can perform row-by-row correction on the aligned object surface image B to obtain the final target image D. During the correction process, a fourth blank image D to be filled can be prepared for filling, whose pixel height is the same as the object surface image, and the width W is a pre-specified value, such as W=1024; then the r-th row image of B is extracted in sequence = , r=1,…,R,c=1,…,L, is the brightness value of the rth row and cth column of image B. Finally, this application can perform the following mapping to After segment alignment and sampling, fill to the fourth blank image:

[0171]

[0172] in, It can be selected as linear mapping or nonlinear mapping. = W / (K-1). This mapping is equivalent to lie in The brightness curve segment in is sampled linearly or nonlinearly, and the number of sampling points is , and then fill the sampling data into the corresponding position of the fourth blank image to obtain the final target image; the target image is as follows Figure 13 As shown, compare it with Figure 10 By comparing the target image with the full grating alignment correction algorithm, it can be seen that the effect is not much different from the target image obtained after correction using the full grating alignment correction algorithm. However, since this algorithm only needs to perform row-by-row peak detection on a region of interest in the grating image, this algorithm can greatly shorten the correction time, thereby meeting the real-time requirements.

[0173] In one embodiment, the present application further provides an acquisition card, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the distortion correction method as described in any one of the above embodiments.

[0174] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0175] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0176] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A laser confocal scanning microscope, comprising a laser light source, a resonant galvanometer, a galvanometer, a spectroscope, an excitation optical path, a confocal optical path, and a first detector, characterized in that: The microscope further comprises: a transflective mirror fixedly mounted between the resonant galvanometer mirror and the galvanometer galvanometer mirror, a monitoring optical path and a second detector arranged in the transmission direction of the transflective mirror, and an acquisition card connected to the first detector and the second detector respectively; When the microscope scans the sample, the light beam emitted by the laser light source is reflected by the beam splitter to the resonant galvanometer mirror. The resonant galvanometer mirror deflects the incident light beam for the first time and then emits it to the transflective mirror. The transflective mirror reflects most of the light beam to the galvanometer galvanometer mirror. The galvanometer galvanometer mirror deflects the incident light beam for the second time and then focuses the deflected light beam onto the sample surface through the excitation optical path. The reflected light excited on the sample surface returns along the original path, is transmitted through the beam splitter to the confocal optical path, and reaches the first detector through the confocal optical path. The transflective mirror transmits a small part of the light beam to the monitoring optical path, and the monitoring optical path converts the real-time deflection angle of the resonant galvanometer mirror into a grating signal and sends it to the second detector. The acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, and generates an object plane image corresponding to the confocal signal and a grating image corresponding to the grating signal, and then uses the grating image to perform distortion correction on the object plane image to obtain a target image.

2. The laser confocal scanning microscope according to claim 1, characterized in that: The monitoring optical path includes an attenuation plate and a Ronchi grating; The attenuation plate attenuates the transmitted light beam and then transmits it to the Ronchi grating. The Ronchi grating converts the real-time deflection angle of the resonant galvanometer into a grating signal according to the attenuated light beam.

3. A distortion correction method, applied to the acquisition card in the laser confocal scanning microscope according to claim 1 or 2, characterized in that: The method comprises: synchronously and isochronously collecting the confocal signal output by the first detector and the grating signal output by the second detector, and generating an object plane image corresponding to the confocal signal and a grating image corresponding to the grating signal; The object plane image is subjected to distortion correction using the grating image to obtain a target image.

4. The distortion correction method according to claim 3, wherein: The method of performing distortion correction on the object plane image by using the grating image to obtain a target image includes: determining a position of at least one grid in the raster image at each pixel row; The object plane image is corrected row by row according to the position of the at least one grid in each pixel row to obtain a target image.

5. The distortion correction method according to claim 4, wherein: Determining the position of at least one grid in the raster image at each pixel row includes: If the current scene is a high-precision scene, determining the positions of all grids in the raster image in each pixel row; If the current scene is a real-time scene and the resonant galvanometer scans in one direction, the position of the grid of the region of interest in the grating image in each pixel row is determined.

6. The distortion correction method according to claim 5, wherein: Determining the positions of all grids in the raster image at respective pixel rows comprises: Determining a row brightness curve corresponding to brightness values ​​of each column in each pixel row of the raster image; Smoothing the row brightness curve of each pixel row, and performing peak fitting on the smoothed row brightness curve until the number of peaks in each pixel row is equal, thereby obtaining a first peak sequence for each pixel row; The positions of all grids in the raster image in each pixel row are determined according to the first peak sequence of each pixel row.

7. The distortion correction method according to claim 6, wherein: The smoothing of the row brightness curves of each pixel row and peak fitting of the smoothed row brightness curves until the number of peaks of each pixel row is equal to obtain a first peak sequence of each pixel row includes: Set the current value of the smoothing radius; Smoothing the row brightness curve of each pixel row based on the current value, and performing peak fitting on the smoothed row brightness curve to obtain a preliminary peak position sequence for each pixel row; Comparing the preliminary peak position sequences of each pixel row, if the number of peaks of at least one pixel row is not equal to the number of peaks of the other pixel rows, updating the current value, and continuing to perform the smoothing process on the row brightness curve of each pixel row based on the current value and subsequent steps until the number of peaks of each pixel row is equal; If the number of peaks in each pixel row is equal, the preliminary peak position sequence of each pixel row is used as the final first peak sequence.

8. The distortion correction method according to claim 5, wherein: Determining the position of the grid of the region of interest in the raster image in each pixel row includes: Determining a column mean brightness curve corresponding to the brightness value of each pixel row in each column of the grating image, and performing peak fitting on the column mean brightness curve of each column to obtain a second peak sequence; The region of interest of the grating image is cropped according to the second peak sequence, and the peak position of each pixel row is extracted from the cropped grating image as the position of the grid of the region of interest in the grating image in each pixel row.

9. The distortion correction method according to claim 8, wherein: The step of clipping the region of interest of the grating image according to the second peak sequence includes: determining the number of peaks in the second peak sequence; determining a starting position and an ending position when clipping the grating image according to the number of peaks in the second peak sequence; The region of interest of the raster image is cropped according to the starting position and the ending position.

10. The distortion correction method according to claim 5, wherein: The step of performing row-by-row correction on the object plane image according to the position of each pixel row of the at least one grid to obtain a target image includes: Acquire a first blank image to be filled, wherein the image height of the first blank image is the same as the image height of the object plane image; Based on the positions of all grids in the grating image in each pixel row, the brightness values ​​of the corresponding pixel rows in the object surface image are segmented and sampled, and the sampled brightness values ​​are sequentially filled into the first blank image to obtain a target image.

11. The distortion correction method according to claim 5, wherein: The step of performing row-by-row correction on the object plane image according to the position of each pixel row of the at least one grid to obtain a target image includes: Aligning the grating image row by row according to the positions of the grids of the region of interest in each pixel row in the grating image to obtain an aligned grating image; According to the position of the grid of the region of interest in the grating image in each pixel row, the object plane image is aligned row by row to obtain an aligned object plane image; The aligned object plane image is corrected line by line according to the aligned grating image to obtain a target image.

12. The distortion correction method according to claim 11, wherein: The step of aligning the grating image row by row according to the positions of the grids of the region of interest in each pixel row to obtain an aligned grating image comprises: Determining the maximum and minimum values ​​of positions in all pixel rows according to the positions of the grids of the region of interest in the raster image in each pixel row, and determining the original image width of the raster image; determining a sampling interval for resampling the raster image according to the maximum value, the minimum value, and the original image width; Acquire a second blank image to be filled, wherein the image height of the second blank image is the same as the image height of the grating image; The grating image is resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the second blank image to obtain an aligned grating image.

13. The distortion correction method according to claim 12, wherein: The step of aligning the object plane image row by row according to the position of the grid of the region of interest in each pixel row in the grating image to obtain the aligned object plane image comprises: Acquire a third blank image to be filled, wherein the image height of the third blank image is the same as the image height of the object plane image; The object plane image is resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the third blank image to obtain an aligned object plane image.

14. The distortion correction method according to claim 11, wherein: The step of performing line-by-line correction on the aligned object plane image according to the aligned grating image to obtain a target image comprises: Determining a column mean curve corresponding to the brightness value of each pixel row in each column of the aligned grating image, and performing peak fitting on the column mean curve of each column to obtain a third peak sequence; Acquire a fourth blank image to be filled, wherein the image height of the fourth blank image is the same as the image height of the aligned object plane image; Based on the position of the third peak sequence, the brightness values ​​of the corresponding pixel rows in the aligned object plane image are segmented and sampled, and the sampled brightness values ​​are sequentially filled into the fourth blank image to obtain a target image.

15. A capture card, characterized in that: The acquisition card stores computer-readable instructions, which, when executed by one or more processors, enable the one or more processors to perform the steps of the distortion correction method according to any one of claims 3 to 14.

Images