A laser confocal scanning microscope, a distortion correction method, and a data acquisition card.

Abstract

This application provides a laser confocal scanning microscope, a distortion correction method, and a data acquisition card. The laser confocal scanning microscope has a reflector fixedly installed between a resonant galvanometer mirror and a galvanometer mirror, a monitoring optical path and a second detector set in the transmission direction of the reflector mirror, and a data acquisition card connected to the first detector and the second detector respectively. The confocal signal and grating signal acquired synchronously and isochronously by the data acquisition card are temporal and spatial twin images. The grating image formed by the grating signal depicts the movement of the resonant galvanometer mirror during the scanning process. Therefore, this application can perform distortion correction on the twin object surface image by analyzing the distortion of the grating image, thereby effectively improving the correction accuracy and being less affected by factors such as environment, lifespan, and optical path deviation.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a laser confocal scanning microscope, a distortion correction method, and a data acquisition card. Background Technology

[0002] Industrial confocal microscopes, as high-precision, high-value-added measuring instruments, are powerful tools for measuring the surface morphology of objects, playing a crucial role in the research, development, production, and testing of materials including ceramics, metals, and semiconductors. Currently, confocal laser scanning has evolved into various scanning imaging methods, including galvanometer / resonance-based dual-mirror scanning, MEMS (Micro-Electro-Mechanical Systems) based galvanometer scanning, Nipkow disk scanning, and line scanning. Among these, the galvanometer / resonance-based dual-mirror scanning method has become the mainstream due to its cost and technological maturity.

[0003] Theoretically, resonant scanning is designed to perform simple harmonic motion at a fixed resonant frequency, with its angular velocity being maximum at the center and zero at both ends. This causes image distortion when using isochronous sampling. Methods to eliminate this distortion can be divided into hardware correction and software correction. Hardware correction generally refers to non-isochronous sampling using a pixel clock. However, the scanning curve of the resonant galvanometer changes with environmental factors such as temperature and humidity, while a fixed pixel clock cannot be changed. Furthermore, a fixed pixel clock limits the image size and pixel count, thus reducing system scalability. Software correction involves obtaining a distorted image using ordinary isochronous sampling and then correcting the image. One method is sinusoidal correction based on a standard sine curve, which uses a sine curve to represent the galvanometer's motion. However, this assumes the galvanometer is in an ideal working state, while in actual use, the galvanometer is often affected by environmental factors, lifespan, and optical path deviations, causing it to deviate from the standard sine curve. Therefore, the image corrected using this method still has significant distortion.

[0004] In summary, the hardware or software calibration methods used in the prior art have low calibration accuracy and are easily affected by factors such as environment, lifespan, and optical path deviation. Summary of the Invention

[0005] The purpose of this application is to at least solve one of the above-mentioned technical defects, in particular the technical defects of the hardware or software correction methods used in the prior art, which have low correction accuracy and are easily affected by factors such as environment, lifespan and optical path deviation.

[0006] This application provides a laser confocal scanning microscope, including a laser source, a resonant galvanometer, a galvanometer galvanometer, a beam splitter, an excitation optical path, a confocal optical path, and a first detector. The microscope also includes: a transflector fixedly installed between the resonant galvanometer and the galvanometer galvanometer, a monitoring optical path and a second detector arranged in the transmission direction of the transflector, and a data acquisition card connected to the first detector and the second detector respectively.

[0007] In this process, when the microscope scans the sample, the laser light emitted by the laser source is reflected by the beam splitter to the resonant mirror. The resonant mirror deflects the incident light beam for the first time and then emits it to the transmission mirror. The transmission mirror reflects most of the light beam to the galvanometer mirror. The galvanometer mirror deflects the incident light beam a second time and then focuses the deflected light beam onto the sample surface through the excitation optical path. The reflected light excited by the sample surface returns along the original path, is transmitted through the beam splitter to the confocal optical path, and reaches the first detector through the confocal optical path. The transmission mirror transmits a small portion of the light beam to the monitoring optical path, and the monitoring optical path converts the real-time deflection angle of the resonant mirror into a grating signal, which is then sent to the second detector.

[0008] The acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, and generates an object surface image corresponding to the confocal signal and a grating image corresponding to the grating signal. Then, it uses the grating image to perform distortion correction on the object surface image to obtain the target image.

[0009] Optionally, the monitoring optical path includes an attenuator and a Ronche grating;

[0010] The attenuator attenuates the transmitted light beam before it is emitted to the Ronchi grating, and the Ronchi grating converts the real-time deflection angle of the resonant mirror into a grating signal based on the attenuated light beam.

[0011] This application also provides a distortion correction method, applied to the acquisition card in the laser confocal scanning microscope described in any of the above embodiments, the method comprising:

[0012] The confocal signal output by the first detector and the grating signal output by the second detector are acquired synchronously and isochronously, and an object surface image corresponding to the confocal signal and a grating image corresponding to the grating signal are generated.

[0013] The object surface image is distorted using the raster image to obtain the target image.

[0014] Optionally, the step of using the raster image to correct the distortion of the object surface image to obtain the target image includes:

[0015] Determine the position of at least one grid cell in the raster image within each pixel row;

[0016] The object image is corrected row by row based on the position of the at least one grid in each pixel row to obtain the target image.

[0017] Optionally, determining the position of at least one raster in the raster image within each pixel row includes:

[0018] If the current scene is a high-precision scene, then determine the position of all grids in the raster image in each pixel row;

[0019] If the current scene is a real-time scene and the resonant galvanometer is scanned in one direction, then the position of the grid of the region of interest in the grating image in each pixel row is determined.

[0020] Optionally, determining the position of all grids in the raster image within each pixel row includes:

[0021] Determine the row brightness curve corresponding to the column brightness value in each pixel row of the raster image;

[0022] The row brightness curves of each pixel row are smoothed, and peak fitting is performed on the smoothed row brightness curves until the number of peaks in each pixel row is equal, thus obtaining the first peak sequence of each pixel row.

[0023] The position of all gratings in each pixel row of the raster image is determined based on the first peak sequence of each pixel row.

[0024] Optionally, the step of smoothing the row brightness curves of each pixel row and performing peak fitting on the smoothed row brightness curves until the number of peaks in each pixel row is equal, to obtain the first peak sequence of each pixel row, includes:

[0025] Set the current value of the smoothing radius;

[0026] Based on the current values, the row brightness curves of each pixel row are smoothed, and peak fitting is performed on the smoothed row brightness curves to obtain the preliminary peak position sequence of each pixel row.

[0027] The initial peak position sequence of each pixel row is compared. If the number of peaks in at least one pixel row is not equal to the number of peaks in other pixel rows, the current value is updated, and the smoothing process of the row brightness curve of each pixel row based on the current value and subsequent steps are continued until the number of peaks in each pixel row is equal.

[0028] If the number of peaks in each pixel row is equal, then the preliminary peak position sequence of each pixel row is taken as the final first peak sequence.

[0029] Optionally, determining the position of the raster grid in the region of interest of the raster image in each pixel row includes:

[0030] The column mean brightness curve corresponding to the brightness value of each pixel row in each column of the raster image is determined, and the second peak sequence is obtained by performing peak fitting on the column mean brightness curve of each column.

[0031] The region of interest in the raster image is cropped according to the second peak sequence, and the peak position of each pixel row is extracted from the cropped raster image as the position of the raster grid of the region of interest in each pixel row.

[0032] Optionally, cropping the region of interest in the raster image based on the second peak sequence includes:

[0033] Determine the number of peaks in the second peak sequence;

[0034] Based on the number of peaks in the second peak sequence, determine the start and end positions when cropping the raster image;

[0035] The region of interest in the raster image is cropped based on the starting position and the ending position.

[0036] Optionally, the step of performing line-by-line correction on the object surface image based on the position of the at least one grid in each pixel row to obtain the target image includes:

[0037] Obtain a first blank image to be filled, wherein the image height of the first blank image is the same as the image height of the object surface image;

[0038] Based on the position of all grids in the raster image in each pixel row, the brightness values ​​of the corresponding pixel rows in the object image are sampled in segments, and the sampled brightness values ​​are sequentially filled into the first blank image to obtain the target image.

[0039] Optionally, the step of performing line-by-line correction on the object surface image based on the position of the at least one grid in each pixel row to obtain the target image includes:

[0040] Based on the position of the raster grid in the region of interest in each pixel row, the raster image is aligned row by row to obtain the aligned raster image;

[0041] Based on the position of the raster grid in the region of interest in the raster image in each pixel row, the object surface image is aligned row by row to obtain the aligned object surface image;

[0042] The aligned object surface image is corrected line by line based on the aligned raster image to obtain the target image.

[0043] Optionally, the step of aligning the raster image row by row according to the position of the raster grid in the region of interest in each pixel row to obtain the aligned raster image includes:

[0044] Based on the position of the raster grid in the region of interest in the raster image in each pixel row, determine the maximum and minimum values ​​of the positions in all pixel rows, and determine the original image width of the raster image;

[0045] The sampling interval for resampling the raster image is determined based on the maximum value, the minimum value, and the original image width;

[0046] Obtain a second blank image to be filled, the image height of the second blank image being the same as the image height of the raster image;

[0047] The raster image is resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the second blank image to obtain an aligned raster image.

[0048] Optionally, the step of aligning the object surface image row by row according to the position of the raster grid in the region of interest in each pixel row to obtain the aligned object surface image includes:

[0049] Obtain the third blank image to be filled, wherein the image height of the third blank image is the same as the image height of the object surface image;

[0050] The object surface image is resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the third blank image to obtain the aligned object surface image.

[0051] Optionally, the step of performing line-by-line correction on the aligned object surface image based on the aligned raster image to obtain the target image includes:

[0052] The column mean curve corresponding to the brightness value of each pixel row in each column of the aligned raster image is determined, and the third peak sequence is obtained by performing peak fitting on the column mean curve of each column.

[0053] Obtain the fourth blank image to be filled, wherein the image height of the fourth blank image is the same as the image height of the aligned object surface image;

[0054] Based on the position of the third peak sequence, the brightness values ​​of the corresponding pixel rows in the aligned object image are sampled in segments, and the sampled brightness values ​​are sequentially filled into the fourth blank image to obtain the target image.

[0055] This application also provides a data acquisition card that stores computer-readable instructions. When executed by one or more processors, the computer-readable instructions cause the one or more processors to perform the steps of the distortion correction method as described in any of the above embodiments.

[0056] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0057] This application provides a laser confocal scanning microscope, a distortion correction method, and a data acquisition card. The laser confocal scanning microscope includes not only a laser source, a resonant mirror, a galvanometer mirror, a beam splitter, an excitation optical path, a confocal optical path, and a first detector, but also a transflecting mirror fixedly installed between the resonant mirror and the galvanometer mirror, a monitoring optical path and a second detector arranged in the transmission direction of the transflecting mirror, and a data acquisition card connected to the first and second detectors respectively. When scanning a sample, the laser beam emitted by the microscope is reflected by the beam splitter to the resonant mirror. The resonant mirror initially deflects the incident beam before reflecting it to the transflecting mirror, which then reflects most of the beam to the galvanometer. The galvanometer galvanometer deflects the incident light beam twice, then focuses the deflected beam onto the sample surface through the excitation optical path. The reflected light excited by the sample surface returns along the original path, is transmitted through a beam splitter to the confocal optical path, and then reaches the first detector. A mirror transmits a small portion of the beam to the monitoring optical path, which converts the real-time deflection angle of the resonant galvanometer into a grating signal and sends it to the second detector. The acquisition card synchronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, and generates an object surface image corresponding to the confocal signal and a grating image corresponding to the grating signal. The object surface image is then distorted using the grating image to obtain the target image. During this process, the confocal signal and grating signal acquired synchronously and isochronously by the acquisition card are twin images in time and space. The grating image formed by the grating signal depicts the motion of the resonant mirror during the scanning process. Therefore, this application can perform distortion correction on the object surface image twinned with it by analyzing the distortion of the grating image, thereby effectively improving the correction accuracy and making it less susceptible to the influence of factors such as environment, lifespan, and optical path deviation. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 A schematic diagram of the structure of a laser confocal scanning microscope provided in this application embodiment;

[0060] Figure 2 This application provides an example of an optical path diagram illustrating beam splitting using a mirror.

[0061] Figure 3 An illustration of a raster image provided in an embodiment of this application;

[0062] Figure 4 An illustration of the object surface image provided in the embodiments of this application;

[0063] Figure 5 A schematic flowchart illustrating a distortion correction method provided in an embodiment of this application;

[0064] Figure 6 An illustration showing a deviation in the brightest position between two adjacent rows of the same grating provided in this application embodiment;

[0065] Figure 7 A schematic diagram of the row brightness curve corresponding to a pixel row in a raster image provided in an embodiment of this application;

[0066] Figure 8 A schematic diagram of the peak fitting result for one pixel row provided in an embodiment of this application;

[0067] Figure 9 This is a diagram illustrating the region of interest at the center of a raster image provided in an embodiment of this application.

[0068] Figure 10 This is a diagram illustrating the target image after distortion correction of the object surface image using the full raster alignment distortion correction algorithm, as provided in an embodiment of this application.

[0069] Figure 11 This application provides a magnified comparison of the raster images before and after alignment in an embodiment of the application.

[0070] Figure 12 A magnified comparison of the object surface images before and after alignment, provided in an embodiment of this application;

[0071] Figure 13This is a diagram illustrating the target image after distortion correction of the object surface image using a local raster alignment correction algorithm, as provided in an embodiment of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] In one embodiment, such as Figure 1 As shown, Figure 1 This is a schematic diagram of the structure of a laser confocal scanning microscope provided in an embodiment of this application. This application provides a laser confocal scanning microscope, including a laser source, a resonant galvanometer, a galvanometer, a beam splitter, an excitation optical path, a confocal optical path, and a first detector. The microscope also includes: a transflector fixedly installed between the resonant galvanometer and the galvanometer, a monitoring optical path and a second detector arranged in the transmission direction of the transflector, and a data acquisition card connected to the first detector and the second detector respectively.

[0074] In this process, when the microscope scans the sample, the laser beam emitted by the laser source is reflected by the beam splitter to the resonant mirror. The resonant mirror deflects the incident beam for the first time and then emits it to the transmission mirror. The transmission mirror reflects most of the beam to the galvanometer mirror. The galvanometer mirror deflects the incident beam a second time and then focuses the deflected beam onto the sample surface through the excitation optical path. The reflected light excited by the sample surface returns along the original path, is transmitted through the beam splitter to the confocal optical path, and reaches the first detector through the confocal optical path. The transmission mirror transmits a small portion of the beam to the monitoring optical path, and the monitoring optical path converts the real-time deflection angle of the resonant mirror into a grating signal, which is then sent to the second detector.

[0075] The acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, and generates an object surface image corresponding to the confocal signal and a grating image corresponding to the grating signal. Then, it uses the grating image to perform distortion correction on the object surface image to obtain the target image.

[0076] In this embodiment, as Figure 1As shown, this application adds a motion acquisition optical path for a resonant galvanometer to the existing laser confocal scanning microscope equipped with a two-dimensional galvanometer. This acquisition optical path consists of a reflective mirror positioned between the resonant galvanometer and the galvanometer mirror, a monitoring optical path and a second detector positioned in the transmission direction of the reflective mirror, and acquisition cards connected to the first and second detectors respectively. Since the reflective mirror can selectively transmit or reflect light according to its wavelength, this application can reflect most of the excitation light to the sample while transmitting a small portion for monitoring the motion of the resonant galvanometer. This allows for real-time recording of the nonlinear motion (such as a sinusoidal waveform) of the resonant galvanometer using the monitoring optical path and the second detector. The acquisition card in this application can record two signals using the same timestamp, ensuring pixel-level alignment and allowing the pixel position remapping of the object surface image acquired by the first detector using the galvanometer motion data acquired by the second detector, thereby eliminating sinusoidal scanning distortion.

[0077] In this application, the transflector is fixedly mounted in the optical path between the resonant galvanometer and the galvanometer galvanometer. This ensures a constant beam splitting ratio and direction, and simplifies system calibration. Furthermore, the beam splitting ratio of the transflector can be set. For example, 95% of the energy in the beam received by the transflector can be used for sample scanning, and 5% can be used to feedback the galvanometer status. The specific ratio can be set according to actual conditions and is not limited here. This ensures that a high-resolution object surface image is obtained while the grating image acquired by the second detector eliminates scanning distortion of the object surface image.

[0078] Furthermore, the laser source of this application can provide monochromatic, high-brightness excitation light (such as Argon ion laser, semiconductor laser, etc.); the resonant mirror of this application can be a high-frequency resonant mirror (such as 4-12 kHz), and the galvanometer mirror can be a low-speed galvanometer mirror (such as 1-30 Hz); the excitation optical path of this application can include a scanning lens and an objective lens, which can focus the beam deflected by the galvanometer mirror onto the sample surface and collect the reflection / fluorescence signal; the confocal optical path of this application is located in front of the first detector to block the defocused light signal and improve the axial resolution; the first and second detectors of this application can use photomultiplier tubes (PMTs) or avalanche photodiodes (APDs) to convert the optical signal into an electrical signal.

[0079] For example, when scanning a sample, the resonant galvanometer oscillates at a high frequency in the form of a sine wave, driving the beam to scan rapidly back and forth across the sample surface. This process only utilizes the linear range of the sine wave (such as near the peaks or troughs) to acquire signals. During the retrace phase (nonlinear range), the laser is typically turned off or data is discarded. During this process, the galvanometer galvanometer moves slowly in a sawtooth wave pattern, moving one step (corresponding to one row of pixels in the image) after completing each X-axis scan, and quickly resetting to the starting position at the end of the frame. After being deflected by the two-dimensional galvanometer and reflected by the mirror, the laser beam is focused onto the sample surface through the scanning lens and objective lens. The fluorescence or reflected light excited by the sample returns along the original path, is separated by a beam splitter, and reaches the first detector through a confocal optical path. The first detector converts the optical signal into an electrical signal and synchronizes it with the galvanometer position, constructing a two-dimensional image pixel by pixel, i.e., the object surface image in this application. Furthermore, the mirror in this application can also transmit a small portion of the laser beam. This portion of the laser beam is collected by the monitoring optical path to convert the real-time deflection angle of the resonant mirror into a grating signal. The monitoring optical path is connected to a second detector, allowing the grating signal to be sent to the acquisition card. After image processing, the acquisition card obtains a grating image. Next, this application can use the grating image to correct distortion in the object surface image, thus obtaining the target image. For example, by determining the location of each grid cell in the grating image (the location of the maximum local brightness), the distortion level between two adjacent locations can be determined, and distortion correction can be performed based on this distortion level.

[0080] In the above embodiments, the laser confocal scanning microscope includes not only a laser source, a resonant mirror, a galvanometer mirror, a beam splitter, an excitation optical path, a confocal optical path, and a first detector, but also a transflector mirror fixedly installed between the resonant mirror and the galvanometer mirror, a monitoring optical path and a second detector arranged in the transmission direction of the transflector mirror, and a data acquisition card connected to the first detector and the second detector respectively. When the microscope scans a sample, the laser beam emitted by the laser source is reflected by the beam splitter to the resonant mirror. The resonant mirror initially deflects the incident beam before transmitting it to the transflector mirror. The transflector mirror reflects most of the beam to the galvanometer mirror, and the galvanometer mirror reflects the incident beam to the transflector mirror. After the light beam undergoes a secondary deflection, it is focused onto the sample surface through the excitation optical path. The reflected light excited by the sample surface returns along the original path, is transmitted through the beam splitter to the confocal optical path, and then reaches the first detector. The transmission mirror transmits a small portion of the light beam to the monitoring optical path, and the monitoring optical path converts the real-time deflection angle of the resonant mirror into a grating signal, which is then sent to the second detector. The acquisition card synchronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, and generates an object surface image corresponding to the confocal signal and a grating image corresponding to the grating signal. The object surface image is then distorted using the grating image to obtain the target image. During this process, the confocal signal and grating signal acquired synchronously and isochronously by the acquisition card are twin images in time and space. The grating image formed by the grating signal depicts the motion of the resonant mirror during the scanning process. Therefore, this application can perform distortion correction on the object surface image twinned with it by analyzing the distortion of the grating image, thereby effectively improving the correction accuracy and making it less susceptible to the influence of factors such as environment, lifespan, and optical path deviation.

[0081] In one embodiment, such as Figure 2 As shown, Figure 2 This is a diagram illustrating the optical path for beam splitting using a mirror, as provided in an embodiment of this application. The monitoring optical path may include an attenuator and a Ronchi grating.

[0082] The attenuator attenuates the transmitted light beam before it is emitted to the Ronchi grating, and the Ronchi grating converts the real-time deflection angle of the resonant mirror into a grating signal based on the attenuated light beam.

[0083] In this embodiment, the monitoring optical path set in the transmission direction of the transflector may include an attenuator and a Ronche grating. The attenuator attenuates the transmitted light beam before it is emitted to the Ronche grating, preventing high-power laser (transmitted beam) from damaging the second detector. The Ronche grating converts the real-time deflection angle of the resonant mirror into a grating signal based on the attenuated beam. For example, the Ronche grating of this application can utilize the deflection of the resonant mirror to move the light beam on the grating, thereby generating a periodic light intensity signal. This periodic light intensity signal is the grating signal of this application, which directly maps to the real-time deflection angle of the resonant mirror. After processing by the acquisition card, the signal can be obtained as shown below. Figure 3 The grating image shown corresponds to the direction of motion of the resonant mirror in the horizontal direction. Each row of the image corresponds to one cycle of the resonant mirror motion. The distance between the grids reflects the motion of the resonant mirror, that is, the distortion level at each position.

[0084] Simultaneously, the acquisition card of this application can also process the acquired confocal signal, which reflects the fluorescence / reflection light intensity distribution of the sample and is spatiotemporally aligned with the grating signal. Therefore, this application can generate an image such as... after image processing of the confocal signal. Figure 4 The object surface image is shown, and then the galvanometer motion data reflected by the grating image is used to correct the distortion of the object surface image, thereby effectively improving the accuracy of distortion correction while avoiding the influence of factors such as environment, lifespan and optical path deviation.

[0085] In one embodiment, such as Figure 5 As shown, Figure 5 This application provides a schematic flowchart of a distortion correction method according to an embodiment of the present application; the present application also provides a distortion correction method applied to the acquisition card in the laser confocal scanning microscope described in any of the above embodiments, the method including:

[0086] S110: Synchronously acquire the confocal signal output by the first detector and the grating signal output by the second detector, and generate an object surface image corresponding to the confocal signal and a grating image corresponding to the grating signal.

[0087] S120: Use a raster image to correct the distortion of the object surface image to obtain the target image.

[0088] In this embodiment, as Figure 1As shown, this application adds a motion acquisition optical path for a resonant galvanometer to the existing laser confocal scanning microscope equipped with a two-dimensional galvanometer. This acquisition optical path consists of a reflective mirror positioned between the resonant galvanometer and the galvanometer mirror, a monitoring optical path and a second detector positioned in the transmission direction of the reflective mirror, and acquisition cards connected to the first and second detectors respectively. Since the reflective mirror can selectively transmit or reflect light according to its wavelength, this application can reflect most of the excitation light to the sample while transmitting a small portion for monitoring the motion of the resonant galvanometer. This allows for real-time recording of the nonlinear motion (such as a sinusoidal waveform) of the resonant galvanometer using the monitoring optical path and the second detector. The acquisition card in this application can record two signals using the same timestamp, ensuring pixel-level alignment and allowing the pixel position remapping of the object surface image acquired by the first detector using the galvanometer motion data acquired by the second detector, thereby eliminating sinusoidal scanning distortion.

[0089] In one specific implementation, when scanning the sample, the resonant galvanometer oscillates at a high frequency in the form of a sine wave, driving the beam to scan rapidly back and forth across the sample surface. This process only utilizes the linear interval of the sine wave (such as near the peaks or troughs) to acquire signals; during the retrace phase (nonlinear interval), the laser is typically turned off or data is discarded. During this process, the galvanometer galvanometer moves slowly in the form of a sawtooth wave, moving one step (corresponding to one row of pixels in the image) after completing each X-axis scan, and quickly resetting to the starting position at the end of the frame. After being deflected by the two-dimensional galvanometer and reflected by the mirror, the laser beam is focused onto the sample surface through the scanning lens and objective lens. The fluorescence or reflected light excited by the sample returns along the original path, is separated by a beam splitter, and reaches the first detector through a confocal optical path. The first detector converts the optical signal into an electrical signal and synchronizes it with the galvanometer position, constructing a two-dimensional image pixel by pixel, i.e., the object surface image in this application. Furthermore, the mirror in this application can also transmit a small portion of the laser beam. This portion of the laser beam is collected by the monitoring optical path to convert the real-time deflection angle of the resonant mirror into a grating signal. The monitoring optical path is connected to a second detector, allowing the grating signal to be sent to the acquisition card. After image processing, the acquisition card obtains a grating image. Next, this application can use the grating image to correct distortion in the object surface image, thus obtaining the target image. For example, by determining the location of each grid cell in the grating image (the location of the maximum local brightness), the distortion level between two adjacent locations can be determined, and distortion correction can be performed based on this distortion level.

[0090] In the above embodiments, the confocal signal and grating signal acquired synchronously and isochronously by the acquisition card are twin images in time and space. The grating image formed by the grating signal depicts the motion of the resonant mirror during the scanning process. Therefore, this application can perform distortion correction on the object surface image twinned with it by analyzing the distortion of the grating image, thereby effectively improving the correction accuracy and making it less susceptible to the influence of factors such as environment, lifespan and optical path deviation.

[0091] In one embodiment, the process of using the raster image to correct the distortion of the object surface image to obtain the target image in step S120 may include:

[0092] S121: Determine the position of at least one grid cell in the raster image in each pixel row.

[0093] S122: The object image is corrected row by row according to the position of the at least one grid in each pixel row to obtain the target image.

[0094] In this embodiment, when using a raster image to correct distortion of an object surface image, the position of at least one grid in the raster image in each pixel row can be determined first. In this way, the object surface image can be corrected according to the position of at least one grid in each pixel row to obtain the target image.

[0095] In this application, when determining the position of at least one grid in a raster image within each pixel row, the number of grids to be determined can be determined based on the current scene. For example, if the current scene requires high precision, the object image can be corrected by determining the positions of all grids in the raster image within each pixel row. If the current scene requires relatively low precision but high real-time performance, the object image can be corrected by filtering local grids, thus meeting the real-time requirements.

[0096] In one embodiment, determining the position of at least one raster in the raster image in each pixel row in step S121 may include:

[0097] S1211: If the current scene is a high-precision scene, then determine the position of all grids in the raster image in each pixel row.

[0098] S1212: If the current scene is a real-time scene and the resonant galvanometer is scanned in one direction, then determine the position of the grid of the region of interest in the grating image in each pixel row.

[0099] In this embodiment, when determining the position of at least one grid in the raster image within each pixel row, the scene characteristics of the current scene can be determined first. If the current scene is a high-precision scene, i.e., a scene where the accuracy requirement for image distortion correction is higher than a preset accuracy threshold, such as micro-nano fabrication, super-resolution imaging, precision measurement, and other application scenarios, the preset accuracy threshold can be a threshold in one dimension, such as error or resolution, or it can be a threshold in multiple dimensions. The specific threshold can be set according to the actual situation and is not limited here. When the current scene is determined to be a high-precision scene, this application can correct the object surface image by using the positions of all grids in the raster image within each pixel row, thereby achieving precise distortion correction of the object surface image.

[0100] Furthermore, if the current scene is a real-time scene, and the resonant galvanometer is unidirectionally scanning—that is, a scene where the real-time requirement for image distortion correction is higher than a preset duration threshold, and the adjacent motion cycles of the resonant galvanometer only deviate at the starting point—such as applications like laser processing, high-speed imaging, and dynamic displays, the preset duration threshold in these scenarios can be set according to the actual situation and is not limited here. When this application determines that the current scene is a real-time scene, the raster region of interest in the raster image can be determined by filtering local raster grids. This raster region of interest can be a raster at the center of the raster image or a raster at a non-center location, depending on the actual situation and is not limited here. This application corrects the object surface image using the raster region of interest, which can greatly shorten the processing time for distortion correction and thus meet the real-time requirements.

[0101] In one embodiment, determining the position of all grids in the raster image in each pixel row in step S1211 may include:

[0102] S12111: Determine the row brightness curve corresponding to the column brightness value in each pixel row of the raster image.

[0103] S12112: Smooth the row brightness curve of each pixel row, and perform peak fitting on the smoothed row brightness curve until the number of peaks in each pixel row is equal, thus obtaining the first peak sequence of each pixel row.

[0104] S12113: Determine the position of all grids in the raster image in each pixel row based on the first peak sequence of each pixel row.

[0105] In this embodiment, when determining the position of all grids in each pixel row of the grating image, due to mechanical motion errors and electronic noise, there is a certain degree of misalignment between the brightest positions of the same grid in adjacent rows (corresponding to two adjacent motion cycles of the resonant mirror). Indicatively, as shown... Figure 6 As shown, Figure 6 An illustration showing a deviation in the brightest position between two adjacent rows of the same grating provided in this application embodiment; Figure 6 The positional deviation can be attributed to the discrepancy between the starting points of adjacent motion cycles of the resonant mirror and the time it takes to reach the same grid. This means that it is not possible to simply treat a grid as a position for correction.

[0106] Based on this, when determining the position of each grid in a raster image, this application can first determine the brightness value of each column in each pixel row of the raster image, and form a row brightness curve corresponding to each pixel row. = c=1, ..., C (where C is the image width of the raster image, i.e., the number of pixel columns). Let be the brightness value of the pixel in the r-th row and c-th column of the raster image. This brightness value represents the lightness or color intensity at that location and can be directly determined from the raster image. (Illustratively, as shown...) Figure 7 As shown, Figure 7 This is a schematic diagram of the row brightness curve corresponding to a pixel row in a raster image provided in an embodiment of this application. Figure 7 In the grid, each peak corresponds to one grid cell.

[0107] Next, this application can smooth the row brightness curve of each pixel row to highlight the main trend or structure of the row brightness curve by suppressing noise or irrelevant details, thereby improving the readability, analyzability or visual effect of the row brightness curve, and providing a more accurate data basis for subsequent peak fitting.

[0108] After smoothing the row brightness curves of each pixel row in this application, peak fitting can be performed on the smoothed row brightness curves. This allows the determination of the number of peaks in each pixel row. When the number of peaks in each pixel row is equal, the first peak sequence of each pixel row can be obtained. i=1, ..., ,in Let be the number of peaks obtained by fitting the row brightness curve of row r to the current smoothing radius, schematically, as shown below. Figure 8 As shown, Figure 8 A schematic diagram of the peak fitting result for one pixel row provided in an embodiment of this application; Figure 8 The red line in the graph represents the position of the brightest pixel in the grid area, and the green line represents the fitted position. Obviously, the fitted position is more reasonable.

[0109] Based on the above processing, this application can obtain the first peak sequence corresponding to each pixel row. Since this first peak sequence indicates the peak positions of multiple peaks fitted to each pixel row under the current smoothing radius, this application can determine the positions of all grids in each pixel row of the raster image based on the first peak sequence of each pixel row, and form a corresponding grid position matrix. Where r=1, ..., R, i=1, ..., N, R is the image height of the raster image, i.e., the number of pixel rows, and N is the number of raster cells per row. Let be the position (i.e., the horizontal coordinate) of the i-th grid in the r-th row within the raster image.

[0110] In one embodiment, step S12112 involves smoothing the row brightness curves of each pixel row and performing peak fitting on the smoothed row brightness curves until the number of peaks in each pixel row is equal, thus obtaining the first peak sequence for each pixel row. This step may include:

[0111] S121121: Sets the current value of the smoothing radius.

[0112] S121122: Based on the current value, smooth the row brightness curve of each pixel row, and perform peak fitting on the smoothed row brightness curve to obtain the preliminary peak position sequence of each pixel row.

[0113] S121123: Compare the preliminary peak position sequence of each pixel row. If the number of peaks in at least one pixel row is not equal to the number of peaks in other pixel rows, update the current value and continue to perform the smoothing process of the row brightness curve of each pixel row based on the current value and subsequent steps until the number of peaks in each pixel row is equal.

[0114] S121124: If the number of peaks in each pixel row is equal, then the preliminary peak position sequence of each pixel row is taken as the final first peak sequence.

[0115] In this embodiment, when smoothing the row brightness curve of each pixel row, the choice of smoothing radius (or smoothing window size) directly affects the quality of the processing effect and the degree of feature preservation. Therefore, this application can first set the current value of the smoothing radius, which can be an initial value. Then, based on the current value, the row brightness curve of each pixel row is smoothed, and then peak fitting is performed. After obtaining the preliminary peak position sequence of each pixel row, the number of peaks in the preliminary peak position sequence of each pixel row can be compared. If the number of peaks in at least one pixel row is not equal to the number of peaks in other pixel rows, the current value of the smoothing radius is updated. For example, when the initial value s=1, the updated current value can be s=s+1. Then, this application can continue to smooth the row brightness curve of each pixel row based on the updated current value, and perform peak fitting on the smoothed row brightness curve to obtain the preliminary peak position sequence of each pixel row. The number of peaks in the preliminary peak position sequence of each pixel row is compared until the number of peaks in each pixel row is equal. At this time, the preliminary peak position sequence of each pixel row can be used as the final first peak sequence. This application can determine the position of all grids in each pixel row of the raster image based on the first peak sequence of each pixel row, and perform distortion correction on the object image based on the position of each grid in each pixel row, so as to obtain accurate correction results.

[0116] In one embodiment, determining the position of the raster grid in the region of interest of the raster image in each pixel row in S1212 may include:

[0117] S12121: Determine the column mean brightness curve corresponding to the brightness value of each pixel row in each column of the raster image, and obtain the second peak sequence by performing peak fitting on the column mean brightness curve of each column.

[0118] S12122: The region of interest in the raster image is cropped according to the second peak sequence, and the peak position of each pixel row is extracted from the cropped raster image as the position of the raster of interest in each pixel row.

[0119] In this embodiment, if the current scene is a real-time scene, and the resonant galvanometer is unidirectionally scanning—that is, a scene where the real-time requirement for image distortion correction is higher than a preset time threshold, and the adjacent motion cycles of the resonant galvanometer only deviate at the starting point—such as applications like laser processing, high-speed imaging, and dynamic display, the raster of interest in the raster image can be determined by filtering local raster grids. This raster of interest can be a raster at the center of the raster image or a raster at a non-center location; the choice depends on the actual situation and is not limited here. This application corrects the object surface image using the raster of interest region, which can greatly shorten the distortion correction processing time and thus meet real-time requirements.

[0120] Specifically, when determining the position of the raster grid in each pixel row of the region of interest in a raster image, this application can first determine the column mean brightness curve corresponding to the brightness value of each pixel row in each column of the raster image, and then perform peak fitting on the column mean brightness curve of each column to obtain a second peak sequence. Then, this application can crop the region of interest in the raster image according to the second peak sequence, and extract the peak position of each pixel row from the cropped raster image. The peak position is the position of the raster grid in each pixel row of the region of interest in the raster image.

[0121] For example, when this application calculates the column mean brightness curve corresponding to the brightness value of each pixel row in each column of the raster image... After that, you can... Peak fitting was performed to obtain the second peak sequence. Let k = 1, ..., K, where K is the number of peaks. Since this second peak sequence indicates the number of grids in the raster image, and the resonant mirror in this application is unidirectionally scanning, this application can select the region with the lowest distortion level in the raster image as the region of interest based on the scanning characteristics of the resonant mirror and the number of grids in the raster image. After cropping the region of interest, the peak position of each pixel row is extracted from the cropped raster image. This peak position is the position of the grid of the region of interest in each pixel row of the raster image. In this way, during distortion correction, only one region of interest in the raster image needs to be detected row by row, thereby greatly shortening the correction time and meeting the real-time requirements.

[0122] In one embodiment, cropping the region of interest of the raster image according to the second peak sequence in step S12122 may include:

[0123] S121221: Determine the number of peaks in the second peak sequence.

[0124] S121222: Determine the starting and ending positions when cropping the raster image based on the number of peaks in the second peak sequence.

[0125] S121223: The region of interest of the raster image is cropped according to the starting position and the ending position.

[0126] In this embodiment, when cropping the region of interest of the raster image according to the second peak sequence, the region of interest can be the central region of the raster image or a non-central region. The specific choice can be made according to the actual situation and is not limited here.

[0127] In one specific implementation, when the application selects the central region of the raster image as the region of interest, the number of peaks in the second peak sequence can be determined first, and then the starting position and ending position for cropping the raster image can be determined based on the number of peaks in the second peak sequence. In this way, the region of interest of the raster image can be cropped based on the starting position and the ending position.

[0128] For example, when the second peak sequence of this application is When k=1, ...,K, where K is the number of peaks, this application can set the start position and end position as follows:

[0129]

[0130]

[0131] in, That is, the midpoint of the second peak sequence. Starting position The final position, The midpoint peak of the second peak sequence. The peak preceding the midpoint of the second peak sequence. The peak following the midpoint of the second peak sequence, after cropping the raster image using the aforementioned start and end positions, can be obtained as follows: Figure 9 The region of interest at the center of the raster image shown contains only the brightness data of one raster. By using this region of interest to perform line-by-line correction on the object surface image, the final target image can be obtained, thus meeting the real-time requirements.

[0132] In one embodiment, S122, which involves performing row-by-row correction on the object surface image based on the position of the at least one raster in each pixel row to obtain a target image, may include:

[0133] S1221: Obtain the first blank image to be filled, wherein the image height of the first blank image is the same as the image height of the object surface image.

[0134] S1222: Based on the position of all grids in the raster image in each pixel row, the brightness values ​​of the corresponding pixel rows in the object image are sampled in segments, and the sampled brightness values ​​are sequentially filled into the first blank image to obtain the target image.

[0135] In this embodiment, if the current scene is a high-precision scene, that is, a scene where the accuracy requirement for image distortion correction is higher than the preset accuracy threshold, such as micro-nano fabrication, super-resolution imaging, precision measurement and other application scenarios, this application can correct the object surface image by the position of all grids in each pixel row in the grating image, thereby achieving accurate distortion correction of the object surface image.

[0136] Specifically, this application can obtain a first blank image to be filled. The image height of the first blank image The image height can be consistent with the object surface image, and the image width W can be a pre-specified value, such as W=1024. Next, this application can sequentially extract the r-th row of the object surface image. = r=1, ..., R, c=1, ..., C, where R is the maximum number of rows in the object plane image and C is the maximum number of columns in the object plane image. Let the brightness value be the value in the r-th row and c-th column of the object surface image. Then, this application can perform the following mapping:

[0137]

[0138] Where T can be either a linear or nonlinear mapping, and J = W / (N-1). This mapping is equivalent to... lie in The brightness curve segment in the image is linearly or non-linearly sampled, with J sampling points. The number of sampling points can be set according to the actual situation. Then, the sampled data is filled into the corresponding positions of the first blank image to obtain the final target image, which is shown below. Figure 10 As shown, by Figure 10 As can be seen, this application can achieve accurate distortion correction of the object surface image by performing distortion correction on the object surface image using the above distortion correction method.

[0139] In one embodiment, S122, which involves performing row-by-row correction on the object surface image based on the position of the at least one raster in each pixel row to obtain a target image, may include:

[0140] S221: Based on the position of the raster grid in the region of interest in the raster image in each pixel row, the raster image is aligned row by row to obtain the aligned raster image.

[0141] S222: Based on the position of the raster grid in the region of interest in the raster image in each pixel row, the object surface image is aligned row by row to obtain the aligned object surface image.

[0142] S223: The aligned object surface image is corrected line by line based on the aligned raster image to obtain the target image.

[0143] In this embodiment, if the current scene is a real-time scene and the resonant galvanometer is a unidirectional scanning scene, that is, a scene where the real-time requirement for image distortion correction is higher than the preset time threshold and the adjacent motion cycles of the resonant galvanometer only deviate at the starting point, such as laser processing, high-speed imaging, dynamic display and other application scenarios, the grating of the region of interest in the raster image can be determined by filtering local gratings, and the object surface image can be corrected by using the grating of the region of interest. This can greatly shorten the processing time of distortion correction and thus meet the real-time requirements.

[0144] Specifically, when performing line-by-line correction of the object surface image based on the position of the raster grid in the region of interest in each pixel row, this application can first align the raster image line by line based on the position of the raster grid in the region of interest in each pixel row to obtain an aligned raster image. Then, the object surface image can be aligned line by line based on the position of the raster grid in the region of interest in each pixel row to obtain an aligned object surface image. Finally, the aligned object surface image is used to perform line-by-line correction, thus obtaining the target image. Moreover, the time required to obtain the target image by this method is significantly shorter than the time required to obtain the target image by performing line-by-line correction of the object surface image based on the position of all rasters in the raster image in each pixel row. Therefore, it can largely meet the real-time requirements.

[0145] In one embodiment, step S221, aligning the raster image row by row according to the position of the raster grid in the region of interest in each pixel row to obtain an aligned raster image, may include:

[0146] S2211: Based on the position of the raster grid in the region of interest in the raster image in each pixel row, determine the maximum and minimum values ​​of the positions in all pixel rows, and determine the original image width of the raster image.

[0147] S2212: Determine the sampling interval for resampling the raster image based on the maximum value, the minimum value, and the original image width.

[0148] S2213: Obtain the second blank image to be filled, wherein the image height of the second blank image is the same as the image height of the raster image.

[0149] S2214: The raster image is resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the second blank image to obtain the aligned raster image.

[0150] In this embodiment, when aligning the raster image row by row according to the position of the raster grid in the region of interest in each pixel row, this application can first determine the maximum and minimum values ​​of the positions in all pixel rows according to the position of the raster grid in the region of interest in each pixel row, and determine the original image width of the raster image. In this way, the sampling interval for resampling the raster image can be determined according to the maximum and minimum values ​​of the positions in all pixel rows and the original image width. Then, this application can obtain a second blank image to be filled. The image height of the second blank image is the same as the image height of the raster image. In this way, the raster image can be resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the second blank image to obtain the aligned raster image. The image height of the aligned raster image is the same as the image height before alignment, but the image width is changed.

[0151] In one specific implementation, this application can extract the peak position on each pixel row of the region of interest in the raster image. r=1, ..., R, where R is the image height, i.e., the number of pixel rows. Then, the raster image is aligned row by row using the following mapping to obtain the aligned raster image, denoted as A:

[0152]

[0153] in, and These are the r-th row images of raster image I before alignment and raster image A after alignment, respectively. It can be selected as a linear mapping or a nonlinear mapping, which is equivalent to... In the interval Resampling is performed during this process. L is the original image width of the raster image, and A is the pixel width of A. This represents the maximum value at all row peak positions. Let be the minimum value at all row peak positions, and we have:

[0154] +

[0155]

[0156]

[0157] After the above mapping, we can obtain the following: Figure 11 The image shown is a magnified comparison of the raster images before and after alignment. Figure 11 The left side shows the local raster image before alignment, and the right side shows the local raster image after alignment. Figure 11 As can be seen, the misalignment between the brightest positions of the same grid in adjacent rows of the aligned raster image is small. By using this aligned raster image to perform row-by-row correction on the object surface image, a more accurate correction result can be obtained.

[0158] In one embodiment, step S222, aligning the object surface image row by row according to the position of the raster grid in the region of interest of the raster image in each pixel row to obtain the aligned object surface image, may include:

[0159] S2221: Obtain the third blank image to be filled, wherein the image height of the third blank image is the same as the image height of the object surface image.

[0160] S2222: The object surface image is resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the third blank image to obtain the aligned object surface image.

[0161] In this embodiment, when aligning the object surface image row by row according to the position of the raster grid in the region of interest in each pixel row, this application can first obtain the third blank image to be filled. The image height of the third blank image is the same as the image height of the object surface image. Then, this application can align the object surface image row by row using the above-described method of aligning the raster image to obtain the aligned object surface image, denoted as B.

[0162]

[0163] in, and These are the r-th row images of the object surface image G before alignment and the object surface image B after alignment, respectively. It can be selected as a linear mapping or a nonlinear mapping, which is equivalent to... In the interval Resampling is performed during the process, and the magnified comparison of the object surface images before and after alignment is shown in the following figure. Figure 12 As shown, Figure 12 The left side shows a magnified view of the area before alignment, and the right side shows a magnified view of the area after alignment. Figure 12 As can be seen, the aligned object image of this application is relatively clear, and some distortion has been eliminated.

[0164] In one embodiment, step-by-line correction of the aligned object surface image based on the aligned raster image in step S223 to obtain the target image may include:

[0165] S2231: Determine the column mean curve corresponding to the brightness value of each pixel row in each column of the aligned raster image, and obtain the third peak sequence by performing peak fitting on the column mean curve of each column.

[0166] S2232: Obtain the fourth blank image to be filled, wherein the image height of the fourth blank image is the same as the image height of the aligned object surface image.

[0167] S2233: Based on the position of the third peak sequence, the brightness values ​​of the corresponding pixel rows in the aligned object image are sampled in segments, and the sampled brightness values ​​are sequentially filled into the fourth blank image to obtain the target image.

[0168] In this embodiment, after obtaining the aligned raster image and the aligned object surface image, this application can perform line-by-line correction on the aligned object surface image based on the aligned raster image to obtain the target image.

[0169] Specifically, this application can first determine the column mean curve corresponding to the brightness value of each pixel row in each column of the aligned raster image, and then perform peak fitting on the column mean curve of each column to obtain the third peak sequence. Then, a fourth blank image to be filled is obtained. The image height of the fourth blank image is the same as the image height of the aligned object image. In this way, based on the position of the third peak sequence, the brightness value of the corresponding pixel row in the aligned object image can be sampled in segments, and the sampled brightness values ​​can be sequentially filled into the fourth blank image to obtain the target image.

[0170] For example, this application can first calculate the column mean curve corresponding to the brightness value of each pixel row in each column of the aligned raster image A. Then on Peak fitting was performed to obtain the third peak sequence. Let k = 1, ..., K, where K is the number of peaks. Next, this application can perform line-by-line correction on the aligned object surface image B to obtain the final target image D. During the correction process, a fourth blank image D to be filled can be prepared first, with its pixel height the same as the object surface image and its width W a pre-specified value, such as W = 1024. Then, the r-th row of B is extracted sequentially. = , r=1,…,R,c=1,…,L, Given the brightness value of image B in row r and column c, this application can finally perform the following mapping for... After segmentation, alignment, and sampling, fill the remaining space into the fourth blank image:

[0171]

[0172] in, It can be a linear mapping or a nonlinear mapping. = W / (K-1). This mapping is equivalent to... lie in The brightness curve segment is sampled linearly or nonlinearly, with the number of sampling points being [number missing]. Then, the sampled data is filled into the corresponding positions in the fourth blank image to obtain the final target image; the target image is as follows: Figure 13 As shown, compare it with Figure 10 A comparison with the target image shows that the effect is not much different from the target image obtained by using the full raster alignment correction algorithm. However, since this algorithm only needs to perform row-by-row peak detection on a region of interest in the raster image, it can greatly shorten the correction time and thus meet the real-time requirements.

[0173] In one embodiment, this application also provides a data acquisition card storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the distortion correction method as described in any of the above embodiments.

[0174] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0175] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0176] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A laser confocal scanning microscope, comprising a laser source, a resonant mirror, a galvanometer mirror, a beam splitter, an excitation optical path, a confocal optical path, and a first detector, characterized in that, The microscope further includes: a transflecting mirror fixedly installed between the resonant galvanometer mirror and the galvanometer mirror; a monitoring optical path and a second detector arranged in the transmission direction of the transflecting mirror; and a data acquisition card connected to the first detector and the second detector respectively. In this process, when the microscope scans the sample, the laser light emitted by the laser source is reflected by the beam splitter to the resonant mirror. The resonant mirror deflects the incident light beam for the first time and then emits it to the transmission mirror. The transmission mirror reflects most of the light beam to the galvanometer mirror. The galvanometer mirror deflects the incident light beam a second time and then focuses the deflected light beam onto the sample surface through the excitation optical path. The reflected light excited by the sample surface returns along the original path, is transmitted through the beam splitter to the confocal optical path, and reaches the first detector through the confocal optical path. The transmission mirror transmits a small portion of the light beam to the monitoring optical path, and the monitoring optical path converts the real-time deflection angle of the resonant mirror into a grating signal, which is then sent to the second detector. The acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, and generates an object surface image corresponding to the confocal signal and a grating image corresponding to the grating signal. Then, it uses the grating image to perform distortion correction on the object surface image to obtain the target image. The monitoring optical path includes an attenuator and a Ronchi grating; The attenuator attenuates the transmitted light beam before it is emitted to the Ronchi grating, and the Ronchi grating converts the real-time deflection angle of the resonant mirror into a grating signal based on the attenuated light beam.

2. A distortion correction method, applied to the acquisition card in the laser confocal scanning microscope of claim 1, characterized in that, The method includes: The confocal signal output by the first detector and the grating signal output by the second detector are acquired synchronously and isochronously, and an object surface image corresponding to the confocal signal and a grating image corresponding to the grating signal are generated. The object surface image is distorted using the raster image to obtain the target image.

3. The distortion correction method according to claim 2, characterized in that, The step of using the raster image to perform distortion correction on the object surface image to obtain the target image includes: Determine the position of at least one grid cell in the raster image within each pixel row; The object image is corrected row by row based on the position of the at least one grid in each pixel row to obtain the target image.

4. The distortion correction method according to claim 3, characterized in that, Determining the position of at least one raster in the raster image within each pixel row includes: If the current scene is a high-precision scene, then determine the position of all grids in the raster image in each pixel row; If the current scene is a real-time scene and the resonant galvanometer is scanned in one direction, then the position of the grid of the region of interest in the grating image in each pixel row is determined.

5. The distortion correction method according to claim 4, characterized in that, Determining the position of all grids in the raster image within each pixel row includes: Determine the row brightness curve corresponding to the column brightness value in each pixel row of the raster image; The row brightness curves of each pixel row are smoothed, and peak fitting is performed on the smoothed row brightness curves until the number of peaks in each pixel row is equal, thus obtaining the first peak sequence of each pixel row. The position of all gratings in each pixel row of the raster image is determined based on the first peak sequence of each pixel row.

6. The distortion correction method according to claim 5, characterized in that, The process of smoothing the row brightness curves of each pixel row and performing peak fitting on the smoothed row brightness curves until the number of peaks in each pixel row is equal, thus obtaining the first peak sequence of each pixel row, includes: Set the current value of the smoothing radius; Based on the current values, the row brightness curves of each pixel row are smoothed, and peak fitting is performed on the smoothed row brightness curves to obtain the preliminary peak position sequence of each pixel row. The initial peak position sequence of each pixel row is compared. If the number of peaks in at least one pixel row is not equal to the number of peaks in other pixel rows, the current value is updated, and the smoothing process of the row brightness curve of each pixel row based on the current value and subsequent steps are continued until the number of peaks in each pixel row is equal. If the number of peaks in each pixel row is equal, then the preliminary peak position sequence of each pixel row is taken as the final first peak sequence.

7. The distortion correction method according to claim 4, characterized in that, Determining the position of the raster grid in each pixel row of the region of interest in the raster image includes: The column mean brightness curve corresponding to the brightness value of each pixel row in each column of the raster image is determined, and the second peak sequence is obtained by performing peak fitting on the column mean brightness curve of each column. The region of interest in the raster image is cropped according to the second peak sequence, and the peak position of each pixel row is extracted from the cropped raster image as the position of the raster grid of the region of interest in each pixel row.

8. The distortion correction method according to claim 7, characterized in that, The step of cropping the region of interest in the raster image based on the second peak sequence includes: Determine the number of peaks in the second peak sequence; Based on the number of peaks in the second peak sequence, determine the start and end positions when cropping the raster image; The region of interest in the raster image is cropped based on the starting position and the ending position.

9. The distortion correction method according to claim 4, characterized in that, The step of performing line-by-line correction on the object surface image based on the position of the at least one raster in each pixel row to obtain the target image includes: Obtain a first blank image to be filled, wherein the image height of the first blank image is the same as the image height of the object surface image; Based on the position of all grids in the raster image in each pixel row, the brightness values ​​of the corresponding pixel rows in the object image are sampled in segments, and the sampled brightness values ​​are sequentially filled into the first blank image to obtain the target image.

10. The distortion correction method according to claim 4, characterized in that, The step of performing line-by-line correction on the object surface image based on the position of the at least one raster in each pixel row to obtain the target image includes: Based on the position of the raster grid in the region of interest in each pixel row, the raster image is aligned row by row to obtain the aligned raster image; Based on the position of the raster grid in the region of interest in the raster image in each pixel row, the object surface image is aligned row by row to obtain the aligned object surface image; The aligned object surface image is corrected line by line based on the aligned raster image to obtain the target image.

11. The distortion correction method according to claim 10, characterized in that, The step of aligning the raster image row by row according to the position of the raster grid in the region of interest in each pixel row to obtain the aligned raster image includes: Based on the position of the raster grid in the region of interest in the raster image in each pixel row, determine the maximum and minimum values ​​of the positions in all pixel rows, and determine the original image width of the raster image; The sampling interval for resampling the raster image is determined based on the maximum value, the minimum value, and the original image width; Obtain a second blank image to be filled, the image height of the second blank image being the same as the image height of the raster image; The raster image is resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the second blank image to obtain an aligned raster image.

12. The distortion correction method according to claim 11, characterized in that, The step of aligning the object surface image row by row according to the position of the raster grid in the region of interest in each pixel row to obtain the aligned object surface image includes: Obtain the third blank image to be filled, wherein the image height of the third blank image is the same as the image height of the object surface image; The object surface image is resampled within the sampling interval, and the resampled brightness values ​​are sequentially filled into the third blank image to obtain the aligned object surface image.

13. The distortion correction method according to claim 10, characterized in that, The step of performing line-by-line correction on the aligned object surface image based on the aligned raster image to obtain the target image includes: The column mean curve corresponding to the brightness value of each pixel row in each column of the aligned raster image is determined, and the third peak sequence is obtained by performing peak fitting on the column mean curve of each column. Obtain the fourth blank image to be filled, wherein the image height of the fourth blank image is the same as the image height of the aligned object surface image; Based on the position of the third peak sequence, the brightness values ​​of the corresponding pixel rows in the aligned object image are sampled in segments, and the sampled brightness values ​​are sequentially filled into the fourth blank image to obtain the target image.

14. A data acquisition card, characterized in that, The acquisition card stores computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the distortion correction method as described in any one of claims 2 to 13.

Images