Laser confocal scanning microscope, scanned image distortion correction method and acquisition card

Abstract

According to the laser confocal scanning microscope, the scanning image distortion correction method and the acquisition card provided by the invention, the transflective mirror is fixedly arranged between the resonance galvanometer and the galvanometer galvanometer, and the monitoring light path and the second detector are arranged in the transmission direction of the transflective mirror. The acquisition card is respectively connected with the first detector and the second detector; the confocal signal and the grating signal which are synchronously acquired by the acquisition card in an isochronous manner are twin images in time and space, and the grating image formed by the grating signal depicts the movement of the resonance galvanometer in the scanning process, so that distortion correction can be performed on the twin object plane image of the grating image through distortion analysis on the grating image, and the distortion correction accuracy is improved. Therefore, the correction accuracy is effectively improved, and the method is not easily influenced by factors such as environment, service life and optical path deviation. Moreover, in the distortion correction process, correction is carried out through the odd-numbered line sub-graphs and the even-numbered line sub-graphs, so that the distortion correction accuracy is further improved.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and particularly to a laser confocal scanning microscope, a scanning image distortion correction method, and an acquisition card. Background Art

[0002] As a high-precision and high-value-added measuring instrument, an industrial confocal microscope is a powerful tool for measuring the surface topography of an object and plays an important role in the research, production, and detection of materials such as ceramics, metals, and semiconductors. At present, various scanning imaging methods have been developed for confocal laser scanning, including galvanometer / resonance-based dual galvanometer scanning, MEMS (Micro-Electro-Mechanical Systems) galvanometer scanning, Nipkow disk scanning, and line scanning. Among them, the galvanometer / resonance-based dual galvanometer scanning method has become the mainstream due to its cost and technological maturity. Dual galvanometer scanning includes two scanning methods: unidirectional scanning and bidirectional scanning. In unidirectional scanning, the galvanometer mirror on the Y-axis takes a step immediately when the resonance mirror on the X-axis moves to an end point (such as the left end point). After the step is completed, the X-axis starts the next line of scanning, and the above process is repeated. In bidirectional scanning, the galvanometer mirror takes a step when the resonance mirror moves to both end points, thereby realizing fast scanning of the sample.

[0003] Theoretically, resonance scanning is designed to perform simple harmonic motion at an inherent resonant frequency, and its angular velocity is the largest at the central position and zero at both ends. This causes distortion in the images acquired using isochronous sampling. Moreover, due to factors such as possible mechanical motion errors and electrical signal delays, the image signals acquired during bidirectional scanning will also produce aliasing in the horizontal direction, resulting in a greater degree of distortion in the finally obtained images and lower image quality. Summary of the Invention

[0004] The purpose of the present application is to at least solve one of the above technical defects, especially the technical defect that when the sample is scanned using the galvanometer / resonance-based dual galvanometer scanning method in the prior art, affected by factors such as mechanical motion errors and electrical signal delays, the scanned images are distorted, thereby affecting the image quality.

[0005] The present application provides a laser confocal scanning microscope, including a laser light source, a resonance mirror, a galvanometer mirror, a beam splitter, an excitation optical path, a confocal optical path, and a first detector. The microscope further includes: a dichroic mirror fixedly installed between the resonance mirror and the galvanometer mirror, a monitoring optical path and a second detector arranged in the transmission direction of the dichroic mirror, and an acquisition card respectively connected to the first detector and the second detector;

[0006] Among them, when the microscope scans a sample, the beam emitted by the laser light source is reflected by the beam splitter to the resonant galvanometer. After the resonant galvanometer makes a first deflection of the incident beam, it emits the beam to the dichroic mirror. The dichroic mirror reflects most of the beam to the galvanometer scanner. After the galvanometer scanner makes a second deflection of the incident beam, the deflected beam is focused onto the surface of the sample through the excitation optical path. The reflected light excited on the surface of the sample returns along the original path, is transmitted by the beam splitter to the confocal optical path, and reaches the first detector through the confocal optical path. The dichroic mirror transmits a small part of the beam to the monitoring optical path, and after the monitoring optical path converts the real-time deflection angle of the resonant galvanometer into a grating signal, it sends the signal to the second detector;

[0007] The acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, generates a surface image corresponding to the confocal signal and a grating image corresponding to the grating signal, then decomposes the surface image into odd-row surface sub-images and even-row surface sub-images, and decomposes the grating image into odd-row grating sub-images and even-row grating sub-images. Then, it corrects the distortion of the odd-row surface sub-images using the odd-row grating sub-images, corrects the distortion of the even-row surface sub-images using the even-row grating sub-images, and merges the corrected odd-row surface sub-images and the corrected even-row surface sub-images to obtain the target image.

[0008] Optionally, the monitoring optical path includes an attenuator and a Ronchi grating;

[0009] The attenuator attenuates the transmitted beam and then emits it to the Ronchi grating, and the Ronchi grating converts the real-time deflection angle of the resonant galvanometer into a grating signal according to the attenuated beam.

[0010] This application also provides a method for correcting the distortion of a scanned image, which is applied to the acquisition card in the laser confocal scanning microscope described in any one of the above embodiments. The method includes:

[0011] Synchronously and isochronously acquire the confocal signal output by the first detector and the grating signal output by the second detector, and generate a surface image corresponding to the confocal signal and a grating image corresponding to the grating signal;

[0012] Decompose the surface image into odd-row surface sub-images and even-row surface sub-images, and decompose the grating image into odd-row grating sub-images and even-row grating sub-images;

[0013] Correct the distortion of the odd-row surface sub-images using the odd-row grating sub-images, and correct the distortion of the even-row surface sub-images using the even-row grating sub-images;

[0014] Merge the corrected odd - row object - surface sub - images and the corrected even - row object - surface sub - images to obtain the target image.

[0015] Optionally, before using the odd - row grating sub - image to correct the distortion of the odd - row object - surface sub - image and using the even - row grating sub - image to correct the distortion of the even - row object - surface sub - image, it further includes:

[0016] When it is detected that the resonant galvanometer uses bidirectional scanning, align the grid sequences of the odd - row grating sub - image and the even - row grating sub - image, and use the aligned odd - row grating sub - image to correct the distortion of the odd - row object - surface sub - image, and use the aligned even - row grating sub - image to correct the distortion of the even - row object - surface sub - image.

[0017] Optionally, the step of aligning the grid sequences of the odd - row grating sub - image and the even - row grating sub - image to obtain the aligned odd - row grating sub - image and even - row grating sub - image includes:

[0018] Perform peak fitting on each column of the odd - row grating sub - image to obtain a first peak sequence, and determine a first interval sequence according to the intervals between adjacent peaks in the first peak sequence;

[0019] Perform peak fitting on each column of the even - row grating sub - image to obtain a second peak sequence, and determine a second interval sequence according to the intervals between adjacent peaks in the second peak sequence;

[0020] According to the first interval sequence and the second interval sequence, extract a first subsequence overlapping with the second peak sequence from the first peak sequence, and extract a second subsequence overlapping with the first peak sequence from the second peak sequence;

[0021] Extract the grids located in the interval determined by the head and tail of the first subsequence from the odd - row grating sub - image to form the aligned odd - row grating sub - image, and extract the grids located in the interval determined by the head and tail of the second subsequence from the even - row grating sub - image to form the aligned even - row grating sub - image.

[0022] Optionally, the step of extracting a first subsequence overlapping with the second peak sequence from the first peak sequence and extracting a second subsequence overlapping with the first peak sequence from the second peak sequence according to the first interval sequence and the second interval sequence includes:

[0023] Determine the overlapping subsequences with a similarity exceeding a preset similarity threshold in the first interval sequence and the second interval sequence;

[0024] Based on the overlapping subsequences, a first subsequence overlapping with the second peak sequence is extracted from the first peak sequence, and a second subsequence overlapping with the first peak sequence is extracted from the second peak sequence.

[0025] Optionally, the using the odd-row raster sub-image to perform distortion correction on the odd-row object surface sub-image to obtain a target image includes:

[0026] Determine the positions of at least one grid in each pixel row of the odd-row raster sub-image;

[0027] Perform row-by-row correction on the odd-row object surface sub-image according to the positions of the at least one grid in each pixel row to obtain a target image.

[0028] Optionally, the determining the positions of at least one grid in each pixel row of the odd-row raster sub-image includes:

[0029] If the current scenario is a high-precision scenario, determine the positions of all grids in each pixel row of the odd-row raster sub-image;

[0030] If the current scenario is a real-time scenario and the resonant galvanometer scans unidirectionally, determine the positions of the grids in the region of interest in each pixel row of the odd-row raster sub-image.

[0031] Optionally, the determining the positions of all grids in each pixel row of the odd-row raster sub-image includes:

[0032] Determine the row brightness curve corresponding to the brightness values of each column in each pixel row of the odd-row raster sub-image;

[0033] Smooth the row brightness curves of each pixel row, and perform peak fitting on the smoothed row brightness curves until the number of peaks in each pixel row is equal, to obtain the third peak sequence of each pixel row;

[0034] Determine the positions of all grids in each pixel row of the odd-row raster sub-image according to the third peak sequence of each pixel row.

[0035] Optionally, the smoothing the row brightness curves of each pixel row, and performing peak fitting on the smoothed row brightness curves until the number of peaks in each pixel row is equal, to obtain the third peak sequence of each pixel row includes:

[0036] Set the current value of the smoothing radius;

[0037] Based on the current value, smooth the row brightness curves of each pixel row, and perform peak fitting on the smoothed row brightness curves to obtain the preliminary peak position sequence of each pixel row;

[0038] Compare the preliminary peak position sequences of each pixel row. If the number of peaks in at least one pixel row is not equal to the number of peaks in other pixel rows, then after updating the current value, continue to execute the steps of smoothing the row brightness curves of each pixel row based on the current value and subsequent steps until the number of peaks in each pixel row is equal;

[0039] If the number of peaks in each pixel row is equal, then use the preliminary peak position sequences of each pixel row as the final third peak sequence.

[0040] Optionally, determining the positions of the grids in the region of interest in the odd-row raster sub-image in each pixel row includes:

[0041] Determine the column mean brightness curves corresponding to the brightness values of each pixel row in each column of the odd-row raster sub-image, and after performing peak fitting on the column mean brightness curves of each column, obtain a fourth peak sequence;

[0042] Crop the region of interest in the odd-row raster sub-image according to the fourth peak sequence, and extract the peak positions of each pixel row from the cropped odd-row raster sub-image as the positions of the grids in the region of interest in the odd-row raster sub-image in each pixel row.

[0043] Optionally, the cropping the region of interest in the odd-row raster sub-image according to the fourth peak sequence includes:

[0044] Determine the number of peaks in the fourth peak sequence;

[0045] According to the number of peaks in the fourth peak sequence, determine the starting position and the ending position for cropping the odd-row raster sub-image;

[0046] Crop the region of interest in the odd-row raster sub-image according to the starting position and the ending position.

[0047] Optionally, the performing row-by-row correction on the odd-row object surface sub-image according to the positions of the at least one grid in each pixel row to obtain a corrected odd-row object surface sub-image includes:

[0048] Obtain a first blank image to be filled, where the image height of the first blank image is the same as the image height of the odd-row object surface sub-image;

[0049] Based on the positions of all the grids in the odd-row raster sub-image in each pixel row, perform segmented sampling on the brightness values of the corresponding pixel rows in the odd-row object surface sub-image, and sequentially fill the sampled brightness values into the first blank image to obtain a corrected odd-row object surface sub-image.

[0050] Optionally, performing line-by-line correction on the odd-row object sub-images according to the positions of the at least one grid in each pixel row to obtain corrected odd-row object sub-images, including:

[0051] Performing line-by-line alignment on the odd-row raster sub-images according to the positions of the grids in the region of interest in the odd-row raster sub-images in each pixel row to obtain aligned odd-row raster sub-images;

[0052] Performing line-by-line alignment on the odd-row object sub-images according to the positions of the grids in the region of interest in the odd-row raster sub-images in each pixel row to obtain aligned odd-row object sub-images;

[0053] Performing line-by-line correction on the aligned odd-row object sub-images according to the aligned odd-row raster sub-images to obtain corrected odd-row object sub-images.

[0054] Optionally, performing line-by-line alignment on the odd-row raster sub-images according to the positions of the grids in the region of interest in the odd-row raster sub-images in each pixel row to obtain aligned odd-row raster sub-images, including:

[0055] Determining the maximum and minimum values of the positions in all pixel rows and determining the original image width of the odd-row raster sub-images according to the positions of the grids in the region of interest in the odd-row raster sub-images in each pixel row;

[0056] Determining a resampling interval for resampling the odd-row raster sub-images according to the maximum value, the minimum value, and the original image width;

[0057] Obtaining a second blank image to be filled, where the image height of the second blank image is the same as the image height of the odd-row raster sub-images;

[0058] Performing resampling on the odd-row raster sub-images within the resampling interval and sequentially filling the resampled luminance values into the second blank image to obtain aligned odd-row raster sub-images.

[0059] Optionally, performing line-by-line alignment on the odd-row object sub-images according to the positions of the grids in the region of interest in the odd-row raster sub-images in each pixel row to obtain aligned odd-row object sub-images, including:

[0060] Obtaining a third blank image to be filled, where the image height of the third blank image is the same as the image height of the odd-row object sub-images;

[0061] Resample the odd-row object surface sub-image within the sampling interval, and fill the brightness values after resampling into the third blank image in sequence to obtain the aligned odd-row object surface sub-image.

[0062] Optionally, the step of performing line-by-line correction on the aligned odd-row object surface sub-image according to the aligned odd-row grating sub-image to obtain the corrected odd-row object surface sub-image includes:

[0063] Determine the column mean curve corresponding to the brightness values of each pixel row in each column of the aligned odd-row grating sub-image, and perform peak fitting on the column mean curves of each column to obtain a plurality of fifth peak sequences;

[0064] Obtain a fourth blank image to be filled, where the image height of the fourth blank image is the same as the image height of the aligned odd-row object surface sub-image;

[0065] Based on the positions of each fifth peak sequence, perform segmented sampling on the brightness values of the corresponding pixel rows in the aligned odd-row object surface sub-image, and fill the sampled brightness values into the fourth blank image in sequence to obtain the corrected odd-row object surface sub-image.

[0066] This application also provides an acquisition card, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the scanning image distortion correction method described in any one of the above embodiments.

[0067] It can be seen from the above technical solutions that the embodiments of this application have the following advantages:

[0068] The provided laser confocal scanning microscope, scanning image distortion correction method and acquisition card of the present application. The laser confocal scanning microscope not only includes a laser light source, a resonant galvanometer, a galvanometer scanner, a beam splitter, an excitation optical path, a confocal optical path and a first detector, but also includes a dichroic mirror fixedly installed between the resonant galvanometer and the galvanometer scanner, a monitoring optical path and a second detector arranged in the transmission direction of the dichroic mirror, and an acquisition card respectively connected to the first detector and the second detector; wherein, when the microscope of the present application scans a sample, the beam emitted by the laser light source is reflected by the beam splitter to the resonant galvanometer, the resonant galvanometer first deflects the incident beam and then emits it to the dichroic mirror, the dichroic mirror reflects most of the beam to the galvanometer scanner, the galvanometer scanner second deflects the incident beam, and then focuses the deflected beam to the surface of the sample through the excitation optical path. The reflected light excited by the surface of the sample returns along the original path, is transmitted by the beam splitter to the confocal optical path, and reaches the first detector through the confocal optical path. The dichroic mirror transmits a small part of the beam to the monitoring optical path, and the monitoring optical path converts the real-time deflection angle of the resonant galvanometer into a grating signal and then sends it to the second detector; the acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, generates an object plane image corresponding to the confocal signal and a grating image corresponding to the grating signal, then decomposes the object plane image into odd-row object plane sub-images and even-row object plane sub-images, and decomposes the grating image into odd-row grating sub-images and even-row grating sub-images, and uses the odd-row grating sub-images to correct the distortion of the odd-row object plane sub-images, and at the same time uses the even-row grating sub-images to correct the distortion of the even-row object plane sub-images, and merges the corrected odd-row object plane sub-images and the corrected even-row object plane sub-images to obtain the target image. In this process, the confocal signal and the grating signal synchronously and isochronously acquired by the acquisition card are twin images in terms of time and space. Among them, the grating image formed by the grating signal depicts the movement of the resonant galvanometer during the scanning process. Therefore, the present application can correct the distortion of the object plane image twin to it by analyzing the distortion of the grating image, thereby effectively improving the correction accuracy and being not easily affected by factors such as the environment, lifespan and optical path deviation. And, the present application fully considers the influencing factors such as the possible delay error generated during the movement of the galvanometer and the data acquisition process. During the distortion correction process, both the grating image and the object plane image are decomposed into corresponding odd-row sub-images and even-row sub-images, and the odd-row sub-images and the even-row sub-images are respectively used for correction to further improve the accuracy of distortion correction. Brief Description of the Drawings

[0069] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0070] Figure 1 Structural schematic diagram of a laser confocal scanning microscope provided by an embodiment of the present application;

[0071] Figure 2 Optical path display diagram of splitting light using a transmissive-reflective lens provided by an embodiment of the present application;

[0072] Figure 3 Display diagram of a grating image during single-direction scanning provided by an embodiment of the present application;

[0073] Figure 4 Display diagram of a grating image during bidirectional scanning provided by an embodiment of the present application;

[0074] Figure 5 Display diagram of an object surface image during single-direction scanning provided by an embodiment of the present application;

[0075] Figure 6 Display diagram of an object surface image during bidirectional scanning provided by an embodiment of the present application;

[0076] Figure 7 Flow schematic diagram of a scanning image distortion correction method provided by an embodiment of the present application;

[0077] Figure 8 Display diagram when there is a deviation in the brightest position between adjacent two rows of the same grating provided by an embodiment of the present application;

[0078] Figure 9 Schematic diagram of the row brightness curve corresponding to a pixel row in the grating image provided by an embodiment of the present application;

[0079] Figure 10 Schematic diagram of the peak fitting result of one of the pixel rows provided by an embodiment of the present application;

[0080] Figure 11 Display diagram of the region of interest at the center of the odd-row grating sub-image provided by an embodiment of the present application;

[0081] Figure 12 Display diagram of the image after distortion correction of the odd-row object surface sub-image using the full-grating alignment distortion correction algorithm provided by an embodiment of the present application;

[0082] Figure 13This is a partially enlarged comparison diagram of the odd - numbered row raster sub - images before and after alignment provided by the embodiments of the present application;

[0083] Figure 14 This is a partially enlarged comparison diagram of the odd - numbered row object - surface sub - images before and after alignment provided by the embodiments of the present application;

[0084] Figure 15 This is an image display diagram of the odd - numbered row object - surface sub - image after distortion correction using the local raster alignment correction algorithm provided by the embodiments of the present application. Detailed implementation manners

[0085] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0086] In one embodiment, as Figure 1 shown, Figure 1 This is a schematic structural diagram of a laser confocal scanning microscope provided by the embodiments of the present application; The present application provides a laser confocal scanning microscope, including a laser light source, a resonant galvanometer mirror, a galvanometer mirror, a beam splitter, an excitation optical path, a confocal optical path, and a first detector. The microscope further includes: a dichroic mirror fixedly installed between the resonant galvanometer mirror and the galvanometer mirror, a monitoring optical path and a second detector arranged in the transmission direction of the dichroic mirror, and an acquisition card respectively connected to the first detector and the second detector.

[0087] Among them, when the microscope scans a sample, the beam emitted by the laser light source is reflected by the beam splitter to the resonant galvanometer mirror. The resonant galvanometer mirror deflects the incident beam for the first time and then emits it to the dichroic mirror. The dichroic mirror reflects most of the beam to the galvanometer mirror. After the galvanometer mirror deflects the incident beam for the second time, the deflected beam is focused onto the surface of the sample through the excitation optical path. The reflected light excited by the sample surface returns along the original path, is transmitted by the beam splitter to the confocal optical path, and reaches the first detector through the confocal optical path. The dichroic mirror transmits a small part of the beam to the monitoring optical path. After the monitoring optical path converts the real - time deflection angle of the resonant galvanometer mirror into a grating signal, it is sent to the second detector.

[0088] The acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, generates a surface image corresponding to the confocal signal and a grating image corresponding to the grating signal, then decomposes the surface image into an odd-row surface sub-image and an even-row surface sub-image, and decomposes the grating image into an odd-row grating sub-image and an even-row grating sub-image. After that, the odd-row grating sub-image is used to correct the distortion of the odd-row surface sub-image, the even-row grating sub-image is used to correct the distortion of the even-row surface sub-image, and the corrected odd-row surface sub-image and the corrected even-row surface sub-image are merged to obtain a target image.

[0089] In this embodiment, as Figure 1 shown, in the present application, an acquisition optical path for the movement process of the resonant galvanometer is added to the original laser confocal scanning microscope equipped with a two-dimensional galvanometer. The acquisition optical path is composed of a dichroic mirror arranged between the resonant galvanometer and the galvanometer, a monitoring optical path and a second detector arranged in the transmission direction of the dichroic mirror, and an acquisition card respectively connected to the first detector and the second detector. Since the dichroic mirror can selectively transmit or reflect light according to the wavelength of the light, therefore, in the present application, most of the excitation light can be reflected by the dichroic mirror to the sample, and a small part of the excitation light is transmitted for monitoring the movement of the resonant galvanometer. In this way, the non-linear movement (such as a sine wave form) of the resonant galvanometer can be recorded in real time through the monitoring optical path and the second detector. And the acquisition card of the present application can record the two signals with the same time stamp, which can not only ensure pixel-level alignment, but also use the galvanometer movement data collected by the second detector to perform pixel position remapping on the surface image collected by the first detector, thereby eliminating the sine scanning distortion.

[0090] Among them, the dichroic mirror of the present application is fixedly installed on the optical path between the resonant galvanometer and the galvanometer, which can not only ensure the constancy of the beam splitting ratio and direction, but also simplify the system calibration. And the present application can also set the beam splitting ratio of the dichroic mirror. For example, the present application can use 95% of the energy of the beam received by the dichroic mirror for sample scanning and 5% of the energy for feedback of the galvanometer state. The specific ratio can be set according to the actual situation and is not limited here. In this way, while ensuring a high-resolution surface image, the scanning distortion of the surface image can be eliminated by the grating image collected by the second detector.

[0091] Furthermore, the laser light source of the present application can provide monochromatic and high-brightness excitation light (such as Argon ion laser, semiconductor laser, etc.); the resonant galvanometer of the present application can adopt a high-frequency resonant galvanometer (such as 4 - 12 kHz), and the galvanometer mirror can adopt a low-speed galvanometer mirror (such as 1 - 30 Hz); the excitation optical path of the present application can include a scanning lens and an objective lens, and the scanning lens and the objective lens can focus the beam deflected by the galvanometer onto the sample surface and collect the reflected / fluorescent signals; the confocal optical path of the present application is located in front of the first detector and is used to block the out-of-focus light signals and improve the axial resolution; the first detector and the second detector of the present application can use a photomultiplier tube (PMT) or an avalanche photodiode (APD) to convert the optical signal into an electrical signal.

[0092] For example, when the present application performs one-way or two-way scanning on the sample, the resonant galvanometer oscillates at a high frequency in the form of a sine wave, driving the beam to quickly scan back and forth on the sample surface. Only the linear interval of the sine wave (such as near the peak or trough) is used to collect signals during this process. During one-way scanning, the laser is usually turned off or the data is discarded during the retrace stage (non-linear interval), while during two-way scanning, the laser beam performs effective processing in both forward and reverse directions. During this process, the galvanometer mirror steps in the form of a sawtooth wave and moves at least one step size (corresponding to one row of pixels in the image) after each row of X-axis scanning is completed, and quickly resets to the starting position at the end of the frame. After the laser beam is deflected by the two-dimensional galvanometer and reflected by the dichroic mirror, it is focused onto the sample surface through the scanning lens and the objective lens. The fluorescence or reflected light excited by the sample returns along the original path, and after being separated by the dichroic mirror, it reaches the first detector through the confocal optical path. The first detector converts the optical signal into an electrical signal and synchronizes with the position of the galvanometer to construct a two-dimensional image pixel by pixel, that is, the object plane image in the present application. In addition, the dichroic mirror of the present application can also transmit a small part of the laser beam, and this part of the laser beam is collected by the monitoring optical path to convert the real-time deflection angle of the resonant galvanometer into a grating signal, and the monitoring optical path is connected to the second detector. In this way, the grating signal can be sent to the acquisition card through the second detector, and after the acquisition card processes the grating signal, the grating image can be obtained. Then, the present application can use the grating image to correct the distortion of the object plane image, and in this way, the target image can be obtained. For example, when the present application determines the positions of each grid in the grating image (the positions of the local brightness maxima), the distortion level between two adjacent positions can be determined, and then the distortion correction can be completed according to this distortion level.

[0093] Furthermore, during the process of using the grating image to correct the distortion of the object surface image in this application, it is particularly noted that when the user selects the bidirectional scanning method for the scanning operation, the galvanometer mirror of the Y-axis takes one step at both key endpoints (such as the starting end and the return end) of the resonant mirror of the X-axis. Based on this phenomenon, we can clearly observe that all the processes of data acquisition starting from the starting end ultimately form the odd-numbered row part of the image without exception; correspondingly, those data acquisition activities starting from the return end logically form the even-numbered row part of the image. In view of this, in order to more precisely correct the image distortion and fully consider the influencing factors such as the possible delay errors during the movement of the mirror and in the data acquisition process, this application proposes an innovative processing method: First, this application can decompose the object surface image into two parts, namely the odd-numbered row object surface sub-image and the even-numbered row object surface sub-image; similarly, this application also correspondingly decomposes the grating image used for correction into the odd-numbered row grating sub-image and the even-numbered row grating sub-image. On this basis, the odd-numbered row grating sub-image is respectively used to perform targeted distortion correction on the odd-numbered row object surface sub-image, and at the same time, the even-numbered row grating sub-image is used to perform corresponding distortion correction on the even-numbered row object surface sub-image. Finally, the accurately corrected odd-numbered row object surface sub-image and the even-numbered row object surface sub-image are seamlessly merged. Through this series of processing steps, a target image with effectively corrected distortion and significantly improved quality is finally generated.

[0094] In the above embodiments, the laser confocal scanning microscope not only includes a laser light source, a resonant galvanometer, a galvanometer scanner, a beam splitter, an excitation optical path, a confocal optical path, and a first detector, but also includes a dichroic mirror fixedly installed between the resonant galvanometer and the galvanometer scanner, a monitoring optical path and a second detector arranged in the transmission direction of the dichroic mirror, and an acquisition card respectively connected to the first detector and the second detector; wherein, when the microscope of the present application scans a sample, the beam emitted by the laser light source is reflected by the beam splitter to the resonant galvanometer, the resonant galvanometer deflects the incident beam for the first time and then emits it to the dichroic mirror, the dichroic mirror reflects most of the beam to the galvanometer scanner, after the galvanometer scanner deflects the incident beam for the second time, the deflected beam is focused on the surface of the sample through the excitation optical path, the reflected light excited by the surface of the sample returns along the original path, is transmitted by the beam splitter to the confocal optical path, and reaches the first detector through the confocal optical path, the dichroic mirror transmits a small part of the beam to the monitoring optical path, and after the monitoring optical path converts the real-time deflection angle of the resonant galvanometer into a grating signal, it is sent to the second detector; the acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, generates a surface image corresponding to the confocal signal and a grating image corresponding to the grating signal, then decomposes the surface image into an odd-row surface sub-image and an even-row surface sub-image, and decomposes the grating image into an odd-row grating sub-image and an even-row grating sub-image, and uses the odd-row grating sub-image to correct the distortion of the odd-row surface sub-image, and at the same time uses the even-row grating sub-image to correct the distortion of the even-row surface sub-image, and merges the corrected odd-row surface sub-image and the corrected even-row surface sub-image, then the target image can be obtained. In this process, the confocal signal and the grating signal synchronously and isochronously acquired by the acquisition card are twin images in time and space, and the grating image formed by the grating signal depicts the movement of the resonant galvanometer during the scanning process. Therefore, in the present application, through the distortion analysis of the grating image, the distortion correction of the twin surface image can be performed, thereby effectively improving the correction accuracy and being not easily affected by factors such as the environment, lifespan, and optical path deviation. Moreover, the present application fully considers the influencing factors such as the possible delay error generated during the movement of the galvanometer and the data acquisition process. During the distortion correction process, both the grating image and the surface image are decomposed into corresponding odd-row sub-images and even-row sub-images, and the odd-row sub-images and the even-row sub-images are respectively used for correction, so as to further improve the accuracy of the distortion correction.

[0095] In one embodiment, as Figure 2 shown, Figure 2 is an optical path display diagram for splitting light by using a dichroic mirror provided by an embodiment of the present application; the monitoring optical path may include an attenuation sheet and a Ronchi grating.

[0096] The attenuation sheet attenuates the transmitted light beam and then emits it to the Ronchi grating, and the Ronchi grating converts the real-time deflection angle of the resonant galvanometer into a grating signal according to the attenuated light beam.

[0097] In this embodiment, the monitoring optical path arranged in the transmission direction of the transmissive and reflective mirror may include an attenuation sheet and a Ronchi grating. Among them, the attenuation sheet can attenuate the transmitted light beam and then emit it to the Ronchi grating to avoid damage to the second detector by high-power laser (the transmitted light beam). The Ronchi grating can convert the real-time deflection angle of the resonant galvanometer into a grating signal according to the attenuated light beam. For example, the Ronchi grating of the present application can use the deflection of the resonant galvanometer to move the light beam on the grating, thereby generating a periodic light intensity signal, which is the grating signal of the present application. This grating signal directly maps the real-time deflection angle of the resonant galvanometer. After being processed by the acquisition card, the grating image generated during one-way scanning as shown in Figure 3 can be obtained. The horizontal direction of the image of this grating image corresponds to the movement direction of the resonant galvanometer. Each image row corresponds to a movement period of the resonant galvanometer. The distance between the grids reflects the movement of the resonant galvanometer, that is, the distortion level at each position. Or the grating image generated during two-way scanning as shown in Figure 4 can also be obtained. Since the galvanometer mirror on the Y-axis steps one step at both key endpoints (such as the starting end and the return end) of the resonant galvanometer on the X-axis during two-way scanning, the grating signals (grids) of the forward journey and the return journey overlap or stick together at multiple positions. The present application can process this through subsequent distortion correction to obtain a target image with higher image quality.

[0098] At the same time, the acquisition card of the present application can also process the collected confocal signal, which reflects the fluorescence / reflection light intensity distribution of the sample and is spatio-temporally aligned with the grating signal. Therefore, the present application can generate the object surface image during one-way scanning as shown in Figure 5 after image processing of the confocal signal, or the object surface image during two-way scanning as shown in Figure 6 and then use the galvanometer movement data reflected by the grating image to perform distortion correction on the object surface image, thereby effectively improving the accuracy of distortion correction while avoiding the influence of factors such as the environment, lifespan, and optical path deviation.

[0099] In one embodiment, as shown in Figure 7 is a schematic flowchart of a method for correcting the distortion of a scanned image provided by an embodiment of the present application; the present application also provides a method for correcting the distortion of a scanned image, which is applied to the acquisition card in the laser confocal scanning microscope described in any one of the above embodiments. The method may include: Figure 7

[0100] ​S110: Synchronously and isochronously collect the confocal signal output by the first detector and the grating signal output by the second detector, and generate a surface image corresponding to the confocal signal and a grating image corresponding to the grating signal;

[0101] S120: Decompose the surface image into odd-row surface sub-images and even-row surface sub-images, and decompose the grating image into odd-row grating sub-images and even-row grating sub-images;

[0102] S130: Use the odd-row grating sub-images to correct the distortion of the odd-row surface sub-images, and use the even-row grating sub-images to correct the distortion of the even-row surface sub-images;

[0103] S140: Merge the corrected odd-row surface sub-images and the corrected even-row surface sub-images to obtain the target image.

[0104] In this embodiment, as Figure 1 shown, in the present application, an acquisition optical path for the movement process of the resonant galvanometer is added to the original laser confocal scanning microscope equipped with a two-dimensional galvanometer. This acquisition optical path is composed of a dichroic mirror arranged between the resonant galvanometer and the galvanometer, a monitoring optical path and a second detector arranged in the transmission direction of the dichroic mirror, and an acquisition card respectively connected to the first detector and the second detector. Since the dichroic mirror can selectively transmit or reflect light according to the wavelength of the light, therefore, in the present application, most of the excitation light can be reflected by the dichroic mirror to the sample, and a small part of the excitation light is transmitted for monitoring the movement of the resonant galvanometer, so that the non-linear movement (such as a sine wave form) of the resonant galvanometer can be recorded in real time through the monitoring optical path and the second detector. And the acquisition card of the present application can record the two signals with the same time stamp, which can not only ensure pixel-level alignment, but also use the galvanometer movement data collected by the second detector to perform pixel position remapping on the surface image collected by the first detector, thereby eliminating the sine scan distortion.

[0105] In a specific implementation manner, when the present application performs single - direction or two - direction scanning on a sample, the resonant galvanometer oscillates at a high frequency in the form of a sine wave, driving the light beam to scan rapidly back and forth on the sample surface. During this process, signals are collected only using the linear interval of the sine wave (such as near the wave peak or wave trough). During single - direction scanning, the laser is usually turned off or data is discarded during the return scan stage (non - linear interval), while during two - direction scanning, the laser beam performs effective processing in both forward and reverse directions. During this process, the galvanometer mirror steps in the form of a sawtooth wave, moving at least one step (corresponding to one row of pixels in the image) after each row of X - axis scanning is completed, and quickly resetting to the starting position at the end of the frame. After the laser beam is deflected by the two - dimensional mirror and reflected by the beam splitter - reflector, it is focused on the sample surface through the scanning lens and the objective lens. The fluorescence or reflected light excited by the sample returns along the original path, is separated by the beam splitter, and reaches the first detector through the confocal optical path. The first detector converts the optical signal into an electrical signal and synchronizes with the position of the mirror to construct a two - dimensional image pixel by pixel, that is, the object - plane image in the present application. In addition, a small part of the laser beam can also be transmitted by the beam splitter - reflector of the present application. This part of the laser beam is collected through the monitoring optical path to convert the real - time deflection angle of the resonant galvanometer into a grating signal, and the monitoring optical path is connected to the second detector. In this way, the grating signal can be sent to the acquisition card through the second detector. After the acquisition card performs image processing on the grating signal, a grating image can be obtained. Then, the present application can use the grating image to perform distortion correction on the object - plane image, and thus a target image can be obtained. For example, when the present application determines the positions of each grid in the grating image (the positions where the local brightness maxima are located), the distortion level between two adjacent positions can be determined, and then the distortion correction can be completed according to this distortion level.

[0106] Furthermore, during the process of using the grating image to correct the distortion of the object surface image in this application, it is particularly noted that when the user selects the two-way scanning method for scanning operations, the galvanometer mirror in the Y-axis takes a step at both key endpoints (such as the starting end and the return end) of the resonant mirror in the X-axis. Based on this phenomenon, it can be clearly observed that all the processes of data acquisition starting from the starting end ultimately form the odd-numbered row part of the image without exception; correspondingly, those data acquisition activities starting from the return end logically form the even-numbered row part of the image. In view of this, in order to more precisely correct the image distortion and fully consider the influencing factors such as the possible delay errors during the movement of the mirror and in the data acquisition process, this application proposes an innovative processing method: First, this application can decompose the object surface image into two parts, namely the odd-numbered row object surface sub-image and the even-numbered row object surface sub-image; similarly, this application also correspondingly decomposes the grating image used for correction into the odd-numbered row grating sub-image and the even-numbered row grating sub-image. On this basis, the odd-numbered row grating sub-image is used to specifically correct the distortion of the odd-numbered row object surface sub-image, and at the same time, the even-numbered row grating sub-image is used to correct the distortion of the even-numbered row object surface sub-image accordingly. Finally, the accurately corrected odd-numbered row object surface sub-image and the even-numbered row object surface sub-image are seamlessly merged. Through this series of processing steps, a target image with effectively corrected distortion and significantly improved quality is ultimately generated.

[0107] Among them, when this application decomposes the object surface image into the odd-numbered row object surface sub-image and the even-numbered row object surface sub-image, and decomposes the grating image into the odd-numbered row grating sub-image and the even-numbered row grating sub-image, it can be decomposed according to the traditional odd-even decomposition method, or an image processing algorithm, such as image segmentation technology, can be used to accurately decompose the image into odd-numbered and even-numbered row sub-images. This process ensures that each row of data can match its corresponding mirror movement state, thereby improving the accuracy of distortion correction. At the same time, during the process of decomposing the image, this application can also preprocess the image, such as denoising and enhancing contrast, etc., to improve the subsequent distortion correction effect. When using the odd-numbered row grating sub-image and the even-numbered row grating sub-image to correct the distortion of the corresponding object surface sub-images, this application can adopt advanced image registration and distortion correction algorithms. These algorithms can accurately calculate the position offset of each pixel point according to the grid distribution and distortion situation in the grating image, and adjust the pixel positions in the object surface sub-images accordingly. This process can not only eliminate the image distortion caused by the non-linear movement of the mirror, but also reduce the impact of the delay error existing in the data acquisition process on the image quality. After completing the distortion correction, this application can also seamlessly merge the accurately corrected odd-numbered row object surface sub-image and the even-numbered row object surface sub-image. During the merging process, this application can also adopt an image fusion algorithm to ensure the visual consistency and continuity of the merged image.

[0108] Image processing algorithms, such as image segmentation techniques, can be employed to precisely decompose an image into odd - row and even - row sub - images. This process ensures that each row of data can be matched with its corresponding galvanometer movement state, thereby improving the accuracy of distortion correction. Meanwhile, during the process of decomposing the image, the present application also pre - processes the image, such as denoising and enhancing contrast, etc., to improve the effect of subsequent distortion correction. When using the odd - row grating sub - image and the even - row grating sub - image to perform distortion correction on the corresponding object - surface sub - images, the present application adopts advanced image registration and distortion correction algorithms. These algorithms can accurately calculate the position offset of each pixel point according to the grid distribution and distortion situation in the grating image, and make corresponding adjustments to the pixel positions in the object - surface sub - images. This process can not only eliminate the image distortion caused by the non - linear movement of the galvanometer, but also reduce the influence of the delay error existing in the data acquisition process on the image quality. After completing the distortion correction, the present application seamlessly merges the accurately corrected odd - row object - surface sub - images and even - row object - surface sub - images. During the merging process, the present application adopts an image fusion algorithm to ensure that the merged image maintains visual consistency and continuity. Through this series of processing steps, the present application can finally generate a target image with effectively corrected distortion and significantly improved quality. This innovative processing method not only improves the image quality of the laser confocal scanning microscope, but also provides a more accurate and reliable data basis for subsequent image analysis and processing.

[0109] In the above - mentioned embodiment, the confocal signal and the grating signal synchronously and isochronously collected by the acquisition card are twin images in terms of time and space. Among them, the grating image formed by the grating signal depicts the movement of the resonant galvanometer during the scanning process. Therefore, through the distortion analysis of the grating image, the present application can perform distortion correction on the object - surface image that is twin to it, thereby effectively improving the correction accuracy and being less susceptible to factors such as environment, lifespan, and optical path deviation. Moreover, the present application fully considers the influencing factors such as those that may be generated during the movement of the galvanometer and the delay error existing in the data acquisition process. During the distortion correction process, both the grating image and the object - surface image are decomposed into corresponding odd - row sub - images and even - row sub - images, and the odd - row sub - images and even - row sub - images are respectively used for correction to further improve the accuracy of distortion correction.

[0110] In one embodiment, before using the odd - row grating sub - image to perform distortion correction on the odd - row object - surface sub - image and using the even - row grating sub - image to perform distortion correction on the even - row object - surface sub - image in S120, it may further include:

[0111] S111: When it is detected that the resonant galvanometer uses bidirectional scanning, align the grid sequences of the odd-row raster sub-images and the even-row raster sub-images, and use the aligned odd-row raster sub-images to correct the distortion of the odd-row object surface sub-images, and use the aligned even-row raster sub-images to correct the distortion of the even-row object surface sub-images.

[0112] In this embodiment, when the resonant galvanometer uses bidirectional scanning, since the galvanometer mirror on the Y-axis takes a step at both key endpoints (such as the starting end and the return end) of the resonant galvanometer on the X-axis, this will cause a misalignment of the grid sequences between the odd-row raster sub-images and the even-row raster sub-images. To ensure the accuracy of distortion correction, before using the odd-row raster sub-images and the even-row raster sub-images to correct the distortion of the corresponding object surface sub-images, this application can first perform alignment processing on the grid sequences of the odd-row raster sub-images and the even-row raster sub-images.

[0113] For example, this application can identify the grid positions in the odd-row raster sub-images and the even-row raster sub-images through an image processing algorithm, then calculate the misalignment amount between the two based on these grid positions, and finally perform corresponding translation transformations on the odd-row raster sub-images or the even-row raster sub-images to achieve the alignment of the grid sequences. Or perform alignment in other ways. The aligned odd-row raster sub-images and even-row raster sub-images will more accurately reflect the actual movement of the resonant galvanometer, so that the odd-row object surface sub-images and the even-row object surface sub-images can be more effectively corrected for distortion. The addition of this step further improves the distortion correction accuracy of this application in the bidirectional scanning mode and ensures that the finally obtained target image has higher quality.

[0114] In one embodiment, aligning the grid sequences of the odd-row raster sub-images and the even-row raster sub-images in S111 to obtain the aligned odd-row raster sub-images and even-row raster sub-images may include:

[0115] S1111: Perform peak fitting on each column of the odd-row raster sub-image to obtain a first peak sequence, and determine a first interval sequence according to the intervals between adjacent peaks in the first peak sequence.

[0116] S1112: Perform peak fitting on each column of the even-row raster sub-image to obtain a second peak sequence, and determine a second interval sequence according to the intervals between adjacent peaks in the second peak sequence.

[0117] S1113: According to the first interval sequence and the second interval sequence, extract a first subsequence overlapping with the second peak sequence from the first peak sequence, and extract a second subsequence overlapping with the first peak sequence from the second peak sequence.

[0118] S1114: Extract the grids within the interval determined by the head and tail of the first subsequence from the odd-row raster sub-images, and form the aligned odd-row raster sub-images. Similarly, extract the grids within the interval determined by the head and tail of the second subsequence from the even-row raster sub-images, and form the aligned even-row raster sub-images.

[0119] In this embodiment, to accurately align the grid sequences of the odd-row raster sub-images and the even-row raster sub-images, this application adopts a series of image processing steps. First, this application can perform peak fitting on each column of the odd-row raster sub-images, thereby identifying the grid peak positions on each column, and determining the first interval sequence based on the intervals between adjacent peaks. This step can accurately depict the distribution of the grids in the odd-row raster sub-images. Similarly, this application also performs peak fitting on each column of the even-row raster sub-images, and obtains the second peak sequence and the corresponding second interval sequence.

[0120] With this basic data, this application can further analyze the misalignment of the grid sequences between the odd-row raster sub-images and the even-row raster sub-images. Specifically, this application can extract the first subsequence overlapping with the second peak sequence from the first peak sequence according to the first interval sequence and the second interval sequence, and at the same time extract the second subsequence overlapping with the first peak sequence from the second peak sequence. The extraction of these two subsequences provides key information for determining the misalignment amount between the odd-row raster sub-images and the even-row raster sub-images. Finally, this application can extract the grids within the intervals determined by the head and tail of the corresponding subsequences from the odd-row raster sub-images and the even-row raster sub-images respectively, and form the aligned odd-row raster sub-images and even-row raster sub-images. This process ensures that the grid sequences of the odd-row raster sub-images and the even-row raster sub-images can be accurately aligned in the bidirectional scanning mode, thereby improving the accuracy of subsequent distortion correction.

[0121] By performing the above alignment process on the grid sequences of the odd-row raster sub-images and the even-row raster sub-images, this application can achieve high-quality distortion correction for the laser confocal scanning microscope images.

[0122] In one embodiment, S1113, extracting the first subsequence overlapping with the second peak sequence from the first peak sequence according to the first interval sequence and the second interval sequence, and extracting the second subsequence overlapping with the first peak sequence from the second peak sequence, may include:

[0123] S11131: Determine the overlapping subsequences in the first interval sequence and the second interval sequence whose similarity exceeds a preset similarity threshold.

[0124] S11132: Based on the overlapping subsequence, extract a first subsequence overlapping with the second peak sequence from the first peak sequence, and extract a second subsequence overlapping with the first peak sequence from the second peak sequence.

[0125] In this embodiment, when determining the alignment manner of the grid sequences of the odd - row raster sub - images and the even - row raster sub - images, the present application can identify the overlapping subsequences with a similarity exceeding a preset similarity threshold in both the first interval sequence and the second interval sequence through in - depth analysis. Among them, the setting of the preset similarity threshold can be determined according to the actual image quality and distortion situation to ensure that the true corresponding relationship between the grid sequences can be accurately captured.

[0126] Once the overlapping subsequences are determined, the present application can, based on these overlapping parts, extract a first subsequence overlapping with the second peak sequence from the first peak sequence, and at the same time extract a second subsequence overlapping with the first peak sequence from the second peak sequence. The accurate extraction of these two subsequences is crucial for the subsequent alignment processing of the grid sequences.

[0127] In a specific implementation manner, the present application decomposes the raster image to obtain an odd - row raster sub - image composed of odd rows and an even - row raster sub - image composed of even rows ; then, the present application can calculate the column - mean brightness curves of the odd - row raster sub - image and the even - row raster sub - image and , and respectively perform peak fitting on and to obtain a first peak sequence and a second peak sequence , where and are respectively the number of peaks of and . Then, the present application can calculate a first interval sequence and a second interval sequence , where , = . Further, when the first interval sequence and the second interval sequence are obtained, the present application can calculate the overlapping subsequences and of the first interval sequence and the second interval sequence through the following formula, with the length being , and the specific formula is as follows:

[0128]

[0129] Among them, The search range is from to , where d is a distance metric, which can be, for example, the Pearson correlation coefficient. is a preset similarity threshold, which can be, for example, T = 0.9.

[0130] The overlapping subsequences are calculated through the above formula and After that, the present application can extract the first subsequence overlapping with the second peak sequence from the first peak sequence , and, extract the second subsequence overlapping with the first peak sequence from the second peak sequence .

[0131] Through the above image processing steps, the present application not only achieves the precise alignment of the raster sub-images of odd rows and even rows of the raster, but also further improves the accuracy and stability of the distortion correction.

[0132] In one embodiment, when using the raster sub-image of odd rows to correct the distortion of the object surface sub-image of odd rows in S130 to obtain a target image, it may include:

[0133] S131: Determine the positions of at least one grid in each pixel row of the raster sub-image of odd rows.

[0134] S132: Perform row-by-row correction on the object surface sub-image of odd rows according to the positions of the at least one grid in each pixel row to obtain a target image.

[0135] In this embodiment, when using the raster sub-image of odd rows to correct the distortion of the object surface sub-image of odd rows, the positions of at least one grid in each pixel row of the raster sub-image of odd rows can be determined first, so that the object surface sub-image of odd rows can be corrected according to the positions of the at least one grid in each pixel row to obtain a target image.

[0136] Among them, when the present application determines the positions of at least one grid in each pixel row of the raster sub-image of odd rows, the number of grids to be determined can be determined according to the current scene. For example, if the current scene is a scene with high precision requirements, the object surface sub-image of odd rows can be corrected by the positions of all grids in each pixel row of the raster sub-image of odd rows. If the current scene is a scene with relatively low precision requirements and high real-time requirements, the object surface sub-image of odd rows can be corrected by screening local grids, which can meet the real-time requirements.

[0137] In one embodiment, determining the positions of at least one grid in each pixel row of the raster sub-image of odd rows in S131 may include:

[0138] S1311: If the current scenario is a high-precision scenario, determine the positions of all grids in the odd-row raster sub-map in each pixel row.

[0139] S1312: If the current scenario is a real-time scenario and the resonant galvanometer scans unidirectionally, determine the positions of the grids in the region of interest in the odd-row raster sub-map in each pixel row.

[0140] In this embodiment, when determining the positions of at least one grid in the odd-row raster sub-map in each pixel row, the scenario characteristics of the current scenario can be determined first. If the current scenario is a high-precision scenario, that is, a scenario where the accuracy requirement for image distortion correction is higher than a preset accuracy threshold, such as application scenarios like micro-nano processing, super-resolution imaging, and precision measurement. The preset accuracy threshold can be a threshold in a certain dimension, such as error, resolution, etc., or it can be thresholds in multiple dimensions, which can be specifically set according to the actual situation and are not limited here. When it is determined that the current scenario is a high-precision scenario, the present application can correct the odd-row object sub-map through the positions of all grids in the odd-row raster sub-map in each pixel row, so as to achieve precise distortion correction of the odd-row object sub-map.

[0141] Furthermore, if the current scenario is a real-time scenario and the resonant galvanometer scans unidirectionally, that is, a scenario where the real-time requirement for image distortion correction is higher than a preset duration threshold and the adjacent motion cycles of the resonant galvanometer only have deviations at the starting point, such as application scenarios like laser processing, high-speed imaging, and dynamic display. The preset duration threshold in these scenarios can be set according to the actual situation and is not limited here. When the present application determines that the current scenario is a real-time scenario, the grids in the region of interest in the odd-row raster sub-map can be determined by screening local grids. The grids in the region of interest can be the grids at the center position of the odd-row raster sub-map or the grids at non-center positions, which can be specifically selected according to the actual situation and are not limited here either. The present application corrects the odd-row object sub-map through the grids in the region of interest, which can greatly shorten the processing duration of distortion correction and thus meet the real-time requirement.

[0142] In one embodiment, determining the positions of all grids in the odd-row raster sub-map in each pixel row in S1311 may include:

[0143] S13111: Determine the row brightness curves corresponding to the brightness values of each column in each pixel row of the odd-row raster sub-map.

[0144] S13112: Smooth the row brightness curves of each pixel row, and perform peak fitting on the smoothed row brightness curves until the number of peaks in each pixel row is equal, and then obtain the third peak sequence of each pixel row.

[0145] S12113: Determine the positions of all grids in each pixel row of the odd-row raster sub-image according to the third peak sequence of each pixel row.

[0146] In this embodiment, the process of using the odd-row raster sub-image to correct the distortion of the odd-row object sub-image is the same as the process of using the even-row raster sub-image to correct the distortion of the even-row object sub-image. Therefore, this application mainly describes the process of using the odd-row raster sub-image to correct the distortion of the odd-row object sub-image. Similarly, the process of correcting the distortion of the even-row object sub-image can be obtained.

[0147] Among them, when this application determines the positions of all grids in the odd-row raster sub-image, due to reasons such as mechanical motion error and electronic noise, there is a certain degree of misalignment between the brightest positions of the same grid in adjacent rows (corresponding to two adjacent motion cycles of the resonant galvanometer) in the odd-row raster sub-image. Schematically, as Figure 8 shown, Figure 8 is a display diagram of the deviation of the brightest positions between two adjacent rows of the same raster provided in the embodiment of this application; Figure 8 The position deviation in can be attributed to the fact that there is a certain deviation in the starting point and the time to reach the same grid between adjacent motion cycles of the resonant galvanometer, which means that it is not possible to simply regard a grid as a position for correction.

[0148] Based on this, when this application determines the positions of each grid in the odd-row raster sub-image, it can first determine the brightness values of each column in each pixel row of the odd-row raster sub-image and form a row brightness curve corresponding to each pixel row = , c = 1, ……, C (C is the image width of the odd-row raster sub-image, that is, the number of pixel columns, is the brightness value of the pixel in the r-th row and c-th column of the odd-row raster sub-image, and this brightness value represents the light and dark or color intensity of this position, which can be directly determined according to the odd-row raster sub-image). Schematically, as Figure 7 shown, Figure 9 is a schematic diagram of the row brightness curve corresponding to a pixel row in the odd-row raster sub-image provided in the embodiment of this application, Figure 9 in which each peak corresponds to a grid.

[0149] Further, since the odd - row raster sub - images and even - row raster sub - images decomposed during unidirectional scanning have the same number of grids and their positions are strictly aligned, no alignment processing is required. For the odd - row raster sub - images obtained by bidirectional scanning, since alignment processing is needed, when the present application determines the positions of all grids in each pixel row of the odd - row raster sub - image, grids located in the interval determined by the head and tail of the first subsequence can be extracted from the odd - row raster sub - image to form an aligned odd - row raster sub - image, and the row - brightness curve corresponding to the brightness values of each column in each pixel row of the aligned odd - row raster sub - image can be determined.

[0150] Next, the present application can perform smoothing processing on the row - brightness curve of each pixel row to highlight the main trend or structure of the row - brightness curve by suppressing noise or irrelevant details, thereby improving the readability, analyzability, or visual effect of the row - brightness curve and providing a relatively accurate data basis for subsequent peak fitting.

[0151] After the present application performs smoothing processing on the row - brightness curves of each pixel row, peak fitting can be performed on the smoothed row - brightness curves, so that the number of peaks in each pixel row can be determined. When the number of peaks in each pixel row is equal, the third peak sequence of each pixel row can be obtained. The third peak sequence , i = 1, ……, , where is the number of peaks obtained by fitting the row - brightness curve of the r - th row under the current smoothing radius. Schematically, as Figure 10 shown, Figure 10 is a schematic diagram of the peak - fitting result of one of the pixel rows provided by the embodiment of the present application; Figure 10 The red line in

[0152] represents the position of the pixel with the maximum brightness in the grid area, and the green line represents the fitting position. Obviously, the fitting position is more reasonable. Based on the above - mentioned processing process, the present application can obtain the third peak sequence corresponding to each pixel row. Since this third peak sequence indicates the peak positions of multiple peaks obtained by fitting each pixel row under the current smoothing radius, the present application can determine the positions of all grids in the odd - row raster sub - image in each pixel row according to the third peak sequence of each pixel row and form a corresponding grid - position matrix , where r = 1, ……, R, i = 1, ……, N, R is the image height of the odd - row raster sub - image, that is, the number of pixel rows, N is the number of grids in each row. When bidirectional scanning is performed, , is the position (i.e., the horizontal - direction coordinate) of the i - th grid in the r - th row in the odd - row raster sub - image.

[0153] In one embodiment, smoothing the row luminance curves of each pixel row in S13112, and performing peak fitting on the smoothed row luminance curves until the number of peaks of each pixel row is equal, to obtain the third peak sequence of each pixel row, may include:

[0154] S131121: Set the current value of the smoothing radius.

[0155] S131122: Smooth the row luminance curves of each pixel row based on the current value, and perform peak fitting on the smoothed row luminance curves to obtain the preliminary peak position sequence of each pixel row.

[0156] S131123: Compare the preliminary peak position sequences of each pixel row. If the number of peaks of at least one pixel row is not equal to the number of peaks of other pixel rows, then update the current value and continue to perform the steps of smoothing the row luminance curves of each pixel row based on the current value and subsequent steps until the number of peaks of each pixel row is equal.

[0157] S131124: If the number of peaks of each pixel row is equal, then use the preliminary peak position sequences of each pixel row as the final third peak sequence.

[0158] In this embodiment, when smoothing the row luminance curves of each pixel row, since the selection of the smoothing radius (or the size of the smoothing window) will directly affect the quality of the processing effect and the degree of feature retention. Therefore, the present application can first set the current value of the smoothing radius, and this current value can be an initial value. Then, based on this current value, smooth the row luminance curves of each pixel row, and then perform peak fitting. After obtaining the preliminary peak position sequences of each pixel row, the number of peaks of the preliminary peak position sequences of each pixel row can be compared. If the number of peaks of at least one pixel row is not equal to the number of peaks of other pixel rows, then update the current value of the smoothing radius. For example, when the initial value s = 1, the updated current value can be s = s + 1. Then, the present application can continue to smooth the row luminance curves of each pixel row based on the updated current value, and perform peak fitting on the smoothed row luminance curves to obtain the preliminary peak position sequences of each pixel row, and compare the number of peaks of the preliminary peak position sequences of each pixel row until the number of peaks of each pixel row is equal. At this time, the preliminary peak position sequences of each pixel row can be used as the final third peak sequence. The present application can determine the positions of all grids in the odd-row raster sub-image in each pixel row according to the third peak sequence of each pixel row, and perform distortion correction on the odd-row object surface sub-image according to the positions of each grid in each pixel row, so as to obtain an accurate correction result.

[0159] In one embodiment, determining the positions of the grids of the region of interest in the odd-row raster sub-image in each pixel row in S1312 may include:

[0160] S13121: Determine the column mean brightness curve corresponding to the brightness values of each pixel row in each column of the odd-row raster sub-image, and after performing peak fitting on the column mean brightness curves of each column, obtain a fourth peak sequence.

[0161] S13122: Crop the region of interest of the odd-row raster sub-image according to the fourth peak sequence, and extract the peak positions of each pixel row from the cropped odd-row raster sub-image as the positions of the grids of the region of interest in the odd-row raster sub-image in each pixel row.

[0162] In this embodiment, if the current scenario is a real-time scenario and the resonant galvanometer performs one-way scanning, that is, the real-time requirement for image distortion correction is higher than the preset duration threshold, and the adjacent motion periods of the resonant galvanometer only have deviations at the starting point, such as application scenarios like laser processing, high-speed imaging, and dynamic display, etc., the grids of the region of interest in the odd-row raster sub-image can be determined by screening local grids. The grids of this region of interest can be the grids at the center position of the odd-row raster sub-image or the grids at non-center positions, and can be specifically selected according to the actual situation, and no limitation is made here. The present application corrects the odd-row object sub-image through the grids of the region of interest, which can greatly shorten the processing duration of distortion correction and thus meet the real-time requirement.

[0163] Specifically, when the present application determines the positions of the grids of the region of interest in the odd-row raster sub-image in each pixel row, it can first determine the column mean brightness curve corresponding to the brightness values of each pixel row in each column of the odd-row raster sub-image, and after performing peak fitting on the column mean brightness curves of each column, obtain a fourth peak sequence. Then, the present application can crop the region of interest of the odd-row raster sub-image according to the fourth peak sequence, and extract the peak positions of each pixel row from the cropped odd-row raster sub-image. This peak position is the position of the grid of the region of interest in the odd-row raster sub-image in each pixel row.

[0164] For example, when the present application calculates the column mean brightness curve corresponding to the brightness values of each pixel row in each column of the odd-row raster sub-image after that, it can perform peak fitting on to obtain a fourth peak sequence , k = 1, ……, K, where K is the number of peaks; since this fourth peak sequence indicates the number of grids in the odd - row raster sub - graph, and the resonant galvanometer of this application scans unidirectionally, therefore, this application can select the area with relatively low or the lowest distortion degree in the odd - row raster sub - graph as the region of interest according to the scanning characteristics of the resonant galvanometer and the number of grids in the odd - row raster sub - graph. After cropping this region of interest, the peak positions of each pixel row are extracted from the cropped odd - row raster sub - graph, and this peak position is the position of the grid of the region of interest in the odd - row raster sub - graph in each pixel row. In this way, during distortion correction, only one region of interest in the odd - row raster sub - graph needs to be detected for peaks row by row, thereby greatly shortening the correction time and meeting the real - time requirement.

[0165] In one embodiment, cropping the region of interest of the odd - row raster sub - graph according to the fourth peak sequence in S13122 may include:

[0166] S131221: Determine the number of peaks in the fourth peak sequence.

[0167] S131222: Determine the starting position and the ending position for cropping the odd - row raster sub - graph according to the number of peaks in the fourth peak sequence.

[0168] S131223: Crop the region of interest of the odd - row raster sub - graph according to the starting position and the ending position.

[0169] In this embodiment, when cropping the region of interest of the odd - row raster sub - graph according to the fourth peak sequence, this region of interest can be the central region of the odd - row raster sub - graph or a non - central region, which can be specifically selected according to the actual situation and is not limited here.

[0170] In a specific implementation manner, when this application selects the central region of the odd - row raster sub - graph as the region of interest, it can first determine the number of peaks in the fourth peak sequence, and then determine the starting position and the ending position for cropping the odd - row raster sub - graph according to the number of peaks in the fourth peak sequence, so that the region of interest of the odd - row raster sub - graph can be cropped according to the starting position and the ending position.

[0171] For example, when the fourth peak sequence of this application is , k = 1, ……, K, where K is the number of peaks, this application can set the starting position and the ending position as:

[0172]

[0173] where, , that is, the midpoint of the fourth peak sequence, is the starting position, is the ending position, is the midpoint peak of the fourth peak sequence, is a peak before the midpoint of the fourth peak sequence, is a peak after the midpoint of the fourth peak sequence. After cropping the odd - row raster sub - image using the above - mentioned starting position and ending position, an interested region at the center of the odd - row raster sub - image as shown in Figure 11 can be obtained. This interested region only contains the luminance data of one grid. By performing row - by - row correction on the odd - row object sub - image using this interested region, the final target image can be obtained, thus meeting the real - time requirement.

[0174] In one embodiment, step S132 of performing row - by - row correction on the odd - row object sub - image according to the positions of the at least one grid in each pixel row to obtain a corrected odd - row object sub - image may include:

[0175] S1321: Obtain a first blank image to be filled, where the image height of the first blank image is the same as the image height of the odd - row object sub - image.

[0176] S1322: Based on the positions of all grids in the odd - row raster sub - image in each pixel row, perform segmented sampling on the luminance values of the corresponding pixel rows in the odd - row object sub - image, and sequentially fill the sampled luminance values into the first blank image to obtain a corrected odd - row object sub - image.

[0177] In this embodiment, if the current scene is a high - precision scene, that is, a scene where the accuracy requirement for image distortion correction is higher than a preset accuracy threshold, such as application scenarios like micro - nano processing, super - resolution imaging, and precision measurement, the present application can correct the odd - row object sub - image through the positions of all grids in the odd - row raster sub - image in each pixel row, so as to achieve precise distortion correction of the odd - row object sub - image.

[0178] Specifically, the present application can obtain a first blank image to be filled , and the image height of this first blank image can be the same as the image height of the odd - row object sub - image. The image width W can be a pre - specified value, such as W = 1024. Then, the present application can sequentially extract the r - th row image of the odd - row object sub - image = , where r = 1, ……, R, c = 1, ……, C, R is the maximum number of rows of the odd - row object sub - image, and C is the maximum number of columns of the odd - row object sub - image, is the luminance value of the c - th column in the r - th row of the odd - row object sub - image. Then, the present application can perform the following mapping:

[0179]

[0180] Among them, T can be optionally a linear mapping or a non-linear mapping, and J = W / (N - 1). This mapping is equivalent to linearly or non-linearly sampling the brightness curve segment located in Located in The number of sampling points is J, and the number of sampling points can be set according to the actual situation. Then, the sampling data is filled into the corresponding positions of the first blank image to obtain the corrected odd-row object sub-image. The corrected odd-row object sub-image is as shown in Figure 12 Shown, from Figure 12 It can be seen that through the above distortion correction method, the odd-row object sub-image can be accurately corrected after distortion correction.

[0181] In one embodiment, step S132 of performing row-by-row correction on the odd-row object sub-image according to the positions of the at least one grid in each pixel row to obtain the corrected odd-row object sub-image may include:

[0182] S321: Align the odd-row raster sub-image row by row according to the positions of the grids in the region of interest in the odd-row raster sub-image in each pixel row to obtain the aligned odd-row raster sub-image.

[0183] S322: Align the odd-row object sub-image row by row according to the positions of the grids in the region of interest in the odd-row raster sub-image in each pixel row to obtain the aligned odd-row object sub-image.

[0184] S323: Perform row-by-row correction on the aligned odd-row object sub-image according to the aligned odd-row raster sub-image to obtain the corrected odd-row object sub-image.

[0185] In this embodiment, if the current scenario is a real-time scenario and the resonant galvanometer performs one-way scanning, that is, in scenarios where the real-time requirement for image distortion correction is higher than the preset duration threshold and there is only a deviation at the starting point in adjacent motion cycles of the resonant galvanometer, such as application scenarios like laser processing, high-speed imaging, and dynamic display, the grids in the region of interest in the odd-row raster sub-image can be determined by screening local grids, and the odd-row object sub-image can be corrected through the grids in the region of interest, which can greatly shorten the processing duration of distortion correction and thus meet the real-time requirement.

[0186] Specifically, when the present application performs line-by-line correction on the odd-numbered rows of the object surface sub-image according to the positions of the grids in the region of interest in the odd-numbered rows of the raster sub-image, it can first perform line-by-line alignment on the odd-numbered rows of the raster sub-image according to the positions of the grids in the region of interest in each pixel row to obtain the aligned odd-numbered rows of the raster sub-image. Then, perform line-by-line alignment on the odd-numbered rows of the object surface sub-image according to the positions of the grids in the region of interest in each pixel row to obtain the aligned odd-numbered rows of the object surface sub-image. Next, use the aligned odd-numbered rows of the raster sub-image to perform line-by-line correction on the aligned odd-numbered rows of the object surface sub-image, so as to obtain the corrected odd-numbered rows of the object surface sub-image. Moreover, the time of the target image obtained by the present application in this way is significantly shorter than the time of the corrected odd-numbered rows of the object surface sub-image obtained by performing line-by-line correction on the odd-numbered rows of the object surface sub-image according to the positions of all the grids in the odd-numbered rows of the raster sub-image. Therefore, the real-time requirement can be satisfied to a great extent.

[0187] In one embodiment, in S321, performing line-by-line alignment on the odd-numbered rows of the raster sub-image according to the positions of the grids in the region of interest in the odd-numbered rows of the raster sub-image to obtain the aligned odd-numbered rows of the raster sub-image may include:

[0188] S3211: Determine the maximum and minimum values of the positions in all pixel rows according to the positions of the grids in the region of interest in the odd-numbered rows of the raster sub-image, and determine the original image width of the odd-numbered rows of the raster sub-image.

[0189] S3212: Determine the resampling interval for resampling the odd-numbered rows of the raster sub-image according to the maximum value, the minimum value, and the original image width.

[0190] S3213: Obtain a second blank image to be filled, where the image height of the second blank image is the same as the image height of the odd-numbered rows of the raster sub-image.

[0191] S3214: Resample the odd-numbered rows of the raster sub-image within the resampling interval, and sequentially fill the resampled luminance values into the second blank image to obtain the aligned odd-numbered rows of the raster sub-image.

[0192] In this embodiment, when aligning the odd - row raster sub - images row by row according to the positions of the grids in the region of interest in the odd - row raster sub - images on each pixel row, the present application can first determine the maximum and minimum values of the positions in all pixel rows and the original image width of the odd - row raster sub - images according to the positions of the grids in the region of interest in the odd - row raster sub - images on each pixel row. In this way, the sampling interval for resampling the odd - row raster sub - images can be determined based on the maximum and minimum values of the positions in all pixel rows and the original image width. Then, the present application can obtain a second blank image to be filled, and the image height of the second blank image is the same as that of the odd - row raster sub - images. In this way, the odd - row raster sub - images can be resampled within the sampling interval, and after filling the resampled luminance values into the second blank image in sequence, the aligned odd - row raster sub - images are obtained. The image height of the aligned odd - row raster sub - images is the same as that before alignment, while the image width has changed.

[0193] In a specific implementation manner, the present application can extract the peak positions on each pixel row in the region of interest of the above - mentioned odd - row raster sub - images , r = 1, …, R, where R is the image height, that is, the number of pixel rows. Then, the odd - row raster sub - images are aligned row by row through the following mapping to obtain the aligned odd - row raster sub - images, denoted as A:

[0194]

[0195] Among them, and are the r - th row images of the odd - row raster sub - image I before alignment and the odd - row raster sub - image A after alignment respectively. can be optionally a linear mapping or a non - linear mapping, which is equivalent to resampling in the interval . is the original image width of the odd - row raster sub - images, L is the pixel width of A, is the maximum value of the peak positions of all rows, is the minimum value of the peak positions of all rows, and there is:

[0196]

[0197] After the above mapping, the local enlarged comparison diagrams of the odd - row raster sub - images before and after alignment as shown in Figure 11 can be obtained. Figure 13 The left side of Figure 13It can be seen that the misalignment between the brightest positions of the same grid in adjacent rows of the odd-row raster sub-image after alignment is small. By using the aligned odd-row raster sub-image to perform row-by-row correction on the odd-row object surface sub-image, a relatively accurate correction result can be obtained.

[0198] In one embodiment, in S322, according to the positions of the grids in the region of interest in the odd-row raster sub-image in each pixel row, performing row-by-row alignment on the odd-row object surface sub-image to obtain the aligned odd-row object surface sub-image may include:

[0199] S3221: Obtain a third blank image to be filled, where the image height of the third blank image is the same as the image height of the odd-row object surface sub-image.

[0200] S3222: Resample the odd-row object surface sub-image within the sampling interval, and sequentially fill the resampled brightness values into the third blank image to obtain the aligned odd-row object surface sub-image.

[0201] In this embodiment, when performing row-by-row alignment on the odd-row object surface sub-image according to the positions of the grids in the region of interest in the odd-row raster sub-image, the present application may first obtain a third blank image to be filled, where the image height of the third blank image is the same as the image height of the odd-row object surface sub-image. Then, the present application may perform row-by-row alignment on the odd-row object surface sub-image in the above-mentioned manner of performing row-by-row alignment on the odd-row raster sub-image to obtain the aligned odd-row object surface sub-image, denoted as B:

[0202]

[0203] Wherein, and are respectively the r-th row images of the odd-row object surface sub-image G before alignment and the odd-row object surface sub-image B after alignment, can be optionally a linear mapping or a non-linear mapping, which is equivalent to resampling in the interval In the interval The local enlarged comparison diagrams of the odd-row object surface sub-images before and after alignment are as shown in Figure 14 shown, Figure 14 The left side in Figure 14 is the local enlarged diagram before alignment, and the right side is the local enlarged diagram after alignment. It can be seen from

[0204] In one embodiment, in S323, according to the aligned odd-row raster sub-image, performing row-by-row correction on the aligned odd-row object surface sub-image to obtain the corrected odd-row raster sub-image may include:

[0205] S3231: Determine the column mean curves corresponding to the luminance values of each pixel row in each column of the aligned odd - row raster sub - image, and after performing peak fitting on the column mean curves of each column, obtain multiple fifth peak sequences.

[0206] S3232: Obtain a fourth blank image to be filled, where the image height of the fourth blank image is the same as the image height of the aligned odd - row object sub - image.

[0207] S3233: Based on the positions of each fifth peak sequence, perform segmented sampling on the luminance values of the corresponding pixel rows in the aligned odd - row object sub - image, and sequentially fill the sampled luminance values into the fourth blank image to obtain a corrected odd - row raster sub - image.

[0208] In this embodiment, after obtaining the aligned odd - row raster sub - image and the aligned odd - row object sub - image, the present application can perform line - by - line correction on the aligned odd - row object sub - image according to the aligned odd - row raster sub - image to obtain a corrected odd - row raster sub - image.

[0209] Specifically, the present application can first determine the column mean curves corresponding to the luminance values of each pixel row in each column of the aligned odd - row raster sub - image, and after performing peak fitting on the column mean curves of each column, obtain multiple fifth peak sequences. Then, obtain a fourth blank image to be filled, where the image height of the fourth blank image is the same as the image height of the aligned odd - row object sub - image. In this way, based on the positions of each fifth peak sequence, perform segmented sampling on the luminance values of the corresponding pixel rows in the aligned odd - row object sub - image, and after sequentially filling the sampled luminance values into the fourth blank image, obtain a corrected odd - row raster sub - image.

[0210] For example, the present application can first calculate the column mean curves corresponding to the luminance values of each pixel row in each column of the aligned odd - row raster sub - image A , and then perform peak fitting on to obtain the fifth peak sequence , k = 1, …, K, where K is the number of peaks. Then, the present application can perform line - by - line correction on the aligned odd - row object sub - image B to obtain the final target image D. During the correction process, a fourth blank image D for filling can be prepared first, with its pixel height being the same as that of the odd - row object sub - image and its width W being a pre - specified value, such as W = 1024. Then, sequentially extract the r - th row image of B = , r = 1, …, R, c = 1, …, L, being the luminance value of the c - th column and r - th row of image B. Finally, the present application can perform the following mapping on After performing segmentation alignment and sampling, fill it into the fourth blank image:

[0211]

[0212] Among them, It can be a linear mapping or a non-linear mapping, = W / (K - 1). This mapping is equivalent to performing linear or non-linear sampling on the luminance curve segment located in Located in The number of sampling points is , and then fill the sampling data into the corresponding positions of the fourth blank image to obtain the corrected odd-line raster sub-image; the corrected odd-line raster sub-image is as shown in Figure 15 Shown, comparing it with the corrected odd-line raster sub-image of Figure 12 It can be seen that its effect is not much different from the corrected odd-line raster sub-image obtained by using the full raster alignment correction algorithm. However, since this algorithm only needs to perform row-by-row peak detection on an interested area of the odd-line raster sub-image, this algorithm can greatly shorten the correction time, thus meeting the real-time requirement.

[0213] In one embodiment, the present application also provides an acquisition card. The acquisition card stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the distortion correction method described in any one of the above embodiments.

[0214] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0215] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0216] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A laser confocal scanning microscope, comprising a laser light source, a resonant galvanometer mirror, a galvanometer mirror, a beam splitter, an excitation optical path, a confocal optical path, and a first detector, characterized in that, The microscope further includes: a dichroic mirror fixedly installed between the resonant galvanometer mirror and the galvanometer mirror, a monitoring optical path and a second detector arranged in the transmission direction of the dichroic mirror, and an acquisition card respectively connected to the first detector and the second detector; Wherein, when the microscope scans a sample, the beam emitted by the laser light source is reflected by the beam splitter to the resonant galvanometer mirror. After the resonant galvanometer mirror deflects the incident beam for the first time, it emits the beam to the dichroic mirror. The dichroic mirror reflects most of the beam to the galvanometer mirror. After the galvanometer mirror deflects the incident beam for the second time, the deflected beam is focused onto the surface of the sample through the excitation optical path. The reflected light excited from the surface of the sample returns along the original path, is transmitted by the beam splitter to the confocal optical path, and reaches the first detector through the confocal optical path. The dichroic mirror transmits a small part of the beam to the monitoring optical path, and the monitoring optical path converts the real-time deflection angle of the resonant galvanometer mirror into a grating signal and sends it to the second detector; The acquisition card synchronously and isochronously acquires the confocal signal output by the first detector and the grating signal output by the second detector, generates a surface image corresponding to the confocal signal and a grating image corresponding to the grating signal, then decomposes the surface image into an odd-row surface sub-image and an even-row surface sub-image, and decomposes the grating image into an odd-row grating sub-image and an even-row grating sub-image. Then, the odd-row grating sub-image is used to correct the distortion of the odd-row surface sub-image, the even-row grating sub-image is used to correct the distortion of the even-row surface sub-image, and the corrected odd-row surface sub-image and the corrected even-row surface sub-image are merged to obtain a target image.

2. The laser confocal scanning microscope according to claim 1, wherein The monitoring optical path includes an attenuation sheet and a Ronchi grating; The attenuation sheet attenuates the transmitted beam and then emits it to the Ronchi grating, and the Ronchi grating converts the real-time deflection angle of the resonant galvanometer mirror into a grating signal according to the attenuated beam.

3. A method for correcting distortion of a scanned image, which is applied to the acquisition card in the laser confocal scanning microscope according to claim 1 or 2 above, and is characterized in that, The method includes: Synchronously and isochronously acquiring the confocal signal output by the first detector and the grating signal output by the second detector, and generating a surface image corresponding to the confocal signal and a grating image corresponding to the grating signal; Decomposing the surface image into an odd-row surface sub-image and an even-row surface sub-image, and decomposing the grating image into an odd-row grating sub-image and an even-row grating sub-image; Using the odd-row grating sub-image to correct the distortion of the odd-row surface sub-image, and using the even-row grating sub-image to correct the distortion of the even-row surface sub-image; Merging the corrected odd-row surface sub-image and the corrected even-row surface sub-image to obtain a target image.

4. The scanning image distortion correction method according to claim 3, characterized in that, Before using the odd-row grating sub-image to correct the distortion of the odd-row surface sub-image and using the even-row grating sub-image to correct the distortion of the even-row surface sub-image, it further includes: When it is detected that the resonant galvanometer adopts bidirectional scanning, align the raster sequences of the odd-row raster sub-images and the even-row raster sub-images, and use the aligned odd-row raster sub-images to correct the distortion of the odd-row object surface sub-images, and use the aligned even-row raster sub-images to correct the distortion of the even-row object surface sub-images.

5. The scanning image distortion correction method according to claim 4, wherein The alignment of the raster sequences of the odd-row raster sub-images and the even-row raster sub-images to obtain the aligned odd-row raster sub-images and even-row raster sub-images includes: Perform peak fitting on each column of the odd-row raster sub-image to obtain a first peak sequence, and determine a first interval sequence according to the intervals between adjacent peaks in the first peak sequence; Perform peak fitting on each column of the even-row raster sub-image to obtain a second peak sequence, and determine a second interval sequence according to the intervals between adjacent peaks in the second peak sequence; According to the first interval sequence and the second interval sequence, extract a first subsequence overlapping with the second peak sequence from the first peak sequence, and extract a second subsequence overlapping with the first peak sequence from the second peak sequence; Extract the grids located in the interval determined by the head and tail of the first subsequence from the odd-row raster sub-image to form the aligned odd-row raster sub-image, and extract the grids located in the interval determined by the head and tail of the second subsequence from the even-row raster sub-image to form the aligned even-row raster sub-image.

6. The scanning image distortion correction method according to claim 5, wherein, The extraction of a first subsequence overlapping with the second peak sequence from the first peak sequence and the extraction of a second subsequence overlapping with the first peak sequence from the second peak sequence according to the first interval sequence and the second interval sequence include: Determine the overlapping subsequences in the first interval sequence and the second interval sequence whose similarity exceeds a preset similarity threshold; Based on the overlapping subsequences, extract a first subsequence overlapping with the second peak sequence from the first peak sequence, and extract a second subsequence overlapping with the first peak sequence from the second peak sequence.

7. The scanning image distortion correction method according to any one of claims 3-6, characterized in that, The use of the odd-row raster sub-image to correct the distortion of the odd-row object surface sub-image to obtain a target image includes: Determine the positions of at least one grid in each pixel row of the odd-row raster sub-image; Perform row-by-row correction on the odd-row object surface sub-image according to the positions of the at least one grid in each pixel row to obtain a target image.

8. The scanning image distortion correction method according to claim 7, wherein, The determination of the positions of at least one grid in each pixel row of the odd-row raster sub-image includes: If the current scenario is a high-precision scenario, determine the positions of all grids in each pixel row of the odd-row raster sub-image; If the current scenario is a real-time scenario, determine the positions of the grids in the region of interest in the odd-row raster sub-image in each pixel row.

9. The scanning image distortion correction method according to claim 8, characterized in that, The determination of the positions of all grids in each pixel row of the odd-row raster sub-image includes: Determine the row brightness curves corresponding to the column brightness values in each pixel row of the odd-row raster sub-image; Smooth the row brightness curves of each pixel row, and perform peak fitting on the smoothed row brightness curves until the number of peaks in each pixel row is equal, then obtain the third peak sequence of each pixel row; Determine the positions of all grids in the odd - row raster sub - map in each pixel row according to the third peak sequence of each pixel row.

10. The scanning image distortion correction method according to claim 9, characterized in that, The step of smoothing the row brightness curves of each pixel row and performing peak fitting on the smoothed row brightness curves until the number of peaks in each pixel row is equal, then obtaining the third peak sequence of each pixel row, includes: Set the current value of the smoothing radius; Based on the current value, smooth the row brightness curves of each pixel row and perform peak fitting on the smoothed row brightness curves to obtain the preliminary peak position sequence of each pixel row; Compare the preliminary peak position sequences of each pixel row. If the number of peaks in at least one pixel row is not equal to the number of peaks in other pixel rows, then update the current value and continue to perform the steps of smoothing the row brightness curves of each pixel row based on the current value and subsequent steps until the number of peaks in each pixel row is equal; If the number of peaks in each pixel row is equal, then use the preliminary peak position sequences of each pixel row as the final third peak sequence.

11. The scanning image distortion correction method according to claim 8, wherein The step of determining the positions of the grids in the region of interest in the odd - row raster sub - map in each pixel row includes: Determine the column - mean brightness curves corresponding to the brightness values of each pixel row in each column of the odd - row raster sub - map, and perform peak fitting on the column - mean brightness curves of each column to obtain the fourth peak sequence; Crop the region of interest of the odd - row raster sub - map according to the fourth peak sequence, and extract the peak positions of each pixel row from the cropped odd - row raster sub - map as the positions of the grids in the region of interest in the odd - row raster sub - map in each pixel row.

12. The scanning image distortion correction method according to claim 11, wherein The step of cropping the region of interest of the odd - row raster sub - map according to the fourth peak sequence includes: Determine the number of peaks in the fourth peak sequence; According to the number of peaks in the fourth peak sequence, determine the starting position and ending position for cropping the odd - row raster sub - map; Crop the region of interest of the odd - row raster sub - map according to the starting position and the ending position.

13. The scanning image distortion correction method according to claim 8, wherein The step of performing row - by - row correction on the odd - row object - surface sub - map according to the positions of the at least one grid in each pixel row to obtain the corrected odd - row object - surface sub - map includes: Obtain a first blank image to be filled, and the image height of the first blank image is the same as the image height of the odd - row object - surface sub - map; Based on the positions of all grids in the odd - row raster sub - map in each pixel row, perform segmented sampling on the brightness values of the corresponding pixel rows in the odd - row object - surface sub - map, and sequentially fill the sampled brightness values into the first blank image to obtain the corrected odd - row object - surface sub - map.

14. The scanning image distortion correction method according to claim 8, wherein, The step of performing row - by - row correction on the odd - row object - surface sub - map according to the positions of the at least one grid in each pixel row to obtain the corrected odd - row object - surface sub - map includes: Align the odd - row raster sub - images row - by - row according to the positions of the grids in the regions of interest in each pixel row of the odd - row raster sub - images, to obtain the aligned odd - row raster sub - images; Align the odd - row object surface sub - images row - by - row according to the positions of the grids in the regions of interest in each pixel row of the odd - row raster sub - images, to obtain the aligned odd - row object surface sub - images; Perform row - by - row correction on the aligned odd - row object surface sub - images according to the aligned odd - row raster sub - images, to obtain the corrected odd - row object surface sub - images.

15. The scanning image distortion correction method according to claim 14, characterized in that, The step of aligning the odd - row raster sub - images row - by - row according to the positions of the grids in the regions of interest in each pixel row of the odd - row raster sub - images, to obtain the aligned odd - row raster sub - images, includes: Determine the maximum and minimum values of the positions in all pixel rows, and determine the original image width of the odd - row raster sub - images according to the positions of the grids in the regions of interest in each pixel row of the odd - row raster sub - images; Determine the resampling interval for resampling the odd - row raster sub - images according to the maximum value, the minimum value, and the original image width; Obtain a second blank image to be filled, where the image height of the second blank image is the same as the image height of the odd - row raster sub - images; Resample the odd - row raster sub - images within the resampling interval, and sequentially fill the resampled luminance values into the second blank image to obtain the aligned odd - row raster sub - images.

16. The scanning image distortion correction method according to claim 15, wherein The step of aligning the odd - row object surface sub - images row - by - row according to the positions of the grids in the regions of interest in each pixel row of the odd - row raster sub - images, to obtain the aligned odd - row object surface sub - images, includes: Obtain a third blank image to be filled, where the image height of the third blank image is the same as the image height of the odd - row object surface sub - images; Resample the odd - row object surface sub - images within the resampling interval, and sequentially fill the resampled luminance values into the third blank image to obtain the aligned odd - row object surface sub - images.

17. The scanning image distortion correction method according to claim 14, wherein The step of performing row - by - row correction on the aligned odd - row object surface sub - images according to the aligned odd - row raster sub - images, to obtain the corrected odd - row object surface sub - images, includes: Determine the column - mean curves corresponding to the luminance values of each pixel row in each column of the aligned odd - row raster sub - images, and after performing peak fitting on the column - mean curves of each column, obtain a plurality of fifth - peak sequences; Obtain a fourth blank image to be filled, where the image height of the fourth blank image is the same as the image height of the aligned odd - row object surface sub - images; Based on the positions of each fifth - peak sequence, perform segmented sampling on the luminance values of the corresponding pixel rows in the aligned odd - row object surface sub - images, and sequentially fill the sampled luminance values into the fourth blank image to obtain the corrected odd - row object surface sub - images.

18. A capture card, characterized in that, The acquisition card stores computer - readable instructions, which when executed by one or more processors, cause the one or more processors to execute the steps of the scanning image distortion correction method according to any one of claims 3 to 17.

Images