A Method, Device and Medium for Calibrating the Step Size of a Two-Dimensional Scanning Module in a Scanning Light Field Imaging System

Abstract

The present invention discloses a method, device and medium for calibrating the step size of a two-dimensional scanning module in a scanning light field imaging system, which relates to the technical field of light field microscopy imaging. The method includes: collecting two-dimensional original scanning light field data with an initial scanning step size and rearranging it into four-dimensional phase space data to obtain a multi-view rearranged image; selecting the middle view of all views in the multi-view rearranged image to obtain a non-scanning center view sub-image; calculating a sub-pixel displacement matrix of all non-scanning center view sub-images relative to the first non-scanning center view sub-image; calculating an average displacement amount according to the sub-pixel displacement matrix, and calculating a step size adjustment coefficient according to the average displacement amount; updating the correct scanning step size according to the step size adjustment coefficient. The present invention calibrates the scanning step size of two-dimensional scanning imaging with higher speed and higher precision, and improves the spatial resolution of scanning imaging.

Description

Technical Field

[0001] The present invention relates to the technical field of light field microscopy imaging, and more particularly to a method, device, and medium for calibrating the step size of a two-dimensional scanning module in a scanning light field imaging system. Background Art

[0002] In a light field microscopy imaging system, a microlens is used to achieve multi-view light field imaging. By sacrificing a certain spatial resolution, the angular information of light rays is obtained, and the three-dimensional tomography result of a sample can be obtained through a phase space reconstruction method. On the premise of ensuring the angular resolution, in order to improve the lateral resolution of the light field microscopy imaging system, a two-dimensional galvanometer is added to scan the sample two-dimensionally, realizing time-space multiplexing scanning imaging. The images obtained from multiple scans are stitched according to the scanning positions to obtain an image with higher spatial resolution.

[0003] Taking a 3*3 scan as an example, it is necessary to strictly control the scan step size to be 1 / 3 of the physical size of a single microlens in order to effectively improve the resolution of the stitched scanned images. A too large step size will result in data redundancy and cannot improve the resolution; a too small step size will result in a discontinuous stitched image. Therefore, obtaining the optimal scan step size is crucial for improving the spatial resolution of scanning light field imaging.

[0004] Therefore, how to provide a method for calibrating the step size of a two-dimensional scanning module in a scanning light field imaging system is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a method, device, and medium for calibrating the step size of a two-dimensional scanning module in a scanning light field imaging system, which can calibrate the scan step size of two-dimensional scanning imaging at a faster speed and higher precision, and improve the spatial resolution of scanning imaging.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for calibrating the step size of a two-dimensional scanning module in a scanning light field imaging system includes:

[0008] Collecting two-dimensional raw scanning light field data at an initial scan step size, rearranging it into four-dimensional phase space data, and obtaining a multi-view rearranged image;

[0009] Selecting the middle view of all views in the multi-view rearranged image to obtain a non-scanning center view sub-image;

[0010] Calculating the sub-pixel displacement matrix of all non-scanning center view sub-images relative to the first non-scanning center view sub-image;

[0011] Calculating the average displacement amount according to the sub-pixel displacement matrix, and calculating the step size adjustment coefficient according to the average displacement amount;

[0012] Update the correct scanning step according to the step adjustment coefficient.

[0013] Preferably, calculate the sub-pixel displacement matrix of all non-scanning center view sub-images relative to the first non-scanning center view sub-image, specifically including:

[0014] Calculate the cross-correlation of all non-scanning center view sub-images and the first non-scanning center view sub-image respectively, and the maximum value of the cross-correlation is the relative displacement of the two images;

[0015]

[0016]

[0017] Among them, , represents the coordinates of two-dimensional scanning, represents the number of scans; , represents the two-dimensional pixel coordinates of the image, , represents the two-dimensional cross-correlation calculation variable; , represents the pixel height and pixel width of the multi-view rearranged image, represents the non-scanning center view sub-image, represents the sub-pixel displacement matrix.

[0018] Preferably, the average displacement in the X-axis direction is:

[0019]

[0020] Among them, represents the x displacement of the Nth column of the sub-pixel displacement matrix, represents the x displacement of the first column of the sub-pixel displacement matrix;

[0021] The average displacement in the Y-axis direction is:

[0022]

[0023] Among them, represents the y displacement of the Nth row of the sub-pixel displacement matrix, represents the y displacement of the first row of the sub-pixel displacement matrix;

[0024] The step adjustment coefficient in the X-axis direction is:

[0025]

[0026] Among them, Represents the average displacement in the X-axis direction;

[0027] Step adjustment coefficient in the Y-axis direction is:

[0028]

[0029] wherein, Represents the average displacement in the Y-axis direction.

[0030] Preferably, the correct scanning step is:

[0031]

[0032] wherein, Represents the correct scanning step in the X-axis direction, Represents the correct scanning step in the Y-axis direction, Represents the initial scanning step in the X-axis direction, Represents the initial scanning step in the Y-axis direction.

[0033] A computer device includes: a memory and a processor. A computer program that can run on the processor is stored in the memory. When the processor executes the computer program, a method for calibrating the step size of a two-dimensional scanning module of a scanning light field imaging system is implemented.

[0034] A computer-readable storage medium has a computer program stored thereon. When the computer program is executed by a processor, a method for calibrating the step size of a two-dimensional scanning module of a scanning light field imaging system is implemented.

[0035] Through the above technical solutions, compared with the prior art, the present invention discloses a method, device, and medium for calibrating the step size of a two-dimensional scanning module of a scanning light field imaging system. By using the sub-pixel registration method, the sub-pixel position offset between multiple scanned images is calculated, and the best scanning step is adjusted according to the calculation result, so as to achieve the highest spatial resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0037] Figure 1 Is a flowchart of a method for calibrating the step size of a two-dimensional scanning module of a scanning light field imaging system provided by the present invention.

[0038] Figure 2 The scanning image displacement matrix under the conditions of step error and correctness of the two-dimensional scanning module provided by the present invention.

[0039] Figure 3 The imaging effects of the scanned images under the conditions that the step size of the two-dimensional scanning module provided by the present invention is too small, too large, and correct. Detailed implementation manners

[0040] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0041] An embodiment of the present invention discloses a method for calibrating the step size of a two-dimensional scanning module of a scanning light field imaging system, as Figure 1 shown, including:

[0042] Collect two-dimensional original scanning light field data with an initial scanning step size, rearrange it into four-dimensional phase space data, and obtain a multi-view rearranged image;

[0043] Select the middle view of all views in the multi-view rearranged image to obtain a non-scanning center view sub-image;

[0044] Calculate the sub-pixel displacement matrix of all non-scanning center view sub-images relative to the first non-scanning center view sub-image;

[0045] Calculate the average displacement amount between scanning images according to the sub-pixel displacement matrix, and calculate the scanning step size adjustment coefficient according to the average displacement amount;

[0046] Set the minimum threshold and maximum threshold for terminating the iteration, and update the scanning step size according to the step size adjustment coefficient.

[0047] If the step size adjustment coefficient is greater than the maximum threshold or less than the minimum threshold, update the scanning step size and repeat the acquisition, calibration, and update steps; if the step size adjustment coefficient is greater than the minimum threshold and less than the maximum threshold, stop the update and obtain the correct scanning step size.

[0048] The scanning light field imaging system includes, but is not limited to: the scanning light field imaging system based on a microlens array and the scanning light field imaging system based on a camera array. In this invention, the scanning light field microscope system based on a microlens is taken as an example. The steps of this invention are described in detail below: In the scanning light field microscopy imaging system, a two-dimensional galvanometer scanning module is added to perform two-dimensional scanning on the sample to be measured, so as to improve the spatial resolution. The two-dimensional galvanometer module realizes scanning imaging through two parameters: the scanning step sizes VX and VY and the number of scans N. Taking the N*N scan as an example, that is, the number of two-dimensional scans, the horizontal scan is N times, the vertical scan is N times, and the total number of scans is N*N. Given that the current scanning step sizes are VX and VY respectively, collect pieces of two-dimensional original scanned light field data. Optionally, a two-dimensional serpentine scanning path is adopted to reduce the mechanical return error. In specific implementation, the horizontal scanning direction scans in the reverse direction in the even rows to form a continuous motion trajectory.

[0049] Rearrange the two-dimensional original scanned light field data into four-dimensional phase space data. The specific steps include: Rearrange the pieces of two-dimensional original scanned light field data to obtain pieces of multi-view rearranged images with the shape of u*v*h*w. Among them, u and v are the index coordinates of the horizontal and vertical views of the multi-view rearranged images, and h and w are the pixel heights and pixel widths of each multi-view rearranged image respectively. Specifically, the longitudinal size and the horizontal size of the two-dimensional light field map are u*h and v*w respectively. For the index coordinates of the fixed view, obtain the pixel values at different spatial coordinates of this view coordinate, and form the spatial coordinate sub-image of this view coordinate; the spatial coordinate sub-images of all view coordinates are arranged in order to jointly form the four-dimensional phase space light field data of the two-dimensional light field map.

[0050] Obtain the non-scanning center view sub-images. The specific steps include: Select the middle view of all views in the multi-view rearranged images, that is, the sub-view image at the u / 2-th row horizontally and the v / 2-th row vertically, and obtain pieces of non-scanning center view sub-images CV with the shape of . Use cross-correlation to calculate the sub-pixel displacement matrix S of the pieces of non-scanning center view sub-images relative to the first non-scanning center view sub-image. The shape and size are . Among them, the sub-pixel displacement matrix characterizes the pixel offset generated between images due to two-dimensional scanning. Use cross-correlation to calculate the pixel offset between image values. The specific steps include: Calculate the cross-correlation of the pieces of non-scanning center view sub-images and the first non-scanning center view sub-image respectively. The maximum value of the cross-correlation is the relative displacement of the two images:

[0051]

[0052]

[0053] Among them, , represents the coordinates of two-dimensional scanning, represents the number of scans; , represents the two-dimensional pixel coordinates of the image, , represents the two-dimensional cross-correlation calculation variable; , represents the pixel height and pixel width of the multi-view rearranged image, represents the sub-image of the non-scanning center view, represents the sub-pixel displacement matrix.

[0054] Optionally, bicubic interpolation is performed in the 5*5 pixel area around the cross-correlation integer pixel peak to improve the accuracy of calculating the sub-pixel displacement matrix.

[0055] Calculate the step adjustment coefficient , , and the specific steps include: calculating

[0056] the average displacement amounts X, Y and the step adjustment coefficient , :

[0057]

[0058]

[0059] Among them, represents the y displacement of the Nth row of the sub-pixel displacement matrix, represents the y displacement of the first row of the sub-pixel displacement matrix; among them, here N has a corresponding relationship with N*N sub-images of the non-scanning center view, that is, there are a total of N*N sub-images of the non-scanning center view, and the first two dimensions corresponding to the sub-pixel displacement matrix are N*N;

[0060]

[0061]

[0062] Among them, represents the x displacement of the Nth column of the sub-pixel displacement matrix, represents the x displacement of the first column of the sub-pixel displacement matrix;

[0063] It should be noted that the sub-pixel displacement matrix between two images represents the displacements of corresponding pixels in the horizontal and vertical directions of the two images. Due to issues such as the accuracy of the displacement stage, the return error, and the calculation accuracy of the algorithm, the scanning step sizes between different scanning positions are not exactly the same. Taking the average value of the displacement amounts at all scanning positions can reduce the error.

[0064] Set the minimum threshold for terminating iteration to 0.9 and the maximum threshold to 1.1, and update the scanning step size according to the step size adjustment coefficient interval. If the step size adjustment coefficient is greater than the maximum threshold or less than the minimum threshold, update the scanning step size and repeat the acquisition, calibration, and update steps; if the step size adjustment coefficient is greater than the minimum threshold and less than the maximum threshold, stop the update and obtain the correct scanning step size.

[0065]

[0066] Figure 2 In [the figure], the left figure shows that there is a large deviation between the displacement matrix (solid line) when the scanning step size is incorrect and the standard displacement matrix (dashed line); the right figure shows that when the method converges, the correct displacement matrix (solid line) and the standard displacement matrix (dashed line) coincide well.

[0067] Figure 3 In [the figure], from left to right are the scanning imaging results under the conditions of a relatively small scanning step size, a correct scanning step size, and a relatively large scanning step size. When the scanning step size is relatively small or relatively large, scanning artifacts will be generated at the edge of the sample to be measured, while when the scanning step size is correct, the edge of the sample to be measured is relatively sharp.

[0068] This embodiment provides a computer device, including: a memory and a processor. A computer program that can run on the processor is stored in the memory. When the processor executes the computer program, a method for calibrating the step size of a two-dimensional scanning module of a scanning light field imaging system is implemented.

[0069] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, a method for calibrating the step size of a two-dimensional scanning module of a scanning light field imaging system is implemented.

[0070] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above method embodiments are executed; and the foregoing storage medium includes: various media such as a removable storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0071] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For related parts, reference can be made to the description in the method section.

[0072] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for calibrating the step size of a two-dimensional scanning module in a scanning light field imaging system, characterized in that, Including: Perform two-dimensional scanning on the sample to be measured, with N horizontal scans and N vertical scans, for a total of N*N scans, and collect N 2 two-dimensional original scanned light field data, and rearrange it into four-dimensional phase space data to obtain a multi-view rearranged image; Select the middle view among all views in the multi-view rearranged image to obtain a non-scanning center view sub-image; Calculate the sub-pixel displacement matrix of all non-scanning center view sub-images relative to the first non-scanning center view sub-image, where the sub-pixel displacement matrix characterizes the pixel offset generated between images due to two-dimensional scanning; Calculate the average displacement according to the sub-pixel displacement matrix, and calculate the step adjustment coefficient according to the average displacement; Update the correct scanning step according to the step adjustment coefficient; Calculate the sub-pixel displacement matrix of all non-scanning center view sub-images relative to the first non-scanning center view sub-image, specifically including: Calculate the cross-correlation between all non-scanning center view sub-images and the first non-scanning center view sub-image respectively, and the maximum value of the cross-correlation is the relative displacement between the two images; Wherein, i, j represent the coordinates of two-dimensional scanning, N represents the number of scans; x, y represent the two-dimensional pixel coordinates of the image, m, n represent the two-dimensional cross-correlation calculation variables; h, w represent the pixel height and width of the multi-view rearranged image, CV(.) represents the non-scanning center view sub-image, and S(i, j) represents the sub-pixel displacement matrix; The average displacement in the X-axis direction is: Wherein, S[N, :, 1] represents the x displacement of the Nth column of the sub-pixel displacement matrix, and S[1, :, 1] represents the x displacement of the first column of the sub-pixel displacement matrix; The average displacement in the Y-axis direction is: Wherein, S[:, N, 2] represents the y displacement of the Nth row of the sub-pixel displacement matrix, and S[:, 1, 2] represents the y displacement of the first row of the sub-pixel displacement matrix; The step adjustment coefficient KX in the X-axis direction is: Wherein, X represents the average displacement in the X-axis direction; The step adjustment coefficient KY in the Y-axis direction is: Wherein, Y represents the average displacement in the Y-axis direction.

2. The step calibration method for the two-dimensional scanning module of a scanning light field imaging system according to claim 1, characterized in that, The correct scanning step is: SX = VX * KX SY = VY * KY Wherein, SX represents the correct scanning step in the X-axis direction, SY represents the correct scanning step in the Y-axis direction, VX represents the initial scanning step in the X-axis direction, and VY represents the initial scanning step in the Y-axis direction.

3. A computer device, characterized in that, Including: A memory and a processor, wherein the memory stores a computer program that can run on the processor, and when the processor executes the computer program, the method described in claim 1 or 2 is implemented.

Images