Method and device for calibrating step length of two-dimensional scanning module of scanning light field imaging system and medium

Abstract

The invention discloses a two-dimensional scanning module step length calibration method and device of a scanning light field imaging system and a medium, and relates to the technical field of light field microscopic imaging, and the method comprises the steps: collecting two-dimensional original scanning light field data according to an initial scanning step length, and carrying out the rearrangement of the two-dimensional original scanning light field data into four-dimensional phase space data, and obtaining a multi-view rearrangement image; selecting a middle view angle of all view angles in the multi-view-angle rearrangement image to obtain a non-scanning center view angle sub-image; calculating a sub-pixel displacement matrix of all non-scanning center view angle sub-images relative to the first non-scanning center view angle sub-image; calculating an average displacement according to the sub-pixel displacement matrix, and calculating a step length adjustment coefficient according to the average displacement; according to the method, the scanning step length of two-dimensional scanning imaging is calibrated at a higher speed and higher precision, and the spatial resolution of scanning imaging is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of light field microscopic imaging, and more particularly to a method, device and medium for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system. Background Art

[0002] In the light field microscopy system, microlenses are used to achieve multi-view light field imaging, sacrificing a certain amount of spatial resolution to obtain the angle information of the light. The three-dimensional tomography results of the sample can be obtained through the phase space reconstruction method. On the premise of ensuring the angular resolution, in order to improve the lateral resolution of the light field microscopy system, a two-dimensional galvanometer is added to perform two-dimensional scanning of the sample, realizing time-space multiplexing scanning imaging. The images obtained by multiple scans are spliced ​​according to the scanning position to obtain images with higher spatial resolution.

[0003] Taking 3*3 scanning as an example, the scanning step length needs to be strictly controlled to 1 / 3 of the physical size of a single microlens in order to effectively improve the resolution of the scanned images after stitching. A step length that is too large will lead to data redundancy and fail to improve the resolution; a step length that is too small will lead to discontinuous images after stitching. Therefore, obtaining the optimal scanning step length is crucial to improving the spatial resolution of scanning light field imaging.

[0004] Therefore, how to provide a method for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0005] In view of this, the present invention provides a method, device and medium for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system, which can calibrate the scanning step length of two-dimensional scanning imaging at a faster speed and higher accuracy, thereby improving the spatial resolution of the scanning imaging.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A method for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system, comprising:

[0008] Collecting two-dimensional raw scanning light field data with an initial scanning step length, and rearrange it into four-dimensional phase space data to obtain a multi-view rearranged image;

[0009] Selecting the middle viewing angle of all viewing angles in the multi-view rearranged image to obtain a non-scanned central viewing angle sub-image;

[0010] Calculate the sub-pixel displacement matrix of all non-scanned central viewing angle sub-images relative to the first non-scanned central viewing angle sub-image;

[0011] Calculate the average displacement according to the sub-pixel displacement matrix, and calculate the step size adjustment coefficient according to the average displacement;

[0012] Update the correct scan step size according to the step size adjustment factor.

[0013] Preferably, calculating the sub-pixel displacement matrix of all non-scanning central viewing angle sub-images relative to the first non-scanning central viewing angle sub-image specifically includes:

[0014] Calculate the cross-correlation between all non-scanned central viewing angle sub-images and the first non-scanned central viewing angle sub-image respectively, and the maximum value of the cross-correlation is the relative displacement of the two images;

[0015]

[0016]

[0017] in, , represents the coordinates of the two-dimensional scan, Indicates the number of scans; , represents the two-dimensional pixel coordinates of the image, , represents the two-dimensional cross-correlation calculation variable; , Represents the pixel height and pixel width of the multi-view rearranged image, represents the non-scanned central viewing angle sub-image, Represents the sub-pixel displacement matrix.

[0018] Preferably, the average displacement in the X-axis direction is:

[0019]

[0020] in, represents the x displacement of the Nth column of the sub-pixel displacement matrix, represents the x displacement of the first column of the sub-pixel displacement matrix;

[0021] The average displacement in the Y-axis direction is:

[0022]

[0023] in, represents the y displacement of the Nth row of the sub-pixel displacement matrix, represents the y displacement of the first row of the sub-pixel displacement matrix;

[0024] X-axis step adjustment factor for:

[0025]

[0026] in, Indicates the average displacement in the X-axis direction;

[0027] Y-axis step adjustment factor for:

[0028]

[0029] in, Indicates the average displacement in the Y-axis direction.

[0030] Preferably, the correct scanning step length is:

[0031]

[0032] in, Indicates the correct scanning step length in the X-axis direction, Indicates the correct scanning step length in the Y-axis direction, represents the initial scanning step length in the X-axis direction, Indicates the initial scan step size in the Y-axis direction.

[0033] A computer device comprises: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, a step length calibration method for a two-dimensional scanning module of a scanning light field imaging system is implemented.

[0034] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, a method for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system is implemented.

[0035] It can be seen from the above technical solution that compared with the prior art, the present invention discloses a method, device and medium for calibrating the step size of a two-dimensional scanning module of a scanning light field imaging system. The sub-pixel position offset between multiple scanned images is calculated by using a sub-pixel registration method, and the optimal scanning step size is adjusted according to the calculation results to achieve the highest spatial resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 A flow chart of a step length calibration method for a two-dimensional scanning module of a scanning light field imaging system provided by the present invention.

[0038] Figure 2 The present invention provides a scanning image displacement matrix when the step size of the two-dimensional scanning module is wrong or correct.

[0039] Figure 3 The imaging effect of the two-dimensional scanning module provided by the present invention when the step size is too small, too large, or correct when scanning an image. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The embodiment of the present invention discloses a method for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system. Figure 1 As shown, including:

[0042] Collecting two-dimensional raw scanning light field data with an initial scanning step length, and rearrange it into four-dimensional phase space data to obtain a multi-view rearranged image;

[0043] Selecting the middle viewing angle of all viewing angles in the multi-view rearranged image to obtain a non-scanned central viewing angle sub-image;

[0044] Calculate the sub-pixel displacement matrix of all non-scanned central viewing angle sub-images relative to the first non-scanned central viewing angle sub-image;

[0045] Calculate the average displacement between scanned images according to the sub-pixel displacement matrix, and calculate the scanning step adjustment coefficient according to the average displacement;

[0046] Set the minimum and maximum thresholds for terminating iterations, and update the scanning step size according to the step size adjustment coefficient.

[0047] If the step adjustment coefficient is greater than the maximum threshold or less than the minimum threshold, the scanning step is updated and the acquisition, calibration and update steps are repeated; if the step adjustment coefficient is greater than the minimum threshold and less than the maximum threshold, the update is stopped to obtain the correct scanning step.

[0048] The scanning light field imaging system includes but is not limited to: a scanning light field imaging system based on a microlens array, and a scanning light field imaging system based on a camera array. The present invention takes a scanning light field microscope system based on a microlens as an embodiment. The steps of the present invention are described in detail below: In the scanning light field microscope imaging system, a two-dimensional galvanometer scanning module is added to perform two-dimensional scanning on the sample to be tested to improve the spatial resolution. The two-dimensional galvanometer module realizes scanning imaging through two parameters: scanning step length VX, VY and the number of scans N. Taking N*N scanning as an example, that is, the number of two-dimensional scans, N horizontal scans, N vertical scans, a total of N*N scans, it is known that the current scanning step lengths are VX and VY respectively, and the acquisition The two-dimensional raw scanning light field data is optionally scanned in a two-dimensional serpentine scanning path to reduce the mechanical return error. In a specific implementation, the transverse scanning direction is reversely scanned in the even-numbered rows to form a continuous motion trajectory.

[0049] The two-dimensional original scanning light field data is rearranged into four-dimensional phase space data. The specific steps include: The original two-dimensional scanning light field data is rearranged to obtain A multi-view rearranged image of shape u*v*h*w, where u and v are the index coordinates of the horizontal and vertical view of the multi-view rearranged image, and h and w are the pixel height and pixel width of each multi-view rearranged image. Specifically, the vertical and horizontal dimensions of the two-dimensional light field image are u*h and v*w, respectively, where the index coordinate of the fixed view is used to obtain the pixel value of the view coordinate at different spatial coordinates to form the spatial coordinate sub-image of the view coordinate; the spatial coordinate sub-images of all view coordinates are arranged in order to form the four-dimensional phase space light field data of the two-dimensional light field image.

[0050] The non-scanning central view sub-image is obtained, and the specific steps include: selecting the middle view of all the views in the multi-view rearrangement image, that is, the sub-view image of the horizontal u / 2 and the vertical v / 2, and obtaining A non-scanned central-view sub-image CV, the shape is . Use cross-correlation to calculate The sub-pixel displacement matrix S of the first non-scanned central viewing angle sub-image relative to the first non-scanned central viewing angle sub-image is of shape and size: , where the sub-pixel displacement matrix represents the pixel offset between images caused by two-dimensional scanning. The pixel offset between image values ​​is calculated using cross-correlation. The specific steps include: The maximum value of the cross-correlation is the relative displacement of the two images:

[0051]

[0052]

[0053] in, , represents the coordinates of the two-dimensional scan, Indicates the number of scans; , represents the two-dimensional pixel coordinates of the image, , represents the two-dimensional cross-correlation calculation variable; , Represents the pixel height and pixel width of the multi-view rearranged image, represents the non-scanned central viewing angle sub-image, Represents the sub-pixel displacement matrix.

[0054] Optionally, bicubic interpolation is performed in a 5*5 pixel area around the cross-correlation integer pixel peak to improve the accuracy of calculating the sub-pixel displacement matrix.

[0055] Calculate the step size adjustment factor , The specific steps include: calculating according to the sub-pixel displacement matrix S

[0056] Average displacement X, Y and step size adjustment factor , :

[0057]

[0058]

[0059] in, represents the y displacement of the Nth row of the sub-pixel displacement matrix, represents the y displacement of the first row of the sub-pixel displacement matrix; where N corresponds to the N*N non-scanning central view sub-images, that is, there are a total of N*N non-scanning central view sub-images, and the first two dimensions of the corresponding sub-pixel displacement matrix are N*N;

[0060]

[0061]

[0062] in, represents the x displacement of the Nth column of the sub-pixel displacement matrix, represents the x displacement of the first column of the sub-pixel displacement matrix;

[0063] It should be noted that the sub-pixel displacement matrix between the two images represents the displacement of the corresponding pixels in the horizontal and vertical directions of the two images. Due to the accuracy of the translation stage, the return error and the calculation accuracy of the algorithm, the scanning step lengths between different scanning positions are not exactly the same. Taking the average of the displacements of all scanning positions can reduce the error.

[0064] Set the minimum threshold for terminating iteration to 0.9 and the maximum threshold to 1.1, and update the scanning step length according to the step length adjustment coefficient interval. If the step length adjustment coefficient is greater than the maximum threshold or less than the minimum threshold, update the scanning step length and repeat the acquisition, calibration and update steps; if the step length adjustment coefficient is greater than the minimum threshold and less than the maximum threshold, stop updating and obtain the correct scanning step length.

[0065]

[0066] Figure 2 In the figure, the left figure shows that when the scanning step size is wrong, there is a large deviation between the displacement matrix (solid line) and the standard displacement matrix (dashed line); the right figure shows that when the method converges, the correct displacement matrix (solid line) and the standard displacement matrix (dashed line) are in good agreement.

[0067] Figure 3 In the figure, from left to right are the scanning imaging results when the scanning step length is too small, correct, and too large. When the scanning step length is too small or too large, scanning artifacts will be generated at the edge of the sample to be tested, while when the scanning step length is correct, the edge of the sample to be tested is sharper.

[0068] This embodiment provides a computer device, including: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, a method for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system is implemented.

[0069] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, a method for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system is implemented.

[0070] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, etc. Various media that can store program codes.

[0071] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0072] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system, characterized in that: include: Collecting two-dimensional raw scanning light field data with an initial scanning step length, and rearrange it into four-dimensional phase space data to obtain a multi-view rearranged image; Selecting the middle viewing angle of all viewing angles in the multi-view rearranged image to obtain a non-scanned central viewing angle sub-image; Calculate the sub-pixel displacement matrix of all non-scanned central viewing angle sub-images relative to the first non-scanned central viewing angle sub-image; Calculate the average displacement according to the sub-pixel displacement matrix, and calculate the step size adjustment coefficient according to the average displacement; Update the correct scan step size according to the step size adjustment factor.

2. The method for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system according to claim 1, characterized in that: Calculate the sub-pixel displacement matrix of all non-scanned central view sub-images relative to the first non-scanned central view sub-image, specifically including: Calculate the cross-correlation between all non-scanned central viewing angle sub-images and the first non-scanned central viewing angle sub-image respectively, and the maximum value of the cross-correlation is the relative displacement of the two images; ； ； in, , represents the coordinates of the two-dimensional scan, Indicates the number of scans; , represents the two-dimensional pixel coordinates of the image, , represents the two-dimensional cross-correlation calculation variable; , Represents the pixel height and pixel width of the multi-view rearranged image, represents the non-scanned central viewing angle sub-image, Represents the sub-pixel displacement matrix.

3. The method for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system according to claim 1, characterized in that: The average displacement in the X-axis direction is: ； in, represents the x displacement of the Nth column of the sub-pixel displacement matrix, represents the x displacement of the first column of the sub-pixel displacement matrix; The average displacement in the Y-axis direction is: ； in, represents the y displacement of the Nth row of the sub-pixel displacement matrix, represents the y displacement of the first row of the sub-pixel displacement matrix; X-axis step adjustment factor for: ； in, Indicates the average displacement in the X-axis direction; Y-axis step adjustment factor for: ； in, Indicates the average displacement in the Y-axis direction.

4. The method for calibrating the step length of a two-dimensional scanning module of a scanning light field imaging system according to claim 3, characterized in that: The correct scanning step size is: ； in, Indicates the correct scanning step length in the X-axis direction, Indicates the correct scanning step length in the Y-axis direction, represents the initial scanning step length in the X-axis direction, Indicates the initial scan step size in the Y-axis direction.

5. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 4.

Images