An optimization method and device for scanning motion artifacts in light field imaging

Abstract

The application discloses a kind of optimization method and device for motion artifact in scanning light field imaging, the method includes the following steps: obtaining the single-frame original light field image to be optimized;Original light field image is rearranged as space-angle image according to space and angle information by pixel rearrangement;Periodic scanning is carried out to space-angle image, and subpixel in different space positions and angle directions is obtained, and pixel is realigned;According to scanning order, the image after realignment is disassembled, and multiple space-angle subgraphs in time dimension are obtained;Space-angle subgraph is input into the scanning correction network trained, image correction fusion is carried out, and high-resolution space-angle image removing motion artifact is obtained;Scanning correction network is based on neural network model construction.The application is processed by pixel rearrangement, periodic scanning, pixel realignment and disassembly, and then input into scanning correction network, while eliminating motion artifact, the higher spatial resolution of image is maintained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of scanning light field imaging, and more particularly to a method and device for optimizing motion artifacts in scanning light field imaging. BACKGROUND

[0002] At present, light field microscopy has become a representative technology of high-speed three-dimensional microscopic imaging, but its low spatial resolution limits its application in biomedical imaging.

[0003] The scanning light field microscope successfully improves the spatial resolution to the vicinity of the diffraction limit by periodically moving the image plane with a step size smaller than the pitch of each microlens through a two-dimensional galvanometer system. However, the physical scanning process inevitably reduces the three-dimensional imaging speed. When the sample moves rapidly or the laser energy output is unstable, resulting in intensity changes, motion artifacts will be introduced into the image after pixel rearrangement, thereby affecting the reconstruction quality.

[0004] The prior art uses a time-weighted algorithm to weaken motion artifacts. The time-weighted algorithm essentially utilizes the information of adjacent time points by weighted summation. For the fast-moving part of the image, the time-weighted algorithm can effectively eliminate motion artifacts. However, for the relatively stationary part of the image, the time-weighted algorithm will introduce a certain degree of blurring, resulting in a decrease in the spatial resolution of the image.

[0005] Therefore, it is an urgent problem for those skilled in the art to provide a method for optimizing motion artifacts in scanning light field imaging, which eliminates motion artifacts while maintaining high spatial resolution. SUMMARY

[0006] Therefore, the present application provides a method and device for optimizing motion artifacts in scanning light field imaging, which processes images through pixel rearrangement, periodic scanning, pixel realignment, and disassembly, and corrects images through a scanning correction network, thereby eliminating motion artifacts while maintaining high spatial resolution of the images.

[0007] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0008] In a first aspect, the present application provides a method for optimizing motion artifacts in scanning light field imaging, comprising the following steps:

[0009] S1, obtaining a single-frame original light field image to be optimized; wherein the original light field image contains motion artifacts;

[0010] S2, rearranging the original light field image into a spatial-angle image according to spatial and angular information through pixel rearrangement;

[0011] S3, periodically scanning the space-angle image to obtain sub-pixels of different spatial positions and angle directions, and performing pixel realignment;

[0012] S4, disassembling the realigned image according to the scanning order to obtain multiple space-angle sub-images in the time dimension;

[0013] S5, inputting the space-angle sub-images into a trained scanning correction network to perform image correction fusion to obtain a high-resolution space-angle image with motion artifacts removed; the scanning correction network is constructed based on a neural network model.

[0014] Further, step S2 specifically includes:

[0015] The original light field image Nh×Nw is rearranged into a space-angle image of N 2 ×h×w through pixel rearrangement; wherein N 2 represents the angle resolution of the light field, h and w represent the height and width of a single sub-aperture image respectively; the space-angle image contains spatial information and angle information.

[0016] Further, step S3 specifically includes:

[0017] The space-angle image is periodically scanned by X×Y; wherein X represents an equal-interval mirror scanning displacement in the horizontal direction, and Y represents an equal-interval mirror scanning displacement in the vertical direction;

[0018] During the scanning process, each pixel on the sensor corresponds to different spatial positions and angle directions;

[0019] The pixels of different sub-pixel scanning positions are realigned.

[0020] Further, step S4 specifically includes:

[0021] The realigned image is disassembled in the time dimension according to the scanning order to obtain X×Y space-angle sub-images of different spatial positions and angle directions in the time dimension.

[0022] Further, the processing process of the scanning correction network includes:

[0023] Multiple space-angle sub-images are input into the scanning correction network, and single-frame multi-channel information of each sub-image is extracted layer by layer through a feature extraction module;

[0024] The multi-channel information of the continuous sub-images is interactively fused by using a feature fusion module based on a channel attention mechanism;

[0025] The fused features are remapped and convolutionally fused to obtain a single-frame single-channel output image.

[0026] Further, the training process of the scanning correction network comprises:

[0027] adding interference factors in the open-source dataset to construct a training data pair; the interference factors include light source instability and / or inaccurate scanning position;

[0028] training the model using the training data pair and constructing a model total loss to optimize the model;

[0029] fine-tuning the trained model using a microscopic dataset to obtain a final model.

[0030] Further, the open-source dataset is a REDS dataset in the biomedical imaging field;

[0031] the microscopic dataset is microscopic data photographed by a small number of users using a scanning light field microscope.

[0032] Further, the model total loss comprises a regular loss and a high-frequency loss;

[0033] the regular loss is directly calculated from a model predicted image and a real image;

[0034] the high-frequency loss is calculated from a high-pass filtered model predicted image and a high-pass filtered real image.

[0035] In a second aspect, the present application provides an optimization device for motion artifacts in scanning light field imaging, which applies the optimization method of any one of the first aspect, comprising the following modules:

[0036] an acquisition module: acquiring a single-frame original light field image to be optimized; wherein the original light field image contains motion artifacts;

[0037] a pixel rearrangement module: rearranging the original light field image into a spatial-angular image according to spatial and angular information through pixel rearrangement;

[0038] a pixel realignment module: performing periodic scanning on the spatial-angular image to obtain sub-pixels at different spatial positions and angular directions, and performing pixel realignment;

[0039] a disassembly module: disassembling the realigned image according to the scanning order to obtain multiple spatial-angular sub-images in the time dimension;

[0040] a motion artifact removal module: inputting the spatial-angular sub-images into a trained scanning correction network to perform image correction fusion and obtain a high-resolution spatial-angular image with removed motion artifacts; the scanning correction network is constructed based on a neural network model.

[0041] Compared with the prior art, the scanning light field imaging motion artifact optimization method and device provided by the application has the following beneficial effects:

[0042] The application fully utilizes the characteristics and time correlation of periodic scanning, combines a scanning correction network, and better mines the time correlation of the static part of the scene by modeling the relative motion between the camera and the scene.

[0043] The application reextracts the high-resolution space-angle image after the pixel realignment of the motion artifact, and generates a high-resolution space-angle image without motion artifacts through a neural network.

[0044] Since the motion artifact exists in the high-frequency region of the image frequency domain, the model output image and the reference image are high-pass filtered to calculate the high-frequency loss, which is added to the total loss function to further guide the network to identify and suppress the motion artifact. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0046] Figure 1 The application provides a scanning light field imaging motion artifact optimization method flowchart.

[0047] Figure 2 The application provides a pixel rearrangement schematic diagram.

[0048] Figure 3 The application provides a pixel realignment schematic diagram.

[0049] Figure 4 The application provides a scanning correction network framework diagram.

[0050] Figure 5 The application provides a model total loss display diagram.

[0051] Figure 6 The application provides a scanning correction network and time weighting algorithm image processing result comparison diagram.

[0052] Figure 7A scanning light field imaging motion artifact optimization device structure diagram is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of the present application.

[0054] Embodiment 1

[0055] The embodiments of the present application disclose a scanning light field imaging motion artifact optimization method, comprising the following steps:

[0056] S1, acquiring a single-frame original light field image to be optimized; wherein the original light field image contains motion artifacts;

[0057] S2, rearranging the original light field image according to spatial and angular information through pixel rearrangement to obtain a spatial-angular image;

[0058] S3, periodically scanning the spatial-angular image to obtain sub-pixels at different spatial positions and angular directions, and performing pixel realignment;

[0059] S4, disassembling the realigned image according to the scanning order to obtain multiple spatial-angular sub-images in the time dimension;

[0060] S5, inputting the spatial-angular sub-images into a trained scanning correction network to perform image correction fusion, and obtaining a high-resolution spatial-angular image with motion artifacts removed; the scanning correction network is constructed based on a neural network model.

[0061] A light field microscope inserts a microlens array in a traditional wide-field microscopic light path, so that the spatial information and angular information of a three-dimensional sample are simultaneously encoded into a single 2D snapshot, realizing fast volume imaging at a camera frame rate. However, the trade-off between spatial information and angular information leads to a decrease in the spatial resolution of the light field microscope, which in turn limits its application in biomedicine.

[0062] In the field of biomedical imaging, the embodiments of the present application use a scanning light field microscope to perform three-dimensional imaging of cells, increase the spatial sampling density by moving the image plane for periodic scanning, so that the spatial resolution is improved to near the diffraction limit, and realize high-resolution, multi-angle cell images. Referring to Figure 1 As shown in the figure, in the first step, an original light field image with a size of Nh×Nw is acquired by an image acquisition device; in the second step, the original light field image is pixel rearranged to obtain a spatial-angular image with a size of2 XhXw spatial-angular image; where, N 2 represents the angular resolution of the light field, h and w represent the spatial size of a single sub-aperture image; the third step, through the XxY periodic scanning of the scanning galvanometer, the pixel realignment is generated N 2 XhXw high-resolution spatial-angular image; the fourth step, according to the scanning order, it is decomposed into N 2 XxYxhXw low-resolution spatial-angular image in time dimension, and finally the fifth step, input the trained scanning correction neural network for reconstruction, get N 2 XhXw high-resolution spatial-angular image without artifacts.

[0063] In this embodiment, the optimal scanning effect is X=3, Y=3. X represents X times of equal interval galvanometer scanning displacement in the x-axis direction, and each displacement is 1 / X of the microlens pitch; Y represents Y times of equal interval galvanometer scanning displacement in the y-axis direction, and each displacement is 1 / Y of the microlens pitch.

[0064] The steps of this embodiment will be described in detail below.

[0065] The first step is to collect a single original light field image with a size of NhXNw through an image acquisition device.

[0066] The image acquisition device is used to capture and record sample images under a microscope. It has the characteristics of high resolution, high sensitivity and high-speed acquisition, and common types include scientific cameras, industrial cameras, CCD cameras and CMOS cameras, etc.

[0067] The image acquisition device of this embodiment is usually used in cooperation with the optical system of the microscope to capture light field images encoded by the microlens array or pinhole array.

[0068] For the light field microscope system with an angle resolution of NxN in this embodiment, the size of the single-frame original light field image collected by the camera is NhXNw.

[0069] The second step, as shown in Figure 2 , the original light field image is rearranged into N 2 XhXw spatial-angular image through pixel rearrangement.

[0070] Figure 2 The left part of the figure in Figure 2 is a 6x6 matrix, which represents a part of the single-frame original light field image collected by the camera. Figure 2 Each number (1, 2, 3, 4) in Figure 2 actually represents different sub-aperture images, and the color helps to distinguish the position of these sub-aperture images in the original light field image. In this embodiment, N=2, i.e. the angle resolution is 2x2.

[0071] The pixels in the original light field image are rearranged according to a rule to form a space-angle image. In this process, each layer ultimately represents a sub-aperture image at a single angle.

[0072] Figure 2 The right portion of the image shows the rearranged image, forming a 4x3x3 spatial-angular image. Here, the first dimension (the 4-dimensional portion) represents angular information, while the second and third dimensions (the 3x3-dimensional portion) represent spatial information. Pixels from the same subaperture image, originally scattered throughout the original light field image, are clustered together to form a clear subaperture image layer.

[0073] Each layer represents a sub-aperture image at a specific angle. Figure 2 Marked with different colors; Figure 2 All the red 1s in the image are rearranged to the same layer to form a complete sub-aperture image. Similarly, the blue 2s, green 3s, and yellow 4s also form their corresponding sub-aperture image layers, and finally the image size is N. 2 ×h×w spatial-angular image.

[0074] Step 3: Refer to Figure 3 As shown, a 3×3 periodic scan is performed by the scanning galvanometer, and a pixel size of N is generated after pixel realignment. 2 ×3h×3w high-resolution spatial-angular image.

[0075] Figure 3 The left portion of the figure is a 3×3 sub-aperture image from a single perspective, where each number 1 represents a portion of the sub-aperture image at a specific angle obtained in step 2. In this embodiment, N=2, meaning the angular resolution is 2x2.

[0076] Next, the system performs a 3x3 periodic scan with a step size smaller than the pitch of each microlens. During this scanning process, each pixel on the sensor corresponds to a different spatial position and angular orientation. This scanning method captures finer spatial information, thereby improving the resolution of the final image.

[0077] Figure 3 The right part of the figure is the result of pixel realignment after sub-pixel scanning. Figure 3 It can be seen that each original "1" is expanded into a 3x3 matrix, forming a larger image block. This process is to rearrange and align the pixels at different sub-pixel scanning positions to generate a higher resolution space-angle image; the image size is N 23h x 3w, in this embodiment, 4 x 9 x 9. Each layer still represents a sub-aperture image at a single angle, but the size of each sub-aperture image is changed from the original h x w to 3h x 3w, thus achieving the improvement of spatial resolution.

[0078] In this embodiment, the original spatial-angle image is converted into a high-resolution spatial-angle image through sub-pixel scanning and pixel realignment. This process not only preserves the original angle information, but also significantly improves the spatial resolution, enabling subsequent light field microscopic analysis and reconstruction to obtain clearer and more detailed images.

[0079] Finally, the high-resolution three-dimensional image of the sample can be obtained by Richardson-Lucy deconvolution algorithm, and the three-dimensional structure of the sample can be observed. However, when shooting high dynamic samples, the motion of the sample during scanning, the instability of the laser light source, sample bleaching, and the instability of the scanning galvanometer will introduce motion artifacts, and the artifacts will be amplified during volume reconstruction, thus reducing the image quality of volume reconstruction.

[0080] According to the fourth step, the low-resolution spatial-angle image of N 2 x 9 x h x w is decomposed into N

[0081] The third step obtains a higher-resolution spatial-angle image of N 2 x 3h x 3w; in order to facilitate processing and effectively eliminate motion artifacts, the 3h x 3w sub-aperture image at each angle is further decomposed into 9 low-resolution sub-images. Each sub-image corresponds to the result of one sub-pixel scan, and the size is h x w. This means that the original large image at one angle is divided into 9 smaller parts, which are arranged according to the step and order of scanning.

[0082] During the decomposition process, the arrangement order of the sub-images is determined according to the actual sub-pixel scanning path. In this embodiment, the scanning order involves a systematic scanning mode, including from left to right and from top to bottom. The original sub-aperture image size in this embodiment is 3h x 3w, which is divided into 3 rows and 3 columns, a total of 9 small blocks, each with a size of h x w. The 9 small blocks represent 9 consecutive sub-pixel scanning results, and each small block contains spatial information at a specific position.

[0083] Finally, according to the fifth step, the trained scanning correction neural network is input for reconstruction, and a high-resolution spatial-angle image of N 2 x 3h x 3w is obtained, which eliminates artifacts.

[0084] In this embodiment, a scanning correction network is built, which refers to Figure 4As shown, the single-frame multi-channel information N of each scanning frame is extracted layer by layer by the feature extraction module 2 ×9×c×h×w, and the multi-channel information of the continuous 9 scanning frames is fully interacted and fused by the feature fusion module based on the channel attention mechanism N 2 ×9c×h×w, and finally the features are remapped to the initial position N 2 ×c×3h×3w, and are fused by convolution into a single-frame single-channel final output image N 2 ×3h×3w.

[0085] The feature extraction module uses a convolutional neural network to extract multi-channel information of each scanning frame layer by layer. Specifically, for an input of N 2 ×9×h×w, after feature extraction, a feature map of N 2 ×9×c×h×w is obtained, where c represents the number of extracted feature channels.

[0086] In this embodiment, the multi-channel feature information is fused by stacking the feature fusion module, which includes a series of convolutional layers, activation functions and channel attention layers to gradually fuse the features; the convolutional layers use multiple 3×3 convolutional kernels to extract features in local regions and fuse features between different channels. The activation function adopts ReLU (Rectified Linear Unit) or Leaky ReLU, etc., which is used to introduce non-linear characteristics to enable the network to learn more complex feature representations. The activation function acts on the output of the convolutional layer to enhance the expression ability of the model. By adding the channel attention layer, the importance weights of different channels of the feature map are dynamically learned, and the feature responses of key channels are enhanced to better fuse the feature information between different channels.

[0087] The feature fusion of this embodiment is stacked by a feature fusion module composed of multiple layers of convolution, activation function and channel attention layer. With the increase of the number of stacked layers, the multi-scale feature information between different channels is fully fused. After the above convolutional neural network processing, the input N²×9×h×w image is converted into a feature map of N²×9×c×h×w. Finally, the motion artifact correction of the image is completed by the upsampling module, and a high-resolution motion artifact-free image of N 2 ×3h×3w is obtained.

[0088] The training process of the scanning correction network in this embodiment is carried out in a supervised learning manner, aiming to optimize the neural network parameters so that the network can accurately identify and remove the motion artifacts caused by sample motion, especially those artifacts concentrated in the high-frequency region of the image Fourier transform. Therefore, when designing the loss function, a high-frequency component constraint term high-frequency loss is introduced to guide the model to pay more attention to the high-frequency noise in the image and effectively suppress its influence.

[0089] The training process of the scanning correction network includes:

[0090] The data set is constructed, and the output image of the data set is a high-quality reference image without motion artifacts. In this embodiment, it is obtained by multi-frame averaging or static sample acquisition. The input image is a plurality of sub-images of different spatial positions and angular directions disassembled from each output image, which is a low-resolution sub-image of N2x9xhxw and contains motion artifacts.

[0091] In order to more effectively remove motion artifacts, especially the artifacts concentrated in the high-frequency region of the image, this embodiment adopts a double-path loss function; as shown in Figure 5 , including a conventional pixel-level loss and a high-frequency loss.

[0092] The conventional pixel-level loss, i.e., the overall loss, is used to measure the difference between the network output image and the real label GT image; the high-frequency loss takes into account that motion artifacts mainly exist in the high-frequency region of the image, and introduces a loss term based on a high-pass filter, respectively. Two-dimensional Fourier transform (FFT) is performed, a high-pass filter (such as ideal high-pass, Butterworth, Gaussian HPF) is applied in the frequency domain, and the high-frequency component is retained; the filtered image is converted back to the spatial domain; the difference between the two is calculated as the high-frequency loss. Figure 5 In the formula, the gain factor σ is set to 1.25, and the intensity of the high-frequency component of the image after high-pass filtering will increase by 25%; this loss term forces the network to pay special attention to eliminating high-frequency noise and artifacts in the image during the training process.

[0093] This embodiment also uses a macro data set to fully pre-train the scanning correction network, so that it can correct and combine the b x 9 x h x w multi-frame continuous shooting images into a b x 3h x 3w high-resolution image, where b represents the batch size of the network single run processing. Then use a small amount of micro data set to fine-tune the pre-trained model, so that it has better effect in microscopic imaging.

[0094] This embodiment reassembles the N 2 x 3h x 3w high-resolution spatial-angle image obtained by pixel realignment into N 2 x 9 x h x w low-resolution spatial-angle images in the time dimension according to the scanning order of different positions. When inputting the neural network, it will be converted into a tensor of b x N 2 x 9 x h x w size, this embodiment combines the angle resolution N 2 into the batch dimension to obtain bN 2a tensor of size 9xh xw, so that the neural network will only utilize the information of the time dimension but not the information between different views to correct each sub-aperture image, and the multi-view information will be fused in the downstream 3D reconstruction.

[0095] The dimension-adjusted tensor is input into the trained neural network for reconstruction to obtain N 2 high-resolution spatial-angle images of size 3x3h x3w.

[0096] The embodiment is also compared with the method of weakening motion artifacts by time weighting algorithm; as shown in Figure 6 , in the time weighting algorithm, the spatial resolution and the motion artifact removal effect can be balanced by adjusting the time weighting coefficient r. When r is equal to 1, it is equivalent to using only the single-frame original data of the current time point t0, and the motion artifacts cannot be eliminated. When r is equal to 0, it is equivalent to using only the interpolation result of the adjacent frame, completely ignoring the original data of the current frame t0, which can effectively eliminate the motion artifacts but leads to a significant reduction in spatial resolution. In contrast, the scanning correction network proposed in the embodiment can not only effectively eliminate the motion artifacts, but also ensure higher spatial resolution. At the same time, since the noise factor is considered in training the network in the embodiment, the scanning correction network can also remove the noise effect at the same time.

[0097] The experimental results show that the deep learning scanning correction result significantly reduces the motion artifacts compared with simple pixel realignment, while maintaining a relatively high spatial resolution.

[0098] The scanning light field microscope can improve the spatial sampling density by 3x3 periodic scanning, and improve the static imaging resolution to near the diffraction limit. However, if there is a strong morphological or intensity change in the sample during the scanning process of 9 camera frames, the problem of motion artifacts will be introduced. The time weighting algorithm can use the information of adjacent time points to eliminate motion artifacts, but at the same time it will also cause a reduction in spatial resolution. The present application aims to solve the above problems existing in the time weighting algorithm, while eliminating motion artifacts and maintaining a relatively high spatial resolution.

[0099] Embodiment 2

[0100] The embodiment of the application discloses an optimization device for motion artifacts in scanning light field imaging, which applies the optimization method of embodiment 1, and refers to Figure 7 , which includes the following modules:

[0101] The acquisition module acquires a single-frame original light field image to be optimized; wherein the original light field image contains motion artifacts.

[0102] The pixel rearrangement module rearranges the original light field image into a spatial-angle image according to spatial and angular information through pixel rearrangement.

[0103] pixel realignment module: periodically scanning the spatial-angle image to obtain sub-pixels at different spatial positions and angle directions, and performing pixel realignment;

[0104] disassembly module: disassembling the realigned image according to the scanning order to obtain multiple spatial-angle sub-images in the time dimension;

[0105] motion artifact removal module: inputting the spatial-angle sub-images into a trained scan correction network to perform image correction fusion, and obtaining a high-resolution spatial-angle image with removed motion artifacts; the scan correction network is constructed based on a neural network model.

[0106] The device of the embodiment is applied in the field of biomedical imaging, fully utilizes the characteristics and time correlation of fixed sub-pixel offset machine position dynamic scanning acquisition, and is different from the general video super-resolution task which can only fuse the information of the dynamic part in the scene. The scan correction network can better mine the time correlation of the static part in the scene (the previous video super-resolution network is difficult to effectively improve the resolution of the repeated static scene, that is, it will degrade to the performance of single-image super-resolution), utilize the mechanical advantage of sub-pixel sampling and the time continuity between the scanning frames to improve the resolution of imaging, and effectively suppress the motion artifact problem caused by improper scanning imaging processing.

[0107] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0108] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for optimizing motion artifacts in scanning light field imaging, characterized in that: The following steps are involved: S1. Acquire a single-frame original light field image to be optimized; wherein the original light field image contains motion artifacts; S2. Rearranging the original light field image into a space-angle image according to space and angle information by pixel rearrangement; S3, periodically scanning the space-angle image to obtain sub-pixels at different spatial positions and angular directions, and performing pixel realignment; specifically comprising: Performing an X×Y periodic scan on the spatial-angle image; wherein X represents the galvanometer scanning displacement with equal spacing in the horizontal direction, and Y represents the galvanometer scanning displacement with equal spacing in the vertical direction; During the scanning process, each pixel on the sensor corresponds to a different spatial position and angular orientation; realigning pixels at different sub-pixel scan positions; S4, disassemble the realigned image according to the scanning order to obtain multiple space-angle sub-images in the time dimension; S5. Input the space-angle sub-image into a trained scan correction network to perform image correction and fusion to obtain a high-resolution space-angle image with motion artifacts removed; the scan correction network is constructed based on a neural network model.

2. The method for optimizing motion artifacts in scanning light field imaging according to claim 1, wherein: Step S2 specifically includes: The original light field image Nh×Nw is rearranged into a space-angle image of N2×h×w by pixel rearrangement; wherein N2 represents the angular resolution of the light field, h and w represent the height and width of a single sub-aperture image, respectively; the space-angle image contains spatial information and angular information.

3. The method for optimizing motion artifacts in scanning light field imaging according to claim 1, wherein: Step S4 specifically includes: The realigned image is disassembled in the time dimension according to the scanning order to obtain X×Y spatial-angular sub-images at different spatial positions and angular directions in the time dimension.

4. The method for optimizing motion artifacts in scanning light field imaging according to claim 1, wherein: The processing process of the scanning correction network includes: Multiple spatial-angle sub-images are input into the scan correction network, and the feature extraction module extracts the single-frame multi-channel information of each sub-image layer by layer; A feature fusion module based on the channel attention mechanism is used to interactively fuse the multi-channel information of continuous subgraphs; The fused features are remapped and convolutionally fused to obtain a single-frame single-channel output image.

5. The method for optimizing motion artifacts in scanning light field imaging according to claim 1, wherein: The training process of the scan correction network includes: Add interference factors to the open source dataset to construct training data pairs; the interference factors include unstable light source and / or inaccurate scanning position; Using the training data to train the model, constructing a total model loss, and optimizing the model; The trained model is fine-tuned using the microscopy dataset to obtain the final model.

6. The method for optimizing motion artifacts in scanning light field imaging according to claim 5, wherein: The open source dataset is the REDS dataset in the field of biomedical imaging; The microscopic data set is microscopic data captured by a small amount of users using a scanning light field microscope.

7. The method for optimizing motion artifacts in scanning light field imaging according to claim 5, wherein: The total loss of the model includes conventional loss and high-frequency loss; The conventional loss is directly calculated from the model's predicted image and the real image; The high-frequency loss is calculated from the model prediction image after high-pass filtering and the real image after high-pass filtering.

8. An optimization device for motion artifacts in scanning light field imaging, characterized in that: The optimization method according to any one of claims 1 to 7 is applied, comprising the following modules: Acquisition module: Acquisition of a single frame of original light field image to be optimized; wherein the original light field image contains motion artifacts; Pixel rearrangement module: rearranges the original light field image into a space-angle image according to space and angle information through pixel rearrangement; Pixel realignment module: Periodically scans the space-angle image to obtain sub-pixels at different spatial positions and angular directions, and performs pixel realignment; specifically, it includes: Performing an X×Y periodic scan on the spatial-angle image; wherein X represents the galvanometer scanning displacement with equal spacing in the horizontal direction, and Y represents the galvanometer scanning displacement with equal spacing in the vertical direction; During the scanning process, each pixel on the sensor corresponds to a different spatial position and angular orientation; realigning pixels at different sub-pixel scan positions; Disassembly module: disassembles the realigned image according to the scanning order to obtain multiple spatial-angular sub-images in the time dimension; Motion artifact removal module: The space-angle sub-image is input into a trained scan correction network to perform image correction fusion to obtain a high-resolution space-angle image with motion artifacts removed; the scan correction network is constructed based on a neural network model.

Images