High-speed image acquisition and processing method and application thereof in scanning imaging device

Abstract

The present invention relates to the technical field of image data processing. Disclosed are a high-speed image acquisition and processing method and an application thereof in a scanning imaging device. The method comprises the following steps: randomly generating a scanning template on the basis of the type of an object to be scanned in combination with a required sampling rate and uniform distribution; a scanning imaging device using the generated scanning template to transversely scan said object at the given sampling point positions in the scanning template; gathering the transverse scanning images obtained at the sampling point positions to obtain a masking scanning image; and performing a reconstruction network algorithm on the masking scanning image to acquire a reconstructed image, and comparing the reconstructed image with a target image to obtain loss function data. According to the present invention, joint optimization is carried out on the scanning template and the reconstruction network algorithm at the same time, so that a global optimal solution can be obtained, thereby further reducing the number of points needing to be sampled during scanning, and ensuring the quality of the reconstructed image.

Description

A high-speed image acquisition and processing method and its application in scanning imaging device Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a high-speed image acquisition and processing method and its application in a scanning imaging device. Background Art

[0002] With the development of modern industry and technology, the demand for fast imaging is becoming increasingly strong, especially for imaging large objects or large areas. There are currently two main technical approaches to improving fast imaging: one is to increase the imaging (scanning) speed of imaging hardware, and the other is to increase the speed of image data processing. Currently, with the development of imaging hardware technology and the emergence of volume scanning technology, the computing speed of image data processing technology faces greater challenges.

[0003] For example, in biological research, understanding the structure of biological tissues and organs greatly facilitates understanding their functions and provides a strong scientific basis for diagnosing various functional diseases. However, due to the large size of tissues and organs (such as the brain), the imaging range is limited, and multiple regional imaging sessions are required to obtain complete structural data for a single plane. Furthermore, due to the specificity of organisms, imaging studies of a large number of samples are required to obtain statistical results. Therefore, when performing structural imaging on large biological samples, it is necessary to increase data acquisition throughput and shorten imaging time.

[0004] Raster scanning is a common imaging technique. Its basic principle is to capture and reconstruct images through a combination of linear scanning and point scanning. During raster scanning, the imaging system scans line by line and point by point along a specific direction to obtain pixel data for the image. Because its data acquisition method is compatible with existing image storage formats (pictures, i.e., 2D matrices), raster scanning is widely used in imaging systems such as scanning electron microscopes (SEMs), transmission electron microscopes (TEMs), atomic force microscopes (AFMs), and confocal microscopes.

[0005] However, the current raster scanning image data acquisition and processing technology wastes a lot of time scanning the object point by point and line by line. In the same scanning area, there are too many scanning points, which naturally results in slow scanning speed. The data processing system needs to process more data, which in turn leads to slow imaging speed and difficulty in quickly obtaining accurate scanned images.

[0006] Currently, no effective solutions have been proposed for the problems in related technologies.

[0007] Summary of the Invention

[0008] In view of the deficiencies of the prior art, the present invention provides a high-speed image acquisition and processing method and its application in a scanning imaging device.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] A high-speed image acquisition and processing method comprises the following steps:

[0011] S1. Randomly generate a scanning template based on the type of object to be scanned, combined with the required sampling rate and uniform distribution;

[0012] S2. The scanning imaging device uses the scanning template generated in step S1 to perform a transverse scan on the object to be scanned at each sampling point position given in the scanning template;

[0013] S3, the horizontal scanning images obtained at each sampling point are aggregated to obtain a masked scanning image;

[0014] S4, obtaining a reconstructed image by reconstructing the masked scan image through a reconstruction network algorithm, and performing a comparison operation between the reconstructed image and the target image to obtain loss function data;

[0015] S5. Continuously iteratively train the reconstruction network algorithm using the loss function data to continuously obtain a more accurate reconstruction network algorithm to obtain a more accurate reconstructed image.

[0016] Preferably, the data source for the iterative training of the reconstruction network algorithm in step S5 is the loss function data between the reconstructed image and the real image; the loss function data is a set of weighted average values ​​of any multiple data combinations selected from the L1 norm, L2 norm, MSE, and perceptual loss function data of the data corresponding to the positions of each acquisition point between the reconstructed image and the real image.

[0017] Preferably, the iterative training step of the reconstruction network algorithm in step S5 is:

[0018] S5.1. Build a target image dataset and generate a scanning template.

[0019] S5.2, feeding the target image into a reconstruction network algorithm, thereby successively generating a masked scan image and a reconstructed image;

[0020] S5.3. Calculate the loss function of the reconstructed image and the target image, and perform gradient backpropagation on the loss function data obtained by the loss function calculation, so as to update the parameters of the reconstruction network algorithm until the loss function error between the reconstructed image and the target image converges.

[0021] Preferably, step S1 is replaced by a random template generation algorithm that generates a random sampling template according to the type of object to be scanned, and uses the random sampling template as the generated scanning template; in step S2, the random sampling template is used as the generated scanning template, and the scanning imaging device uses the random sampling template to perform a horizontal scan on the object to be scanned at each sampling point position given in the random sampling template.

[0022] Preferably, the loss function data also includes the difference between the sampling rate of the sampling template and the target sampling rate; the loss function data can also be combined with noise input to jointly train the random template generation algorithm to continuously obtain more accurate sampling templates; wherein, the data source of the noise input is: noise randomly generated according to a specific distribution.

[0023] Preferably, in step S2, a feasible path can be determined based on the random sampling template, and the scanning imaging device moves along the determined feasible path to sequentially perform horizontal scanning on the object to be scanned at each sampling point position given in the scanning template; wherein, the scanning path is determined by using a graph construction algorithm or a depth-first search algorithm at each sampling point of the random sampling template to find a path covering all sampling points.

[0024] Preferably, step S1 is replaced by the path generation algorithm directly generating a feasible path according to the type of object to be scanned, and reproducing a reproduction template based on the feasible path; the method for reproducing the reproduction template from the feasible path is as follows: the feasible path is obtained by moving on an N*N matrix, and the feasible path finally generated by the path generation algorithm can be regarded as a sequence of plane coordinates; when it is necessary to reproduce the reproduction template, the sequence of plane coordinates is traversed, and the coordinate points appearing in the sequence are marked as 1, and those not appearing are marked as 0. If a coordinate of a certain point appears repeatedly in the sequence, the repeated coordinate point is still marked as 1, and the N*N matrix finally obtained is the sampling template, wherein the coordinate point marked as 1 is the sampling point in the reproduction template; in step S2, the scanning imaging device moves along the determined feasible path to sequentially perform horizontal scanning on the object to be scanned at each sampling point position given in the reproduction template.

[0025] Preferably, the specific steps of the path generation algorithm are:

[0026] S6.1. Starting from a random initial position, take the action from the initial position to each possible position coordinate as the action. There are several actions at this time.

[0027] S6.2. Calculate the rewards for several different actions; the reward values ​​are provided by a real-time loss function.

[0028] S6.3. Move to the next position coordinate corresponding to the action with the largest reward value;

[0029] S6.4. Repeat the above steps S6.2 and S6.3 until the stopping condition is met, where the stopping condition is that the proportion of the passed position coordinates in the entire N*N matrix meets a specific sampling rate.

[0030] Preferably, the loss function data can also be combined with noise input to jointly train the path generation algorithm to continuously obtain more effective feasible paths and more accurate reproduction templates.

[0031] The invention discloses an application of a high-speed image acquisition and processing method in a scanning imaging device.

[0032] Compared with the prior art, the present invention provides a high-speed image acquisition and processing method and its application in a scanning imaging device, which has the following beneficial effects:

[0033] 1. This high-speed image acquisition and processing method reduces the number of points required for horizontal scanning based on various scanning templates, thereby reducing the acquisition time required to scan the same area. At the same time, by using a reconstruction network algorithm, a high-quality image with similar quality to the original image can be reconstructed from the aforementioned undersampled image points.

[0034] 2. This high-speed image acquisition and processing method can obtain a global optimal solution by simultaneously optimizing the scanning template and the reconstruction network algorithm, thereby further reducing the number of points required for sampling during scanning and ensuring the quality of the reconstructed image.

[0035] 3. This high-speed image acquisition and processing method generates a feasible scanning path from the scanning template based on the physical limitations of the actual scanning device, and uses this path to drive the actual scanning device, thereby reducing the deviation between the actual scanning template of the scanning device and the designed scanning template, thereby further improving the imaging quality.

[0036] 4. This high-speed image acquisition and processing method jointly optimizes the scanning path and reconstruction network algorithm, introduces device physical limitations when generating the scanning path, and obtains a global optimal solution, thereby improving imaging quality in actual application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is a schematic diagram of the steps of a high-speed image acquisition and processing method according to a first embodiment of the present invention;

[0038] FIG2 is a comparison diagram of a reconstructed image and a target image when the sampling rate is 50% according to the first embodiment of the present invention;

[0039] FIG3 is a comparison diagram of a reconstructed image and a target image when the sampling rate is 10% according to the first embodiment of the present invention;

[0040] FIG4 is a schematic diagram of the steps of a high-speed image acquisition and processing method according to a second embodiment of the present invention;

[0041] FIG5 is a schematic diagram of the steps of a high-speed image acquisition and processing method according to a third embodiment of the present invention;

[0042] FIG6 is a schematic diagram of the steps of a high-speed image acquisition and processing method according to a fourth embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] As introduced in the background technology, there are deficiencies in the existing technology. In order to solve the above technical problems, this application proposes a high-speed image acquisition and processing method and its application in a scanning imaging device.

[0045] Example 1:

[0046] Referring to FIG1 , a high-speed image acquisition and processing method includes the following steps:

[0047] S1. Randomly generate a scanning template based on the type of object to be scanned, combined with the required sampling rate and uniform distribution;

[0048] The sampling rate is determined according to the type of object to be scanned. Please refer to Figures 2 and 3, which are the comparison diagrams of the reconstructed image and the target image when the sampling rate is 50% and 10%, respectively.

[0049] Specifically, step S1 generates an N*N matrix for the entire projection surface of the object to be scanned, generates a set of N*N digital sequences for each element point (N*N element points) in the N*N matrix according to a uniform distribution of 0-1, and then performs threshold processing according to the sampling rate (taking the sampling rate of 50% and the sampling rate of 10% as examples):

[0050] When the sampling rate is determined to be 50% based on the type of object to be scanned, the numbers in the digital sequence that are greater than 0.5 are marked as 0, and the numbers in the digital sequence that are less than 0.5 are marked as 1, thereby generating an N*N matrix scanning template;

[0051] When the sampling rate is determined to be 10% based on the type of object to be scanned, the numbers in the digital sequence that exceed 0.1 are marked as 0, and the numbers in the digital sequence that do not exceed 0.1 are marked as 1, thereby generating an N*N matrix scanning template;

[0052] Among them, the points marked as 1 element are sampling points in the scanning template, and each sampling point also has a unique and determined coordinate in the N*N matrix corresponding to the scanning template.

[0053] S2. The scanning imaging device uses the scanning template generated in step S1 to perform a transverse scan on the object to be scanned at each sampling point position given in the scanning template;

[0054] The scanning imaging device includes but is not limited to a MEMS galvanometer or other reflective mirrors with random addressing capabilities, and scans the object to be scanned according to the positions of each sampling point given by the scanning template.

[0055] The acquisition of the masked scanning image is mainly achieved through a high-speed scanning element (the high-speed scanning element usually adopts a MEMS galvanometer or DLP. The specific parameter setting and specific operation principle of the MEMS galvanometer can be found at https: / / www.mirrorcletech.com / wp / products / mems-mirrors / , which will not be described in detail here). Taking the OCT system using a MEMS galvanometer as an example (the OCT system in this embodiment adopts the OCT system in the prior art. The specific parameter setting and specific operation principle of the OCT system can be found at https: / / www.proquest.com / docview / 1978063193, which will not be described in detail here):

[0056] First, the scanning sampling rate of the MEMS galvanometer is set to be the same as the imaging rate of the OCT system, and they are synchronized; that is, every time the MEMS galvanometer moves one position, the OCT system collects one acquisition;

[0057] Then, the coordinate sequence corresponding to the scanning template obtained in step S1 (each sampling point also has a unique determined coordinate in the N*N matrix corresponding to the scanning template, and the determined coordinates corresponding to all sampling points are arranged in a certain order to generate the coordinate sequence here) is input into the galvanometer controller, and then the galvanometer is controlled to move point by point on the sample surface according to the coordinate position. Each time it moves to the sampling point position, an acquisition is performed.

[0058] S3, the horizontal scanning images obtained at each sampling point are aggregated to obtain a masked scanning image;

[0059] After the MEMS galvanometer has traversed the coordinate sequence corresponding to the scanning template, the OCT system can obtain the required masked scanning image based on the acquisition information obtained from each sampling point. The technical solution of the OCT system here to obtain the masked scanning image based on the aggregation of each acquisition point is an existing technical solution in the OCT system, and its specific technical solution will not be described in detail here.

[0060] S4, obtaining a reconstructed image by reconstructing the masked scan image through a reconstruction network algorithm, and performing a comparison operation between the reconstructed image and the target image to obtain loss function data;

[0061] The reconstruction network algorithm is a neural network algorithm, and the reconstruction network algorithm is any one or a combination of Unet, SRCNN, EDSR, and resnet algorithms.

[0062] In this embodiment, the reconstruction network algorithm has 2 input channels and 1 output channel, adopts a U-Net architecture network, has four downsampling (and corresponding upsampling) blocks, the initial feature of the input layer is set to 32 features, the downsampling block uses a LeaklyReLU (negative slope of 0.2) activation function, the upsampling block uses a ReLU activation function, all blocks use batch normalization, and each block has two convolutional layers.

[0063] S5. Continuously iteratively train the reconstruction network algorithm through loss function data to continuously obtain a more accurate reconstruction network algorithm to obtain a more accurate reconstructed image.

[0064] The data source for iterative training of the reconstruction network algorithm is the loss function data between the reconstructed image and the real image;

[0065] The loss function data is a set of weighted average values ​​of any multiple data combinations selected from the L1 norm, L2 norm, MSE, and perceptual loss function data of the data corresponding to each acquisition point between the reconstructed image and the real image.

[0066] In this embodiment, the loss function data of the reconstruction network algorithm uses the L2, L1 norm and perceptual loss to calculate the difference between the target image and the true value image. The specific functional relationship is:

[0067] in, is the total loss function, is the L2 norm of the difference between the target image and the true value image; λ1 is The weight of is the L1 norm of the difference between the target image and the true value image; λ2 is The weight of is the perceptual loss between the target image and the ground-truth image;

[0068] The iterative training steps of the reconstruction network algorithm are:

[0069] S5.1. Build a target image dataset and generate a scanning template.

[0070] In this embodiment, the target dataset includes commonly used, public natural image datasets such as ImageNet or DIV2K, and may also include specialized OCT image datasets such as the Kenyan Ophthalmology OCT dataset or a private OCT image dataset;

[0071] The target data set includes a two-dimensional grayscale image, and the two-dimensional grayscale image needs to be resampled to be consistent with the size of the scan template.

[0072] S5.2, feeding the target image into a reconstruction network algorithm, thereby successively generating a masked scan image and a reconstructed image;

[0073] S5.3. Perform loss function calculation on the reconstructed image and the target image, and perform gradient backpropagation (SGD or ADAM) operation on the loss function data obtained by the loss function calculation, so as to update the parameters of the reconstruction network algorithm until the loss function error between the reconstructed image and the target image converges.

[0074] In this example, the training framework for the reconstruction network algorithm used a batch size of 1, an initial learning rate of 0.001, and an AdamW optimizer with a momentum of (0.9, 0.999) for 100 cycles. The learning rate was then reduced using a cosine decay strategy. Convergence was determined when the change in the loss function was less than 0.0001. All experiments were trained and tested using an NVIDIA GeForce RTX 3090 GPU.

[0075] Example 2:

[0076] Referring to FIG4 , a high-speed image acquisition and processing method includes the following steps:

[0077] S1. Based on the type of object to be scanned, a random template generation algorithm (mask generator) generates a random sampling template, and uses the random sampling template as the generated scanning template;

[0078] The random template generation algorithm uses a mask neural network. The input of the mask neural network is random noise. It outputs two sets of feature maps through a U-Net architecture with four layers of upsampling and four layers of downsampling. Each upsampling or downsampling layer contains two convolutional layers. The last two sets of feature maps are activated by the Gumbel Softmax function to generate a random sampling template.

[0079] The Gumbe Softmax activation function is:

[0080] Where D and are the outputs of the mask neural network and the Gumbel Softmax activation function respectively; randomly selected A channel of is used as a random sampling template; g is the random noise of independent and identical sampling in Gumbel (0,1) distribution, τ is the value that controls the density of Gumbel distribution, p and c are pixel index and channel index respectively, Represents the output of the Gumbel Softmax activation function with channel c and pixel p; Represents the output of the mask neural network with channel c and pixel p; Represents the output of random noise with channel c and pixel p;

[0081] S2. Using the random sampling template as a generated scanning template, the scanning imaging device performs a transverse scan of the object to be scanned at each sampling point position given in the random sampling template using the random sampling template;

[0082] S3, the horizontal scanning images obtained at each sampling point are aggregated to obtain a masked scanning image;

[0083] S4, obtaining a reconstructed image by reconstructing the masked scan image through a reconstruction network algorithm, and performing a comparison operation between the reconstructed image and the target image to obtain loss function data;

[0084] The loss function data includes the loss function data of the first embodiment and also includes the difference between the sampling rate of the sampling template and the target sampling rate;

[0085] S5. Continuously iteratively train the reconstruction network algorithm using the loss function data to continuously obtain a more accurate reconstruction network algorithm. The loss function data can also be combined with noise input to jointly train the random template generation algorithm to continuously obtain more accurate sampling templates, thereby obtaining a more accurate reconstructed image.

[0086] The data source of the noise input is: noise randomly generated according to a specific distribution. The method of randomly generating noise with this specific distribution is the generation mode in GAN, and a signal is generated from the noise.

[0087] The iterative training steps of the random template generation algorithm and the reconstruction network algorithm are the same as the iterative training steps of the reconstruction network algorithm in Example 1. The difference is that the loss function data can also be combined with noise input to jointly train the random template generation algorithm to continuously obtain more accurate sampling templates.

[0088] Example 3:

[0089] Referring to FIG5 , a high-speed image acquisition and processing method includes the following steps:

[0090] S1. Based on the type of object to be scanned, a random template generation algorithm generates a random sampling template, and uses the random sampling template as the generated scanning template;

[0091] S2. Using the random sampling template as a generated scanning template, a feasible path can be determined based on the random sampling template, and the scanning imaging device moves along the determined feasible path to sequentially perform horizontal scanning on the object to be scanned at each sampling point position given in the scanning template;

[0092] The scanning path is determined by using a graph construction algorithm or a depth-first search algorithm at each sampling point of a random sampling template to find a path that covers all sampling points.

[0093] The graph construction algorithm is as follows: First, consider an N*N matrix as a graph, where each element is a node and the connections between nodes represent the distances between them. If the length of adjacent edges must not exceed a certain value, only nodes that meet this condition are connected.

[0094] Among them, the depth-first search algorithm is: using the depth-first search (DFS) algorithm to find a path covering all nodes of 1. DFS is an algorithm that can explore all nodes of a graph, which can be used to find a path that meets specific conditions.

[0095] S3, the horizontal scanning images obtained at each sampling point are aggregated to obtain a masked scanning image;

[0096] S4, obtaining a reconstructed image by reconstructing the masked scan image through a reconstruction network algorithm, and performing a comparison operation between the reconstructed image and the target image to obtain loss function data;

[0097] The loss function data includes the loss function data of the first embodiment and also includes the difference between the sampling rate of the sampling template and the target sampling rate;

[0098] S5. The reconstruction network algorithm is continuously iteratively trained through the loss function data to continuously obtain a more accurate reconstruction network algorithm. The loss function data can also be combined with noise input to jointly train the random template generation algorithm to continuously obtain more accurate sampling templates to obtain more accurate reconstructed images.

[0099] Example 4:

[0100] Referring to FIG6 , a high-speed image acquisition and processing method includes the following steps:

[0101] S1. The path generation algorithm directly generates a feasible path according to the type of object to be scanned, and reproduces a reproduction template based on the feasible path;

[0102] The specific steps of the path generation algorithm are:

[0103] S6.1. Starting from a random initial position, take the action from the initial position to each possible position coordinate as the action. There are several actions at this time.

[0104] S6.2. Calculate the rewards for several different actions; the reward values ​​are provided by a real-time loss function.

[0105] S6.3. Move to the next position coordinate corresponding to the action with the largest reward value;

[0106] S6.4. Repeat the above steps S6.2 and S6.3 until the stopping condition is met, where the stopping condition is that the proportion of the passed position coordinates in the entire N*N matrix meets a specific sampling rate.

[0107] In this embodiment, the specific sampling rate may be the sampling rate of 50% or the sampling rate of 10% mentioned above;

[0108] In addition, the stopping condition can also be that the proportion of the passed position coordinates in the entire N*N matrix reaches the PSNR or SSIM value;

[0109] Peak signal-to-noise ratio (PSNR) is an engineering term that represents the ratio of the maximum possible signal power to the destructive noise power that affects its representation accuracy. Because many signals have a very wide dynamic range, PSNR is often expressed in logarithmic decibels. It is often used as a measure of signal reconstruction quality in fields such as image compression and is often simply defined as the mean square error (MSE).

[0110] SSIM (Structural Similarity) is an indicator for measuring the similarity between two images.

[0111] The specific values ​​of the PSNR or SSIM values ​​are selected according to the different sampling objects. The values ​​are common values ​​in the prior art and their specific values ​​are not listed here one by one.

[0112] In addition, in actual use, the path generation algorithm may also adopt common path generation algorithms in the prior art, such as breadth-first search (BFS), depth-first search (DFS), etc.

[0113] Among them, breadth-first search (BFS) is an algorithm for graph search and traversal. It starts from the starting node and traverses all adjacent nodes layer by layer until the target node is found. The characteristic of breadth-first search is that it can find the shortest path. It uses a queue to store the nodes to be traversed. Each time a node is taken out of the queue, its adjacent unvisited nodes are added to the queue until the queue is empty or the target node is found.

[0114] Depth-first search (DFS) is an algorithm for graph search and traversal. It starts from the starting node and searches downward along a path until it reaches the deepest node or finds the target node. It then backtracks to the previous layer of nodes and continues searching for adjacent unvisited nodes. The characteristic of depth-first search is that it can quickly find a path, but it is not necessarily the shortest path.

[0115] The method for reproducing the recurrence template from the feasible path is as follows: the feasible path is obtained by moving on an N*N matrix. The feasible path finally generated by the path generation algorithm can be regarded as a sequence of plane coordinates. When the recurrence template needs to be reproduced, the sequence of plane coordinates is traversed, and the coordinate points that appear in the sequence are marked as 1, and those that do not appear are marked as 0. If a coordinate point appears repeatedly in the sequence, the repeated coordinate point is still marked as 1. The N*N matrix finally obtained is the sampling template, in which the coordinate points marked as 1 are the sampling points in the recurrence template.

[0116] S2. The scanning imaging device moves along a determined feasible path to sequentially perform transverse scanning on the object to be scanned at each sampling point position given in the reproduction template.

[0117] S3, the horizontal scanning images obtained at each sampling point are aggregated to obtain a masked scanning image;

[0118] S4, obtaining a reconstructed image by reconstructing the masked scan image through a reconstruction network algorithm, and performing a comparison operation between the reconstructed image and the target image to obtain loss function data;

[0119] The loss function data includes the loss function data of the first embodiment and also includes the difference between the sampling rate of the sampling template and the target sampling rate;

[0120] S5. The reconstruction network algorithm is continuously iteratively trained using the loss function data to continuously obtain a more accurate reconstruction network algorithm. The loss function data can also be combined with noise input to jointly train the path generation algorithm to continuously obtain more effective feasible paths and more accurate reproduction templates to obtain more accurate reconstructed images.

[0121] Embodiment 5:

[0122] A high-speed image acquisition and processing method is applied to a scanning imaging device. In this embodiment, a high-speed image acquisition and processing method of any one of Embodiments 1 to 4 can be used. In this embodiment, the scanning imaging device can be a scanning electron microscope (SEM), a transmission electron microscope (TEM), an atomic force microscope (AFM), a confocal microscope, or other imaging device.

[0123] Any of the above-mentioned scanning devices can perform scanning of each scanning point of the scanning template.

[0124] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A high-speed image acquisition and processing method, characterized in that: The steps include: S1. Randomly generate a scanning template based on the type of object to be scanned, combined with the required sampling rate and uniform distribution; S2, the scanning imaging device uses the scanning template generated in step S1 to perform horizontal scanning on the object to be scanned at each sampling point position given in the scanning template; S3, the lateral scanning images obtained at each sampling point are aggregated to obtain a masked scanning image; S4, obtaining a reconstructed image by reconstructing the masked scanned image through a reconstruction network algorithm, and performing a comparison operation between the reconstructed image and the target image to obtain loss function data; S5. Continuously iteratively train the reconstruction network algorithm through the loss function data to continuously obtain a more accurate reconstruction network algorithm to obtain a more accurate reconstructed image.

2. A high-speed image acquisition and processing method according to claim 1, characterized in that: The data source for the iterative training of the reconstruction network algorithm in step S5 is the loss function data between the reconstructed image and the real image; The loss function data is a set of weighted average values ​​of any combination of data selected from the L1 norm, L2 norm, MSE, and perceptual loss function data of the data at the corresponding positions of each acquisition point between the reconstructed image and the real image.

3. A high-speed image acquisition and processing method according to claim 2, characterized in that: The iterative training steps of the reconstruction network algorithm described in step S5 are: S5.

1. Build a target image dataset and generate a scanning template; S5.2, sending the target image to the reconstruction network algorithm, thereby successively generating a masked scan image and a reconstructed image; S5.

3. Calculate the loss function of the reconstructed image and the target image, and perform gradient back-propagation operation on the loss function data obtained by the loss function calculation, so as to update the parameters of the reconstruction network algorithm until the loss function error between the reconstructed image and the target image converges.

4. The high-speed image acquisition and processing method according to claim 1, characterized in that: The S1 step is replaced by the random template generation algorithm generating a random sampling template according to the type of the object to be scanned, and the random sampling template is used as the generated scanning template; In step S2, the random sampling template is used as the generated scanning template, and the scanning imaging device uses the random sampling template to perform a transverse scan on the object to be scanned at each sampling point position given in the random sampling template.

5. A high-speed image acquisition and processing method according to claim 4, characterized in that: The loss function data also includes the difference between the sampling rate of the sampling template and the target sampling rate; The loss function data can also be combined with noise input to jointly train the random template generation algorithm to continuously obtain more accurate sampling templates; The data source of the noise input is: noise randomly generated according to a specific distribution.

6. A high-speed image acquisition and processing method according to claim 4, characterized in that: In step S2, a feasible path can be determined according to the random sampling template, and the scanning imaging device moves along the determined feasible path to sequentially perform horizontal scanning on the object to be scanned at each sampling point position given in the scanning template; The scanning path is determined by using a graph construction algorithm or a depth-first search algorithm at each sampling point of a random sampling template to find a path that covers all sampling points.

7. A high-speed image acquisition and processing method according to claim 1, characterized in that: The S1 step is replaced by the path generation algorithm directly generating a feasible path according to the type of object to be scanned, and reproducing a reproduction template based on the feasible path; The method of reproducing the recurrence template from the feasible path is as follows: the feasible path is obtained by moving on an N*N matrix, and the feasible path finally generated by the path generation algorithm can be regarded as a sequence of plane coordinates; when the recurrence template needs to be reproduced, the sequence of plane coordinates is traversed, and the coordinate points appearing in the sequence are marked as 1, and those not appearing are marked as 0. If a certain point coordinate appears repeatedly in the sequence, the repeated coordinate point is still marked as 1. The N*N matrix finally obtained is the sampling template, and the coordinate points marked as 1 are the sampling points in the recurrence template; In step S2, the scanning imaging device moves along a determined feasible path to sequentially perform transverse scanning on the object to be scanned at each sampling point position given in the reproduction template.

8. A high-speed image acquisition and processing method according to claim 7, characterized in that: The specific steps of the path generation algorithm are: S6.1, starting from a random initial position, the movement from the initial position to each possible position coordinate as the next step is taken as an action, and there are several actions at this time; S6.

2. Calculate the rewards obtained from several different actions; the reward values ​​are provided by the real-time loss function; S6.3, move to the next step coordinate position corresponding to the action with the largest reward value; S6.

4. Repeat the above steps S6.2 and S6.3 until the stopping condition is met, where the stopping condition is that the proportion of the passed position coordinates in the entire N*N matrix meets a specific sampling rate.

9. A high-speed image acquisition and processing method according to claim 8, characterized in that: The loss function data can also be combined with noise input to jointly train the path generation algorithm to continuously obtain more effective feasible paths and more accurate reproduction templates.

10. Application of a high-speed image acquisition and processing method in a scanning imaging device, characterized in that: A high-speed image acquisition and processing method as described in any one of claims 1 to 9 is used.

Images