A method for obtaining a scanned image of a biological sample

Abstract

An embodiment of the present application provides a method for obtaining a scanned image of a biological sample. The method includes: obtaining a cross-sectional sample of the biological sample to be scanned; controlling a transmission electron microscope to perform transmission on the cross-sectional sample based on multiple sets of focusing parameters, and collecting a set of current signals of the focal plane generated by the cross-sectional sample under each set of focusing parameters by using a composite detector during the transmission process, where the set of current signals includes multiple current signals; performing image processing on the set of current signals under each focal plane to obtain multiple signal images under each focal plane, and the image processing includes multi-stage multiplication amplification, signal transformation, sampling, algorithm operation, and AI image extraction; using an image filtering algorithm to perform image filtering on the multiple signal images under each focal plane to obtain multiple layer-scanned images, and the total number of the layer-scanned images is different from the total number of the signal images.

Description

Technical Field

[0001] This application relates to the field of electron microscopy imaging technology, and particularly to a method for obtaining a scanned image of a biological sample. Background Art

[0002] Obtaining the internal structure information of a biological sample highly depends on the quality of the scanning electron microscope (SEM) image (scanned image), especially its axial resolution, which is crucial for ensuring the complete presentation of the internal structure information. The axial resolution of the scanned image is closely related to the slice thickness of the biological sample: the thinner the slice, the higher the axial resolution, and thus the internal characteristics of the biological sample can be revealed more finely.

[0003] However, the current methods for reducing the slice thickness mainly focus on physical slicing techniques, but these traditional techniques face significant limitations. Although they perform well in processing millimeter-scale samples, they are inadequate when pursuing nanoscale slice thickness. Therefore, the current physical slicing techniques are significantly insufficient in meeting the demand for high-precision and high-resolution imaging of the internal structure of biological samples, and how to provide a method for obtaining a scanned image with high axial resolution at the nanoscale has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a method for obtaining a scanned image of a biological sample to at least solve the above technical problems existing in the prior art.

[0005] According to the first aspect of this application, a method for obtaining a scanned image of a biological sample is provided, and the method includes:

[0006] Obtain a cross-section sample of the biological sample to be scanned;

[0007] Control a scanning transmission electron microscope to perform scanning transmission on the cross-section sample based on multiple sets of focusing parameters, and during the scanning transmission process, use a composite detector to collect a group of current signals of the focal plane generated by the cross-section sample under each set of focusing parameters, and the group of current signals includes multiple current signals;

[0008] Perform image processing on the group of current signals under each focal plane to obtain multiple signal images under each focal plane, and the image processing includes multi-stage multiplication amplification, signal transformation, sampling, algorithm operation, and AI image extraction;

[0009] Use an image filtering algorithm to perform image filtering on the multiple signal images under each focal plane to obtain multiple layered scanned images, and the total number of the layered scanned images is different from the total number of the signal images.

[0010] According to an embodiment of the present application, before the controlled through-scanning electron microscope performs through-scanning on the cross-section sample based on multiple sets of focusing parameters, the method further includes obtaining multiple sets of focusing parameters. Correspondingly,

[0011] The obtaining of multiple sets of focusing parameters includes:

[0012] Determine the scaling size of the cross-section sample according to the original field of view size and the set limit size of the through-scanning electron microscope, and adjust the first gain and the second gain of the through-scanning electron microscope according to the scaling size;

[0013] Obtain the required number of sets of focusing parameters, and determine multiple layers of the cross-section sample according to the required number of sets;

[0014] For each layer, use a parameter optimization algorithm to obtain the focusing parameters of each layer, and obtain multiple sets of focusing parameters. The focusing parameters include coarse adjustment parameters and fine adjustment parameters.

[0015] According to an embodiment of the present application, obtain a cross-section sample of the biological sample to be scanned;

[0016] Control the through-scanning electron microscope to perform through-scanning on the cross-section sample based on multiple sets of focusing parameters, and use a composite detector to collect a set of current signals of the focused surface generated by the cross-section sample under each set of focusing parameters during the through-scanning process. The set of current signals includes multiple current signals;

[0017] Perform image processing on the set of current signals under each focused surface to obtain multiple layer scanning images of the cross-section sample. The image processing includes multi-stage multiplication amplification, signal transformation, sampling, algorithm operation, and AI image extraction.

[0018] According to an embodiment of the present application, the point-by-point and line-by-line through-scanning includes:

[0019] The point-by-point and line-by-line through-scanning includes:

[0020] The converging lens of the through-scanning electron microscope focuses the high-voltage focused incident electron beam into an electron beam probe, and performs point-by-point through-scanning on the cross-section sample based on the electron beam probe;

[0021] The deflector of the through-scanning electron microscope deflects the electron beam probe so that the electron beam probe performs line-by-line through-scanning on the cross-section sample;

[0022] Wherein, the point-by-point through-scanning and line-by-line through-scanning methods are continuous large angles.

[0023] According to an embodiment of the present application, the using of an image filtering algorithm to perform image filtering on multiple signal images under each focused surface to obtain multiple layer scanning images includes:

[0024] Adaptive filtering denoising and spatial registration are performed on all signal images of each focal plane to obtain aligned images corresponding to all signal images;

[0025] Based on the sharpness, noise level, and structural similarity of each aligned image, calculate the quality scores of each aligned image, and screen for valid images according to the quality scores of each aligned image;

[0026] Taking the valid images as input, initialize the point spread function according to a preset kernel function, and reconstruct the image through an iterative deconvolution algorithm guided by noise confidence;

[0027] According to the gradient features and noise confidence of the reconstructed image, screen multiple to-be-processed hierarchical scanning images;

[0028] Perform brightness equalization and contrast enhancement on multiple to-be-processed hierarchical scanning images to obtain multiple hierarchical scanning images;

[0029] Among them, the noise confidence is obtained through local variance and global variance;

[0030] During the process of reconstructing the image, adjust the point spread function parameters according to the structural similarity gradient, and control the update intensity through a noise weight matrix.

[0031] According to an embodiment of the present application, the multiple current signals include at least two of the following: secondary electrons, backscattered electrons, Auger electrons, X-ray energy spectra, bright field information, and dark field information.

[0032] According to an embodiment of the present application, the section thickness of the section sample is 30 - 100 nm.

[0033] According to an embodiment of the present application, the composite detector includes a secondary electron detector, a backscattered electron detector, a bright field detector, and a dark field detector. The secondary electron detector is arranged directly above the section sample, the backscattered electron detector is arranged on the upper side of the section sample, the bright field detector is arranged directly below the section sample, and the dark field detector is arranged on the lower side of the section sample.

[0034] According to an embodiment of the present application, the number of groups of the multiple sets of focusing parameters is determined according to the axial resolution requirements of the scanning image;

[0035] The axial resolution requirements of the scanning image are determined according to the nanometer-level requirements.

[0036] According to an embodiment of the present application, the section sample of the to-be-scanned biological sample is obtained from the sample processing device for the to-be-scanned biological sample. The sample processing device includes a slicing device, a plasma processing device, and a film coating device; correspondingly,

[0037] The cross-section sample of the biological sample to be scanned is obtained by the sample processing device through the following operations:

[0038] Using the slicing device to continuously cut the biological sample to be scanned to obtain a plurality of sample slices to be processed;

[0039] Using the plasma processing device and the film coating device to perform plasma thinning and surface conductive film coating on the plurality of sample slices to obtain a plurality of sample slices;

[0040] Determining the cross-section sample from the plurality of sample slices.

[0041] According to the second aspect of the present application, there is provided a device for obtaining a scanned image of a biological sample, the device comprising:

[0042] An acquisition module for acquiring a cross-section sample of the biological sample to be scanned;

[0043] An acquisition module for controlling a transmission electron microscope to perform transmission on the cross-section sample based on multiple sets of focusing parameters, and using a composite detector to acquire a set of current signals of the focused plane generated by the cross-section sample under each set of focusing parameters during the transmission process, the set of current signals including multiple current signals;

[0044] An image processing module for performing image processing on the set of current signals under each focused plane to obtain multiple signal images under each focused plane, the image processing including multi-stage multiplication amplification, signal transformation, sampling, algorithm operation, and AI image extraction;

[0045] An image filtering module for filtering the multiple signal images under each focused plane using an image filtering algorithm to obtain multiple layer-scanned images, the total number of the layer-scanned images being different from the total number of the signal images.

[0046] According to the third aspect of the present application, there is provided an electronic device, comprising:

[0047] At least one processor; and

[0048] A memory communicatively connected to the at least one processor; wherein,

[0049] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the present application.

[0050] According to the fourth aspect of the present application, there is provided a non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to cause a computer to execute the method described in the present application.

[0051] In an embodiment of the present application, a method for obtaining a scanned image of a biological sample is provided. A cross-sectional sample of the biological sample to be scanned is obtained; a scanning transmission electron microscope is controlled to perform scanning transmission on the cross-sectional sample based on multiple sets of focusing parameters, and during the scanning transmission process, a composite detector is used to collect a set of current signals of the focusing plane generated by the cross-sectional sample under each set of focusing parameters, where the set of current signals includes multiple current signals; image processing is performed on the set of current signals under each focusing plane to obtain multiple layer scanning images of the cross-sectional sample, and the image processing includes multi-stage multiplication amplification, signal transformation, sampling, algorithm operation, and AI image extraction. By finely dividing the focusing plane of the sample section through multiple sets of focusing parameters, "secondary sectioning" of the sample section is achieved, greatly improving the axial resolution and nanoscale level of the scanned image.

[0052] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present application will become easily understood. In the drawings, several embodiments of the present application are shown in an exemplary rather than restrictive manner, where:

[0054] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0055] Figure 1 It shows a schematic flowchart of the implementation of the method for obtaining a scanned image of a biological sample provided by an embodiment of the present application;

[0056] Figure 2 It shows an example diagram of the arrangement of the composite detector of the method for obtaining a scanned image of a biological sample provided by an embodiment of the present application;

[0057] Figure 3 It shows a schematic diagram of the composition structure of the device for obtaining a scanned image of a biological sample provided by an embodiment of the present application;

[0058] Figure 4 It shows a schematic diagram of the composition structure of the electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] To make the objectives, features, and advantages of the present application more obvious and understandable, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0060] Figure 1 The figure shows a schematic implementation flowchart of a method for acquiring a scanned image of a biological sample provided by an embodiment of the present application.

[0061] Referring to Figure 1 , an embodiment of the present application provides a method for acquiring a scanned image of a biological sample, and the method includes:

[0062] Operation 101: Obtain a cross-sectional sample of the biological sample to be scanned.

[0063] To obtain a scanned image of a thinner biological sample, it is necessary to slice the biological sample to be scanned. When generating a scanned image of the biological sample, a cross-sectional sample obtained after slicing the biological sample to be scanned is obtained.

[0064] In an embodiment of the present application, obtaining a cross-sectional sample of the biological sample to be scanned can be regarded as sending an acquisition signal to the device for slicing the biological sample, so that the device for slicing transmits the cross-sectional sample of the biological sample to the corresponding processing position. To ensure the randomness of obtaining the scanned image, obtaining the cross-sectional sample of the biological sample can also be regarded as obtaining the cross-sectional sample from the user, that is, when receiving the scanned image acquisition instruction sent by the user, receiving the cross-sectional sample placed by the user at the corresponding processing position.

[0065] In an embodiment of the present application, obtaining a cross-sectional sample of the biological sample to be scanned is to obtain a cross-sectional sample of the biological sample to be scanned from a sample processing device. The sample processing device includes a slicing device, a plasma processing device, and a film coating device; correspondingly, the cross-sectional sample of the biological sample to be scanned is obtained by the sample processing device through the following operations: continuously cutting the biological sample to be scanned by the slicing device to obtain a plurality of sample slices to be processed; performing plasma thinning and surface conductive film coating on the plurality of sample slices by the plasma processing device and the film coating device to obtain a plurality of sample slices; and determining the cross-sectional sample from the plurality of sample slices.

[0066] The cross-sectional sample can be obtained by slicing the biological sample to be scanned based on a sample processing device. The sample processing device continuously cuts, significantly thins the plasma, and coats the surface with a conductive film on the biological sample by controlling its own slicing device, plasma processing device, and film coating device, to obtain multiple sample slices. Then, one of the sample slices is selected as the cross-sectional sample.

[0067] In an embodiment of the present application, the slice thickness of the cross-sectional sample is 30~100nm.

[0068] The slice thickness of the cross-sectional sample is the minimum cross-sectional thickness. The minimum cross-sectional thickness is limited by the diamond knife, slicing device, and process. Currently, the minimum cross-sectional thickness that can be obtained is 30~100nm.

[0069] Operation 102: Control the scanning transmission electron microscope to perform scanning transmission on the cross-sectional sample based on multiple sets of focusing parameters, and use a composite detector to collect a set of current signals of the focusing plane generated by the cross-sectional sample under each set of focusing parameters during the scanning transmission process. The set of current signals includes multiple current signals.

[0070] The scanned image mainly depends on the electron microscope for acquisition. The scanning electron microscope (SEM) generally uses a secondary electron detector (SED) and a backscattered electron detector (BSED). However, the SEM can only obtain information such as the morphology and composition of the sample surface, and cannot detect the internal information of the sample. The transmission electron microscope (TEM) generally uses a bright field detector (BFD) and a dark field detector (DFD), and has a higher resolution than the SEM, and can observe the internal information of the sample to obtain the elemental and spatial structure information of the sample. Therefore, in order to obtain more internal structure information of the biological sample, the embodiment of the present application uses a scanning transmission electron microscope (STEM) with the comprehensive characteristics of SEM and TEM to perform scanning transmission on the sample slice to obtain various internal information of the sample and obtain the three-dimensional structure of the sample slice. Among them, scanning transmission means scanning and transmission.

[0071] The slice thickness of the cross-sectional sample obtained by slicing using physical technology may not meet the nanometer-level requirements. Therefore, the embodiment of the present application needs to perform secondary cutting on the cross-sectional sample through a scanning transmission electron microscope to cut the cross-sectional sample into a sample that meets the nanometer level. Specifically, by setting multiple sets of focusing parameters and adjusting the scanning transmission electron microscope based on the multiple sets of focusing parameters, the scanning transmission electron microscope can be controlled to divide the cross-sectional sample into multiple focusing planes at different depths, reducing the slice thickness of the sample slice. Among them, each set of focusing parameters corresponds to a depth of the sample slice.

[0072] In order to obtain various current signals generated at each focal plane during the penetration of the cross-section sample, a composite detector is arranged around the sample section. The composite detector consists of multiple detectors, and each detector can obtain one type of current signal. During the penetration process, the current signal group of the cross-section sample at each focal plane is continuously obtained based on the composite detector. Among them, the current signal group includes multiple current signals.

[0073] In an embodiment of the present application, the number of groups of multiple sets of focusing parameters is determined according to the axial resolution requirement of the scanned image; the axial resolution requirement of the scanned image is determined according to the nanometer-level requirement.

[0074] The axial resolution is the minimum distance between the scanned images corresponding to two focal planes. If it is an equal distance, the numerical value is the original section thickness of the cross-section sample. The more focal planes, the more layers of the cross-section sample, and the higher the axial resolution. Therefore, the axial resolution can be controlled by controlling the data volume of the focal planes to achieve a scanned image with an axial resolution of several nanometers. Among them, each set of focusing parameters in the multiple sets of focusing parameters is arranged in size, and the distance between every two adjacent sets of focusing parameters is the same.

[0075] For example, when the section thickness of the cross-section sample is 30 nanometers, by setting four sets of equally spaced focusing parameters, an axial resolution of several nanometers can be obtained, that is, 30 nanometers / 4.

[0076] In an embodiment of the present application, multiple sets of focusing parameters also need to be obtained. The multiple sets of focusing parameters can be obtained through the following operations: Determine the scaling size of the cross-section sample according to the original field of view size and the set limit size of the penetration electron microscope, and adjust the first gain and the second gain of the penetration electron microscope according to the scaling size; obtain the required number of groups of focusing parameters, and determine multiple layers of the cross-section sample according to the required number of groups; for each layer, use a parameter optimization algorithm to obtain the focusing parameters of each layer to obtain multiple sets of focusing parameters, and the focusing parameters include coarse adjustment parameters and fine adjustment parameters.

[0077] First, it is necessary to determine the number of layers for dividing the cross-section sample. Each layer requires a set of focusing parameters. Therefore, obtaining the number of layers of the cross-section sample can also be understood as obtaining the number of groups of focusing parameters. After determining the layers, it is necessary to determine the focusing parameters of each layer. In the embodiment of the present application, a parameter optimization algorithm is preferably used to determine the focusing parameters of each layer.

[0078] Before using the parameter optimization algorithm to obtain the focusing parameters, in order to enable the algorithm to be used, the field of view of the penetration electron microscope is also preprocessed, that is, the image displayed by the penetration electron microscope is scaled. The process of image scaling is a gain adjustment process. The specific process of field of view preprocessing includes:

[0079] When the original field of view size Meet When Perform scaling:

[0080]

[0081] where s is the scaling ratio, i.e., the scaling size.

[0082] After preprocessing the field of view, determining the focusing parameters for each layer, the process of obtaining the focusing parameters for each layer based on the parameter optimization algorithm may include:

[0083] 1) Coarse-grained search ( denotes coarse, the coarse adjustment parameter, denotes fine, the fine adjustment parameter), where the coarse adjustment parameter represents the coarse adjustment value of the objective lens, and the fine adjustment parameter represents the fine adjustment value of the objective lens:

[0084] Initialization: ;

[0085] Traversal: ;

[0086] Update: where is the current sharpness index;

[0087] Boundary detection: If If ;

[0088] Repeat the traversal, update, and boundary detection steps until

[0089] 2) Medium-grained search;

[0090] Step size: ;

[0091] Search domain: ;

[0092] Update: ;

[0093] 3) Fine-grained search;

[0094] Step size: ;

[0095] Search domain: ;

[0096] Update: ;

[0097] 4) Sub-pixel interpolation optimization (quadratic curve interpolation initialized to 0);

[0098] If not at the boundary of the search domain, calculate the conic section difference :

[0099]

[0100]

[0101] 5) Output the focusing parameters of each layer

[0102] Take as the final focusing parameter and restore the first gain to the initial value.

[0103] Operation 103: Perform image processing on the current signal groups under each focusing plane to obtain multiple signal images under each focusing plane. The image processing includes multi-stage multiplication amplification, signal transformation, sampling, algorithm operation, and AI image extraction.

[0104] Perform multi-stage multiplication amplification, signal transformation, sampling, algorithm operation, and AI (Artificial Intelligence) extraction processing on the current signal groups of each focusing plane, and then the signal images of each focusing plane can be obtained.

[0105] The current signals generated in the scanning transmission electron microscope are often very weak, so multi-stage amplification is required to ensure that the signals are strong enough for subsequent processing. To ensure that the amplified scanning transmission current signals meet the requirements of subsequent sampling and algorithm operations, the amplified current signals need to be subjected to signal transformation. The signal transformation can include, but is not limited to, filtering, modulation, etc. The purpose of signal conversion is to remove noise, improve the signal-to-noise ratio, or change the form of the current signal. For the convenience of subsequent processing, the continuous scanning transmission current signals are also converted into discrete signals by sampling. The purpose of sampling is to convert the scanning transmission current signals into digital signals.

[0106] The sampled digital signals need to be subjected to various algorithm operations to extract useful information. Among them, the algorithm operations can include statistical analysis, frequency domain analysis, and image reconstruction of the signals, aiming to extract the information reflecting the sample characteristics from the original digital signals.

[0107] The last step is image extraction, that is, converting the signals processed by algorithm operations into visual images. Among them, the image extraction can include processing such as image enhancement, filtering, and segmentation. The image extraction can be carried out by using conventional AI image extraction methods or image conversion algorithms, which will not be elaborated here.

[0108] Operation 104: Use an image filtering algorithm to filter multiple signal images under each focal plane to obtain multiple layer-scanned images. The total number of layer-scanned images is different from the total number of signal images.

[0109] Due to situations such as current signal loss, the images obtained after processing may be defective. In the case where the images obtained after image processing are defective, the defective images need to be filtered out. Therefore, all signal images are also filtered based on the image filtering algorithm to obtain the final multiple layer-scanned images.

[0110] In an embodiment of the present application, since the multiple layer-scanned images are obtained by filtering all signal images, the number of layer-scanned images is less than the number of signal images.

[0111] In an embodiment of the present application, use an image filtering algorithm to filter multiple signal images under each focal plane to obtain multiple layer-scanned images of multiple cross-section samples, including: perform adaptive filtering denoising and spatial registration on all signal images of each focal plane to obtain aligned images corresponding to all signal images; calculate the quality scores of each aligned image based on the clarity, noise level, and structural similarity of each aligned image, and screen out valid images according to the quality scores of each aligned image; use the valid images as inputs, initialize the point spread function according to a preset kernel function, and reconstruct the image through an iterative deconvolution algorithm guided by noise confidence; screen multiple layer-scanned images to be processed according to the gradient characteristics and noise confidence of the reconstructed image; perform brightness equalization and contrast enhancement on the multiple layer-scanned images to be processed to obtain multiple layer-scanned images; where the noise confidence is obtained through local variance and global variance; during the process of reconstructing the image, adjust the point spread function parameters according to the structural similarity gradient, and control the update intensity through a noise weight matrix.

[0112] For example, taking the number of focal planes as n, each focal plane includes k signal images, and m (m ≤ k × n) layer-scanned images are obtained by image filtering as an example, the process of filtering k × n signal images includes:

[0113] 1) Adaptive filtering denoising

[0114] Perform image denoising using Adaptive Non-Local Means (Adaptive NLM) to retain the detailed features of biological samples as much as possible while suppressing noise. The specific design of Adaptive Non-Local Means is as follows:

[0115] For image I, the filtered value of pixel point x

[0116] Among them, the weight function :

[0117]

[0118] and is an adaptive block centered at the pixel coordinates (side length , is the local standard deviation of the neighborhood), is the set of all pixel abscissas in the image that contain of .

[0119] Distance metric uses the Gaussian weighted Euclidean distance:

[0120]

[0121] Among them, the standard deviation of the Gaussian function: , is the index variable, then is the Gaussian kernel function, which is positively correlated with k and .

[0122] Adaptive smoothing parameter :

[0123] , is the custom weight, usually taken as 0.8.

[0124] is the normalization factor:

[0125]

[0126] 2) Registration

[0127] The image is registered based on the Anisotropic Guided Registration algorithm to obtain the aligned image. The design of the Anisotropic Guided Registration algorithm is as follows:

[0128] Let the reference image be , and the image to be registered be . The transformation matrix is solved by minimizing the anisotropic loss function:

[0129]

[0130] Gradient direction constraint :

[0131] ​

[0132] Among them, is determined by the main gradient direction of the reference image at points:

[0133]

[0134] A convolution kernel representing the horizontal and vertical coordinates;

[0135] Anisotropic weight :

[0136]

[0137] Higher weights are assigned in the edge region;

[0138] Transformation model : Affine transformation parameterization:

[0139]

[0140] Among them, , , , , , are the internal parameters of the affine matrix, are the horizontal and vertical coordinates of the pixel point;

[0141] 3) Image quality score calculation

[0142] The image is screened for defects by designing an image quality evaluation function for calculating the quality score.

[0143] Image quality evaluation function:

[0144]

[0145] Among them, Tenengrad gradient (sharpness):

[0146] , , is the Sobel operator;

[0147] Noise level: Median of local variance ;

[0148] SSIM: Structural similarity;

[0149] Weight .

[0150] For each focusing layer of the signal images, retain before the value one ( ) or eliminate images below the threshold to obtain valid images, where and the threshold are configured according to actual requirements;

[0151] 5) Reconstruct the image

[0152] Use the noise-adaptive Richardson-Lucy deconvolution algorithm (noise confidence-guided iterative deconvolution algorithm) for tomographic blurring, and reconstruct the blurred images. Through iterative optimization, screen out layers of images, and estimate the point spread function (PSF) of each layer , where represents the depth of focus, that is, for each layer, the process is as follows:

[0153] a. Initialize

[0154] Input the blurred image and the initial PSF estimate (preset kernel function), and calculate the global noise variance . The calculation method for the noise variance is: perform 3-layer wavelet decomposition on the image to obtain the high-frequency subbands , variance:

[0155]

[0156] Noise confidence:

[0157] , is the attenuation coefficient, defaulting to 10, is the local variance of the sliding window.

[0158] b. Iterative optimization

[0159] Dynamic PSF correction (update every 5 times), is the learning rate ， where the loss function :

[0160]

[0161] is the total variation regularization of the PSF, is the balance coefficient, defaulting to 0.01;

[0162] Noise weight update: Calculate the weight according to the local variance Dynamically adjust the iteration step at the pixel level:

[0163]

[0164] wherein, is an empirical parameter, with a default value of 0.1. The greater the noise, the smaller the number of iteration steps, suppressing the amplification of noise.

[0165] Regularization term RL iteration:

[0166] ,

[0167] wherein, the gradient term : The gradient descent direction based on the structural similarity index, and the coefficient is adaptively adjusted according to the noise level:

[0168] , is a small constant to prevent division by zero (such as 10^-6);

[0169] c. Hierarchical screening

[0170] Calculate the gradient of the result of each layer and the noise confidence , and retain the layer and The layer , and are the gradient and noise confidence thresholds, which can be configured according to actual needs.

[0171] Thus, through multiple sets of focusing parameters, the embodiments of the present application finely divide the focusing plane of the sample section, realizing "secondary slicing" of the sample section, and greatly improving the axial resolution and nanoscale level of the scanned image.

[0172] In an embodiment of the present application, controlling the scanning transmission electron microscope to perform scanning transmission on the cross-section sample based on multiple sets of focusing parameters includes: based on multiple sets of focusing parameters, sequentially adjusting the focusing parameters of the scanning transmission electron microscope, and controlling the scanning transmission electron microscope to generate a high-voltage focused incident electron beam corresponding to each set of focusing parameters, and performing point-by-point and line-by-line scanning transmission on the focusing plane generated by the cross-section sample under each set of focusing parameters.

[0173] In an embodiment of the present application, point-by-point and line-by-line penetration includes: the convergence lens of the transmission electron microscope focuses the high-voltage focused incident electron beam into an electron beam probe, and performs point-by-point penetration on the cross-sectional sample based on the electron beam probe; the deflector of the scanning electron microscope deflects the electron beam probe so that the electron beam probe performs line-by-line penetration on the cross-sectional sample. Among them, the point-by-point penetration and line-by-line penetration methods are continuous large angles.

[0174] By adjusting the focusing parameters of the high-voltage focused incident electron beam, n focusing planes of the sample section can be obtained, and the n focusing planes are penetrated. Then, signals are collected for each focusing plane based on the composite detector.

[0175] Among them, during the penetration process, it is necessary to penetrate the sample section point by point and line by line. The process of point-by-point penetration is: based on the convergence lens, the incident electron beam is focused into an electron beam "probe", and point-by-point penetration is performed on the sample. The process of line-by-line penetration includes: using the deflector to deflect the electron beam so that the electron beam performs line-by-line penetration on the sample. When performing point-by-point and line-by-line penetration, a continuous large-angle method is adopted.

[0176] In an embodiment of the present application, the multiple current signals include at least two of the following: secondary electrons (SE, Secondary Electron), backscattered electrons (BSE, Backscattered Electrons), Auger electrons (AE, Auger Electron), energy dispersive X-ray spectroscopy (EDX, Energy Dispersive X-ray spectroscopy), bright field information (BF, Bright Field), and dark field information (DF, Dark Field).

[0177] Figure 2 The figure shows an example diagram of the arrangement of the composite detector for the method of obtaining a scanning image of a biological sample provided by an embodiment of the present application.

[0178] Reference Figure 2 , in an embodiment of the present application, the composite detector includes a secondary electron detector, a backscattered electron detector, a bright field detector, and a dark field detector. The secondary electron detector is arranged directly above the cross-sectional sample, the backscattered electron detector is arranged on the upper side of the cross-sectional sample, the bright field detector is arranged below the cross-sectional sample, and the dark field detector is arranged on the lower side of the cross-sectional sample.

[0179] Figure 3 The figure shows a schematic diagram of the composition structure of the device for obtaining a scanning image of a biological sample provided by an embodiment of the present application.

[0180] Reference Figure 3, based on the above method for obtaining a scanned image of a biological sample, an embodiment of the present application further provides a device for obtaining a scanned image of a biological sample. The device includes: an acquisition module 301, configured to acquire a cross-section sample of the biological sample to be scanned; a collection module 302, configured to control a transmission electron microscope to perform transmission on the cross-section sample based on multiple sets of focusing parameters, and during the transmission process, use a composite detector to collect a set of current signals of the focal plane generated by the cross-section sample under each set of focusing parameters, where the set of current signals includes multiple current signals; an image processing module 303, configured to perform image processing on the set of current signals under each focal plane to obtain multiple signal images under each focal plane, and the image processing includes multi-stage multiplication amplification, signal transformation, sampling, algorithm operation, and AI image extraction; an image filtering module 304, configured to perform image filtering on the multiple signal images under each focal plane using an image filtering algorithm to obtain multiple layer-scanned images, and the total number of layer-scanned images is different from the total number of signal images.

[0181] It should be noted that the description of the device in the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects to the method embodiment, so it will not be elaborated here. For the technical details not described in the device for obtaining a scanned image of a biological sample provided in the embodiment of the present application, they can be understood according to Figures 1 to 2 the description of any one of the accompanying drawings.

[0182] According to an embodiment of the present application, the present application further provides an electronic device and a non-transitory computer-readable storage medium.

[0183] Figure 4 FIG. shows a schematic block diagram of an exemplary electronic device 40 that can be used to implement the embodiments of the present application. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0184] As Figure 4As shown, the electronic device 40 includes a computing unit 401, which can perform various appropriate actions and processes according to computer programs stored in a read-only memory (ROM) 402 or computer programs loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 40 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0185] Multiple components in the electronic device 40 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a magnetic disk, an optical disc, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the electronic device 40 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0186] The computing unit 401 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 401 executes the various methods and processes described above, such as the method for acquiring a scanned image of a biological sample. For example, in some embodiments, the method for acquiring a scanned image of a biological sample can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 40 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the method for acquiring a scanned image of a biological sample described above can be executed. Alternatively, in other embodiments, the computing unit 401 can be configured to execute the method for acquiring a scanned image of a biological sample in any other appropriate manner (e.g., by means of firmware).

[0187] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on a chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0188] The program code for implementing the methods of this application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on the remote machine or server.

[0189] In the context of this application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0190] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0191] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0192] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client - server relationship is created by computer programs running on respective computers and having a client - server relationship with each other. The server can be a cloud server, or a server of a distributed system, or a server incorporating blockchain.

[0193] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved, and no limitation is imposed herein.

[0194] As described above, the above are only specific embodiments of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claimed rights.

Claims

1. A method for obtaining a scanned image of a biological sample, characterized in that, The method includes: Obtaining a cross-sectional sample of the biological sample to be scanned; Controlling a scanning transmission electron microscope to perform scanning transmission on the cross-sectional sample based on multiple sets of focusing parameters, and during the scanning transmission process, using a composite detector to collect a set of current signals of the focused plane generated by the cross-sectional sample under each set of focusing parameters, the set of current signals including multiple current signals; the composite detector includes a secondary electron detector, a backscattered electron detector, a bright field detector, and a dark field detector, the secondary electron detector is arranged directly above the cross-sectional sample, the backscattered electron detector is arranged on the upper side of the cross-sectional sample, the bright field detector is arranged directly below the cross-sectional sample, and the dark field detector is arranged on the lower side of the cross-sectional sample; the multiple current signals include at least two of the following: secondary electrons, backscattered electrons, Auger electrons, X-ray energy spectrum, bright field information, and dark field information; Performing image processing on the set of current signals under each focused plane to obtain multiple signal images under each focused plane, the image processing including multi-stage multiplication amplification, signal transformation, sampling, algorithm operation, and AI image extraction; Using an image filtering algorithm to filter the multiple signal images under each focused plane to obtain multiple layer scanning images, the total number of layer scanning images being different from the total number of signal images; Wherein, before controlling the scanning transmission electron microscope to perform scanning transmission on the cross-sectional sample based on multiple sets of focusing parameters, the method further includes obtaining multiple sets of focusing parameters, correspondingly, The obtaining of multiple sets of focusing parameters includes: Determining the scaling size of the cross-sectional sample according to the original field of view size and the set limit size of the scanning transmission electron microscope, and adjusting the first gain and the second gain of the scanning transmission electron microscope according to the scaling size; Obtaining the required number of sets of focusing parameters, and determining multiple layers of the cross-sectional sample according to the required number of sets; For each layer, using a parameter optimization algorithm to obtain the focusing parameters of each layer to obtain multiple sets of focusing parameters, the focusing parameters including coarse adjustment parameters and fine adjustment parameters.

2. The method according to claim 1, wherein Controlling the scanning transmission electron microscope to perform scanning transmission on the cross-sectional sample based on multiple sets of focusing parameters includes: Based on multiple sets of focusing parameters, sequentially adjusting the focusing parameters of the scanning transmission electron microscope, and controlling the scanning transmission electron microscope to generate a high-voltage focused incident electron beam corresponding to each set of focusing parameters, and performing point-by-point and row-by-row scanning transmission on the focused plane generated by the cross-sectional sample under each set of focusing parameters.

3. The method according to claim 2, wherein The point-by-point and row-by-row scanning transmission includes: The converging lens of the scanning transmission electron microscope focuses the high-voltage focused incident electron beam into an electron beam probe, and performs point-by-point scanning transmission on the cross-sectional sample based on the electron beam probe; The deflector of the scanning transmission electron microscope deflects the electron beam probe so that the electron beam probe performs row-by-row scanning transmission on the cross-sectional sample; Wherein, the point-by-point scanning transmission and the row-by-row scanning transmission are in a continuous large angle manner.

4. The method according to claim 1, characterized in that, The using of an image filtering algorithm to filter the multiple signal images under each focused plane to obtain multiple layer scanning images includes: Performing adaptive filtering denoising and spatial registration on all the signal images of each focused plane to obtain aligned images corresponding to all the signal images; Based on the clarity, noise level, and structural similarity of each aligned image, calculate the quality scores of the aligned images, and screen the valid images according to the quality scores of the aligned images; Using the valid images as input, initialize the point spread function according to a preset kernel function, and reconstruct the image through an iterative deconvolution algorithm guided by noise confidence; According to the gradient features and noise confidence of the reconstructed image, screen multiple to-be-processed layer-scanned images; Perform brightness equalization and contrast enhancement on the multiple to-be-processed layer-scanned images to obtain multiple layer-scanned images; Among them, the noise confidence is obtained through local variance and global variance; During the process of reconstructing the image, adjust the point spread function parameters according to the structural similarity gradient, and control the update intensity through a noise weight matrix.

5. The method according to claim 1, wherein The section thickness of the section sample is 30 - 100 nm.

6. The method according to claim 1, characterized in that, The number of groups of the multiple sets of focusing parameters is determined according to the axial resolution requirements of the scanned image; The axial resolution requirements of the scanned image are determined according to the nanoscale requirements.

7. The method according to claim 1, wherein The section sample of the to-be-scanned biological sample is obtained from a sample processing device for the to-be-scanned biological sample, and the sample processing device includes a slicing device, a plasma processing device, and a film coating device; correspondingly, The section sample of the to-be-scanned biological sample is obtained by the sample processing device through the following operations: Continuously cut the to-be-scanned biological sample using the slicing device to obtain multiple to-be-processed sample sections; Perform plasma thinning and surface conductive film coating on the multiple sample sections using the plasma processing device and the film coating device to obtain multiple sample sections; Determine the section sample from the multiple sample sections.

Images