Method for processing an image, computer program product, and image processing device

By exchanging specific pixel values ​​when the particle beam device generates images, the image noise problem is solved, image quality and training efficiency are improved, and images are quickly generated for training are achieved, which improves the performance of noise reduction function.

CN113297896BActive Publication Date: 2024-11-01CARL ZEISS MICROSCOPY GMBH
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202110198664.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-02-21
Filing Date
2021-02-22
Publication Date
2024-11-01
Estimated Expiration
2041-02-22

AI Technical Summary

Technical Problem

In the prior art, when using particle beam devices to generate images, the image often contains noise, resulting in low image quality, and the number of images required to train a machine learning system to recognize the noise reduction function is cumbersome and takes a long time.

Method used

By exchanging specific pixel values ​​in the image, new images are generated to increase the number of images trained by the machine learning system, improve the quality of the noise reduction function, and generate images for training in a short time.

Benefits of technology

The quality and efficiency of image denoising is improved, the training time is shortened, and the image generated for training is achieved in a short time, which improves the performance of the noise reduction function.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113297896B_ABST
    Figure CN113297896B_ABST
Patent Text Reader

Abstract

The present invention relates to a method for processing an image. The method comprises: providing at least one first image of an object, the first image comprising n pixels P11 to P1 n ; Provide at least one second image of the object, the second image comprising m pixels P21 to P2 m ; Identify pixels P11 to P1 n At least one pixel P1 in i , the identified pixel P1 i The position PL1 in the first image i Identify pixels P21 to P2 m At least one pixel P2 in j , the identified pixel P2 j The position PL2 in the second image j By using the second image and by identifying the pixel P1 i Introduced into the identified pixel P2 j Location PL2 j to generate at least one third image, and / or by using the first image and by placing the identified pixel P2 j Introduced into the identified pixel P1 i Location PL1 i generating at least one fourth image from the processor; training the processor unit using at least one of the third image and the fourth image to identify a noise reduction function; and storing the noise reduction function in a memory unit.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention described herein relates to a method for processing an image generated by imaging an object using a beam device, in particular a particle beam device and / or a light microscope, such as a laser scanning microscope. Furthermore, the invention described herein relates to a computer program product which is loaded into a processor and which, when executed, controls an image processing device in such a way that the method according to the invention is performed. Furthermore, the invention described herein relates to a beam device having such an image processing device. Background Art

[0002] It is known in the art to use light microscopes, such as laser scanning microscopes, to generate images of an object (also referred to as a specimen).

[0003] Furthermore, it is known in the art to use electron beam devices, in particular scanning electron microscopes (hereinafter also SEM) and / or transmission electron microscopes (hereinafter also TEM), to examine objects to gain insights into their properties and behavior under certain conditions.

[0004] In SEM, a beam generator is used to generate an electron beam (hereinafter also referred to as a primary electron beam), and a beam guiding system is used to focus the electron beam on the object to be inspected. The objective lens is used for focusing purposes. A deflection device is used to guide the primary electron beam on the surface of the object to be inspected. This is also called scanning. The area scanned by the primary electron beam is also called the scanning area. In this area, the electrons of the primary electron beam interact with the object to be inspected. Interaction particles and / or interaction radiation appear due to this interaction. For example, the interaction particles are electrons. In particular, the electrons are emitted by the object (so-called secondary electrons), and the electrons of the primary electron beam are backscattered (so-called backscattered electrons). The interaction particles form a so-called secondary particle beam and are detected by at least one particle detector. The particle detector generates a detection signal for generating an image of the object. Thus, an image of the object to be inspected is obtained. For example, the interaction radiation is X-ray radiation or cathode luminescence. At least one radiation detector is used to detect the interaction radiation.

[0005] In the case of TEM, a beam generator is also used to generate a primary electron beam, and a beam guidance system is used to guide the primary electron beam to the object to be inspected. The primary electron beam passes through the object to be inspected. When the primary electron beam passes through the object to be inspected, the electrons of the primary electron beam interact with the material of the object to be inspected. The electrons passing through the object to be inspected or the electrons emitted by the object are imaged onto a light-emitting screen or a detector (for example, in the form of a camera) through a system including an objective lens. For example, the above-mentioned system further includes a projection lens. Imaging can also be performed in the scanning mode of TEM. This type of TEM is generally referred to as STEM. In addition, it can be provided that at least one additional detector is used to detect particles backscattered from the object to be inspected and / or secondary particles emitted by the object to be inspected, so as to image the object to be inspected. Additionally or alternatively, in TEM or STEM, the electrons of the primary electron beam are used to ablate or modify the object.

[0006] It is known to combine the functionality of STEM and SEM in a single particle beam device. Thus, an object can be examined using this particle beam device using the SEM functionality and / or the STEM functionality.

[0007] Furthermore, particle beam devices in the form of ion beam columns are known. An ion beam generator arranged in the ion beam column is used to generate ions for treating an object. For example, a gas is used to ablate material of the object or to apply material to the object during treatment. Additionally or alternatively, the ions are used for imaging by generating interaction particles and / or interaction radiation generated by the interaction of the ions with the object when impacting the object, wherein the interaction particles are, for example, secondary electrons, and wherein the interaction radiation is, for example, X-ray radiation.

[0008] In addition, the prior art has disclosed the practice of using electrons on the one hand and ions on the other hand to analyze and / or process objects in particle beam devices. For example, an electron beam column with SEM functionality is arranged at the particle beam device. In addition, an ion beam column as explained above is arranged at the particle beam device. The electron beam column with SEM functionality is particularly used for further inspection of processed or unprocessed objects, and is also used for processing objects.

[0009] Images generated by beam devices in the form of particle beam devices or in the form of laser scanning microscopes usually include noise. In the following, these images are also referred to as noisy images. The noise in the image is usually random variations in brightness and is based on electronic noise. The noise may be generated by the image sensor of the particle detector and / or the photodetector and / or any electronic circuit used in the image processing.

[0010] It is known in the art to process images generated by the above-mentioned beam device using a noise reduction function (e.g., a filter) to improve the quality of these images. After processing these images through the noise reduction function, the processed images have been denoised. In other words, the processed images have reduced noise and / or are substantially free of noise.

[0011] It is also known in the art to use the so-called noise-to-clean method to identify the noise reduction function. This method will be explained below.

[0012] The images generated by the beam device are used as training images for training a machine learning system used in a noise-clean method to identify a noise reduction function. In particular, a first image comprising noise is identified. The first image is used as a training image. Furthermore, a second image comprising no noise or only little noise is identified. The data of the first image comprising noise is used as input to an artificial neural network for generating a denoised first image. The denoised first image is used to identify a noise reduction function, as further explained below.

[0013] An artificial neural network consists of several neurons (also called nodes). Figure 1 The figure schematically shows a single neuron 800. Neuron 800 includes several inputs x1, x2, ..., x n-1 、x n . Figure 1 Three inputs x1, x2, x3 are shown as an example. Data of one pixel or some pixels of a noisy image in the form of a first image are used as inputs x1, x2, ..., x3 of the neuron 800. n-1 、x n , where n is an integer. Input x1, x2, ..., x n-1 、x n is the value of a characteristic of a pixel, such as brightness. Input x1, x2, ..., x n-1 、x n Each input in is weighted using a specific weighting factor w1, w2, ..., w n-1 、w n For example, the input x1, x2, ..., x n-1 、x n Each input in is multiplied by weighting factors w1, w2, ..., w n-1 、w n In addition to these inputs x1, x2, ..., x n-1 、x n In addition, neuron 800 also receives another input in the form of a bias value. In addition, neuron 800 includes an activation function This activation function is used to use the input (i.e., input x1, x2, ..., x n-1 、x n and / or bias value) to calculate the output value of neuron 800.

[0014] Figure 2 An example of an artificial neural network 900 with n inputs, K hidden layers, and three outputs is shown. Each layer consists of a number of neurons 800, each of which performs a weighted sum operation on these inputs and compares the result of the weighted sum with a threshold to generate an output. The neurons 800 may have interconnections between different layers of the artificial neural network 900. The artificial neural network 900 typically has more than three layers of neurons and has as many output neurons as input neurons, where n is the number of pixels in the reconstructed image. Synapses (i.e., connections between neurons 800) store values ​​in the form of weighting factors that manipulate data in calculations.

[0015] The output of artificial neural network 900 depends on: (i) the interconnection pattern between different layers of neurons 800, (ii) a learning process for updating the weighting factors of the interconnections, and (iii) an activation function that converts the weighted input of neuron 800 into its output activation.

[0016] Mathematically, the network function m(x) of neuron 800 is defined as the composition of other functions ni(x), which in turn can be defined as the composition of other functions. This can be conveniently represented as a network structure, where arrows depict dependencies between variables, such as Figure 2 For example, the artificial neural network 900 may use a nonlinear weighted sum in the form

[0017] m(x)=KΣ i w i n i (x)

[0018] K is a predefined function, namely, an activation function, such as a sigmoid function, a hyperbolic tangent function, or a rectified linear unit (ReLU).

[0019] Figure 2 The neuron 800 shown in is depicted by a circle surrounding the threshold function. Also, Figure 2 The inputs shown in are depicted as circles surrounding the linear function, and the arrows indicate the connections between neurons 800.

[0020] The artificial neural network 900 operates to achieve a specific task, such as denoising an image. This can be achieved, for example, by defining a loss function. The loss function is a measure of how far a particular solution to the problem to be solved (i.e., achieving good image quality by adequately denoising the image) is from the optimal solution. The learning algorithm iterates through the solution space to find a function with the smallest possible loss.

[0021] The artificial neural network 900 may be a convolutional neural network having beneficial properties for image processing and therefore having particular relevance for denoising images. The convolutional neural network may use a feed-forward artificial neural network 900, wherein the connection pattern between neurons 800 may represent a convolution in image processing.

[0022] When the noise reduction function is initially identified, the weighting factors w1, w2, ..., w2 of each neuron 800 of the artificial neural network 900 are n-1 、w n is selected randomly or based on previous experience. If the pixel data of a noisy image in the form of a first image is used as input to the neurons 800 of the first layer, the output of the artificial neural network 900 is the pixel data of the first image after denoising. Accordingly, a denoised first image is generated. The denoised first image is compared with a second image which does not include noise or only includes little noise as described above. The comparison enables the identification of a difference function, for example in the form of a loss function.

[0023] Based on the difference function, it can be determined whether the denoised first image corresponds or almost corresponds to the second image. If the denoised first image does not correspond to the second image, the noise in the denoised first image is too high. If the noise is too high, a back propagation algorithm known in the art is used in particular to identify those neurons 800 in the artificial neural network 900 that contribute significantly to the denoised first image. All weighting factors w1, w2, ..., w2 of all neurons 800 are changed in such a way that the loss function is minimized. n-1 、w n In other words, the weighting factors and / or bias values ​​are changed in such a way as to move toward the direction of the steepest gradient descent or the like. n-1 、w n After adjusting the image quality and / or bias values, data of another noisy image is used as input to the artificial neural network 900.

[0024] The above-described method of generating a denoised image using the artificial neural network 900 is repeated until the denoised image corresponds to or almost corresponds to an image with no noise or with little noise. In other words, if the noise in the denoised image is low enough, the parameters used in the artificial neural network 900, i.e., the weighting factors w1, w2, ..., w2, are set in such a way that a random image having similar noise characteristics and whose data is used as the input of the artificial neural network 900 is properly denoised. n-1 、w n and / or bias value. Accordingly, the weighting factors w1, w2, ..., w n-1 、w n The set bias value is the basis of the noise reduction function and is stored in the memory unit. Whenever an image with similar noise characteristics needs to be denoised, the weighting factors w1, w2, ..., w1 set above are used. n-1 、w n The image data is used as input to the artificial neural network 900 using the set bias value and / or the set bias value.

[0025] It is also known in the art to use the so-called noise-to-noise method to identify a denoising function. The noise-to-noise method is based on the noise-clean method as described above. However, instead of using a second image without noise or with little noise, a second noisy image is used to compare with the denoised first image. In other words, the denoised first image is compared with the second image including noise. The comparison allows the identification of a difference function, for example in the form of a loss function. Based on the difference function, it can be determined whether the denoised first image corresponds to or almost corresponds to the second image. It is explicitly mentioned that the difference function using the noise-to-noise method can also lead to the desired conclusion, i.e., whether the noise in the denoised first image is too high. For example, this is explained in the publication "Supplemental Material (Noise2Noise)" by Lehtinen et al. in the proceedings of the 35th International Conference on Machine Learning, PMLR 80, Stockholm, Sweden, 2018. If the noise is too high, a back propagation algorithm known in the art is used in particular to identify those neurons 800 in the artificial neural network 900 that contribute significantly to the de-noised first image. All weighting factors w1, w2, ..., w2 of all neurons 800 are changed in such a way that the loss function is minimized. n-1 、w n In other words, the weighting factors and / or bias values ​​are changed in such a way as to move toward the direction of the steepest gradient descent or the like. n-1 、wn After the denoised image is generated, the data of another noisy image is used as input to the artificial neural network 900. The above method of generating a denoised image using the artificial neural network 900 is repeated until the denoised image corresponds to or almost corresponds to an image with no noise or with little noise. In other words, if the noise in the denoised image is low enough, the parameters used in the artificial neural network 900, i.e., the weighting factors w1, w2, ..., w2, are set in such a way that a random image with similar noise characteristics and whose data is used as input to the artificial neural network 900 is properly denoised. n-1 、w n and / or bias value. Accordingly, the weighting factors w1, w2, ..., w n-1 、w n The set bias value is the basis of the noise reduction function and is stored in the memory unit. Whenever the image needs to be denoised, the weighting factors w1, w2, ..., w1 are used. n-1 、w n And / or the set bias value uses the data of the image as input to the artificial neural network 900.

[0026] Reference is made to the following prior art documents: EP 3 467 766 A1; CN 109118435 A; the publication by Zhang et al., entitled “Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising”, IEEE Transactions on Image Processing, 2017, Vol. 26, No. 7, pp. 3142-3155; the publication by Lehtinen et al., entitled “Noise2Noise: Learning Image Restoration without Clean Data”, in Proceedings of the 35th International Conference on Machine Learning, PMLR 80, Stockholm, Sweden, 2018; and the publication by Shorten et al., entitled “A survey on Image Data Augmentation for Deep Learning”, Journal of Big Data, 2019, 6: 60.

[0027] If the number of images used to train the machine learning system increases, the quality of the denoising function will improve. However, generating images with a particle beam device and / or a laser scanning microscope can be quite time-consuming. For example, generating a single image using a particle beam device can sometimes take hours or even a day. Accordingly, generating hundreds of training images can take a while. Summary of the invention

[0028] It is therefore desirable to provide a method, an image processing apparatus and a beam processing apparatus for performing the method, whereby the number of images used to train a machine learning system to identify a denoising function is increased in a short period of time.

[0029] According to the invention, this object is achieved by a method according to the following description. The features described below give a computer program product comprising a program code for controlling an image processing device. The features described below give an image processing device. Furthermore, the features described below give a beam device having an image processing device. Further features of the invention become clear from the following description and / or the accompanying drawings.

[0030] The method according to the invention is used to process an image generated by imaging an object using a beam device. The beam device may be a light microscope, in particular a laser scanning microscope and / or a particle beam device. The particle beam device may be an electron beam device and / or an ion beam device. The particle beam device may include a particle beam generator that generates charged particles. These charged particles may be electrons and / or ions.

[0031] The method according to the invention comprises the following steps: providing at least one first image of the object, wherein the first image comprises n pixels P11 to P1 n , where n ≥ 2, and where n is an integer. A pixel is an image element having, for example, a two-dimensional extension and an intensity. The intensity is a variable and represents, for example, a grayscale value and / or a color value. The n pixels P11 to P1 n Each pixel P1 in i The position PL1 arranged in the first image relative to the first image coordinate system i where i is an integer of 1≤i≤n, and where each position PL1 in the first image i Only the n pixels P11 to P1 n In other words, each position PL1 i No more than these n pixels P11 to P1 n A single pixel in. Each position PL1 i It is clearly identified by its coordinates relative to a first image coordinate system (which may be, for example, a Cartesian coordinate system).

[0032] The method according to the invention further comprises the following steps: providing at least one second image of the object, wherein the second image comprises m pixels P21 to P2 m , where m ≥ 2, and where m is an integer. In addition, it is applicable that m ≠ n or m = n. The m pixels P21 to P2 m Each pixel P2 in j The position PL2 arranged in the second image relative to the second image coordinate system j where j is an integer of 1≤j≤m, and where each position PL2 in the second image j Only these m pixels P21 to P2 m In other words, each position PL2 j Excluding more than these m pixels P21 to P2 m A single pixel in. Each position PL2 j It is clearly identified by its coordinates relative to a second image coordinate system (which may be, for example, a Cartesian coordinate system).

[0033] The first image of the object may be generated using the beam device and / or by loading the first image of the object from a database, such as a memory unit, into the processor unit. The database may include several first images of the object, wherein these first images have been generated using the beam device for implementing the present invention or using any other beam device. The database may be provided by the manufacturer of the beam device and / or by a user of the beam device.

[0034] The second image of the object may be generated using the beam device and / or by loading the second image of the object from a database, such as a memory unit, into the processor unit. The database may include several second images of the object, wherein these second images have been generated using the beam device for implementing the present invention or using any other beam device. The database may be provided by the manufacturer of the beam device and / or by the user of the beam device.

[0035] Furthermore, the method according to the invention comprises the following steps: identifying the n pixels P11 to P1 n At least one pixel P1 in i , where the identified pixel P1 i The position PL1 in the first image i The following will further describe the identification of the n pixels P11 to P1 of the first image. n Pixel P1 in i different ways.

[0036] In addition, the method according to the present invention comprises the following steps: identifying the m pixels P21 to P2 of the second image m At least one pixel P2 inj , where the identified pixel P2 j The position PL2 in the second image j The identification of the m pixels P21 to P2 of the second image will be further described below. m Pixel P2 in j different ways.

[0037] The method according to the invention further comprises the following steps: i The identified pixel P2 introduced into the second image j Location PL2 j At least one third image is generated at position PL2. In other words, another image, namely, a third image, is generated. The generated third image is based on the second image. However, not at position PL2 j The pixel P2 j , the generated third image is at position PL2 j The identified pixel P1 of the first image is included i .

[0038] Additionally or alternatively, the method according to the invention comprises the following steps: by using the first image and by transforming the identified pixel P2 of the second image j The identified pixel P1 introduced into the first image i Location PL1 i At least one fourth image is generated at position PL1. In other words, another image, namely, a fourth image, is generated. The generated fourth image is based on the first image. However, not at position PL1 i The pixel P1 i , the fourth image generated is at position PL1 i The identified pixel P2 of the second image is included at j .

[0039] In other words, the identified pixel P1 of the first image i (It is located at position PL1 in the first image i ) and the identified pixel P2 of the second image j (It is located at position PL2 in the second image j In other words, the identified pixel P1 of the first image is exchanged. i The value (eg, grayscale value) of j Location PL2 j Furthermore, the identified pixel P2 of the second image j The value (eg, grayscale value) of iLocation PL1 i By placing the identified pixel P1 i With the identified pixel P2 j By exchanging the pixels of the first image (or by exchanging the entire row, as further explained below), a new image is generated. The generated third image is based on at least one pixel (ie, P1) of the first image. i In addition, the generated fourth image is based on at least one pixel (ie, P2) of the second image. j )’s first image.

[0040] The method according to the invention comprises the following steps: using at least one of the generated third image and the generated fourth image to train a processor unit (e.g. the above-mentioned processor unit) for identifying a denoising function. In other words, the generated third image and / or the generated fourth image are used as input to the processor unit. The processor unit calculates a denoising function that can be used to denoise any image generated by using the beam device. An exemplary manner of calculating the denoising function is further explained below.

[0041] The method according to the invention further comprises the step of storing the identified noise reduction function in a memory unit, such as the above-mentioned memory unit.

[0042] The present invention provides a simple way to generate images during training of a machine learning system to identify a denoising function. By increasing the number of images used to train the machine learning system, the quality of the denoising function is improved. Thus, when the denoising function is used on any noisy image, the quality of denoising the noisy image is higher than the prior art. Furthermore, the present invention provides for generating images in a relatively short time (e.g., a few minutes) during training of the machine learning system. This is not time consuming at all compared to the prior art.

[0043] The method according to the invention can be carried out by a manufacturer of a beam device and / or by a user of a beam device.

[0044] In an embodiment of the method according to the invention, it is additionally or alternatively provided that the n pixels P11 to P1 n Pixel P1 in i The steps include one of the following: (i) randomly identifying the n pixels P11 to P1 n Pixel P1 in i (ii) identifying the n pixels P11 to P1 of the first image according to the first random pattern n Pixel P1 in i , and (iii) identifying the n pixels P11 to P1 of the first image according to the first given pattern n Pixel P1 in iThe first random pattern and / or the first given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the first random pattern and / or the first given pattern may be a line, a cross or a circle.

[0045] Furthermore, in an embodiment of the method according to the invention it is additionally or alternatively provided that the m pixels P21 to P2 m Pixel P2 in j The steps include one of the following: (i) randomly identifying the m pixels P21 to P2 m Pixel P2 in j (ii) identifying the m pixels P21 to P2 of the second image according to the second random pattern m Pixel P2 in j , and (iii) identifying the m pixels P21 to P2 of the second image according to the second given pattern m Pixel P2 in j The second random pattern and / or the second given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the second random pattern and / or the second given pattern may be a line, a cross or a circle.

[0046] The method according to the invention is not limited to identifying a single pixel P1 i and a single pixel P2 j And a single pixel P1 i With a single pixel P2 j Instead, the method according to the present invention also provides for identifying a number of pixels P1 i and several pixels P2 j And the identified pixel P1 i With the identified pixel P2 j In other words, in an embodiment of the method according to the invention, it is additionally or alternatively provided that the n pixels P11 to P1 n The identified pixel P1 in i is the first pixel P1 identified i , and the m pixels P21 to P2 of the second image m The identified pixel P2 in j is the first pixel P2 identified j An embodiment of the method according to the present invention further comprises: identifying the n pixels P11 to P1 n At least one second pixel P1 in o , where the identified second pixel P1 o The position PL1 in the first image oWhere o is an integer of 1≤o≤n and o≠i. The identification of the n pixels P11 to P1 of the first image will be further described below. n The second pixel P1 in o In addition, an embodiment of the method according to the present invention further comprises: identifying the m pixels P21 to P2 of the second image m At least one second pixel P2 p , where the identified second pixel P2 p The position PL2 in the second image p Where p is an integer of 1≤p≤m and p≠j. The identification of the m pixels P21 to P2 of the second image will be further described below. m The second pixel P2 in p In this embodiment of the method according to the invention, the step of generating the third image comprises: o The identified second pixel P2 introduced into the second image p Location PL2 p In addition, the step of generating the fourth image comprises the step of: p The identified second pixel P1 introduced into the first image o Location PL1 o Place.

[0047] In other words, the identified second pixel P1 of the first image o (It is located at position PL1 in the first image o ) and the identified second pixel P2 of the second image p (It is located at position PL2 in the second image p In other words, the identified second pixel P1 of the first image is exchanged. o The value (eg grayscale value) of p Location PL2 p Furthermore, the identified second pixel P2 of the second image p The value (eg grayscale value) of o Location PL1 o The identified second pixel P1 o The second pixel P2 identified p The swapping helps generate the third and fourth images.

[0048] It is explicitly mentioned that the method according to the invention is not limited to generating only the third image and the fourth image. Rather, a further embodiment of the method according to the invention provides for generating several new images based on already generated images, wherein the several new images are generated by exchanging pixels of the already generated images as explained above, and wherein the several new images are generated during training of the processor unit and are used for training the processor unit. In particular, it is additionally or alternatively provided that the method of the invention comprises providing several first images and several second images, wherein each of the several first images and each of the several second images is implemented as described above. Moreover, the method according to this embodiment comprises: identifying these n pixels P11 to P1 of each of the several first images n At least one pixel P1 in i , where the identified pixel P1 i Having the features as described above; and identifying the m pixels P21 to P2 of each image in the plurality of second images m At least one pixel P2 in j , where the identified pixel P2 j Furthermore, the method according to the present embodiment includes generating a plurality of third images, wherein each of the plurality of third images is generated by using one of the plurality of second images and by transforming the identified pixel P1 of one of the plurality of first images to the image. i The identified pixel P2 of the second image used in the introduction of the second image j Location PL2 j Furthermore, the method according to the present embodiment comprises generating a plurality of fourth images, wherein each of the plurality of fourth images is generated by using one of the plurality of first images and by converting the identified pixel P2 of one of the plurality of second images into j The identified pixel P1 of the first image used in the first image is introduced i Location PL1 i In addition, the step of training the processor includes using at least one of the generated third images and at least one of the generated fourth images to identify a noise reduction function. The identified noise reduction function is stored in the memory unit.

[0049] In an embodiment of the method according to the invention, it is additionally or alternatively provided that the n pixels P11 to P1 n The second pixel P1 in o The steps include one of the following: (i) randomly identifying the n pixels P11 to P1 n The second pixel P1 in o(ii) identifying the n pixels P11 to P1 of the first image according to the third random pattern n The second pixel P1 in o , and (iii) identifying the n pixels P11 to P1 of the first image according to a third given pattern n The second pixel P1 in o The third random pattern and / or the third given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the third random pattern and / or the third given pattern may be a line, a cross or a circle.

[0050] Furthermore, in an embodiment of the method according to the invention it is additionally or alternatively provided that the m pixels P21 to P2 m The second pixel P2 in p The steps include one of the following: (i) randomly identifying the m pixels P21 to P2 m The second pixel P2 in p (ii) identifying the m pixels P21 to P2 of the second image according to the fourth random pattern m The second pixel P2 in p , and (iii) identifying the m pixels P21 to P2 of the second image according to the fourth given pattern m The second pixel P2 in p The fourth random pattern and / or the fourth given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the fourth random pattern and / or the fourth given pattern may be a line, a cross or a circle.

[0051] Furthermore, in an embodiment of the method according to the invention, it is additionally or alternatively provided that the first random pattern is identical to the third random pattern. Furthermore, in an embodiment of the method according to the invention, it is additionally or alternatively provided that the first given pattern is identical to the third given pattern. Additionally or alternatively provided in an embodiment of the method according to the invention, it is additionally or alternatively provided that the second random pattern is identical to the fourth random pattern. Additionally or alternatively provided in an embodiment of the method according to the invention, it is additionally or alternatively provided that the second given pattern is identical to the fourth given pattern.

[0052] In an embodiment of the method according to the invention it is additionally or alternatively provided that the n pixels P11 to P1 n The first pixel P1 in i and the n pixels P11 to P1 of the first image n The second pixel P1 in o The n pixels P11 to P1 of the first image are identified in a manner adjacent to each other. n The first pixel P1 ini and the n pixels P11 to P1 of the first image n The second pixel P1 in o For example, the n pixels P11 to P1 n The first pixel P1 in i and the n pixels P11 to P1 of the first image n The second pixel P1 in o A line can be formed.

[0053] Furthermore, in an embodiment of the method according to the invention it is additionally or alternatively provided that the m pixels P21 to P2 m The first pixel P2 in j and the m pixels P21 to P2 of the second image m The second pixel P2 in p The m pixels P21 to P2 of the second image are identified in a manner adjacent to each other. m The first pixel P2 in j and the m pixels P21 to P2 of the second image m The second pixel P2 in p For example, the m pixels P21 to P2 of the second image m The first pixel P2 in j and the m pixels P21 to P2 of the second image m The second pixel P2 in p A line can be formed.

[0054] Moreover, in an embodiment of the method according to the present invention, it is additionally or alternatively provided that the step of training the processor unit includes using a machine learning system. The machine learning system uses algorithms and statistical models to perform specific tasks without the use of explicit instructions. In addition, the machine learning system uses algorithms based on training data to make predictions or decisions without explicit programming to perform tasks. With respect to the present invention, the above-mentioned specific task is to generate a denoising function and use the images generated using the present invention as training data. In a further embodiment of the method according to the present invention, it is additionally or alternatively provided that the machine learning system is at least one of the following: an artificial neural network, a support vector machine, a Bayesian network, a random decision forest, and association rule learning.

[0055] In addition, in a further embodiment of the method according to the present invention, it is additionally or alternatively provided that the step of training the processor unit includes: (a) denoising the generated third image and generating a denoised third image; (b) comparing the generated third image with the generated denoised third image; (c) calculating a first difference between the generated third image and the generated denoised third image, wherein the first difference can be calculated using a loss function; and (d) using the calculated first difference to identify the denoising function.

[0056] In an embodiment of the method according to the present invention, it is additionally or alternatively provided that the step of training the processor unit comprises: (a) denoising the generated fourth image and generating a denoised fourth image; (b) comparing the generated fourth image with the generated denoised fourth image; (c) calculating a second difference between the generated fourth image and the generated denoised fourth image, in particular using a second loss function to calculate the second difference; and (d) using the calculated second difference to identify the denoising function.

[0057] Furthermore, in an embodiment of the method according to the invention it is additionally or alternatively provided that the step of training the processor unit comprises using a noise-noise machine learning system.The noise-noise machine learning system has been further described above.

[0058] Furthermore, in an embodiment of the method according to the invention, additionally or alternatively, it is provided that the method comprises using the denoising function to denoise other images generated by imaging the object using the beam device or any other beam device that has not been used to generate the first image, the second image, the third image and / or the fourth image. Additionally or alternatively, the method according to the invention comprises: loading the denoising function from the memory unit into the processor unit, and using the denoising function to denoise other images generated by imaging the object using the beam device or any other beam device that has not been used to generate the first image, the second image, the third image and / or the fourth image.

[0059] The present invention also relates to a computer program product comprising a program code which can be loaded or loaded into a processor and which, when executed, controls an image processing device in the following manner: performs a method comprising at least one of the steps described above or further below or a combination of at least two of the steps described above or further below.

[0060] The present invention also relates to an image processing device, which comprises a memory unit and a processor unit having a processor, into which a computer program product is loaded, wherein the computer program product comprises at least one of the features described above or further below or a combination of at least two of the features described above or further below.

[0061] The invention also relates to a beam arrangement for imaging and / or analyzing an object. The beam arrangement comprises at least one beam generator for generating a beam and at least one image processing device, the image processing device comprising at least one of the features described above or further below or a combination of at least two of the features described above or further below.

[0062] The beam generator of an embodiment of the beam device according to the invention is a beam generator for generating a light beam. In particular, the beam generator is a generator for generating a laser beam. Thus, the beam device may be a light microscope and / or a laser scanning microscope.

[0063] Embodiments of the beam device according to the present invention additionally or alternatively include: the beam device is a particle beam device, wherein the beam generator is a particle beam generator for generating a primary particle beam with charged particles. The charged particles can be, for example, electrons or ions. Moreover, the particle beam device according to the present invention also has at least one objective lens for focusing the primary particle beam onto an object. In addition, the particle beam device according to the present invention has at least one detector for detecting interacting particles and / or interacting radiation, which are generated when the primary particle beam hits the object. The interacting particles can be secondary particles and / or backscattered particles, in particular secondary electrons and backscattered electrons. The interacting radiation can be X-ray radiation and / or cathode luminescence.

[0064] In the embodiment of the beam device according to the present invention, additionally or alternatively, the particle beam generator of the particle beam device is a first particle beam generator, and the primary particle beam is a first primary particle beam comprising a first charged particle. The objective lens is a first objective lens for focusing the first primary particle beam onto an object. The particle beam device according to an embodiment of the present invention further comprises a second particle beam generator for generating a second primary particle beam comprising a second charged particle and a second objective lens for focusing the second primary particle beam onto an object. The second charged particle can be an electron and / or an ion.

[0065] In a further embodiment of the beam device according to the present invention, it is additionally or alternatively provided that the particle beam device is at least one of the following: an electron beam device and an ion beam device. In particular, the particle beam device can be both an electron beam device and an ion beam device. The electron beam device and the ion beam device can be arranged at a certain angle to each other, for example at an angle in the range of 45° to 90°, wherein the range includes the boundaries. In particular, the electron beam device and the ion beam device can be arranged at an angle of 54° to each other. However, the present invention is not limited to the above-mentioned angles. Instead, any angle between the electron beam device and the ion beam device suitable for the present invention can be used. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Embodiments of the invention described herein are explained in more detail below with reference to the accompanying drawings, in which:

[0067] Figure 1 shows a schematic diagram of a neuron of an artificial neural network according to the prior art;

[0068] Figure 2 shows a schematic diagram of an artificial neural network according to the prior art;

[0069] Figure 3 A first exemplary embodiment of a particle beam device is shown;

[0070] Figure 4 A second exemplary embodiment of a particle beam apparatus is shown;

[0071] Figure 5 A third exemplary embodiment of a particle beam apparatus is shown;

[0072] Figure 6 An exemplary embodiment of a light microscope is shown;

[0073] Figure 7 An exemplary embodiment of an image processing apparatus is shown;

[0074] Figure 8 An exemplary embodiment of a method for processing an image is shown;

[0075] Fig. 9 An exemplary embodiment of a first image is shown;

[0076] Fig.10 An exemplary embodiment of a second image is shown;

[0077] Fig.11 An exemplary embodiment of a first image having identified pixels is shown;

[0078] Fig.12 An exemplary embodiment of a second image having identified pixels is shown;

[0079] Fig.13 An exemplary embodiment of the generated third image is shown;

[0080] Fig.14 shows an exemplary embodiment of a fourth image generated;

[0081] Fig.15 A further exemplary embodiment of a first image with identified pixels is shown;

[0082] Fig.16 A further exemplary embodiment of a second image with identified pixels is shown;

[0083] Fig.17 A further exemplary embodiment of the generated third image is shown;

[0084] Fig.18 shows a further exemplary embodiment of a generated fourth image;

[0085] Fig.19 The steps of an exemplary embodiment of a method for processing an image are shown;

[0086] Fig. 20 Further steps of an exemplary embodiment of a method for processing an image are shown;

[0087] Fig.21 further steps of a further exemplary embodiment of a method for processing an image are shown; and

[0088] Fig. 22 The steps of a further exemplary embodiment of a method for processing an image are shown. DETAILED DESCRIPTION

[0089] The particle beam device in the form of SEM and combined device is now used as an example to explain the present invention in more detail, and the combined device has an electron beam column and an ion beam column. Moreover, the light microscope is now used as an example to explain the present invention in more detail. The following fact is clearly mentioned: the present invention can be used for any particle beam device, particularly any electron beam device and / or any ion beam device and any light beam device, such as a laser scanning microscope.

[0090] Figure 3A schematic illustration of a SEM 100 is shown. The SEM 100 comprises a first beam generator in the form of an electron source 101, which is implemented as a cathode. Further, the SEM 100 is provided with an extraction electrode 102 and an anode 103, which is arranged at one end of a beam guide tube 104 of the SEM 100. By way of example, the electron source 101 is implemented as a thermal field emitter. However, the present invention is not limited to such an electron source 101. Rather, any electron source may be utilized.

[0091] The electrons emitted from the electron source 101 form a primary electron beam. These electrons are accelerated to the anode potential due to the potential difference between the electron source 101 and the anode 103. In the exemplary embodiment depicted here, the anode potential is 1 kV to 20 kV, for example 5 kV to 15 kV, in particular 8 kV, relative to the ground potential of the housing of the object chamber 120. However, alternatively, the anode potential can be at the ground potential.

[0092] Two condenser lenses, namely a first condenser lens 105 and a second condenser lens 106, are arranged at the beam guide tube 104. Figure 3 105 and then the second condenser lens 106. Explicitly mentioned is the fact that further exemplary embodiments of the SEM 100 may have only a single condenser lens. The first aperture unit 108 is arranged between the anode 103 and the first condenser lens 105. Together with the anode 103 and the beam guide tube 104, the first aperture unit 108 is at a high voltage potential (i.e. the potential of the anode 103), or it is grounded. The first aperture unit 108 has a plurality of first apertures 108A, which are arranged at Figure 3 One of them is depicted in . For example, there may be two first apertures 108A. Each of these many first apertures 108A has a different aperture diameter. By means of the adjustment mechanism 126, the desired first aperture 108A can be set on the optical axis OA of the SEM 100. For example, the first aperture unit 108 can be moved in the x-direction (i.e. the first aperture unit axis), in the y-direction (i.e. the second aperture unit axis) and in the z-direction (i.e. the third aperture unit axis) by using the adjustment mechanism 126, which directions are perpendicular to each other. The adjustment mechanism 126 can be a drive unit, in particular a motor, such as a stepper motor or a piezoelectric motor. It is explicitly mentioned that the drive unit is not limited to the above-described embodiments. Instead, the drive unit can be any drive unit suitable for the present invention.

[0093] The fact that in further exemplary embodiments the first aperture unit 108 may be provided with only a single aperture 108A is explicitly mentioned. In these exemplary embodiments the adjustment mechanism may be omitted. The first aperture unit 108 is then designed to be fixed.

[0094] The fixed second aperture unit 109 is arranged between the first condenser lens 105 and the second condenser lens 106. As an alternative thereto, provision may be made that the second aperture unit 109 is movable.

[0095] The first objective lens 107 has a pole piece 110 in which a hole is formed. The beam guide tube 104 is guided through this hole. A coil 111 is arranged in the pole piece 110.

[0096] The electrostatic delay device is arranged in the lower region of the beam guide tube 104. It has a single electrode 112 and a tubular electrode 113. The tubular electrode 113 is arranged at one end of the beam guide tube 104 facing the object 125 arranged on the object carrier 114. The tubular electrode 113 together with the beam guide tube 104 is at the potential of the anode 103, while the single electrode 112 and the object 125 are at a lower potential relative to the potential of the anode 103. In the present case, this is the ground potential of the housing of the object chamber 120. In this way, the electrons of the primary electron beam can be decelerated to the desired energy required for the examination of the object 125.

[0097] The SEM 100 further comprises a scanning device 115, by which the primary electron beam can be deflected and scanned over the object 125. In doing so, the electrons of the primary electron beam interact with the object 125. Due to the interaction, interaction particles are generated, which are detected. In particular, electrons are emitted from the surface of the object 125 (so-called secondary electrons), or electrons of the primary electron beam are backscattered (so-called backscattered electrons) as interaction particles.

[0098] The object 125 and the single electrode 112 may also be at different potentials and potentials different from ground potential. This allows the position of the delay of the primary electron beam relative to the object 125 to be set. For example, if the delay occurs very close to the object 125, the imaging aberration becomes smaller.

[0099] A detector arrangement comprising a first detector 116 and a second detector 117 is arranged in the beam guide tube 104 for detecting secondary electrons and / or backscattered electrons, wherein in the beam guide tube 104 the first detector 116 is arranged on the source side along the optical axis OA and the second detector 117 is arranged on the object side along the optical axis OA. The first detector 116 and the second detector 117 are arranged offset from each other in the direction of the optical axis OA of the SEM 100. The first detector 116 and the second detector 117 each have a passage opening through which the primary electron beam can pass. The first detector 116 and the second detector 117 are approximately at the potential of the anode 103 and the beam guide tube 104. The optical axis OA of the SEM 100 extends through the respective passage openings.

[0100] The second detector 117 is mainly used to detect secondary electrons. When emitted from the object 125, the secondary electrons initially have low kinetic energy and an arbitrary direction of movement. The secondary electrons are accelerated in the direction of the first objective lens 107 by the strong extraction field emitted from the tubular electrode 113. The secondary electrons enter the first objective lens 107 approximately in parallel. The beam diameter of the secondary electron beam is also kept small in the first objective lens 107. Then, the first objective lens 107 has a strong effect on the secondary electrons and generates a relatively short secondary electron focus at a sufficiently steep angle relative to the optical axis OA, so that the secondary electrons diverge away from each other downstream of the focus and impinge on the effective area of ​​the second detector 117. In contrast, only a small portion of the electrons that are backscattered at the object 125 (i.e., backscattered electrons, which have relatively higher kinetic energy than the secondary electrons emitted from the object 125) are detected by the second detector 117. The high kinetic energy of the backscattered electrons when emerging from the object 125 and the angle relative to the optical axis OA have the effect that the beam waist (i.e. the beam region with the smallest diameter) of the backscattered electrons is located near the second detector 117. Most of the backscattered electrons pass through the channel opening of the second detector 117. Therefore, the first detector 116 is essentially used to detect the backscattered electrons.

[0101] In a further embodiment of the SEM 100, the first detector 116 may additionally be embodied as an opposite field grid 116A. The opposite field grid 116A is arranged on a side of the first detector 116 facing the object 125. The opposite field grid 116A has a negative potential relative to the potential of the beam guide tube 104, so that only backscattered electrons with high energy pass through the opposite field grid 116A to the first detector 116. Additionally or alternatively, the second detector 117 has another opposite field grid, which is similar in design and function to the opposite field grid 116A of the first detector 116 described above.

[0102] The detection signals generated by the first detector 116 and the second detector 117 are used to generate one or more images of the surface of the object 125 .

[0103] The following fact is mentioned explicitly: for the sake of clarity, in the figures, the apertures of the first aperture unit 108 and the second aperture unit 109 and the channel openings of the first detector 116 and the second detector 117 appear disproportionately large. The channel openings of the first detector 116 and the second detector 117 have an extension perpendicular to the optical axis OA in the range of 0.5 mm to 5 mm. By way of example, these channel openings have an annular design and have a diameter perpendicular to the optical axis OA in the range of 1 mm to 3 mm.

[0104] The second aperture unit 109 is configured as a pinhole aperture in the exemplary embodiment depicted here and is provided with a second aperture 118 for letting the primary electron beam pass through, the second aperture having an extension in the range of 5 μm to 500 μm (for example 35 μm). As an alternative thereto, in a further embodiment it is provided that the second aperture unit 109 is provided with a plurality of apertures which can be mechanically displaced relative to the primary electron beam or can be reached by the primary electron beam by using electrostatic and / or magnetic deflection elements. The second aperture unit 109 is implemented as a pressure stage unit. The second aperture unit separates a first area from a second area, in which the electron source 101 is arranged and in which an ultrahigh vacuum (10 -7 hPa to 10 -12 hPa), the second region has a high vacuum (10 -3 hPa to 10 -7 hPa). The second region is an intermediate pressure region of the beam guide tube 104 , which opens into the object chamber 120 .

[0105] The object chamber 120 is in a vacuum state. In order to generate the vacuum, a pump (not shown) is arranged at the object chamber 120. Figure 3 In the exemplary embodiment shown, the object chamber 120 operates within a first pressure range or within a second pressure range. The first pressure range only includes pressures less than or equal to 10 -3 hPa, and the second pressure range only includes pressures greater than 10 -3 To ensure the pressure range, the object chamber 120 is vacuum-tight.

[0106] The object stage 114 is implemented to be movable in three directions arranged perpendicularly to each other, namely, the x direction (first stage axis), the y direction (second stage axis) and the z direction (third stage axis). Moreover, the object stage 114 can rotate around two rotation axes arranged perpendicularly to each other (namely, the first stage rotation axis and the second stage rotation axis).

[0107] The SEM 100 further comprises a third detector 121 arranged in the object chamber 120. More precisely, the third detector 121 is arranged downstream of the object stage 114, as seen along the optical axis OA from the electron source 101. The object stage 114 can be rotated in such a way that the primary electron beam can be radiated through the object 125. When the primary electron beam passes through the object 125 to be inspected, the electrons of the primary electron beam interact with the material of the object 125 to be inspected. The third detector 121 detects the electrons that pass through the object 125 to be inspected.

[0108] A radiation detector 500 is arranged at the object chamber 120 for detecting interaction radiation, such as X-ray radiation and / or cathode luminescence. The radiation detector 500, the first detector 116 and the second detector 117 are connected to an image processing device 123 having a monitor 124 and a database 129. The third detector 121 is also connected to the image processing device 123. For the sake of clarity, this connection is not shown. The image processing device 123 processes the detection signals generated by the first detector 116, the second detector 117, the third detector 121 and / or the radiation detector 500, and displays the detection signals on the monitor 124 in the form of images.

[0109] Further, the SEM 100 has a movable chamber detector 119. The chamber detector 119 may be a particle detector. For example, the chamber detector 119 may be moved to or away from a specific detection position in the object chamber 120 of the SEM 100. The chamber detector 119 is connected to a drive unit 127 and may be moved in the x direction (i.e., the first unit direction), in the y direction (i.e., the second unit direction), and in the z direction (i.e., the third direction) using the drive unit 127. All directions are perpendicular to each other. In addition, the chamber detector 119 may be rotated around a first unit rotation axis and a second unit rotation axis arranged perpendicular to the first unit rotation axis, wherein the drive unit 127 is used for the rotation. The drive unit 127 may be a motor, such as a stepper motor or a piezoelectric motor. It is explicitly mentioned that the drive unit 127 is not limited to the above-described embodiments. Instead, the drive unit 127 may be any drive unit suitable for the present invention.

[0110] The SEM 100 further includes a high voltage control unit 135 for adjusting an acceleration voltage of electrons of the primary electron beam.

[0111] Furthermore, the SEM 100 includes a current control unit 136 for adjusting the current of the first objective lens 107 .

[0112] The SEM 100 further includes a stigmator 137 for reducing astigmatism in the SEM 100. The stigmator 137 is connected to a stigmator control unit 141.

[0113] Furthermore, the SEM 100 includes a condenser control unit 139 connected to the first condenser lens 105 and the second condenser lens 106 .

[0114] Furthermore, the SEM 100 includes a scanning control unit 140 connected to the scanning device 115 .

[0115] Furthermore, the SEM 100 comprises a processor unit 128 having a processor in which a program code is loaded for controlling the image processing device 123 of the SEM 100 in such a way that the method according to the present invention is carried out.

[0116] Figure 4 A particle beam device is shown in the form of a combined device 200. The combined device 200 has two particle beam columns.

[0117] On the one hand, if Figure 4 As depicted in FIG. 1 , the combined device 200 is provided with the SEM 100 but without the object chamber 120. Instead, the SEM 100 is arranged in the object chamber 201. The object chamber 201 is in vacuum. To generate vacuum, a vacuum system 202 including a pump is connected to a valve 203 arranged at the object chamber 201. The vacuum system 202 and the valve 203 are connected to a vacuum control unit 204. Figure 4 In the exemplary embodiment shown, the object chamber 201 operates within a first pressure range or within a second pressure range. The first pressure range only includes pressures less than or equal to 10 -3 hPa, and the second pressure range only includes pressures greater than 10 -3 In order to ensure the pressure range, the object chamber 201 is vacuum-tight.

[0118] The third detector 121 is arranged in the object chamber 201. The SEM 100 is used to generate a first particle beam (i.e. a primary electron beam as described further above) and has an optical axis as specified above, which is at Figure 4 It is denoted by reference numeral 709 and is also referred to as the first beam axis.

[0119] On the other hand, the combined device 200 is provided with an ion beam device 300, which is also arranged at the object chamber 201. The ion beam device 300 also has an optical axis, which is Figure 4 It is denoted by reference numeral 710 and is also referred to as the second beam axis.

[0120] The SEM 100 is arranged vertically relative to the object chamber 201. In contrast, the ion beam device 300 is arranged to be inclined at an angle of approximately 50° relative to the SEM 100. The ion beam device has a second beam generator in the form of an ion beam generator 301. The ion beam generator 301 generates ions, which form a second particle beam in the form of an ion beam. These ions are accelerated by an extraction electrode 302 that is in a predeterminable potential. Then, the second particle beam passes through the ion optical device of the ion beam device 300, wherein the ion optical device includes a bunching lens 303 and a second objective lens 304. The second objective lens 304 finally generates an ion probe, which is focused on an object 125 arranged on the object carrier 114.

[0121] The adjustable or optional aperture unit 306, the first electrode arrangement 307 and the second electrode arrangement 308 are arranged above the second objective lens 304 (i.e., in the direction of the ion beam generator 301), wherein the first electrode arrangement 307 and the second electrode arrangement 308 are implemented as scanning electrodes. The second particle beam is scanned on the surface of the object 125 by the first electrode arrangement 307 and the second electrode arrangement 308, wherein the first electrode arrangement 307 acts in a first direction and the second electrode arrangement 308 acts in a second direction, and the second direction is opposite to the first direction. In this way, scanning is performed in the x direction, for example. The scanning in the y direction perpendicular thereto is realized by another electrode (not depicted here) rotated 90 ° at the first electrode arrangement 307 and the second electrode arrangement 308.

[0122] exist Figure 4 In the exemplary embodiment shown in , the object stage 114 is also implemented to be movable in three directions arranged perpendicularly to each other, namely, the x direction (first stage axis), the y direction (second stage axis) and the z direction (third stage axis). Moreover, the object stage 114 can rotate around two rotation axes arranged perpendicularly to each other (namely, the first stage rotation axis and the second stage rotation axis).

[0123] In order to better illustrate the various units of the combined device 200, Figure 4 The distances between the individual units of the combined device 200 depicted in FIG. 2 appear disproportionately large.

[0124] A radiation detector 500 is arranged at the object chamber 201 for detecting interaction radiation, such as X-ray radiation and / or cathode luminescence. The radiation detector 500 is connected to an image processing device 123, which has a monitor 124 and a database 129. The image processing device 123 processes the images obtained by the first detector 116, the second detector 117 ( Figure 4 ), the third detector 121 and / or the radiation detector 500 generate detection signals, and the detection signals are displayed on the monitor 124 in the form of images.

[0125] Further, the combined device 200 has a movable chamber detector 119. The chamber detector 119 can be a particle detector. For example, the chamber detector 119 can be moved to or away from a specific detection position in the object chamber 201 of the combined device 200. The chamber detector 119 is connected to a drive unit 127 and can be moved in the x direction (i.e., the first unit direction), in the y direction (i.e., the second unit direction), and in the z direction (i.e., the third direction) using the drive unit 127. All directions are perpendicular to each other. In addition, the chamber detector 119 can rotate around the first unit rotation axis and the second unit rotation axis arranged perpendicular to the first unit rotation axis, wherein the drive unit 127 is used for the rotation. The drive unit 127 can be a motor, such as a stepper motor or a piezoelectric motor. It is explicitly mentioned that the drive unit 127 is not limited to the above-mentioned embodiment. Instead, the drive unit 127 can be any drive unit suitable for the present invention.

[0126] Furthermore, the combination device 200 comprises a processor unit 128 having a processor in which a program code is loaded for controlling the image processing device 123 of the combination device 200 in such a way that the method according to the present invention is carried out.

[0127] Figure 5 4 is a schematic diagram of a further exemplary embodiment of a particle beam device according to the present invention. This exemplary embodiment of a particle beam device is represented by reference numeral 400, and the exemplary embodiment includes a reflector corrector for correcting, for example, chromatic aberration and / or spherical aberration. The particle beam device 400 includes a particle beam column 401, which is implemented as an electron beam column and substantially corresponds to the electron beam column of the corrected SEM. However, the particle beam device 400 is not limited to a SEM with a reflector corrector. Instead, the particle beam device can include any type of corrector unit.

[0128] The particle beam column 401 includes a particle beam generator in the form of an electron source 402 (cathode), an extraction electrode 403 and an anode 404. For example, the electron source 402 is implemented as a thermal field emitter. The electrons emitted from the electron source 402 are accelerated to reach the anode 404 due to the potential difference between the electron source 402 and the anode 404. Accordingly, a particle beam in the form of an electron beam is formed along the first optical axis OA1.

[0129] After the particle beam is emitted from the electron source 402, the particle beam is guided along a beam path corresponding to the first optical axis OA1. The first electrostatic lens 405, the second electrostatic lens 406, and the third electrostatic lens 407 are used to guide the particle beam.

[0130] In addition, a beam guide is used to adjust the particle beam along the beam path. The beam guide of this exemplary embodiment includes a source adjustment unit having two magnetic deflection units 408 arranged along the first optical axis OA1. Moreover, the particle beam device 400 includes an electrostatic beam deflection unit. A first electrostatic beam deflection unit 409 (also implemented as a quadrupole in a further embodiment) is arranged between the second electrostatic lens 406 and the third electrostatic lens 407. The first electrostatic beam deflection unit 409 is also arranged downstream of the magnetic deflection unit 408. A first multipole unit 409A in the form of a first magnetic deflection unit is arranged on one side of the first electrostatic beam deflection unit 409. Moreover, a second multipole unit 409B in the form of a second magnetic deflection unit is arranged on the other side of the first electrostatic beam deflection unit 409. The first electrostatic beam deflection unit 409, the first multipole unit 409A and the second multipole unit 409B are arranged to adjust the direction of the particle beam relative to the axis of the third electrostatic lens 407 and the entrance window of the beam deflection device 410. The first electrostatic beam deflection unit 409 , the first multipole unit 409A and the second multipole unit 409B may interact like a Wien filter. A further magnetic deflection element 432 is arranged at the entrance of the beam deflection device 410 .

[0131] The beam deflector 410 is used as a particle beam deflector, which is used to deflect the particle beam in a specific manner. The beam deflector 410 includes a plurality of magnetic sectors, namely a first magnetic sector 411A, a second magnetic sector 411B, a third magnetic sector 411C, a fourth magnetic sector 411D, a fifth magnetic sector 411E, a sixth magnetic sector 411F and a seventh magnetic sector 411G. The particle beam enters the beam deflector 410 along the first optical axis OA1, and the particle beam is deflected by the beam deflector 410 along the direction of the second optical axis OA2. The beam deflection of 30° to 120° angles is performed by means of the first magnetic sector 411A, by means of the second magnetic sector 411B and by means of the third magnetic sector 411C. The second optical axis OA2 is oriented at an angle of the same size relative to the first optical axis OA1. The beam deflector 410 also deflects the particle beam guided along the second optical axis OA2 in the direction of the third optical axis OA3. The beam deflection is provided by the third magnetic sector 411C, the fourth magnetic sector 411D and the fifth magnetic sector 411E. Figure 5In the exemplary embodiment of the present invention, the deflection relative to the second optical axis OA2 and relative to the third optical axis OA3 is provided by deflecting the particle beam by 90 ° angle. Therefore, the third optical axis OA3 extends coaxially with the first optical axis OA1. However, the following fact is mentioned: the particle beam device 400 according to the invention described herein is not limited to a deflection angle of 90 °. Instead, the beam deflection device 410 can select any suitable deflection angle, such as 70 ° or 110 °, so that the first optical axis OA1 does not extend coaxially with the third optical axis OA3. For more details about the beam deflection device 410, please refer to WO 2002 / 067286 A2.

[0132] After the particle beam has been deflected by the first magnetic sector 411A, the second magnetic sector 411B and the third magnetic sector 411C, the particle beam is guided along the second optical axis OA2. The particle beam is guided to the electrostatic mirror 414 and travels along the fourth electrostatic lens 415, the third multipole unit 416A in the form of a magnetic deflection unit, the second electrostatic beam deflection unit 416, the third electrostatic beam deflection unit 417 and the fourth multipole unit 416B in the form of a magnetic deflection unit on its path to the electrostatic mirror 414. The electrostatic mirror 414 includes a first mirror electrode 413A, a second mirror electrode 413B and a third mirror electrode 413C. The electrons of the particle beam reflected back at the electrostatic mirror 414 travel along the second optical axis OA2 again and reenter the beam deflection device 410. Then, these electrons are deflected to the third optical axis OA3 by the third magnetic sector 411C, the fourth magnetic sector 411D and the fifth magnetic sector 411E.

[0133] The electrons of the particle beam are emitted from the beam deflection device 410, and the electrons are guided along the third optical axis OA3 to the object 425 intended to be inspected and arranged on the object stage 424. On its path to the object 425, the particle beam is guided along the fifth electrostatic lens 418, the beam guide tube 420, the fifth multipole unit 418A, the sixth multipole unit 418B and the objective lens 421. The fifth electrostatic lens 418 is an electrostatic immersion lens. With the help of the fifth electrostatic lens 418, the particle beam is decelerated or accelerated to the potential of the beam guide tube 420.

[0134] By means of objective lens 421, particle beam is focused into the focal plane where object 425 is arranged. Object 425 is arranged on movable object stage 424. Movable object stage 424 is arranged in object chamber 426 of particle beam device 400. Object stage 424 is implemented to be movable in three directions arranged perpendicularly to each other, i.e., x direction (first stage axis), y direction (second stage axis) and z direction (third stage axis). Moreover, object stage 424 can rotate around two rotation axes (i.e., first stage rotation axis and second stage rotation axis) arranged perpendicularly to each other.

[0135] The object chamber 426 is in a vacuum state. In order to generate the vacuum, a pump (not shown) is arranged at the object chamber 426. Figure 5 In the exemplary embodiment shown, the object chamber 426 operates within a first pressure range or within a second pressure range. The first pressure range only includes pressures less than or equal to 10 -3 hPa, and the second pressure range only includes pressures greater than 10 -3 In order to ensure the pressure range, the object chamber 426 is vacuum-tight.

[0136] The object lens 421 can be implemented as a combination of a magnetic lens 422 and a sixth electrostatic lens 423. Further, the end of the beam guide tube 420 can be an electrode of an electrostatic lens. After exiting from the beam guide tube 420, the particles of the particle beam device 400 are decelerated to the potential of the object 425. The object lens 421 is not limited to the combination of the magnetic lens 422 and the sixth electrostatic lens 423. Instead, the object lens 421 can take any suitable form. For example, the object lens 421 can also be implemented as a pure magnetic lens or a pure electrostatic lens.

[0137] The particle beam focused on the object 425 interacts with the object 425. Interaction particles are generated. In particular, secondary electrons are emitted from the object 425, or backscattered electrons are backscattered from the object 425. The secondary electrons or backscattered electrons are accelerated again and guided into the beam guide tube 420 along the third optical axis OA3. Specifically, the trajectories of the secondary electrons and the backscattered electrons extend in the opposite direction to the particle beam on the route of the beam path of the particle beam.

[0138] The particle beam device 400 includes a first analysis detector 419, which is arranged along the beam path between the beam deflection device 410 and the objective lens 421. Secondary electrons traveling in a direction oriented at a large angle relative to the third optical axis OA3 are detected by the first analysis detector 419. Backscattered electrons and secondary electrons having a small axial distance from the third optical axis OA3 at the position of the first analysis detector 419 (i.e., backscattered electrons and secondary electrons having a small distance from the third optical axis OA3 at the position of the first analysis detector 419) enter the beam deflection device 410 and are deflected by the fifth magnetic sector 411E, the sixth magnetic sector 411F and the seventh magnetic sector 411G along the detection beam path 427 to the second analysis detector 428. By way of example, the deflection angle is 90° or 110°.

[0139] The first analysis detector 419 generates a detection signal mainly generated by the emitted secondary electrons. The detection signal generated by the first analysis detector 419 is guided to the image processing device 123 and is used to obtain information about the characteristics of the interaction area between the focused particle beam and the object 425. Specifically, the focused particle beam is scanned over the object 425 using a scanning device 429. Afterwards, an image of the scanned area of ​​the object 425 can be generated by the detection signal generated by the first analysis detector 419, and the image can be displayed on a display unit. The display unit is, for example, a monitor 124 arranged at the image processing device 123. Moreover, the image processing device 123 includes a database 129.

[0140] The second analysis detector 428 is also connected to the image processing device 123. The detection signal generated by the second analysis detector 428 is supplied to the image processing device 123 and is used to generate an image of the scanned area of ​​the object 425 and display the image on a display unit. The display unit is, for example, a monitor 124 arranged at the image processing device 123.

[0141] A radiation detector 500 is arranged at the object chamber 426 for detecting interaction radiation, such as X-ray radiation and / or cathode luminescence. The radiation detector 500 is connected to an image processing device 123 including a monitor 124. The image processing device 123 processes the detection signals generated by the radiation detector 500 and displays these detection signals on the monitor 124 in the form of images.

[0142] Further, the particle beam device 400 has a movable chamber detector 119. The chamber detector 119 can be a particle detector. For example, the chamber detector 119 can be moved to or away from a specific detection position in the object chamber 426 of the particle beam device 400. The chamber detector 119 is connected to a drive unit 127, and can be moved in the x direction (i.e., the first unit direction), in the y direction (i.e., the second unit direction) and in the z direction (i.e., the third direction) using the drive unit 127. All directions are perpendicular to each other. In addition, the chamber detector 119 can rotate around the first unit rotation axis and the second unit rotation axis arranged perpendicular to the first unit rotation axis, wherein the drive unit 127 is used for the rotation. The drive unit 127 can be a motor, such as a stepping motor or a piezoelectric motor. It is clearly mentioned that the drive unit 127 is not limited to the above-mentioned embodiment. Instead, the drive unit 127 can be any drive unit suitable for the present invention.

[0143] Furthermore, particle beam device 400 comprises a processor unit 128 having a processor in which a program code is loaded for controlling image processing device 123 of particle beam device 400 in such a way that the method according to the present invention is carried out.

[0144] Figure 6 600, in particular a laser scanning microscope). The light microscope 600 comprises a light beam generator 601, in particular a laser beam generator, and an objective lens 602, which focuses the light beam generated by the light beam generator 601 onto an object 125 arranged on an object stage 114. The object stage 114 is implemented to be movable in three directions arranged perpendicularly to each other, namely, the x-direction (first stage axis), the y-direction (second stage axis) and the z-direction (third stage axis). Moreover, the object stage 114 can rotate around two rotation axes arranged perpendicularly to each other, namely the first stage rotation axis and the second stage rotation axis. The light microscope 600 also comprises an image processing device 123, which has a monitor 124, a processor unit 128 and a database 129.

[0145] The processor unit 128 of the light microscope 600 has a processor into which a program code is loaded for controlling the image processing device 123 of the light microscope 600 in such a way that the method according to the present invention is carried out.

[0146] Figure 7 1 is a schematic illustration of an exemplary embodiment of an independent image processing device 123 that is not part of a specific beam device. The image processing device 123 of this embodiment is used only for image processing. The image processing device 123 has a monitor 124, a processor unit 128 and a database 129. Several images of an object generated by a beam device such as SEM 100, a combined device 200, a particle beam device 400 and / or a light microscope 600 are stored in the database 129. The processor unit 128 has a processor in which a program code is loaded for controlling the image processing device 123 in a manner that performs the method according to the present invention.

[0147] Figure 8 An exemplary embodiment of a method according to the invention using a beam device in the form of a SEM 100, a combined device 200, a particle beam device 400 and / or a light microscope 600 is shown. The method is explained in the following in an exemplary manner based on the operation of the SEM 100. Statements made with respect to processing an image generated by imaging an object 125 using the SEM 100 also apply mutatis mutandis to methods for processing images generated by imaging an object 125, 425 using other beam devices in the form of a combined device 200, a particle beam device 400 and / or a light microscope 600.

[0148] In method step S1, at least one first image of object 125 is provided. The first image Fig. 9100 and denoted by reference numeral 610. The first image 610 of the object 125 may be generated using the SEM 100 and / or by loading the first image 610 of the object 125 from the database 129 into the image processing device 123. The database 129 may include several first images 610 of the object 125, wherein the first images 610 have been generated using the SEM 100. The content of the database 129 may be provided by the manufacturer of the SEM 100 and / or by a user of the SEM 100.

[0149] The provided first image 610 includes n pixels P11 to P1 n , where n ≥ 2, and where n is an integer such as n ≥ 850.000. The n pixels P11 to P1 n Each pixel P1 in i The position PL1 arranged in the first image 610 relative to the first image coordinate system i where i is an integer of 1≤i≤n, and where each position PL1 in the first image 610 i Only the n pixels P11 to P1 n A single pixel in P1 i In other words, each position PL1 i No more than these n pixels P11 to P1 n A single pixel P1 in i . Each position PL1 i It is clearly identified by its coordinates relative to a first image coordinate system (which may be, for example, a Cartesian coordinate system). Fig. 9 The embodiment of the first image 610 shown includes 16 pixels. Therefore, n=16. It is explicitly noted that Fig. 9 The illustrated embodiment of the first image 610 is for illustration purposes only. In practice, the first image 610 typically comprises more than 16 pixels. As mentioned above, n ≥ 850.000.

[0150] like Fig. 9 As shown, an exemplary embodiment of the first image 610 includes a pixel P11 located at a position PL11, a pixel P12 located at a position PL12, a pixel P13 located at a position PL13, a pixel P14 located at a position PL14, a pixel P15 located at a position PL15, a pixel P16 located at a position PL16, a pixel P17 located at a position PL17, a pixel P18 located at a position PL18, a pixel P19 located at a position PL19, a pixel P20 located at a position PL21, a pixel P21 located at a position PL22, a pixel P22 located at a position PL23, a pixel P23 located at a position PL24, a pixel P24 located at a position PL25, a pixel P25 located at a position PL26, a pixel P26 located at a position PL27, a pixel P27 located at a position PL28, a pixel 10 Pixel P1 10 , located at position PL1 11 Pixel P1 11 , located at position PL1 12Pixel P1 12 , located at position PL1 13 Pixel P1 13 , located at position PL1 14 Pixel P1 14 , located at position PL1 15 Pixel P1 15 and at location PL1 16 Pixel P1 16 . Each position PL1 i The coordinates x, y of the pixel position PL18 are clearly identified relative to the first image coordinate system. For example, the coordinates of the position PL18 of the pixel P18 are (4, 3). The statements made about the coordinates of the position PL18 of the pixel P18 also apply, mutatis mutandis, to all other positions of all other pixels of the first image 610.

[0151] In method step S2, at least one second image of object 125 is provided. The second image Fig.10 100 and denoted by reference numeral 611. The second image 611 of the object 125 may be generated using the SEM 100 and / or by loading the second image 611 of the object 125 from the database 129 into the image processing device 123. The database 129 may include several second images 611 of the object 125, wherein the second images 611 have been generated using the SEM 100. As described above, the content of the database 129 may be provided by the manufacturer of the SEM 100 and / or by a user of the SEM 100.

[0152] The provided second image 611 includes m pixels P21 to P2 m , where m ≥ 2, and where m is an integer such as m ≥ 850.000. In addition, the following conditions may apply: m ≠ n or m = n. The m pixels P21 to P2 m Each pixel P2 in j The position PL2 arranged in the second image 611 with respect to the second image coordinate system j Where j is an integer of 1≤j≤m, and where each position PL2 in the second image 611 j Only these m pixels P21 to P2 m A single pixel in P2 j In other words, each position PL2 j Does not include more than these m pixels P11 to P1 m A single pixel P2 in j . PL2 per position j It is clearly identified by its coordinates relative to a second image coordinate system (which may be, for example, a Cartesian coordinate system). Fig.10The embodiment of the second image 611 shown comprises 16 pixels. Therefore, m=16. It is explicitly noted that Fig.10 The illustrated embodiment of the second image 611 is for illustration purposes only. In practice, the second image 611 typically comprises more than 16 pixels. As mentioned above, m ≥ 850.000.

[0153] like Fig.10 As shown, the exemplary embodiment of the second image 611 includes a pixel P21 located at a position PL21, a pixel P22 located at a position PL22, a pixel P23 located at a position PL23, a pixel P24 located at a position PL24, a pixel P25 located at a position PL25, a pixel P26 located at a position PL26, a pixel P27 located at a position PL27, a pixel P28 located at a position PL28, a pixel P29 located at a position PL29, and a pixel P21 located at a position PL21. 10 Pixel P2 10 , located at position PL2 11 Pixel P2 11 , located at position PL2 12 Pixel P2 12 , located at position PL2 13 Pixel P2 13 , located at position PL2 14 Pixel P2 14 , located at position PL2 15 Pixel P2 15 and located at position PL2 16 Pixel P2 16 . PL2 per position j The coordinates x, y of the pixel position PL28 are clearly identified relative to the second image coordinate system. For example, the coordinates of the position PL28 of the pixel P28 are (4, 3). The statements made about the coordinates of the position PL28 of the pixel P28 also apply to all other positions of all other pixels of the second image 611, mutatis mutandis.

[0154] In method step S3, the n pixels P11 to P1 n At least one pixel P1 in i , where the identified pixel P1 i The position PL1 in the first image 610 i For example, the n pixels P11 to P1 of the first image 610 are identified by one of the following: n Pixel P1 in i : (i) randomly identifying the n pixels P11 to P1 n Pixel P1 in i , (ii) identifying the n pixels P of the first image 610 according to the first random patternx1 To P1 n Pixel P1 in i , and (iii) identifying the pixel P1 among the n pixels P11 to P1x of the first image 610 according to the first given pattern i The first random pattern and / or the first given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the first random pattern and / or the first given pattern may be a line, a cross or a circle.

[0155] In method step S3, more than one pixel P1 may be identified in the first image 610. i For example, the identified pixel P1 i is the first pixel P1 to be identified i Furthermore, the n pixels P11 to P1 of the first image 610 are identified. n At least one second pixel P1 in o , where the identified second pixel P1 o The position PL1 in the first image 610 o Where o is an integer of 1≤o≤n and o≠i. The n pixels P11 to P1 of the first image 610 may be identified by one of the following: n The second pixel P1 in o : (i) randomly identifying the n pixels P11 to P1 n The second pixel P1 in o , (ii) identifying the n pixels P11 to P1 of the first image 610 according to the third random pattern n The second pixel P1 in o , and (iii) identifying the n pixels P11 to P1 of the first image 610 according to the third given pattern n The second pixel P1 in o The third random pattern and / or the third given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the third random pattern and / or the third given pattern may be a line, a cross or a circle.

[0156] like Fig.11 As shown, an exemplary embodiment of the first image 610 includes a first pixel P11 in the form of a pixel P11 located at a position PL11. i and is located at position PL1 16 Pixel P1 at 16 The second pixel P1 of the form o , which pixels have already been identified and selected in method step S3 .

[0157] In method step S3, the n pixels P11 to P1 n The first pixel P1 in i and the second pixel P1 o The n pixels P11 to P1 of the first image 610 are identified in a manner that they are adjacent to each other. n The first pixel P1 in i and the second pixel P1 o For example, the n pixels P11 to P1 n Several first pixels P1 in i and a plurality of second pixels P1 o A line can be formed. Fig.11 As shown, an exemplary embodiment of the first image 610 includes a pixel P15 located at a position PL15, a pixel P16 located at a position PL16, a pixel P17 located at a position PL17, and a pixel P18 located at a position PL18, which have been identified and selected in method step S3. Pixels P15, P16, P17, and P18 form a line, which can be a scan line for scanning the object 125 using the SEM 100.

[0158] In method step S4, the m pixels P21 to P2 m At least one pixel P2 in j , where the identified pixel P2 j The position PL2 in the second image 611 j For example, the m pixels P21 to P2 of the second image 611 are identified by one of the following: m Pixel P2 in j : (i) Randomly identify the m pixels P21 to P2 of the second image 611 m Pixel P2 in j , (ii) identifying the m pixels P21 to P2 of the second image 611 according to the second random pattern m Pixel P2 in j , and (iii) identifying the m pixels P21 to P2 of the second image 611 according to the second given pattern m Pixel P2 in j The second random pattern and / or the second given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the second random pattern and / or the second given pattern may be a line, a cross or a circle.

[0159] In method step S4, more than one pixel P2 can be identified in the second image 611 j For example, the identified pixel P2j is the first pixel P2 to be identified j Furthermore, the m pixels P21 to P2 of the second image 611 are identified. m At least one second pixel P2 p , where the identified second pixel P2 p The position PL2 in the second image 611 p Where p is an integer of 1≤p≤m and p≠j. The m pixels P21 to P2 of the second image 611 can be identified by one of the following: m The second pixel P2 in p : (i) Randomly identify the m pixels P21 to P2 of the second image 611 m The second pixel P2 in p , (ii) identifying the m pixels P21 to P2 of the second image 611 according to the fourth random pattern m The second pixel P2 in p , and (iii) identifying the m pixels P21 to P2 of the second image 611 according to the fourth given pattern m The second pixel P2 in p The fourth random pattern and / or the fourth given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the fourth random pattern and / or the fourth given pattern may be a line, a cross or a circle.

[0160] like Fig.12 As shown, an exemplary embodiment of the second image 611 includes a first pixel P2 in the form of a pixel P21 located at a position PL21. j and is located at position PL2 16 Pixel P2 at 16 The second pixel P2 of the form p , which pixels have already been identified and selected in method step S4 .

[0161] In method step S4, the m pixels P21 to P2 m The first pixel P2 in j and the second pixel P2 p The m pixels P21 to P2 of the second image 611 are identified in a manner that they are adjacent to each other. m The first pixel P2 in j and the second pixel P2 p For example, the m pixels P21 to P2 m Several first pixels P2 in j and a plurality of second pixels P2 p A line can be formed. Fig.12As shown, an exemplary embodiment of the second image 611 includes a pixel P25 at position PL25, a pixel P26 at position PL26, a pixel P27 at position PL27, and a pixel P28 at position PL28, which have been identified and selected in method step S4. Pixels P25, P26, P27, and P28 form a line, which can be a scan line for scanning the object 125 using the SEM 100.

[0162] In method step S5, at least one third image is generated. Fig.13 The third image 612 is schematically shown in FIG. 6 and is denoted by reference numeral 612. The third image 612 is based on the second image 611. However, not all pixels P2 of the second image 611 have j , but the generated third image 612 includes some (some) pixels P2 of the second image 611 j and / or P2 p and a certain pixel (or pixels) P1 of the first image 610 i and / or P1 o More specifically, the identified pixel(s) P1 of the first image 610 are i The identified pixel(s) P2 introduced into the second image 611 j Location PL2 j Furthermore, the identified pixel(s) P1 of the first image 610 are o The identified pixel(s) P2 introduced into the second image 611 p Location PL2 p In other words, the identified pixel(s) P1 of the first image 610 is i The value (eg, grayscale value) of is introduced into the identified pixel(s) P2 in the second image 611. j Location(s) PL2 j and the identified pixel(s) P1 of the first image 610 o The value (eg, grayscale value) of is introduced into the identified pixel(s) P2 in the second image 611. p Location(s) PL2 p to generate the third image 612. Accordingly, the generated third image 612 includes the following pixels: pixel P11 at position PL21, pixel P22 at position PL22, pixel P23 at position PL23, pixel P24 at position PL24, pixel P15 at position PL25, pixel P16 at position PL26, pixel P17 at position PL27, pixel P18 at position PL28, pixel P29 at position PL29, pixel P30 at position PL31, pixel P31 at position PL32, pixel P33 at position PL33, pixel P34 at position PL34 10Pixel P2 10 , located at position PL2 11 Pixel P2 11 , located at position PL2 12 Pixel P2 12 , located at position PL2 13 Pixel P2 13 , located at position PL2 14 Pixel P2 14 , located at position PL2 15 Pixel P2 15 and located at position PL2 16 Pixel P1 16 .

[0163] In method step S6, at least one fourth image is generated. Fig.14 The fourth image 613 is schematically shown in FIG. 6 and is denoted by reference numeral 613. The fourth image 613 is based on the first image 610. However, not all pixels P1 of the first image 610 are i , but the generated fourth image 613 includes some pixel (s) P1 of the first image 610 i and / or P1 o and a certain pixel (or pixels) P2 of the second image 611 j and / or P2 p More specifically, the identified pixel(s) P2 of the second image 611 are j The identified pixel(s) P1 introduced into the first image 610 i Location PL1 i Furthermore, the identified pixel(s) P2 of the second image 611 are p The identified pixel(s) P1 introduced into the first image 610 o Location PL1 o In other words, the identified pixel(s) P2 of the second image 611 are j The value (eg, grayscale value) of is introduced into the identified pixel(s) P1 in the first image 610. i Location(s) PL1 i and the identified pixel(s) P2 of the second image 611 p The value (eg, grayscale value) of is introduced into the identified pixel(s) P1 in the first image 610. o Location(s) PL1 oto generate the fourth image 613. Accordingly, the generated fourth image 613 includes the following pixels: a pixel P21 at position PL11, a pixel P12 at position PL12, a pixel P13 at position PL13, a pixel P14 at position PL14, a pixel P25 at position PL15, a pixel P26 at position PL16, a pixel P27 at position PL17, a pixel P28 at position PL18, a pixel P19 at position PL19, a pixel P20 at position PL11, a pixel P21 at position PL12, a pixel P22 at position PL13, a pixel P23 at position PL14, a pixel P24 at position PL15, a pixel P25 at position PL16, a pixel P26 at position PL17, a pixel P27 at position PL18, a pixel P28 at position PL19, a pixel P19 at position PL11 10 Pixel P1 10 , located at position PL1 11 Pixel P1 11 , located at position PL1 12 Pixel P1 12 , located at position PL1 13 Pixel P1 13 , located at position PL1 14 Pixel P1 14 , located at position PL1 15 Pixel P1 15 and at location PL1 16 Pixel P2 16 .

[0164] In other words, the identified pixel P1 of the first image 610 may be i (It is located at position PL1 in the first image 610 i ) and the identified pixel P2 of the second image 611 j (It is located at position PL2 in the second image 611 j In addition, the identified pixel P1 of the first image 610 may be o (It is located at position PL1 in the first image 610 o ) and the identified pixel P2 of the second image 611 p (It is located at position PL2 in the second image 611 p place) for exchange.

[0165] By identifying the pixel P1 i With the identified pixel P2 j exchange, and / or by placing the identified pixel P1 o With the identified pixel P2 p The third image 612 and the fourth image 613 are exchanged to generate new images. The generated third image 612 is based on at least one pixel (ie, P1) of the first image 610. i and / or P1 o) of the second image 611. In addition, the generated fourth image 613 is based on at least one pixel (ie, P2 j and / or P2 p )'s first image 610.

[0166] Figures 11 to 14 The embodiment shown in FIG. 6 is based on the idea of ​​transforming the identified pixel P1 of the first image 610 into i (wherein, pixel P1 i The position PL1 in the first image 610 i ) is arranged at the identified pixel P2 j Location PL2 j At, the identified pixel P2 j Location PL2 j The pixel P1 corresponding to the first image 610 is i Location PL1 i In other words, the position PL1 of the first image 610 in the first coordinate system i The coordinates of the second image 611 are the same as the position PL2 in the second coordinate system j Furthermore, if pixel P1 is identified o , then the identified pixel P1 of the first image 610 o (wherein, pixel P1 o The position PL1 in the first image 610 o ) is arranged at the identified pixel P2 p Location PL2 p At, the identified pixel P2 p Location PL2 p The pixel P1 corresponding to the first image 610 is o Location PL1 o In other words, the position PL1 of the first image 610 in the first coordinate system o The coordinates of the second image 611 are the same as the position PL2 in the second coordinate system p The coordinates are the same.

[0167] Figures 15 to 18 Embodiments based on different concepts are shown. Figures 15 to 18 In the embodiment shown, pixel P2 of the second image 611 j Location PL2 j does not correspond to the identified pixel P1 in the first image 610 i Location PL1 i In other words, the identified pixel P1 in the first coordinate system of the first image 610 i Location PL1 iThe coordinates of the pixel P2 in the second coordinate system of the second image 611 are not the same as j Location PL2 j This is explained as follows.

[0168] exist Figures 15 to 18 In the illustrated embodiment, in method step S3, the n pixels P11 to P1 n At least one pixel P1 in i The identified pixel P1 i The position PL1 in the first image 610 i For example, the n pixels P11 to P1 of the first image 610 are identified by one of the following: n Pixel P1 in i : (i) randomly identifying the n pixels P11 to P1 n Pixel P1 in i , (ii) identifying the n pixels P11 to P1 of the first image 610 according to the first random pattern n Pixel P1 in i , and (iii) identifying the n pixels P11 to P1 of the first image 610 according to the first given pattern n Pixel P1 in i The first random pattern and / or the first given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the first random pattern and / or the first given pattern may be a line, a cross or a circle. Fig.15 As shown, the exemplary embodiment of the first image 610 comprises a pixel P11 at a position PL11 , which has been identified and selected in method step S3 .

[0169] exist Figures 15 to 18 In the illustrated embodiment, in method step S4, the m pixels P21 to P2 m At least one pixel P2 in j The identified pixel P2 j The position PL2 in the second image 611 j For example, the m pixels P21 to P2 of the second image 611 are identified by one of the following: m Pixel P2 in j : (i) Randomly identify the m pixels P21 to P2 of the second image 611 m Pixel P2 in j , (ii) identifying the m pixels P21 to P2 of the second image 611 according to the second random pattern m Pixel P2 in j, and (iii) identifying the m pixels P21 to P2 of the second image 611 according to the second given pattern m Pixel P2 in j The second random pattern and / or the second given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the second random pattern and / or the second given pattern may be a line, a cross or a circle. Fig.16 As shown, the exemplary embodiment of the second image 611 includes a 16 Pixel P2 at 16 , which pixel has already been identified and selected in method step S4.

[0170] exist Figures 15 to 18 In the embodiment shown, at least one third image is generated in method step S5. Fig.17 The third image 612 is schematically shown in FIG. 6 and is denoted by reference numeral 612. The third image 612 is based on the second image 611. However, not all pixels P2 of the second image 611 have j , but the generated third image 612 includes some pixels P2 of the second image 611 j and some pixels P1 of the first image 610 i More specifically, the identified pixel P1 of the first image 610 is i The identified pixel P2 introduced into the second image 611 j Location PL2 j In other words, the identified pixel P1 of the first image 610 is i The value (eg, grayscale value) of is introduced into the identified pixel P2 in the second image 611 j Location PL2 j to generate the third image 612. Accordingly, the generated third image 612 includes the following pixels: a pixel P21 at position PL21, a pixel P22 at position PL22, a pixel P23 at position PL23, a pixel P24 at position PL24, a pixel P25 at position PL25, a pixel P26 at position PL26, a pixel P27 at position PL27, a pixel P28 at position PL28, a pixel P29 at position PL29, and a pixel P21 at position PL21. 10 Pixel P2 10 , located at position PL2 11 Pixel P2 11 , located at position PL2 12 Pixel P2 12 , located at position PL2 13 Pixel P2 13 , located at position PL2 14 Pixel P214 , located at position PL2 15 Pixel P2 15 and located at position PL2 16 Pixel P11.

[0171] exist Figures 15 to 18 In the embodiment shown, at least one fourth image is generated in method step S6. Fig.18 The fourth image 613 is schematically shown in FIG. 6 and is denoted by reference numeral 613. The fourth image 613 is based on the first image 610. However, not all pixels P1 of the first image 610 are i , but the generated fourth image 613 includes some pixels P1 of the first image 610 i and some pixels P2 of the second image 611 j More specifically, the identified pixel P2 of the second image 611 is j The identified pixel P1 introduced into the first image 610 i Location PL1 i In other words, the identified pixel P2 of the second image 611 is j The value (eg, grayscale value) of is introduced into the identified pixel P1 in the first image 610 i Location PL1 i , for generating the fourth image 613. Accordingly, the generated fourth image 613 includes the following pixels: pixel P2 at position PL11 16 , pixel P12 at position PL12, pixel P13 at position PL13, pixel P14 at position PL14, pixel P15 at position PL15, pixel P16 at position PL16, pixel P17 at position PL17, pixel P18 at position PL18, pixel P19 at position PL19, pixel P20 at position PL21, pixel P21 at position PL22, pixel P22 at position PL23, pixel P23 at position PL24, pixel P24 at position PL25 10 Pixel P1 10 , located at position PL1 11 Pixel P1 11 , located at position PL1 12 Pixel P1 12 , located at position PL1 13 Pixel P1 13 , located at position PL1 14 Pixel P1 14 , located at position PL1 15 Pixel P1 15 and at location PL1 16 Pixel P1 16 .

[0172] In other words, the identified pixel P1 of the first image 610 isi (It is located at position PL1 in the first image 610 i ) and the identified pixel P2 of the second image 611 j (It is located at position PL2 in the second image 611 j By replacing the identified pixel P1 i With the identified pixel P2 j The third image 612 and the fourth image 613 are exchanged to generate new images. The generated third image 612 is based on at least one pixel (ie, P1) of the first image 610. i ) of the second image 611. In addition, the generated fourth image 613 is based on at least one pixel (ie, P2 j )'s first image 610.

[0173] It is clearly pointed out that in Figures 15 to 18 In the embodiment shown, more than one pixel may be identified. In other words, in method step S3, more than one pixel P1 may be identified in the first image 610. i For example, the identified pixel P1 i is the first pixel P1 to be identified i Furthermore, the n pixels P11 to P1 of the first image 610 are identified. n At least one second pixel P1 in o , where the identified second pixel P1 o The position PL1 in the first image 610 o Where o is an integer of 1≤o≤n and o≠i. The n pixels P11 to P1 of the first image 610 may be identified by one of the following: n The second pixel P1 in o : (i) randomly identifying the n pixels P11 to P1 n The second pixel P1 in o , (ii) identifying the n pixels P11 to P1 of the first image 610 according to the third random pattern n The second pixel P1 in o , and (iii) identifying the n pixels P11 to P1 of the first image 610 according to the third given pattern n The second pixel P1 in o The third random pattern and / or the third given pattern may have any embodiment suitable for carrying out the method according to the invention. For example, the third random pattern and / or the third given pattern may be a line, a cross or a circle. Furthermore, in method step S4, more than one pixel P2 may be identified in the second image 611. j For example, the identified pixel P2j is the first pixel P2 to be identified j Furthermore, the m pixels P21 to P2 of the second image 611 are identified. m At least one second pixel P2 p , where the identified second pixel P2 p The position PL2 in the second image 611 p Where p is an integer of 1≤p≤m and p≠j. The m pixels P21 to P2 of the second image 611 can be identified by one of the following: m The second pixel P2 in p : (i) Randomly identify the m pixels P21 to P2 of the second image 611 m The second pixel P2 in p , (ii) identifying the m pixels P21 to P2 of the second image 611 according to the fourth random pattern m The second pixel P2 in p , and (iii) identifying the m pixels P21 to P2 of the second image 611 according to the fourth given pattern m The second pixel P2 in p The fourth random pattern and / or the fourth given pattern may have any embodiment suitable for performing the method according to the present invention. For example, the fourth random pattern and / or the fourth given pattern may be a line, a cross or a circle. i and P2 j The statements made also apply mutatis mutandis to the exchange of a number of pixels P1 i 、P1 o 、P2 j and P2 p .

[0174] As described above, an embodiment of the method according to the invention provides for exchanging pixels forming a line. This embodiment is advantageous if the first image 610 and / or the second image 611 include noise correlations such as horizontal noise correlations and / or vertical noise correlations. Noise correlations may occur in the SEM 100 if, for example, the first detector 116 and the second detector 117 of a beam device such as the SEM 100 acquire detection signals of a first pixel of the scanned area of ​​the object 125 despite the fact that the primary electron beam is no longer focused on the first pixel but has been focused on a second pixel of the scanned area of ​​the object 125. If there is no noise correlation (i.e., if the first value of the first noise signal at the first position of the first pixel is independent of the second value of the second noise signal at the second position of the second pixel), the pixels can be arbitrarily identified and exchanged.

[0175] As described above, at least one first image 610 of the object 125 is provided in method step S1. Furthermore, at least one second image 611 of the object 125 is provided in method step S2. In an embodiment of the method according to the invention, several first images 610 of the object 125 are provided in method step S1. The several first images 610 of the object 125 may be generated using the SEM 100 and / or by loading the several first images 610 of the object 125 from a database 129 into the image processing device 123. The content of the database 129 may be provided by the manufacturer of the SEM 100 and / or by a user of the SEM 100. Furthermore, in an embodiment of the method according to the invention, several second images 611 of the object 125 are provided in method step S2. The several second images 611 of the object 125 may be generated using the SEM 100 and / or by loading the several second images 611 of the object 125 from a database 129 into the image processing device 123. The content of database 129 may be provided by the manufacturer of SEM 100 and / or by a user of SEM 100. In addition, a number of third images 612 and a number of fourth images 613 are generated. The statements made regarding the generation of third images 612 and fourth images 613 also apply mutatis mutandis to the generation of the number of third images 612 and the number of fourth images 613.

[0176] like Figure 8 As shown, method step S7 provides for training a processor unit 128 of a processor including the SEM 100. The processor unit 128 is trained using at least one of the generated (multiple) third images 612 and the generated (multiple) fourth images 613 for identifying a denoising function. In other words, the generated (multiple) third images 612 and / or the generated (multiple) fourth images 613 are used as inputs to the processor unit 128. The processor unit 128 calculates a denoising function that can be used to denoise any image generated by using the SEM 100. Training the processor unit 128 may include using a machine learning system, such as a noise-noise machine learning system. The machine learning system is at least one of an artificial neural network, a support vector machine, a Bayesian network, a random decision forest, and association rule learning. An exemplary embodiment of a method according to the present invention will now be described with respect to using an artificial neural network.

[0177] Fig.19 A first embodiment of training the processor unit 128 in method step 7 is schematically shown. Method step S7 comprises method steps S7A to S7D which will be explained in the following.

[0178] In method step S7A, an artificial neural network (eg Figure 2The artificial neural network 900 shown in the figure denoises the generated third image 612. At the beginning, the weighting factor of each neuron 800 of the artificial neural network 900 is randomly selected or selected based on previous experience. The pixel data of the generated third image 612 is used as the input of the neuron 800 of the first layer. The output of the artificial neural network 900 is the pixel data of the denoised third image 612. In method step S7B, the generated third image 612 is compared with the denoised third image 612. Moreover, in method step S7C, the difference between the generated third image 612 and the denoised third image 612 is calculated, wherein the difference can be calculated using a loss function. In particular, the back propagation algorithm known in the art is used to identify those neurons 800 in the artificial neural network 900 that have a significant contribution to the denoised third image 612. All weighting factors and / or bias values ​​of all neurons 800 are changed in a way that minimizes the loss function. In other words, the change of the above-mentioned weighting factors and / or bias values ​​is performed in a manner that moves towards the direction of the steepest gradient descent or a similar descent. After changing the weighting factors and / or bias values, method steps S1 to S6 are repeated to generate another third image 612. The pixel data of the generated another third image 612 is used as input to the neurons 800 of the first layer. The output of the artificial neural network 900 is the pixel data of the denoised another third image 612. In method step S7B, the generated another third image 612 is compared with the denoised another third image 612. Moreover, in method step S7C, the difference between the generated another third image 612 and the denoised another third image 612 is calculated, wherein the difference can be calculated using a loss function.

[0179] If the difference increases, the back propagation algorithm known in the art is used in particular to identify those neurons 800 in the artificial neural network 900 that have a significant contribution to the generated denoised further third image 612. All weighting factors and / or bias values ​​are changed in such a way that the loss function is minimized. In other words, the change of the above-mentioned weighting factors and / or bias values ​​is performed in such a way that the direction of the steepest gradient descent or a similar descent is advanced. After changing the weighting factors and / or bias values, method steps S1 to S6 are repeated to generate another third image 612 during training the processor unit 128, and it is used to train the processor unit 128 in method step S7.

[0180] If the difference decreases, the method provides weighting factors and / or bias values ​​in the right direction towards achieving a good quality denoised image. Method steps S1 to S7 are repeated. If the difference reaches a plateau, the noise in the denoised further third image 612 is sufficiently low. The weighting factors and / or bias values ​​are set in such a way that random images with similar noise characteristics and whose data are used as input to the artificial neural network 900 are properly denoised. Accordingly, the weighting factors and / or bias values ​​are the basis of a denoising function, which can be stored in the database 129 in method step S8.

[0181] Fig. 20 A second embodiment of training the processor unit 128 in method step 7 is schematically shown. In addition to method steps S7A to S7D, method step S7 comprises method steps S7E to S7G which will be explained below.

[0182] In method step S7E, an artificial neural network (e.g. Figure 2The method of claim 1 ) is to perform denoising on the generated fourth image 613 using the artificial neural network 900 shown in the figure. If the artificial neural network 900 is used for the first time, the weighting factor and / or bias value of each neuron 800 of the artificial neural network 900 is randomly selected. If, for example, the artificial neural network 900 has been used for the generated third image 612, the already set weighting factor and / or the set bias value of each neuron 800 of the artificial neural network 900 are used. The pixel data of the generated fourth image 613 is used as the input of the neurons 800 of the first layer. The output of the artificial neural network 900 is the pixel data of the denoised fourth image 613. In method step S7F, the generated fourth image 613 is compared with the denoised fourth image 613. Moreover, in method step S7G, the difference between the generated fourth image 613 and the denoised fourth image 613 is calculated, wherein the difference can be calculated using a loss function. In particular, a back propagation algorithm known in the art is used to identify those neurons 800 in the artificial neural network 900 that have a significant contribution to the denoised fourth image 613. All weighting factors and / or bias values ​​of all neurons 800 are changed in a way that minimizes the loss function. In other words, the change of the weighting factors and / or bias values ​​is performed in a way that moves towards the direction of the steepest gradient descent or a similar descent. After the weighting factors and / or bias values ​​have been changed, method steps S1 to S6 are repeated to generate another fourth image 613. The pixel data of the generated another fourth image 613 is used as input to the neurons 800 of the first layer. The output of the artificial neural network 900 is the pixel data of the denoised another fourth image 613. In method step S7F, the generated another fourth image 613 is compared with the denoised another fourth image 613. Moreover, in method step S7G, the difference between the generated another fourth image 613 and the denoised another fourth image 613 is calculated, wherein the difference can be calculated using a loss function.

[0183] If the difference increases, the back propagation algorithm known in the art is used in particular to identify those neurons 800 in the artificial neural network 900 that have a significant contribution to the denoised further fourth image 613. All weighting factors and / or bias values ​​of all neurons 800 are changed in such a way that the loss function is minimized. In other words, the change of the above-mentioned weighting factors and / or bias values ​​is performed in such a way that the direction of the steepest gradient descent or a similar descent is advanced. After the change of the weighting factors and / or bias values, the method steps S1 to S6 are repeated for generating another fourth image 613 during the training of the processor unit 128, and for training the processor unit 128 in the method step S7.

[0184] If the difference decreases, the method provides weighting factors and / or bias values ​​in the right direction towards achieving a good quality denoised image. Method steps S1 to S7 are repeated. If the difference reaches a plateau, the noise is sufficiently low. The weighting factors and / or bias values ​​are set in such a way that random images with similar noise characteristics and whose data are used as input to the artificial neural network 900 are properly denoised. Accordingly, the weighting factors and / or bias values ​​are the basis of a denoising function, which can be stored in the database 129 in method step S8.

[0185] Fig.21 A further embodiment of the method according to the invention is shown. In method step S7H, an artificial neural network (e.g. Figure 2 The artificial neural network 900 shown in FIG. 9A ) denoises the generated third image 612 and the generated fourth image 613. If the artificial neural network 900 is used for the first time, the weighting factor and / or bias value of each neuron 800 of the artificial neural network 900 is randomly selected. If, for example, the artificial neural network 900 has been used for the generated third image 612, the already set weighting factor and / or the set bias value of each neuron 800 of the artificial neural network 900 is used. The pixel data of the generated third image 612 and the generated fourth image 613 are used as inputs of the neurons 800 of the first layer. The output of the artificial neural network 900 is the pixel data of the denoised third image 612 and the denoised fourth image 613. In method step S7I, the generated third image 612 is compared with the denoised fourth image. Moreover, the generated fourth image 613 is compared with the denoised third image 612. In addition, the denoised third image 612 is compared with the denoised fourth image 613.

[0186] Furthermore, in method step S7J, the difference between the generated third image 612 and the denoised fourth image 613 is calculated. Furthermore, the difference between the generated fourth image 613 and the denoised third image 612 is calculated. Furthermore, the difference between the denoised third image 612 and the denoised fourth image 613 is calculated. A loss function may be used to calculate the difference. The loss function depends on the generated third image 612, the denoised third image 612, the generated fourth image 613 and the denoised fourth image 613. For example, the following loss function may be used:

[0187] L=α1L a (third image, denoised fourth image) + α2L b (fourth image, denoised third image) + α3L c (Denoised third image, Denoised fourth image)

[0188] Among them, α1, α2, α3, a, b and c are variables that can be appropriately selected.

[0189] In particular, a back propagation algorithm known in the art is used to identify those neurons 800 in the artificial neural network 900 that have a significant contribution to the denoised third image 612 and the denoised fourth image 613. All weighting factors and / or bias values ​​of all neurons 800 are changed in a way that minimizes the loss function. In other words, the change of the weighting factors and / or bias values ​​is performed in a way that moves towards the direction of the steepest gradient descent or a similar descent. After the weighting factors and / or bias values ​​are changed, method steps S1 to S6 are repeated to generate another third image 612 and another fourth image 613. The pixel data of the generated another third image 612 and the generated another fourth image 613 are used as inputs to the neurons 800 of the first layer. The output of the artificial neural network 900 is the pixel data of the denoised another third image 612 and the denoised another fourth image 613. In method step S7J, the difference between the generated another third image 612 and the denoised another fourth image 613 is calculated. Furthermore, a difference is calculated between the generated another fourth image 613 and the denoised another third image 612. Furthermore, a difference is calculated between the denoised another third image 612 and the denoised another fourth image 613. The difference may be calculated using the above-mentioned loss function.

[0190] If the difference according to the above-mentioned loss function increases, the back propagation algorithm known in the art is particularly used to identify those neurons 800 in the artificial neural network 900 that have a significant contribution to the denoised further third image 612 and the denoised further fourth image 613. All weighting factors and / or bias values ​​of all neurons 800 are changed in such a way that the loss function is minimized. In other words, the change of the above-mentioned weighting factors and / or bias values ​​is performed in such a way that the direction of the steepest gradient descent or a similar descent is advanced. After changing the weighting factors and / or bias values, method steps S1 to S6 are repeated to generate another third image 612 and another fourth image 613 during the training of the processor unit 128, and use them for training the processor unit 128 in method step S7.

[0191] If the difference according to the above loss function decreases, the method provides weighting factors and / or bias values ​​in the right direction towards achieving a good quality denoised image. Method steps S1 to S7 are repeated. If the difference reaches a plateau, the noise is sufficiently low. The weighting factors and / or bias values ​​are set in such a way that random images with similar noise characteristics and whose data are used as input to the artificial neural network 900 are properly denoised. Accordingly, the weighting factors and / or bias values ​​are the basis of a denoising function, which can be stored in the database 129 in method step S8.

[0192] All embodiments of the method according to the present invention provide iteration by repeating method steps S1 to S7. In each iteration, weighting factors and / or bias values ​​are changed. In other words, the back propagation algorithm known in the art is particularly used to identify those neurons 800 in the artificial neural network 900 that have a significant contribution to the denoised third image 612 and the denoised fourth image 613. All weighting factors and / or bias values ​​of all neurons 800 are changed. After changing the weighting factors and / or bias values, method steps S1 to S6 are repeated to generate another third image 612 and another fourth image 613 during training the processor unit 128, and use them to train the processor unit 128 in method step S7.

[0193] If the difference according to the loss function L decreases, the method provides weighting factors and / or bias values ​​in the right direction towards achieving a good quality denoised image. Method steps S1 to S7 are repeated. If the difference reaches a plateau, the noise is sufficiently low. The weighting factors and / or bias values ​​are set in such a way that random images with similar noise characteristics and whose data are used as input to the artificial neural network 900 are properly denoised. Accordingly, the weighting factors and / or bias values ​​of the neuron 800 are the basis of a denoising function, which can be stored in the database 129 in method step S8.

[0194] Fig. 22 Further method steps according to an embodiment of the method according to the invention are shown. These further method steps may be performed after performing method step S8. In method step S9, the denoising function is loaded from a memory unit in the form of a database 129 into the processor unit 128. Furthermore, in method step S10, the denoising function is used to denoise other images generated by imaging the object using the SEM 100.

[0195] The present invention provides a simple way to generate images during training a machine learning system (e.g., artificial neural network 900) to identify a denoising function. By increasing the number of images used to train the machine learning system, the quality of the denoising function is improved. Therefore, when the denoising function is used on any noisy image, the quality of denoising the noisy image is higher than the prior art. In addition, the present invention provides for generating images in a relatively short time (e.g., a few minutes) during training the machine learning system. Compared to the prior art, this is not time-consuming at all.

[0196] The method according to the invention can be carried out by a manufacturer of a beam device and / or by a user of a beam device.

[0197] The various embodiments discussed herein may be combined with each other in appropriate combinations in conjunction with the systems described herein. In addition, in some cases, the order of steps in the flowchart, process and / or described process processing may be modified in appropriate places. Further, various aspects of the system described herein may be implemented using software, hardware, a combination of software and hardware, and / or other computer-implemented modules or devices having the described features and performing the described functions. The system may further include a display and / or other computer components for providing an appropriate interface with a user and / or with other computers.

[0198] The software implementation of aspects of the system described herein may include executable code, which is stored in a computer-readable medium and executed by one or more processors. The computer-readable medium may include volatile memory and / or non-volatile memory, and may include, for example, a computer hard drive, ROM, RAM, flash memory, cloud storage, portable computer storage media, such as a CD-ROM, DVD-ROM, SO card, flash drive or other drive with, for example, a universal serial bus (USB) interface, and / or any other suitable tangible or non-temporary computer-readable medium or computer memory on which the executable code may be stored and executed by a processor. The system described herein may be used in conjunction with any suitable operating system.

[0199] The features of the invention disclosed in this specification, the drawings and the claims may be essential for implementing the invention in its various embodiments, either individually or in any combination. The invention is not limited to the described embodiments. They may be modified within the scope of the claims, taking into account the knowledge of the relevant technical personnel in the field.

[0200] *******

[0201] List of Reference Numerals

[0202] 100 SEM

[0203] 101 Electron Source

[0204] 102 Leading electrode

[0205] 103 Anode

[0206] 104 beam guide tube

[0207] 105 First beam focusing lens

[0208] 106 Second beam focusing lens

[0209] 107 First Objective

[0210] 108 First aperture unit

[0211] 108A First Aperture

[0212] 109 Second aperture unit

[0213] 110 Pole piece

[0214] 111 Coil

[0215] 112 Single Electrode

[0216] 113 Tubular Electrode

[0217] 114 Object carrier

[0218] 115 Scanning Device

[0219] 116 First Detector

[0220] 116A Opposite Field Grid

[0221] 117 Second Detector

[0222] 118 Second Aperture

[0223] 119 Chamber Detector

[0224] 120 Object Room

[0225] 121 Third Detector

[0226] 123 Image processing device

[0227] 124 Monitor

[0228] 125 Objects

[0229] 126 Adjustment mechanism

[0230] 127 Driver Unit

[0231] 128 Processor Unit with processor

[0232] 129 Database

[0233] 135 High voltage control unit

[0234] 136 Current Control Unit

[0235] 137 Asteresis Device

[0236] 139 beamforming control unit

[0237] 140 Scanning control unit

[0238] 141 Stigmator control unit

[0239] 200 Combination Devices

[0240] 201 Object Room

[0241] 202 Vacuum system

[0242] 203 valve

[0243] 204 Vacuum Control Unit

[0244] 300 Ion Beam Device

[0245] 301 Ion Beam Generator

[0246] 302 Extraction electrode in ion beam device

[0247] 303 Focusing lens

[0248] 304 Second objective lens

[0249] 306 Adjustable or selectable aperture units

[0250] 307 First Electrode Arrangement

[0251] 308 Second electrode arrangement

[0252] 400 Particle beam device with corrector unit

[0253] 401 particle beam column

[0254] 402 Electron Source

[0255] 403 Lead electrode

[0256] 404 Anode

[0257] 405 First Electrostatic Lens

[0258] 406 Second electrostatic lens

[0259] 407 The third electrostatic lens

[0260] 408 Magnetic Deflection Unit

[0261] 409 First electrostatic beam deflection unit

[0262] 409A First Multipole Unit

[0263] 409B Second multipole unit

[0264] 410 Beam Deflection Device

[0265] 411A First magnetic sector

[0266] 411B Second magnetic sector

[0267] 411C The third magnetic sector

[0268] 411D The fourth magnetic sector

[0269] 411E Fifth magnetic sector

[0270] 411F Sixth magnetic sector

[0271] 411G seventh magnetic sector

[0272] 413A First reflector electrode

[0273] 413B Second reflector electrode

[0274] 413C Third reflector electrode

[0275] 414 Electrostatic Mirror

[0276] 415 The fourth electrostatic lens

[0277] 416 Second electrostatic beam deflection unit

[0278] 416A Third multipole unit

[0279] 416B Fourth multipole unit

[0280] 417 Third Electrostatic Beam Deflection Unit

[0281] 418 Fifth Electrostatic Lens

[0282] 418A Fifth Multipole Unit

[0283] 418B Sixth multipole unit

[0284] 419 First Analytical Detector

[0285] 420 beam guide tube

[0286] 421 Objective

[0287] 422 Magnetic Lens

[0288] 423 Sixth Electrostatic Lens

[0289] 424 Object Carrier

[0290] 425 Objects

[0291] 426 Object Room

[0292] 427 Detection beam path

[0293] 428 Second analysis detector

[0294] 429 Scanning Device

[0295] 432 Additional magnetic deflection elements

[0296] 500 Radiation Detector

[0297] 600 Light Microscope

[0298] 601 Beam Generator

[0299] 602 Objectives of light microscopes

[0300] 610 First Image

[0301] 611 Second Image

[0302] 612 Third Image

[0303] 613 Fourth Image

[0304] 709 First Beam Axis

[0305] 710 Second beam axis

[0306] 800 Neurons

[0307] 900 Artificial Neural Networks

[0308] OA optical axis

[0309] OA1 First optical axis

[0310] OA2 Second optical axis

[0311] OA3 Third optical axis

[0312] S1 to S10 Method steps

[0313] S7A to S7G Method Steps

[0314] Activation Function

[0315] Activation Function

[0316] Activation Function

[0317] Activation Function

[0318] Activation Function

[0319] Activation Function

[0320] Activation Function

[0321] Activation Function

[0322] Activation Function

[0323] Activation Function

[0324] x1, x2, x3 input

[0325] w1, w2, w3 weighting factors

[0326] *******

Claims

1. A method for processing an image generated by imaging an object using a beam device, the method comprising the steps of: - providing at least one first image of the object, wherein the first image comprises n pixels P11 to P1 n , where n ≥ 2, where n is an integer, where these n pixels P11 to P1 n Each pixel P1 in i The position PL1 arranged in the first image relative to the first image coordinate system i where i is an integer of 1≤i≤n, and where each position PL1 in the first image i Only the n pixels P11 to P1 n A single pixel in - providing at least one second image of the object, wherein the second image comprises m pixels P21 to P2 m , where m ≥ 2, where m is an integer, where these m pixels P21 to P2 m Each pixel P2 in j The position PL2 arranged in the second image relative to the second image coordinate system j where j is an integer of 1≤j≤m, and where each position PL2 in the second image j Only these m pixels P21 to P1 are included m A single pixel in - Identify the n pixels P11 to P1 of the first image n At least one pixel P1 in i , where the identified pixel P1 i The position PL1 in the first image i Department; - Identify the m pixels P21 to P2 of the second image m At least one pixel P2 in j , where the identified pixel P2 j The position PL2 in the second image j Department; - by using the second image and by i The identified pixel P2 introduced into the second image j Location PL2 j to generate at least one third image, and / or by using the first image and by applying the identified pixel P2 of the second image to the j The identified pixel P1 introduced into the first image i Location PL1 i to generate at least one fourth image; - using at least one of the generated third image and the generated fourth image to train a processor unit for identifying a noise reduction function; and - Saving the identified noise reduction function in a memory unit.

2. The method according to claim 1, further comprising at least one of the following: - Identify the n pixels P11 to P1 of the first image n Pixel P1 in i The steps include one of the following: (i) randomly identifying the n pixels P11 to P1 n Pixel P1 in i (ii) identifying the n pixels P11 to P1 of the first image according to the first random pattern n Pixel P1 in i , and (iii) identifying the n pixels P11 to P1 of the first image according to the first given pattern n Pixel P1 in i ; - Identify the m pixels P21 to P2 of the second image m Pixel P2 in j The steps include one of the following: (i) randomly identifying the m pixels P21 to P2 m Pixel P2 in j (ii) identifying the m pixels P21 to P2 of the second image according to the second random pattern m Pixel P2 in j , and (iii) identifying the m pixels P21 to P2 of the second image according to the second given pattern m Pixel P2 in j .

3. The method according to claim 1 or 2, wherein: The n pixels P11 to P1 of the first image n The identified pixel P1 in i is the first pixel P1 identified i , wherein the m pixels P21 to P2 of the second image m The identified pixel P2 in j is the first pixel P2 identified j , and wherein the method further comprises: - Identify the n pixels P11 to P1 of the first image n At least one second pixel P1 o , where the identified second pixel P1 o The position PL1 in the first image o where o is an integer 1≤o≤n and o≠i; - Identify the m pixels P21 to P2 of the second image m At least one second pixel P2 p , where the identified second pixel P2 p The position PL2 in the second image p where p is an integer 1≤p≤m and p≠j; in The step of generating the third image comprises: o The identified second pixel P2 introduced into the second image p Location PL2 p and among them, The step of generating the fourth image comprises: p The identified second pixel P1 introduced into the first image o Location PL1 o Place.

4. The method according to claim 3, further comprising at least one of the following: - Identify the n pixels P11 to P1 of the first image n The second pixel P1 in o The steps include one of the following: (i) randomly identifying the n pixels P11 to P1 n The second pixel P1 in o (ii) identifying the n pixels P11 to P1 of the first image according to the third random pattern n The second pixel P1 in o , and (iii) identifying the n pixels P11 to P1 of the first image according to a third given pattern n The second pixel P1 in o ; - Identify the m pixels P21 to P2 of the second image m The second pixel P2 in p The steps include one of the following: (i) randomly identifying the m pixels P21 to P2 m The second pixel P2 in p (ii) identifying the m pixels P21 to P2 of the second image according to the fourth random pattern m The second pixel P2 in p , and (iii) identifying the m pixels P21 to P2 of the second image according to the fourth given pattern m The second pixel P2 in p .

5. The method according to claim 3, further comprising at least one of the following: - so that the n pixels P11 to P1 of the first image n The first pixel P1 in i and the n pixels P11 to P1 of the first image n The second pixel P1 in o The n pixels P11 to P1 of the first image are identified in a manner adjacent to each other. n The first pixel P1 in i and the n pixels P11 to P1 of the first image n The second pixel P1 in o ; - so that the m pixels P21 to P2 of the second image m The first pixel P2 in j and the m pixels P21 to P2 of the second image m The second pixel P2 in p The m pixels P21 to P2 of the second image are identified in a manner adjacent to each other.m The first pixel P2 in j and the m pixels P21 to P2 of the second image m The second pixel P2 in p .

6. The method according to claim 1 or 2, further comprising: -The step of training the processor unit includes using a machine learning system.

7. The method according to claim 6, wherein: The machine learning system is at least one of: an artificial neural network, a support vector machine, a Bayesian network, a random decision forest, and association rule learning.

8. The method according to claim 1 or 2, further comprising one of the following: (i) The step of training the processor unit comprises: (a) denoising the generated third image and generating a denoised third image; (b) comparing the generated third image with the generated denoised third image; (c) calculating a first difference between the generated third image and the generated denoised third image; and (d) identifying the noise reduction function using the calculated first difference; (ii) The step of training the processor unit includes: (a) denoising the generated third image and generating a denoised third image; (b) comparing the generated third image with the generated denoised third image; (c) using a first loss function to calculate a first difference between the generated third image and the generated denoised third image; and (d) using the calculated first difference to identify the denoising function.

9. The method according to claim 1 or 2, further comprising one of the following: (i) The step of training the processor unit comprises: (a) denoising the generated fourth image and generating a denoised fourth image; (b) comparing the generated fourth image with the generated denoised fourth image; (c) calculating a second difference between the generated fourth image and the generated denoised fourth image; and (d) using the calculated second difference to identify the noise reduction function; (ii) the step of training the processor unit includes: (a) denoising the generated fourth image and generating a denoised fourth image; (b) comparing the generated fourth image with the generated denoised fourth image; (c) using a second loss function to calculate a second difference between the generated fourth image and the generated denoised fourth image; and (d) using the calculated second difference to identify the denoising function.

10. The method according to claim 1 or 2, further comprising: -The step of training the processor unit includes using a noise-noise machine learning system.

11. The method according to claim 1 or 2, further comprising one of the following: (i) using the noise reduction function to denoise other images generated by imaging the object using the beam device; (ii) loading the denoising function from the memory unit into the processor unit and using the denoising function to denoise other images generated by imaging the object using the beam arrangement. 12 . A computer program product comprising a program code which is loaded into a processor and which, when executed, controls an image processing device in such a way that the method according to claim 1 is performed.

13. An image processing device comprising at least one memory unit and a processor unit having a processor, into which the computer program product according to claim 12 is loaded.

14. A beam device for imaging and / or analyzing an object, the beam device comprising - at least one beam generator for generating a beam; and - At least one image processing device according to claim 13.

15. The beam device according to claim 14, wherein: The beam generator is a beam generator for generating a light beam.

16. The beam device according to claim 15, wherein: The light beam is a laser beam.

17. The beam device according to any one of claims 14 to 16, wherein: The beam device is a particle beam device, wherein the beam generator is a particle beam generator for generating a primary particle beam having charged particles, and wherein the beam device further comprises: - at least one objective lens for focusing the primary particle beam onto the object; and - at least one detector for detecting interacting particles and / or interacting radiation generated when the primary particle beam impinges on the object.

18. The beam device according to claim 17, wherein: The particle beam generator is a first particle beam generator, wherein the primary particle beam is a first primary particle beam having first charged particles, wherein the objective lens is a first objective lens for focusing the first primary particle beam onto the object, and wherein the beam device further includes: a second particle beam generator for generating a second primary particle beam having second charged particles and a second objective lens for focusing the second primary particle beam onto the object.

19. The beam device according to claim 17, wherein: The beam device is at least one of: an electron beam device and an ion beam device.

Citation Information

Patent Citations

  • Depth residual convolution neural network image denoising method based on PReLU

    CN109118435A

  • Medical image processing apparatus and medical image processing system

    EP3467766A1

  • Particle beam system comprising a mirror corrector

    WO2002067286A2

  • Method and apparatus for image enhancement and edge verification using at least one additional image

    CN104272323A

  • Neural network training image generation system

    CN108830827A