Observation system and artifact correction method thereof

Abstract

Provided is technology capable of uniformly reducing artifacts that are in a reconstructed image and that change due to a sampling coordinate group. This observation system comprises an image capture device and a processor subsystem. The processor subsystem sets a sparse sampling coordinate group for a sample, acquires a pixel value group corresponding to the sampling coordinate group on the sample, and gives, to a correction engine, the pixel value group or a first reconstructed image that has been generated on the basis of the pixel value group, so that a second reconstructed image is generated.　The correction engine is trained using the following data (1) to (3) pertaining to the sample or a training sample: (1) the sampling coordinate group; (2) the pixel value group or a reconstructed image which has artifacts and which has been reconstructed from the pixel value group using the sampling coordinate group; and (3) an image which contains no artifacts or in which artifacts are reduced.

Description

[Technical field]

[0001] The present invention relates to an observation system and an artifact correction method thereof. [Background technology]

[0002] In image reconstruction from sparsely sampled data in observation devices such as a charged particle microscope device, a magnetic resonance imaging (MRI) device, and a computed tomography (CT) device, artifacts, which are secondary image patterns that do not exist in the original imaging subject, may occur in the reconstructed image. A technology for reducing such artifacts has been proposed, for example, in Patent Document 1.

[0003] In the method described in Patent Document 1, artifacts in a reconstructed image are reduced by using a correction engine that has learned to reduce artifacts in a reconstructed image. However, in this method, artifacts differ depending on the imaging method, so the effect of artifact reduction varies, making it difficult to reduce artifacts uniformly. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-99667 A Summary of the Invention [Problem to be solved by the invention]

[0005] In order to solve the above problem, an object of the present invention is to provide an observation system and an artifact correction method that can uniformly reduce artifacts that vary due to sampling coordinate groups in a reconstructed image. [Means for solving the problem]

[0006] In order to solve the above problems, an observation system according to the present invention includes an imaging device and a processor subsystem, which sets a sparse set of sampling coordinates for a sample, obtains a set of pixel values ​​corresponding to the set of sampling coordinates on the sample, and generates a second reconstructed image by providing a correction engine with either the set of pixel values ​​or a first reconstructed image generated based on the set of pixel values. The correction engine is trained using the following data (1) to (3) related to the sample or training sample: (1) The sampling coordinate group (2) a reconstructed image having artifacts, the image being reconstructed from the pixel values ​​or the pixel values ​​using the sampling coordinates; (3) Images that do not contain artifacts or have reduced artifacts Effect of the Invention

[0007] According to the present invention, it is possible to provide an observation system and an artifact correction method that can uniformly reduce artifacts that vary due to sampling coordinate groups in a reconstructed image. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram showing an example of a schematic configuration of a scanning electron microscope apparatus 100 according to a first embodiment. [Diagram 2] FIG. 2 is a schematic diagram for explaining an overview of artifact correction in the scanning electron microscope apparatus 100 according to the first embodiment. [Diagram 3] 10 is a flowchart illustrating an example of an execution procedure of artifact correction processing in the first embodiment. [Figure 4] 4 is a flowchart illustrating a specific example of a learning sequence of a correction engine used in step S305 of FIG. 3. [Diagram 5]10A and 10B show schematic diagrams of a reconstructed image 510 of the region of interest 203, an image 520 obtained by mapping the degree of artifact occurrence of the reconstructed image 510, and an example of a sparse sampling coordinate group 530 that is additionally set in the region of interest 203. [Figure 6] 13 is a flowchart illustrating details of a procedure for additionally setting a sampling coordinate group in an area with a high degree of artifact occurrence in step S308. [Figure 7] 13 is a flowchart according to a first modified example of the first embodiment. [Figure 8] 13 is a flowchart according to a second modified example of the first embodiment. [Figure 9] 13 shows an example of a GUI screen that displays an image in which the degree of artifact occurrence is mapped together with a reconstructed image on the screen of the display 156 in the device according to the first embodiment. [Figure 10] 10 is a flowchart illustrating an example of an execution procedure of artifact correction processing in the second embodiment. [Figure 11] 13 is a flowchart illustrating an example of a procedure for learning a correction engine according to the second embodiment. [Figure 12] 13 is a flowchart illustrating an example of an execution procedure of artifact correction processing in the third embodiment. [Figure 13] 13 is a flowchart illustrating a specific example of a learning sequence of the correction engine used in step S1206. [Figure 14] 13 is a flowchart illustrating an example of a specific procedure for aligning a sampling coordinate group based on design data in step S1203 according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the present embodiment will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be indicated by the same numbers. Note that the accompanying drawings show embodiments and implementation examples according to the principles of the present disclosure, but these are for understanding the present disclosure and are not to be used to interpret the present disclosure in a limiting manner. The description in this specification is merely a typical example and does not limit the scope or application of the present disclosure in any sense.

[0010] In the present embodiment, the description is given in sufficient detail for a person skilled in the art to implement the present disclosure, but it should be understood that other implementations and forms are possible, and that changes in configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present disclosure. Therefore, the following description should not be interpreted as being limited thereto.

[0011] [First embodiment] A scanning electron microscope apparatus 100, which is an example of an observation system according to the first embodiment, will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing an example of the schematic configuration of the scanning electron microscope apparatus 100 according to the first embodiment. Note that in this figure, no distinction is made between physical components and logical components.

[0012] The scanning electron microscope device 100 is, for example, composed of a scanning electron microscope 101, an input / output unit 121, a control unit 122, a processing unit 123, a storage unit 124, and an image processing unit 125. The scanning electron microscope 101 is an imaging unit that captures an image of a sample 106 and acquires the image. The input / output unit 121, the control unit 122, the processing unit 123, the storage unit 124, and the image processing unit 125 are realized, for example, by a computer 200 (processor subsystem). The computer 200 may be a general-purpose computer and may be composed, for example, of a CPU 151 (processor), a ROM 152, a RAM 153, a hard disk drive 154, an input device 155, and a display 156 (display unit). The computer 200 may be connected to the scanning electron microscope 101 via a network. The CPU 151 is one aspect of a processor, and may be a GPU or other semiconductor device capable of arithmetic processing, or a combination thereof. The computer 200 may have multiple processors and memories, and the configuration of the computer 200 in Fig. 1 is understood to be an example. The input / output unit 121, the control unit 122, the processing unit 123, the storage unit 124, and the image processing unit 125 may be realized by a common computer 200 or by separate computers 200.

[0013] The scanning electron microscope 101 generates an electron beam 103 (charged particle beam) from an electron gun 102, and focuses the electron beam 103 on the surface of a sample 106 by passing it through a condenser lens 104 and an objective lens 105. Secondary electrons or backscattered electrons generated from the sample 106 are detected by a detector 108, and an image generated according to the detection signal is stored in a storage unit 124. The storage unit 124 may be composed of a RAM 153 and a hard disk drive 154 (memory) of the computer 200. In other words, it can be said that the storage unit 124 is a memory. The scanning electron microscope 101 and the storage unit 124 constitute an image acquisition unit that acquires an image of the sample.

[0014] Regarding detector 108, multiple detectors of the same type may be provided in one scanning electron microscope 100. Detector 108 may also be a combination of detectors of different types (for example, a combination of a detector that detects electrons and a detector that detects electromagnetic waves, a combination of detectors that detect only particles whose energy or spin direction is within a specific range, a combination of a secondary electron detector and a backscattered electron detector, etc.), or multiple detectors of the same type may be provided at different positions. When multiple detectors are provided, multiple images can usually be obtained in one imaging session.

[0015] The sample 106 can be placed on a stage 107. By moving the stage 107 in the horizontal direction, it is possible to obtain an image at any position on the sample 106. In addition, by changing the direction of the electron beam 103 two-dimensionally with a beam deflector 109, the sample 106 can be scanned with the electron beam 103.

[0016] The input / output unit 121 has a function of inputting instructions relating to the image capturing position and image capturing conditions from the outside using an input device 155 or the like, and visualizing the obtained image and outputting / displaying it, for example, on a display 156. Note that the scanning electron microscope apparatus 100 may omit the display 156 and may omit the input device 155. The input device 155 may be a network interface.

[0017] The control unit 122 is configured to perform various controls on the scanning electron microscope 101 (for example, control of the voltage applied to the electron gun 102, etc., control of the focal position of the condenser lens 104 and the objective lens 105, control of the position of the stage 107, control of the deflection degree of the beam deflector 109, etc.). The scanning electron microscope 101 of the following embodiment is capable of performing sparse sampling, and the control unit 122 realizes this sparse sampling by controlling the electron gun 102, the condenser lens 104, the objective lens 105, the beam deflector 109, etc. The control unit 122 can be realized by a computer program stored in the CPU 151, the ROM 152, etc. of the computer 200. The computer program can also be recorded on a portable recording medium. The portable storage medium is a non-volatile storage medium.

[0018] The control unit 122 is configured to control the input / output unit 121, the processing unit 123, the storage unit 124, and the image processing unit 125. The processing unit 123 performs various processes, such as a process of generating random numbers for setting random sampling coordinates, and a calculation process related to automatic focusing required to focus the electron beam 103 on the surface of the sample 106. The storage unit 124 stores (preserves) information about the region of interest, a sparse sampling coordinate group, a reconstructed image, a trained model of artifact reduction processing, data related to the degree of artifact occurrence in the reconstructed image, a training data set, and various processing parameters. The image processing unit 125 is responsible for image processing of the acquired data. Note that the computer and operator that train the correction engine may be the same as or different from the computer and operator that execute the artifact correction of this embodiment using the trained correction engine, etc.

[0019] The image processing unit 125 is realized by a computer 200 and image processing software stored in the computer 200. The image processing unit 125 includes, for example, a sampling coordinate setting unit 131, a reconstruction image generating unit 132, an artifact reduction processing unit 133, an artifact occurrence degree evaluating unit 134, a sampling coordinate adjusting unit 135, and a correction engine learning unit 136. Each unit 131 to 136 is realized by a CPU 151 (processor) of the computer 200 reading and executing image processing software stored in a memory (ROM 152, etc.).

[0020] The sampling coordinate setting unit 131 sets a sparse sampling coordinate group in the region of interest of the sample 106. In this specification, the expression "setting a sampling coordinate group" is used to mean not only generating a new sampling coordinate group but also updating a sampling coordinate group that has already been set. The sampling coordinate group includes data on the coordinates of a plurality of sampling points. The reconstructed image generating unit 132 generates a reconstructed image (first reconstructed image) based on a pixel value group acquired by irradiating an electron beam onto the sparse sampling coordinate group in the region of interest. As an example, the value of secondary electrons or the like corresponding to the pixels of an image signal obtained from the scanning electron microscope 100 (imaging device) can be digitized to obtain a pixel value, and a set of the pixel values ​​can be used as a pixel value group. Alternatively, the value of secondary electrons or the like can be converted to a certain value once, and then the value can be further converted to a pixel value to obtain a pixel value group.

[0021] The artifact reduction processing unit 133 executes artifact reduction processing on the first reconstructed image to generate a reconstructed image (second reconstructed image) with fewer artifacts compared to the first reconstructed image. The artifact reduction processing unit 133 also generates a correction engine that performs artifact reduction processing on the reconstructed image by training the correction engine using a plurality of sets of training data sets (teaching data) including a sparse sampling coordinate group, a reconstructed image with artifacts, and a fully sampled captured image (an image with few or no artifacts) as one set. Instead of (or in addition to) the fully sampled captured image, it is also possible to train the correction engine using a reconstructed image after artifact reduction processing. For example, when a fully sampled captured image cannot be obtained, training can be performed using a reconstructed image after artifact reduction processing instead of the fully sampled captured image. It is also possible to train the correction engine using an ideal image artificially created, such as a simulation image, instead of the fully sampled captured image.

[0022] The artifact occurrence degree evaluation unit 134 evaluates the degree of occurrence of artifacts in the reconstructed image based on a predetermined evaluation criterion. The sampling coordinate adjustment unit 135 adds a sampling coordinate group to the sampling coordinate group set by the sampling coordinate setting unit 131 based on the artifact occurrence degree. The correction engine learning unit 136 learns the correction engine using (i) the sampling coordinate group, (ii) a pixel value group corresponding to the sampling coordinate group, or a reconstructed image having artifacts reconstructed from the pixel value group using the sampling coordinate group, and (iii) an image that does not contain artifacts or has reduced artifacts. The correction engine learning unit 136 may acquire a sampling coordinate group and a reconstructed image based on the sample 106 imaged by the scanning electron microscope 101 and learn the correction engine based on these data, but instead, the correction engine may be learned using a sampling coordinate group and a reconstructed image obtained based on a learning sample different from the sample 106 obtained by another imaging device or simulation device.

[0023] An overview of artifact correction in the scanning electron microscope apparatus 100 of the first embodiment will be described with reference to Fig. 2. In artifact correction, first, a low-magnification image 201 of a sample is captured at low magnification, as shown in the upper left of Fig. 2. A region of interest 203 is set in a sample image 202 contained in this low-magnification image 201.

[0024] After the region of interest 203 is set, a sparse sampling coordinate set 205 is set in the region of interest 203. For example, the sparse sampling coordinate set 205 is set in only one frame. Reference numeral 204 in FIG. 2 indicates a distribution range of sampling points set in the sparse sampling coordinate set 205.

[0025] It is desirable to set the positions of the sampling points in the sparse sampling coordinate set 205 so as to have high randomness. On the other hand, if the randomness of the positions of the sampling points cannot be increased due to hardware constraints, artifacts tend to occur more easily in the reconstructed image.

[0026] Image 211 on the middle left side of Fig. 2 is a schematic diagram of a high-magnification image captured by fully sampling the region of interest 203. On the other hand, image 212 on the middle right side of Fig. 2 is a reconstructed image reconstructed based on a sparse pixel group obtained by sampling the region of interest 203 with a sparse sampling coordinate group 205. This image 212 is a reconstructed image containing artifacts. Reconstruction from a sparse pixel group causes degradation of signal information and artifacts, so the reconstructed image 212 generally has lower image quality than captured image 211 captured by fully sampling.

[0027] Moreover, the image 213 on the lower left side of FIG. 2 is an image in which the artifacts of the reconstructed image 212 have been corrected using a correction engine after learning the artifact reduction process using the aforementioned training data set, and represents a reconstructed image in which artifacts have been reduced (or a small amount of artifacts remain).

[0028] When the sampling rate in the region of interest 203 is sufficient for image reconstruction, the correction engine can sufficiently reduce artifacts in the reconstructed image. On the other hand, when the sampling rate is not sufficient for image reconstruction, it may be difficult to sufficiently reduce artifacts using only the correction engine. In such a case, in this embodiment, it is possible to set an additional sampling coordinate group, image the sample 106, and acquire a reconstructed image (third reconstructed image) again. The image 214 on the lower right side of FIG. 2 is an image in which image reconstruction is performed based on the sparse sampling coordinate group 205 of the region of interest 203 and the sparse pixel group obtained from the additional sampling coordinate group, and the artifacts of the reconstructed image are corrected using the correction engine. The image 214 is a reconstructed image with almost no artifacts obtained using the additional sampling coordinate group.

[0029] An example of an execution procedure of the artifact correction process in the first embodiment will be described with reference to the flowchart in Fig. 3. It is assumed that the imaging conditions and information input or instructed by the user to be used in the following process are set in advance as parameter information.

[0030] In step S301, an operator sets a region of interest 203 on the sample 106 via the input / output unit 121 to determine an imaging region of the sample 106. This corresponds to setting the region of interest 203 in the sample image 202 in the low-magnification image 201 (FIG. 2).

[0031] Next, in step S302, a sparse sampling coordinate group 205 is set in the region of interest 203. There are several possible methods for setting the sampling points of the sparse sampling coordinate group 205, as described below. (i) An image analysis is performed on the low magnification image 201, and a range in which a significant signal exists is extracted based on the analysis result, and sampling points are set in the extracted range. (ii) Sampling points are set based on information previously taught by the user. In this embodiment, a method of setting sampling points other than the above may be adopted. The sparse sampling coordinate group 205 may be set by dividing it into multiple frames instead of one frame. Also, the same pixel may be sampled multiple times. When the same pixel is not sampled multiple times, the sampling coordinate group may be represented as a binary image (for example, sampled pixels are white and non-sampled pixels are black). On the other hand, when the same pixel is sampled multiple times, the sampling coordinate group may be represented as a multi-value image.

[0032] In step S303, the scanning electron microscope 101 irradiates the electron beam from the electron gun 102 to positions on the sample 106 that correspond to the sampling points of the sampling coordinate group 205, and obtains a corresponding sparse pixel value group. Then, in step S304, a reconstructed image is generated from the obtained sparse pixel value group. This reconstructed image corresponds to the reconstructed image 212 with artifacts.

[0033] Here, as a method for generating the reconstructed image, for example, a compressed sensing method can be used. In one compressed sensing method, it is assumed that the sampling data is sparse in a representation space using a dictionary image, and an image close to the captured image at the time of all sampling can be reconstructed from sampling data that is smaller than the number of data of all sampling by a linear sum of the dictionary images. The dictionary image may be a dictionary image created based on a general discrete cosine transform, or may be a dictionary image specified by the user. Another method is to use a neural network that is trained to input a sparse pixel value group and output a reconstructed image.

[0034] When the reconstructed image is generated in this way, in the next step S305, artifacts in the reconstructed image are corrected using a correction engine that has learned artifact reduction processing. That is, as illustrated in FIG. 2, a pixel value group or a reconstructed image having artifacts generated based on the pixel value group is provided to the correction engine to an artifact reduction processing unit 133, whereby a reconstructed image 212 having artifacts is corrected to obtain a reconstructed image 213 in which artifacts are reduced (or remain to a small extent). The correction engine is a network that has learned (trained) to receive a sampling coordinate group and a reconstructed image having artifacts as inputs and output a reconstructed image in which artifacts are reduced or removed. For example, a convolutional neural network is used as the network of the correction engine.

[0035] Here, a specific example of a learning sequence for the correction engine used in step S305 will be described with reference to the flowchart of Fig. 4. Fig. 4 shows an example of a learning sequence for a correction engine that performs artifact reduction processing on a reconstructed image.

[0036] In step S401, a variable i=1 is set, and steps S402 to S406 are looped until the number of training data candidates i=N. A training data set is generated by this loop. If the number of samples to be imaged is N1, the number of regions of interest set for each sample is N2, and the number of types of sparse sampling coordinate groups of the regions of interest is N3, then the number of training data candidates N is N=N1×N2×N3.

[0037] In step S402, steps S301 to S304 described in FIG. 3 are executed, the sampling coordinate setting unit 131 sets the region of interest and the sampling coordinate group, and the reconstructed image generating unit 132 generates a reconstructed image. In the following step S403, the artifact occurrence degree of the generated reconstructed image is evaluated in the artifact occurrence degree evaluation unit 134. This corresponds to evaluating the artifact occurrence degree of the reconstructed image 213 in which a small amount of artifact remains. In the method of evaluating the artifact occurrence degree, for example, the feature amount of each artifact can be modeled based on the discontinuity of the image pattern. The feature amount of the artifact is calculated for each local region of the reconstructed image, and a region with a large value of the feature amount is evaluated as a region with a high artifact occurrence degree. In another method, a difference image is generated between an image obtained by super-resolving the region of interest in the low magnification image and the reconstructed image. A region with a large pixel value of the difference image is evaluated as a region with a high artifact occurrence degree.

[0038] Yet another method for evaluating the degree of artifact occurrence is to use an evaluation engine that has been trained to output a distribution of the degree of artifact occurrence. The evaluation engine is a network that has been trained to output a distribution of the degree of artifact occurrence using a sampling coordinate group and a reconstructed image containing artifacts as input. The network of the evaluation engine uses, for example, a convolutional neural network. A specific example of the learning sequence of the evaluation engine will be described later.

[0039] After the degree of artifact occurrence is evaluated, the presence or absence of artifacts in the reconstructed image is determined in the following step S404. If there are no artifacts (YES), the process proceeds to step S407, and if there are artifacts (NO), the process proceeds to step S405.

[0040] If there are no artifacts in the reconstructed image, the reconstructed image is excluded from the candidates for training data (steps S405 and S406 are omitted and the process proceeds to step S407). On the other hand, if there are artifacts in the reconstructed image, in step S405, the region of interest is fully sampled to obtain an image without artifacts. Then, in step S406, the sparse sampling coordinate group, the reconstructed image with artifacts, and the image without artifacts are added to the training data set as a set of data. Note that when pixel value groups are obtained from a plurality of detectors, a plurality of corresponding reconstructed images with artifacts and images without artifacts may be added to form a set of data. In step S407, if i = N, the loop of steps S402 to S406 is terminated, and if i < N, the process returns to step S401 and the above procedure is repeated. When N training data sets are obtained, the process proceeds to step S408.

[0041] In step S408, the N obtained training data sets are stored in the storage unit 124. In the next step S409, the variable j = 1 is set, and steps S410 to S413 are looped until the number of training times j = M. By this loop, the learning of the correction engine for performing the artifact reduction process is performed by the correction engine learning unit 136.

[0042] First, in step S410, based on the artifact correction parameters of the correction engine, a reconstructed image with corrected artifacts is output from the sparse sampling coordinate group and the reconstructed image with artifacts. When j = 1, the artifact correction parameters of the correction engine may be initialized with random numbers or with parameters learned from another training data set.

[0043] In the next step S411, the reconstructed image in which the artifacts have been corrected is compared with an image without the artifacts to calculate a correction error. The correction error may be calculated, for example, as a mean square error. Then, in step S412, it is determined whether the correction error is equal to or smaller than a predetermined threshold value. If it is equal to or smaller than the threshold value (YES), the process proceeds to step S415, and if it is larger than the threshold value (NO), the process proceeds to step S413.

[0044] If the correction error is equal to or smaller than the predetermined threshold (YES), the process proceeds to step S415, where the calculated artifact correction parameters are stored, and the procedure in FIG. 4 ends. On the other hand, if the correction error is greater than the predetermined threshold, in step S413, the artifact correction parameters are updated so that the correction error becomes smaller. In updating the parameters, for example, an error backpropagation method can be used. After that, in step S414, if j is the number of times of learning M, the loop in step S409 ends, and if j is less than M, the operations in and after step S409 are repeated.

[0045] Returning to FIG. 3, the description will be resumed from step S306. In step S306, the degree of artifact occurrence in the generated reconstructed image is evaluated. The details are the same as in step S403. In the following step S307, it is determined whether the artifacts have been sufficiently corrected in step S305. If YES, the process proceeds to step S310, and if NO, an additional sampling coordinate group is required, and the process proceeds to step S308. The determination of whether the artifacts have been sufficiently corrected is made, for example, by determining whether the degree of artifact occurrence is greater than a threshold value. The threshold value for determining whether the artifacts have been sufficiently corrected may be set based on a user input.

[0046] In step S308, a sparse sampling coordinate group is additionally set in the region of interest 203. For example, if the sampling rate of the region of interest 203 is set to 10% in step S302, a sampling coordinate group is additionally set so that the sampling rate of the region of interest 203 becomes 20% while maintaining the randomness of the sampling coordinate group. Other specific examples of step S308 will be described later.

[0047] In step S309, an electron beam is irradiated to positions on the sample 106 corresponding to the sampling points of the added sampling coordinate group in the region of interest 203 to obtain corresponding sparse pixel value groups, and the process returns to step S304, and steps S304 to S307 are executed again. At that time, if it is determined in step S307 that the artifacts have been sufficiently corrected, the process proceeds to step S310. In step S310, the reconstructed image is displayed on the GUI. That is, at least one reconstructed image 214 with almost no artifacts is displayed on the GUI.

[0048] As described above, in the first embodiment, when generating a learning data set for the correction engine, the sampling coordinate group is included in the teaching data, so that the correction engine can learn the relationship between the sampling coordinate group and artifacts in the reconstructed image. Therefore, it is possible to uniformly reduce artifacts that change due to the sampling coordinate group in the reconstructed image. In addition, by evaluating the occurrence degree of artifacts in the reconstructed image and adjusting the sampling rate to the minimum necessary to correct the artifacts, it is possible to correct the artifacts in the reconstructed image while suppressing damage to the sample.

[0049] FIG. 5 shows an example of a reconstructed image 510 of the region of interest 203, an image 520 obtained by mapping the degree of artifact occurrence of the reconstructed image 510, and a sparse sampling coordinate group 530 that is additionally set in the region of interest 203.

[0050] 2 in which some artifacts remain, but the region 511 with some artifacts is different, and the image is one in which many artifacts occur. Image 520 is an image in which the degree of artifact occurrence is mapped, and region 521 is a region in which the degree of artifact occurrence is high. Sampling coordinate group 530 is a sampling coordinate group that is additionally set in the region of interest 203, and reference numeral 531 indicates the distribution range of sampling points of the additionally set sparse sampling coordinate group 530. For example, the sparse sampling coordinate group 530 is set in only one frame.

[0051] The procedure for additionally setting a sampling coordinate group in an area where the occurrence rate of artifacts is high in step S308 will be described in detail with reference to the flowchart of Fig. 6. The description will also refer to Fig. 5 as necessary.

[0052] First, in step S601, an instruction is given to additionally set a sampling coordinate group. In accordance with this instruction, an additional sampling coordinate group is set according to the provisionally set sampling rate and sampling point distribution range 204. For example, a sampling coordinate group having a sampling rate different from the sampling rate of the sampling coordinate group set in step 302 but the same sampling point distribution range 204 may be provisionally set as the additional sampling coordinate group.

[0053] In step S602, based on the evaluation of the artifact occurrence degree (step S306), a region where the artifact occurrence degree is high is extracted, and an image 520 in which the artifact occurrence degree is mapped as shown in FIG.

[0054] In the next step S603, a mask image is generated to separate regions with a high degree of artifact occurrence from other regions. One method for generating the mask image is to binarize the pixel values ​​of regions with a higher degree of artifact occurrence than a predetermined threshold by setting the pixel values ​​to 1 and other pixel values ​​to 0. Since pixel values ​​near regions with a high degree of artifact occurrence are also effective for artifact correction, it is also possible to expand the regions with pixel values ​​of 1 by performing expansion processing on the binary image.

[0055] Then, in step S604, the generated mask image is used to exclude sampling coordinate groups in regions with a low occurrence rate of artifacts. This makes it possible to additionally set a sparse sampling coordinate group 530 (see FIG. 5) in which sampling coordinates with pixel values ​​of 0 in the mask image are excluded from the provisional sparse sampling coordinate group (step S601). In this case, a distribution range 531 of sampling points in the sparse sampling coordinate group 530 corresponds to a region with pixel values ​​of 1 in the mask image. By additionally setting sampling coordinate groups only in regions with a high occurrence rate of artifacts through the above steps, additional imaging is performed locally to the minimum necessary, and damage to the sample can be suppressed.

[0056] It should be noted that the procedure of additionally setting the sampling coordinate group in steps S307 to S309 above may be omitted under certain conditions. By appropriately setting the sampling coordinate group setting in step S302 and the conditions for generating the correction engine, the artifact reduction process may be performed sufficiently to the extent that subsequent correction of artifacts is not required. In such a case, steps S307 to S309 may be omitted. It is also possible to configure the device so that steps S307 to S309 are not executed regardless of the conditions.

[0057] Fig. 7 is a first modification of the first embodiment, showing a modification of the learning sequence of the correction engine used in step S305. Some steps are similar to the first example of Fig. 4, so detailed explanations will be omitted and only different parts will be explained in detail. The procedure of Fig. 7 updates artifact correction parameters of the correction engine based on the degree of artifact occurrence.

[0058] First, in step S701, a learning data set is generated and saved in the same manner as in steps S401 to S408. Then, in step S702, a variable j is set to 1, and steps S703 to S707 are looped until the number of learning times j reaches M. This loop updates parameters for artifact correction in the correction engine.

[0059] In step S703, similarly to step S410, a reconstructed image in which artifacts have been corrected is output from the sparse sampling coordinate group and the reconstructed image having artifacts, based on the artifact correction parameters of the correction engine.

[0060] In the next step S704, the degree of artifact occurrence in the reconstructed image generated in step S703 during learning of the artifact correction parameters is evaluated. The method of evaluating the degree of artifact occurrence may be the same as that in step S403.

[0061] Then, in step S705, based on the evaluation result of the artifact occurrence degree, the error between the artifact-corrected reconstructed image and the artifact-free image is calculated as a correction error. For example, the correction error may be the mean square error between the artifact-corrected reconstructed image and the artifact-free image, plus the mean square error of the artifact occurrence degree between them. Note that the artifact occurrence degree of the artifact-free image may be assumed to be zero.

[0062] As described above, in this first modification example, in the learning of the correction engine, the parameters of the correction engine are updated based on the evaluation result of the degree of occurrence of artifacts. As a result, the correction engine can accurately learn the artifact reduction process of the reconstructed image.

[0063] FIG. 8 is a flowchart for explaining a specific example of a learning sequence of an evaluation engine that outputs the distribution of the degree of occurrence of artifacts in step S306 according to a second modification example of the first embodiment.

[0064] First, in steps S801 to S804, the same operations as steps S401 to S404 in FIG. 4 are executed. In step S804, the presence or absence of artifacts in the reconstructed image is determined. If there are no artifacts (YES), the process proceeds to step S807; if there are (NO), the process proceeds to step S805.

[0065] When there are no artifacts in the reconstructed image, the reconstructed image is excluded from the candidates for learning data (steps S805 and S806 are omitted and the process proceeds to step S807). On the other hand, when there are artifacts in the reconstructed image, in step S805, the distribution of the degree of occurrence of artifacts in the reconstructed image is output as a GUI on the display 156 to prompt the user to make changes. If the user finds an error in the distribution of the degree of occurrence of artifacts, the user teaches the correct distribution. The distribution of the degree of occurrence of artifacts may be represented by binary values or multi - values.

[0066] In the subsequent step S806, a learning data set consisting of a sparse sampling coordinate group, a reconstructed image with artifacts, and the distribution of the degree of occurrence of artifacts is added. In step S807, if i = N, the loop of steps S802 to S806 ends; if i < N, the process returns to step S801 and the above procedure is repeated. When N learning candidate data sets are processed, the process proceeds to step S808.

[0067] When the learning data set is saved in step S808, in step S809, the variable j=1 is set, and steps S810 to S813 are looped until the number of learning times j=M. Through this loop, learning is performed for the evaluation engine that outputs the distribution of the artifact occurrence rate.

[0068] In step S810, a distribution of the artifact occurrence degree is output from the sparse sampling coordinate group and the reconstructed image having the artifact based on the artifact occurrence degree evaluation parameter of the evaluation engine. When j=1, the artifact occurrence degree evaluation parameter of the evaluation engine may be initialized with a random number or may be initialized with a parameter learned from another learning data set.

[0069] In step S811, the difference between the distribution of the artifact occurrence degree output in step S810 and the distribution of the artifact occurrence degree of the learning data set is calculated as a distribution error. The distribution error can be calculated as, for example, a mean square error.

[0070] In step S812, it is determined whether the distribution error is equal to or smaller than a predetermined threshold value. If it is equal to or smaller than the threshold value (YES), the process proceeds to step S815, and if it is greater than the threshold value (NO), the process proceeds to step S813. In step S813, an artifact occurrence degree evaluation parameter is updated so as to reduce the distribution error. As a method for updating the parameter, for example, an error backpropagation method can be used. In step S814, if j is the number of times of learning M, the loop of steps S810 to S813 is terminated. In step S815, the artifact occurrence degree evaluation parameter of the evaluation engine is stored in the storage unit 124.

[0071] As described above, in the second modified example, an evaluation engine that has been trained to output the distribution of the occurrence rate of artifacts is used. This makes it possible to accurately evaluate the distribution of the occurrence rate of artifacts, which is difficult to model using rule-based processing.

[0072] FIG. 9 shows an example of a GUI screen that displays an image in which the degree of artifact occurrence is mapped together with a reconstructed image on the screen of the display 156 in the device according to the first embodiment.

[0073] The GUI screen 900 in FIG. 9 includes, as an example, a parameter setting window 901. (a) Reconstructed images 510, 214, (b) Images 520, 909 mapping the degree of artifact occurrence (c) Sparsely sampled coordinate sets 905, 907, 910 Here, the sparse sampling coordinate groups 905, 907, and 910 simply display sampling coordinates 906, 908, and 911. Note that the sparse sampling coordinate groups 905, 907, and 910 respectively indicate a sampling coordinate group for the initial setting, a sampling coordinate group for the additional setting, and a sampling coordinate group obtained by combining the initial setting and the additional setting.

[0074] The parameter setting window 901 is configured to enable setting of a sampling rate 902 of the initially set sampling coordinate group, an increase rate 903 from the initially set sampling coordinate group in an additionally set sampling coordinate group, and a threshold value 904 of the artifact occurrence degree.

[0075] The display example in Fig. 9 is merely an example, and these may be displayed at different times or simultaneously. For example, in the procedure in Fig. 3, the initially set sampling coordinate group 905 is first displayed on the display 156 in step S302, and then the reconstructed image 510 in which artifacts have been reduced (some artifacts remain) is displayed in step S305. Then, in step S306, an image 520 in which the artifact occurrence degree is mapped is displayed, and in step S308, the additionally set sampling coordinate group 907 can be displayed. Here, the sampling coordinates 908 included in the sampling coordinate group 907 are located within the range 531 set based on the region 521 in which the artifact occurrence degree is high. Furthermore, the sampling coordinate group 910 including the initially set and the additionally set sampling coordinates is displayed.

[0076] In the second step S305 after the sampling coordinate group is additionally set, a reconstructed image 214 with almost no artifacts may be displayed on the display 156. In addition, in the second step S306, an image 909 in which the artifact occurrence degree is mapped may be displayed. Here, since the artifact occurrence degree of the reconstructed image 214 is smaller than the threshold value 904, there are no areas with a high artifact occurrence degree (image 909). In this way, by displaying the reconstructed image together with the artifact occurrence degree distribution, it becomes possible to smoothly set the minimum necessary sampling coordinate group while checking the allowable artifact occurrence degree.

[0077] [Second embodiment] A scanning electron microscope apparatus 100, which is an observation apparatus according to the second embodiment, will be described with reference to Fig. 10 and Fig. 11. The overall configuration of the scanning electron microscope apparatus 100 according to the second embodiment is similar to that of the first embodiment, so duplicated explanations will be omitted below. This second embodiment differs from the first embodiment in that it uses a correction engine that has learned the process of generating a reconstructed image (second reconstructed image) that is almost free of artifacts from a sparse pixel value group without using a reconstructed image (first reconstructed image) that has artifacts.

[0078] An example of artifact correction processing in the device according to the second embodiment will be described with reference to the flowchart in Fig. 10. First, in steps S1001 to S1004, the same processing as in steps S301 to S304 is performed to set a region of interest and a sampling coordinate group, and a reconstructed image is generated.

[0079] Steps S1005 to S1009 are similar to steps S306 to S310, and detailed description thereof will be omitted. However, in this second embodiment, in step S1004, a reconstructed image is generated from a sparse pixel value group obtained from a sampling coordinate group in the region of interest using a correction engine that has learned a process of generating a reconstructed image substantially free of artifacts from a sparse pixel value group. This corresponds to generating a reconstructed image 214 substantially free of artifacts from a sparse pixel value group obtained from sparse sampling coordinates 205 set in the region of interest 203. The correction engine is a network that has learned to receive a sampling coordinate group and a sparse pixel value group as input and output a reconstructed image free of artifacts. The correction engine network uses, for example, a convolutional neural network.

[0080] An example of a procedure for learning a correction engine in the second embodiment will be described with reference to the flowchart of Fig. 11. According to the procedure of Fig. 11, a correction engine that obtains a reconstructed image that is almost free of artifacts is learned.

[0081] In step S1101, variable i=1 is set, and steps S1102 to S1104 are repeatedly executed (looped) until the number of pieces of training data i=K. A training data set is generated by this loop. If the number of samples to be imaged is N1, the number of regions of interest set for each sample is N2, and the number of types of sparsely sampled coordinate groups of the regions of interest is N3, then the number of pieces of training data K is K=N1×N2×N3.

[0082] In step S1102, a sparse sampling coordinate group and a sparse pixel value group are obtained in the same procedure as steps S301 to S303. In step S1103, the region of interest 203 is fully sampled to obtain an artifact-free image.

[0083] In step S1104, the sparse sampling coordinate group, the sparse pixel value group, and the artifact-free image are added to the learning dataset as a set of data. In step S1105, if i = K, the loop of steps S1102 to S1104 is terminated. If i < K, the process returns to step S1101 and the above procedure is repeated. In step S1106, the learning dataset is stored in the storage unit 124.

[0084] In step S1107, the variable j = 1 is set, and steps S1108 to S1112 are repeatedly executed (looped) until the number of learning times i = M. Through this loop, the correction engine is trained to generate an artifact-free reconstructed image. In step S1108, based on the reconstruction parameters of the correction engine, a reconstructed image with almost no artifacts is output from the sparse sampling coordinate group and the sparse pixel value group. When j = 1, the reconstruction parameters of the correction engine may be initialized with random numbers or with parameters learned from another learning dataset.

[0085] In step S1109, during the learning of the reconstruction parameters, the degree of artifact generation in the reconstructed image is evaluated. The method for evaluating the degree of artifact generation may be the same as that in step S403.

[0086] In step S1110, based on the degree of artifact generation determined in S1109, the difference between the reconstructed image and the artifact-free image is calculated as the reconstruction error. As the reconstruction error, for example, the mean squared error between the reconstructed image and the artifact-free image plus the mean squared error of their degrees of artifact generation can be used. Note that the degree of artifact generation in the artifact-free image may be assumed to be zero.

[0087] In step S1111, it is determined whether the reconstruction error is equal to or less than a predetermined threshold. If it is equal to or less than the threshold (YES), the process proceeds to step S1114, and if it is greater than the threshold (NO), the process proceeds to step S1112. If the reconstruction error is equal to or less than the predetermined threshold, the reconstruction parameters calculated in step S1114 are saved, and the procedure in FIG. 11 ends.

[0088] In step S1112, the artifact correction parameters are updated so that the reconstruction error is reduced. As a parameter updating method, for example, the backpropagation method can be used. In step S1113, if j=M, the loop of steps S1108 to S1112 is ended, and the process proceeds to step S1114.

[0089] In the second embodiment, a reconstructed image that is almost free of artifacts is generated from a sparse group of pixel values ​​without going through a reconstructed image that contains artifacts, so there are fewer processing steps than in the first embodiment, making it possible to improve usability.

[0090] [Third embodiment] A scanning electron microscope apparatus 100, which is an observation system according to the third embodiment, will be described with reference to Fig. 12. The overall configuration of the scanning electron microscope apparatus 100 of the third embodiment is similar to that of the first embodiment, so duplicated explanations will be omitted below. This third embodiment differs from the previous embodiments in that it is configured to effectively set sampling coordinate groups and evaluate the degree of artifact occurrence by acquiring layout design data (hereinafter, design data) of a pattern on a sample 106 to be observed in advance.

[0091] As an example, when the sample 106 is a semiconductor, the design data may be a file in which edge information of the design shape of a semiconductor circuit pattern is written as coordinate data. Specifically, the design data is written in a format such as GDSII or OASIS. By utilizing the design data, it is possible to obtain layout information of the pattern without actually imaging the sample 106.

[0092] An example of an execution procedure of the artifact correction process in the third embodiment will be described with reference to the flowchart in Fig. 12. First, in step S1201, design data of the sample 106 is read. In the case of a sample for which design data is not available, a super-resolution image of a low-magnification image of the sample may be used instead of the design data.

[0093] Next, in step S1202, similarly to step S301 (FIG. 3), a region of interest 203 is set on the sample 106 to determine an imaging region of the sample 106. Furthermore, subsequently, in step S1203, a sparse sampling coordinate group is set in the region of interest 203 based on design data.

[0094] For example, in the third embodiment, a sample is imaged in advance using various sampling coordinate groups to generate a reconstructed image, and the analysis results of the reconstructed image are stored as a database in the storage unit 124. When design data is read in step S1201, an optimal sampling coordinate group is selected and set according to the analysis results of a sample that is the same as or similar to the design data.

[0095] In setting a sampling coordinate group based on the design data (step S1203), for example, instead of setting uniform sampling coordinates for the entire region of interest, a sampling coordinate group may be set by locally changing the sampling density. Even if data matching the set region of interest is not searched for in the database, data similar to the region of interest may be searched for in the database, and a sampling coordinate group corresponding to the similar data may be set for the region of interest.

[0096] A sampling coordinate group may be set for the sample related to the design data by using an estimation engine trained based on the database. The estimation engine is, for example, a network trained to input design data and output a sparse sampling coordinate group. For example, the loaded design data, the design data referenced from the database, and candidates for the sampling coordinate group may be displayed on the GUI of the display 156 to prompt the user to select the sampling coordinate group.

[0097] After the sampling coordinate group is set in step S1203, the sampling coordinate group 205 set in the region of interest 203 is imaged in step S1204 in the same manner as in step S303 in Fig. 3, and a corresponding sparse pixel value group is obtained. Then, in step S1205, a reconstructed image is generated from the obtained sparse pixel value group in the same manner as in step S304. In the following step S1206, artifacts in the reconstructed image are corrected using a correction engine that has learned artifact reduction processing. The learning sequence of the correction engine will be described later.

[0098] In step S1207, the degree of artifact occurrence is evaluated based on the design data. In the following step S1208, it is determined whether the artifact correction in step S1206 has been performed sufficiently. If YES, the process proceeds to step S1211, and if NO, the process proceeds to step S1209.

[0099] In step S1209, a sparse sampling coordinate group is additionally set in the region of interest. The specific method is the same as step S308 in FIG. 3. In step S1210, an electron beam is irradiated to positions on the sample 106 corresponding to the sampling points of the added sampling coordinate group in the region of interest to obtain a corresponding sparse pixel value group, and the process returns to step S1205, and steps S1205 to S1208 are executed again. At that time, if it is determined in step S1208 that the artifacts have been sufficiently corrected, the process proceeds to step S1211. In step S1211, a reconstructed image is displayed on the GUI. That is, at least one reconstructed image with almost no artifacts is displayed on the GUI.

[0100] In the third embodiment, by setting a sampling coordinate group based on design data, an optimal sampling coordinate group for a sample can be set, so that it is possible to improve the robustness of the image reconstruction process and the artifact correction process for the sample. Also, by evaluating the degree of artifact occurrence based on design data, the design data can be used as the evaluation standard, so that it is possible to accurately evaluate the degree of artifact occurrence.

[0101] A specific example of the learning sequence of the correction engine used in step S1206 will be described with reference to the flowchart of FIG. Step S1301 corresponds to step S401, Step S1302 corresponds to step S402 (steps S1201 to S1205), Steps S1304 and S1305 correspond to steps S404 and S405, Steps S1307 to S1310 correspond to steps S407 to S410, Step S1312 corresponds to step S705; Steps S1313 to S1316 correspond to steps S412 to S415. Therefore, in the following, overlapping explanations will be omitted, and only the points that differ from the above-mentioned embodiment will be explained.

[0102] In step S1303, the degree of artifact occurrence is evaluated based on the design data. As a method for evaluating the degree of artifact occurrence based on the design data, for example, a method may be adopted in which edge images of the reconstructed image and the design data are generated using a Sobel filter or the like, and a region where the difference between these edge images is large is evaluated as a region where the degree of artifact occurrence is high.

[0103] Another method may be to generate a difference image between an image obtained by converting design data into an image of a quality similar to that of a captured image using an image conversion engine and a reconstructed image, and to evaluate an area with a large pixel value of the difference image as an area with a high degree of artifact occurrence. The image conversion engine is, for example, a network that has been trained to receive design data of the area of ​​interest as an input and output a captured image of the area of ​​interest.

[0104] In step S1306, the design data, the sparsely sampled coordinate group, the reconstructed image with the artifacts, and the image without the artifacts are added as a set of data to a training data set.

[0105] In step S1311, the degree of artifact occurrence of a reconstructed image in which artifacts have been corrected during learning of the artifact correction parameters is evaluated based on the design data. The method of evaluating the degree of artifact occurrence is the same as in step S1303.

[0106] An example of a specific procedure for aligning the sampling coordinate group based on the design data in step S1203 of the third embodiment will be described with reference to the flowchart in Fig. 14. In step S1401, a sparse sampling coordinate group for alignment is set in the region of interest. In order to avoid damaging the sample, the sampling coordinate group for alignment may have a lower sampling rate than the sampling coordinate group for image reconstruction. Also, the sampling coordinate group for alignment may be set locally rather than over the entire region of interest.

[0107] In the next step S1402, an electron beam is irradiated onto positions on the sample 106 that correspond to the sampling points of the sampling coordinate group for alignment, and a sparse pixel value group is acquired. Then, in step S1403, a reconstructed image is generated from the sparse pixel value group. The method of generating the reconstructed image may be the same as that in step S304.

[0108] In step S1404, the amount of misalignment of the sparse sampling coordinate group is calculated based on the design data and the reconstructed image for alignment obtained in step S1403. For example, a method of calculating the amount of misalignment is to generate edge images of the reconstructed image and the design data using a Sobel filter or the like, and calculate the amount of misalignment from the peak position in a normalized cross-correlation map of those edge images. Then, in steps S1405 and S1406, the sampling coordinate group is corrected based on the calculated amount of misalignment, and the corrected sampling coordinate group is set. In this way, by aligning the sampling coordinate group for generating the reconstructed image in the region of interest using the sampling coordinate group for alignment, a pixel value group can be obtained from the coordinates optimized based on the design data, and an image can be reconstructed with high accuracy.

[0109] The present invention is not limited to the above-mentioned embodiment, and various modifications are included. For example, the above-mentioned embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those having all the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. In addition, each of the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example, by designing them as an integrated circuit. [Explanation of symbols]

[0110] 100...scanning electron microscope apparatus, 101...scanning electron microscope, 102...electron gun, 103...electron beam (charged particle beam), 104...condenser lens, 105...objective lens, 106...sample, 107...stage, 108...detector, 121...input / output section, 122...control section, 123...processing section, 124...storage section, 125...image processing section, 131...sampling coordinate setting section, 132...reconstructed image generating section, 133...artifact reduction processing section, 134...artifact occurrence degree evaluation section, 135...sampling coordinate adjustment section, 136...correction engine learning section, 151...CPU, 152...ROM, 153...RAM, 154...hard disk drive, 155...input device, 156...display, 200...computer, 201...low magnification image, 202... sample image; 203... region of interest; 204... distribution range of sampling points; 205... sampling coordinate group; 211... high magnification image captured by sampling the entire region of interest 203; 212... reconstructed image reconstructed based on a sparse pixel group obtained by sampling the region of interest 203 with a sparse sampling coordinate group 205; 213... image in which artifacts of the reconstructed image 212 are corrected using a correction engine (reconstructed image with reduced artifacts); 214... reconstructed image with almost no artifacts obtained using an additional sampling coordinate group; 510... reconstructed image; 520... image in which the degree of artifact occurrence is mapped; 530... sampling coordinate group; 531... distribution range of sampling points; 900... GUI screen; 901... parameter setting window; 905, 907, 910... sparse sampling coordinate group; 906, 908, 911... sampling coordinates; 909…Image showing the degree of artifact occurrence

Claims

1. An imaging device; Processor Subsystem and Equipped with The processor subsystem includes: A sparse sampling coordinate set including coordinate data of a plurality of sampling points is set for the sample; obtaining pixel values ​​corresponding to the sparsely sampled coordinates on the specimen; providing a correction engine with either the set of pixel values ​​or a first reconstructed image generated based on the set of pixel values ​​to generate a second reconstructed image having reduced artifacts; An observational system, wherein the correction engine is trained using a plurality of training data sets relating to the sample or training samples, the training data sets including: (1) to (3) below: (1) A sparse set of training sampling coordinates including data on the coordinates of a plurality of sampling points set on the sample or training sample. (2) a training pixel value group corresponding to the sparse training sampling coordinate group, or a training reconstructed image having artifacts generated based on the training pixel value group; (3) A learning image that does not include or has reduced artifacts, the learning image being generated based on pixel values ​​corresponding to a coordinate group including at least the learning sampling coordinate group.

2. 2. The observation system according to claim 1, The processor subsystem includes: Evaluating the degree of artifact occurrence in the second reconstructed image; determining whether or not the group of sampling coordinates needs to be added based on the occurrence rate of the artifacts; generating an additional sampling coordinate group according to the result of the determination of whether or not the additional sampling coordinate group is required; an observation system configured to obtain pixel values ​​according to the additional sampling coordinates to generate a third reconstructed image.

3. 3. The observation system according to claim 2, The additional sampling coordinates are added to areas with a high incidence of artifacts.

4. 2. The observation system according to claim 1, The processor subsystem includes: updating a parameter of the correction engine based on a degree of occurrence of artifacts in a reconstructed image generated by the correction engine; An observation system comprising:

5. 3. The observation system according to claim 2, the processor subsystem uses an evaluation engine that is trained to output a distribution of the degree of occurrence of artifacts using the sampling coordinate group and the first reconstructed image as inputs in evaluating the degree of occurrence of artifacts.

6. 3. The observation system according to claim 2, The observation system further comprises a display unit that displays an image indicating the degree of occurrence of the artifact.

7. An imaging device; Processor Subsystem and Equipped with The processor subsystem includes: A sparse sampling coordinate set including coordinate data of a plurality of sampling points is set for the sample; obtaining pixel values ​​corresponding to the sparsely sampled coordinates on the specimen; providing a correction engine with either the set of pixel values ​​or a first reconstructed image generated based on the set of pixel values ​​to generate a second reconstructed image having reduced artifacts; The correction engine is trained using a plurality of training data sets relating to the sample or training samples, the training data sets including the following (1) to (3): (1) A sparse set of training sampling coordinates including data on the coordinates of a plurality of sampling points set on the sample or training sample. (2) a training pixel value group corresponding to the sparse training sampling coordinate group, or a training reconstructed image having artifacts generated based on the training pixel value group; (3) A learning image that does not include or has reduced artifacts, the learning image being generated based on pixel values ​​corresponding to a coordinate group including at least the learning sampling coordinate group. The processor subsystem includes: An observation system configured to read design data of the sample and set the sampling coordinate group based on the design data.

8. 8. The observation system according to claim 7, The processor subsystem includes: Evaluating the degree of occurrence of artifacts in the second reconstructed image based on the design data; determining whether or not the group of sampling coordinates needs to be added based on the occurrence rate of the artifacts; generating an additional sampling coordinate group according to the result of the determination of whether or not the additional sampling coordinate group is required; A viewing system for obtaining pixel values ​​according to the additional sampling coordinates to generate a third reconstructed image.

9. 8. The observation system according to claim 7, setting a group of sampling coordinates for alignment with respect to the sample; Obtaining pixel values ​​according to the set of sampling coordinates for alignment to generate a reconstructed image for alignment; calculating a positional deviation amount of the sparse sampling coordinate group with respect to the sample based on the design data and the reconstructed image for alignment; and an observation system that corrects the sparsely sampled coordinate group based on the amount of positional deviation.

10. 1. A method for artifact correction in a processor subsystem, the method comprising: A sparse sampling coordinate set including coordinate data of a plurality of sampling points is set for the sample; obtaining pixel values ​​corresponding to the sparsely sampled coordinates on the specimen; generating a second reconstructed image having reduced artifacts by providing a correction engine with the set of pixel values ​​or a first reconstructed image generated based on the set of pixel values, 13. A method of artifact correction, wherein the correction engine is trained using a plurality of training data sets relating to the sample or training samples, the training data comprising: A sparse set of sampling coordinates for learning, including data on the coordinates of a plurality of sampling points set for the sample or learning sample. a training pixel value set corresponding to the sparse training sampled coordinate set, or a training reconstructed image having artifacts generated based on the training pixel value set; A training image that does not include or has reduced artifacts, the training image being generated based on pixel values ​​corresponding to a set of coordinates including at least the training sampling coordinate set.

11. 11. The method of claim 10, Evaluating the degree of artifact occurrence in the second reconstructed image; determining whether or not the group of sampling coordinates needs to be added based on the occurrence rate of the artifacts; generating an additional sampling coordinate group according to the result of the determination of whether or not the additional sampling coordinate group is required; obtaining pixel values ​​according to the additional sampling coordinates to generate a third reconstructed image; Artifact correction methods.

12. 12. The method of claim 11, The method of claim 1, wherein the additional sampling coordinates are added to areas with a high incidence of artifacts.

13. 11. The method of claim 10, An artifact correction method, comprising: updating parameters of the correction engine based on a degree of occurrence of artifacts in a reconstructed image generated by the correction engine.

14. 12. The method of claim 11, an artifact correction method, in which the evaluation of the degree of occurrence of artifacts is performed using an evaluation engine that has been trained to output a distribution of the degree of occurrence of artifacts using the sampling coordinate group and the first reconstructed image as inputs.

15. 12. The method of claim 11, and displaying an image showing the degree of occurrence of the artifact.

16. An artifact correction program that causes a processor subsystem to execute the artifact correction method according to any one of claims 11 to 15.

Images