Computer program, information processing method, and information processing device

Abstract

The present invention provides a computer program, an information processing method, and an information processing device that can be expected to detect various patterns from a target image without preliminarily performing machine learning of a pattern to be detected.　A computer program according to the present embodiment causes a computer to execute processing for detecting a predetermined pattern formed on a substrate processed by a substrate processing apparatus from a target image capturing the substrate. The computer program causes the computer to execute processing involving: acquiring a template image showing a pattern to be detected; extracting a feature amount from each of the template image and the target image; and detecting the pattern to be detected from the target image on the basis of a result of comparison between the feature amount of the template image and the feature amount of the target image.

Description

Computer program, information processing method, and information processing device

[0001] The present disclosure relates to a computer program, an information processing method, and an information processing device.

[0002] Patent Literature 1 proposes a pattern inspection system that inspects an image of a pattern to be inspected using a classifier configured by machine learning based on an image of the pattern to be inspected of an electronic device and data used to manufacture the pattern to be inspected. This pattern inspection system includes a storage unit that stores a plurality of pattern images of the electronic device and pattern data used to manufacture the pattern of the electronic device, and an image selection unit that selects a learning pattern image to be used for machine learning from the plurality of pattern images based on the pattern data and pattern image stored in the storage unit.

[0003] Japanese Patent Application Laid-Open No. 2020-35282

[0004] The present disclosure provides a computer program, an information processing method, and an information processing device that are expected to be able to detect various patterns from a target image without prior machine learning of the detection target pattern.

[0005] A computer program according to one embodiment causes a computer to execute a process for detecting a predetermined pattern formed on a substrate from a target image of the substrate that has been processed by a substrate processing apparatus, the computer acquiring a template image onto which the pattern to be detected is captured, extracting features from the template image and the target image, and detecting the pattern to be detected from the target image based on a comparison result between the features of the template image and the features of the target image.

[0006] According to the present disclosure, it is expected that various patterns can be detected from a target image without prior machine learning of the detection target pattern.

[0007] FIG. 1 is a schematic diagram for explaining an overview of an information processing system according to the present embodiment. FIG. 2 is a block diagram showing an example of a configuration of an information processing device according to the present embodiment. FIG. 3 is a schematic diagram for explaining an example of a method for acquiring a template image. FIG. 4 is a schematic diagram for explaining an example of a pattern detection process. FIG. 5 is a schematic diagram for explaining an example of a pattern detection process. FIG. 6 is a flowchart showing an example of a procedure for a pattern detection process performed by an information processing device according to embodiment 1. FIG. 7 is a flowchart showing an example of a procedure for a pattern detection process performed by an information processing device according to embodiment 2. FIG. 8 is a flowchart showing an example of a procedure for a pattern detection process performed by an information processing device according to embodiment 3. FIG. 9 is a flowchart showing an example of a procedure for a pattern detection process performed by an information processing device according to embodiment 4.

[0008] Specific examples of information processing systems according to embodiments of the present disclosure will be described below with reference to the drawings. Note that the present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope of the claims.

[0009] First Embodiment <System Overview> FIG. 1 is a schematic diagram illustrating an overview of an information processing system according to this embodiment. The information processing system according to this embodiment includes an information processing apparatus 1, a substrate processing apparatus 101, a scanning electron microscope 102, and the like. The substrate processing apparatus 10 is an apparatus such as a process chamber that performs processing such as etching on semiconductor wafers (substrates). After processing such as etching in the substrate processing apparatus 101, the surface shape of the processed substrate is photographed by a scanning electron microscope 102. The scanning electron microscope 102 is an apparatus that observes the object by irradiating the object with an electron beam and detecting secondary electrons or transmitted electrons emitted from the object, and outputs a so-called SEM (Scanning Electron Microscope) image of the object. The SEM image output by the scanning electron microscope 102 is provided to the information processing apparatus 1. Note that in this embodiment, processing is performed on an SEM image photographed by the scanning electron microscope 102; however, the present invention is not limited to this, and processing may also be performed on an image photographed by, for example, a transmission electron microscope or an optical microscope.

[0010] Structures of various shapes are formed on the surface of a substrate processed by the substrate processing apparatus 101. For example, as shown in the SEM image in the center of FIG. 1 , multiple structures of the same shape may be formed on the substrate. Hereinafter, in this embodiment, each of these structures of the same shape will be referred to as a pattern. However, even if the formation of patterns of the same shape is intended, the size or shape of each pattern will differ due to processes such as film formation, etching, and exposure. A pattern whose size or shape deviates beyond a predetermined threshold is determined to be abnormal. The information processing system according to this embodiment is a system that detects multiple patterns formed on a target substrate based on an SEM image of the target substrate captured by a scanning electron microscope 102 and determines the state of each detected pattern.

[0011] In this embodiment, the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 will be described as separate apparatuses, but this is not limiting. For example, the substrate processing apparatus 101 and the scanning electron microscope 102 may be a single apparatus, the scanning electron microscope 102 and the information processing apparatus 1 may be a single apparatus, or the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be a single apparatus. Furthermore, each of the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be configured by combining a plurality of apparatuses.

[0012] The information processing device 1 acquires an SEM image of a target substrate captured by a scanning electron microscope 102. In this embodiment, the information processing device 1 also acquires an image in which a pattern to be detected is captured as a template image. The information processing device 1 detects the pattern captured in the template image from the SEM image of the target substrate. The lower part of FIG. 1 shows a state in which the pattern detected from the SEM image is surrounded by a bounding box. The information processing device 1 outputs, for each detected pattern, information such as the position and size of the bounding box relative to the SEM image as the pattern detection result.

[0013] In this embodiment, the information processing device 1 extracts feature points from a template image and also from a target SEM image, and performs a matching process between the extracted feature points of both images to detect a pattern of the template image from the SEM image. By employing a feature point matching process for pattern detection, the information processing system according to this embodiment does not require pre-processing, such as having a learning model learn the shape of the pattern to be detected in advance through machine learning, and allows the user to arbitrarily specify the pattern to be detected. The user can specify the pattern to be detected by providing the information processing device 1 with a template image, such as an image with a pattern or an image with a pattern drawn on it. Details of the pattern detection process based on feature points will be described later.

[0014] The information processing device 1 also determines the state of each pattern detected from the SEM image. Even if multiple patterns formed on a substrate are intended to have the same shape, differences in size, shape, etc. may occur due to each substrate processing process. A pattern whose size, shape, etc. deviates beyond a predetermined threshold is considered abnormal, while a pattern whose size, shape, etc. is within the threshold is considered normal. Therefore, for example, the information processing device 1 measures the length, width, etc. of each pattern detected from the SEM image based on a bounding box. The information processing device 1 can determine whether each pattern is normal or abnormal based on whether the measured length, etc. of each pattern is within an appropriate range. Furthermore, for example, the information processing device 1 calculates the similarity between the pattern of the template image and each pattern detected from the SEM image. The information processing device 1 considers the pattern of the template image to be normal, and determines the detected pattern to be normal if the similarity exceeds a threshold (i.e., the pattern of the template image and the detected pattern are similar), and determines the detected pattern to be abnormal if the similarity does not exceed the threshold. The method for determining the state of a pattern detected from an SEM image is not limited to the above method, and various other methods may be adopted. Furthermore, the state of a pattern detected from an SEM image may be determined to be three or more types of states instead of just two types, normal and abnormal.

[0015] <Device Configuration> Fig. 2 is a block diagram showing an example configuration of an information processing device 1 according to this embodiment. The information processing device 1 according to this embodiment can be realized by installing a predetermined application program or the like in a general-purpose information processing device such as a personal computer or a server computer. However, the information processing device 1 may also be a dedicated information processing device that controls the substrate processing device 101 or the scanning electron microscope 102. The information processing device 1 according to this embodiment is configured to include a processing unit 11, a storage unit 12, a communication unit 13, a display unit 14, an operation unit 15, etc. Note that, although this embodiment will be described assuming that processing is performed by a single information processing device 1, the processing of the information processing device 1 may be distributed among multiple devices.

[0016] The processing unit 11 is configured using an arithmetic processing device such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit) or a quantum processor, a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processing unit 11 reads and executes a program 12a stored in the storage unit 12 to perform various processes such as a process of acquiring a template image and an SEM image, a process of detecting a pattern from the SEM image, and a process of determining the state of the detected pattern.

[0017] The storage unit 12 is configured using a large-capacity storage device such as a hard disk or an SSD (Solid State Drive). The storage unit 12 stores various programs executed by the processing unit 11 and various data required for the processing of the processing unit 11. In this embodiment, the storage unit 12 stores a program 12a executed by the processing unit 11. The storage unit 12 also includes a template image storage unit 12b that stores and accumulates a template image on which a pattern to be detected from an SEM image is copied.

[0018] In this embodiment, the program (computer program, program product) 12a is provided in a form recorded on a recording medium 99 such as a memory card or an optical disc, and the information processing device 1 reads the program 12a from the recording medium 99 and stores it in the storage unit 12. However, the program 12a may also be written to the storage unit 12, for example, during the manufacturing stage of the information processing device 1. Alternatively, the program 12a may be distributed by a remote server device or the like and acquired by the information processing device 1 via communication. For example, the program 12a may be read from the recording medium 99 by a writing device and written to the storage unit 12 of the information processing device 1. The program 12a may be provided in a form distributed via a network or in a form recorded on the recording medium 99.

[0019] The template image storage unit 12b stores and accumulates template images used for pattern detection. The information processing device 1 acquires a template image generated by, for example, a user by cutting out a portion of an SEM image and stores the acquired template image in the template image storage unit 12b. The template image storage unit 12b stores text information, such as title information of the template image set by the user or explanatory information regarding the pattern depicted in the template image, in association with information such as the date and time the template image was acquired, along with the template image. The information processing device 1 can acquire a template image that will be the pattern to be detected by, for example, displaying a list of multiple template images stored in the template image storage unit 12b as candidates and accepting the selection of one from the multiple candidates.

[0020] The communication unit 13 is connected to the scanning electron microscope 102 via a cable such as a communication line or a signal line, and transmits and receives data to and from the scanning electron microscope 102 via this cable. In this embodiment, the communication unit 13 receives data of an SEM image of a substrate transmitted from the scanning electron microscope 102, and provides the received data to the processing unit 11. Note that in this embodiment, the SEM image is transmitted from the scanning electron microscope 102 to the information processing device 1 via communication, but this is not limiting, and the SEM image may also be transmitted and received via a recording medium such as a memory card.

[0021] The display unit 14 is configured using a liquid crystal display or the like, and displays various images, characters, etc. based on the processing of the processing unit 11. In this embodiment, the display unit 14 displays, for example, an SEM image acquired from the scanning electron microscope 102, and the results of pattern detection from the SEM image.

[0022] The operation unit 15 accepts user operations and notifies the processing unit 11 of the accepted operations. For example, the operation unit 15 accepts user operations using input devices such as mechanical buttons or a touch panel provided on the surface of the display unit 14. Furthermore, for example, the operation unit 15 may be input devices such as a mouse and a keyboard, and these input devices may be configured to be detachable from the information processing device 1.

[0023] The storage unit 12 may be an external storage device connected to the information processing device 1. The information processing device 1 may be a multi-computer including multiple computers, or may be a virtual machine virtually constructed by software. The information processing device 1 is not limited to the above configuration, and may not include, for example, the display unit 14 and the operation unit 15.

[0024] In the information processing device 1 according to this embodiment, the processing unit 11 reads and executes a program 12a stored in the storage unit 12, thereby realizing an image acquisition unit 11a, a feature point extraction unit 11b, a matching processing unit 11c, a pattern detection unit 11d, a state determination unit 11e, a display processing unit 11f, and the like as software functional units in the processing unit 11. Note that in this drawing, functional units related to the processing of detecting a pattern from an SEM image are shown as functional units of the processing unit 11, and functional units related to other processing are not shown.

[0025] The image acquisition unit 11a performs processing to acquire an SEM image (target image) of a substrate processed by the substrate processing apparatus 101, taken by the scanning electron microscope 102, and a template image in which a pattern to be detected is captured. The image acquisition unit 11a acquires the SEM image of the substrate taken by the scanning electron microscope 102 by communicating with the scanning electron microscope 102 via the communication unit 13.

[0026] The image acquisition unit 11a also displays an SEM image acquired from the scanning electron microscope 102 on the display unit 14, and receives from the user a designation of a partial image area from the SEM image, thereby acquiring this partial image area as a template image. The image acquisition unit 11a receives input of text information, such as title information and explanatory information, related to the acquired template image from the user, and stores the template image together with the received information in the template image storage unit 12b. The image acquisition unit 11a also displays a list of multiple template images stored in the template image storage unit 12b as candidates on the display unit 14, receives selection of one of the displayed candidates, and acquires the selected candidate as the template image. The user can appropriately select which method to use to designate the template image, for example, on a menu screen or the like.

[0027] The feature point extraction unit 11b performs a process of extracting feature points from the SEM image and the template image acquired by the image acquisition unit 11a. The feature point extraction process is a process of identifying characteristic points (pixels) such as edges or corners contained in the image, and can be performed using an existing method such as SIFT (Scale Invariant Feature Transformation) or AKAZE (Accelerated-KAZE). Each feature point extracted from the image by the feature point extraction unit 11b includes information on the position of the feature point in the image (e.g., x, y coordinates) and a numerical value (feature amount) indicating the characteristic of the point.

[0028] The matching processing unit 11c matches the feature points of the SEM image extracted by the feature point extraction unit 11b with the feature points of the template image. For example, the matching processing unit 11c determines which of the multiple feature points extracted from the SEM image each feature point extracted from the template image corresponds to by comparing the positions and feature amounts of the feature points. The matching processing unit 11c outputs, as a result of the matching processing, information indicating which of the feature points of the SEM image each feature point of the template image corresponds to (matches).

[0029] Based on the results of the matching process performed by the matching processing unit 11c, the pattern detection unit 11d performs a process of detecting a pattern from the SEM image that is identical to the pattern captured in the template image. Note that a pattern that is "same" as the pattern captured in the template image does not mean that the pattern is completely identical. The "same" pattern also includes patterns that are identical in design but have different sizes or shapes due to various substrate processing processes. First, the pattern detection unit 11d identifies an image area in the SEM image that is the same size as the template image and contains the largest number of feature points that match the feature points in the template image. The pattern detection unit 11d then extracts from the SEM image an image area that includes the identified image area in the SEM image but is larger than this image area (for example, an image area that is 1.2 times larger in both length and width than the identified image area).

[0030] Next, the pattern detection unit 11d causes the feature point extraction unit 11b to extract feature points and the matching processing unit 11c to match feature points between the template image and the image region extracted from the SEM image, and obtains the feature point matching results. Based on the re-matching results, the pattern detection unit 11d creates a bounding box that encloses the matched feature points in the extracted image region. The area surrounded by this bounding box corresponds to the detected pattern, and the pattern detection unit 11d outputs information about the bounding box (e.g., information such as the position of the bounding box in the original SEM image) as the pattern detection result.

[0031] The pattern detection unit 11d can detect one pattern identical to the template image from the SEM image through the above processing. The pattern detection unit 11d repeatedly performs the same processing on the same SEM image to detect multiple patterns from the SEM image. During this processing, the pattern detection unit 11d repeats the processing, excluding an image region corresponding to a bounding box enclosing a detected pattern from the SEM image as a target for feature point extraction and feature point matching. Furthermore, during each repetition, the pattern detection unit 11d calculates the similarity between the image region of the bounding box enclosing the detected pattern and the template image, and terminates the repeated pattern detection when the similarity falls below a predetermined threshold. Note that the calculation of the similarity may be performed by any method. For example, if information such as the degree of matching is obtained during the feature point matching processing, this information may be used as the similarity.

[0032] The state determination unit 11e performs a process of determining the state of each pattern detected from the SEM image by the pattern detection unit 11d. For example, the state determination unit 11e measures the length and width of each pattern based on information about a bounding box surrounding each pattern, and determines whether each pattern is normal (good) or abnormal (defective) depending on whether the measured lengths are within a predetermined range. Furthermore, for example, the state determination unit 11e may determine whether a pattern is normal or abnormal by considering a pattern in a template image as a normal pattern or by receiving an image of a normal pattern input from a user and determining whether the similarity between the normal pattern and the detected pattern exceeds a predetermined threshold. The state determination of each pattern by the state determination unit 11e is not limited to the above, and any method or state determination may be used.

[0033] The display processing unit 11f performs processing to display information such as the detection results of patterns from the SEM image by the pattern detection unit 11d and / or the state determination results of each pattern by the state determination unit 11e on the display unit 14. For example, the display processing unit 11f generates an image in which bounding boxes surrounding each pattern detected by the pattern detection unit 11d are superimposed on the SEM image acquired from the scanning electron microscope 102, and displays the image on the display unit 14. Furthermore, for example, the display processing unit 11f notifies the user of an abnormal pattern by highlighting (for example, by changing the color or blinking) the bounding box corresponding to a pattern determined to be abnormal by the state determination unit 11e, out of the bounding boxes of each pattern superimposed on the SEM image.

[0034] <Pattern Detection Processing> FIG. 3 is a schematic diagram illustrating an example of a method for acquiring a template image. The information processing device 1 according to this embodiment displays an SEM image of a target substrate acquired from, for example, a scanning electron microscope 102 on the display unit 14 and accepts a user's designation of an area on the displayed SEM image. The information processing device 1 accepts a selection of a rectangular image area on the SEM image, for example, based on an operation of a mouse on the operation unit 15 (see the upper part of FIG. 3). The information processing device 1 acquires the rectangular image area selected by the user as a template image (see the lower left of FIG. 3). Furthermore, the information processing device 1 treats an image obtained by removing the image area corresponding to the template image from the original SEM image as the SEM image for pattern detection in subsequent processing (see the lower right of FIG. 3).

[0035] The information processing device 1 stores the template image selected based on the user's input in the template image storage unit 12b. Instead of acquiring a template image based on the user's input from an SEM image acquired from the scanning electron microscope 102, the information processing device 1 may, for example, display a list of the template images stored in the template image storage unit 12b, accept selection of one of the displayed template images from the user, and use the selected template image for subsequent pattern detection processing. Note that the method by which the information processing device 1 acquires a template image can be appropriately selected by the user, for example, on a menu screen or the like.

[0036] 4 to 6 are schematic diagrams illustrating an example of pattern detection processing. The information processing device 1 acquires an SEM image and a template image to be processed, and then performs processing to extract feature points from each of the SEM image and the template image. The upper part of FIG. 4 shows an example of the result of feature point extraction, in which the extracted feature points are superimposed on the SEM image and the template image. Each feature point has coordinate information in the SEM image and the template image, and feature amount information that indicates the characteristics of each point. The information processing device 1 can extract feature points from the SEM image and the template image using an existing feature point extraction method, such as SIFT or AKAZE.

[0037] Next, the information processing device 1 performs a matching process between the feature points of the SEM image and the feature points of the template image. In the matching process, the information processing device 1 determines which of the multiple feature points of the SEM image each feature point of the template image corresponds to, for example, based on the position and feature amount of each feature point. In the lower part of Fig. 4, the SEM image and the template image are displayed side by side, and an example of the result of the feature point matching process is shown by connecting the feature points of the SEM image and the feature points of the template image with straight lines.

[0038] Based on the results of the matching process, the information processing device 1 identifies an image area from the SEM image that is the same size as the template image so as to include as many feature points of the SEM image as possible that match the feature points of the template image. The information processing device 1 then extracts from the SEM image an image area that is larger than this identified image area (for example, an image area that is obtained by enlarging the identified image area by a predetermined amount in all directions). The upper part of Figure 5 shows an example of an image area extracted from the SEM image based on the results of the matching process, enclosed in a rectangular frame.

[0039] The information processing device 1 again performs a process of extracting feature points and a process of matching the extracted feature points on the image region extracted from the SEM image and the template image. The information processing device 1 further identifies an image region of the same size as the template image from this image region so as to include as many feature points in the image region that match the feature points of the template image as possible. The lower part of Figure 5 shows an example of an image region finally identified by the information processing device 1. The information processing device 1 determines the identified image region as an image region in which the same pattern as that of the template image is captured, and outputs the coordinates of the bounding box surrounding this image region as the pattern detection result.

[0040] After detecting one pattern from an SEM image, the information processing device 1 excludes the image region corresponding to the detected pattern from the original SEM image. The upper part of Fig. 6 shows an example of an SEM image from which the detected region has been excluded. The information processing device 1 can detect multiple patterns from one SEM image by repeatedly performing the above-described pattern detection procedure on the SEM image from which the image region of the detected pattern has been excluded. The lower part of Fig. 6 shows an example of an SEM image from which pattern detection has been repeatedly performed and multiple image regions corresponding to the detected patterns have been excluded.

[0041] The information processing device 1 repeatedly detects patterns from the SEM image as described above, and calculates the similarity between the detected pattern and the pattern of the template image each time a pattern is detected. When the similarity falls below a predetermined threshold, that is, when the information processing device 1 determines that the detected pattern is not similar to the pattern of the template image, the information processing device 1 ends the repeated pattern detection.

[0042] The information processing device 1 determines the state of each pattern detected from the SEM image, for example, whether there is an abnormality. For example, the information processing device 1 measures a distance such as the length or width of each detected pattern based on a bounding box and determines whether the measured distance is within a normal range, thereby determining whether each pattern is normal or abnormal. Furthermore, for example, the information processing device 1 calculates the similarity between each detected pattern and a pattern in a template image and determines whether the similarity exceeds a threshold, thereby determining whether each pattern is normal or abnormal. The information processing device 1 may determine the state of a detected pattern using various other methods.

[0043] The information processing device 1 displays the pattern detection results from the SEM image and the state determination results for the detected patterns on the display unit 14. The information processing device 1 notifies the user of the location of patterns in the SEM image and the state of each pattern by displaying an image in which bounding boxes surrounding the detected patterns are superimposed on the target SEM image acquired from the scanning electron microscope 102, for example, and highlighting the bounding boxes corresponding to patterns determined to be abnormal.

[0044] 7 is a flowchart showing an example of the procedure of the pattern detection process performed by the information processing device 1 according to embodiment 1. The image acquisition unit 11a of the processing unit 11 of the information processing device 1 according to embodiment 1 communicates with the scanning electron microscope 102 via the communication unit 13, and acquires an SEM image of the target substrate photographed by the scanning electron microscope 102 (step S1).

[0045] The image acquisition unit 11a displays the SEM image acquired in step S1 on the display unit 14, and acquires a template image onto which the pattern to be detected is transferred from the SEM image by accepting an operation to select an image area on the SEM image via the operation unit 15 (step S2). Note that the image acquisition unit 11a may acquire a template image onto which the pattern to be searched is transferred by accepting, from the user, a selection of one template image from multiple template images stored in the template image storage unit 12b. The image acquisition unit 11a stores the template image acquired in step S2 in the template image storage unit 12b (step S3).

[0046] The feature point extraction unit 11b of the processing unit 11 extracts feature points from the SEM image acquired in step S1 and the template image acquired in step S2 (step S4). The matching processing unit 11c of the processing unit 11 performs a feature point matching process to determine which feature points of the template image correspond to which feature points of the SEM image, based on the feature points extracted in step S4 (step S5).

[0047] Based on the results of the matching process in step S5, the pattern detection unit 11d of the processing unit 11 identifies an image area that is the same size as the template image and contains the most feature points that match those of the template image, and extracts an image area from the SEM image that is a constant multiple of the identified image area (step S6). The feature point extraction unit 11b re-extracts feature points from the image area extracted in step S6 (step S7). The matching processing unit 11c re-matches the feature points of the template image with the feature points of the image area re-extracted in step S7 (step S8). Based on the results of the re-matching in step S8, the pattern detection unit 11d creates a bounding box for the detected pattern, for example, by creating a bounding box to surround the matched feature points (step S9).

[0048] The pattern detection unit 11d calculates the similarity between the pattern in the template image and the pattern surrounded by the bounding box created in step S9, and determines whether this similarity is equal to or less than a predetermined threshold (step S10). If the similarity exceeds the threshold (S10: NO), the pattern detection unit 11d deletes the area corresponding to the detected pattern from the SEM image to be processed (step S11), returns the process to step S4, and performs the processes of steps S4 to S10 on the SEM image from which the area of ​​the detected pattern has been deleted.

[0049] If the similarity is equal to or less than the threshold (S10: YES), the state determination unit 11e of the processing unit 11 determines the state of each pattern detected from the SEM image, such as whether it is normal or abnormal (step S12). The display processing unit 11f of the processing unit 11 displays the detection result of the pattern indicated by the bounding box created in step S9 and the state determination result of each pattern performed in step S12 on the display unit 14 (step S13), and the processing ends.

[0050] <Summary> In the information processing system according to the present embodiment configured as described above, the information processing device 1 uses the scanning electron microscope 102 to capture an image of a substrate that has been subjected to processing such as etching by the substrate processing device 101, and detects a predetermined pattern formed on the substrate from the target image (SEM image) obtained by capturing the image. The information processing device 1 acquires a template image on which the detection target pattern is captured, extracts feature amounts such as feature points from each of the target image and the template image, and detects the detection target pattern from the target image based on the comparison result between the feature amounts of the template image and the feature amounts of the target image. As a result, the information processing system according to the present embodiment is expected to be able to detect various patterns from the target image without having to previously learn the detection target pattern by machine learning.

[0051] In the information processing system according to this embodiment, the information processing device 1 extracts feature points from each of a template image and a target image, matches the feature points of the template image with the feature points of the target image, and detects a target pattern from the target image based on the matching results. Existing methods such as SIFT can be used to extract feature points, and the information processing system according to this embodiment is expected to detect various patterns from the target image with higher accuracy without prior machine learning of the target pattern.

[0052] Furthermore, in the information processing system according to this embodiment, the information processing device 1 extracts an image area from the target image that is larger than the template image based on the result of matching of feature points, re-extracts feature points from this image area, matches the feature points of the template image with the feature points of the image area, and detects the target pattern from this image area. As a result, the information processing system according to this embodiment first extracts an image area from the target image that is likely to contain the target pattern, and can detect the pattern from this image area, which is expected to more accurately extract the target pattern from the target image.

[0053] Furthermore, in the information processing system according to this embodiment, the information processing device 1 extracts feature points from the target image excluding the detected pattern, matches the extracted feature points with feature points of a template image, extracts an image region based on the matching result, re-matches the feature points of the extracted image region with feature points of the template image, and detects patterns from the target image based on the re-matching result, thereby detecting multiple patterns from the target image. As a result, the information processing system according to this embodiment can detect multiple similar patterns when multiple similar patterns are arranged in a single target image.

[0054] In the information processing system according to this embodiment, the information processing device 1 calculates the similarity between a pattern detected from a target image and a pattern in a template image, and ends the above-described repeated process when the similarity falls below a threshold. This prevents the information processing system according to this embodiment from detecting a pattern with a low similarity as the target pattern, and is expected to prevent erroneous pattern detection.

[0055] Furthermore, in the information processing system according to this embodiment, the information processing device 1 accepts from the user a selection operation of an image area in which a detection target pattern is captured from a target image acquired from the scanning electron microscope 102, and sets the accepted image area as a template image. As a result, the information processing system according to this embodiment can be expected to acquire, for example, one pattern from a target image in which a plurality of similar patterns are arranged, based on the user's selection, and use the acquired pattern as a template image to detect a similar pattern from the target image.

[0056] Furthermore, in the information processing system according to this embodiment, the information processing device 1 stores a template image in the template image storage unit 12 b, accepts a selection from a plurality of template images stored in the template image storage unit 12 b, and detects a pattern from a target image based on the selected template image. As a result, in the information processing system according to this embodiment, the user can reuse the template image stored in the storage unit when, for example, detecting a pattern similar to a previously detected pattern from another target image, and therefore, improved convenience can be expected.

[0057] Furthermore, in the information processing system according to this embodiment, the information processing device 1 determines the state of each pattern detected from the target image, for example, whether it is normal or abnormal, etc. As a result, the information processing system according to this embodiment can be expected to provide the user with information about the state of each detected pattern in addition to the detection results of the patterns included in the target image.

[0058] 3 to 6 show schematic SEM images of one or more holes formed by etching a substrate in the vertical direction, and the process performed by the information processing device 1 using these holes as the pattern to be detected has been described. However, the pattern to be detected is not limited to such holes. The pattern may be, for example, functional elements such as transistors or capacitors formed in multiple numbers on a substrate, or various other formations such as thin films. The same applies to the following embodiments.

[0059] [Embodiment 2] In embodiment 2, a modified example of a repeating process for detecting a first pattern from an SEM image based on a template image and then detecting a second and subsequent patterns will be described. The information processing device 1 according to embodiment 2 acquires an SEM image acquired from the scanning electron microscope 102 and a template image on which a pattern to be detected is captured, and performs the process up to detecting the first pattern from the SEM image based on a matching result of feature points between both images, in the same procedure as the process performed by the information processing device 1 according to embodiment 1 described above.

[0060] After detecting the first pattern from the SEM image, the information processing device 1 according to the second embodiment replaces the old template image with an image of the detected pattern (i.e., an image within the bounding box obtained as the detection result) as a new template image. The information processing device 1 according to the second embodiment detects the second and subsequent patterns from the SEM image by performing so-called template matching processing using this new template image.

[0061] Template matching is a technique in which an image area of ​​the same size as a template image is extracted from a target image (SEM image), and if the similarity between the extracted image and the template image exceeds a threshold, it is determined that the target pattern is captured in this image area. By moving the extraction position of the image area from the target image and repeating the same similarity determination, and performing image area extraction and similarity determination at all positions in the target image, it is possible to detect one or more patterns from the target image.

[0062] 8 is a flowchart showing an example of the procedure of the pattern detection process performed by the information processing device 1 according to the second embodiment. Similar to the procedure of the flowchart shown in FIG. 7 , the processing unit 11 of the information processing device 1 according to the second embodiment performs the following operations: acquiring an SEM image of the target substrate (step S1), acquiring a template image (step S2), storing the template image (step S3), extracting feature points (step S4), matching the feature points (step S5), extracting an image region from the SEM image (step S6), re-extracting the feature points (step S7), re-matching the feature points (step S8), and creating a bounding box for the detected pattern (step S9). Note that the processes of steps S1 to S9 performed by the information processing device 1 according to the second embodiment are the same as the processes of steps S1 to S9 shown in FIG. 7 , and therefore some of the processes of steps S1 to S9 are omitted from FIG. 8 .

[0063] The pattern detection unit 11d of the processing unit 11 deletes areas corresponding to the patterns detected in steps S1 to S9 from the SEM image acquired in step S1 (step S31).The pattern detection unit 11d also replaces the images corresponding to the patterns detected in steps S1 to S9 as new template images (step S32).

[0064] Next, the pattern detection unit 11d extracts an image area of ​​the same size as the template image from the SEM image from which the area corresponding to the pattern detected in step S31 has been deleted (step S33). The pattern detection unit 11d calculates the similarity between the image area extracted in step S33 and the template image (step S34). The pattern detection unit 11d determines whether the similarity calculated in step S34 exceeds a predetermined threshold (step S35). If the similarity exceeds the threshold (S35: YES), the pattern detection unit 11d determines that the target pattern is captured in this image area, stores the position of the image area of ​​this pattern in the SEM image (step S36), and proceeds to step S37. If the similarity does not exceed the threshold (S35: NO), the pattern detection unit 11d determines that the target pattern is not captured in this image area, and proceeds to step S37. The pattern detection unit 11d determines whether the process of extracting an image region and determining the similarity with the template image has been completed for all positions in the SEM image (step S37). If this process has not been completed for all positions (S37: NO), the pattern detection unit 11d returns to step S32, moves the extraction position, and repeats the processes of steps S32 to S37.

[0065] When these processes have been completed for all positions (S37: YES), the state determination unit 11e of the processing unit 11 determines the state of each pattern detected from the SEM image, such as whether it is normal or abnormal, based on the positions stored in step S36 (step S38). The display processing unit 11f of the processing unit 11 displays on the display unit 14 a bounding box surrounding each pattern detected from the SEM image and the result of the state determination for each pattern performed in step S38 (step S39), and the process ends.

[0066] In the information processing system according to the second embodiment configured as described above, the information processing device 1 updates the template image based on a pattern detected from the SEM image, extracts multiple image regions of the same size as the template image from the SEM image, calculates the similarity between the updated template image and each extracted image region, and detects the image region in which the target pattern is captured based on the calculated similarity. As a result, the information processing system according to this embodiment can be expected to accurately detect these patterns from a target image in which, for example, multiple similar patterns are arranged.

[0067] The other configurations of the information processing system according to the second embodiment are the same as those of the information processing system according to the first embodiment, so the same parts are given the same reference numerals and detailed description thereof will be omitted.

[0068] [Third Embodiment] In an information processing system according to a third embodiment, the information processing device 1 presents a plurality of template image candidates to a user from an SEM image acquired from a scanning electron microscope 102 and acquires a template image by accepting a selection from the candidates. The information processing device 1 according to the third embodiment extracts feature points from the SEM image acquired from the scanning electron microscope 102 and creates a plurality of template image candidates by extracting appropriate image regions from the SEM image based on the extracted feature points.

[0069] When multiple patterns of the same shape are repeatedly formed on a substrate, the multiple feature points that can be extracted from the SEM image may include portions where similar feature point arrangements are repeated. The information processing device 1 according to the third embodiment searches for such repeated feature point arrangements from the entire set of multiple feature points extracted from the SEM image. The information processing device 1 can select one of the repeated feature point arrangements based on the search results, extract a rectangular image area including the multiple feature points included in this arrangement, and use it as a candidate for the template image. The information processing device 1 may extract multiple candidates from one SEM image. Note that the method of candidate extraction by the information processing device 1 is not limited to the above method, and any method may be adopted.

[0070] The information processing device 1 displays a list of multiple candidates for template images extracted from an SEM image and created on, for example, the display unit 14. The information processing device 1 accepts a user operation at the operation unit 15 to select one candidate from the multiple candidates displayed in the list, acquires the selected candidate as a template image, and performs processing to detect the pattern captured in this template image from the SEM image.

[0071] 9 is a flowchart showing an example of the procedure of pattern detection processing performed by the information processing device 1 according to embodiment 3. The procedure of pattern detection processing performed by the information processing device 1 according to embodiment 3 is the same as the procedure of pattern detection processing performed by the information processing device 1 according to embodiment 1 shown in FIG. 7 , except that step S2, in which a template image is acquired based on a user's input, is replaced with steps S51 to S52, in which the information processing device 1 presents template image candidates to the user from an SEM image and acquires a template image. Since step S3 and subsequent steps in the pattern detection processing performed by the information processing device 1 according to embodiment 3 are the same as those shown in FIG. 7 , some of the processing steps S3 to S13 are not shown in FIG.

[0072] The image acquisition unit 11a of the processing unit 11 of the information processing device 1 according to the third embodiment acquires an SEM image of a target substrate captured by the scanning electron microscope 102 (step S1). The feature point extraction unit 11b of the processing unit 11 extracts feature points from the SEM image acquired in step S1 (step S51). The image acquisition unit 11a performs processing, such as searching for repetitions of a specific arrangement of feature points, based on the feature points of the SEM image extracted in step S2, to extract appropriate image regions from the SEM image, thereby creating template image candidates (step S52). The display processing unit 11f of the processing unit 11 displays one or more template image candidates extracted from the SEM image in step S52 on the display unit 14 (step S53). The image acquisition unit 11a accepts the selection of a template image from the user by selecting one of the candidates displayed in step S53 via the operation unit 15 (step S54).

[0073] The image acquisition unit 11a stores the template image acquired in step S54 in the template image storage unit 12b (step S3). The process from step S3 onwards is the same as that shown in the flowchart of FIG. 7, and therefore a detailed description thereof will be omitted.

[0074] In the information processing system according to the third embodiment configured as described above, the information processing device 1 extracts feature points from a target SEM image and creates template image candidates from the SEM image based on the extracted feature points. The information processing device 1 displays the created candidates on the display unit 14, accepts a selection operation from a plurality of candidates from the user, and performs pattern detection from the SEM image using the selected candidate as a template image. As a result, the information processing system according to this embodiment can assist the user in creating template images, and is expected to improve user convenience.

[0075] The other configurations of the information processing system according to the third embodiment are the same as those of the information processing systems according to the first and second embodiments, so the same reference numerals are used for the same parts and detailed description thereof will be omitted.

[0076] [Fourth Embodiment] In an information processing system according to a fourth embodiment, an information processing device 1 accepts a natural language input by a user and selects a template image from a plurality of candidates based on the input natural language. The natural language to be accepted as input is, for example, a character string such as a word or a sentence in a language such as Japanese or English. To handle natural language, the information processing device 1 according to the fourth embodiment uses an LLM (Large Language Model) that has undergone machine learning in advance to interpret natural language. Note that the LLM may operate on the information processing device 1, or may operate on a device different from the information processing device 1, such as a remote server device.

[0077] In the information processing system according to the fourth embodiment, a user can input natural language such as "hole" or "mask" to the information processing device 1 to display candidate images depicting "hole" or "mask," and can determine the template image by selecting one of these candidates. The user can also add any modifier, such as "long, narrow hole" or "large mask," to cause the information processing device 1 to display candidate template images.

[0078] The information processing device 1 according to the fourth embodiment extracts template image candidates from an SEM image acquired from the scanning electron microscope 102 by the method described in the third embodiment, selects a candidate corresponding to the input natural language from among the plurality of candidates, and presents the selected candidates to the user as a list. The information processing device 1 may also select a template image corresponding to the input natural language from among the template images stored in the template image storage unit 12b, and present the selected template images to the user as a list.

[0079] The information processing device 1 converts, for example, a natural language input by a user into a feature vector using an LLM, and then converts multiple images serving as template image candidates into feature vectors using a learning model (encoder) that has undergone machine learning. The information processing device 1 calculates the similarity between the feature vector of the input natural language and the feature vector of each candidate, selects a predetermined number of candidates in descending order of similarity, and presents them to the user, thereby presenting candidates corresponding to the natural language input by the user. The learning model that converts images into feature vectors can be generated in advance by machine learning using training data that associates pre-collected template images with natural languages ​​that express the patterns of the template images (or feature vectors converted from the natural languages ​​using the LLM). Note that the method of presenting candidates corresponding to natural languages ​​to the user using the LLM described above is merely an example and is not limited thereto; any other method may be employed.

[0080] 10 is a flowchart showing an example of the procedure of pattern detection processing performed by the information processing device 1 according to embodiment 4. The procedure of pattern detection processing performed by the information processing device 1 according to embodiment 4 is the same as the procedure of pattern detection processing performed by the information processing device 1 according to embodiment 1 shown in FIG. 7 except that step S2 of acquiring a template image based on a user's input is replaced with steps S71 to S77 of presenting template image candidates in response to a natural language input from the user and acquiring a template image. Since step S3 and subsequent steps of the pattern detection processing performed by the information processing device 1 according to embodiment 4 are the same as those shown in FIG. 7, some of the processing steps S3 to S13 are not shown in FIG. 10.

[0081] The image acquisition unit 11a of the processing unit 11 of the information processing device 1 according to the fourth embodiment acquires an SEM image of a target substrate photographed by the scanning electron microscope 102 (step S1). The feature point extraction unit 11b of the processing unit 11 extracts feature points from the SEM image acquired in step S1 (step S71). Based on the feature points of the SEM image extracted in step S71, the image acquisition unit 11a performs processing such as searching for repetition of a specific arrangement of feature points to create candidate template images (step S72).

[0082] The image acquisition unit 11a accepts a natural language input from the user using the operation unit 15, such as a keyboard (step S73). The image acquisition unit 11a inputs the accepted natural language into the LLM, thereby interpreting the natural language using the LLM (step S74). Based on the result of the natural language interpretation using the LLM performed in step S74, the image acquisition unit 11a selects one or more candidates corresponding to the natural language input from the one or more candidates created in step S72 (step S75). The display processing unit 11f of the processing unit 11 displays the one or more candidates selected in step S75 on the display unit 14 (step S76). The image acquisition unit 11a accepts a template image selection from the user by selecting one of the candidates displayed in step S76 using the operation unit 15 (step S77).

[0083] The image acquisition unit 11a stores the template image acquired in step S77 in the template image storage unit 12b (step S3). The process from step S3 onwards is the same as that shown in the flowchart of FIG. 7, and therefore a detailed description thereof will be omitted.

[0084] In the information processing system according to the fourth embodiment having the above configuration, the information processing device 1 acquires natural language information about a pattern to be detected based on input from a user, and creates template image candidates from an SEM image according to the acquired natural language information. The information processing device 1 displays the created candidates on the display unit 14, accepts a selection operation from the user from among multiple candidates, and performs pattern detection from the SEM image using the selected candidate as a template image. As a result, the information processing system according to the fourth embodiment is expected to further improve user convenience in creating template images.

[0085] In the fourth embodiment, candidates are selected according to natural language by using LLM, but the method of selecting candidates according to natural language is not limited to the method using LLM. For example, natural language tags such as names or words that indicate the characteristics of the objects depicted in the images can be attached to images that can be candidates, and the corresponding tags can be searched for based on the natural language input by the user, and images with tags that match the search can be used as candidates for template images.

[0086] The other configurations of the information processing system according to the fourth embodiment are the same as those of the information processing systems according to the first to third embodiments, so the same reference numerals are used for the same parts and detailed explanations are omitted.

[0087] The embodiments disclosed herein are to be considered as illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0088] The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations, regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used.

[0089] REFERENCE SIGNS LIST 1 Information processing device (computer) 11 Processing unit 11a Image acquisition unit 11b Feature point extraction unit 11c Matching processing unit 11d Pattern detection unit 11e State determination unit 11f Display processing unit 12 Storage unit 12a Program 12b Template image storage unit 13 Communication unit 14 Display unit 15 Operation unit 99 Recording medium 101 Substrate processing device 102 Scanning electron microscope

Claims

1. A computer program that causes a computer to execute a process of detecting a predetermined pattern formed on a substrate from a target image of the substrate that has been processed by a substrate processing apparatus, the computer executing the process of: acquiring a template image onto which the pattern to be detected is captured; extracting feature amounts from the template image and the target image; and detecting the target pattern from the target image based on a comparison result between the feature amounts of the template image and the feature amounts of the target image.

2. The computer program according to claim 1, which extracts feature points from the template image and the target image, matches the feature points of the template image with the feature points of the target image, and detects the target pattern from the target image based on the matching result.

3. The computer program according to claim 2, further comprising: extracting an image area from the target image that is larger than the template image based on the matching result; matching feature points of the template image with feature points of the image area; and detecting the target pattern from the image area based on the matching result.

4. The computer program of claim 3, further comprising: repeating steps of extracting feature points of the target image excluding a detected pattern, matching the extracted feature points with feature points of the template image, extracting an image region based on the result of the matching, matching the feature points of the template image with feature points of the image region, and detecting a pattern based on the result of the matching, thereby detecting a plurality of target patterns from the target image.

5. The computer program according to claim 4, further comprising: calculating a similarity between a pattern detected from the target image and a pattern transferred to the template image; and terminating repeated detection when the similarity becomes equal to or less than a threshold value.

6. The computer program of claim 1, further comprising: updating the template image based on the detected pattern; extracting a plurality of image regions from the target image; and detecting an image region in which the target pattern is captured from the plurality of image regions based on the similarity between the template image and each image region.

7. The computer program according to claim 1, further comprising: receiving a selection operation of an image area into which a detection target pattern is captured from the target image; and setting the received image area as the template image.

8. The computer program according to claim 1, further comprising: extracting features from the target image; extracting a plurality of template image candidates from the target image based on the extracted features; accepting a selection operation from among the extracted plurality of candidates; and setting the accepted candidate as the template image.

9. The computer program according to claim 1, further comprising: acquiring natural language information related to the detection target; extracting template image candidates from the target image according to the natural language information; accepting a selection operation from among the extracted multiple candidates; and setting the accepted candidate as the template image.

10. A computer program as claimed in any one of claims 7 to 9, further comprising: storing the template image in a memory unit; accepting a selection operation from among a plurality of the template images stored in the memory unit; and detecting a detection target pattern from the target image based on the accepted template image.

11. The computer program according to claim 1, further comprising: determining a state of a pattern detected from the target image.

12. An information processing method in which an information processing device detects a predetermined pattern formed on a substrate from a target image captured of the substrate that has been processed by a substrate processing device, the information processing device acquiring a template image in which the pattern to be detected is captured, extracting features from the template image and the target image, respectively, and detecting the target pattern from the target image based on a comparison result between the features of the template image and the features of the target image.

13. An information processing device comprising a processing unit that performs processing to detect a predetermined pattern formed on a substrate from a target image captured of the substrate that has been processed by a substrate processing device, wherein the processing unit obtains a template image onto which the pattern to be detected is captured, extracts features from the template image and the target image, respectively, and detects the target pattern from the target image based on a comparison result between the features of the template image and the features of the target image.

Images