Computer program, information processing method, and information processing device

Abstract

The present application provides a computer program, an information processing method and an information processing device capable of assisting in detection processing related to a prescribed pattern formed on a substrate. The computer program causes a computer to perform detection processing related to a plurality of patterns of a prescribed shape formed on a substrate on one or more captured images obtained by capturing the substrate after being processed by a substrate processing device, and to perform processing of detecting an image area in which an existing pattern is reflected from the captured images, detecting the outline of the pattern in the image area, and detecting a length measurement site for the pattern based on the detection result of the outline. In at least one of the detection processing of the image area, the detection processing of the outline, and the detection processing of the length measurement site, condition information including at least one of coordinate designation, area designation, or text as a detection condition is acquired, and the detection processing related to other patterns is performed based on the condition information acquired with respect to one pattern.

Description

Technical Field

[0001] This disclosure relates to computer programs, information processing methods, and information processing apparatus. Background Technology

[0002] Patent Document 1 discloses a substrate processing apparatus in which an inspection unit is provided in the middle of the substrate transport path. This unit consists of a film thickness measuring device, a line width measuring device, an overlap measuring device, and a macroscopic defect inspector stacked together. The substrate to be processed is selectively transported into these inspection units, thereby enabling the substrate to be inspected appropriately within the apparatus as needed.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2002-151403 Summary of the Invention

[0004] This disclosure provides a computer program, information processing method, and information processing apparatus that are expected to assist in detection processing related to a specified pattern formed on a substrate.

[0005] One embodiment of the computer program involves a computer program that performs detection processing on one or more captured images obtained by photographing a substrate processed by a substrate processing apparatus. The detection processing is related to a plurality of patterns formed on the substrate in a predetermined shape. The computer performs the following processing: detecting an image region reflecting the pattern from the captured image; detecting the outline of the pattern in the image region; and detecting a length measurement portion of the pattern based on the detection result of the outline. In at least one of the detection processing of the image region, the detection processing of the outline, and the detection processing of the length measurement portion, condition information including at least one of coordinate specification, region specification, or text as a detection condition is obtained. Based on the condition information obtained with respect to one pattern, detection processing related to other patterns is performed.

[0006] According to this disclosure, it is expected that inspection processes related to the specified patterns formed on the substrate can be assisted. Attached Figure Description

[0007] Figure 1 This is a schematic diagram used to illustrate the general outline of the information processing system involved in this embodiment.

[0008] Figure 2 This is a schematic diagram illustrating an example of a pattern formed on a substrate.

[0009] Figure 3 This is a schematic diagram illustrating the outline of the detection processing performed by the information processing apparatus according to this embodiment.

[0010] Figure 4This is a block diagram illustrating one configuration example of the information processing apparatus according to this embodiment.

[0011] Figure 5 This is a flowchart illustrating an example of the pattern detection processing steps performed by the information processing apparatus according to this embodiment.

[0012] Figure 6 This is a schematic diagram illustrating a configuration example of the contour detection model of this embodiment.

[0013] Figure 7 This is a schematic diagram illustrating an example of contour detection processing.

[0014] Figure 8 This is a schematic diagram representing a construction example of a similar image contour detection model.

[0015] Figure 9 This is a schematic diagram illustrating an example of a learning method for a similar image contour detection model.

[0016] Figure 10 This is a flowchart illustrating an example of the contour detection processing steps performed by the information processing apparatus according to this embodiment.

[0017] Figure 11 This is a schematic diagram illustrating a structural example of a length measurement component detection model based on article information.

[0018] Figure 12 This is a schematic diagram illustrating a structural example of a similar image length measurement part detection model.

[0019] Figure 13 This is a schematic diagram illustrating an example of a learning method used to detect length measurement parts in similar images.

[0020] Figure 14 This is a flowchart illustrating an example of the length measurement part detection process performed by the information processing apparatus according to this embodiment.

[0021] Figure 15 This is a flowchart illustrating an example of the state determination process performed by the information processing apparatus according to this embodiment.

[0022] Figure 16 This is a flowchart illustrating an example of the contour detection process performed by the information processing apparatus according to Embodiment 2.

[0023] Figure 17 This is a diagram illustrating the method of appending information to a database.

[0024] Figure 18This is a schematic diagram illustrating an example of a screen display of an information processing apparatus according to this embodiment.

[0025] Figure 19 This is a schematic diagram illustrating an example of a screen display of an information processing apparatus according to this embodiment.

[0026] Figure 20 This is a schematic diagram illustrating an example of a screen display of an information processing apparatus according to this embodiment.

[0027] Figure 21 This is a schematic diagram illustrating an example of a screen display of an information processing apparatus according to this embodiment. Detailed Implementation

[0028] Hereinafter, specific examples of the information processing system according to embodiments of the present disclosure will be described with reference to the accompanying drawings. Furthermore, the present disclosure is not limited to these examples, but is defined by the claims and is intended to include all modifications of the same meaning and scope as the claims.

[0029] [Implementation Method 1]

[0030] <System Overview>

[0031] Figure 1 This is a schematic diagram used to explain the overview of the information processing system according to this embodiment. The information processing system according to this embodiment is configured to include an information processing device 1, a substrate processing device 101, and a scanning electron microscope 102, etc. The substrate processing device 101 is, for example, a process chamber for performing processes such as etching on a semiconductor wafer (substrate). The substrate after being processed by etching or the like by the substrate processing device 101 is photographed by the scanning electron microscope 102 to capture its surface shape, etc. The scanning electron microscope 102 is a device that observes an object by irradiating it with electron beams and detecting secondary electrons or transmitted electrons emitted from the object, and outputs a so-called SEM (Scanning Electron Microscope) image obtained by photographing the object. The SEM image output by the scanning electron microscope 102 is provided to the information processing device 1. Furthermore, in this embodiment, the SEM image obtained by the scanning electron microscope 102 is processed as the object, but it is not limited to this. For example, images obtained by the transmission electron microscope or optical microscope may also be processed as the object.

[0032] Various shapes of structures are formed on the surface of the substrate after it has been processed by the substrate processing apparatus 101, such as... Figure 1As shown in the central SEM image, sometimes multiple structures of the same shape are formed on a substrate. Hereinafter, in this embodiment, such structures of the same shape will be referred to as patterns. However, in reality, even if patterns of the same shape are intended to be formed, their size or shape may differ from one pattern to another due to various processes such as film deposition, etching, or exposure. Patterns whose size or shape deviates beyond a predetermined threshold are judged as abnormal. The information processing system according to this embodiment is a system that detects multiple patterns formed on a substrate based on an SEM image of the substrate taken by a scanning electron microscope 102, detects the outline of each detected pattern, and detects the length measurement portion of each pattern based on the detected outline.

[0033] Furthermore, in this embodiment, the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 are described as separate devices, but this is not a limitation. For example, the substrate processing apparatus 101 and the scanning electron microscope 102 may be a single device, the scanning electron microscope 102 and the information processing apparatus 1 may be a single device, or the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be a single device. Additionally, each of the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be further configured by combining multiple devices.

[0034] Figure 2 This is a schematic diagram illustrating an example of a pattern formed on a substrate. Figure 2 The left side shows an example of a scanning electron microscope (SEM) image of a substrate surface taken by a scanning electron microscope 102, which corresponds to a top view of the substrate structure. In this example, the SEM image shows a structure with multiple rectangular patterns (indicated by thick black lines) arranged at approximately equal intervals, but these rectangular patterns are the structures to be inspected. Figure 2 The right side schematically shows the structure of the object. In this example, the object is a frustum shape with a roughly rectangular upper surface and a thickness that gradually increases from top to bottom. Figure 2 The rectangle shown by the thick black line in the image on the left is equivalent to Figure 2 The inclined portion of the side of the structure shown on the right.

[0035] Figure 3 This is a schematic diagram illustrating the general outline of the detection processing performed by the information processing apparatus 1 according to this embodiment. The information processing apparatus 1 according to this embodiment acquires an SEM image of a target substrate obtained by a scanning electron microscope 102, and first performs pattern detection processing on the SEM image. Sometimes multiple patterns of the same shape are formed on the target substrate, and sometimes multiple patterns of the same shape are reflected in the SEM image acquired by the information processing apparatus 1. Figure 3 The top image shows an example of a SEM image, in which multiple rectangular frames with thick black lines are arranged horizontally and vertically. In this example, the structure with the shape of these rectangular frames is used as an object pattern, and the information processing device 1 performs detection processing.

[0036] Furthermore, in this example, assuming that multiple patterns are reflected in a single SEM image, the information processing device 1 acquires one SEM image and performs detection processing on each pattern, but is not limited to this. For example, the information processing device 1 may also acquire multiple SEM images reflecting multiple object patterns and perform detection processing on the multiple patterns reflected in the multiple SEM images. Alternatively, for example, the information processing device 1 may also acquire multiple SEM images reflecting a single object pattern and perform detection processing on the patterns reflected in the multiple SEM images.

[0037] The pattern detection processing performed by the information processing apparatus 1 is the process of detecting, for example, rectangular image regions that reflect a pattern from an SEM image acquired by a scanning electron microscope 102. By performing pattern detection on the acquired SEM image, the information processing apparatus 1 can obtain information such as the coordinates and dimensions of the rectangular boxes surrounding each pattern, also known as bounding boxes, for one or more patterns reflected in the SEM image. Based on the results of the pattern detection processing, the information processing apparatus 1 individually segments the bounding box regions from the SEM image, thereby extracting the image regions reflecting each pattern. Figure 3 The second figure from the top shows image regions of multiple object patterns extracted from SEM images.

[0038] Next, the information processing device 1 performs contour detection processing on one or more image regions obtained as a result of the pattern detection processing, identifying the outline of the object pattern. In this embodiment, the contour detection processing performed by the information processing device 1 is a process of detecting pixels in the image region that reflect the object pattern, a so-called segmentation process. Based on the result of the segmentation process, the information processing device 1 can consider the outermost pixels among the pixels reflecting the object pattern as the outline of the pattern. The information processing device 1 can perform segmentation processing on each image region obtained as a result of the pattern detection processing to obtain information indicating which pixel reflects the object pattern, a so-called mask image, as a result of the contour detection processing. Figure 3 The third figure from the top shows the result of overlaying the mask image obtained through contour detection onto the image region of the object pattern.

[0039] Next, the information processing device 1 performs the following processing: based on a mask image obtained as a result of contour detection processing, indicating which pixel reflects each pattern, the length measurement portion of each pattern is detected. Even if multiple patterns formed on a substrate are intended to have the same shape in their design, differences in size or shape may occur depending on various processes such as film deposition, etching, or exposure. In the information processing system according to this embodiment, the length of a specific portion of each pattern is measured, and based on whether the measured length is within the normal range, it is determined whether each pattern on the substrate is formed into the desired shape. The length measurement portion detection processing performed by the information processing device 1 is a process of detecting the portion of each pattern whose length should be measured to determine whether it is normal or abnormal.

[0040] exist Figure 3 In the fourth image from the top, as a result of the length measurement location detection processing, two arrows representing the length measurement location are superimposed on the image area of ​​each pattern. One end of each arrow represents the starting point of the length measurement location, and the other end represents the ending point. The straight-line distance connecting the starting point and the ending point is the measured distance. In this example, two arrows are shown as the length measurement location, and the length of two locations is measured for one pattern. That is, in this example, for each pattern shown within the rectangle with thick black lines, the vertical length and horizontal length within the rectangle are measured. The information processing device 1 can obtain images of length measurement locations depicting arrows or other markings representing the length measurement location, or coordinate information of the starting and ending points of the length measurement location in the image area of ​​each pattern, as a result of the length measurement location detection processing.

[0041] Next, the information processing device 1 performs the following processing: based on the length-measuring part image or coordinate information of the length-measuring part obtained as a result of the length-measuring part detection processing, it measures the length of the part. The information processing device 1 detects arrows and other markings depicted in the length-measuring part images of each pattern obtained as a result of the length-measuring part detection processing, calculates the length between the two ends of the detected arrows, etc., and acquires information such as the magnification during imaging by the scanning electron microscope 102, calculating the actual length of the length-measuring part. By comparing the measured length with predetermined upper and lower threshold values, the information processing device 1 can determine whether each pattern is normal or abnormal based on whether the length is within a specified range.

[0042] Thus, in the information processing system according to this embodiment, the information processing device 1 performs pattern detection processing, contour detection processing, and length measurement region detection processing based on the SEM image of the substrate captured by the scanning electron microscope 102, and determines whether each pattern is normal. When the information processing device 1 performs these detection processes, the user needs to set information such as detection-related conditions for the information processing device 1. For example, the user sets which of the multiple patterns reflected in the SEM image will be used as the detection object in the pattern detection processing, as a detection condition. Additionally, for example, the user sets which object pattern will be detected in the contour detection processing, i.e., which part of the mask image is desired as the result of the segmentation processing. Furthermore, for example, the user sets information related to which part of the object pattern should be used as the measurement region in the length measurement region detection processing.

[0043] The information processing system according to this embodiment facilitates user access to the system and improves its convenience by providing information input functions, such as assisting with conditions related to these detection processes. The information processing device 1 utilizes a learning model obtained through pre-processed machine learning, such as an LLM (Large Language Model), to receive user input of text-based information in natural language. Based on the received information, it determines the conditions for detection processing and performs detection processing according to the determined conditions. Furthermore, in addition to natural language-based information input, the information processing device 1 also receives input based on information specifying coordinates or regions of the displayed image, and performs detection processing accordingly. Hereinafter, in this embodiment, the user's natural language-based information input and the information input based on specifying coordinates or regions of the image will be referred to as "prompt"-based information input, and the input information will be referred to as "prompt." However, prompts may also include information input other than natural language, coordinate specification, and region specification.

[0044] In the information processing system of this embodiment, the aforementioned prompt-based information input is accepted in the three detection processes performed by the information processing device 1: pattern detection processing, contour detection processing, and length measurement part detection processing. However, the information processing system only needs to have a structure that accepts the aforementioned prompt-based information input in at least one of the three detection processes: pattern detection processing, contour detection processing, and length measurement part detection processing.

[0045] <Device Structure>

[0046] Figure 4This is a block diagram illustrating one configuration example of the information processing apparatus 1 according to this embodiment. The information processing apparatus 1 according to this embodiment can be implemented, for example, by installing a prescribed application program on a general-purpose information processing device such as a personal computer or server computer. However, the information processing apparatus 1 may also be a dedicated information processing device for controlling the substrate processing apparatus 101 or the scanning electron microscope 102. The information processing apparatus 1 according to this embodiment is configured to include a processing unit 11, a storage unit 12, a communication unit 13, a display unit 14, and an operation unit 15. Furthermore, in this embodiment, processing is described as being performed by a single information processing apparatus 1, but processing may also be performed by multiple devices distributed among the information processing apparatus 1.

[0047] The processing unit 11 is configured using computing devices such as CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), or quantum processor, as well as ROM (Read Only Memory) and RAM (Random Access Memory). The processing unit 11 performs various processes, including detecting patterns from SEM images of the substrate, detecting pixels corresponding to existing detected patterns, and detecting measurement areas of the detected patterns, by reading and executing the program 12a stored in the storage unit 12.

[0048] The storage unit 12 is configured using a high-capacity storage device such as a hard disk or an SSD (Solid State Drive). The storage unit 12 stores various programs executed by the processing unit 11, as well as various data required for processing by the processing unit 11. In this embodiment, the storage unit 12 stores program 12a executed by the processing unit 11. Additionally, the storage unit 12 includes a prompt storage unit 12b that stores and accumulates information input by the user during the testing process.

[0049] In this embodiment, the program (computer program, program product) 12a is provided in the form of a recording medium 99 such as a memory card or optical disc, and the information processing device 1 reads the program 12a from the recording medium 99 and stores it in the storage unit 12. However, the program 12a may also be written to the storage unit 12 during the manufacturing stage of the information processing device 1, for example. Additionally, the program 12a may be obtained by the information processing device 1 via communication from a program distributed by a remote server device, for example. Alternatively, the program 12a may be read by a writing device from a program recorded on the recording medium 99 and written to the storage unit 12 of the information processing device 1. The program 12a may be provided either by distribution via a network or by being recorded on the recording medium 99.

[0050] The prompt storage unit 12b stores prompt information input by the user in each detection process, including pattern detection processing, contour detection processing, and length measurement part detection processing. The prompt information includes text information in natural language, or information such as coordinates or regions specified by the user for an image. The prompt storage unit 12b stores various information, such as information indicating which detection process the prompt is related to, timestamp information such as the date / time of the prompt input, input information such as the image that is the object in the detection process based on the prompt, and / or output information such as the image that is the result of the detection process based on the prompt, together with the input prompt information. When the information processing device 1 receives prompt information input from the user during detection processing, it can read past prompt information from the prompt storage unit 12b and display it at a glance, accepting the user's selection from among them, thereby accepting the input of prompt information.

[0051] The communication unit 13 is connected to the scanning electron microscope 102 via a communication cable or signal cable, and transmits and receives data with the scanning electron microscope 102 via this cable. In this embodiment, the communication unit 13 receives SEM image data of the substrate transmitted from the scanning electron microscope 102 and provides the received data to the processing unit 11. Furthermore, in this embodiment, SEM images are transmitted from the scanning electron microscope 102 to the information processing device 1 via communication, but it is not limited to this; for example, SEM images can also be transmitted and received via a recording medium such as a memory card.

[0052] The display unit 14 is configured using a liquid crystal display or the like, and displays various images and text based on the processing of the processing unit 11. In this embodiment, the display unit 14 displays, for example, SEM images acquired from the scanning electron microscope 102, and displays information related to the results of various detection processes performed on the SEM images.

[0053] The operation unit 15 accepts user operations and notifies the processing unit 11 of the accepted operations. For example, the operation unit 15 accepts user operations via input devices such as mechanical buttons or a touch panel provided on the surface of the display unit 14. Alternatively, the operation unit 15 may be an input device such as a mouse and keyboard, and these input devices may be detachable from the information processing device 1.

[0054] Furthermore, the storage unit 12 may also be an external storage device connected to the information processing device 1. Additionally, the information processing device 1 may be a multi-computer system comprising multiple computers, or a virtual machine constructed via software. Furthermore, the information processing device 1 is not limited to the above-described structure; for example, it may not include a display unit 14 and an operation unit 15.

[0055] Furthermore, in the information processing apparatus 1 according to this embodiment, the processing unit 11 reads and executes the program 12a stored in the storage unit 12, thereby implementing functional units such as the image acquisition unit 11a, the pattern detection processing unit 11b, the contour detection processing unit 11c, the length measurement part detection processing unit 11d, the state determination unit 11e, and the display processing unit 11f as software in the processing unit 11. In addition, in this figure, the functional units of the processing unit 11 are shown as those related to SEM image-based detection processing; functional units related to other processing are omitted from the illustration.

[0056] The image acquisition unit 11a performs processing to acquire a SEM image, which is obtained by scanning electron microscope 102 taking a picture of the substrate after it has been processed by substrate processing apparatus 101. The image acquisition unit 11a communicates with scanning electron microscope 102 via communication unit 13, thereby acquiring the SEM image of the substrate taken by scanning electron microscope 102, and storing the acquired SEM image in storage unit 12.

[0057] The pattern detection processing unit 11b performs a process of detecting one or more patterns formed on the substrate using an SEM image of the substrate acquired from the image acquisition unit 11a. For example, the pattern detection processing unit 11b displays the SEM image acquired by the image acquisition unit 11a on the display unit 14, and selects a portion of the image area from the SEM image from the user's input, thereby acquiring that portion of the image area as a template image. Alternatively, the pattern detection processing unit 11b can also acquire a template image by selecting one or more template images stored in the prompt storage unit 12b from the user's input. The pattern detection processing unit 11b uses structures or the like reflected in the acquired template image as patterns to be detected, for example, by performing template matching processing, thereby detecting one or more patterns from the SEM image.

[0058] Template matching is a method that extracts an image region of the same size as a template image from an object image (SEM image). If the similarity between the extracted image and the template image exceeds a threshold, it is determined that the image region reflects a pattern of the target object. By moving the extracted image region from the object image and repeating the same similarity determination, image region extraction and similarity determination are performed at all locations in the object image, thereby enabling the detection of one or more patterns from the object image. Furthermore, the method by which the pattern detection processing unit 11b detects patterns from an SEM image based on a template image is not limited to the template matching method described above; various other methods can be employed. For example, the pattern detection processing unit 11b can extract feature points from the template image, extract feature points from the SEM image, and perform feature point matching between the two images, thereby detecting one or more patterns from the SEM image.

[0059] In this embodiment, when the pattern detection processing unit 11b acquires a template image that reflects a pattern as the object of detection, it can accept input from a user's prompt and acquire a template image corresponding to the input prompt. The prompt may be, for example, an article describing the shape features of the pattern of the object of detection in natural language. Furthermore, in this embodiment, the article accepted as a prompt does not need to have a complete subject, object, and predicate; for example, it may be an incomplete article containing only one word, such as "quadrilateral," or a list of multiple words. Additionally, the prompt may be, for example, a region specified in the SEM image that surrounds the pattern of the object of detection, the coordinates of one or more pixels contained in the pattern of the object of detection, or the coordinates of one or more pixels not contained in the pattern of the object of detection.

[0060] The pattern detection processing unit 11b uses a learning model such as LLM, obtained through pre-processed machine learning, to acquire a template image corresponding to the input prompt, and detects patterns from the SEM image that are identical to the patterns reflected in the template image. For example, the pattern detection processing unit 11b extracts one or more image regions from the SEM image that reflect the detected patterns, and outputs the extracted one or more image regions as the pattern detection result. Alternatively, for example, the pattern detection processing unit 11b may also output the coordinate information of the bounding box surrounding the pattern detected from the SEM image as the pattern detection result.

[0061] The contour detection processing unit 11c performs contour detection processing on one or more patterns detected by the pattern detection processing unit 11b from the SEM image. In this embodiment, the contour detection processing unit 11c detects the contours of the patterns by performing a so-called segmentation process. The contour detection processing unit 11c uses a general, fully learned learning model for image segmentation, such as SAM (Segment Anything Model), to segment the image regions of each pattern detected by the pattern detection processing unit 11b, thereby detecting the contours of the patterns reflected in each image region. Based on the result of the segmentation process, the contour detection processing unit 11c creates a mask image indicating which pixel in the image region reflects the pattern, and outputs the mask image as the result of the contour detection process.

[0062] In this embodiment, the contour detection processing unit 11c uses a SAM (Search Engine Model) in the segmentation process. This SAM is a learning model obtained by pre-processing machine learning on the input image and prompts, and by segmenting the input image into objects corresponding to the prompts. The contour detection processing unit 11c can, for example, perform specified operations on pixels included in the pattern of the detected object (reflecting the pattern of the detected object) or on pixels not included in the pattern of the detected object (not reflecting the pattern of the detected object) from the user's input, and use the coordinate information of the specified image region as prompt information. Alternatively, the contour detection processing unit 11c can also use information from a text in natural language describing features related to the object pattern from the user's input as prompts. The contour detection processing unit 11c inputs the image region obtained as a result of pattern detection and the prompt information obtained from the user's input to the SAM, and obtains a mask image of the segmentation result output by the SAM, thereby detecting the contour of the pattern from the image region.

[0063] Furthermore, when multiple image regions are extracted from a SEM image through pattern detection processing, the contour detection processing unit 11c performs contour detection on each of these multiple image regions individually. In this embodiment, when performing contour detection processing on multiple image regions, the contour detection processing unit 11c accepts user input prompts when performing contour detection processing on the first image region. However, for contour detection processing on the second and subsequent image regions, it does not accept input prompts individually, but rather performs contour detection processing on the second and subsequent image regions based on the prompts received regarding the first image region and / or the segmentation results related to the first image region.

[0064] The length measurement part detection processing unit 11d performs the following processing: based on the mask images of each pattern obtained as a result of contour detection performed by the contour detection processing unit 11c, it detects one or more length measurement parts for the patterns reflected in each image region. The length measurement part detection processing unit 11d outputs, for example, an image of the length measurement part depicting an arrow or line indicating the position of the length measurement part, or coordinate information of pixels in the image region that serves as the start and end points of the length measurement, as the detection result of the length measurement part.

[0065] In this embodiment, the length measurement part detection processing unit 11d uses a length measurement part detection model. This model is obtained by performing machine learning in advance, taking as input a mask image obtained as a result of contour detection processing and information such as text or coordinates as prompts, and outputting a length measurement part image (or coordinate information of the length measurement part) depicting arrows or lines representing the length measurement part, in response to the input mask image. For example, the length measurement part detection model can be implemented by changing a part of the SAM used in the segmentation process described above (changing it to output a length measurement part image or coordinates of the length measurement part instead of an output mask image).

[0066] The length measurement part detection processing unit 11d, for example, receives information from a user-received article describing the length measurement parts of a pattern in natural language as a prompt. It then inputs a mask image of the pattern's contour detection result into the length measurement part detection model, obtains the length measurement part image or the coordinates of the length measurement part output by the model, and performs length measurement part detection. Furthermore, when multiple patterns are detected from an SEM image, and multiple image regions and mask images are obtained, for the mask image of the first pattern, the length measurement part detection processing unit 11d performs length measurement part detection based on the user-received prompt input. For the mask images of the second and subsequent patterns, the length measurement part detection processing unit 11d does not individually accept the prompt input, but rather performs length measurement part detection processing related to the second and subsequent mask images based on the prompt received regarding the first mask image and / or the length measurement part detection results related to the first mask image.

[0067] Furthermore, regarding the mask image of the first pattern, the length measurement location detection processing unit 11d may not accept prompts based on the text, but instead overlay a mask image corresponding to, for example, the image area reflecting the existing pattern, and use the user's specified coordinates of the measurement position displayed in that image as a prompt. In this case, the length measurement location detection processing unit 11d provides the length measurement location as a prompt for the mask image of the first pattern, so it is not necessary to perform length measurement location detection using the length measurement location detection model for the first mask image. Regarding the mask images of the second and subsequent patterns, the length measurement location detection processing unit 11d performs length measurement location detection processing based on the prompts input by the user (the length measurement location detection result for the mask image of the first pattern).

[0068] The state determination unit 11e performs processing to determine the state of the pattern based on the length measurement portion detected by the length measurement portion detection processing unit 11d. The state determination unit 11e detects markers such as arrows or lines from the length measurement portion image obtained as a length measurement portion detection result, and calculates the length of the arrow based on, for example, the coordinates of its two ends. Alternatively, the state determination unit 11e calculates the length (distance) between the start and end points based on the coordinate information of the start and end points obtained as a length measurement result. The state determination unit 11e calculates the actual length of the length measurement portion of the pattern based on information such as the calculated distance on the image area and the magnification when the SEM image of the extracted image area was captured. The state determination unit 11e compares the calculated length with, for example, predetermined upper and lower limits. If the measured length is within the range of the upper and lower limits, the pattern is determined to be normal; otherwise, it is determined to be abnormal. Furthermore, the state determination unit 11e's state determination for each pattern is not limited to the above; any method can be used to determine any state.

[0069] The display processing unit 11f performs the following processing: it displays information such as the SEM image acquired from the scanning electron microscope 102, the detection results of the pattern detection processing unit 11b, the contour detection processing unit 11c, and the length measurement detection processing unit 11d on the SEM image, and the state determination results of the state determination unit 11e on each pattern on the display unit 14. For example, the display processing unit 11f generates an image by superimposing the bounding boxes surrounding each pattern detected by the pattern detection processing unit 11b onto the SEM image acquired from the scanning electron microscope 102 and displays it on the display unit 14. In addition, for example, the display processing unit 11f superimposes the image region of the pattern extracted from the SEM image based on the pattern detection results with the contour detection results, i.e., the mask image, for the image region, and displays the contour detection results on the display unit 14. In addition, for example, the display processing unit 11f displays the length measurement image obtained as a result of the length measurement detection processing on the display unit 14, or superimposes the marks such as arrows or lines onto the image of the contour detection results to display the coordinates of the start and end points obtained as a result of the feature part detection processing. Additionally, for example, the display processing unit 11f displays the length obtained by measuring the length of each pattern detected from the SEM image on the display unit 14, as well as information indicating whether the length is normal or abnormal.

[0070] <Pattern Detection and Processing>

[0071] In the information processing system of this embodiment, the scanning electron microscope 102 photographs the substrate after it has undergone substrate processing such as etching by the substrate processing apparatus 101. The information processing apparatus 1 acquires the photographed image (SEM image) obtained by the photograph and performs a state determination of the patterns formed on the substrate. For this state determination, the information processing apparatus 1 first performs pattern detection processing to detect multiple patterns formed on the substrate from the SEM image.

[0072] In pattern detection processing, the information processing device 1 displays an SEM image acquired from the scanning electron microscope 102 on the display unit 14 and prompts the user to input condition information related to the pattern being detected, specifically information for obtaining a template image reflecting the existing pattern of the object being detected. Conversely, the user can, for example, specify a region surrounding one of the patterns of the object being detected on the displayed SEM image. The information processing device 1 extracts the image region specified by the user from the SEM image and uses the extracted image region as a template image reflecting the existing pattern of the object being detected.

[0073] In this embodiment, the information processing device 1 may, for example, receive an input from a user that describes the shape of a pattern of a detection object in natural language, as conditional information for obtaining a template image. The information processing device 1 extracts regions from the SEM image that reflect patterns corresponding to the input natural language text, and uses the extracted image regions as template images. The natural language input from the user is, for example, strings of words or text based on languages ​​such as Japanese or English. To process natural language, the information processing device 1 in this embodiment uses an LLM (Local Language Model) obtained by pre-performing machine learning in a manner that interprets natural language. Furthermore, the LLM can operate within the information processing device 1, or it can operate within a different device, such as a remote server device.

[0074] In the information processing system of this embodiment, a user can input natural language terms such as "hole" or "mask" into the information processing device 1 to display candidate images showing "holes" or "masks," and can determine the template image by selecting one of these candidates. Furthermore, the user can add arbitrary modifiers such as "elongated hole" or "large mask" to display candidate template images on the information processing device 1.

[0075] The information processing device 1 extracts feature points from the SEM image acquired from the scanning electron microscope 102, and extracts appropriate image regions from the SEM image based on the extracted feature points, thereby creating multiple candidate template images. When multiple patterns of the same shape are repeatedly formed on a substrate, the multiple feature points that can be extracted from the SEM image may contain repeated configurations of similar feature points. The information processing device 1 searches for such repetitions of similar feature point configurations from the entirety of the multiple feature points extracted from the SEM image. Based on the search results, the information processing device 1 can select any location in the configuration of repeated feature points, extract a rectangular image region containing the multiple feature points contained in that configuration, and use it as a candidate template image. The information processing device 1 can also extract multiple candidates from a single SEM image. Furthermore, the method by which the information processing device 1 extracts candidates is not limited to the above method; any method can be used.

[0076] Feature point extraction processing, for example, involves determining feature points (pixels) such as edges or corners contained in an image. This can be performed using existing methods such as SIFT (Scale Invariant Feature Transformation) or AKAZE (Accelerated-KAZE). Each feature point extracted from the image by the information processing device 1 includes the location of the feature point in the image (e.g., x, y coordinates) and information about the numerical value (feature quantity) representing the feature of that point.

[0077] Information processing device 1, which extracts candidate template images from SEM images, selects a candidate corresponding to the input natural language from a plurality of candidates, displays a list of selected candidates, and prompts the user. Alternatively, information processing device 1 can also select a template image corresponding to the input natural language from template images stored in prompt storage unit 12b, display the selected template image as a candidate list, and prompt the user. Information processing device 1 receives the user's operation unit 15 to select any candidate from the multiple candidates displayed, and obtains the selected candidate as a template image.

[0078] Information processing device 1, for example, uses LLM to convert user-input natural language into feature vectors. Multiple candidate images, which will serve as template images, are converted into feature vectors using a pre-obtained machine learning model (encoder). Information processing device 1 calculates the similarity between the feature vector of the input natural language and each candidate feature vector, selects a predetermined number of candidates in descending order of similarity, and presents them to the user. This allows it to provide the user with candidate suggestions corresponding to the user's input natural language. The learning model for converting images into feature vectors can be pre-generated using machine learning with learning data, such as pre-collected template images and the natural language representing the pattern of the template image (or feature vectors obtained by converting the natural language using LLM). Furthermore, the method described above using LLM to provide candidate suggestions corresponding to natural language is an example, and is not limited to this; any other method can be used.

[0079] In this embodiment, the conditional information obtained from the user regarding the pattern to be detected, such as the aforementioned area specification information and natural language text information, is referred to as a prompt. The information processing device 1 stores and accumulates information related to the input prompt in the prompt storage unit 12b. For example, the information processing device 1 stores information indicating that the prompt is related to pattern detection processing, timestamp information such as the date / time of accepting the prompt input, and a template image extracted from the SEM image based on the prompt, along with the prompt (area specification or text, etc.) input during pattern detection processing, in the prompt storage unit 12b. The information processing device 1 can obtain the template image from the SEM image and can also obtain the template image stored in the prompt storage unit 12b. The user can choose which method to use to obtain the template image.

[0080] Information processing apparatus 1, which acquires a template image reflecting a pattern of a target object, detects patterns identical to those reflected in the template image from a SEM image. For example, information processing apparatus 1 detects one or more identical patterns reflected in the SEM image based on existing template matching methods. Alternatively, information processing apparatus 1 may extract feature points from both the template and SEM images, and match these feature points between the two images, thereby detecting one or more patterns from the SEM image. Furthermore, the method by which information processing apparatus 1 detects patterns identical to those in the template image from the SEM image is not limited to template matching or feature point matching; any method can be employed.

[0081] Information processing device 1, which detects one or more patterns in a SEM image through pattern detection processing, extracts image regions from the SEM image that reflect each detected pattern. The image regions of each pattern extracted from the SEM image become input information for subsequent contour detection processing.

[0082] Furthermore, in pattern detection processing, when a template image is extracted from a SEM image based on prompts from user-inputted text information, extracting this template image can be considered pattern detection processing for detecting the first pattern from the SEM image. Processing for detecting one or more patterns from a SEM image based on a template image can be considered as a second or subsequent pattern detection process utilizing the result of the first pattern detection. Additionally, when a template image is extracted by the user specifying a region in the SEM image, the user-specified region information serves as input for the pattern detection processing, and the extracted template image can be considered as the detection result of the first pattern.

[0083] Figure 5This is a flowchart illustrating an example of the pattern detection processing steps performed by the information processing apparatus 1 according to this embodiment. The image acquisition unit 11a of the processing unit 11 of the information processing apparatus 1 according to this embodiment communicates with the scanning electron microscope 102 via the communication unit 13, acquiring an SEM image of the target substrate captured by the scanning electron microscope 102 (step S1). The display processing unit 11f of the processing unit 11 displays the SEM image acquired by the image acquisition unit 11a in step S1 on the display unit 14 (step S2). The pattern detection processing unit 11b of the processing unit 11 accepts input of prompts (article information or area specification information, etc.) related to the detected pattern by accepting user operations on the operation unit 15 (step S3).

[0084] The pattern detection processing unit 11b determines whether the prompt received in step S3 is article information (step S4). If the prompt is article information (S4: "Yes"), the pattern detection processing unit 11b, based on the received article information, uses a learning model such as LLM to extract multiple image regions from the SEM image obtained in step S1 that have features consistent with the article information, thereby extracting candidate template images (step S5). The display processing unit 11f displays the candidate template images extracted in step S5 on the display unit 14 (step S6). The pattern detection processing unit 11b accepts the user's operation using the operation unit 15, and selects a candidate template image from the multiple candidates displayed on the display unit 14 (step S7), and proceeds to step S9.

[0085] If the prompt received in step S3 is not article information (S4: "No"), the pattern detection processing unit 11b extracts the specified region from the SEM image obtained in step S1 as a template image based on the information specified as the region of the prompt by the received input (step S8), and proceeds to step S9.

[0086] The pattern detection processing unit 11b stores the prompt received in step S3 and the template image extracted based on the prompt, along with information such as information indicating that the prompt is related to pattern detection processing and timestamp information such as the date / time of the prompt input, in the prompt storage unit 12b (step S9). Based on the obtained template image, the pattern detection processing unit 11b performs processing such as template matching or feature point matching, thereby detecting one or more patterns matching the template image from the SEM image (step S10). For the one or more patterns detected in step S10, the pattern detection processing unit 11b extracts the image region reflecting the pattern from the SEM image (step S11). The display processing unit 11f displays the image regions of the one or more patterns extracted in step S11 on the display unit 14 (step S12), and the processing ends.

[0087] <Contour Detection Processing>

[0088] Information processing apparatus 1, which obtains multiple image regions reflecting existing patterns from SEM images through pattern detection processing, performs contour processing to detect the pattern contours of each image region. In this embodiment, information processing apparatus 1 detects the contours of patterns from each image region by performing a so-called segmentation process, which involves processing the pixels reflecting existing patterns in the image regions. Information processing apparatus 1 uses a pre-learned contour detection model obtained through machine learning to perform contour detection processing on each of the multiple image regions obtained as a result of pattern detection processing.

[0089] Figure 6 This is a schematic diagram illustrating one configuration example of the contour detection model in this embodiment. The contour detection model 200 used in the contour detection processing of the information processing apparatus 1 according to this embodiment is a learned model that accepts an input image and prompts containing various information related to the object to be segmented as input, and outputs mask information indicating which pixel in the input image reflects the existing object as the contour detection result. The contour detection model 200 is, for example, a learned model that has been trained for segmentation using existing methods such as SAM.

[0090] The prompts input to the contour detection model 200 may include, for example, a mask image, coordinate information, region specification, and text information. However, in this embodiment, a mask image may not be used as the prompt input. The coordinate information input to the contour detection model 200 may be, for example, information specifying the coordinates of one of the multiple pixels contained in the object reflected in the input image, or, for example, information specifying the coordinates of one of the multiple pixels not contained in the object reflected in the input image. Region specification may be, for example, information such as the coordinates of the bounding box (rectangle) that surrounds the object in the input image to specify the region in the input image that reflects the segmented object. Text information may be information such as articles or words that describe the shape or color of the segmented object in natural language. The mask image input as a prompt may be, for example, information specifying a portion of the input image that fills the segmented object, i.e., information about the multiple pixels in the input image that reflect the object.

[0091] The contour detection model 200 of this embodiment is configured with an image encoder 201, a mask encoder 202, a prompt encoder 203, and a mask decoder 204. The image encoder 201 converts the input image into feature values. The mask encoder 202 converts the mask image input as a prompt into feature values. The feature value synthesized from the feature values ​​output by the image encoder 201 and the mask encoder 202 is input to the mask decoder 204. The prompt encoder 203 converts the coordinate information, region specification, or text information input as a prompt into feature values. The mask decoder 204 performs the following processing: based on the feature value synthesized from the outputs of the image encoder 201 and the mask encoder 202, and the feature value output by the prompt encoder 203, it generates a mask image as a segmentation result of the input image.

[0092] Figure 7 This is a schematic diagram illustrating an example of contour detection processing. Figure 7 The first image from the left is an example of an input image for contour detection processing, and an example of an image region reflecting a pattern detected by the information processing device 1 through pattern detection processing. In this example, the processing aims to detect the contour of the rectangular frame with thick black lines reflected in the input image as a pattern. The information processing device 1 selects an appropriate one from the multiple image regions of the pattern obtained as a result of the pattern detection processing and displays it on the display unit 14, based on input from a user prompt.

[0093] In this example, the user inputs the coordinates of a point near the left end of a rectangle that serves as the object to be detected. Based on the user's input, the information processing device 1 sends the image area of ​​the displayed pattern and the received input to the contour detection model 200, and obtains a mask image output by the contour detection model 200 accordingly. Figure 7 The image of detection result 1 shown in the second figure from the left is an image in which the mask image of the processing result is superimposed on the image area of ​​the original pattern and displayed on the display unit 14 by the information processing device 1. In the detection result 1 of this example, the interior of the pattern of the target rectangle is included in the result of the segmentation process (contour detection process). The user in this example intends to use the pattern that does not include the interior of the rectangle as the result of the segmentation process, and the detection result 1 is different from the result desired by the user.

[0094] In this situation, the user can correct the detection results by adding further input prompts. In this example, the detection results are corrected by adding a prompt to provide the coordinates of a point (1) of a pixel that is not included in the detected object. Figure 7The image of detection result 2, shown in the third figure from the left, is the result corrected by the input of an additional prompt. As an additional prompt for correction, a point 1 near the center of the area inside the rectangle is specified. The user can repeatedly correct the detection result of the information processing device 1. Alternatively, if the initial detection result is the desired result, the user may choose not to make any corrections.

[0095] The information processing device 1 stores information such as prompts received from user input and mask images obtained through contour detection based on the prompts in the prompt storage unit 12b. When a user inputs a prompt, the information processing device 1 displays the prompts and mask images stored in the prompt storage unit 12b at a glance, and selects one or more prompts from the stored past prompts, thereby enabling the user to input the current prompt.

[0096] Information processing device 1, for one of the multiple image regions obtained as a result of pattern detection processing, performs segmentation processing using contour detection model 200 based on prompts input from the user, and outputs a mask image obtained as a contour detection result. The information processing device 1 of this embodiment performs contour detection processing on other image regions by reflecting the prompts for the first image region and the contour detection result in other image regions. That is, for the second and subsequent image regions obtained as a result of pattern detection processing, the information processing device 1 of this embodiment does not need to re-receive the prompts from the user, but can perform contour detection processing based on the prompts obtained for the first image region and the mask image obtained as the contour detection result of the first image region.

[0097] The method for performing contour detection on the second and subsequent image regions based on the prompts and contour detection results for the first image region can be any one of the three methods shown in (1) to (3).

[0098] (1) A method that uses contour detection results of similar images as input for a contour detection model

[0099] (2) A method for generating prompts based on feature quantities

[0100] (3) Method of generating prompts using rule base

[0101] (1) A method that uses contour detection results of similar images as input for a contour detection model

[0102] Figure 8This is a schematic diagram illustrating one configuration example of a similar image contour detection model. In the information processing apparatus 1 of this embodiment, the similar image contour detection model 210 used in contour detection processing for the second and subsequent image regions is a learned model obtained through pre-processing machine learning, which takes a reference image, the contour detection result of the reference image, an object image, and a filled image as input, and outputs a mask image of the contour detection result of the object image. In this embodiment, the reference image and the contour detection result of the reference image input to the similar image contour detection model 210 correspond to the first image region and the mask image obtained by the contour detection model 200 for that image region. Furthermore, in this embodiment, the object image input to the similar image contour detection model 210 corresponds to the second and subsequent image regions that are the detection targets.

[0103] Furthermore, in this embodiment, the filled image input to the similar image contour detection model 210 is an image obtained by filling the entire object image. The filled image is the input information required by the similar image contour detection model 210 during learning. When performing inference using the similar image contour detection model 210, the information processing device 1 generates an image obtained by filling the object image and inputs it into the similar image contour detection model 210.

[0104] Figure 9 This is a schematic diagram illustrating an example of a learning method for the similar image contour detection model 210. For the machine learning of the similar image contour detection model 210, learning data is prepared in advance, which corresponds to an image of the object undergoing contour detection processing (segmentation processing) and a mask image representing the contour detection result for that image. The information processing device 1 appropriately extracts two sets from multiple sets of the object image and the mask image, generating a filled image obtained by filling a portion of the mask image. In the illustrated example, the filled image is generated by filling three parts of the mask image with rectangular regions; however, this is just an example, and the filled image can be generated using any method.

[0105] Information processing device 1 establishes a correspondence between the four input information of similar image contour detection model 210: the input reference image is used as the first object image, the contour detection result of the reference image is used as the filled image generated based on the first mask image, the object image is used as the second object image, and the filled image is used as the filled image generated based on the second mask image. Information processing device 1 establishes a correspondence between the output information of similar image contour detection model 210, i.e., the contour detection result, and the second mask image, and uses this correspondence as the positive solution value of the input information for so-called supervised machine learning, thereby enabling machine learning of similar image contour detection model 210. Furthermore, the machine learning processing of similar image contour detection model 210 can also be performed outside of information processing device 1, but in a device different from information processing device 1, such as a server device.

[0106] The information processing device 1 uses the learned similar image contour detection model 210 to repeatedly perform contour detection processing on the second and subsequent image regions, and obtains a mask image as the result of contour detection for each image region. In this repeated process, the information processing device 1 can always use the reference image input to the similar image contour detection model 210 and its contour detection result as the first image region and its mask image, or it can use the image region and its mask image used in the previous contour detection process.

[0107] (2) A method for generating prompts based on feature quantities

[0108] Information processing device 1 generates a prompt for contour detection processing of the object image based on feature values ​​of a reference image (the first image region) and feature values ​​of the object image (the second and subsequent image regions). Thus, information processing device 1 can perform contour detection processing on the object image based on the generated prompt and... Figure 6 The contour detection model 200 shown is used to obtain a mask image by performing contour detection processing on the object image.

[0109] Information processing device 1, for example, uses an image encoder to convert a reference image (the first image region) and an object image (the second and subsequent image regions) into feature quantity information. Here, the information obtained through conversion is, for example, information assigning feature quantities to each pixel in the image. Information processing device 1 performs so-called feature point matching processing, which involves comparing the feature quantities of the reference image with those of the object image to determine pixels in the reference image that are similar to those in the object image. Based on the result of the matching processing, information processing device 1 determines which pixel in the object image a masked pixel in a mask image of the reference image corresponds to, and generates a prompt (coordinate specification or region specification, etc.) such that the contour detection model 200 detects the corresponding pixel in the object image.

[0110] Furthermore, the information processing device 1 can, for example, utilize an existing learning model called Matcher, which is a derivative technique of SAM, to generate the aforementioned prompts and a mask image based on the generated prompts. Matcher is a learning model that performs segmentation using feature matching, but since it is prior art, detailed description is omitted.

[0111] (3) Method of generating prompts using rule base

[0112] Information processing device 1 generates prompts for object images (second and subsequent image regions) according to predetermined rules, for example, based on prompts input by the user during contour detection processing of a reference image (first image region). The rules for generating the prompts may include, but are not limited to, the following.

[0113] When the user inputs text information as a prompt for the reference image, the information processing device 1 directly uses that text information as a prompt for the object image. When the prompt is a coordinate specification or a region specification, the information processing device 1 assumes the object image has the same size as the reference image and converts the coordinates of the reference image prompt to the coordinates of the object image. If the object image and the reference image have the same size, the information processing device 1 can directly use the coordinate specification and region specification prompts as prompts for the object image.

[0114] When contour detection is performed on the second and subsequent image regions using any of the methods described in (1) to (3) above, the information processing device 1 also displays an image, for example, overlaid with a mask image corresponding to each image region, as a summary of the contour detection results for the second and subsequent image regions on the display unit 14. At this time, the information processing device 1 can also display the contour detection results for the first image region and the contour detection results for the second and subsequent image regions together. The user can select one of the multiple contour detection results displayed in the summary and correct the contour detection result. For example, the user can perform operations such as specifying the coordinates of pixels included in the pattern, specifying the coordinates of pixels not included in the pattern, or inputting the correction area as text. The information processing device 1, which accepts the operation, appends the input correction information to the prompt and performs the contour detection processing again, so that the user's correction of one image region is reflected in other image regions.

[0115] Figure 10This is a flowchart illustrating an example of the contour detection processing steps performed by the information processing apparatus 1 according to this embodiment. The contour detection processing unit 11c of the processing unit 11 of the information processing apparatus 1 according to this embodiment acquires an image region of the detected pattern, which is then used as the detection result of the pattern detection processing unit 11b detecting the pattern from the SEM image (step S21). The contour detection processing unit 11c appropriately selects an image region from the plurality of image regions acquired in step S21 (step S22). The display processing unit 11f of the processing unit 11 displays the image region selected in step S22 on the display unit 14 (step S23).

[0116] The contour detection processing unit 11c, by accepting the operation of the operation unit 15, receives input from the user as a prompt (article information, coordinate specification, or area specification, etc.) regarding the conditions for contour detection of the image area of ​​the pattern displayed in step S23 (step S24). The contour detection processing unit 11c then sends the image area selected in step S22 and the information from the prompt accepted in step S24 to... Figure 6 The contour detection model 200 shown is input (step S25). The contour detection processing unit 11c acquires the mask image, which is the result of contour detection, and information indicating which pixel reflects the existing pattern, as output by the contour detection model 200 based on the information input in step S25 (step S26).

[0117] The contour detection processing unit 11c selects an image region from the multiple image regions acquired in step S21 that have not been selected so far (step S27). The contour detection processing unit 11c uses the image region selected in step S27 as the object image, the image region selected in step S22 as the reference image, and the mask image acquired in step S26 as the reference image, and applies the contour detection results to... Figure 8 The similar image contour detection model 210 is input (step S28). Furthermore, at this time, the contour detection processing unit 11c sets the filled image input to the similar image contour detection model 210 as an image obtained by filling the entire object image. The contour detection processing unit 11c acquires the mask image output by the similar image contour detection model 210 as the result of contour detection based on the information input in step S28 (step S29).

[0118] The contour detection processing unit 11c determines whether to end the contour detection processing for all image regions acquired in step S21 (step S30). If the processing for all image regions has not ended (S30: "No"), the contour detection processing unit 11c returns the processing to step S27, selects one unprocessed image region, and repeats the contour detection. If the processing for all image regions has ended (S30: "Yes"), the display processing unit 11f displays the image obtained, for example, by overlaying a mask image onto each image region, as the result of the contour detection processing on the display unit 14 (step S31). The contour detection processing unit 11c stores, for example, the image region selected in step 22, the prompt received in step S24, and the mask image acquired in step S26, along with information such as information indicating that the prompt is related to the contour detection processing and timestamp information such as the date / time of the prompt input, in the prompt storage unit 12b (step S32), and ends the processing.

[0119] <Length Measurement Part Inspection and Processing>

[0120] Information processing device 1, which obtains mask information indicating which pixel of each pattern is being represented from the image region representing each pattern through contour detection processing, performs processing to detect the length measurement portion of the pattern in each image region. In this embodiment, information processing device 1 acquires multiple groups of pattern-related image regions and mask images, selects an appropriate group from these groups, and displays an image obtained by superimposing the image region and mask image of the selected group on the display unit 14. The user inputs a prompt to specify the length measurement portion of the displayed image.

[0121] When a user inputs, for example, text information describing the features of the measurement area in natural language, as a prompt, the information processing device 1 uses a pre-learned measurement area detection model obtained through machine learning to detect the measurement area of ​​the pattern based on a mask image of the object pattern and the input text information. Figure 11 This is a schematic diagram illustrating one configuration example of a length measurement region detection model based on article information. In the length measurement region detection process used in the information processing apparatus 1 of this embodiment, the length measurement region detection model 220 is a learned model that accepts an input image and a prompt as input, and outputs a length measurement region image depicting markers such as arrows or lines representing the length measurement regions of the input image as the detection result. In this embodiment, the prompt input to the length measurement region detection model 220 is article information, and the input image is a mask image of a pattern.

[0122] The length measurement part detection model 220 of this embodiment is composed of an image encoder 221, a prompt encoder 222, and a length measurement part decoder 223. The image encoder 221 performs processing to convert the input image into feature quantities. The prompt encoder 222 performs processing to convert the text information input as a prompt into feature quantities. The length measurement part decoder 223 performs the following processing: based on the feature quantities output by the image encoder 221 and the feature quantities output by the prompt encoder 222, it generates a length measurement part image as the detection result of the length measurement part of the input image.

[0123] The length measurement part detection model 220, for example, can be improved through relearning, etc. Figure 6 The mask decoder 204 of the contour detection model 200 (or an existing SAM learning model, etc.) shown is replaced with the length measurement part decoder 223 to generate the model. The length measurement part detection model 220 can be generated by using training data to perform so-called supervised machine learning and update the parameters of the length measurement part decoder 223. The training data establishes a correspondence between the mask image of the contour detection result of the pattern, the article information related to the length measurement part, and the length measurement part image with arrows and other markings that become the detection result of the length measurement part for the mask image.

[0124] The information processing device 1 inputs the mask image obtained as a result of contour detection processing for each pattern and the article information obtained from user input as a prompt to the length measurement part detection model 220, and acquires the length measurement part image output by the length measurement part detection model 220 accordingly, thereby detecting the length measurement part. Furthermore, the information processing device 1 overlays the length measurement part image as the detection result onto the image area of ​​each pattern and the mask image and displays it on the display unit 14. Based on the correction of the user input prompt, the length measurement part detection processing is performed again.

[0125] Furthermore, when the user inputs information such as the coordinates of the starting and ending points of the actual specified measurement area as a prompt, the information processing device 1 can process the input coordinate information as a prompt, and can also process it as the detection result of the measurement area of ​​the pattern displayed in the first image area.

[0126] The information processing device 1 stores the prompts received from user input and the information of the measurement location based on the prompts in the prompt storage unit 12b. When the user inputs a prompt related to the measurement location, the information processing device 1 displays the prompts and measurement location information stored in the prompt storage unit 12b at a glance, and selects one or more prompts or measurement locations from the stored past prompts and measurement locations, thereby enabling the user to input the current prompt.

[0127] The information processing device 1 performs length measurement region detection processing using the length measurement region detection model 220 based on one of the multiple mask images obtained as a result of contour detection processing and the prompt from the text information input by the user. It then outputs the length measurement region image obtained from the length measurement region detection model 220 as the length measurement region detection result. Furthermore, when the user inputs coordinate information of the length measurement region as a prompt, the information processing device 1 generates an image based on the input coordinate information, for example, an image depicting an arrow or straight line connecting the coordinates of the starting point and the ending point on a black or white background of the same size as the corresponding pattern, and uses this image as the length measurement region detection result.

[0128] Furthermore, the information processing device 1 can display the detection result of the length measurement part based on the input of text information as a prompt on the display unit 14, and accept the user's correction operation for the detection result. At this time, the user can input the correction part as text information, or input the coordinate information of the length measurement part. When the information processing device 1 accepts the input of text information as correction information, it adds the text information related to the correction to the prompt of the already acquired text information, and performs the length measurement part detection again using the length measurement part detection model 220, thereby correcting the length measurement part detection result. In addition, when the information processing device 1 accepts the input of the coordinate information of the length measurement part as correction information, it can use the coordinate information as a new prompt and length measurement part detection result.

[0129] The information processing apparatus 1 according to this embodiment performs length measurement part detection processing on other mask images by reflecting the length measurement part detection result of the mask image for the first pattern in other mask images. That is, for the second and subsequent image regions of the multiple image regions obtained as a result of the pattern detection processing, the information processing apparatus 1 according to this embodiment does not need to receive input from the user again, but can perform length measurement part detection processing based on the length measurement part image obtained as a length measurement part detection result related to the mask image of the pattern reflected in the first image region.

[0130] In this embodiment, the information processing device 1 uses a length measurement part detection model based on the length measurement part detection results of similar input images to perform length measurement part detection on the mask image of the second and subsequent patterns. Figure 12This is a schematic diagram illustrating one configuration example of a similar image length measurement region detection model. In this embodiment, the similar image length measurement region detection model 230 used in the information processing apparatus 1 for length measurement region detection processing of the second and subsequent image regions is a learned model obtained through pre-processing machine learning, which takes a reference image, the length measurement region detection result of the reference image, an object image, and a fill image as input, and outputs a length measurement region image as the length measurement region detection result of the object image. In this embodiment, the reference image and the length measurement region detection result of the reference image input to the similar image length measurement region detection model 230 correspond to the mask image of the first pattern and the length measurement region image obtained by using the length measurement region detection model 220 on that mask image. Furthermore, in this embodiment, the object image input to the similar image length measurement region detection model 230 corresponds to the mask image of the second and subsequent pattern that is the detection object.

[0131] Furthermore, in this embodiment, the filled image input to the similar image length measurement part detection model 230 is an image obtained by filling the entire object image. The filled image is the input information required by the similar image length measurement part detection model 230 during learning. When performing inference using the similar image length measurement part detection model 230, the information processing device 1 generates an image obtained by filling the object image and inputs it into the similar image length measurement part detection model 230.

[0132] Figure 13 This is a schematic diagram illustrating an example of a learning method for a similar image length measurement part detection model 230. For the machine learning of the similar image length measurement part detection model 230, learning data is prepared in advance, corresponding to a mask image obtained as the contour detection result of a pattern, which will be the object of the length measurement part detection processing, and a length measurement part image that is the length measurement part detection result for that image. The information processing device 1 appropriately extracts two sets from multiple sets of the mask image and the length measurement part image, and generates a filled image obtained by filling a portion of the length measurement part image. In the illustrated example, the filled image is generated by filling two parts of the length measurement part image with rectangular regions; however, this is just an example, and the filled image can be generated using any method.

[0133] Information processing device 1, for the four input information of the similar image length measurement part detection model 230, establishes a correspondence by using the input reference image as the first mask image, the length measurement part detection result of the reference image as the fill image generated based on the first length measurement part image, the object image as the second mask image, and the fill image as the fill image generated based on the second length measurement part image. Information processing device 1 establishes a correspondence between the length measurement part detection result, which is the output information of the similar image length measurement part detection model 230, and the second mask image, and uses this as the positive solution value of the input information to perform so-called supervised machine learning, thereby enabling machine learning of the similar image length measurement part detection model 230. Furthermore, the machine learning processing of the similar image length measurement part detection model 230 can also be performed outside of information processing device 1, but in a device different from information processing device 1, such as a server device.

[0134] The information processing device 1 uses the learned similar image length measurement part detection model 230 to repeatedly perform length measurement part detection processing on the mask images of the second and subsequent patterns, and obtains a length measurement part image as the result of the length measurement part detection for each mask image. In this repeated process, the information processing device 1 can always use the reference image input to the similar image length measurement part detection model 230 and its length measurement part detection result as the first mask image and its length measurement part image, or it can use the mask image and its length measurement part image used in the previous length measurement part detection process.

[0135] The information processing device 1 displays multiple length measurement images as a summary of length measurement detection results for the second and subsequent image regions on the display unit 14. Alternatively, the information processing device 1 can display length measurement images for the mask image of the first pattern and length measurement images for the mask images of the second and subsequent patterns together. The user can select one of the multiple length measurement images displayed and correct the length measurement detection result. For example, the user can input the correction location as text information or input the coordinate information of the length measurement location for correction. Upon receiving the user's correction, the information processing device 1 appends the input correction information to the prompt and then performs the aforementioned length measurement detection processing, thereby reflecting the user's correction of one length measurement image in the other length measurement images.

[0136] Figure 14This is a flowchart illustrating an example of the steps involved in the length measurement part detection process performed by the information processing apparatus 1 according to this embodiment. The length measurement part detection processing unit 11d of the processing unit 11 of the information processing apparatus 1 according to this embodiment acquires multiple mask images corresponding to multiple patterns, and uses these as contour detection results detected by the contour detection processing unit 11c from the image region of the pattern (step S41). The length measurement part detection processing unit 11d appropriately selects one mask image from the multiple mask images acquired in step S41 (step S42). The display processing unit 11f of the processing unit 11 overlays the mask image selected in step S42 onto the image region of the pattern corresponding to that mask image and displays it on the display unit 14 (step S43).

[0137] The length measurement part detection processing unit 11d receives an input from the user, which is a prompt (article information, coordinate specification, or area specification, etc.) regarding the conditions for length measurement part detection of the image area and mask image displayed in step S43, by accepting the operation of the operation unit 15 (step S44). The length measurement part detection processing unit 11d determines whether the prompt received in step S44 is article information (step S45). If the prompt is article information (S45: "Yes"), the length measurement part detection processing unit 11d performs operations based on the article information. Figure 11 The length measurement part detection model 220 shown detects the length measurement part (step S46) and proceeds to step S48. Furthermore, in step S46, the length measurement part detection processing unit 11d takes the mask image selected in step S42 as the input image, inputs the article information received in step S44 as a prompt to the length measurement part detection model 220, and acquires the length measurement part image output by the length measurement part detection model 220, thereby enabling the detection processing of the length measurement part.

[0138] If the prompt is not article information (S45: "No"), since the prompt received in step S44 is the coordinate information of the length measurement part specified by the user, the length measurement part detection processing unit 11d takes the prompt as the detection result of the length measurement part (step S47) and proceeds to step S48.

[0139] The length measurement part detection processing unit 11d selects a mask image from the multiple mask images acquired in step S41 that have not been selected so far (step S48). Based on the length measurement part detection result for the mask image selected in step S42, the length measurement part detection processing unit 11d utilizes the mask image selected in step S48. Figure 12The similar image length measurement part detection model 230 shown performs length measurement part detection (step S49). At this time, the length measurement part detection processing unit 11d takes the mask image selected in step S48 as the object image, the mask image selected in step S42 as the reference image, and the length measurement part image obtained in step S46 or the length measurement part image generated based on the length measurement part detection result in step S47 as the reference image. It inputs these images into the similar image length measurement part detection model 230 and obtains the length measurement part image output by the similar image length measurement part detection model 230 as the length measurement part detection result, thereby enabling the detection of length measurement parts. At this time, the length measurement part detection processing unit 11d sets the filling image input to the similar image length measurement part detection model 230 to be an image obtained by filling the entire object image.

[0140] The length measurement part detection processing unit 11d determines whether to end the length measurement part detection process for all mask images acquired in step S41 (step S50). If the process for all mask images has not ended (S50: "No"), the length measurement part detection processing unit 11d returns the process to step S48, selects one mask image from the unprocessed images, and repeats the length measurement part detection. If the process for all mask images has ended (S50: "Yes"), the display processing unit 11f displays the length measurement part image obtained as a result of the length measurement part detection on the display unit 14 (step S51). The length measurement part detection processing unit 11d stores, for example, the mask image selected in step 42, the prompt received in step S44, and the length measurement part image obtained as a result of the length measurement part detection process in step S46, along with information such as information related to the length measurement part detection process and timestamp information such as the date / time of the prompt input, in the prompt storage unit 12b (step S52), and ends the process.

[0141] (Modified example)

[0142] Alternatively, information processing device 1 may not require the use of... Figure 12 The method of the similar image length measurement part detection model 230 shown uses... Figure 11 The length measurement detection model 220 shown repeats the length measurement detection process for subsequent mask images. As a prompt specifying the length measurement location for the mask image of the first pattern, the user may input, for example, "I want to measure the horizontal width 100 pixels down from the top inner contour." In this case, the information processing device 1 inputs the mask image and the user-input prompt to the length measurement detection model 220 and acquires the length measurement image output by the length measurement detection model 220.

[0143] Next, for the second and subsequent mask images, the information processing device 1 in the modified example uses the same text information as a prompt, inputs the mask image and the same text information into the length measurement part detection model 220, and obtains the length measurement part image output by the length measurement part detection model 220. By repeating the same processing on the second and subsequent mask images, the information processing device 1 in the modified example is able to obtain the detection results of the length measurement parts for each pattern.

[0144] <State Determination and Processing>

[0145] The information processing apparatus 1 according to this embodiment performs the following processing: based on the result of the length measurement region detection processing, it determines the state of each pattern detected from the SEM image, that is, whether each pattern is normal or abnormal. Based on the result of the length measurement region detection processing, the information processing apparatus 1 obtains the coordinates of the start and end points of the length measurement region in the partial image reflecting each pattern, and calculates the length of the length measurement region in the image based on the coordinates. The information processing apparatus 1 pre-stores information such as the magnification when the scanning electron microscope 102 photographs the substrate, and calculates the actual length of each pattern based on the calculated length of the length measurement region and the magnification during photography.

[0146] After calculating the actual length at the measuring point of each pattern, the information processing device 1 determines the state of each pattern by judging whether the calculated length is within a predetermined normal range. The information processing device 1 may, for example, determine the state based on a comparison with thresholds such as upper and lower limits preset by the user. Alternatively, it may determine the threshold based on statistical values ​​such as the average and dispersion of the lengths calculated from multiple patterns, and then determine the state based on a comparison with the determined threshold. Other methods may also be used to determine the threshold.

[0147] Figure 15 This is a flowchart illustrating an example of the state determination process performed by the information processing apparatus 1 according to this embodiment. The state determination unit 11e of the processing unit 11 of the information processing apparatus 1 according to this embodiment acquires the detection results of the length measurement part by the length measurement part detection processing unit 11d, i.e., multiple length measurement part images corresponding to multiple image regions extracted from the SEM image (step S61). The state determination unit 11e appropriately selects one length measurement part image from the multiple length measurement part images acquired in step S61 (step S62). Based on the length measurement part image selected in step S62, the state determination unit 11e measures, for example, the length of markings such as arrows or lines depicted in the length measurement part image, and calculates the actual length based on the magnification of the SEM image during capture, thereby measuring the length of the pattern (step S63).

[0148] The state determination unit 11e determines the state of the pattern by comparing the length measured in step S63 with predetermined upper and lower limits, etc. (step S64). The state determination unit 11e stores the state determination result performed in step S64 in the storage unit 12 (step S65).

[0149] The state determination unit 11e determines whether the state determination process has ended for all patterns extracted from the SEM image (step S66). If the process has not ended for all patterns (S66: "No"), the state determination unit 11e returns the process to step S62, selects one unprocessed measurement area image, and repeats the state determination. If the process has ended for all patterns (S66: "Yes"), the display processing unit 11f displays the state determination result on the display unit 14 (step S67), for example, by displaying a list of determination results for which of the multiple patterns detected from the SEM image is normal or abnormal, and ends the process.

[0150] <Summary>

[0151] In the information processing system of this embodiment with the above structure, the information processing device 1 captures one or more images of a substrate after it has undergone substrate processing such as etching by the substrate processing device 101 using a scanning electron microscope 102, and performs detection processing related to a predetermined pattern formed on the substrate. The information processing device 1 performs pattern detection processing to detect image regions reflecting the existing pattern from the captured images, contour detection processing to detect the outline of the pattern by segmenting pixels reflecting the existing pattern in the image regions, and length measurement region detection processing to detect length measurement regions of the pattern based on the contour detection results. In at least one of these pattern detection, contour detection, and length measurement region detection processes, the information processing device 1 of this embodiment acquires condition information including at least one of coordinate information, region specification, or text information as detection conditions as a prompt, and performs detection processing related to other patterns based on the prompt regarding the acquisition of one pattern. Therefore, the information processing system of this embodiment reduces the frequency and amount of user input of conditions in detection processing related to multiple patterns of a predetermined shape formed on the substrate, thus expecting to assist the user in performing the detection processing.

[0152] Furthermore, in the information processing system according to this embodiment, the information processing device 1 captures one or more images of a substrate after substrate processing by the substrate processing device 101 using a scanning electron microscope 102, and performs contour detection processing related to a pattern of a predetermined shape formed on the substrate. The information processing device 1 acquires an image region reflecting a pattern extracted from the captured image, acquires condition information including at least one of coordinate information, region specification, or text information as detection conditions as a prompt, and performs contour detection processing related to other patterns based on the prompt regarding the acquisition of one pattern. Therefore, the information processing system according to this embodiment reduces the frequency and amount of user input required for contour detection processing related to a predetermined pattern formed on the substrate, thus expecting to assist the user in performing contour detection processing.

[0153] Furthermore, in the information processing system of this embodiment, the information processing device 1 inputs an image region and a prompt as input to a first segmentation model (contour detection model 200) that has been trained to classify (segment) multiple pixels constituting the image, and obtains the segmentation result output by the first segmentation model, thereby detecting the pixels of the existing pattern in the image region, i.e., the contour detection of the pattern. Additionally, the information processing device 1 inputs an image region, the segmentation result of the reference image, and the object image as input to a second segmentation model (similar image contour detection model 210) that has been trained to segment the object image, and inputs the image region, the segmentation result of the image region, and other image regions, and obtains the segmentation result of the second segmentation model, thereby performing contour detection of the pattern in other image regions. Therefore, for multiple image regions extracted from a SEM image, the user only needs to input a prompt for the image region of the first pattern, and the information processing system of this embodiment can perform contour detection on the image regions of the second and subsequent patterns, thus assisting the user's work in contour detection processing.

[0154] Furthermore, in the information processing system of this embodiment, the information processing device 1 generates prompts for detection processing related to other patterns based on prompts regarding the acquisition of a pattern. The prompts can be generated, for example, using a learned model, or, for example, using a rule base based on pre-determined rules. Therefore, the information processing system of this embodiment can generate prompts related to a second and subsequent pattern based on prompts related to a first pattern, and use the generated prompts to perform detection processing on the second and subsequent patterns, thus expected to assist the user's work in contour detection processing.

[0155] Furthermore, in the information processing system of this embodiment, the information processing device 1 inputs an image region and a prompt as input to a learned segmentation model (contour detection model 200) that classifies (segments) multiple pixels constituting the image, and obtains the segmentation result output by the segmentation model. This allows for the detection of pixels in an image region that reflect an existing pattern, i.e., contour detection of the pattern. Additionally, the information processing device 1 calculates feature values ​​for each pixel in an image region and other image regions, extracts similar pixels in the image region and other image regions based on the calculated feature values, and generates prompts related to other image regions based on the segmentation result associated with an image region and the extracted similar pixels. Therefore, it is expected that the information processing system of this embodiment can generate prompts for image regions of second and subsequent patterns based on prompts related to the image region of the first pattern.

[0156] Furthermore, in the information processing system according to this embodiment, the prompts received by the information processing device 1 include information specifying pixels included in the detected pattern, or information specifying pixels not included in the pattern. Therefore, it is expected that the information processing system according to this embodiment will facilitate user operations for specifying a pattern as the object of contour detection.

[0157] Furthermore, in the information processing system according to this embodiment, the information processing device 1 displays (outputs) the contour detection result on the display unit 14, receives correction information for the detection result from the user, and corrects the detection result based on the correction information. Therefore, it is expected that the information processing system according to this embodiment can assist the user in making corrections while simultaneously trying out input prompts for detection conditions, thereby obtaining the desired detection result.

[0158] Furthermore, in the information processing system according to this embodiment, the information processing device 1 accepts input of correction information for detection results related to a pattern, and corrects detection results related to other patterns based on the correction information. Therefore, the information processing system according to this embodiment can, in cases such as contour detection processing of multiple image regions, reflect corrections made by the user to one image region in the image regions of other patterns, thus potentially assisting in correction operations performed by the user.

[0159] Furthermore, in the information processing system of this embodiment, the information processing device 1 acquires the detection results of an image region reflecting an existing pattern and the contour detection results of the pattern, acquires prompts such as coordinate specifications, region specifications, or text information input by the user, and detects length-measuring portions of the pattern based on the acquired detection results and prompts. Additionally, based on prompts regarding the acquisition of one pattern, the information processing device 1 performs length-measuring portions related to other patterns. Therefore, the information processing system of this embodiment reduces the frequency and amount of user input required for length-measuring portion detection processing related to multiple patterns of a predetermined shape formed on a substrate, thus potentially assisting the user in performing length-measuring portion detection processing.

[0160] Furthermore, in the information processing system of this embodiment, the information processing device 1 inputs a mask image of a pattern and a prompt as input to a first length measurement part detection model (length measurement part detection model 220) that has completed learning and has received the result of contour detection and a prompt, and detects the length measurement part of the pattern, thereby performing length measurement part detection for one pattern. Additionally, the information processing device 1 inputs a mask image of a pattern, the measurement part detection result of the mask image of one pattern, and mask images of other patterns to a second length measurement part detection model (similar image length measurement part detection model 230) that has completed learning and has received the reference image, the length measurement part detection result of the reference image, and the object image, and performs length measurement part detection for the object image, thereby performing length measurement part detection for other patterns. Therefore, based on the contour detection results of each pattern appearing in multiple image regions extracted from the SEM image, the user only needs to input a prompt for the mask image of the first pattern, and the information processing system involved in this embodiment can perform length measurement part detection on the mask images of the second and subsequent patterns, thus expecting to assist the user's work in performing length measurement part detection processing.

[0161] Furthermore, in the information processing system of this embodiment, the information processing device 1 determines the state of the pattern formed on the substrate based on the detection results of the length measurement portion. The information processing device 1 can measure the length of each pattern based on the detection results of the length measurement portion of the pattern reflected in each image region, and determine the state of each pattern, for example, based on whether the length measurement result is within a specified range. Therefore, it is expected that the information processing system of this embodiment can determine the state of each pattern formed on the substrate by the substrate processing device 101 based on the SEM image captured by the scanning electron microscope 102 with high precision.

[0162] Furthermore, in the information processing system of this embodiment, the information processing device 1 stores prompts obtained from user input as conditional information related to the detection process in the prompt storage unit 12b. During detection processing, instead of directly receiving prompts from the user, it retrieves the prompts stored in the prompt storage unit 12b to perform the detection processing. Therefore, in the information processing system of this embodiment, for example, when detecting a pattern that is the same as a pattern previously targeted for detection processing, the user does not need to input prompts again, thus assisting the user's operation in performing the detection processing.

[0163] Furthermore, in this embodiment, for example in Figures 1-3 , Figure 7 , Figure 9 , Figure 11 , Figure 13 The middle figure shows a diagram that patterns the SEM image of the irregularities formed on the substrate, illustrating how the information processing device 1 processes them as patterns of the object to be detected. However, the pattern of the object to be detected is not limited to such a three-dimensional structure. The pattern may be, for example, a plurality of functional elements such as transistors or capacitors formed on the substrate, or various formations such as thin films.

[0164] [Implementation Method 2]

[0165] <Contour Detection Processing>

[0166] As described above, in the information processing system of this embodiment, the SEM image captured by the scanning electron microscope 102 is sequentially subjected to pattern detection processing, contour detection processing, and length measurement region detection processing. The information processing system of Embodiment 2 differs from the information processing system of Embodiment 1 in that the contour detection processing step is performed in these processes. The steps of pattern detection processing and length measurement region detection processing are the same in the information processing systems of Embodiment 1 and Embodiment 2.

[0167] In the information processing system according to Embodiment 2, image regions of a pattern that is the object of contour detection and mask images that become the contour detection results of the pattern are collected in advance. The collected image regions of the pattern are converted into feature quantities, and the image regions of the pattern, feature quantities, and mask images are stored and accumulated in a database. Information processing device 1 may have this database, and devices different from information processing device 1 may also have this database, but at least information processing device 1 can access the database via communication or the like.

[0168] Furthermore, the collection of image regions of patterns stored in the database can be performed in advance by, for example, the designer of the information processing system according to Embodiment 2. Alternatively, data obtained during contour detection processing or similar processes performed by the information processing system of Embodiment 1 can also be stored in the database.

[0169] Furthermore, the transformation of image regions of a pattern into feature quantities can be performed, for example, using a learning model obtained through pre-processing machine learning. The technique of converting images into feature quantities using a learning model is existing technology, therefore a detailed explanation is omitted.

[0170] The information processing device 1 of the information processing system according to Embodiment 2 converts the image region of the pattern obtained by pattern detection processing of the SEM image into feature quantities. The information processing device 1 calculates the similarity (e.g., cosine similarity or L2 norm, etc.) between the converted feature quantities and the feature quantities of the image regions of the patterns stored in the database, and retrieves the image region of the pattern with the highest feature quantity similarity and the mask image group from the database.

[0171] The information processing apparatus 1 according to embodiment 2 utilizes the image region of the pattern and the mask image obtained from the database and Figure 8 The similar image contour detection model 210 shown performs contour detection on the image region of the object pattern. Specifically, the information processing device 1 uses image regions of similar patterns obtained from a database as input information to the similar image contour detection model 210, i.e., a "reference image," uses a mask image obtained from the database as the "contour detection result of the reference image," uses the image region of the pattern obtained through pattern detection processing as the "object image," and uses a mask image obtained by filling the entire object image as the "filled image." By inputting this information to the similar image contour detection model 210 and acquiring the mask image output by the similar image contour detection model 210, the information processing device 1 is able to detect contours from the image region of the object pattern.

[0172] Furthermore, in cases where the above-described method cannot be used to detect contours, or where image regions of similar patterns are not stored in the database, the information processing apparatus 1 according to Embodiment 2 performs contour detection from user input prompts, for example, similar to the information processing apparatus 1 of Embodiment 1. The information processing apparatus 1, having obtained the contour detection result based on the user input prompt, correspondingly stores the image region of the pattern for which contour detection was performed, the feature values ​​of that image region, and the mask image as a result of the contour detection in the database.

[0173] Furthermore, the similar image contour detection model 210 used in the contour detection process by the information processing apparatus 1 of Embodiment 1 can, for example, adopt an existing learned model such as segGPT.

[0174] Figure 16 This is a flowchart illustrating an example of the contour detection processing steps performed by the information processing apparatus 1 according to Embodiment 2. The contour detection processing unit 11c of the processing unit 11 of the information processing apparatus 1 according to Embodiment 2 acquires the image region of the detected pattern and uses it as the detection result of the pattern detection processing unit 11b detecting the pattern from the SEM image (step S71).

[0175] The contour detection processing unit 11c converts the image region of the pattern obtained in step S71 into a feature quantity (step S72). At this time, the contour detection processing unit 11c uses, for example, a learning model obtained by performing machine learning in advance in a way that converts the input image into a feature quantity and outputs it. The image region of the pattern is input into the learning model, and the feature quantity output by the learning model is obtained, thereby enabling the image region of the pattern to be converted into a feature quantity.

[0176] The contour detection processing unit 11c compares the feature values ​​obtained in step S72 with multiple feature values ​​pre-stored in a database, and retrieves an image region from the database that is similar to the image region of the pattern obtained in step S71 (step S73). At this time, the contour detection processing unit 11c calculates, for example, the similarity between the feature values ​​obtained in step S72 and each feature value stored in the database, and retrieves the image region of the pattern corresponding to the feature value with the highest similarity and its corresponding mask image from the database.

[0177] Furthermore, in step S71, when multiple image regions are acquired for a pattern, the contour detection processing unit 11c selects at least one of the image regions as a representative and performs the conversion to feature values ​​and the retrieval of similar image regions.

[0178] The contour detection processing unit 11c inputs the similar image regions and mask images obtained from the database in step S73, and the image regions of the pattern obtained in step S71 (and the filled image obtained by filling the image) into the learning model, namely the similar image contour detection model 210, which has been obtained through machine learning in advance (step S74). The contour detection processing unit 11c obtains the mask image of the contour detection result output by the similar image contour detection model 210 in response to the information input in step S74 (step S75).

[0179] Next, the contour detection processing unit 11c determines whether the contour detection result obtained in step S75 is correct (step S76). Here, the contour detection processing unit 11c can, for example, calculate the similarity between the mask image obtained in step S73 and the mask image obtained in step S76, and determine whether the contour detection result is correct based on whether the calculated similarity exceeds a predetermined threshold. When calculating the similarity between two mask images, the contour detection processing unit 11c can, for example, convert the mask image into a feature quantity and calculate the cosine similarity. Alternatively, it can, for example, use a template matching method to calculate the similarity. It can also, for example, calculate the IoU (Intersection over Union) of the two images as the similarity, or use other methods to calculate the similarity.

[0180] Furthermore, the contour detection processing unit 11c can also use methods other than those described above based on the similarity of the mask image to determine whether the contour detection result is correct. For example, when retrieving similar image regions from the database in step S73, the contour detection processing unit 11c may determine that the contour detection result is incorrect if no similar image regions with a similarity exceeding a predetermined threshold are stored in the database.

[0181] Alternatively, for example, the contour detection processing unit 11c can use a learning model pre-generated through machine learning to determine whether the contour detection result is correct. The learning model is generated, for example, using learning data that establishes a correspondence between a mask image and a flag indicating whether the mask image is correct or not, through so-called supervised machine learning. The generated learning model takes the mask image as input and outputs information indicating whether the mask image is correct or not. The contour detection processing unit 11c inputs the mask image obtained as the contour detection result in step S75 into the learning model and obtains the information output by the learning model, thereby determining whether the mask image is correct or not.

[0182] If the contour detection result obtained in step S75 is correct (S76: "Yes"), the display processing unit 11f of the processing unit 11 displays the contour detection result on the display unit 14 (step S80) and ends the contour detection process.

[0183] If the contour detection result is incorrect (S76: "No"), the contour detection processing unit 11c, for example, displays the image area of ​​the pattern obtained in step S71 on the display unit 14, and receives input from the user containing information required for contour detection of that image area (step S77). Based on the prompt received in step S77, the contour detection processing unit 11c, for example, uses... Figure 6The contour detection model 200 shown performs contour detection (step S78). The contour detection processing unit 11c stores the mask image obtained as a result of the contour detection in step S78, along with the image region of the pattern obtained in step S71 and the feature values ​​of that image region, in the database (step S79). The display processing unit 11f displays the result of the contour detection in step S78 on the display unit 14 (step S80) and ends the contour detection processing.

[0184] (Modified example)

[0185] In the above contour detection process, the information processing device 1 extracts an image region from the database that is similar to the image region of the pattern that is the object of contour detection, but it is not limited to this and can also extract multiple similar image regions.

[0186] exist Figure 16 In steps S73 to S76 of the flowchart shown, the information processing device 1 retrieves the image region and mask image most similar to the image region of the pattern to be the target from the database and performs contour detection, determining whether the contour detection result is correct. If the contour detection result is determined to be incorrect in step S76, the information processing device 1 in the modified example then retrieves similar image regions and mask images from the database and performs contour detection in the same way, determining whether the contour detection result is correct. The information processing device 1 in the modified example repeatedly retrieves image regions and mask images from the database in descending order of similarity until a correct contour detection result is obtained. For example, if the contour detection result based on the image region and mask image retrieved from the database for the Kth time is determined to be incorrect, the contour detection based on user input in steps S77 to S79 ​​is performed. Furthermore, K is a natural number, predetermined by the designer of the information processing system according to this embodiment.

[0187] Alternatively, the information processing apparatus 1 in the modified example may not acquire image regions and mask images similar to the image region of the contour detection object up to the predetermined Kth one, but instead acquire a group of image regions and mask images whose similarity is within a specified range, for example, in descending order of similarity.

[0188] Alternatively, the information processing apparatus 1 in the modified example may also include a similar image contour detection model 210. This model 210 is configured to, for example, accept multiple reference images and contour detection results, an object image, and a fill image as input, and output a mask image as the contour detection result of the object image. In this case, the information processing apparatus 1 in the modified example can input a group of multiple image regions and mask images obtained from a database into the similar image contour detection model 210 and obtain contour detection results for the image regions of the object.

[0189] <Methods for appending information to a database>

[0190] In cases where a correct contour detection result cannot be obtained based on information stored in the database, the information processing device 1 according to Embodiment 2 accepts user input prompts and performs contour detection, storing the contour detection result in the database. Alternatively, the information processing device 1 may store a mask image inverted from the mask image obtained based on the contour detection result from the user input prompts in the database. By using the inverted mask image as the contour detection result of the reference image input to the similar image contour detection model 210, the accuracy of contour detection for the image region of the object is improved.

[0191] Figure 17 This is a diagram illustrating the method of appending information to a database. For example... Figure 8 As shown, the similar image contour detection model 210 involved in this embodiment accepts a reference image and its contour detection result (mask image) and an object image and a filled image as input, and outputs a mask image of the contour detection result of the object image. Figure 16 In step S78 of the flowchart shown, the information processing device 1 acquires a mask image of the contour detection result for the image region of the pattern.

[0192] like Figure 17 As shown in the upper section, the information processing device 1 inputs the image region as both the object image and the reference image into the similar image contour detection model 210, and also inputs the mask image (usually a mask image) of the image region into the similar image contour detection model 210. Furthermore, in Figure 17 The filled image input to the similar image contour detection model 210 is omitted from the illustration. The information processing device 1 acquires a mask image of the contour detection result output by the similar image contour detection model 210, and calculates the similarity by comparing the acquired mask image with the input ordinary mask image.

[0193] In addition, such as Figure 17 As shown in the next section, the information processing device 1 generates an inverted mask image that reverses the normal mask image. The information processing device 1 inputs the original image region as the object image and the reference image into the similar image contour detection model 210, and also inputs the inverted mask image into the similar image contour detection model 210. The information processing device 1 acquires a mask image of the contour detection result output by the similar image contour detection model 210, inverts the acquired mask image, and compares the inverted mask image with the normal mask image to calculate the similarity.

[0194] The information processing device 1 determines whether to store the normal mask image or the inverted mask image by comparing the similarity calculated based on the normal mask image with the similarity calculated based on the inverted mask image. Furthermore, the similarity calculated by the information processing device 1 can be an appropriate value, such as IoU or cosine similarity.

[0195] When the similarity calculated based on the normal mask image is higher, the information processing device 1 stores the original image region and the normal mask image in a corresponding manner in the database. When the similarity calculated based on the inverted mask image is higher, the information processing device 1 stores the original image region and the inverted mask image in a corresponding manner in the database. In this case, the information processing device 1 may also store information indicating that the mask image corresponding to the image region is an inverted mask image together with the image region and the inverted mask image in the database.

[0196] <User Interface>

[0197] Figures 18-21 This is a schematic diagram illustrating an example of the screen display of the information processing apparatus 1 according to this embodiment. The information processing apparatus 1 according to this embodiment, for example, displays an image file selection screen (illustration omitted) and performs a selection operation from SEM images captured by the scanning electron microscope 102 at the user's request. The information processing apparatus 1 reads the files of one or more selected SEM images and, for example... Figure 18 The initial screen shown is displayed on the display unit 14. The initial screen displayed by the information processing device 1 has, for example, an image display area 301 that displays SEM images in the upper part of the screen, and three buttons arranged vertically on the lower left side of the screen: a preprocessing setting button 302, an automatic / manual setting button 303, and a segmentation setting button 304. A length measurement start button 305 is provided in the lower right of the screen.

[0198] The information processing device 1 establishes a corresponding image display area 301 on the initial screen, displaying the SEM image selected by the user and its filename. Figure 18 In the example shown, three SEM images named "File A1," "File A2," and "File A3" are selected. The information processing device 1 arranges the three SEM images horizontally and displays them in the image display area 301. The user, for example, uses a mouse or other operating unit 15 to make a rectangular selection of the SEM images displayed in the image display area 301, thereby selecting the pattern of the object to be measured. Figure 18 In the SEM image, a rectangular frame 306 surrounding the pattern selected by the user is overlaid and displayed. Furthermore, the user can specify a start and end point within the rectangular frame 306 using an operation unit 15, such as a mouse, thereby specifying the measurement portion of the pattern. Figure 18In the SEM image, a double-headed arrow 307 is overlaid to indicate the measurement location specified by the user.

[0199] The preprocessing setting button 302, the automatic / manual setting button 303, and the segmentation setting button 304, located on the initial screen, are buttons for the information processing device 1 to receive various settings related to pattern length measurement from the user. When these buttons are clicked with the mouse, the information processing device 1 displays a setting screen (not shown) for receiving detailed settings on the display unit 14. The preprocessing setting button 302 is used to perform preprocessing settings such as adjusting the brightness or contrast of the SEM image. The automatic / manual setting button 303 is used to set whether the contour detection processing for length measurement is performed automatically or manually by the user. For the information processing device 1, when the user sets it to automatic, contour detection using groups of similar image regions and mask images stored in the database, as described in Embodiment 2, is performed; when the user sets it to manual, contour detection based on user prompts, as described in Embodiment 1, is performed. The segmentation setting button 304 is used to set the type or size of the learning model used in the contour detection processing. In addition, when the length measurement start button 305 set on the initial screen is clicked with a mouse, the information processing device 1 starts the length measurement process with one or more SEM images displayed in the image display area 301 as the target.

[0200] When the length measurement process begins with automatic length measurement enabled, the information processing device 1 performs pattern detection processing on one or more given SEM images, retrieves image regions and mask images similar to the detected pattern image regions from the database, and performs contour detection processing. Furthermore, based on the contour detection results, the information processing device 1 performs length measurement region detection processing, measures the length of the detected length measurement regions, and displays the measurement results, for example, on [the database / platform / etc.]. Figure 19 The length measurement result display screen shown is as follows. The length measurement result display screen displayed by the information processing device 1 has, for example, an image display area 311 at the top of the screen that displays the SEM image, a numerical display area 312 at the bottom left of the screen that displays the numerical values ​​of the length measurement result, and a chart display area 313 at the bottom right of the screen that displays the length measurement result using charts.

[0201] The SEM image displayed by the information processing device 1 in the image display area 311 of the length measurement result display screen is the same as the SEM image displayed in the image display area 301 of the initial screen. However, for each SEM image, the information processing device 1 overlays one or more rectangular boxes surrounding the image area of ​​the detected pattern, bidirectional arrows indicating the length measurement part for each pattern, and identification information added by the information processing device 1 to each pattern. In addition, although the illustration is omitted, the information processing device 1 may also, for example, color-sort the mask image obtained as a contour detection result in the contour detection process according to the pattern and overlay it on the SEM image.

[0202] The information processing device 1 establishes a correspondence between the identification information attached to each pattern and the length measurement result of that pattern, and displays the result as a length measurement result arranged vertically in the numerical display area 312 of the length measurement result display screen. Additionally, the information processing device 1 displays, for example, a histogram summarizing the length measurement results of multiple patterns in the graph display area 313 of the length measurement result display screen. The histogram, for example, uses multiple levels obtained by dividing the length measurement result values ​​(length) within an appropriate range as the horizontal axis, and the number (frequency) of patterns corresponding to each level as the vertical axis.

[0203] also, Figure 19 The length measurement result display shown is the result obtained by the information processing device 1 through contour detection processing. For example... Figure 16 As shown in steps S76 to S79 ​​of the flowchart, when a correct contour detection result cannot be obtained based on the information stored in the database, the information processing device 1 performs contour detection based on the user's input. When a correct detection result cannot be obtained through contour detection processing, the information processing device 1 will... Figure 20 The failure notification screen shown is displayed on the display unit 14, notifying the user of the failure of the automatic contour detection process. The failure notification screen displayed by the information processing device 1 includes, for example, an image display area 321 at the top of the screen displaying an SEM image and a mask image related to the failed pattern, a message display area 322 at the bottom left of the screen notifying the length measurement result of the failure, and a manual length measurement button 323 at the bottom right of the screen for performing manual length measurement.

[0204] The information processing device 1 displays the SEM image of the pattern that failed contour detection processing, the image region of the pattern (object pattern), the mask image (estimated mask) of the contour detection result performed on the image region, and the image region (similar pattern) and mask image (similar mask) obtained from the database as image regions similar to the image region in an image display area 321 of the failure notification screen. The estimated mask displayed here is the incorrect contour detection result obtained through contour detection processing.

[0205] Additionally, the information processing device 1 notifies the user of the automatic length measurement failure by displaying a "failure" message in the message display area 322 of the failure notification screen as the length measurement result. If the user clicks the manual length measurement button 323 located on the failure notification screen, the information processing device 1 will... Figure 21 The manual length measurement screen shown is displayed on display unit 14.

[0206] The manual length measurement screen displayed by the information processing device 1 is divided into two areas, the upper area as a prompt input area 331 for inputting length measurement conditions or settings, and the lower area as a length measurement result display area 332 for displaying the length measurement result.

[0207] The information processing device 1 displays, for example, an SEM image of the object being measured on the left side of the prompt input area 331 of the manual length measurement screen. Based on user mouse operations, it accepts inputs such as a rectangular frame surrounding the object being measured, or the start and end points of the measurement section. To the right of the SEM image, the information processing device 1 displays an image region (object pattern) of the object being measured extracted from the SEM image. For this image region, it accepts inputs from the user regarding points included in the pattern or points not included in the pattern. The information processing device 1 uses this information as prompts to perform contour detection using the contour detection model 200, and displays the resulting mask image (estimated mask) to the right of the object pattern. Furthermore, the information processing device 1 can accept prompt input via natural language-based text input, either in place of rectangular frames or point (coordinate) inputs, or along with these inputs. A text box for text input is provided at the right side of the prompt input area 331. In addition, an application button is provided below the text box in the prompt input area 331. When the user clicks the application button, the information processing device 1 applies the conditions entered in the prompt input area 331 to perform contour detection processing, length measurement part detection processing, and length measurement processing on the same pattern contained in one or more SEM images.

[0208] The information processing device 1 displays one or more SEM images of the object being measured in the upper part of the measurement result display area 332 on the manual measurement screen, and displays the measurement results in a summary view at the bottom, using charts such as histograms. The information displayed in the measurement result display area 332 is consistent with the information displayed on the... Figure 19 The length measurement results displayed on the screen are largely the same, so detailed explanations are omitted.

[0209] A DB append button is located in the lower right corner of the length measurement result display area 332 on the manual length measurement screen. When the DB append button is clicked, the information processing device 1 stores the object pattern and estimated mask shown in the input prompt area 331 into a database. Furthermore, the information processing device 1 can also convert the object pattern into feature quantities and store these feature quantities, along with the object pattern and estimated mask, in the database. This information stored in the database is used as a retrieval object for similar patterns in subsequent length measurement processing.

[0210] <Summary>

[0211] In the information processing system according to Embodiment 2 of the above structure, the information processing device 1 pre-stores the image region of the pattern and a set of mask images that become the contour detection result in a database. Based on the image region of the pattern detected from the SEM image, the information processing device 1 retrieves the image region and the set of mask images from the database, and based on the retrieved image region and the set of mask images, detects the contour of the pattern from the image region detected from the SEM image. Therefore, the information processing system according to Embodiment 2 can perform contour detection using information stored in the database without requiring user input of conditional information (prompts) for contour detection.

[0212] Furthermore, in the information processing system according to Embodiment 2, when contour detection cannot be performed correctly based on information stored in the database, the information processing device 1 obtains a prompt, detects the contour of a pattern from an image region based on the obtained prompt, and stores the image region and a mask image that becomes the contour detection result in the database. Thus, when contour detection cannot be performed correctly based on information stored in the database, the information processing system according to Embodiment 2 can obtain a prompt from the user, perform contour detection, and store the result of the contour detection in the database for future contour detection.

[0213] Furthermore, in the information processing system according to Embodiment 2, the information processing device 1 retrieves a set of image regions and mask images from the database based on the similarity between the image region to be detected and each image region stored in the database. Additionally, if no image region with a similarity exceeding a threshold is stored in the database, the information processing device 1 determines that a contour cannot be detected from the image region of the object and provides a prompt. Therefore, it is expected that the information processing system according to Embodiment 2 can retrieve a set of image regions and mask images useful for contour detection from the database for the image region of the object to be detected.

[0214] Furthermore, in the information processing system according to Embodiment 2, the information processing device 1 determines whether a correct contour can be detected from the image region based on the similarity between a mask image of the contour detection result from the image region of the object and a mask image obtained from the database. Therefore, it is expected that the information processing system according to Embodiment 2 can determine with high accuracy whether contour detection can be performed based on information stored in the database.

[0215] Furthermore, in the information processing system according to Embodiment 2, the information processing device 1 displays one or more SEM images on the display unit 14, and overlays an image region of a pattern detected from the SEM image, a mask image of the contour detection result from the image region, or a length measurement location detected based on a link onto the SEM image, and displays the length measurement result based on the length measurement location. It is expected that the information processing system according to Embodiment 2 will provide the user with detailed information related to the length measurement of the pattern by displaying this information.

[0216] Furthermore, in the information processing system according to Embodiment 2, the information processing device 1 displays a histogram of length measurement results related to multiple patterns detected from the SEM image. Therefore, it is expected that the information processing system according to Embodiment 2 can provide the user with information related to deviations in the shape of the patterns.

[0217] Furthermore, in the information processing system according to Embodiment 2, when the information processing device 1 cannot obtain the correct contour detection result from the SEM image, it displays the original SEM image, the image region of the pattern, the mask image of the incorrect contour detection result obtained for each image region, and a group of image regions and mask images obtained from information stored in the database on the display unit 14. Additionally, the information processing device 1 accepts input prompts regarding the displayed SEM image. Therefore, it is expected that the information processing system according to Embodiment 2, when unable to obtain the correct contour detection result, will provide the user with information for determining the cause, and accept input prompts based on this information.

[0218] Furthermore, in implementation method 2, Figures 18-21 The screen display of the information processing device 1 shown is an example, and is not limited thereto. The information processing device 1 may also display information related to pattern detection processing, contour detection processing, length measurement part detection processing, and length measurement processing in any form on the display unit 14.

[0219] Furthermore, the other structures of the information processing system involved in Embodiment 2 are the same as those of the information processing system involved in Embodiment 1. Therefore, the same reference numerals are used to mark the same parts, and detailed descriptions are omitted.

[0220] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of this disclosure is defined by the claims, not by the foregoing, and is intended to include all modifications within the equivalent meaning and scope of the claims.

[0221] The items described in each embodiment can be combined with each other. Furthermore, the independent and dependent claims described in the claims can be combined with each other in all combinations, regardless of their referencing form. Moreover, the claims can be described in a form that refers to a claim that refers to two or more other claims (multiple claim form), but are not limited to this. It is also possible to use a form that describes a multiple claim that refers to at least one multiple claim (multiple-referencing-multiple-claims).

[0222] <Postscript>

[0223] (Note 1)

[0224] A computer program causes a computer to perform detection processing on one or more images obtained by photographing a substrate processed by a substrate processing apparatus, wherein the detection processing is related to a plurality of patterns formed on the substrate in a predetermined shape.

[0225] The above computer program causes the above computer to perform the following processes:

[0226] Detect the image region that reflects the above pattern from the captured images;

[0227] Detect the outline of the pattern in the image region described above; and

[0228] Based on the detection results of the above contour, the length measurement area for the above pattern is detected.

[0229] In at least one of the above-mentioned image region detection processing, contour detection processing, and length measurement region detection processing, condition information including at least one of coordinate specification, region specification, or text as a detection condition is obtained.

[0230] Based on the conditional information obtained about a pattern, detection processing related to other patterns is performed.

[0231] (Note 2)

[0232] According to the computer program described in Appendix 1

[0233] The storage unit stores multiple sets of image regions that reflect existing patterns and sets of contour detection results for the patterns in the image regions.

[0234] Based on the image region detected from the captured image, a set of image region and contour detection results is obtained from the storage unit.

[0235] Based on the obtained image region and contour detection results, the contour of the pattern is detected from the image region detected from the above-mentioned captured image.

[0236] (Note 3)

[0237] According to the computer program described in Appendix 2

[0238] If the outline of a pattern cannot be detected from the aforementioned image region, the aforementioned conditional information is obtained.

[0239] Based on the obtained conditional information, the outline of the pattern is detected from the image region.

[0240] The detection results of the above image region and the above contour are stored in the above storage unit.

[0241] (Note 4)

[0242] According to the computer program described in Appendix 2

[0243] Based on the similarity between the image region of the detected object and each image region stored in the storage unit, a set of image region and contour detection results is obtained from the storage unit.

[0244] If the image region with similarity exceeding the threshold is not stored in the aforementioned storage unit, it is determined that the contour cannot be detected from the aforementioned image region.

[0245] (Note 5)

[0246] According to the computer program described in Appendix 2

[0247] Based on the similarity between the contour detection result of the image region of the pattern detected from the above-mentioned captured image and the contour detection result obtained from the above-mentioned storage unit, it is determined whether the correct contour of the pattern can be detected from the above-mentioned image region.

[0248] (Note 6)

[0249] According to the computer program described in Appendix 2

[0250] Displays captured images of one or more objects being processed.

[0251] The image region of the pattern detected from the above-mentioned captured image, the outline of the above-mentioned pattern detected from the above-mentioned image region, or the measuring part detected based on the above-mentioned outline are superimposed on the above-mentioned captured image.

[0252] The display shows the measurement results based on the above-mentioned measurement locations.

[0253] (Note 7)

[0254] According to the computer program described in Appendix 6

[0255] Histograms showing the length measurement results related to multiple patterns detected from the aforementioned images.

[0256] (Postscript 8)

[0257] According to the computer program described in Appendix 3,

[0258] If the correct contour cannot be detected from the above image region, the original captured image obtained by detecting the above image region, the above image region, the incorrect contour detection result related to the above image region, and the group of image region and mask information obtained from the above storage unit based on the above image region are displayed.

[0259] (Note 9)

[0260] According to the computer program described in Appendix 8

[0261] Accept input of the aforementioned condition information for the displayed captured image.

[0262] Explanation of reference numerals in the attached figures

[0263] 1…Information processing device (computer); 11…Processing unit; 11a…Image acquisition unit; 11b…Pattern detection processing unit; 11c…Contour detection processing unit; 11d…Length measurement part detection processing unit; 11e…State determination unit; 11f…Display processing unit; 12…Storage unit; 12a…Program (computer program); 12b…Prompt storage unit; 13…Communication unit; 14…Display unit; 15…Operation unit; 99…Recording medium; 101…Substrate processing device; 102…Scanning electron microscope; 200…Contour detection model; 201…Image encoder; 202…Mask encoder; 203…Prompt encoder; 204…Mask decoder; 2 10…Similar image contour detection model; 220…Length measurement part detection model; 221…Image encoder; 222…Prompt encoder; 223…Length measurement part decoder; 230…Similar image length measurement part detection model; 301…Image display area; 302…Preprocessing setting button; 303…Automatic / Manual setting button; 304…Segmentation setting button; 305…Length measurement start button; 311…Image display area; 312…Number display area; 313…Chart display area; 321…Image display area; 322…Message display area; 323…Manual length measurement button; 331…Prompt input area; 332…Length measurement result display area.

Claims

1. A computer program that causes a computer to perform detection processing on one or more images obtained by photographing a substrate processed by a substrate processing apparatus, wherein the detection processing is related to a plurality of patterns formed on the substrate in a predetermined shape. The above computer program causes the above computer to perform the following processes: Detect the image region that reflects the above pattern from the captured images; Detect the outline of the pattern in the image region described above; and Based on the detection results of the above contour, the length measurement area for the above pattern is detected. In at least one of the above-mentioned image region detection processing, contour detection processing, and length measurement region detection processing, condition information including at least one of coordinate specification, region specification, or text as a detection condition is obtained. Based on the conditional information obtained about a pattern, detection processing related to other patterns is performed.

2. A computer program that causes a computer to perform detection processing on one or more images obtained by photographing a substrate processed by a substrate processing apparatus, wherein the detection processing is related to a plurality of patterns formed on the substrate in a predetermined shape. The above computer program causes the above computer to perform the following processes: Obtain an image region containing the aforementioned pattern extracted from the captured image; Obtain conditional information for the detection, including at least one of coordinate specification, region specification, or document; and Based on the above conditional information, the outline of the above pattern is detected in the above image region. Based on the conditional information obtained about a pattern, detection processing related to other patterns is performed.

3. The computer program according to claim 1 or 2, wherein, By inputting an image region and acquired conditional information into the first segmentation model after training, and obtaining the segmentation result of the first segmentation model, the outline of the pattern is detected in the image region. The first segmentation model after training accepts the image and conditional information as input, and classifies the multiple pixels constituting the image based on the conditional information. By inputting the aforementioned image region, the segmentation result of the aforementioned image region, and other image regions into the learned second segmentation model, and obtaining the segmentation result of the aforementioned second segmentation model, the outline of the aforementioned pattern is detected in the aforementioned other image regions. The learned second segmentation model accepts the reference image, the segmentation result of the reference image, and the object image as input, and performs segmentation on the aforementioned object image.

4. The computer program according to claim 1 or 2, wherein, Based on the conditional information obtained about one of the patterns, conditional information for detection processing related to the other patterns is generated.

5. The computer program according to claim 4, wherein, By inputting an image region and acquired conditional information into the learned segmentation model, and obtaining the segmentation result of the model, the outline of the pattern is detected in the image region. The learned segmentation model accepts the image and conditional information as input, and classifies the multiple pixels constituting the image based on the conditional information. Calculate the feature values ​​of each pixel in the aforementioned image region and the other image regions. Based on the calculated feature values, similar pixels from the aforementioned image region and other image regions are extracted. Based on the segmentation results associated with one of the aforementioned image regions and the extracted similar pixels, conditional information for segmentation associated with the other aforementioned image regions is generated.

6. The computer program according to claim 1 or 2, wherein, The above condition information includes information about the pixels contained in the specified detection pattern, or information about pixels not contained in the specified pattern.

7. The computer program according to claim 1 or 2, wherein, Output the detection results of the outline of the above pattern. Accept input of correction information regarding the above test results. Based on the above correction information, the above test results have been corrected.

8. The computer program according to claim 7, wherein, Accept inputs for corrections to detection results related to the aforementioned pattern. Based on the above correction information, the detection results related to the other patterns mentioned above are corrected.

9. The computer program according to claim 2, wherein, Retrieve conditional information for length measurement location detection, including at least one of coordinate specification, region specification, or article. Based on the detection results of the above contour and the above condition information, the length measurement part of the above pattern is detected. Based on the conditional information obtained about a pattern, detection processing related to other patterns is performed.

10. A computer program that causes a computer to perform detection processing on one or more images obtained by photographing a substrate processed by a substrate processing apparatus, said detection processing being related to a plurality of patterns formed on said substrate in a predetermined shape. The above computer program causes the above computer to perform the following processes: Obtain the detection results of the outline of the above pattern in the image region that reflects the existing pattern; Obtain conditional information for the detection, including at least one of coordinate specification, region specification, or document; and Based on the detection results of the above contour and the above condition information, the length measurement part of the above pattern is detected. Based on the conditional information obtained about a pattern, detection processing related to other patterns is performed.

11. The computer program according to any one of claims 1, 9, and 10, wherein, By inputting the detection result of a pattern's contour and the aforementioned conditional information into the first length measurement part detection model after it has been trained, and obtaining the information related to the length measurement part output by the first length measurement part detection model, the length measurement part for the aforementioned pattern is detected. The first length measurement part detection model after it has been trained accepts the detection result of the contour and the aforementioned conditional information as input, and outputs information related to the length measurement part for the aforementioned pattern. By inputting the detection results of the outline of the aforementioned pattern, the detection results of the length measurement part of the aforementioned pattern, and the detection results of the outlines of other patterns into the learned second length measurement part detection model, and obtaining the information related to the length measurement part output by the second length measurement part detection model, the length measurement part of the aforementioned other patterns is detected. The learned second length measurement part detection model accepts the reference image, the length measurement part detection results of the reference image, and the object image as input, and outputs information related to the length measurement part of the object image.

12. The computer program according to any one of claims 1, 2, and 10, wherein, Based on the results of the above detection and processing, the state of the above pattern is determined.

13. The computer program according to any one of claims 1, 9, and 10, wherein, Based on the detection results of the aforementioned measurement points, the length of the aforementioned pattern is measured. Based on the length measurement results, the state of the above pattern is determined.

14. The computer program according to any one of claims 1, 2, and 10, wherein, The above condition information is stored in the storage unit. Obtain the condition information used for detection processing from the aforementioned storage unit.

15. The computer program according to claim 1, wherein, The storage unit stores multiple sets of image regions that reflect existing patterns and sets of contour detection results for the patterns in the image regions. Based on the image region detected from the captured image, a set of image region and contour detection results is obtained from the storage unit. Based on the obtained image region and contour detection results, the contour of the pattern is detected from the image region detected from the above-mentioned captured image.

16. The computer program according to claim 15, wherein, If the outline of a pattern cannot be detected from the aforementioned image region, the aforementioned conditional information is obtained. Based on the obtained conditional information, the outline of the pattern is detected from the image region. The detection results of the above image region and the above contour are stored in the above storage unit.

17. The computer program according to claim 15, wherein, Based on the similarity between the image region of the detected object and each image region stored in the storage unit, a set of image region and contour detection results is obtained from the storage unit. If the image region with similarity exceeding the threshold is not stored in the aforementioned storage unit, it is determined that the contour cannot be detected from the aforementioned image region.

18. The computer program according to claim 15, wherein, Based on the similarity between the contour detection result of the image region of the pattern detected from the above-mentioned captured image and the contour detection result obtained from the above-mentioned storage unit, it is determined whether the contour of the pattern can be detected from the above-mentioned image region.

19. An information processing method comprising an information processing apparatus performing detection processing on one or more captured images obtained by capturing images of a substrate processed by a substrate processing apparatus, wherein the detection processing is related to a plurality of patterns formed on the substrate in a predetermined shape. The aforementioned information processing device performs the following processing: Detect the image region that reflects the above pattern from the captured images; Detect the outline of the pattern in the image region described above; Based on the detection results of the above contour, the length measurement part of the above pattern is detected; In at least one of the above-mentioned image region detection processing, contour detection processing, and length measurement region detection processing, condition information including at least one of coordinate specification, region specification, or text as a detection condition is obtained; and Based on the conditional information obtained about a pattern, detection processing related to other patterns is performed.

20. An information processing apparatus comprising a processing unit that performs detection processing on one or more captured images obtained by capturing images of a substrate processed by a substrate processing apparatus, the detection processing being related to a plurality of patterns formed on the substrate in a predetermined shape. The above processing department performs the following processing: Detect the image region that reflects the above pattern from the captured images; Detect the outline of the pattern in the image region described above; Based on the detection results of the above contour, the length measurement part of the above pattern is detected; In at least one of the above-mentioned image region detection processing, contour detection processing, and length measurement region detection processing, condition information including at least one of coordinate specification, region specification, or text as a detection condition is obtained; and Based on the conditional information obtained about a pattern, detection processing related to other patterns is performed.

Images