Computer program, information processing method, and information processing device

Abstract

Provided is a computer program, an information processing method, and an information processing device that can be expected to assist detection processing related to a prescribed pattern formed on a substrate.　This computer program causes a computer to perform detection processing related to a plurality of patterns that are formed in a prescribed shape on a substrate processed by a substrate processing device, with respect to one or more captured images obtained by imaging the substrate. The computer program causes the computer to execute processing of: detecting, from the captured images, an image region in which one of the patterns is included; detecting the outline of the pattern in the image region; and detecting a length measurement portion with respect to the pattern on the basis of the detection result of the outline. In at least one detection processing from the detection processing of the image region, the detection processing of the outline, and the detection processing of the length measurement portion, condition information including at least one of coordinates designation, region designation, or sentence that is to be a condition for detection is acquired, and on the basis of the condition information acquired for one pattern, detection processing of other patterns is executed.

Description

Computer program, information processing method, and information processing device

[0001] The present disclosure relates to a computer program, an information processing method, and an information processing device.

[0002] Patent Document 1 proposes a substrate processing apparatus in which an inspection unit having a stacked arrangement of a film thickness measuring device, a line width measuring device, an overlay measuring device, and a macro defect inspector is provided midway along the substrate transport path, and by selectively transporting the substrate to be processed into each of these inspection units, it is possible to inspect the substrate within the apparatus as needed.

[0003] Japanese Patent Application Laid-Open No. 2002-151403

[0004] The present disclosure provides a computer program, an information processing method, and an information processing device that are expected to assist in detection processing related to a predetermined pattern formed on a substrate.

[0005] A computer program according to one embodiment causes a computer to perform detection processing for multiple patterns of a predetermined shape formed on a substrate for one or more captured images of the substrate that has been processed by a substrate processing apparatus, and causes the computer to detect an image area in which the pattern is captured from the captured image, detect the contour of the pattern in the image area, and execute a process to detect measurement points for the pattern based on the contour detection results, and in at least one of the detection processes of the image area detection process, the contour detection process, and the measurement point detection process, acquire condition information including at least one of coordinate specification, area specification, or text that is a detection condition, and perform detection processing for another pattern based on the condition information acquired for one pattern.

[0006] According to the present disclosure, it is expected that detection processing relating to a predetermined pattern formed on a substrate can be supported.

[0007] FIG. 1 is a schematic diagram for explaining an overview of an information processing system according to the present embodiment. FIG. 1 is a schematic diagram showing an example of a configuration of a pattern formed on a substrate. FIG. 1 is a schematic diagram for explaining an overview of detection processing performed by an information processing device according to the present embodiment. FIG. 2 is a block diagram showing an example of a configuration of an information processing device according to the present embodiment. FIG. 3 is a flowchart showing an example of the procedure of pattern detection processing performed by the information processing device according to the present embodiment. FIG. 4 is a schematic diagram showing an example of a configuration of a contour detection model according to the present embodiment. FIG. 5 is a schematic diagram for explaining an example of contour detection processing. FIG. 6 is a schematic diagram for explaining an example of a training method for the similar image contour detection model. FIG. 7 is a flowchart showing an example of the procedure of contour detection processing performed by the information processing device according to the present embodiment. FIG. 8 is a schematic diagram showing an example of a configuration of a measurement portion detection model based on text information. FIG. 9 is a schematic diagram for explaining an example of a training method for the similar image measurement portion detection model. FIG. 10 is a flowchart showing an example of the procedure of measurement portion detection processing performed by the information processing device according to the present embodiment. FIG. 11 is a flowchart showing an example of the procedure of state determination processing performed by the information processing device according to the present embodiment. FIG. 12 is a flowchart showing an example of the procedure of contour detection processing performed by an information processing device according to embodiment 2. FIG. 13 is a schematic diagram for explaining a method of adding information to a database. Fig. 1 is a schematic diagram showing an example of a screen display by the information processing device according to the present embodiment. Fig. 2 is a schematic diagram showing an example of a screen display by the information processing device according to the present embodiment. Fig. 3 is a schematic diagram showing an example of a screen display by the information processing device according to the present embodiment. Fig. 4 is a schematic diagram showing an example of a screen display by the information processing device according to the present embodiment.

[0008] Specific examples of information processing systems according to embodiments of the present disclosure will be described below with reference to the drawings. Note that the present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope of the claims.

[0009] First Embodiment <System Overview> FIG. 1 is a schematic diagram illustrating an overview of an information processing system according to this embodiment. The information processing system according to this embodiment is configured to include an information processing apparatus 1, a substrate processing apparatus 101, a scanning electron microscope 102, and the like. The substrate processing apparatus 101 is an apparatus such as a process chamber that performs processing such as etching on semiconductor wafers (substrates). After processing such as etching in the substrate processing apparatus 101, the surface shape of the processed substrate is photographed by a scanning electron microscope 102. The scanning electron microscope 102 is an apparatus that observes the object by irradiating the object with an electron beam and detecting secondary electrons or transmitted electrons emitted from the object, and outputs a so-called SEM (Scanning Electron Microscope) image of the object. The SEM image output by the scanning electron microscope 102 is provided to the information processing apparatus 1. Note that in this embodiment, processing is performed on an SEM image photographed by the scanning electron microscope 102; however, the present invention is not limited to this, and processing may also be performed on an image photographed by, for example, a transmission electron microscope or an optical microscope.

[0010] Structures of various shapes are formed on the surface of a substrate processed by the substrate processing apparatus 101. For example, as shown in the SEM image in the center of FIG. 1 , multiple structures of the same shape may be formed on the substrate. Hereinafter, in this embodiment, each of these structures of the same shape will be referred to as a pattern. However, even if the formation of patterns of the same shape is intended, the size or shape of each pattern will vary due to processes such as film formation, etching, and exposure. A pattern whose size or shape deviates beyond a predetermined threshold is determined to be abnormal. The information processing system according to this embodiment is a system that detects multiple patterns formed on a target substrate based on an SEM image of the target substrate captured by a scanning electron microscope 102, detects the contours of each detected pattern, and detects measurement points for each pattern based on the detected contours.

[0011] In this embodiment, the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 will be described as separate apparatuses, but this is not limiting. For example, the substrate processing apparatus 101 and the scanning electron microscope 102 may be a single apparatus, the scanning electron microscope 102 and the information processing apparatus 1 may be a single apparatus, or the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be a single apparatus. Furthermore, each of the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be configured by combining a plurality of apparatuses.

[0012] FIG. 2 is a schematic diagram showing an example of a configuration of a pattern formed on a substrate. The left side of FIG. 2 shows an example of an image (SEM image) of the substrate surface captured by a scanning electron microscope 102, and this image corresponds to the top view of the substrate. The SEM image of this example shows a configuration in which multiple rectangular patterns, indicated by thick black lines, are arranged vertically and horizontally at approximately equal intervals. Each rectangular pattern in this image is the structure to be detected. The right side of FIG. 2 shows a schematic side view of the target structure. The target structure in this example has a substantially rectangular top surface and is a quadrangular pyramid truncated shape whose thickness gradually increases from top to bottom. The rectangle indicated by thick black lines in the image on the left side of FIG. 2 corresponds to the inclined portion of the side of the structure shown on the right side of FIG. 2.

[0013] FIG. 3 is a schematic diagram for explaining an overview of the detection process performed by the information processing device 1 according to this embodiment. The information processing device 1 according to this embodiment acquires an SEM image of a target substrate captured by a scanning electron microscope 102, and first performs pattern detection processing on this SEM image. Multiple patterns of the same shape may be formed on the target substrate, and the SEM image acquired by the information processing device 1 may contain multiple patterns of the same shape. An example of an SEM image is shown at the top of FIG. 3, in which multiple rectangular frames drawn with thick black lines are arranged vertically and horizontally. In this example, a structure shaped like this rectangular frame is treated as the target pattern, and the information processing device 1 performs detection processing.

[0014] In this example, multiple patterns are captured in one SEM image, and the information processing device 1 acquires one SEM image and performs detection processing on each pattern, but this is not limited to this. For example, the information processing device 1 may acquire multiple SEM images in which multiple target patterns are captured, and perform detection processing on the multiple patterns captured in the multiple SEM images. Furthermore, for example, the information processing device 1 may acquire multiple SEM images in which one target pattern is captured, and perform detection processing on the patterns captured in the multiple SEM images.

[0015] The pattern detection process performed by the information processing device 1 is a process of detecting, for example, a rectangular image area in which a single pattern is captured from an SEM image acquired from the scanning electron microscope 102. By performing pattern detection on the acquired SEM image, the information processing device 1 can obtain information such as the coordinates and size of a rectangular frame surrounding each pattern, a so-called bounding box, for one or more patterns captured in the SEM image. Based on the results of the pattern detection process, the information processing device 1 extracts image areas in which each pattern is captured by individually cutting out bounding box areas from the SEM image. The second from the top of Figure 3 shows multiple image areas of target patterns extracted from the SEM image.

[0016] Next, the information processing device 1 performs a contour detection process for the target pattern on one or more image regions obtained as a result of the pattern detection process. In this embodiment, the contour detection process performed by the information processing device 1 is a so-called segmentation process, which is a process for detecting pixels in an image region where the target pattern is captured. Based on the results of the segmentation process, the information processing device 1 can treat the outermost pixels among the pixels where the pattern is captured as the contour of the pattern. The information processing device 1 performs a segmentation process on each image region obtained as a result of the pattern detection process, and can obtain, as a result of the contour detection process, information indicating which pixels contain the target pattern, a so-called mask image. The third image from the top in Figure 3 shows the mask image obtained by contour detection superimposed on the image region of the target pattern.

[0017] Next, the information processing device 1 performs a process of detecting measurement points for each pattern based on a mask image obtained as a result of the contour detection process, which indicates which pixels each pattern is imaged in. Even if multiple patterns formed on a substrate are intended to have the same shape in design, differences in size or shape occur for each pattern due to processes such as film formation, etching, and exposure. The information processing system according to this embodiment determines whether each pattern on the substrate is formed in the desired shape by measuring the length of a specific point in each pattern and determining whether the measured length is within a normal range. The measurement point detection process performed by the information processing device 1 is a process of detecting points in each pattern where the length should be measured to determine whether the pattern is normal or abnormal.

[0018] In the fourth row from the top of Figure 3, two arrows indicating measurement locations are superimposed on the image area of ​​each pattern as a result of the measurement location detection process. One end of each arrow indicates the start point of the measurement location, and the other end indicates the end point of the measurement location. The linear distance connecting these start and end points is the measured distance. In this example, two arrows are shown as measurement locations, and measurements are performed at two locations for each pattern. That is, in this example, for each pattern indicated by a thick black rectangular frame, the vertical length and horizontal length within the frame are measured. As a result of the measurement location detection process, the information processing device 1 can obtain a measurement location image with indicators such as arrows indicating the measurement locations, or coordinate information for the start and end points of the measurement locations in the image area of ​​each pattern.

[0019] Next, the information processing device 1 performs a process of measuring the length of this location based on the measurement location image or coordinate information of the measurement location obtained as a result of the measurement location detection process. The information processing device 1 detects an indicator such as an arrow drawn in the measurement location image of each pattern obtained as a result of the measurement location detection process, calculates the length between both ends of the detected arrow, etc., and calculates the actual length of the measurement location by obtaining information such as the magnification at the time of image capture by the scanning electron microscope 102. The information processing device 1 can determine whether each pattern is normal or abnormal based on whether the length is within a predetermined range by comparing the measured length with predetermined thresholds such as upper and lower limits.

[0020] As described above, in the information processing system according to this embodiment, the information processing device 1 performs pattern detection processing, contour detection processing, and measurement point detection processing based on the SEM image of the substrate captured by the scanning electron microscope 102, and determines whether each pattern is normal. When the information processing device 1 performs these detection processing, the user must set information such as detection conditions for the information processing device 1. For example, in the pattern detection processing, the user sets, as a detection condition, which of the many patterns captured in the SEM image should be detected. Also, for example, in the contour detection processing, the user sets which pattern is the target for contour detection, i.e., which portion of the target pattern is desired to obtain a mask image, which is the result of the segmentation processing. Also, for example, in the measurement point detection processing, the user sets information regarding which portion of the target pattern should be the measurement point.

[0021] The information processing system according to the present embodiment is provided with a function for supporting the input of information, such as conditions related to the detection process, thereby facilitating user use of the system and improving convenience. The information processing device 1 receives input of information in natural language text from a user using a learning model, such as an LLM (Large Language Model), which has undergone machine learning in advance. The information processing device 1 determines conditions for the detection process based on the received information and performs the detection process according to the determined conditions. In addition to information input in natural language, the information processing device 1 also receives information input, for example, by specifying coordinates or an area on a displayed image, and performs the detection process. Hereinafter, in this embodiment, information input by a user in natural language and information input by specifying coordinates or an area on an image are referred to as information input by a "prompt," and the input information is referred to as a "prompt." However, the prompt may include information input other than the natural language, coordinate specification, and area specification described above.

[0022] In the information processing system according to this embodiment, information input via the above prompt is accepted in the three detection processes of pattern detection processing, contour detection processing, and measurement point detection processing performed by the information processing device 1. However, the information processing system only needs to be configured to accept information input via the above prompt in at least one of the three detection processes of pattern detection processing, contour detection processing, and measurement point detection processing.

[0023] <Device Configuration> Fig. 4 is a block diagram showing an example configuration of an information processing device 1 according to this embodiment. The information processing device 1 according to this embodiment can be realized by installing a predetermined application program or the like in a general-purpose information processing device such as a personal computer or a server computer. However, the information processing device 1 may also be a dedicated information processing device that controls the substrate processing device 101 or the scanning electron microscope 102. The information processing device 1 according to this embodiment is configured to include a processing unit 11, a storage unit 12, a communication unit 13, a display unit 14, an operation unit 15, etc. Note that, although this embodiment will be described assuming that processing is performed by a single information processing device 1, the processing of the information processing device 1 may be distributed among multiple devices.

[0024] The processing unit 11 is configured using an arithmetic processing device such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit) or a quantum processor, a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processing unit 11 reads and executes a program 12a stored in the storage unit 12, thereby performing various processes such as a process of detecting a pattern from an SEM image of a substrate, a process of detecting pixels onto which the detected pattern is imaged, and a process of detecting measurement points for the detected pattern.

[0025] The storage unit 12 is configured using a large-capacity storage device such as a hard disk or a solid-state drive (SSD). The storage unit 12 stores various programs executed by the processing unit 11 and various data required for the processing of the processing unit 11. In this embodiment, the storage unit 12 stores a program 12a executed by the processing unit 11. The storage unit 12 also includes a prompt storage unit 12b that stores and accumulates information on prompts input by the user when performing the detection process.

[0026] In this embodiment, the program (computer program, program product) 12a is provided in a form recorded on a recording medium 99 such as a memory card or an optical disc, and the information processing device 1 reads the program 12a from the recording medium 99 and stores it in the storage unit 12. However, the program 12a may also be written to the storage unit 12, for example, during the manufacturing stage of the information processing device 1. Alternatively, the program 12a may be distributed by a remote server device or the like and acquired by the information processing device 1 via communication. For example, the program 12a may be read from the recording medium 99 by a writing device and written to the storage unit 12 of the information processing device 1. The program 12a may be provided in a form distributed via a network or in a form recorded on the recording medium 99.

[0027] The prompt storage unit 12b stores prompt information input by the user in each detection process, i.e., the pattern detection process, the contour detection process, and the measurement point detection process. The prompt information includes natural language text information, or information such as coordinates or areas specified by the user on an image. The prompt storage unit 12b stores the input prompt information in association with various information, such as information indicating which detection process the prompt relates to, timestamp information such as the date and time when the prompt input was received, input information such as the image targeted in the detection process performed based on the prompt, and / or output information such as the image resulting from the detection process performed based on the prompt. When accepting prompt information input from the user during a detection process, the information processing device 1 reads past prompt information from the prompt storage unit 12b and displays a list, and can accept the input of prompt information by accepting a user selection from among the list.

[0028] The communication unit 13 is connected to the scanning electron microscope 102 via a cable such as a communication line or a signal line, and transmits and receives data to and from the scanning electron microscope 102 via this cable. In this embodiment, the communication unit 13 receives data of an SEM image of a substrate transmitted from the scanning electron microscope 102, and provides the received data to the processing unit 11. Note that in this embodiment, the SEM image is transmitted from the scanning electron microscope 102 to the information processing device 1 via communication, but this is not limiting, and the SEM image may also be transmitted and received via a recording medium such as a memory card.

[0029] The display unit 14 is configured using a liquid crystal display or the like, and displays various images, characters, etc. based on the processing of the processing unit 11. In this embodiment, the display unit 14 displays, for example, an SEM image acquired from the scanning electron microscope 102, and information relating to the results of each detection processing performed on this SEM image.

[0030] The operation unit 15 accepts user operations and notifies the processing unit 11 of the accepted operations. For example, the operation unit 15 accepts user operations using input devices such as mechanical buttons or a touch panel provided on the surface of the display unit 14. Furthermore, for example, the operation unit 15 may be input devices such as a mouse and a keyboard, and these input devices may be configured to be detachable from the information processing device 1.

[0031] The storage unit 12 may be an external storage device connected to the information processing device 1. The information processing device 1 may be a multi-computer including multiple computers, or may be a virtual machine virtually constructed by software. The information processing device 1 is not limited to the above configuration, and may not include, for example, the display unit 14 and the operation unit 15.

[0032] In the information processing device 1 according to this embodiment, the processing unit 11 reads out and executes the program 12a stored in the storage unit 12, whereby an image acquisition unit 11a, a pattern detection processing unit 11b, a contour detection processing unit 11c, a length measurement location detection processing unit 11d, a state determination unit 11e, a display processing unit 11f, etc. are realized as software functional units in the processing unit 11. Note that in this drawing, functional units related to detection processing based on SEM images are shown as functional units of the processing unit 11, and functional units related to other processing are not shown.

[0033] The image acquisition unit 11a performs processing to acquire SEM images taken by the scanning electron microscope 102 of a substrate that has been processed by the substrate processing apparatus 101. The image acquisition unit 11a acquires the SEM images of the substrate taken by the scanning electron microscope 102 by communicating with the scanning electron microscope 102 via the communication unit 13, and stores the acquired SEM images in the memory unit 12.

[0034] The pattern detection processing unit 11b performs processing to detect one or more patterns formed on the substrate from the SEM image of the substrate acquired by the image acquisition unit 11a. For example, the pattern detection processing unit 11b displays the SEM image acquired by the image acquisition unit 11a on the display unit 14 and acquires a partial image area from the SEM image as a template image by receiving a user's designation of the partial image area. The pattern detection processing unit 11b may also acquire a template image by receiving a user's selection from one or more template images stored in the prompt storage unit 12b. The pattern detection processing unit 11b detects one or more patterns from the SEM image by performing, for example, template matching processing, using structures or the like depicted in the acquired template image as patterns to be detected.

[0035] Template matching is a technique in which an image region of the same size as a template image is extracted from a target image (SEM image), and if the similarity between the extracted image and the template image exceeds a threshold, it is determined that the target pattern is captured in this image region. By moving the extraction position of the image region from the target image and repeating the same similarity determination, and extracting image regions and determining similarity at all positions in the target image, one or more patterns can be detected from the target image. Note that the method by which the pattern detection processing unit 11b detects patterns from an SEM image based on a template image is not limited to the template matching technique described above, and various other methods may be employed. For example, the pattern detection processing unit 11b can detect one or more patterns from the SEM image by extracting feature points from the template image, extracting feature points from the SEM image, and matching the feature points between the two images.

[0036] In this embodiment, when acquiring a template image containing a pattern to be detected, the pattern detection processing unit 11b can accept a prompt input from a user and acquire a template image corresponding to the input prompt. The prompt may be, for example, a sentence describing the shape characteristics of the pattern to be detected in natural language. Note that in this embodiment, the sentence accepted as a prompt does not need to include a subject, object, predicate, etc., and may include, for example, a single word such as "rectangle" or an incomplete sentence such as a list of multiple words. Furthermore, the prompt may be, for example, a region specification that specifies a region surrounding the pattern to be detected in the SEM image, the coordinate specification of one or more pixels included in the pattern to be detected, or the coordinate specification of one or more pixels not included in the pattern to be detected.

[0037] The pattern detection processing unit 11b uses a learning model such as an LLM that has been machine-learned in advance to acquire a template image corresponding to the input prompt and detects a pattern identical to the pattern captured in the template image from the SEM image. The pattern detection processing unit 11b, for example, extracts one or more image areas in which the detected pattern is captured from the SEM image and outputs the extracted one or more image areas as pattern detection results. Furthermore, for example, the pattern detection processing unit 11b may output coordinate information of a bounding box surrounding the pattern detected from the SEM image as the pattern detection result.

[0038] The contour detection processing unit 11c performs a process of detecting the contour of each pattern for one or more patterns detected from the SEM image by the pattern detection processing unit 11b. In this embodiment, the contour detection processing unit 11c detects the contour of the pattern by performing a so-called segmentation process. The contour detection processing unit 11c performs a segmentation process on the image region of each pattern detected by the pattern detection processing unit 11b using a general-purpose trained learning model that performs image segmentation, such as a Segment Anything Model (SAM), to detect the contour of the pattern captured in each image region. Based on the results of the segmentation process, the contour detection processing unit 11c creates a mask image that indicates which pixels in the image region contain the pattern, and outputs this mask image as the result of the contour detection process.

[0039] In this embodiment, the SAM used by the contour detection processing unit 11c for segmentation processing is a learning model that has undergone machine learning in advance to accept input of an image and a prompt and perform object segmentation on the input image in accordance with the prompt. The contour detection processing unit 11c can accept, for example, an operation from the user specifying a pixel included in the pattern to be detected (where the pattern to be detected is captured) or an operation specifying a pixel not included in the pattern to be detected (where the pattern to be detected is not captured), and use coordinate information in the specified image area as prompt information. The contour detection processing unit 11c may also accept, from the user, text information describing the characteristics of the target pattern in natural language as a prompt. The contour detection processing unit 11c inputs the image area obtained as a result of pattern detection and the prompt information received from the user to the SAM, and obtains a mask image of the segmentation result output by the SAM to detect the contour of the pattern from the image area.

[0040] Furthermore, when a plurality of image regions are extracted from the SEM image by the pattern detection process, the contour detection processing unit 11c performs contour detection for each of these plurality of image regions individually. In this embodiment, when performing contour detection for a plurality of image regions, the contour detection processing unit 11c accepts a prompt input from the user when performing contour detection for the first image region, and performs contour detection for the second and subsequent image regions based on the prompt received for the first image region and / or the segmentation result for the first image region without accepting individual prompt input for contour detection for the second and subsequent image regions.

[0041] The measurement point detection processing unit 11d performs processing to detect one or more measurement points for the pattern captured in each image area, based on the mask image of each pattern obtained as a result of contour detection by the contour detection processing unit 11c. The measurement point detection processing unit 11d outputs, as a result of the detection of the measurement points, for example, a measurement point image in which indicators such as arrows or lines indicating the positions of the measurement points for each pattern captured in the image area are drawn, or coordinate information of pixels in the image area that are the start and end points of the measurement.

[0042] In this embodiment, the measurement point detection processing unit 11d uses a measurement point detection model that has been machine-learned in advance to accept as input a mask image obtained as a result of the contour detection processing and prompt information such as text or coordinates, and output a measurement point image (or coordinate information of the measurement point) in which an indicator such as an arrow or a line indicating the measurement point is drawn for the input mask image. The measurement point detection model can be realized, for example, by changing a part of the SAM used in the above-mentioned segmentation processing (changing it so that instead of outputting a mask image, a measurement point image or coordinates of the measurement point, etc.)

[0043] The measurement point detection processing unit 11d receives, for example, sentence information describing measurement points for a pattern in natural language as a prompt from the user, inputs a mask image of the pattern contour detection result to the measurement point detection model, and detects measurement points by acquiring information such as the measurement point image or coordinates of the measurement points output by the measurement point detection model. Note that when multiple patterns are detected from an SEM image and multiple image regions and mask images are obtained, the measurement point detection processing unit 11d receives a prompt input from the user for the mask image of the first pattern and performs measurement point detection. The measurement point detection processing unit 11d does not receive individual prompt input for mask images of the second and subsequent patterns, but performs measurement point detection processing for the second and subsequent mask images based on the prompt received for the first mask image and / or the measurement point detection results for the first mask image.

[0044] Furthermore, instead of accepting a text prompt for the mask image of the first pattern, the measurement point detection processing unit 11d may superimpose and display a mask image corresponding to the image area where the pattern is captured, and accept from the user the coordinates of the measurement position for this image display as a prompt. In this case, since the measurement point detection processing unit 11d is given the measurement point for the mask image of the first pattern as a prompt, it is not necessary to detect the measurement point using the measurement point detection model for the first mask image. For the mask images of the second and subsequent patterns, the measurement point detection processing unit 11d performs measurement point detection processing based on the prompt input by the user (the measurement point detection result for the mask image of the first pattern).

[0045] The state determination unit 11e performs a process of determining the state of a pattern based on the measurement points detected by the measurement point detection processing unit 11d. The state determination unit 11e detects an indicator such as an arrow or a line from the measurement point image obtained as the measurement point detection result, and calculates the length of the arrow based on the coordinates of both ends of the arrow. Alternatively, the state determination unit 11e calculates the length (distance) between the start point and the end point based on coordinate information of the start point and the end point obtained as the measurement result. The state determination unit 11e calculates the actual length of the measurement point of the pattern based on the calculated distance in the image area and information such as the magnification at which the SEM image from which the image area was extracted was captured. The state determination unit 11e compares the calculated length with, for example, predetermined upper and lower limits, and determines the pattern as normal if the measured length is within the range of the upper and lower limits, and determines the pattern as abnormal if it is outside the range. Note that the state determination of each pattern by the state determination unit 11e is not limited to the above, and any method and state determination may be used.

[0046] The display processing unit 11f performs processing to display information such as the SEM image acquired from the scanning electron microscope 102, the detection results of the SEM image by the pattern detection processing unit 11b, the contour detection processing unit 11c, and the measurement point detection processing unit 11d, and the state determination results of each pattern by the state determination unit 11e, on the display unit 14. For example, the display processing unit 11f generates an image in which a bounding box surrounding each pattern detected by the pattern detection processing unit 11b is superimposed on the SEM image acquired from the scanning electron microscope 102, and displays the image on the display unit 14. Alternatively, for example, the display processing unit 11f superimposes an image area of ​​a pattern extracted from the SEM image based on the result of pattern detection on a mask image that is the result of contour detection for this image area, and displays the result of contour detection on the display unit 14. Alternatively, for example, the display processing unit 11f displays a measurement point image obtained as a result of the measurement point detection processing on the display unit 14, or displays the coordinates of the start point and end point obtained as a result of the characteristic point detection processing by superimposing indicators such as arrows or lines on the image of the contour detection result. Furthermore, for example, the display processing unit 11f displays on the display unit 14, for each pattern detected from the SEM image, the measured length of the measurement point and information indicating whether this length is normal or abnormal.

[0047] <Pattern Detection Processing> In the information processing system according to this embodiment, the scanning electron microscope 102 photographs a substrate that has been subjected to substrate processing such as etching by the substrate processing apparatus 101, and the photographed image (SEM image) obtained by this photographing is acquired by the information processing apparatus 1 to determine the state of a pattern formed on the substrate. To perform this state determination, the information processing apparatus 1 first performs pattern detection processing to detect multiple patterns formed on the substrate from the SEM image.

[0048] In the pattern detection process, the information processing device 1 displays an SEM image acquired from the scanning electron microscope 102 on the display unit 14 and prompts the user to input condition information related to the patterns to be detected, more specifically, information for acquiring a template image onto which the patterns to be detected are projected. In response, the user can, for example, perform an operation to specify an area on the displayed SEM image that surrounds one of the patterns to be detected. The information processing device 1 extracts the image area specified by the user's operation from the SEM image and sets the extracted image area as a template image onto which the patterns to be detected are projected.

[0049] Furthermore, in this embodiment, the information processing device 1 may accept input from the user of a sentence describing, for example, the shape of the pattern to be detected in natural language as condition information for acquiring a template image. The information processing device 1 extracts an area from the SEM image in which a pattern corresponding to the input natural language sentence is captured, and sets the extracted image area as the template image. The natural language accepted as input by the information processing device 1 from the user is, for example, a character string such as a word or a sentence in a language such as Japanese or English. To handle natural language, the information processing device 1 according to this embodiment uses an LLM that has undergone machine learning in advance to interpret natural language. The LLM may operate on the information processing device 1, or on a device different from the information processing device 1, such as a remote server device.

[0050] In the information processing system according to this embodiment, a user can input natural language such as "hole" or "mask" into the information processing device 1 to display candidate images depicting "hole" or "mask," and can determine the template image by selecting one of these candidates. The user can also add any modifier, such as "long, narrow hole" or "large mask," to cause the information processing device 1 to display candidate template images.

[0051] The information processing device 1 extracts feature points from an SEM image acquired from the scanning electron microscope 102 and creates multiple template image candidates by extracting appropriate image regions from the SEM image based on the extracted feature points. When multiple patterns of the same shape are repeatedly formed on a substrate, the multiple feature points that can be extracted from the SEM image may include areas where similar feature point arrangements are repeated. The information processing device 1 searches for such repeated arrangements of similar feature points from the entire set of feature points extracted from the SEM image. Based on the search results, the information processing device 1 selects one of the repeated feature point arrangements and extracts a rectangular image region containing the multiple feature points included in this arrangement to use as a template image candidate. The information processing device 1 may extract multiple candidates from a single SEM image. Note that the method of candidate extraction by the information processing device 1 is not limited to the above method, and any method may be employed.

[0052] The feature point extraction process is a process of identifying characteristic points (pixels) such as edges or corners contained in an image, and can be performed using existing methods such as SIFT (Scale Invariant Feature Transformation) or AKAZE (Accelerated-KAZE). Each feature point extracted from an image by the information processing device 1 includes information on the position of the feature point in the image (e.g., x, y coordinates) and a numerical value (feature amount) indicating the feature of this point.

[0053] The information processing device 1, which has extracted template image candidates from the SEM image, selects a candidate that corresponds to the input natural language from among the multiple candidates and displays a list of the selected candidates to present to the user. The information processing device 1 may also select a template image that corresponds to the input natural language from among the template images stored in the prompt storage unit 12b and display a list of the selected template images as candidates to present to the user. The information processing device 1 accepts, via the operation unit 15, a user operation to select one of the multiple candidates displayed in the list, and acquires the selected candidate as the template image.

[0054] The information processing device 1 converts, for example, a natural language input by a user into a feature vector using an LLM, and then converts multiple images serving as template image candidates into feature vectors using a learning model (encoder) that has undergone machine learning. The information processing device 1 calculates the similarity between the feature vector of the input natural language and the feature vector of each candidate, selects a predetermined number of candidates in descending order of similarity, and presents them to the user, thereby presenting candidates corresponding to the natural language input by the user. The learning model that converts images into feature vectors can be generated in advance by machine learning using training data that associates pre-collected template images with natural languages ​​that express the patterns of the template images (or feature vectors converted from the natural languages ​​using the LLM). Note that the method of presenting candidates corresponding to natural languages ​​to the user using the LLM described above is merely an example and is not limited thereto; any other method may be employed.

[0055] In this embodiment, condition information for acquiring input from a user regarding a pattern to be detected, such as the above-mentioned area specification information and natural language sentence information, is called a prompt, and the information processing device 1 stores and accumulates information regarding the input prompt in the prompt storage unit 12b. The information processing device 1 stores, in the prompt storage unit 12b, information indicating that the prompt is related to the pattern detection process, timestamp information such as the date and time when the prompt input was received, and information such as a template image extracted from an SEM image based on the prompt, along with a prompt (area specification or sentence, etc.) input during the pattern detection process, for example. The information processing device 1 can acquire a template image from an SEM image and can also acquire a template image stored in the prompt storage unit 12b, and the user may be able to select which method to use to acquire the template image.

[0056] The information processing device 1 acquires a template image on which a detection target pattern is captured, and detects from the SEM image a pattern identical to the pattern captured in the template image. The information processing device 1 detects one or more identical patterns captured in the SEM image, for example, based on an existing template matching technique. The information processing device 1 may also detect one or more patterns from the SEM image by, for example, extracting feature points from the template image and from the SEM image and matching the feature points between both images. Note that the method by which the information processing device 1 detects a pattern identical to the template image from the SEM image is not limited to the above-mentioned template matching or feature point matching, and any method may be employed.

[0057] The information processing device 1 detects one or more patterns in the SEM image by the pattern detection process, and extracts from the SEM image the image area in which each of the detected patterns is captured. The image area of ​​each pattern extracted from the SEM image becomes input information for the subsequent contour detection process.

[0058] In addition, when a template image is extracted from an SEM image based on a prompt containing text information entered by a user in a pattern detection process, the template image extraction process can be considered to be a pattern detection process for detecting a first pattern from an SEM image. The process for detecting one or more patterns from an SEM image based on a template image can be considered to be a second or subsequent pattern detection process using the results of the first pattern detection. Furthermore, when a template image is extracted by a user specifying an area on an SEM image, the information on the user's area specification can be considered to be a prompt input for the pattern detection process, and the extracted template image can be considered to be the detection result of the first pattern.

[0059] 5 is a flowchart showing an example of the procedure of the pattern detection process performed by the information processing device 1 according to this embodiment. The image acquisition unit 11a of the processing unit 11 of the information processing device 1 according to this embodiment communicates with the scanning electron microscope 102 via the communication unit 13 and acquires an SEM image of the target substrate photographed by the scanning electron microscope 102 (step S1). The display processing unit 11f of the processing unit 11 displays the SEM image acquired by the image acquisition unit 11a in step S1 on the display unit 14 (step S2). The pattern detection processing unit 11b of the processing unit 11 accepts input of a prompt (text information, area designation information, etc.) related to the pattern to be detected by accepting a user operation via the operation unit 15 (step S3).

[0060] The pattern detection processing unit 11b determines whether the prompt received in step S3 is text information (step S4). If the prompt is text information (S4: YES), the pattern detection processing unit 11b uses a learning model such as LLM based on the received text information to extract multiple image regions that match the characteristics of the text information from the SEM image acquired in step S1, thereby extracting template image candidates (step S5). The display processing unit 11f displays the template image candidates extracted in step S5 on the display unit 14 (step S6). By receiving a user's operation via the operation unit 15, the pattern detection processing unit 11b accepts selection of a template image candidate from the multiple candidates displayed on the display unit 14 (step S7), and proceeds to step S9.

[0061] If the prompt received in step S3 is not text information (S4: NO), the pattern detection processing unit 11b extracts the specified area from the SEM image acquired in step S1 as a template image based on the area specification information received as the prompt (step S8), and proceeds to step S9.

[0062] The pattern detection processing unit 11b stores the prompt received in step S3 and the template image extracted based on this prompt in the prompt storage unit 12b, along with information such as information indicating that the prompt is related to the pattern detection process and timestamp information, such as the date and time when the prompt was received (step S9). The pattern detection processing unit 11b performs processing such as template matching or feature point matching based on the obtained template image to detect one or more patterns that match the template image from the SEM image (step S10). For one or more patterns detected in step S10, the pattern detection processing unit 11b extracts image areas in which the patterns are captured from the SEM image (step S11). The display processing unit 11f displays the image areas of the one or more patterns extracted in step S11 on the display unit 14 (step S12), and the processing ends.

[0063] <Contour Detection Processing> After acquiring multiple image areas in which patterns are captured from an SEM image through pattern detection processing, the information processing device 1 performs processing to detect the contours of the patterns in each image area. The information processing device 1 according to this embodiment detects the contours of the patterns in each image area by performing a process to detect pixels in which the pattern is captured in the image area, that is, a so-called segmentation process. The information processing device 1 uses a trained contour detection model that has been machine-learned in advance to perform the contour detection processing on each of the multiple image areas obtained as a result of the pattern detection processing.

[0064] 6 is a schematic diagram showing an example of the configuration of a contour detection model according to this embodiment. The contour detection model 200 used in the contour detection process by the information processing device 1 according to this embodiment is a learning model that receives as input an input image and a prompt containing various information related to the object of segmentation, and outputs mask information indicating which pixels in the input image contain the object as a contour detection result. For example, an existing learning model that performs segmentation, such as SAM, is used as the contour detection model 200.

[0065] The prompt input to the contour detection model 200 may include, for example, a mask image, coordinate information, area designation, and text information. However, in this embodiment, a mask image need not be input as a prompt. The coordinate information input to the contour detection model 200 is, for example, information specifying the coordinates of one of multiple pixels included in the object captured in the input image, or information specifying the coordinates of one of multiple pixels not included in the object captured in the input image. The area designation is, for example, information such as the coordinates of a box (rectangular frame) surrounding the object to be segmented, for designating the area in the input image in which the object is captured. The text information is information such as sentences or words describing the shape, color, etc. of the object to be segmented in natural language. The mask image input as a prompt is, for example, information in which a portion of the object to be segmented in the input image is filled in, i.e., information specifying multiple pixels in the input image in which the object is captured.

[0066] The contour detection model 200 according to this embodiment is configured to include an image encoder 201, a mask encoder 202, a prompt encoder 203, and a mask decoder 204. The image encoder 201 converts an input image into features. The mask encoder 202 converts a mask image input as a prompt into features. The features output by the image encoder 201 and the features output by the mask encoder 202 are combined and input to the mask decoder 204. The prompt encoder 203 converts coordinate information, area designation, or text information input as a prompt into features. The mask decoder 204 generates a mask image that is the result of segmentation of the input image, based on the features combined from the outputs of the image encoder 201 and the mask encoder 202 and the features output by the prompt encoder 203.

[0067] 7 is a schematic diagram illustrating an example of the contour detection process. The first image from the left in FIG. 7 is an example of an input image for the contour detection process, and is an example of an image area in which a pattern detected by the information processing device 1 through the pattern detection process is captured. In this example, the process is performed with the objective of detecting the contour of a pattern represented by a thick black rectangular frame captured in this input image. The information processing device 1 selects an appropriate one of the multiple image areas of the pattern obtained as a result of the pattern detection process, displays it on the display unit 14, and accepts a prompt input from the user.

[0068] In this example, the user inputs a prompt specifying the coordinates of a point near the left edge of a rectangular frame, which is the detection target. The information processing device 1 accepts the user's input of the prompt, inputs the image area of ​​the displayed pattern and the accepted prompt to the contour detection model 200, and acquires a mask image output by the contour detection model 200 in response to the input. The image of detection result 1 shown second from the left in FIG. 7 is an image displayed by the information processing device 1 on the display unit 14, with the mask image resulting from the processing superimposed on the image area of ​​the original pattern. In this example, detection result 1 includes the interior of the target rectangular frame pattern in the result of the segmentation process (contour detection process). However, the user in this example aims to obtain a pattern that does not include the interior of the rectangular frame as the result of the segmentation process, and therefore detection result 1 is different from the result desired by the user.

[0069] In such cases, the user can correct the detection results by inputting additional prompts. In this example, the detection results are corrected by inputting an additional prompt specifying the coordinates of a pixel that is not included in the detection target. The image of detection result 2 shown third from the left in Figure 7 is a detection result corrected by inputting an additional prompt, and a point near the center of the area inside the rectangular frame is specified as the additional prompt for correction. The user can repeatedly correct the detection results obtained by the information processing device 1, and if the initial detection results are as desired, the user does not need to make any corrections.

[0070] The information processing device 1 stores, in the prompt storage unit 12b, information such as prompts input by the user and mask images obtained by contour detection based on the prompts. When inputting a prompt, the user can cause the information processing device 1 to display a list of the prompts and information such as mask images stored in the prompt storage unit 12b, and select one or more prompts from the stored past prompts to input the current prompt.

[0071] The information processing device 1 performs segmentation processing on one of the multiple image regions obtained as a result of the pattern detection processing using the contour detection model 200 based on a prompt input by the user, and outputs a mask image obtained as a result of this processing as a contour detection result. The information processing device 1 according to this embodiment performs contour detection processing on the other image regions by reflecting the prompt and contour detection result for the first image region on the other image regions. In other words, the information processing device 1 according to this embodiment can perform contour detection processing on the second and subsequent image regions of the multiple image regions obtained as a result of the pattern detection processing based on the prompt acquired for the first image region and the mask image obtained as a result of contour detection for the first image region, without having to receive a new prompt input from the user.

[0072] The method of detecting contours for the second and subsequent image regions by reflecting the prompt and contour detection results for the first image region can be any of the following three methods (1) to (3): (1) A method that uses a contour detection model that inputs the contour detection results of a similar image; (2) A method that generates a prompt based on feature quantities; and (3) A method that generates a rule-based prompt.

[0073] (1) Method using a contour detection model that inputs contour detection results from a similar image: FIG. 8 is a schematic diagram showing an example of the configuration of a similar image contour detection model. The similar image contour detection model 210 used by the information processing device 1 according to this embodiment for contour detection processing on second and subsequent image regions is a trained learning model that has undergone machine learning in advance to accept a reference image, a contour detection result from the reference image, a target image, and a filled-in image as input, and output a mask image that serves as the contour detection result for the target image. The reference image and the contour detection result from the reference image input to the similar image contour detection model 210 correspond to the first image region and the mask image obtained for this image region using the contour detection model 200 in this embodiment. The target image input to the similar image contour detection model 210 corresponds to the second and subsequent image regions to be detected in this embodiment.

[0074] In this embodiment, the filled-in image input to the similar image contour detection model 210 is an image in which the entire target image is filled in. The filled-in image is input information required when the similar image contour detection model 210 is trained, and when inference is made using the similar image contour detection model 210, the information processing device 1 generates a filled-in image based on the target image and inputs the image to the similar image contour detection model 210.

[0075] 9 is a schematic diagram illustrating an example of a learning method for the similar image contour detection model 210. For the machine learning of the similar image contour detection model 210, learning data is prepared in advance, which associates an image of an object to be subjected to contour detection processing (segmentation processing) with a mask image that is the contour detection result for the image. The information processing device 1 appropriately extracts two pairs from multiple pairs of an object image and a mask image, and generates a filled-in image by filling in a portion of the mask image. In the illustrated example, the filled-in image is generated by filling in three areas of the mask image with rectangular regions, but this is just an example, and the filled-in image may be generated by any method.

[0076] For the four pieces of input information of the similar image contour detection model 210, the information processing device 1 associates the input reference image as the first object image, the contour detection result of the reference image as a filled-in image generated based on the first mask image, the target image as the second object image, and the filled-in image as a filled-in image generated based on the second mask image. The information processing device 1 associates the contour detection result, which is output information of the similar image contour detection model 210, with the second mask image and performs so-called supervised machine learning using this as a correct value for the input information, thereby performing machine learning of the similar image contour detection model 210. Note that the machine learning process of the similar image contour detection model 210 does not have to be performed by the information processing device 1, but may be performed by a device separate from the information processing device 1, such as a server device.

[0077] The information processing device 1 repeatedly performs contour detection processing on the second and subsequent image regions using the trained similar image contour detection model 210, and acquires a mask image that is the result of contour detection for each image region. In this repetition, the information processing device 1 may always use the first image region and its mask image as the reference image and its contour detection result to be input to the similar image contour detection model 210, or may use the image region and its mask image used in the previous contour detection processing.

[0078] (2) Method of Generating Prompt Based on Feature Amounts The information processing device 1 generates a prompt to be used in the contour detection process of the target image based on the feature amounts of the reference image (first image area) and the feature amounts of the target image (second and subsequent image areas). As a result, the information processing device 1 can obtain a mask image as the contour detection process of the target image based on the generated prompt and the contour detection model 200 shown in FIG.

[0079] The information processing device 1 converts the reference image (first image region) and the target image (second and subsequent image regions) into feature information, for example, using an image encoder. The information obtained by this conversion is, for example, information in which a feature is assigned to each pixel in the image. The information processing device 1 performs a process of identifying similar pixels between the reference image and the target image by comparing the feature values ​​of the reference image and the target image, a so-called feature point matching process. Based on the results of the matching process, the information processing device 1 determines which pixels in the target image correspond to the masked pixels in the mask image for the reference image, and generates a prompt (coordinate specification, area specification, etc.) that causes the contour detection model 200 to detect the corresponding pixels in the target image.

[0080] The information processing device 1 can generate the above-described prompt and generate a mask image based on the generated prompt by using, for example, an existing learning model called Matcher, which is a derivative technology of SAM. Matcher is a learning model that performs segmentation by matching features, but because it is an existing technology, detailed description will be omitted.

[0081] (3) Method of generating prompts based on rules The information processing device 1 generates prompts for target images (second and subsequent image areas) in accordance with predetermined rules, based on a prompt input by a user when performing contour detection processing on a reference image (first image area). Possible rules for generating prompts include, but are not limited to, the following:

[0082] If the prompt entered by the user for the reference image is text information, the information processing device 1 uses this text information as the prompt for the target image. If the prompt is a coordinate specification or an area specification, the information processing device 1 converts the coordinates of the prompt for the reference image into the coordinates of the target image, assuming that the target image is the same size as the reference image. If the target image and the reference image are the same size, the information processing device 1 can use the coordinate specification and area specification prompt as the prompt for the target image.

[0083] Regardless of which of the above methods (1) to (3) is used to perform contour detection on the second and subsequent image regions, the information processing device 1 displays, on the display unit 14, a list of images superimposed with a mask image corresponding to each image region as contour detection results for the second and subsequent image regions. In this case, the information processing device 1 may display a list of both the contour detection results for the first image region and the contour detection results for the second and subsequent image regions. The user can select one of the displayed contour detection results and modify the contour detection result. The user may perform an operation such as specifying the coordinates of a pixel included in a pattern, specifying the coordinates of a pixel not included in a pattern, or inputting the correction as text. Upon receiving this operation, the information processing device 1 adds the entered correction information to a prompt and re-executes the contour detection process described above, thereby reflecting the user's correction to one image region in other image regions.

[0084] 10 is a flowchart showing an example of the procedure of the contour detection process performed by the information processing device 1 according to this embodiment. The contour detection processing unit 11c of the processing unit 11 of the information processing device 1 according to this embodiment acquires an image area of ​​the detected pattern as a result of the pattern detection from the SEM image by the pattern detection processing unit 11b (step S21). The contour detection processing unit 11c appropriately selects one image area from the multiple image areas acquired in step S21 (step S22). The display processing unit 11f of the processing unit 11 displays the image area selected in step S22 on the display unit 14 (step S23).

[0085] The contour detection processing unit 11c receives an operation on the operation unit 15, thereby receiving from the user an input of a prompt (information such as text information, coordinate specification, or area specification) that serves as a condition for contour detection for the image area of ​​the pattern displayed in step S23 (step S24). The contour detection processing unit 11c inputs the image area selected in step S22 and the prompt information received in step S24 to the contour detection model 200 shown in Fig. 6 (step S25). The contour detection processing unit 11c obtains a mask image output by the contour detection model 200 as a result of contour detection in response to the information input in step S25, and information indicating which pixels contain the pattern (step S26).

[0086] The contour detection processing unit 11c selects one image area from among the multiple image areas acquired in step S21 that have not been selected (step S27). The contour detection processing unit 11c sets the image area selected in step S27 as the target image, the image area selected in step S22 as the reference image, and the mask image acquired in step S26 as the contour detection result of the reference image, and inputs these to the similar image contour detection model 210 shown in FIG. 8 (step S28). At this time, the contour detection processing unit 11c sets the filled-in image to be input to the similar image contour detection model 210 as an image in which the entire target image is filled-in. The contour detection processing unit 11c acquires the mask image output by the similar image contour detection model 210 as the contour detection result in response to the information input in step S28 (step S29).

[0087] The contour detection processing unit 11c determines whether contour detection processing has been completed for all image regions acquired in step S21 (step S30). If processing has not been completed for all image regions (S30: NO), the contour detection processing unit 11c returns to step S27, selects one of the unprocessed image regions, and repeats contour detection. If processing has been completed for all image regions (S30: YES), the display processing unit 11f displays, on the display unit 14, an image in which a mask image is superimposed on each image region as a result of the contour detection processing (step S31). The contour detection processing unit 11c stores, for example, the image region selected in step S22, the prompt received in step S24, and the mask image acquired in step S26, together with information indicating that the image is related to contour detection processing and timestamp information, such as the date and time when the prompt input was received, in the prompt storage unit 12b (step S32), and then terminates processing.

[0088] <Measurement Point Detection Process> After acquiring mask information indicating which pixels each pattern is captured from the image area where each pattern is captured by the contour detection process, the information processing device 1 performs a process of detecting measurement points for the pattern in each image area. The information processing device 1 according to this embodiment acquires multiple pairs of image areas and mask images related to the patterns, selects an appropriate pair from these multiple pairs, and displays a superimposed image area and mask image of the selected pair on the display unit 14. The user inputs a prompt for specifying measurement points for the displayed image.

[0089] For example, when a user inputs text information describing the characteristics of measurement points in natural language as a prompt, the information processing device 1 uses a trained measurement point detection model that has been machine-learned in advance to detect measurement points of this pattern based on a mask image of the target pattern and the input text information. Fig. 11 is a schematic diagram showing an example configuration of a measurement point detection model based on text information. The measurement point detection model 220 used by the information processing device 1 according to this embodiment in the measurement point detection process is a learning model that accepts an input image and a prompt as input and outputs, as a detection result, a measurement point image on which indicators such as arrows or lines indicating measurement points for the input image are drawn. In this embodiment, the prompt input to this measurement point detection model 220 is text information, and the input image is a mask image of the pattern.

[0090] The measurement point detection model 220 according to this embodiment is configured to include an image encoder 221, a prompt encoder 222, and a measurement point decoder 223. The image encoder 221 performs processing to convert an input image into a feature quantity. The prompt encoder 222 performs processing to convert text information input as a prompt into a feature quantity. The measurement point decoder 223 performs processing to generate a measurement point image, which is the result of detecting measurement points for the input image, based on the feature quantity output by the image encoder 221 and the feature quantity output by the prompt encoder 222.

[0091] 6 (or an existing SAM learning model, etc.) with the measurement point decoder 223 by re-learning, etc. The measurement point detection model 220 can be generated by performing so-called supervised machine learning and updating the parameters of the measurement point decoder 223 using learning data that associates a mask image of the pattern contour detection result, text information related to the measurement points, and a measurement point image on which an indicator such as an arrow is drawn as the detection result of the measurement point for the mask image.

[0092] The information processing device 1 detects measurement points by inputting the mask image obtained as a result of the contour detection process for each pattern and text information input as a prompt from the user to the measurement point detection model 220, and acquiring the measurement point image output by the measurement point detection model 220 in response to this. Note that the information processing device 1 may superimpose the measurement point image on the image area and mask image of each pattern as a result of the detection of the measurement points on the display unit 14, accept corrections to the prompt from the user, and perform the measurement point detection process again based on the accepted prompt.

[0093] Furthermore, when the user inputs, as a prompt, coordinate information or the like that actually specifies the start and end points of the measurement point, the information processing device 1 can treat the input coordinate information or the like as a prompt and also treat it as the detection result of the measurement point of the pattern displayed in the first image area.

[0094] The information processing device 1 stores the prompts input by the user and information on measurement points based on these prompts in the prompt storage unit 12b. When inputting a prompt related to a measurement point, the user can cause the information processing device 1 to display a list of information on the prompts and measurement points stored in the prompt storage unit 12b, and select one or more prompts or measurement points from the stored past prompts and measurement points, thereby inputting the current prompt.

[0095] The information processing device 1 performs measurement point detection processing using the measurement point detection model 220 based on one mask image out of the multiple mask images obtained as a result of the contour detection processing and a prompt of text information input by the user, and outputs the measurement point image obtained from the measurement point detection model 220 as the measurement point detection result. Furthermore, when coordinate information of the measurement point is input by the user as the prompt, the information processing device 1 generates an image based on the input coordinate information, in which an indicator such as an arrow or a line connecting the coordinates of the start point and the coordinates of the end point is drawn against a black or white background or the like of the same size as the corresponding pattern, and sets this image as the measurement point detection result.

[0096] When text information is input as a prompt, the information processing device 1 displays the measurement point detection results based on this prompt on the display unit 14 and can accept a correction operation for this detection result from the user. At this time, the user may input the correction point as text information, or may input coordinate information of the measurement point. When the information processing device 1 accepts input of text information as correction information, it adds text information related to the correction to the prompt of the already acquired text information and re-detects the measurement point using the measurement point detection model 220, thereby correcting the detection result of the measurement point. When the information processing device 1 accepts input of coordinate information of the measurement point as correction information, it can use this coordinate information as a new prompt and measurement point detection result.

[0097] The information processing device 1 according to this embodiment performs measurement point detection processing for the other mask images by reflecting the measurement point detection result for the mask image of the first pattern in the other mask images. That is, the information processing device 1 according to this embodiment can perform measurement point detection processing for the second and subsequent image areas of the multiple image areas obtained as a result of the pattern detection processing, based on the measurement point image obtained as the measurement point detection result for the mask image of the pattern captured in the first image area, without having to accept a new prompt input from the user.

[0098] In this embodiment, the information processing device 1 detects measurement points in mask images of second and subsequent patterns using a measurement point detection model that receives measurement point detection results from a similar image. FIG. 12 is a schematic diagram showing an example of the configuration of the similar image measurement point detection model. The similar image measurement point detection model 230 used by the information processing device 1 according to this embodiment for measurement point detection processing on second and subsequent image regions is a trained learning model that has undergone machine learning in advance to accept a reference image, measurement point detection results from the reference image, a target image, and a filled-in image as input, and output a measurement point image that serves as the measurement point detection result for the target image. The reference image and measurement point detection results input to the similar image measurement point detection model 230 correspond to the mask image of the first pattern and the measurement point image obtained from this mask image using the measurement point detection model 220 in this embodiment. The target image input to the similar image measurement point detection model 230 corresponds to the mask image of the second and subsequent patterns to be detected in this embodiment.

[0099] In this embodiment, the filled-in image input to the similar image measurement portion detection model 230 is an image in which the entire target image is filled in. The filled-in image is input information required when learning the similar image measurement portion detection model 230, and when inference is made using the similar image measurement portion detection model 230, the information processing device 1 generates an image in which the target image is filled in based on the target image and inputs the image to the similar image measurement portion detection model 230.

[0100] 13 is a schematic diagram for explaining an example of a learning method for the similar image measurement point detection model 230. For the machine learning of the similar image measurement point detection model 230, learning data is prepared in advance, which associates a mask image obtained as a result of contour detection of a pattern to be subjected to measurement point detection processing with a measurement point image that is the measurement point detection result for this image. The information processing device 1 appropriately extracts two pairs from multiple pairs of mask images and measurement point images, and generates a filled-in image by filling in part of the measurement point image. In the illustrated example, the filled-in image is generated by filling in two parts of the measurement point image with rectangular areas, but this is just one example, and the filled-in image may be generated by any method.

[0101] For the four pieces of input information of the similar image measurement point detection model 230, the information processing device 1 corresponds the input reference image as a first mask image, the measurement point detection result of the reference image as a filled-in image generated based on the first measurement point image, the target image as a second mask image, and the filled-in image as a filled-in image generated based on the second measurement point image. The information processing device 1 corresponds the measurement point detection result, which is output information of the similar image measurement point detection model 230, to the second mask image and performs so-called supervised machine learning using this as a correct value for the input information, thereby enabling machine learning of the similar image measurement point detection model 230. Note that the machine learning process of the similar image measurement point detection model 230 does not have to be performed by the information processing device 1, but may be performed by a device separate from the information processing device 1, such as a server device.

[0102] The information processing device 1 repeatedly performs measurement point detection processing on mask images of the second and subsequent patterns using the trained similar image measurement point detection model 230, and acquires measurement point images that are the result of measurement point detection for each mask image. In this repetition, the information processing device 1 may always use the first mask image and its measurement point image as the reference image and its measurement point detection result to be input to the similar image measurement point detection model 230, or may use the mask image and its measurement point image used in the previous measurement point detection processing.

[0103] The information processing device 1 displays a list of multiple measurement location images on the display unit 14 as measurement location detection results for the second and subsequent image areas. At this time, the information processing device 1 may display a list of both the measurement location image for the mask image of the first pattern and the measurement location images for the mask images of the second and subsequent patterns. The user can select one of the multiple measurement location images displayed in the list and correct the measurement location detection results. The user may, for example, input the correction location as text information or may make corrections by inputting coordinate information of the measurement location. Upon receiving a correction operation from the user, the information processing device 1 adds the input correction information to a prompt and re-executes the above-described measurement location detection process, thereby reflecting the user's correction to one measurement location image in other measurement location images.

[0104] 14 is a flowchart showing an example of the procedure of the measurement point detection process performed by the information processing device 1 according to this embodiment. The measurement point detection processing unit 11d of the processing unit 11 of the information processing device 1 according to this embodiment acquires a plurality of mask images corresponding to a plurality of patterns as the result of contour detection from the image area of ​​the patterns by the contour detection processing unit 11c (step S41). The measurement point detection processing unit 11d appropriately selects one mask image from the plurality of mask images acquired in step S41 (step S42). The display processing unit 11f of the processing unit 11 displays the mask image selected in step S42 on the display unit 14 by superimposing it on the image area of ​​the pattern corresponding to this mask image (step S43).

[0105] The measurement point detection processing unit 11d accepts an operation on the operation unit 15 to accept input from the user of a prompt (information such as text information, coordinate specification, or area specification) that serves as a condition for detecting measurement points in the image area and mask image displayed in step S43 (step S44). The measurement point detection processing unit 11d determines whether the prompt accepted in step S44 is text information (step S45). If the prompt is text information (S45: YES), the measurement point detection processing unit 11d detects measurement points using the measurement point detection model 220 shown in FIG. 11 based on the text information (step S46), and proceeds to step S48. Note that in step S46, the measurement point detection processing unit 11d uses the mask image selected in step S42 as an input image, inputs the text information accepted in step S44 as a prompt to the measurement point detection model 220, and acquires the measurement point image output by the measurement point detection model 220, thereby performing measurement point detection processing.

[0106] If the prompt is not text information (S45: NO), the measurement point detection processing unit 11d proceeds to step S48, since the prompt received in step S44 is coordinate information or the like of the measurement point specified by the user, and treats this prompt as the detection result of the measurement point (step S47).

[0107] The measurement portion detection processing unit 11d selects one mask image from among the multiple mask images acquired in step S41 that have not been selected (step S48). Based on the measurement portion detection result for the mask image selected in step S42, the measurement portion detection processing unit 11d performs measurement portion detection for the mask image selected in step S48 using the similar image measurement portion detection model 230 shown in Fig. 12 (step S49). At this time, the measurement portion detection processing unit 11d uses the mask image selected in step S48 as the target image, the mask image selected in step S42 as the reference image, and inputs the measurement portion image acquired in step S46 or the measurement portion image generated based on the measurement portion detection result of step S47 as the measurement portion detection result for the reference image to the similar image measurement portion detection model 230, and can detect measurement portions by acquiring the measurement portion image output by the similar image measurement portion detection model 230 as the measurement portion detection result. At this time, the length measurement portion detection processing unit 11d sets the filled-in image to be input to the similar image length measurement portion detection model 230 as an image in which the entire target image is filled in.

[0108] The measurement point detection processing unit 11d determines whether the measurement point detection process has been completed for all mask images acquired in step S41 (step S50). If the process has not been completed for all mask images (S50: NO), the measurement point detection processing unit 11d returns to step S48, selects one of the unprocessed mask images, and repeats the measurement point detection process. If the process has been completed for all mask images (S50: YES), the display processing unit 11f displays the measurement point image obtained as a result of the measurement point detection process on the display unit 14 (step S51). The measurement point detection processing unit 11d stores, for example, the mask image selected in step S42, the prompt received in step S44, and the measurement point image obtained as a result of the measurement point detection process in step S46, together with information indicating that the measurement point detection process is related to the measurement point detection process and timestamp information such as the date and time when the prompt input was received, in the prompt storage unit 12b (step S52), and then terminates the process.

[0109] (Modification) Furthermore, the information processing device 1 may perform the process of detecting measurement points repeatedly for the second and subsequent mask images by utilizing the measurement point detection model 220 shown in Fig. 11 instead of the method using the similar image measurement point detection model 230 shown in Fig. 12. Suppose that the user inputs text information such as "I want to measure the width 100 pixels below the upper inner contour" as a prompt for specifying a measurement point for the mask image of the first pattern. In this case, the information processing device 1 inputs the mask image and the text information of the prompt input by the user to the measurement point detection model 220, and acquires the measurement point image output by the measurement point detection model 220.

[0110] Next, the information processing device 1 according to the modified example uses the same text information as a prompt for the second and subsequent mask images, inputs the mask images and the same text information to the length measurement portion detection model 220, and acquires the length measurement portion images output by the length measurement portion detection model 220. By repeating the same process for the second and subsequent mask images, the information processing device 1 according to the modified example can obtain the detection results of the length measurement portions for each pattern.

[0111] <State Determination Process> The information processing device 1 according to this embodiment performs a process for determining the state of each pattern detected from the SEM image, i.e., whether each pattern is normal or abnormal, based on the result of the measurement point detection process. Based on the result of the measurement point detection process, the information processing device 1 acquires the coordinates of the start and end points of the measurement point in the partial image in which each pattern is captured, and calculates the length of the measurement point in the image from these coordinates. The information processing device 1 stores in advance information such as the magnification used when the scanning electron microscope 102 captured an image of the substrate, and calculates the actual length of each pattern based on the calculated length of the measurement point and the magnification used at the time of capture.

[0112] After calculating the actual length at the measurement point of each pattern, the information processing device 1 determines whether the calculated length is within a predetermined normal range, thereby determining the state of each pattern. The information processing device 1 may perform the state determination based on a comparison with thresholds such as upper and lower limits preset by the user, or may determine a threshold based on statistical values ​​such as the average and variance of lengths calculated for multiple patterns, and perform the state determination based on a comparison with the determined threshold, or may determine the threshold by other methods.

[0113] 15 is a flowchart showing an example of the procedure of the state determination process performed by the information processing device 1 according to this embodiment. The state determination unit 11e of the processing unit 11 of the information processing device 1 according to this embodiment acquires the results of the detection of the measurement points by the measurement point detection processing unit 11d, i.e., multiple measurement point images corresponding to multiple image regions extracted from the SEM image (step S61). The state determination unit 11e appropriately selects one measurement point image from the multiple measurement point images acquired in step S61 (step S62). Based on the measurement point image selected in step S62, the state determination unit 11e measures the length of an indicator such as an arrow or a line drawn in the measurement point image, and calculates the actual length based on the magnification at the time of capturing the SEM image, thereby measuring the length of the pattern (step S63).

[0114] The state determination unit 11e determines the state of this pattern by comparing the length measured in step S63 with predetermined upper and lower limits, etc., and determining whether the measured length is within a normal range (step S64).The state determination unit 11e stores the result of the state determination performed in step S64 in the storage unit 12 (step S65).

[0115] The state determination unit 11e determines whether the state determination process has been completed for all patterns extracted from the SEM image (step S66). If the process has not been completed for all patterns (S66: NO), the state determination unit 11e returns to step S62, selects one of the unprocessed measurement location images, and repeats the state determination. If the process has been completed for all patterns (S66: YES), the display processing unit 11f displays the state determination results on the display unit 14 (step S67), for example, by displaying a list of the determination results as to whether the multiple patterns detected from the SEM image are normal or abnormal, and then ends the process.

[0116] <Summary> In the information processing system according to the present embodiment, configured as described above, the information processing device 1 performs detection processing for a predetermined pattern formed on one or more captured images of a substrate that has been subjected to substrate processing such as etching by the substrate processing device 101, captured by the scanning electron microscope 102. The information processing device 1 performs a pattern detection processing for detecting an image area in which the pattern is captured from the captured image, a contour detection processing for detecting the contour of the pattern by segmentation processing that detects pixels in which the pattern is captured in the image area, and a measurement point detection processing for detecting a measurement point for the pattern based on the results of the contour detection. In at least one of the pattern detection processing, contour detection processing, and measurement point detection processing, the information processing device 1 according to the present embodiment acquires, as a prompt, condition information including at least one of coordinate information, area designation, and text information that constitutes a detection condition, and performs detection processing for another pattern based on the prompt acquired for one pattern. As a result, the information processing system according to the present embodiment reduces the frequency and amount of user input required for conditions, etc., in the detection processing for multiple patterns of predetermined shapes formed on the substrate, and is expected to assist the user in performing the detection processing.

[0117] Furthermore, in the information processing system according to this embodiment, the information processing device 1 performs contour detection processing for a pattern of a predetermined shape formed on one or more captured images of a substrate that has been processed by the substrate processing device 101, captured by the scanning electron microscope 102. The information processing device 1 acquires an image area in which one pattern extracted from the captured image is captured, acquires condition information as a prompt including at least one of coordinate information, area designation, and text information that serve as conditions for detection, and performs contour detection processing for another pattern based on the prompt acquired for one pattern. As a result, the information processing system according to this embodiment is expected to support the user's work in performing contour detection processing, since it reduces the frequency and amount of input of conditions, etc., required by the user in the contour detection processing for a predetermined pattern formed on the substrate.

[0118] In the information processing system according to the present embodiment, the information processing device 1 inputs an image region and a prompt into a trained first segmentation model (contour detection model 200) that receives an image and a prompt as input and classifies (segments) the multiple pixels that make up the image, thereby detecting pixels in which a pattern is captured in the image region, i.e., detecting the contour of the pattern. The information processing device 1 also inputs an image region, the segmentation result of the image region, and another image region into a trained second segmentation model (similar image contour detection model 210) that receives a reference image, the segmentation result of the reference image, and a target image as input and segments the target image, thereby obtaining the segmentation result from the second segmentation model, thereby detecting the contour of the pattern in another image region. As a result, the information processing system according to this embodiment is expected to assist users in performing contour detection processing, since it allows the user to perform contour detection on the image areas of the second and subsequent patterns for multiple image areas extracted from an SEM image simply by inputting a prompt for the image area of ​​the first pattern.

[0119] Furthermore, in the information processing system according to this embodiment, the information processing device 1 generates prompts for detection processing related to another pattern based on a prompt acquired for one pattern. The prompts may be generated, for example, using a trained learning model, or may be generated rule-based based on predetermined rules. As a result, the information processing system according to this embodiment can generate prompts for the second and subsequent patterns based on the prompt for the first pattern and perform detection processing for the second and subsequent patterns using the generated prompts, which is expected to assist the user in performing contour detection processing.

[0120] In the information processing system according to this embodiment, the information processing device 1 inputs an image region and a prompt into a trained segmentation model (contour detection model 200) that receives an image and a prompt as input and classifies (segments) the multiple pixels that make up the image. The information processing device 1 then obtains the segmentation results output by the segmentation model, thereby detecting pixels in which a pattern is captured in the image region, i.e., detecting the contours of the pattern. The information processing device 1 also calculates feature amounts for each pixel in the image region and other image regions, extracts similar pixels in the image region and other image regions based on the calculated feature amounts, and generates a prompt for the other image regions based on the segmentation result for the image region and the extracted similar pixels. This allows the information processing system according to this embodiment to generate prompts for image regions of a second or subsequent patterns based on a prompt for an image region of a first pattern.

[0121] Furthermore, in the information processing system according to this embodiment, the prompt received by the information processing device 1 includes information specifying pixels included in the pattern to be detected, or information specifying pixels not included in the pattern, which is expected to simplify the user's operation for specifying the pattern to be the target of contour detection.

[0122] Furthermore, in the information processing system according to this embodiment, the information processing device 1 displays (outputs) the results of contour detection on the display unit 14, accepts input of correction information for the detection results from the user, and corrects the detection results based on this correction information. As a result, the information processing system according to this embodiment is expected to help the user correct the input of prompts that are detection conditions through trial and error, thereby obtaining the desired detection results.

[0123] Furthermore, in the information processing system according to this embodiment, the information processing device 1 accepts input of correction information for the detection result for one pattern and corrects the detection result for another pattern based on this correction information. As a result, when performing contour detection processing on many image regions, the information processing system according to this embodiment can reflect the correction made by the user to one image region in the image regions of other patterns, and is expected to support the user's correction work.

[0124] Furthermore, in the information processing system according to this embodiment, the information processing device 1 acquires the detection results of the image area onto which a pattern is captured and the results of contour detection of this pattern, acquires prompts such as coordinate specification, area specification, or text information input by the user, and detects measurement points for the pattern based on the acquired detection results and prompts. Furthermore, the information processing device 1 detects measurement points for another pattern based on the prompt acquired for one pattern. As a result, the information processing system according to this embodiment reduces the frequency and amount of requests for the user to input conditions, etc., in the measurement point detection process for multiple patterns of predetermined shapes formed on a substrate, and is therefore expected to support the user's work in performing the measurement point detection process.

[0125] Furthermore, in the information processing system according to this embodiment, the information processing device 1 detects measurement points of one pattern by inputting a mask image of one pattern and a prompt into a trained first measurement point detection model (measurement point detection model 220) that receives as input the contour detection results and a prompt and detects measurement points of this pattern, and acquiring a measurement point image output by the first measurement point detection model. Furthermore, the information processing device 1 detects measurement points of other patterns by inputting a mask image of one pattern, measurement point results of the mask image of the one pattern, and a mask image of another pattern into a trained second measurement point detection model (similar image measurement point detection model 230) that receives as input the reference image, measurement point detection results of the reference image, and a target image and detects measurement points of the target image, and acquiring measurement point detection results by the second measurement point detection model. As a result, the information processing system of this embodiment is expected to assist the user in performing measurement point detection processing, since it allows the user to perform measurement point detection for the mask images of the second and subsequent patterns simply by inputting a prompt for the mask image of the first pattern based on the contour detection results of each pattern displayed in multiple image areas extracted from the SEM image.

[0126] Furthermore, in the information processing system according to this embodiment, the information processing device 1 determines the state of the pattern formed on the substrate based on the measurement point detection results. The information processing device 1 measures each pattern based on the detection results of the measurement point of the pattern captured in each image area, and can determine the state of each pattern, for example, depending on whether the measurement result is within a predetermined range. As a result, the information processing system according to this embodiment can be expected to accurately determine the state of each pattern formed on the substrate by the substrate processing device 101 based on the SEM image captured by the scanning electron microscope 102.

[0127] Furthermore, in the information processing system according to this embodiment, the information processing device 1 stores a prompt input from the user as condition information related to the detection process in the prompt storage unit 12 b, and instead of receiving a prompt directly from the user when performing the detection process, the information processing device 1 acquires the prompt stored in the prompt storage unit 12 b and performs the detection process. As a result, when performing detection process on a pattern similar to a pattern that was previously the target of detection process, the information processing system according to this embodiment is expected to support the user's work in performing the detection process, because the user does not need to input a prompt again.

[0128] 1 to 3, 7, 9, 11, and 13 show schematic SEM images of uneven portions formed on a substrate, and the process performed by the information processing device 1 using these as patterns to be detected has been described. However, the patterns to be detected are not limited to such externally three-dimensional structures. The patterns may be, for example, functional elements such as transistors or capacitors formed in multiple numbers on a substrate, or various other formations such as thin films.

[0129] [Embodiment 2] <Contour Detection Processing> As described above, in the information processing system according to this embodiment, pattern detection processing, contour detection processing, and measurement point detection processing are performed in this order on an SEM image captured by the scanning electron microscope 102. Of these processes, the information processing system according to embodiment 2 differs from the information processing system according to embodiment 1 in the procedure of contour detection processing. The procedures of the pattern detection processing and measurement point detection processing are the same in the information processing system according to embodiment 1 and the information processing system according to embodiment 2.

[0130] In the information processing system according to the second embodiment, an image area of ​​a pattern to be subjected to contour detection and a mask image that is the result of contour detection of this pattern are collected in advance. The collected image area of ​​the pattern is converted into feature quantities, and a set of the image area of ​​the pattern, the feature quantities, and the mask image is stored and accumulated in a database. This database may be provided in the information processing device 1 or in a device separate from the information processing device 1, but at least the information processing device 1 is able to access the database via communication or the like.

[0131] The image areas of the patterns to be stored in the database may be collected in advance by, for example, a designer of the information processing system according to embodiment 2. Alternatively, data obtained in the process of contour detection processing or the like by the information processing system according to embodiment 1 may be stored in the database.

[0132] Furthermore, the conversion of the image region of the pattern into feature quantities can be performed using, for example, a learning model that has been machine-learned in advance. The technology for converting an image into feature quantities using a learning model is already known, and therefore a detailed description thereof will be omitted.

[0133] An information processing device 1 of an information processing system according to the second embodiment converts an image region of a pattern obtained by pattern detection processing of an SEM image into a feature quantity. The information processing device 1 calculates the similarity (e.g., cosine similarity or L2 norm) between the converted feature quantity and the feature quantity of the image region of the pattern stored in a database, and obtains from the database a pair of the image region of the pattern and the mask image with the highest feature similarity.

[0134] The information processing device 1 according to the second embodiment performs contour detection for the image region of a target pattern by utilizing the image region and mask image of the pattern acquired from the database and the similar image contour detection model 210 shown in Fig. 8. That is, the information processing device 1 uses the image region of the similar pattern acquired from the database as the "reference image," which is input information to the similar image contour detection model 210, the mask image acquired from the database as the "reference image contour detection result," the image region of the pattern obtained by the pattern detection process as the "target image," and the mask image obtained by filling in the entire target image as the "filled image." The information processing device 1 inputs this information to the similar image contour detection model 210 and acquires the mask image output by the similar image contour detection model 210, thereby being able to detect a contour from the image region of the target pattern.

[0135] Note that if the information processing device 1 according to the second embodiment cannot detect a contour using the above-described method, or if no image area of ​​a similar pattern is stored in the database, the information processing device 1 according to the second embodiment performs contour detection by accepting a prompt input from the user, for example, in the same manner as the information processing device 1 according to the first embodiment. After obtaining the contour detection result based on the prompt input by the user, the information processing device 1 stores in the database the image area of ​​the pattern for which contour detection was performed, the feature amount of this image area, and the mask image that is the result of contour detection, in association with each other.

[0136] Note that the similar image contour detection model 210 used by the information processing device 1 according to the first embodiment for the contour detection process can be an existing trained learning model such as segGPT.

[0137] 16 is a flowchart showing an example of the procedure of contour detection processing performed by the information processing device 1 according to embodiment 2. The contour detection processing unit 11c of the processing unit 11 of the information processing device 1 according to embodiment 2 acquires an image area of ​​the detected pattern as a result of detection of the pattern from the SEM image by the pattern detection processing unit 11b (step S71).

[0138] The contour detection processing unit 11c converts the image region of the pattern acquired in step S71 into a feature (step S72). At this time, the contour detection processing unit 11c uses, for example, a learning model that has been machine-learned in advance to convert an input image into a feature and output it, inputs the image region of the pattern into this learning model, and acquires the feature output by the learning model, thereby converting the image region of the pattern into a feature.

[0139] The contour detection processing unit 11c compares the feature converted in step S72 with a plurality of feature values ​​previously stored in a database to search the database for an image area similar to the image area of ​​the pattern acquired in step S71 (step S73). At this time, the contour detection processing unit 11c calculates the similarity between the feature converted in step S72 and each feature value stored in the database, and acquires from the database the image area of ​​the pattern corresponding to the feature value with the highest similarity and the corresponding mask image.

[0140] If the contour detection processing unit 11c acquires multiple image areas for one pattern in step S71, it simply selects at least one of these image areas as a representative, converts it into features, and searches for similar image areas.

[0141] The contour detection processing unit 11c inputs the similar image area and mask image acquired from the database in step S73 and the pattern image area acquired in step S71 (and a filled-in image obtained by filling in this image) to the similar image contour detection model 210, which is a learning model that has undergone machine learning in advance (step S74). The contour detection processing unit 11c acquires a mask image of the contour detection result output by the similar image contour detection model 210 in response to the information input in step S74 (step S75).

[0142] Next, the contour detection processing unit 11c determines whether the contour detection result acquired in step S75 is correct (step S76). Here, the contour detection processing unit 11c calculates the similarity between the mask image acquired in step S73 and the mask image acquired in step S76, for example, and can determine whether the contour detection result is correct based on whether the calculated similarity exceeds a predetermined threshold. When calculating the similarity between the two mask images, the contour detection processing unit 11c may, for example, convert the mask images into feature quantities and calculate cosine similarity, or may calculate the similarity using a template matching technique, or may calculate the IoU (Intersection over Union) of the two images as the similarity, or may calculate the similarity using other techniques.

[0143] The contour detection processing unit 11c may determine whether the contour detection result is correct by a method other than the above-described method based on the similarity of the mask images. For example, when searching for a similar image region from the database in step S73, the contour detection processing unit 11c may determine that the contour detection result is incorrect if the database does not store a similar image region whose feature similarity exceeds a predetermined threshold.

[0144] Furthermore, for example, the contour detection processing unit 11c may determine whether the contour detection result is correct by using a learning model previously generated by machine learning. The learning model is generated by so-called supervised machine learning using, for example, learning data in which a mask image is associated with a flag indicating whether the mask image is correct. The generated learning model receives a mask image as input and outputs information indicating whether the mask image is correct. The contour detection processing unit 11c can input the mask image acquired as the contour detection result in step S75 to the learning model and acquire information output by the learning model to determine whether the mask image is correct.

[0145] If the contour detection result acquired in step S75 is correct (S76: YES), the display processing unit 11f of the processing unit 11 displays the contour detection result on the display unit 14 (step S80), and the contour detection process ends.

[0146] If the contour detection result is incorrect (S76: NO), the contour detection processing unit 11c displays the image area of ​​the pattern acquired in step S71 on the display unit 14, for example, and receives a prompt input from the user containing information necessary for contour detection for this image area (step S77). Based on the prompt received in step S77, the contour detection processing unit 11c performs contour detection using, for example, the contour detection model 200 shown in FIG. 6 (step S78). The contour detection processing unit 11c stores the mask image obtained as a result of the contour detection in step S78 in a database together with the image area of ​​the pattern acquired in step S71 and the feature quantities of this image area (step S79). The display processing unit 11f displays the result of the contour detection in step S78 on the display unit 14 (step S80), and the contour detection process ends.

[0147] (Variant example) In the above-described contour detection process, the information processing device 1 extracts one image area from the database that is similar to the image area of ​​the pattern that is the target of contour detection, but this is not limited to this, and multiple similar image areas may be extracted.

[0148] In steps S73 to S76 of the flowchart shown in FIG. 16 , the information processing device 1 acquires from the database the image area and mask image most similar to the image area of ​​the target pattern, performs contour detection, and determines whether the contour detection result is correct. If it determines in step S76 that the contour detection result is incorrect, the information processing device 1 according to the modified example acquires from the database the next most similar image area and mask image, performs contour detection in the same manner, and determines whether the contour detection result is correct. The information processing device 1 according to the modified example repeatedly acquires image areas and mask images from the database in descending order of similarity until a correct contour detection result is obtained. If it determines that the contour detection result based on the image area and mask image acquired from the database for the Kth time is incorrect, for example, it performs contour detection based on the user's input in steps S77 to S79. Note that K is a natural number and is predetermined by the designer of the information processing system according to this embodiment.

[0149] In addition, the information processing device 1 in the modified example may acquire sets of image areas and mask images whose similarity falls within a predetermined range in order of decreasing similarity, rather than acquiring up to a predetermined Kth image area and mask image that are similar to the image area to be subjected to contour detection.

[0150] Furthermore, the information processing device 1 according to the modified example may include a similar image contour detection model 210 configured to receive, for example, a plurality of reference images and contour detection results, and one target image and filled-in image as input, and output a mask image that serves as the contour detection result for the target image. In this case, the information processing device 1 according to the modified example can input a plurality of sets of image regions and mask images acquired from a database to the similar image contour detection model 210, and acquire a contour detection result for the target image region.

[0151] <Method of Adding Information to Database> When the information processing device 1 according to the second embodiment cannot obtain a correct contour detection result based on the information stored in the database, the information processing device 1 accepts a prompt input from the user, performs contour detection, and stores the contour detection result in the database. At this time, the information processing device 1 may store in the database a mask image that is an inverted version of the mask image of the contour detection result based on the user's prompt input. Using the inverted mask image as the contour detection result of the reference image to be input to the similar image contour detection model 210 may improve the accuracy of contour detection for the target image region.

[0152] Fig. 17 is a schematic diagram for explaining a method for adding information to a database. As shown in Fig. 8, the similar image contour detection model 210 according to this embodiment receives as input a reference image, a contour detection result (mask image) from the reference image, a target image, and a filled-in image, and outputs a mask image that is the contour detection result for the target image. In step S78 of the flowchart shown in Fig. 16, the information processing device 1 obtains a mask image of the contour detection result for the image region of the pattern.

[0153] 17, the information processing device 1 inputs this image area as a target image and a reference image to the similar image contour detection model 210, and also inputs a mask image (normal mask image) of this image area to the similar image contour detection model 210. Note that the filled-in image input to the similar image contour detection model 210 is not shown in Fig. 17. The information processing device 1 acquires a mask image of the contour detection result output by the similar image contour detection model 210, and compares the acquired mask image with the input normal mask image to calculate the similarity.

[0154] Furthermore, the information processing device 1 generates an inverted mask image by inverting the normal mask image, as shown in the lower part of Fig. 17. The information processing device 1 inputs the original image region as a target image and a reference image to the similar image contour detection model 210, and also inputs this inverted mask image to the similar image contour detection model 210. The information processing device 1 acquires a mask image of the contour detection result output by the similar image contour detection model 210, inverts the acquired mask image, and compares the inverted mask image with the normal mask image to calculate the similarity.

[0155] The information processing device 1 compares the similarity calculated based on the normal mask image with the similarity calculated based on the inverted mask image to determine whether to store the normal mask image or the inverted mask image. Note that the similarity calculated by the information processing device 1 may be an appropriate value such as IoU or cosine similarity.

[0156] If the similarity calculated based on the normal mask image is higher, the information processing device 1 associates the original image region with the normal mask image and stores them in the database.If the similarity calculated based on the inverted mask image is higher, the information processing device 1 associates the original image region with the inverted mask image and stores them in the database.At this time, the information processing device 1 may store information indicating that the mask image associated with the image region is an inverted mask image in the database, together with the image region and the inverted mask image.

[0157] 18 to 21 are schematic diagrams showing examples of screen displays by the information processing device 1 according to this embodiment. The information processing device 1 according to this embodiment displays, for example, an image file selection screen (not shown) and accepts a selection operation of an SEM image captured by the scanning electron microscope 102 from the user. The information processing device 1 reads one or more selected SEM image files and displays, for example, the initial screen shown in FIG. 18 on the display unit 14. The initial screen displayed by the information processing device 1 has, for example, an image display area 301 at the top of the screen that displays a list of SEM images, a preprocessing setting button 302, an automatic / manual setting button 303, and a segmentation setting button 304 arranged vertically on the left side of the bottom of the screen, and a length measurement start button 305 at the bottom right of the bottom of the screen.

[0158] The information processing device 1 displays SEM images selected by the user in association with their file names in the image display area 301 of the initial screen. In the example shown in FIG. 18 , three SEM images with file names "File A1," "File A2," and "File A3" are selected, and the information processing device 1 displays the three SEM images horizontally arranged in the image display area 301. The user can select a pattern to be measured by selecting a rectangular area on the SEM image displayed in the image display area 301 using the operation unit 15, such as a mouse. In FIG. 18 , a rectangular frame 306 surrounding the pattern selected by the user is superimposed on the SEM image. The user can also specify a measurement point on the pattern by specifying a start point and an end point within the rectangular frame 306 using the operation unit 15, such as a mouse. In FIG. 18 , a bidirectional arrow 307 indicating the measurement point specified by the user is superimposed on the SEM image.

[0159] The preprocessing setting button 302, the automatic / manual setting button 303, and the segmentation setting button 304 provided on the initial screen are buttons that allow the information processing device 1 to accept various settings related to pattern measurement from the user. When these buttons are clicked with a mouse, the information processing device 1 displays a setting screen (not shown) on the display unit 14 for accepting detailed settings. The preprocessing setting button 302 is a button for setting preprocessing that adjusts the brightness or contrast of the SEM image, etc. The automatic / manual setting button 303 is a button for setting whether the contour detection process for the contour measurement is to be performed automatically or manually by the user. When the automatic setting is selected by the user, the information processing device 1 performs contour detection using a set of similar image regions and mask images stored in the database described in the second embodiment. When the manual setting is selected, the information processing device 1 performs contour detection based on the user's prompt input described in the first embodiment. The segmentation setting button 304 is a button for setting the type or size of a learning model to be used in the contour detection process, etc. Furthermore, when a mouse click or the like is performed on the measurement start button 305 provided on the initial screen, the information processing device 1 starts measurement processing for one or more SEM images displayed in the image display area 301.

[0160] When the measurement process is started with the setting for automatic measurement, the information processing device 1 performs a pattern detection process on one or more given SEM images, acquires an image area and a mask image similar to the image area of ​​the detected pattern from a database, and performs a contour detection process. Furthermore, the information processing device 1 performs a measurement point detection process based on the contour detection result, measures the detected measurement points, and displays the measurement results on a measurement result display screen, for example, as shown in Fig. 19. The measurement result display screen displayed by the information processing device 1 has, for example, an image display area 311 at the top of the screen for displaying a list of SEM images, a numeric value display area 312 at the bottom left of the screen for displaying a list of numeric values ​​of the measurement results, and a graph display area 313 at the bottom right of the screen for displaying the measurement results in a graph.

[0161] The SEM image that the information processing device 1 displays in the image display area 311 of the measurement result display screen is the same as the SEM image displayed in the image display area 301 of the initial screen. However, the information processing device 1 superimposes on each SEM image one or more rectangular frames surrounding the image area of ​​the detected pattern, two-way arrows indicating the measurement points for each pattern, and identification information assigned to each pattern by the information processing device 1. Although not shown in the drawings, the information processing device 1 may also superimpose on the SEM image a mask image obtained as a result of contour detection in the contour detection process, for example, by color-coding the mask image for each pattern.

[0162] The information processing device 1 displays a list of the identification information assigned to each pattern as a measurement result and the numerical value of the measurement result of this pattern, which are associated with each other, arranged vertically in a numerical value display area 312 of the measurement result display screen. The information processing device 1 also displays, for example, a histogram graph summarizing the measurement results of a plurality of patterns in a graph display area 313 of the measurement result display screen. The histogram is a graph in which, for example, the horizontal axis represents a plurality of classes obtained by dividing the numerical values ​​(lengths) of the measurement results into appropriate ranges, and the vertical axis represents the number of patterns (frequency) that fall into each class.

[0163] The measurement result display screen shown in Fig. 19 is a screen displayed when the information processing device 1 obtains a correct detection result through the contour detection process. As shown in steps S76 to S79 ​​of the flowchart in Fig. 16, if the information processing device 1 cannot obtain a correct contour detection result based on the information stored in the database, the information processing device 1 performs contour detection based on input from the user. If the information processing device 1 cannot obtain a correct detection result through the contour detection process, the information processing device 1 displays a failure notification screen shown in Fig. 20 on the display unit 14 to notify the user of the failure of the automatic contour detection process. The failure notification screen displayed by the information processing device 1 has, for example, an image display area 321 at the top of the screen for displaying an SEM image and a mask image related to the failed pattern, a message display area 322 at the bottom left of the screen for notifying the user of the failure of the measurement result, and a manual measurement button 323 at the bottom right of the screen for performing manual measurement.

[0164] The information processing device 1 displays, in the image display area 321 of the failure notification screen, an SEM image of the pattern for which the contour detection process failed, the image area of ​​this pattern (target pattern), a mask image (estimated mask) of the contour detection results performed on this image area, and an image area (similar pattern) and mask image (similar mask) acquired from a database as being similar to this image area. The estimated mask displayed here is the incorrect contour detection result obtained by the contour detection process.

[0165] Furthermore, the information processing device 1 notifies the user that the automatic measurement process has failed by displaying a message of "Failed" as the measurement result in the message display area 322 of the failure notification screen. When a click operation or the like is performed on the manual measurement button 323 provided on the failure notification screen, the information processing device 1 displays the manual measurement screen shown in FIG. 21 on the display unit 14.

[0166] The manual measurement screen displayed by the information processing device 1 is divided into two areas, for example, an upper area and a lower area, with the upper area being a prompt input area 331 for inputting information (prompts) such as measurement conditions or settings, and the lower area being a measurement result display area 332 for displaying the measurement results.

[0167] The information processing device 1 displays, for example, an SEM image to be measured at the left end of a prompt input area 331 on the manual measurement screen, and accepts input of a rectangular frame surrounding the pattern to be measured or input of the start and end points of the measurement location based on mouse operation by the user, etc. The information processing device 1 displays an image area of ​​the pattern to be measured (target pattern) extracted from the SEM image to the right of the SEM image, and accepts input of points included or not included in the pattern in this image area from the user. The information processing device 1 performs contour detection using the contour detection model 200 using the information entered by the user as a prompt, and displays a mask image (estimated mask) obtained as a result of the contour detection to the right of the target pattern. Note that the information processing device 1 can accept prompt input by text input in natural language instead of or in addition to input of a rectangular frame or points (coordinates), and a text box for text input is provided at the right end of the prompt input area 331. In addition, the prompt input area 331 has an apply button below the text box, and when the user clicks on the apply button, the information processing device 1 applies the conditions entered in the prompt input area 331 and performs contour detection processing, measurement location detection processing, measurement processing, etc. for the same pattern contained in one or more SEM images.

[0168] The information processing device 1 displays one or more SEM images of the measurement target in a row in the upper part of the measurement result display area 332 of the manual measurement screen, and displays the measurement results in a list in the lower part, as well as in graphs such as histograms. The information displayed in the measurement result display area 332 is approximately the same as the information displayed on the measurement result display screen in Fig. 19, and therefore a detailed description thereof will be omitted.

[0169] A DB Add button is provided in the lower right portion of the measurement result display area 332 on the manual measurement screen. When the DB Add button is clicked, the information processing device 1 stores the set of the target pattern and estimated mask displayed in the prompt input area 331 in a database. At this time, the information processing device 1 may convert the target pattern into a feature and store the feature in the database together with the target pattern and the estimated mask. This information stored in the database is used as a search target for similar patterns in the subsequent measurement processing.

[0170] <Summary> In the information processing system according to the second embodiment configured as described above, the information processing device 1 stores a set of an image area of ​​a pattern and a mask image that is the result of contour detection for the pattern in a database. The information processing device 1 acquires a set of an image area and a mask image from the database based on the image area of ​​the pattern detected from the SEM image, and detects the contour of the pattern from the image area detected from the SEM image based on the acquired set of image area and mask image. This allows the information processing system according to the second embodiment to perform contour detection using the information stored in the database without receiving input of condition information (prompt) for contour detection from the user.

[0171] Furthermore, in the information processing system according to the second embodiment, when the information processing device 1 is unable to perform contour detection correctly based on the information stored in the database, it acquires a prompt, detects the contour of a pattern from the image area based on the acquired prompt, and stores a set of the image area and a mask image that is the contour detection result in the database. As a result, when the information processing system according to the second embodiment is unable to perform contour detection correctly based on the information stored in the database, it acquires a prompt from the user, performs contour detection, stores the contour detection result in the database, and can use it for subsequent contour detection.

[0172] In the information processing system according to the second embodiment, the information processing device 1 acquires a pair of an image region and a mask image from the database based on the similarity between the image region to be subjected to contour detection and each image region stored in the database. Furthermore, if the database does not contain any image regions whose similarity exceeds a threshold, the information processing device 1 determines that a contour cannot be detected from the target image region and acquires a prompt. This allows the information processing system according to the second embodiment to be expected to acquire, from the database, a pair of an image region and a mask image that are useful for contour detection for the image region to be subjected to contour detection.

[0173] In the information processing system according to the second embodiment, the information processing device 1 determines whether a correct contour can be detected from an image region based on the similarity between a mask image resulting from contour detection from a target image region and a mask image acquired from the database. As a result, the information processing system according to the second embodiment is expected to accurately determine whether contour detection is possible based on the information stored in the database.

[0174] In the information processing system according to the second embodiment, the information processing device 1 displays one or more SEM images on the display unit 14, and also displays an image area of ​​a pattern detected from the SEM image, a mask image of the contour detection results from this image area, or measurement points detected based on links, superimposed on the SEM image, and displays the measurement results based on the measurement points. By displaying this information, the information processing system according to the second embodiment can be expected to provide the user with detailed information regarding pattern measurement.

[0175] In the information processing system according to the second embodiment, the information processing device 1 displays a histogram of measurement results for a plurality of patterns detected from an SEM image. This makes it possible to expect that the information processing system according to the second embodiment will provide the user with information on the variation in the shape of the patterns, etc.

[0176] Furthermore, in the information processing system according to the second embodiment, when the information processing device 1 cannot obtain a correct contour detection result from an SEM image, it displays on the display unit 14 the original SEM image, the image area of ​​the pattern, a mask image of the erroneous contour detection result obtained for each image area, and a set of the image area and mask image obtained from the information stored in the database. The information processing device 1 also accepts input of a prompt for the displayed SEM image. As a result, when a correct contour detection result cannot be obtained, the information processing system according to the second embodiment is expected to provide the user with information for determining the cause, etc., and to accept input of a prompt based on this information.

[0177] 18 to 21 in the second embodiment are merely examples and are not limiting. The information processing device 1 may display information relating to the pattern detection process, the contour detection process, the measurement location detection process, the measurement process, etc. on the display unit 14 in any manner.

[0178] Furthermore, other configurations of the information processing system according to the second embodiment are the same as those of the information processing system according to the first embodiment, so the same reference numerals are used for the same parts and detailed description thereof will be omitted.

[0179] The embodiments disclosed herein are to be considered as illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0180] The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations, regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used.

[0181] <Notes> (Note 1) A computer program that causes a computer to perform detection processing for a plurality of patterns of a predetermined shape formed on a substrate for one or a plurality of captured images of the substrate that has been processed by a substrate processing apparatus, the computer program causing the computer to execute the following processes: detect an image area in which the pattern is captured from the captured image; detect a contour of the pattern in the image area; and detect a measurement point for the pattern based on the contour detection result; acquire condition information including at least one of coordinate specification, area specification, or text that is a detection condition in at least one of the detection processes for the image area, the contour detection process, and the measurement point detection process; and perform detection processing for another pattern based on the condition information acquired for one pattern. (Supplementary Note 2) The computer program according to Supplementary Note 1, wherein a storage unit stores a plurality of sets of image areas onto which patterns are captured and contour detection results for the patterns in the image areas, and the program acquires the set of image areas and contour detection results from the storage unit based on the image areas detected from the captured image, and detects the contour of the pattern from the image areas detected from the captured image based on the acquired set of image areas and contour detection results. (Supplementary Note 3) The computer program according to Supplementary Note 2, wherein when the contour of the pattern cannot be detected from the image area, the program acquires the condition information, detects the contour of the pattern from the image area based on the acquired condition information, and stores the set of image areas and contour detection results in the storage unit. (Supplementary Note 4) The computer program according to Supplementary Note 2, wherein the program acquires the set of image areas and contour detection results from the storage unit based on the similarity between the image area for contour detection and each image area stored in the storage unit, and determines that the contour cannot be detected from the image area if no image area for which the similarity exceeds a threshold is stored in the storage unit. (Supplementary Note 5) The computer program according to Supplementary Note 2, wherein the computer program determines whether or not a correct contour of a pattern can be detected from the image area based on a similarity between a contour detection result from an image area of ​​the pattern detected from the captured image and a contour detection result acquired from the memory unit.(Supplementary Note 6) The computer program according to Supplementary Note 2, which displays one or more captured images to be processed, superimposing an image area of ​​a pattern detected from the captured image, a contour of the pattern detected from the image area, or a measurement point detected based on the contour on the captured image, and displays a measurement result based on the measurement point. (Supplementary Note 7) The computer program according to Supplementary Note 6, which displays a histogram of measurement results for a plurality of patterns detected from the captured image. (Supplementary Note 8) The computer program according to Supplementary Note 3, which, when a correct contour cannot be detected from the image area, displays the original captured image from which the image area was detected, the image area, an erroneous contour detection result for the image area, and a set of image area and mask information acquired from the storage unit based on the image area. (Supplementary Note 9) The computer program according to Supplementary Note 8, which accepts input of the condition information for the displayed captured image.

[0182] 1 Information processing device (computer) 11 Processing unit 11a Image acquisition unit 11b Pattern detection processing unit 11c Contour detection processing unit 11d Measurement point detection processing unit 11e State determination unit 11f Display processing unit 12 Storage unit 12a Program (computer program) 12b Prompt storage unit 13 Communication unit 14 Display unit 15 Operation unit 99 Recording medium 101 Substrate processing device 102 Scanning electron microscope 200 Contour detection model 201 Image encoder 202 Mask encoder 203 Prompt encoder 204 Mask decoder 210 Similar image contour detection model 220 Measurement point detection model 221 Image encoder 222 Prompt encoder 223 Measurement point decoder 230 Similar image measurement point detection model 301 Image display area 302 Pre-processing setting button 303 Automatic / manual setting button 304 Segmentation setting button 305 Measurement start button 311 Image display area 312 Numerical value display area 313 Graph display area 321 Image display area 322 Message display area 323 Manual measurement button 331 Prompt input area 332 Measurement result display area

Claims

1. A computer program causing a computer to perform detection processing for multiple patterns of a predetermined shape formed on a substrate for one or more captured images of the substrate that has been processed by a substrate processing apparatus, the computer executing the following processes: detect an image area in which the pattern is captured from the captured image; detect the contour of the pattern in the image area; and detect a measurement point for the pattern based on the detection result of the contour; acquire condition information including at least one of coordinate specification, area specification, or text that is a detection condition for at least one of the detection processes for the image area, the contour detection process, and the measurement point detection process; and perform detection processing for one pattern based on the condition information acquired for the other pattern.

2. A computer program that causes a computer to perform detection processing for multiple patterns of a predetermined shape formed on a substrate for one or more captured images of the substrate that has been processed by a substrate processing apparatus, the computer program causing the computer to: acquire an image area in which one of the patterns extracted from the captured image is captured; acquire detection condition information including at least one of coordinate designation, area designation, or text; execute a process to detect the contour of the pattern in the image area based on the condition information; and perform detection processing for another pattern based on the condition information acquired for one pattern.

3. A computer program as described in claim 1 or claim 2, which detects the contour of the pattern in one image region by inputting an image region and the acquired condition information into a trained first segmentation model that accepts an image and condition information as input and classifies multiple pixels that make up the image based on the condition information and obtaining a segmentation result by the first segmentation model; and detects the contour of the pattern in the other image region by inputting the one image region, the segmentation result of the one image region, and another image region into a trained second segmentation model that accepts a reference image, the segmentation result of the reference image, and a target image as input and segments the target image, and obtaining a segmentation result by the second segmentation model.

4. The computer program according to claim 1 or 2, further comprising generating condition information for a detection process relating to the other pattern based on the condition information acquired relating to the one pattern.

5. The computer program according to claim 4, which detects the contour of the pattern in the one image region by inputting one image region and the acquired condition information into a trained segmentation model that accepts an image and condition information as input and classifies multiple pixels that make up the image based on the condition information and obtains a segmentation result by the segmentation model, calculates features of each pixel of the one image region and the other image region, extracts similar pixels of the one image region and the other image region based on the calculated features, and generates condition information for segmentation of the other image region based on the segmentation result for the one image region and the extracted similar pixels.

6. A computer program according to claim 1 or claim 2, wherein the condition information includes information specifying pixels included in a detection pattern, or information specifying pixels not included in the pattern.

7. The computer program according to claim 1 or 2, further comprising: outputting a detection result of the contour of the pattern; accepting input of correction information for the detection result; and correcting the detection result based on the correction information.

8. The computer program according to claim 7, further comprising: receiving input of correction information for the detection result relating to said one pattern; and correcting the detection result relating to said other pattern based on said correction information.

9. A computer program as claimed in claim 2, which acquires condition information for detecting measurement points, including at least one of coordinate designation, area designation and text, detects measurement points for said pattern based on the contour detection result and said condition information, and performs detection processing for one pattern based on the condition information acquired for another pattern.

10. A computer program that causes a computer to perform detection processing for multiple patterns of a predetermined shape formed on a substrate for one or more captured images of the substrate that has been processed by a substrate processing apparatus, the computer being configured to: acquire detection results for the contours of the patterns in an image area in which the patterns are captured; acquire detection condition information including at least one of coordinate designation, area designation, or text; detect measurement points for the patterns based on the contour detection results and the condition information; and perform detection processing for one pattern based on the condition information acquired for another pattern.

11. A computer program as described in any one of claims 1, 9, or 10, comprising: inputting the contour detection result of one pattern and the condition information into a trained first measurement point detection model that accepts the contour detection result and the condition information as input and outputs information regarding the measurement points for the pattern, thereby detecting the measurement points for the one pattern; and inputting the contour detection result of the one pattern, the measurement point detection result of the one pattern, and the contour detection result of another pattern into a trained second measurement point detection model that accepts a reference image, the measurement point detection result of the reference image, and a target image as input, and outputs information regarding the measurement points for the target image, thereby detecting the measurement points for the other pattern.

12. The computer program according to claim 1, 2 or 10, further comprising: determining a state of the pattern based on a result of the detection process.

13. A computer program according to claim 1, 9 or 10, further comprising: measuring the pattern based on the detection result of the measurement point; and judging a state of the pattern based on the measurement result.

14. A computer program as claimed in any one of claims 1, 2 and 10, further comprising: storing the condition information in a storage unit; and acquiring the condition information used in the detection process from the storage unit.

15. A computer program as described in claim 1, wherein a memory unit stores multiple sets of image areas into which a pattern is captured and the results of contour detection of the pattern in the image areas; the set of image areas and contour detection results is obtained from the memory unit based on the image areas detected from the captured image; and the contour of the pattern is detected from the image areas detected from the captured image based on the obtained set of image areas and contour detection results.

16. The computer program according to claim 15, further comprising: if a pattern contour cannot be detected from the image area, acquiring the condition information; detecting a pattern contour from the image area based on the acquired condition information; and storing a set of the image area and the detection results of the contour in the memory unit.

17. The computer program of claim 15, further comprising: acquiring a pair of an image area and a contour detection result from the memory unit based on the similarity between the image area for which the contour is to be detected and each image area stored in the memory unit; and determining that a contour cannot be detected from the image area if no image area for which the similarity exceeds a threshold is stored in the memory unit.

18. The computer program according to claim 15, which determines whether or not a pattern contour can be detected from the image area based on the similarity between the contour detection result from the image area of ​​the pattern detected from the captured image and the contour detection result obtained from the memory unit.

19. An information processing method in which an information processing device performs detection processing on multiple patterns of a predetermined shape formed on a substrate for one or more captured images of a substrate that has been processed by a substrate processing device, wherein the information processing device: detects an image area in which the pattern is captured from the captured image; detects the contour of the pattern in the image area; detects a measurement point for the pattern based on the detection result of the contour; acquires condition information including at least one of coordinate specification, area specification, or text that is a detection condition in at least one of the detection processes of the image area detection process, the contour detection process, and the measurement point detection process; and performs detection processing on another pattern based on the condition information acquired for one pattern.

20. An information processing device comprising a processing unit which performs detection processing on a plurality of patterns of a predetermined shape formed on a substrate for one or a plurality of captured images of a substrate which has been processed by a substrate processing device, wherein the processing unit: detects an image area in which the pattern is captured from the captured image; detects a contour of the pattern in the image area; detects a measurement point for the pattern based on a result of the contour detection; acquires condition information including at least one of coordinate designation, area designation, or text which is a detection condition for at least one of the detection processes for the image area, the contour detection process, and the measurement point detection process; and performs detection processing on another pattern based on the condition information acquired for one pattern.

Images