Information processing method, computer program, and information processing device

Abstract

Provided are an information processing method, a computer program, and an information processing device which can be expected to accurately detect similar objects from a plurality of non-time series image data items relating to substrate processing.　An information processing method according to the present embodiment involves an information processing device that comprises a time-series segmentation model that sequentially receives time-series image data items as input and sequentially outputs, for each image item, detection results for a region in which an object is captured. The information processing device acquires a plurality of non-time-series image items obtained by imaging an object relating to substrate processing. An acquired image data item is input to the time-series segmentation model to acquire detection results, and the acquired detection results are output.

Description

Information Processing Method, Computer Program, and Information Processing Apparatus

[0001] The present disclosure relates to an information processing method, a computer program, and an information processing apparatus.

[0002] In Patent Document 1, a pattern inspection system is proposed that inspects an image of an inspection target pattern using an identifier configured by machine learning based on a pattern image of an inspection target of an electronic device and pattern data used to manufacture the inspection target pattern. This pattern inspection system includes an image selection unit that selects a learning pattern image for machine learning from a plurality of pattern images based on the pattern data and pattern images stored in the storage unit.

[0003] Japanese Patent Application Laid-Open No. 2022-172401

[0004] The present disclosure provides an information processing method, a computer program, and an information processing apparatus that can be expected to accurately detect similar objects from a plurality of non-sequential image data related to substrate processing.

[0005] An information processing method according to an embodiment includes an information processing apparatus including a time-series segmentation model that sequentially receives time-series image data as input and sequentially outputs detection results of regions in which an object is depicted for each image data, obtains a plurality of non-sequential image data obtained by photographing an object related to substrate processing, inputs the obtained image data into the time-series segmentation model to obtain a detection result, and outputs the obtained detection result.

[0006] According to the present disclosure, it can be expected to accurately detect similar objects from a plurality of non-sequential image data related to substrate processing.

[0007] This is a schematic diagram illustrating the overview of the information processing system according to this embodiment. This is a schematic diagram illustrating the overview of the processing performed by the information processing device according to this embodiment. This is a block diagram illustrating one example configuration of the information processing device according to this embodiment. This is a schematic diagram illustrating the overview of the region detection processing performed by the information processing device according to this embodiment. This is a schematic diagram illustrating one example configuration of a video segmentation model. This is a flowchart illustrating an example of the procedure for region detection processing performed by the information processing device according to this embodiment. This is a schematic diagram illustrating the configuration of modification 1 regarding correct / false determination. This is a schematic diagram illustrating the configuration of modification 2 regarding correct / false determination. This is a schematic diagram showing an example of the operation screen for region detection processing displayed by the information processing device according to this embodiment. This is a schematic diagram illustrating an example of applying the information processing device system according to this embodiment to probe alignment.

[0008] Specific examples of information processing systems according to the embodiments of this disclosure will be described below with reference to the drawings. However, this disclosure is not limited to these examples and is intended to include all changes within the meaning and scope of the claims as indicated by the claims.

[0009] <System Overview> Figure 1 is a schematic diagram illustrating the overview of the information processing system according to this embodiment. The information processing system according to this embodiment is configured to include an information processing device 1, a substrate processing device 101, and a scanning electron microscope 102, etc. The substrate processing device 10 is a device such as a process chamber that performs etching or other processing on semiconductor wafers (substrates). After etching or other processing is performed on the substrate in the substrate processing device 101, the surface shape and other features of the processed substrate are photographed by the scanning electron microscope 102. The scanning electron microscope 102 is a device that observes an object by irradiating it with an electron beam and detecting secondary electrons or transmitted electrons emitted from the object, and outputs a so-called SEM (Scanning Electron Microscope) image of the object. The SEM image output by the scanning electron microscope 102 is provided to the information processing device 1. In this embodiment, processing is performed on the SEM image taken by the scanning electron microscope 102, but it is not limited to this, and processing may be performed on images taken by, for example, a transmission electron microscope or an optical microscope.

[0010] Various structures of different shapes are formed on the surface of a substrate processed by the substrate processing apparatus 101. For example, as shown in the SEM image in the center of Figure 1, multiple structures of the same shape may be formed on the substrate. Hereinafter, in this embodiment, each of these structures of the same shape will be referred to as a pattern. However, in reality, even if the intention is to form patterns of the same shape, the size or shape of each pattern may differ due to processes such as film deposition, etching, or exposure. Patterns whose size or shape deviates beyond a predetermined threshold are judged to be abnormal. The information processing system according to this embodiment is a system that extracts multiple patterns formed on a target substrate based on an SEM image of the target substrate taken by a scanning electron microscope 102, detects a predetermined region included in each pattern by performing segmentation processing on the image of each pattern, and performs length measurement processing to measure the length of a predetermined part of each pattern based on the results of region detection.

[0011] In this embodiment, the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 are described as separate devices, but this is not limited to them. For example, the substrate processing apparatus 101 and the scanning electron microscope 102 may be a single device, the scanning electron microscope 102 and the information processing apparatus 1 may be a single device, or the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be a single device. Furthermore, each of the devices of the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be configured by combining multiple devices.

[0012] Figure 2 is a schematic diagram illustrating the overview of the processing performed by the information processing device 1 according to this embodiment. The information processing device 1 according to this embodiment acquires an SEM image taken by a scanning electron microscope 102 of the target substrate, and first performs pattern detection processing on this SEM image. Multiple patterns of the same shape may be formed on the target substrate, and multiple patterns of the same shape may be captured in the SEM image acquired by the information processing device 1. An example of an SEM image is shown at the top of Figure 2, and in this SEM image, multiple rectangular frames with thick black lines are arranged vertically and horizontally. In this example, the structure with the shape of these rectangular frames is considered the target pattern, and the information processing device 1 performs detection processing.

[0013] In this example, multiple patterns are captured in a single SEM image, and the information processing device 1 acquires one SEM image and performs detection processing on each pattern. However, this is not the only way. For example, the information processing device 1 may acquire multiple SEM images, each containing multiple target patterns, and perform detection processing on the multiple patterns captured in the multiple SEM images. Alternatively, the information processing device 1 may acquire multiple SEM images, each containing a single target pattern, and perform detection processing on the patterns captured in the multiple SEM images.

[0014] The pattern detection process performed by the information processing device 1 is the process of detecting, for example, a rectangular image region in which a single pattern is depicted from the SEM image acquired from the scanning electron microscope 102. By performing pattern detection on the acquired SEM image, the information processing device 1 can obtain information such as the coordinates and size of the rectangular frame, so-called bounding box, surrounding each pattern for one or more patterns depicted in the SEM image. Based on the results of the pattern detection process, the information processing device 1 extracts images (image regions) in which each pattern is depicted by individually cutting out the bounding box region from the SEM image. The second image from the top in Figure 2 shows multiple images of the target pattern extracted from the SEM image.

[0015] In this embodiment, a detailed explanation of the pattern detection process, which extracts images of each pattern from the SEM image, will be omitted. The information processing device 1 appropriately employs an existing pattern detection algorithm to detect the pattern to be measured and acquires an image of the detected pattern. This pattern detection process may also be performed by a device other than the information processing device 1, in which case the information processing device 1 acquires the image of each pattern obtained as a result of the pattern detection process from the other device. In this embodiment, the detailed processing after obtaining the image of each pattern will be explained.

[0016] The information processing device 1 performs a region detection process to detect the region to be measured in one or more images obtained as a result of the pattern detection process. In this embodiment, the information processing device 1 performs the region detection process using a segmentation model that has been generated in advance by machine learning to perform image segmentation. Furthermore, in this embodiment, the information processing device 1 performs a region detection process to detect the region to be measured from multiple non-time-series images using a segmentation model that has been generated to perform segmentation on multiple images arranged in time series (so-called moving images). Through this region detection process, the information processing device 1 obtains information indicating which pixel in the image contains the target region, a so-called mask image. The third image from the top in Figure 2 shows the mask image obtained by region detection of the region to be measured.

[0017] Next, the information processing device 1 performs a length measurement process to measure the length of a predetermined location in each pattern based on the mask image obtained as a result of the region detection process. Even if the multiple patterns formed on the substrate are intended to have the same shape in the design, differences in size or shape may occur for each pattern due to processes such as film formation, etching, or exposure. In the information processing system according to this embodiment, whether or not each pattern on the substrate is formed in the desired shape is determined by measuring the length of a specific location in each pattern and determining whether or not the measured length is within the normal range.

[0018] The information processing device 1 measures the length of a target pattern by calculating, for example, the distance (number of pixels, etc.) between two predetermined locations on the obtained mask image. In the fourth row from the top of Figure 2, two arrows indicating measurement locations are superimposed on the mask image of each pattern. Each arrow has one end indicating the start point of the measurement location and the other end indicating the end point of the measurement location, and the straight-line distance connecting these start and end points is the distance to be measured. In this example, two arrows are shown as measurement locations, and measurements are taken at two locations for each pattern. That is, in this example, for each pattern indicated by the thick black rectangular frame, the vertical length and horizontal length within the frame are measured. By comparing the measured length with predetermined threshold values ​​such as upper and lower limits, the information processing device 1 can determine whether each pattern is normal or abnormal based on whether the length is within a predetermined range.

[0019] <Device Configuration> Figure 3 is a block diagram showing an example configuration of the information processing device 1 according to this embodiment. The information processing device 1 according to this embodiment can be realized by installing a predetermined application program on a general-purpose information processing device such as a personal computer or a server computer. However, the information processing device 1 may be a dedicated information processing device for controlling a substrate processing device 101 or a scanning electron microscope 102. The information processing device 1 according to this embodiment is configured to include a processing unit 11, a storage unit 12, a communication unit 13, a display unit 14, and an operation unit 15, etc. In this embodiment, the explanation will be given assuming that processing is performed by one information processing device 1, but the processing of the information processing device 1 may be distributed among multiple devices.

[0020] The processing unit 11 is composed of an arithmetic processing unit such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), NPU (Neural network Processing Unit), or quantum processor, and a storage device such as ROM (Read Only Memory) and RAM (Random Access Memory). The processing unit 11 reads and executes a program 12a stored in the storage unit 12, thereby performing various processes such as region detection processing on the pattern image extracted from the SEM image, and length measurement processing of the pattern based on the detected region.

[0021] The functions realized by the processing unit 11 can be implemented using any circuit or processing circuitry. For example, the circuit or processing circuitry can be implemented using a general-purpose processor, a special-purpose processor, an integrated circuit, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a conventional circuit. The functions realized by the processing unit 11 can be programmed using one or more programs stored in one or more memories, or configured in other ways to execute the disclosed functions. The functions of the processing unit 11 can be implemented using circuit or processing circuitry including combinations of these.

[0022] The storage unit 12 is configured using a large-capacity storage device such as a hard disk or an SSD (Solid State Drive). The storage unit 12 stores various programs executed by the processing unit 11, and various data necessary for the processing of the processing unit 11. In this embodiment, the storage unit 12 stores the program 12a executed by the processing unit 11. The storage unit 12 is also provided with a model information storage unit 12b that stores information about a trained learning model used when performing region detection processing.

[0023] In this embodiment, the program (computer program, program product) 12a is provided in a form recorded on a recording medium 99 such as a memory card or optical disc, and the information processing device 1 reads the program 12a from the recording medium 99 and stores it in the storage unit 12. However, the program 12a may also be written to the storage unit 12 during the manufacturing stage of the information processing device 1, for example. Alternatively, the program 12a may be distributed by a remote server device or the like and acquired by the information processing device 1 via communication. For example, the program 12a may be read from the recording medium 99 by a writing device and written to the storage unit 12 of the information processing device 1. The program 12a may be provided by distribution via a network, or it may be provided in a form recorded on the recording medium 99.

[0024] The model information storage unit 12b stores information about various pre-generated learning models. Information about learning models may include, for example, information indicating the configuration of the learning model and information such as predetermined values ​​of internal parameters. In this embodiment, the model information storage unit 12b stores information such as the configuration and parameters of a pre-trained video segmentation model (time-series data segmentation model), such as SAM (Segment Anything Model) 2. Note that the video segmentation model is not limited to SAM 2, and various models such as Segment and Track Anything (SAM-Track) or Track Anything may be used. In this embodiment, the model information storage unit 12b also stores information such as the configuration and parameters of a pre-trained image (still image) segmentation model (non-time-series data segmentation model), such as SegGPT.

[0025] Furthermore, information regarding these learning models may not be stored by the information processing device 1, but by a device other than the information processing device 1. In this case, the information processing device 1 may transmit input data for the learning model to the other device and obtain output data that the learning model outputs in response to this input data from the other device. Also, the machine learning process that generates these learning models may be performed by the information processing device 1, or by a device other than the information processing device 1.

[0026] The communication unit 13 is connected to the scanning electron microscope 102 via a cable such as a communication line or signal line, and transmits and receives data with the scanning electron microscope 102 via this cable. In this embodiment, the communication unit 13 receives SEM image data of a substrate transmitted from the scanning electron microscope 102 and provides the received data to the processing unit 11. In this embodiment, SEM images are exchanged between the scanning electron microscope 102 and the information processing device 1 via communication, but this is not the only method, and SEM images may be exchanged via a recording medium such as a memory card.

[0027] The display unit 14 is configured using a liquid crystal display or the like, and displays various images and characters based on the processing of the processing unit 11. In this embodiment, the display unit 14 displays, for example, SEM images acquired from the scanning electron microscope 102, and displays information related to the results of region detection processing and length measurement processing performed based on these SEM images.

[0028] The operation unit 15 receives user input and notifies the processing unit 11 of the received input. For example, the operation unit 15 receives user input via a mechanical button or an input device such as a touch panel provided on the surface of the display unit 14. Alternatively, the operation unit 15 may be an input device such as a mouse and a keyboard, and these input devices may be configured to be detachable from the information processing device 1.

[0029] The memory unit 12 may be an external storage device connected to the information processing device 1. The information processing device 1 may be a multicomputer comprising multiple computers, or it may be a virtual machine virtually constructed by software. Furthermore, the information processing device 1 is not limited to the above configuration, and may not include, for example, a display unit 14 and an operation unit 15.

[0030] Furthermore, in the information processing device 1 according to this embodiment, the processing unit 11 reads and executes the program 12a stored in the storage unit 12, thereby realizing the image acquisition unit 11a, target area acquisition unit 11b, area detection unit 11c, detection result determination unit 11d, length measurement processing unit 11e, and display processing unit 11f, etc., as software-based functional units in the processing unit 11. In this figure, the functional units of the processing unit 11 that relate to area detection processing and length measurement processing based on SEM images are shown, and functional units related to other processing are omitted from the illustration.

[0031] The image acquisition unit 11a performs the process of acquiring SEM images of the substrate processed by the substrate processing apparatus 101, which are captured by the scanning electron microscope 102. The image acquisition unit 11a communicates with the scanning electron microscope 102 via the communication unit 13 to acquire the SEM images of the substrate captured by the scanning electron microscope 102, and stores the acquired SEM images in the storage unit 12. The image acquisition unit 11a also performs pattern detection processing on the SEM images acquired from the scanning electron microscope 102 to detect multiple patterns on the substrate, and acquires image data for each pattern by extracting an image for each pattern. The pattern detection processing can be performed, for example, by matching with a pre-stored template image.

[0032] The target area acquisition unit 11b performs a process to acquire the settings for area detection to be performed on the image data for each pattern acquired by the image acquisition unit 11a. In this embodiment, the target area acquisition unit 11b acquires sample (reference) data indicating the area to be detected for one image data selected appropriately from among a plurality of image data acquired from the SEM image. For example, the target area acquisition unit 11b selects one image data appropriately from a plurality of image data and displays it on the display unit 14, accepts input from the user for area selection for the displayed image data, and acquires data of a mask image indicating the selected area as the setting for the area to be detected. Alternatively, the target area acquisition unit 11b may select one image data appropriately from a plurality of image data, input the selected image data into an image segmentation model such as SegGPT, and acquire data of a mask image of the segmentation result obtained as the setting for the area to be detected. Alternatively, the target area acquisition unit 11b may display the segmentation result of the image segmentation model on the display unit 14, accept user modifications to the area shown as this segmentation result, and acquire data of a mask image indicating the modified area as the setting for the area to be detected.

[0033] The region detection unit 11c performs target region detection on multiple image data acquired by the image acquisition unit 11a based on the target region setting acquired by the target region acquisition unit 11b. In this embodiment, the region detection unit 11c performs region detection on multiple non-time-series image data using a video segmentation model such as SAM2. The region detection unit 11c inputs the multiple image data into the video segmentation model in an appropriate order and sequentially acquires the segmentation results output by the video segmentation model accordingly, thereby obtaining the region detection result for each image data.

[0034] Furthermore, the information processing device 1 according to this embodiment stores intermediate information obtained during the segmentation process by the video segmentation model for a single image data, in order to improve the accuracy of region detection, and feeds this information back into the segmentation of the next image data. The video segmentation model, such as SAM2, used by the information processing device 1 according to this embodiment is equipped with a mechanism called "Attention" that identifies which part of the input data should be focused on. The information processing device 1 according to this embodiment is expected to improve the accuracy of region detection by feeding back the intermediate information obtained during the segmentation process by the video segmentation model to this Attention mechanism, thereby identifying the area to be focused on in the next segmentation process. This utilizes the characteristic that the multiple image data handled in the information processing system according to this embodiment are not time-series image data, but are images obtained by extracting similar patterns on a substrate, and are similar to each other, resulting in similar region detection results.

[0035] The detection result determination unit 11d performs a process to determine whether the result of region detection by the region detection unit 11c is correct or incorrect. The information processing device 1 stores, for example, the detection results of previously performed region detections (correct detection results and / or incorrect detection results) in the storage unit 12, and the detection result determination unit 11d determines whether the current result is correct or incorrect based on a comparison between past detection results and the current detection result. For example, the detection result determination unit 11d can calculate the similarity between past correct detection results and the current detection result, and determine whether the current detection result is correct or incorrect based on whether the calculated similarity exceeds a predetermined threshold.

[0036] Furthermore, as described above, the information processing device 1 according to this embodiment stores intermediate information obtained during the region detection process and feeds it back to subsequent region detection. However, it stores the intermediate information only if the detection result determination unit 11d determines that the detection result is correct, and does not store the intermediate information or provide feedback if it determines that the detection result is incorrect.

[0037] The length measurement processing unit 11e performs a process to measure the length of predetermined locations on the substrate pattern based on the data of the mask image obtained as a result of region detection by the region detection unit 11c. In this embodiment, the length measurement locations on the mask image are predetermined by the user, and the setting information of the length measurement locations is stored in the storage unit 12 in advance. The length measurement processing unit 11e determines the length measurement locations based on the stored setting information for the mask image of the region detection result, and measures the length of the determined length measurement locations. Note that the locations to be measured by the length measurement processing unit 11e may not be predetermined by the user, but may be determined each time, for example, by using a learning model generated in advance by machine learning. Any method may be used to determine the length measurement locations.

[0038] The display processing unit 11f performs the process of displaying various characters and images on the display unit 14. In this embodiment, the display processing unit 11f displays information on the display unit 14 such as, for example, an SEM image acquired from the scanning electron microscope 102, images of each pattern extracted from the SEM image, the results of detecting the area to be measured for each image, and the measurement results of each pattern.

[0039] <Region Detection Processing> Figure 4 is a schematic diagram illustrating the outline of the region detection processing performed by the information processing device 1 according to this embodiment. The information processing device 1 according to this embodiment detects similar patterns on the substrate from an SEM image obtained by photographing the substrate that has undergone substrate processing, and extracts the image region of each detected pattern to obtain a plurality of image data in which the patterns to be measured are individually captured as input image data. In this embodiment, the information processing device 1 obtains a mask image indicating the target region for measurement processing for each image data by performing region detection processing using the moving image segmentation model 110 and the image segmentation model 120 based on these acquired plurality of image data.

[0040] The video segmentation model 110 classifies multiple image data that have sequential dependencies between them into regions that satisfy the conditions input as prompts and regions that do not, and outputs a mask image as a classification result that shows which region each pixel of the image data belongs to. The video segmentation model 110 receives multiple image data in sequential order, stores intermediate information obtained during the processing of one image data, and uses it for processing the next image data, thereby improving the accuracy of segmentation processing for multiple image data that have sequential dependencies. Existing learning models such as SAM2 can be used for the video segmentation model 110.

[0041] The image segmentation model 120 classifies a single image data into regions that satisfy the conditions input as prompts and regions that do not, and outputs a mask image as a classification result that indicates which region each pixel of the image data belongs to. The image segmentation model 120 does not perform segmentation processing that considers the temporal dependencies between preceding and succeeding regions, and outputs independent segmentation results for each image data even if multiple image data are input sequentially. Existing learning models such as SegGPT can be used for the image segmentation model 120. However, the image segmentation model 120 may also be used by turning off the intermediate information feedback function of the video segmentation model 110.

[0042] The information processing device 1 appropriately selects one image data from among multiple acquired image data and displays it on the display unit 14. By accepting operations such as mouse clicks on the displayed image, it accepts input of one or more coordinates included in the detection target area as a user prompt. The information processing device 1 inputs the selected image data and the coordinate information of the received user prompt to the image segmentation model 120 and obtains a mask image of the segmentation result output by the image segmentation model 120. The information processing device 1 treats the combination of the selected image data and this mask image as reference data to be input as a prompt to the moving image segmentation model 110.

[0043] The information processing device 1 may store this reference data in the storage unit 12 and reuse it in subsequent region detection processing. If valid reference data is stored, the above procedure for generating reference data using the image segmentation model 120 may be omitted. Furthermore, the method for creating reference data is not limited to the method described above. For example, instead of using the image segmentation model 120, data obtained by the user performing operations such as filling in the target region on the displayed image may be acquired as reference data.

[0044] After obtaining the necessary reference data, the information processing apparatus 1 performs region detection processing on a plurality of image data using the moving image segmentation model 110. The information processing apparatus 1 selects one piece of image data in a predetermined order from the plurality of image data and inputs it to the moving image segmentation model 110, and inputs the reference data created in advance as a prompt to the moving image segmentation model 110. The moving image segmentation model 110 outputs a mask image indicating which pixels are included in the image region that satisfies the conditions of the prompt according to the input image data and the reference data of the prompt. The information processing apparatus 1 acquires the mask image output by the moving image segmentation model 110, and uses this mask image as the detection result of the measurement target region. The information processing apparatus 1 can obtain a plurality of region detection results for the plurality of image data by sequentially performing the input of the image data to the moving image segmentation model 110 and the acquisition of the mask image output by the moving image segmentation model 110 for the plurality of image data. Note that the information processing apparatus 1 may use one common reference data as the reference data input as a prompt for the plurality of image data.

[0045] FIG. 5 is a schematic diagram for explaining a configuration example of the moving image segmentation model 110. The moving image segmentation model 110 according to the present embodiment includes, for example, an image encoder 111, an Attention mechanism 112, a prompt encoder 113, a mask decoder 114, a memory encoder 115, an intermediate information storage unit 116, and the like. The image encoder 111 converts the input image data into feature amounts of a predetermined size. The Attention mechanism 112 performs an operation of emphasizing the attention points based on the intermediate information stored in the intermediate information storage unit 116 on the feature amounts output by the image encoder 111. When a plurality of intermediate information is stored in the intermediate information storage unit 116, the Attention mechanism 112 repeatedly performs an operation of emphasizing the attention points for each intermediate information.

[0046] The prompt encoder 113 converts the information of the prompt that is a condition for segmentation into a feature amount of a predetermined size. In the present embodiment, the prompt input to the prompt encoder 113 is reference data of a target region created using the moving image segmentation model 110. The mask decoder 114 generates a mask image that is a detection result of the target region based on the feature amount emphasizing the attention point output by the Attention mechanism 112 and the feature amount output by the prompt encoder 113. The mask image output by the mask decoder 114 becomes the output data of the moving image segmentation model 110.

[0047] The memory encoder 115 generates intermediate information for feeding back the attention point to the Attention mechanism 112 based on the mask image output by the mask decoder 114 and the feature amount output by the image encoder 111. The intermediate information storage unit 116 stores the intermediate information output by the memory encoder 115 and gives the stored intermediate information to the Attention mechanism 112. Note that the intermediate information storage unit 116 can store a preset number of intermediate information. The intermediate information storage unit 116 stores the intermediate information output by the memory encoder 115 in order, and when the number of stored intermediate information exceeds the set number, deletes the old ones in order.

[0048] However, the information processing apparatus 1 according to the present embodiment does not store intermediate information for all the input image data, but determines the correctness of the result of the region detection and stores the intermediate information related to the input image data for which a correct result is obtained in the intermediate information storage unit 116. When it is determined that the result of the region detection is incorrect, the information processing apparatus 1 discards the intermediate information without storing it.

[0049] The information processing device 1, for example, stores the correct region detection results in the storage unit 12, and determines whether the region detection results output by the video segmentation model 110 are correct or incorrect by comparing them with the correct results stored in the storage unit 12. The information processing device 1 can, for example, calculate the similarity between the mask image of the detection result obtained from the video segmentation model 110 and the mask image of the stored correct detection result, and if the calculated similarity exceeds a predetermined threshold, it can determine that the new detection result is correct. The method for determining the correctness of the region detection results is not limited to the method described above, and various methods may be employed.

[0050] Furthermore, after determining whether the region detection result is correct or incorrect, the information processing device 1 may store the original image data, the region detection result, and the correct / incorrect determination result in the storage unit 12 in association with each other. The information processing device 1 can use the region detection result that has been determined to be correct as data for subsequent correct / incorrect determinations. The information processing device 1 may also perform re-detection of image data that has been determined to have an incorrect detection result using the same procedure after completing region detection on other image data. In the region detection according to this embodiment, intermediate information obtained during the region detection process can be fed back to the next detection process, so the detection accuracy can be expected to improve with each repetition of region detection. For this reason, even if a correct detection result is not obtained initially, the information processing device 1 can perform region detection processing on a certain amount of image data and then perform re-detection, which may result in obtaining a correct detection result on the re-detection.

[0051] Figure 6 is a flowchart showing an example of the procedure for region detection processing performed by the information processing device 1 according to this embodiment. The image acquisition unit 11a of the processing unit 11 of the information processing device 1 according to this embodiment acquires a plurality of image data in which patterns to be targeted for region detection are individually depicted by extracting image regions in which predetermined patterns are depicted from the SEM images captured by the scanning electron microscope 102 (step S1).

[0052] Next, the target area acquisition unit 11b of the processing unit 11 performs a process to acquire information about the area to be detected, that is, reference data to be input as a prompt to the moving image segmentation model 110 (step S2). At this time, the target area acquisition unit 11b appropriately selects one image data from the multiple image data acquired in step S1, for example, and displays it on the display unit 14, and accepts operations from the user such as coordinate selection for the displayed image data. The target area acquisition unit 11b inputs the selected image data and the coordinate information received from the user to the image segmentation model 120, and obtains the reference data by acquiring the mask image output by the image segmentation model 120.

[0053] The region detection unit 11c of the processing unit 11 appropriately selects one image data from among the multiple image data acquired in step S1 (step S3). The region detection unit 11c inputs the one image data selected in step S3 to the video segmentation model 110 (step S4). At this time, the region detection unit 11c inputs the target region setting (reference data) acquired in step S2 as a prompt to the video segmentation model 110. The region detection unit 11c obtains the region detection result output by the video segmentation model 110 in response to the image data input in step S4 (step S5).

[0054] The detection result determination unit 11d of the processing unit 11 determines whether the region detection result obtained in step S5 is correct or not (step S6). At this time, the detection result determination unit 11d can determine the correctness of the current region detection result based on a comparison with the correct region detection result stored in the storage unit 12. If it is determined that the region detection result is correct (S6: YES), the detection result determination unit 11d stores the intermediate information obtained from the memory encoder 115 of the moving image segmentation model 110 in the intermediate information storage unit 116 during the process of obtaining this region detection result (step S7), and proceeds to step S8. If it is determined that the region detection result is incorrect (S6: NO), the detection result determination unit 11d proceeds to step S8 without storing the intermediate information.

[0055] The processing unit 11 determines whether to terminate region detection based on whether the processing in steps S3 to S7 has been performed on all image data acquired in step S1 (step S8). If it is determined that processing has not been performed on all image data to be determined and region detection should not be terminated (S8: NO), the processing unit 11 returns to step S3, selects another image data, and repeats the same processing. If it is determined that processing has been completed on all image data and region detection should be terminated (S8: YES), the length measurement processing unit 11e of the processing unit 11 performs length measurement processing on the objects captured in each image data based on the region detection results (step S9). The display processing unit 11f of the processing unit 11 displays the result of the length measurement processing in step S9 on the display unit 14 (step S10) and terminates the processing.

[0056] <Modifications Regarding Correct / Failure Determination> (Modification 1) Figure 7 is a schematic diagram illustrating the configuration of Modification 1 regarding correct / failure determination. The information processing device 1 according to Modification 1 includes two video segmentation models: a first video segmentation model 110a and a second video segmentation model 110b. The first video segmentation model 110a is the same model as the video segmentation model 110 described above, and the second video segmentation model 110b corresponds to a model in which the intermediate information feedback function is removed from the video segmentation model 110. The first video segmentation model 110a and the second video segmentation model 110b have the same internal parameters, etc., determined by machine learning.

[0057] In other words, the information processing device 1 is equipped with one video segmentation model 110 and controls the use of an intermediate information feedback function to switch between using and not using it. The video segmentation model 110 that uses the feedback function can be used as the first video segmentation model 110a, and the video segmentation model 110 that does not use the feedback function can be used as the second video segmentation model 110b.

[0058] The information processing device 1 according to Modification 1 inputs the input image data to be subjected to region detection processing into a first video segmentation model 110a to obtain a mask image of the first detection result, and also inputs the same image data into a second video segmentation model 110b to obtain a mask image of the second detection result. These first and second detection results are input to a comparison unit 121, where the two detection results are compared. In Modification 1, the comparison unit 121 performs a calculation to determine the value of IoU (Intersection over Union) for the two mask images of the detection results. IoU is an index that shows the extent to which the two regions overlap.

[0059] The comparison result of the comparison unit 121, i.e., the calculated IoU value, is provided to the selection unit 122. The selection unit 122 performs a process to select which detection result to adopt based on the comparison of the two detection results. In the modified example 1, the selection unit 122 determines whether the IoU value calculated by the comparison unit 121 exceeds a predetermined threshold. If the IoU exceeds the threshold, the selection unit 122 adopts the first detection result as the result of region detection processing on the input image data. The selection unit 122 also stores the intermediate information obtained in the process by which the first video segmentation model 110a calculates the first detection result in the intermediate information storage unit 116 of the first video segmentation model 110a and feeds it back.

[0060] If the IoU does not exceed the threshold, the selection unit 122 adopts the second detection result as the result of region detection processing on the input image data. In this case, the selection unit 122 does not provide intermediate information feedback to the first video segmentation model 110a. However, the selection unit 122 may store the intermediate information obtained in the process by which the second video segmentation model 110b calculates the second detection result in the intermediate information storage unit 116 of the first video segmentation model 110a and provide feedback.

[0061] (Modification 2) Figure 8 is a schematic diagram illustrating the configuration of Modification 2 relating to correctness determination. In this embodiment, the information processing device 1 performs region detection processing on multiple image data extracted from the SEM image. The information processing device 1 according to Modification 2 sequentially inputs multiple (N) image data into the moving image segmentation model 110 in the order they were extracted from the SEM image, and sequentially acquires mask images as a result of region detection. The information processing device 1 according to Modification 2 discards the results of the first round of N region detections on these N image data. However, the information processing device 1 performs a correctness determination of the detection results and stores intermediate information regarding the correct detection results in the intermediate information storage unit 116.

[0062] After completing this first round of processing, the information processing device 1 according to Modification 2 sequentially inputs N image data into the moving image segmentation model 110 in the reverse order of the first round, and sequentially obtains N mask images as the result of region detection. The information processing device 1 according to Modification 2 treats the region detection results obtained in the second round of region detection processing as the final region detection results. The information processing device 1 according to Modification 2 may or may not provide feedback of intermediate information in the second round of region detection processing.

[0063] In this example, the order in which image data is input in the second round is the reverse of the order in the first round, but this is not the only option. The information processing device 1 may, for example, input the first and second rounds in the same order, or it may, for example, input multiple image data randomly in the second round, or it may input the image data in the second round in any other order.

[0064] <Processing Order of Image Data> In the above-described embodiment, there is no specified order in which the multiple image data extracted from the SEM image are input to the video segmentation model 110. The information processing device 1 may, for example, input the image data to the video segmentation model 110 in the order in which it was extracted from the SEM image, or it may input the image data in any order. However, by adopting the method for determining the processing order described below, it is expected that the accuracy of the region detection processing using the video segmentation model 110 can be improved.

[0065] (Method for Determining Processing Order 1) When performing region detection processing of multiple image data using the video segmentation model 110, accuracy can be expected to improve by inputting similar images sequentially. Therefore, the information processing device 1 calculates the similarity between two image data for multiple image data extracted from the SEM image, and determines the processing order of the multiple image data based on the calculated similarity. For example, the information processing device 1 determines the input order of the multiple image data so that the average value of the similarity between the sequentially input image data is the highest. This method for determining the input order is a so-called combinatorial optimization problem, and various existing methods can be adopted. Alternatively, the information processing device 1 may randomly select the first image data, select the image data most similar to this image data, and repeat this process sequentially to input multiple image data.

[0066] (Processing Order Determination Method 2) The image data input to the motion image segmentation model 110 is extracted from SEM images of a substrate that has undergone substrate processing, and positional information such as coordinates indicating where on the substrate the object depicted in each image data is located can be linked to the image data and stored. If such coordinate information is available, the information processing device 1 can acquire the positional information along with the image data that is the target of the region detection processing, and determine the processing order of the image data based on the positional information. For example, if there is knowledge that there may be a difference in the results of substrate processing between the central part and the peripheral part of the substrate, the information processing device 1 can expect to input similar image data sequentially by inputting the image data in order of proximity (or distance) from the center of the substrate.

[0067] <Screen Display> Figure 9 is a schematic diagram showing an example of the operation screen for the region detection process displayed by the information processing device 1 according to this embodiment. The illustrated operation screen includes an "input data" area that displays multiple image data to be processed side by side, an "output result" area that displays the results of region detection, a button labeled "Automatic Reference Data Selection", a button labeled "Create Reference Mask", a button labeled "Execute", a checkbox labeled "Reinfer Abnormal Data", and a checkbox labeled "Bidirectional Inference".

[0068] The "Input Data" area is a region for displaying multiple image data extracted from SEM images in a list. In this example, n images, from Data 1 to Data n, are displayed. The information processing device 1 may, for example, display multiple image data in the "Input Data" area in processing order, accept user requests to rearrange the image data using operations such as drag and drop, and input the rearranged image data into the video segmentation model 110.

[0069] The "Output Data" area is for displaying a list of the original image data with the detected mask image superimposed on it, as a result of region detection. The "Output Data" area further contains the "Normal Group" area, the "Abnormal Group" area, and the "Abnormal Reinference Results" area. The "Normal Group" area is for displaying a list of data for which the region detection result was determined to be correct (normal). The "Abnormal Group" area is for displaying a list of data for which the region detection result was determined to be incorrect (abnormal). The "Abnormal Reinference Results" area is for displaying a list of results for which region detection has been performed again using the video segmentation model 110 on the data displayed in the "Abnormal Group" area.

[0070] The "Automatic Reference Data Selection" button is used to instruct the information processing device 1 to automatically select image data for creating reference data from among the multiple image data displayed in the "Input Data" area. When this button is pressed, the information processing device 1 will, for example, calculate the similarity between multiple image data and select the image data with the highest average similarity with other multiple images as the data for creating reference data. Note that the above method of data selection is just one example and is not limited to this method.

[0071] The "Create Reference Mask" button is used by the user to create a mask image of reference data based on image data automatically selected by the information processing device 1 or image data selected by the user from the "Input Data" area. When this button is pressed, the information processing device 1 displays a screen for creating the mask image (not shown in the figure) on the display unit 14, accepts the user's input for creating the mask image, and creates the mask image. For example, the information processing device 1 displays the selected image data on the mask image creation screen, accepts the user's operation to select coordinates by clicking on several points on the displayed image, and uses the received coordinate information as user prompt information. As shown in Figure 4, the information processing device 1 can create a mask image by inputting the selected image data and the user's prompt into the image segmentation model 120.

[0072] The "Execute" button is used to instruct the information processing device 1 to perform region detection processing on multiple image data displayed in the "Input Data" area. When this button is pressed, the information processing device 1 inputs the multiple image data into the video segmentation model 110 in an appropriate order and executes the region detection processing by sequentially acquiring the mask images of the region detection results output by the video segmentation model 110.

[0073] The "Reinfer Abnormal Data" checkbox allows you to choose whether or not to perform reinference (perform region detection again) on image data for which the region detection process has been determined to be abnormal. If this checkbox is checked when the "Execute" button is pressed, the information processing device 1 performs reinference on the image data for which the detection result has been determined to be abnormal and displays the reinference result in "Abnormal Reinference Result".

[0074] The "Bidirectional Inference" checkbox allows the user to choose whether or not to perform "bidirectional inference," which involves detecting regions on multiple image data in both forward and reverse directions. If this checkbox is checked when the "Execute" button is pressed, the information processing device 1, as shown in Figure 8, performs region detection processing on multiple image data in an appropriate order in the first pass and stores the intermediate information. In the second pass, it performs region detection processing in the reverse order of the first pass, and the results obtained become the final region detection results.

[0075] The "Bidirectional Inference" checkbox mentioned above is for performing two region detections, one in forward and one in reverse order. However, the order in which the two region detections are performed is not limited to forward and reverse order. For example, various orders can be adopted, such as performing two region detections in forward order, or performing the second round in a random order. Therefore, if a different order is adopted, a checkbox for selecting to perform the second round of region detection in a different order may be provided instead of the "Bidirectional Inference" checkbox, or in conjunction with the "Bidirectional Inference" checkbox. Alternatively, the system may be configured so that the user can select the order in which the second round of region detection is performed, for example, via a pull-down menu.

[0076] Furthermore, the screen configuration shown in Figure 9 is merely an example and is not limited to this configuration. The information processing device 1 may display any screen configuration to display information and receive user input.

[0077] <Other Examples of Detectable Objects> In the above-described embodiment, an example was shown in which a region related to the measurement of an object is detected based on an SEM image of an object formed on a substrate that has undergone substrate processing. However, the application of the information processing system according to this embodiment is not limited to this. The information processing system according to this embodiment performs processing to detect a desired region in multiple image data in which similar objects are photographed, and the objects are not limited to those formed on a substrate, but can be various objects.

[0078] Figure 10 is a schematic diagram illustrating an example of applying the information processing system according to this embodiment to probe alignment. When performing electrical testing on a substrate that has undergone substrate processing, a probe (inspection needle) is brought into contact with the substrate surface, and it is necessary to adjust the contact position of the probe with respect to the substrate at this time. In this example, the information processing system acquires an image of the substrate surface in contact with the probe, and the information processing device 1 performs a process to recognize the contact position of the probe, that is, a process to detect the area of ​​the tip portion of the probe that is in contact with the substrate in the captured image.

[0079] The information processing device 1 acquires image data of the substrate surface captured by a camera or the like while the prober is in contact with the substrate. A schematic example of this image data is shown at the top of Figure 10, where a roughly circular black area with a white cross pattern inside represents the tip of the prober in contact with the substrate. The shapes of the tips of multiple probers captured in the image are similar. The information processing device 1 detects the prober tips from the image data acquired from the camera or the like, and divides the image data by extracting an image of a predetermined size for each detected prober tip.

[0080] The information processing device 1 inputs the divided image data into a motion image segmentation model 110 in an appropriate order and obtains the region detection results output by the motion image segmentation model 110. The information processing device 1 generates synthesized image data by superimposing the multiple mask images obtained as region detection results onto the original image data based on the extraction positions of the corresponding image data from the original image data. An example of this generated image data is shown at the bottom of Figure 10, where the region in the original image data where the probe tip is captured is detected and masked.

[0081] The information processing device 1 can calculate the position where the tip of the prober is in contact with the substrate surface based on the region detection result of the tip of the prober, and can perform a correct or incorrect determination of the calculated position.

[0082] <Summary> In the information processing system according to this embodiment with the above configuration, the information processing device 1 is equipped with a moving image segmentation model 110 that sequentially accepts time-series image data as input and sequentially outputs the detection result of the region in which an object is photographed for each image data. The information processing device 1 acquires a plurality of non-time-series image data of an object related to substrate processing, such as a pattern formed on a substrate that has undergone substrate processing, inputs the acquired image data to the moving image segmentation model 110 to acquire region detection results, and displays (outputs) the acquired region detection results. As a result, the information processing system according to this embodiment is expected to accurately detect regions in which similar objects are photographed from a plurality of non-time-series image data related to substrate processing using the moving image segmentation model 110 which handles time-series image data (moving images).

[0083] Furthermore, in the information processing system according to this embodiment, the information processing device 1 determines whether the region detection result obtained from the video segmentation model 110 is correct or incorrect. If the information processing device 1 determines that the detection result is correct, it stores the intermediate information generated by the video segmentation model 110 during the process of outputting this detection result in the intermediate information storage unit 116 and feeds back the stored intermediate information to the region detection processing for the next image data. If the information processing device 1 determines that the detection result is incorrect, it does not store the intermediate information. As a result, the information processing system according to this embodiment can feed back and use the intermediate information obtained when a correct detection result is obtained, and is therefore expected to improve the accuracy of region detection.

[0084] Furthermore, in the information processing system according to this embodiment, the information processing device 1 stores the correct detection result in the storage unit 12, and determines whether the detection result is correct or incorrect by comparing the detection result output by the moving image segmentation model 110 with the stored detection result. As a result, the information processing system according to this embodiment is expected to easily determine whether the detection result output by the moving image segmentation model 110 is correct or incorrect.

[0085] Furthermore, in the information processing system according to this embodiment, the information processing device 1 stores image data for which the detection result has been determined to be incorrect, and after completing region detection for multiple acquired image data, it re-detects the stored image data. As a result, the information processing system according to this embodiment is expected to increase the likelihood of obtaining a correct detection result even for image data for which the initial detection process did not yield a correct result, by performing re-detection using the dynamic image segmentation model 110, which improves detection accuracy by feeding back intermediate information.

[0086] Furthermore, in the information processing system according to this embodiment, the information processing device 1 compares the first detection result output by the first video segmentation model 110a, which uses intermediate information, with the second detection result output by the second video segmentation model 110b, which does not use intermediate information, to determine whether the detection result is correct or incorrect. The information processing device 1 calculates an index (e.g., IoU) that indicates the degree of agreement between the mask image obtained as the first detection result and the mask image obtained as the second detection result, and adopts the first detection result if the calculated index exceeds a threshold, and adopts the second detection result if the index does not exceed the threshold. As a result, the information processing system according to this embodiment can perform region detection processing using two types of video segmentation models and adopt the detection result that has a high probability of being correct, and is therefore expected to improve the accuracy of region detection.

[0087] Furthermore, in the information processing system according to this embodiment, the information processing device 1 sequentially performs a first region detection using the moving image segmentation model 110 on multiple image data, and then performs a second region detection on the multiple image data in a different order than the first, and the result of the second detection is taken as the final region detection result. As a result, the information processing system according to this embodiment is expected to improve the accuracy of region detection on multiple image data.

[0088] Furthermore, in the information processing system according to this embodiment, the information processing device 1 calculates the similarity between multiple image data and determines the order in which the image data is input to the video segmentation model 110 based on the calculated similarity. As a result, the information processing system according to this embodiment can, for example, input similar image data consecutively to the video segmentation model 110, and is expected to improve the accuracy of region detection for multiple image data.

[0089] Furthermore, in the information processing system according to this embodiment, the information processing device 1 determines the order of image data to be input to the moving image segmentation model 110 based on the position of the pattern on the substrate as depicted in the image data. As a result, the information processing system according to this embodiment is expected to input similar image data consecutively to the moving image segmentation model 110, and is expected to improve the accuracy of region detection for multiple image data.

[0090] Furthermore, in the information processing system according to this embodiment, the information processing device 1 displays an operation screen on the display unit 14 that includes an "input data" area for displaying multiple acquired image data, a "normal group" area for displaying detection results determined to be correct, an "abnormal group" area for displaying detection results determined to be incorrect, and an "abnormal reinference result" area for displaying the results of re-detection. The information processing device 1 may also display an "abnormal data reinference" checkbox (first selection reception unit) on the operation screen that accepts the selection of whether or not to perform re-detection on image data for which the detection result was determined to be incorrect, and a "bidirectional inference" checkbox (second selection reception unit) that accepts the selection of whether or not to perform multiple detections in different orders for multiple image data. With these features, the information processing system according to this embodiment is expected to support users performing tasks such as region detection on multiple image data.

[0091] Furthermore, in the information processing system according to this embodiment, the moving image segmentation model 110 receives time-series image data along with reference image data with the detection results of the target object attached as a prompt, and outputs the detection results of the target object for each image data corresponding to the reference image data. The information processing device 1 includes an image segmentation model (non-time-series segmentation model) 120 that has been pre-machine-trained to receive image data in which the target object is photographed and prompts based on user input related to the target object as input, and to output detection results of regions corresponding to the information received for the image data. The information processing device 1 acquires image data and prompt information such as coordinate specification by the user and inputs it to the image segmentation model 120, acquires the detection results output by the image segmentation model 120, and uses the image data with the detection result mask image attached as the reference image data. Thus, the information processing system according to this embodiment is expected to support the creation of reference image data by the user.

[0092] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims, not in the sense described above, and all modifications within the meaning and scope equivalent to the claims are intended.

[0093] The matters described in each embodiment can be combined with each other. Furthermore, the independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. In addition, the claims use a form in which claims referencing two or more other claims (multi-claim form), but are not limited to this. A form in which multi-claims referencing at least one multi-claim (multi-multi-claim) may also be used.

[0094] 1 Information Processing Device (Computer) 11 Processing Unit 11a Image Acquisition Unit 11b Target Area Acquisition Unit 11c Area Detection Unit 11d Detection Result Determination Unit 11e Length Measurement Processing Unit 11f Display Processing Unit 12 Storage Unit 12a Program (Computer Program) 12b Model Information Storage Unit 13 Communication Unit 14 Display Unit 15 Operation Unit 101 Substrate Processing Device 102 Scanning Electron Microscope 110 Moving Image Segmentation Model (Time-Series Segmentation Model) 110a First Moving Image Segmentation Model (First Time-Series Segmentation Model) 110b Second Moving Image Segmentation Model (Second Time-Series Segmentation Model) 111 Image Encoder 112 Attention Mechanism 113 Prompt Encoder 114 Mask Decoder 115 Memory Encoder 116 Intermediate Information Storage Unit 120 Image segmentation model (non-time-series segmentation model) 121 Comparison section 122 Selection section

Claims

1. An information processing device equipped with a time-series segmentation model that sequentially accepts time-series image data as input and sequentially outputs detection results for the region in which an object is photographed for each image data; an information processing method comprising: acquiring multiple non-time-series image data of an object related to substrate processing; inputting the acquired image data into the time-series segmentation model to acquire detection results; and outputting the acquired detection results.

2. The information processing method according to claim 1, wherein the information processing device determines whether the acquired detection result is correct, and if it determines that the detection result is correct, it stores the intermediate information generated by the time-series segmentation model in order to output the detection result, and uses the stored intermediate information for region detection of the next image data.

3. The information processing method according to claim 2, wherein the information processing device does not store the intermediate information if it determines that the detection result is incorrect.

4. The information processing method according to claim 2 or 3, wherein the information processing device stores the correct detection result and determines whether the detection result is correct or incorrect based on a comparison between the detection result output by the time-series segmentation model and the stored detection result.

5. The information processing method according to claim 2 or 3, wherein the information processing device stores image data for which the detection result has been determined to be incorrect, and performs re-detection on the stored image data.

6. The information processing method according to any one of claims 1 to 3, wherein the information processing device inputs acquired image data to a first time-series segmentation model that performs region detection using intermediate information to obtain a first detection result, inputs acquired image data to a second time-series segmentation model that performs region detection without using intermediate information to obtain a second detection result, and determines whether the detection result is correct or incorrect based on a comparison of the acquired first detection result and the acquired second detection result.

7. The information processing method according to claim 6, wherein the information processing device calculates an index indicating the degree of agreement between the first detection result and the second detection result, adopts the first detection result if the calculated index exceeds a threshold, and adopts the second detection result if the calculated index does not exceed a threshold.

8. The information processing method according to any one of claims 1 to 3, wherein the information processing device sequentially performs a first region detection using the time-series segmentation model on a plurality of image data, and performs a second region detection using the time-series segmentation model on a plurality of image data in a different order from the first region detection.

9. The information processing method according to any one of claims 1 to 3, wherein the information processing device calculates the similarity between a plurality of acquired image data, and determines the order in which to perform region detection using the time-series segmentation model of the plurality of image data based on the calculated similarity.

10. The information processing method according to any one of claims 1 to 3, wherein the plurality of image data are image data of a plurality of patterns formed on the substrate that has undergone the substrate processing, and the information processing device determines the order in which to perform region detection using the time-series segmentation model of the plurality of image data based on the position of the patterns on the substrate.

11. The information processing method according to claim 2 or 3, wherein the information processing device displays a screen that includes an area for displaying the acquired plurality of image data, an area for displaying detection results determined to be correct, an area for displaying detection results determined to be incorrect, and an area for displaying the results of re-detection performed on image data for which the detection result was determined to be incorrect.

12. The information processing method according to claim 11, wherein the information processing device displays a first selection receiving unit on the screen that accepts a selection of whether or not to perform re-detection on image data for which the detection result has been determined to be incorrect.

13. The information processing method according to claim 11, wherein the information processing device displays a second selection receiving unit on the screen that accepts a selection of whether or not to perform multiple detections on the plurality of image data in different order.

14. The information processing method according to any one of claims 1 to 3, wherein the time-series segmentation model receives time-series image data along with reference image data to which the detection results of the object are attached, and outputs the detection results of the object for each image data corresponding to the reference image data.

15. The information processing method according to claim 14, wherein the information processing device includes a non-time-series segmentation model that receives image data of an object and information relating to a region to be detected for the object as input, and outputs a region detection result corresponding to the information for the image data, acquires image data of an object and information relating to a region to be detected for the object, inputs the acquired image data and the information to the non-time-series segmentation model, acquires the detection result output by the non-time-series segmentation model, and uses the data obtained by attaching the acquired detection result to the image data input to the non-time-series segmentation model as the reference image data.

16. The information processing method according to any one of claims 1 to 3, wherein the plurality of image data are image data of a plurality of patterns formed on a substrate that has undergone substrate processing.

17. A computer program that causes a computer equipped with a time-series segmentation model that sequentially accepts time-series image data as input and sequentially outputs detection results for the region in which an object is photographed for each image data to acquire multiple non-time-series image data of an object related to substrate processing, inputs the acquired image data into the time-series segmentation model to acquire detection results, and outputs the acquired detection results.

18. An information processing device comprising: a time-series segmentation model that sequentially accepts time-series image data as input and sequentially outputs detection results for the region in which an object is photographed for each image data; and a processing unit that performs detection processing using the time-series segmentation model, wherein the processing unit acquires a plurality of non-time-series image data of an object related to substrate processing, inputs the acquired image data to the time-series segmentation model to acquire detection results, and outputs the acquired detection results.

Images