Information processing method, computer program, and information processing device

Abstract

Provided are an information processing method, a computer program, and an information processing device that can be expected to detect, on the basis of image data obtained by capturing an image of a substrate, defects related to a plurality of inspection objects expected to be regularly arranged on the substrate.　An information processing method according to the present embodiment includes the information processing device acquiring image data obtained by imaging a plurality of inspection objects arranged on a substrate, detecting the plurality of inspection objects from the acquired image data, assessing regularity related to the arrangement of the plurality of inspection objects on the basis of the acquired image data, and detecting a defect of the substrate on the basis of the result of detecting the plurality of inspection objects and the result of assessing the regularity related to the arrangement of the plurality of inspection objects.

Description

Information Processing Method, Computer Program, and Information Processing Apparatus

[0001] The present disclosure relates to an information processing method, a computer program, and an information processing apparatus.

[0002] In Patent Document 1, an image estimation model created by machine learning using a captured image of a substrate before processing by a substrate processing apparatus and a captured image after processing is used to obtain an estimated image of an inspection target substrate after processing based on the captured image of the inspection target substrate before processing, and a substrate inspection apparatus that determines the presence or absence of defects in the inspection target substrate based on the captured image of the inspection target substrate after processing by the substrate processing apparatus and the estimated image estimated by the image estimation model has been proposed.

[0003] Japanese Patent Application Laid-Open No. 2024-96135

[0004] The present disclosure provides an information processing method, a computer program, and an information processing apparatus that can be expected to detect defects related to a plurality of inspection targets that are expected to be regularly arranged on a substrate based on image data obtained by photographing the substrate.

[0005] An information processing method according to an embodiment includes an information processing apparatus acquiring image data obtained by photographing a plurality of inspection targets arranged on a substrate, detecting the plurality of inspection targets from the acquired image data, determining the regularity of the arrangement of the plurality of inspection targets based on the acquired image data, and detecting a defect in the substrate based on the detection result of the plurality of inspection targets and the determination result of the regularity of the arrangement of the plurality of inspection targets.

[0006] According to the present disclosure, it can be expected to detect defects related to a plurality of inspection targets that are expected to be regularly arranged on a substrate based on image data obtained by photographing the substrate.

[0007] This is a schematic diagram illustrating the overview of the information processing system according to this embodiment. This is a schematic diagram illustrating the overview of the processing performed by the information processing device according to this embodiment. This is a schematic diagram illustrating the overview of the processing performed by the information processing device according to this embodiment. This is a block diagram illustrating an example configuration of the information processing device according to this embodiment. This is a schematic diagram showing an example configuration of a learning model used in object detection processing. This is a flowchart illustrating an example of the procedure for generating a learning model performed by the information processing device according to this embodiment. This is a flowchart illustrating an example of the procedure for determining regularity processing performed by the information processing device according to this embodiment. This is a schematic diagram illustrating an example of image data taken of a cross-section of a substrate with multiple holes formed therein. This is a flowchart illustrating an example of the procedure for defect detection processing performed by the information processing device according to this embodiment. This is a schematic diagram illustrating the defect detection processing performed by the information processing system according to Embodiment 2. This is a schematic diagram illustrating the overview of the information processing system according to Embodiment 3.

[0008] Specific examples of information processing systems according to the embodiments of this disclosure will be described below with reference to the drawings. However, this disclosure is not limited to these examples, and all changes are intended to be made within the meaning and scope of the claims as indicated by the claims.

[0009] <System Overview> Figure 1 is a schematic diagram illustrating the overview of the information processing system according to this embodiment. The information processing system according to this embodiment is configured to include an information processing device 1, a substrate processing device 101, and a scanning electron microscope 102, etc. The substrate processing device 101 is a device such as a process chamber that performs processing such as etching on a semiconductor wafer (substrate). The substrate is not limited to a semiconductor wafer, but may be various types such as a glass substrate or a substrate for a flat panel display. Furthermore, the substrate processing device 101 is not limited to one that performs etching as a substrate processing, but may be a device that processes a substrate on which a structure has been formed, such as by film deposition, exposure, coating and development, or ashing, or a device that performs processing to form a structure.

[0010] After the substrate has been processed by etching or other processes in the substrate processing apparatus 101, its surface shape and other characteristics are photographed using a scanning electron microscope 102. The scanning electron microscope 102 is a device that observes an object by irradiating it with an electron beam and detecting secondary electrons or transmitted electrons emitted from the object, and outputs image data of a so-called SEM (Scanning Electron Microscope) image of the object. The image data output by the scanning electron microscope 102 is provided to the information processing apparatus 1. In this embodiment, processing is performed on image data captured by the scanning electron microscope 102, but this is not limited to this, and processing may also be performed on images captured by, for example, a transmission electron microscope or an optical microscope.

[0011] Various structures of different shapes are formed on the surface of a substrate processed by the substrate processing apparatus 101. For example, as schematically shown in the image data in the center of Figure 1, multiple structures of the same shape may be formed on the substrate in a regular arrangement. The image data in this example is a photograph of a substrate on which multiple holes are formed as structures on the surface in a regular arrangement vertically and horizontally. In substrate processing in which multiple structures of the same shape are regularly formed on the surface of the substrate, some of the multiple structures may not be formed correctly and may be considered defective. The information processing system according to this embodiment targets these structures that are expected to be regularly formed on the substrate and performs processing to detect areas where the target is not formed correctly as defects. The defect detection processing is performed by the information processing apparatus 1, which outputs (displays) information such as the presence or absence of defects and the location of defects as defect detection results. For example, in the defect detection result shown at the bottom of Figure 1, the information processing apparatus 1 shows the location of the detected defects enclosed in a rectangular frame on the image data acquired from the scanning electron microscope 102.

[0012] In this embodiment, the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 are described as separate devices, but this is not limited to them. For example, the substrate processing apparatus 101 and the scanning electron microscope 102 may be a single device, the scanning electron microscope 102 and the information processing apparatus 1 may be a single device, or the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be a single device. Furthermore, each of the devices of the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be configured by combining multiple devices.

[0013] Figures 2 and 3 are schematic diagrams illustrating the general process performed by the information processing device 1 according to this embodiment. Figure 2 shows a case where no defect is detected, and Figure 3 shows a case where a defect is detected. The information processing device 1 according to this embodiment acquires image data of the target substrate captured by a scanning electron microscope 102. The upper part of Figures 2 and 3 shows an example of image data of multiple objects formed on the substrate, and this image data captures multiple holes that are regularly arranged in a 3x3 matrix.

[0014] The information processing device 1 according to this embodiment performs a process to detect multiple objects that are expected to be formed in a regular arrangement on the surface of a substrate as depicted in the acquired image data. For example, the information processing device 1 is equipped with a learning model that has been pre-trained to accept image data as input and output the positions of the objects depicted in the image data. The information processing device 1 inputs image data acquired from a scanning electron microscope 102 into this learning model and performs object detection processing from the image data by acquiring the position information of the objects output by the learning model. In the left center of Figures 2 and 3, a rectangular frame (bounding box) surrounding the detected objects is shown superimposed on the original image data.

[0015] Furthermore, the information processing device 1 according to this embodiment performs a process to determine the regularity of the arrangement of multiple objects based on the acquired image data. For example, the information processing device 1 calculates the autocorrelation value of the image data using an autocorrelation function and determines the presence or absence of regularity and the regularity based on the calculated autocorrelation value. The information processing device 1 repeatedly performs a process to calculate the autocorrelation value between the original image data and data obtained by sliding this image data by a predetermined number of pixels (e.g., 1 pixel) in the horizontal direction (x direction) or vertical direction (y direction), and by comparing the magnitudes of the multiple autocorrelation values ​​obtained in this way, it determines a period in which a high autocorrelation value can be obtained. In this way, the information processing device 1 can determine a period in which a high autocorrelation value can be obtained as a regularity of the arrangement of objects, for example, every 10 pixels in the horizontal direction and every 20 pixels in the vertical direction. The right center of Figures 2 and 3 shows an example in which a grid (lattice) indicating the positions in which multiple objects should be placed is generated based on the period obtained as regularity.

[0016] The information processing device 1 according to this embodiment performs a process to detect defects in objects that are expected to be regularly formed on a substrate, based on the object detection results obtained from image data using a learning model and the regularity obtained from the image data using an autocorrelation function. As shown in the lower part of Figures 2 and 3, the information processing device 1 superimposes the object detection results and a grid generated based on the regularity, and detects defects by determining whether or not an object is detected at the intersection of the grid. In the example shown in the lower part of Figure 2, objects are detected at all grid intersections. In contrast, in the example shown in the lower part of Figure 3, no object is detected at the central intersection of the grid, and this intersection is a defect.

[0017] <Device Configuration> Figure 4 is a block diagram showing an example configuration of the information processing device 1 according to this embodiment. The information processing device 1 according to this embodiment can be realized by installing a predetermined application program on a general-purpose information processing device such as a personal computer or a server computer. However, the information processing device 1 may be a dedicated information processing device that controls a substrate processing device 101 or a scanning electron microscope 102. The information processing device 1 according to this embodiment is configured to include a processing unit 11, a storage unit 12, a communication unit 13, a display unit 14, and an operation unit 15, etc. In this embodiment, the explanation will be given assuming that processing is performed by one information processing device 1, but the processing of the information processing device 1 may be distributed among multiple devices.

[0018] The processing unit 11 is composed of an arithmetic processing unit such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), NPU (Neural Network Processing Unit), or quantum processor, and a storage device such as ROM (Read Only Memory) and RAM (Random Access Memory). The processing unit 11 reads and executes a program 12a stored in the storage unit 12, and performs various processes such as detecting defects in objects that are expected to be regularly placed on the substrate as depicted in the image data.

[0019] The functions realized by the processing unit 11 can be implemented using any circuit or processing circuitry. For example, the circuit or processing circuitry can be implemented using a general-purpose processor, a special-purpose processor, an integrated circuit, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a conventional circuit. The functions realized by the processing unit 11 can be programmed using one or more programs stored in one or more memories, or configured in other ways to execute the disclosed functions. The functions of the processing unit 11 can be implemented using circuit or processing circuitry including combinations of these.

[0020] The storage unit 12 is configured using a large-capacity storage device such as a hard disk or an SSD (Solid State Drive). The storage unit 12 stores various programs executed by the processing unit 11, and various data necessary for the processing of the processing unit 11. In this embodiment, the storage unit 12 stores the program 12a executed by the processing unit 11. The storage unit 12 is also provided with a model information storage unit 12b that stores information about a trained learning model used when performing the process of detecting an object captured in image data.

[0021] In this embodiment, the program (computer program, program product) 12a is provided in a form recorded on a recording medium 99 such as a memory card or optical disc, and the information processing device 1 reads the program 12a from the recording medium 99 and stores it in the storage unit 12. However, the program 12a may also be written to the storage unit 12 during the manufacturing stage of the information processing device 1, for example. Alternatively, the program 12a may be distributed by a remote server device or the like and acquired by the information processing device 1 via communication. For example, the program 12a may be read from the recording medium 99 by a writing device and written to the storage unit 12 of the information processing device 1. The program 12a may be provided by distribution via a network, or it may be provided in a form recorded on the recording medium 99.

[0022] The model information storage unit 12b stores information about various pre-generated learning models. The information about the learning models may include, for example, information indicating the configuration of the learning model and information such as predetermined values ​​of internal parameters. In this embodiment, the information processing device 1 stores information about a learning model for detecting objects captured in image data in the model information storage unit 12b. This learning model may be a neural network such as a CNN (Convolutional Neural Network) or Transformer, or a deep neural network, or an existing object detection learning model such as YOLO (You Only Look Once) or DETR (DEtection Transformer).

[0023] Furthermore, information regarding these learning models may not be stored by the information processing device 1, but by a device other than the information processing device 1. In this case, the information processing device 1 may transmit input data for the learning model to the other device and obtain output data that the learning model outputs in response to this input data from the other device. Also, the machine learning process that generates these learning models may be performed by the information processing device 1, or by a device other than the information processing device 1.

[0024] The communication unit 13 is connected to the scanning electron microscope 102 via a cable such as a communication line or signal line, and transmits and receives data with the scanning electron microscope 102 via this cable. In this embodiment, the communication unit 13 receives image data of the substrate transmitted from the scanning electron microscope 102 and provides the received image data to the processing unit 11. In this embodiment, image data is exchanged between the scanning electron microscope 102 and the information processing device 1 via communication, but this is not the only method, and image data may be exchanged via a recording medium such as a memory card.

[0025] The display unit 14 is configured using a liquid crystal display or the like, and displays various images and characters based on the processing of the processing unit 11. In this embodiment, the display unit 14 displays, for example, image data acquired from a scanning electron microscope 102, and information regarding the results of defect detection processing performed based on this image data.

[0026] The operation unit 15 receives user input and notifies the processing unit 11 of the received input. For example, the operation unit 15 receives user input via a mechanical button or an input device such as a touch panel provided on the surface of the display unit 14. Alternatively, the operation unit 15 may be an input device such as a mouse and a keyboard, and these input devices may be configured to be detachable from the information processing device 1.

[0027] The memory unit 12 may be an external storage device connected to the information processing device 1. The information processing device 1 may be a multicomputer comprising multiple computers, or it may be a virtual machine virtually constructed by software. Furthermore, the information processing device 1 is not limited to the above configuration, and may not include, for example, a display unit 14 and an operation unit 15.

[0028] Furthermore, in the information processing device 1 according to this embodiment, the processing unit 11 reads and executes the program 12a stored in the storage unit 12, thereby realizing the image acquisition unit 11a, object detection unit 11b, regularity determination unit 11c, defect detection unit 11d, display processing unit 11e, and learning processing unit 11f, etc., as software-based functional units in the processing unit 11. In this figure, the functional units of the processing unit 11 that are related to defect detection processing based on image data are shown, and functional units related to other processing are omitted from the illustration.

[0029] The image acquisition unit 11a performs the process of acquiring image data of the substrate processed by the substrate processing device 101, which is then photographed by the scanning electron microscope 102. The image acquisition unit 11a communicates with the scanning electron microscope 102 via the communication unit 13 to acquire image data of the substrate photographed by the scanning electron microscope 102, and stores the acquired image data in the storage unit 12.

[0030] The object detection unit 11b performs a process to detect objects such as holes that are expected to be regularly formed on the substrate from the image data acquired by the image acquisition unit 11a. In this embodiment, the information processing device 1 stores a pre-machine-learned object detection model in the model information storage unit 12b, and the object detection unit 11b uses this learning model to detect objects. The object detection learning model, for example, accepts image data as input and outputs location information of the object. The object detection unit 11b can detect objects from the image data by inputting the image data acquired by the image acquisition unit 11a into the learning model and obtaining the location information output by the learning model.

[0031] The regularity determination unit 11c performs a process to determine the regularity of multiple objects captured in the image data acquired by the image acquisition unit 11a. In this embodiment, the regularity determination unit 11c calculates the autocorrelation value of the image data using an autocorrelation function and determines the regularity based on the calculated autocorrelation value. For example, the regularity determination unit 11c calculates the autocorrelation value of the target image data and the image data obtained by sliding this image data horizontally and / or vertically by a predetermined number of pixels. The regularity determination unit 11c obtains multiple autocorrelation values ​​by repeatedly performing the process of sliding the image data by a predetermined number of pixels and calculating the autocorrelation value, and calculates the period in which high autocorrelation values ​​appear. As a result, the regularity determination unit 11c can determine that the autocorrelation value is high at a predetermined period in the horizontal and / or vertical directions for the target image data, and outputs this period as the determination result.

[0032] The defect detection unit 11d performs a process to detect defects in the substrate captured in the image data, based on the object detection result by the object detection unit 11b and the regularity determination result by the regularity determination unit 11c. In this embodiment, the defect detection unit 11d compares the horizontal and / or vertical period obtained as a result of the regularity determination with the position of the object detected from the image data by the learning model, and determines whether the detected objects are arranged regularly according to the determined period. The defect detection unit 11d detects as defects locations where an object is not detected in a position where an object should exist according to the determined period, and locations where an object is detected in a position where an object should not exist according to the determined period, and outputs the location information of the detected defects.

[0033] The display processing unit 11e performs the process of displaying various characters and images on the display unit 14. In this embodiment, the display processing unit 11e displays various information on the display unit 14, such as image data acquired from the scanning electron microscope 102 and location information of defects detected from this image data.

[0034] The learning processing unit 11f performs machine learning processing to generate a learning model that the object detection unit 11b uses to detect objects from image data. In this embodiment, image data of a substrate on which multiple objects are regularly arranged is collected in advance, and information such as bounding boxes indicating the positions of the objects is added to this image data through annotation. The learning processing unit 11f generates a learning model for detecting objects from image data by performing machine learning processing using the image data with object position information attached as training data. The learning processing unit 11f may not generate a learning model for detecting objects from scratch, but may acquire a learning model that has been pre-trained and generate a desired learning model by performing additional machine learning processing on the acquired learning model. The process of generating the learning model may also be performed on a device other than the information processing device 1, in which case the information processing device 1 may acquire information on the trained learning model from the other device that performed machine learning and store it in the model information storage unit 12b. Furthermore, if sufficient object detection accuracy can be obtained using a learning model that has only been pre-trained without additional training, the information processing device 1 may acquire and use a learning model that has only been pre-trained without additional training.

[0035] <Object Detection Process> In the information processing system according to this embodiment, in order to detect defects in a substrate based on image data acquired from a scanning electron microscope 102, the information processing device 1 first performs a process to detect objects from the image data and a process to determine the regularity of the image data. In this embodiment, the object detection process and the regularity determination process are performed using the same image data as input, but there is no dependency between the processes, and the information processing device 1 may perform either process first or both processes in parallel.

[0036] Figure 5 is a schematic diagram showing one example configuration of a learning model used for object detection processing. In the information processing system according to this embodiment, a learning model 110 is generated in advance so that the information processing device 1 can perform the process of detecting an object from image data. In this embodiment, the learning model 110 used by the information processing device 1 for object detection processing receives image data as input and outputs position information of the object depicted in the input image data. The position information output by the learning model 110 is, for example, the coordinates of the bounding box surrounding the object in the image data.

[0037] Figure 6 is a flowchart showing an example of the procedure for generating a learning model 110 performed by the information processing device 1 according to this embodiment. The learning processing unit 11f of the processing unit 11 of the information processing device 1 according to this embodiment acquires image data of the substrate captured by the scanning electron microscope 102 (step S1). At this time, the learning processing unit 11f may acquire image data directly from the scanning electron microscope 102 by communicating with the scanning electron microscope 102, for example, via the communication unit 13. Alternatively, the learning processing unit 11f may acquire image data via a recording medium such as a memory card on which image data is recorded. Alternatively, the learning processing unit 11f may acquire image data by reading image data previously stored in the storage unit 12.

[0038] The learning processing unit 11f, for example, displays the image data acquired in step S1 on the display unit 14 and accepts an input operation by the user to specify the position of the object at the operation unit 15, thereby acquiring the position information of the object captured in the image data (step S2). The processing performed in step S2 is the process of accepting so-called annotation work by the user. The learning processing unit 11f associates the image data acquired in step S1 with the position information acquired in step S2 and stores it in the storage unit 12 as training data (step S3). The learning processing unit 11f determines whether the preparation for training to generate the training model 110 is complete based on whether conditions such as a sufficient amount of training data has been accumulated or the annotation work on previously stored image data has been completed have been met (step S4). If the preparation for training is not complete (S4: NO), the learning processing unit 11f returns to step S1 and repeats the processing of steps S1 to S3 to generate training data.

[0039] If the learning preparation is complete (S4: YES), the learning processing unit 11f reads out the learning data stored in the memory unit 12, that is, data in which image data and object position information are associated (step S5). The learning processing unit 11f performs machine learning using the learning data read out in step S5 (step S6) and generates a learning model 110. At this time, if the learning model 110 is a neural network model, the learning processing unit 11f can determine the internal parameters of the learning model 110 by machine learning using a method such as backpropagation and generate the learning model 110. The learning processing unit 11f stores the internal parameters of the learning model 110 obtained by the machine learning process in step S6 in the model information storage unit 12b (step S7) and terminates the process.

[0040] The information processing device 1 configures a learning model 110 based on the information stored in the model information storage unit 12b, acquires image data of the substrate captured by the scanning electron microscope 102 and inputs it to the learning model 110, and acquires the position information of the target object output by the learning model 110. As a result, the information processing device 1 can detect the target object captured in the image data.

[0041] <Regularity Determination Process> The information processing device 1 according to this embodiment performs a process to determine the regularity of multiple objects provided on a substrate, based on image data of the substrate captured by the scanning electron microscope 102, using an autocorrelation function that calculates the autocorrelation value of the image. Since the autocorrelation function and autocorrelation value are existing technologies, a detailed explanation is omitted in this embodiment.

[0042] Figure 7 is a flowchart illustrating an example of the procedure for regularity determination processing performed by the information processing device 1 according to this embodiment. The regularity determination unit 11c of the processing unit 11 of the information processing device 1 according to this embodiment acquires the image data to be processed by, for example, reading from the storage unit 12 (step S21). The regularity determination unit 11c calculates the autocorrelation value between the image data to be processed acquired in step S21 and the image obtained by sliding this image data by one pixel in the x direction (horizontal direction) (step S22). The regularity determination unit 11c stores the autocorrelation value calculated in step S22, along with the amount of slide (number of pixels) relative to the original image data, in the storage unit 12 (step S23). The regularity determination unit 11c repeatedly performs the slide in the x direction relative to the original image data for, for example, the size of the image data in the x direction, and determines whether the slide in the x direction has been completed to the end (step S24). If the sliding in the x-direction has not been completed (S24: NO), the regularity determination unit 11c returns to step S22 and repeatedly performs the process of calculating and storing the autocorrelation value with the original image data for the image that has been slid by one more pixel.

[0043] When the slide in the x direction is completed (S24: YES), the regularity determination unit 11c calculates the autocorrelation value between the image data of the processing target acquired in step S21 and the image obtained by sliding this image data by only one pixel in the y direction (vertical direction) (step S25). The regularity determination unit 11c stores the autocorrelation value calculated in step S25 together with the slide amount with respect to the original image data in the storage unit 12 (step S26). The regularity determination unit 11c repeats the slide in the y direction with respect to the original image data, for example, by the size of the image data in the y direction, and determines whether the slide in the y direction has been completed to the end (step S27). When the slide in the y direction has not been completed (S27: NO), the regularity determination unit 11c returns the process to step S25 and repeats the process of calculating and storing the autocorrelation value between the original image data and the image slid by one pixel further.

[0044] When the slide in the y direction is completed (S27: YES), the regularity determination unit 11c determines the regularity in the x direction and y direction of the substrate depicted in the original image data based on the plurality of combinations of the slide amount and autocorrelation value in the x direction stored in step S23 and the plurality of combinations of the slide amount and autocorrelation value in the y direction stored in step S26 (step S28), and ends the process.

[0045] In step S28, the regularity determination unit 11c determines the presence or absence of regularity in the x direction of the image data by determining, for example, whether the peak of the autocorrelation value appears periodically based on the change in the autocorrelation value accompanying the increase in the slide amount in the x direction. When it is determined that there is regularity, the regularity determination unit 11c calculates the period at which the peak of the autocorrelation value appears and outputs the calculated period as the determination result. Similarly for the y direction, the regularity determination unit 11c can determine the presence or absence of regularity and calculate the period when there is regularity. Thereby, the regularity determination unit 11c can obtain determination results such as a period of 10 pixels in the x direction and a period of 20 pixels in the y direction, for example. In this case, it can be determined that the objects are regularly provided at a period of 10 pixels in the x direction and at a period of 20 pixels in the y direction.

[0046] In the flowchart of FIG. 7, the regularity determination unit 11c independently performs sliding in the x direction and sliding in the y direction, thereby individually determining the regularity regarding the x direction and the regularity regarding the y direction. As a result, for example, the regularity regarding the x direction and the y direction, as represented by the grid shown on the right side in the middle row of FIG. 2, can be obtained. However, the regularity determined by the regularity determination unit 11c is not limited to only these two directions of the x direction and the y direction.

[0047] FIG. 8 is a schematic diagram showing other examples of the regularity regarding the arrangement of the objects. The figure shown in the upper row of FIG. 8 is an example in which the regularity in the diagonal direction is determined. The regularity determination unit 11c can determine the regularity in the 45-degree diagonal direction, for example, by sliding one pixel in both the x direction and the y direction and calculating the autocorrelation value. Also, the figure shown in the middle row of FIG. 8 is an example in which the regularity in the curve is determined. The regularity determination unit 11c can determine the regularity in the curve corresponding to the rotation direction, for example, not by sliding the image data but by rotating it and calculating the autocorrelation value.

[0048] Also, the figure shown in the lower row of FIG. 8 is an example in which the regularity in the vertical and horizontal directions is determined when there is a blank area where no object exists within a region where a plurality of objects are regularly arranged vertically and horizontally. The regularity determination unit 11c, for example, acquires the detection result of the object by the object detection unit 11b and divides the image data into a region where the object exists and a region where the object does not exist. The object detection unit 11b can similarly determine the regularity even for image data with a blank area by individually performing the above-described regularity determination process for one or more regions determined to be regions where the object exists.

[0049] Furthermore, instead of processing an image of the surface (top surface) of a substrate that has been processed by the substrate processing apparatus 101, the apparatus may process image data of a cross-section of the substrate. Figure 9 is a schematic diagram showing an example of image data of a cross-section of a substrate in which multiple holes have been formed. In the case of image data of the cross-sectional structure of a substrate, as shown in this figure, objects such as holes are regularly arranged in the horizontal direction (x direction), but objects are rarely arranged in the vertical direction (y direction). Therefore, the information processing apparatus 1 may determine the regularity in two dimensions (x and y directions) for image data of the top surface of the substrate, and determine the regularity in one dimension (x direction only) for image data of a cross-section of the substrate.

[0050] (Modified Version) Alternatively, the regularity determination unit 11c of the information processing device 1 may determine the regularity based on, for example, the object detection result by the object detection unit 11b, without using autocorrelation values. In the modified version, the regularity determination unit 11c of the information processing device 1 calculates the distance between two adjacent objects in the x-direction, and obtains multiple distance values ​​by calculating this distance for all detected objects. The regularity determination unit 11c sorts the obtained multiple distances and obtains the median of the distances. The regularity determination unit 11c can use this obtained median as the period in the x-direction where the objects are lined up. The regularity determination unit 11c can similarly use the median of the distances as the period in the y-direction.

[0051] Alternatively, the modified regularity determination unit 11c may remove a few percent of the values ​​above and below the calculated distances, calculate the average value of the remaining distances, and use the calculated average value as the period. For example, if 100 numerical values ​​are obtained as distances between objects, the regularity determination unit 11c can remove 25 values ​​each corresponding to the top and bottom 25%, and calculate the average value of the remaining 50 distances corresponding to the remaining 25% to 75%.

[0052] <Defect Detection Processing> The information processing device 1 according to this embodiment performs a process to detect defects in the substrate captured in the image data, based on the results of the object detection processing and the regularity determination processing described above. Figure 10 is a flowchart showing an example of the procedure for the defect detection processing performed by the information processing device 1 according to this embodiment. The image acquisition unit 11a of the processing unit 11 of the information processing device 1 according to this embodiment acquires image data of the substrate captured by the scanning electron microscope 102 (step S41). The object detection unit 11b of the processing unit 11 performs an object detection process to detect multiple objects, such as holes, that are expected to be regularly formed on the substrate from the image data acquired in step S41, using the learning model 110 stored in the model information storage unit 12b (step S42). At this time, the object detection unit 11b can perform the object detection process by inputting the acquired image data to the learning model 110 and acquiring the position information of the objects output by the learning model 110 accordingly. Furthermore, the regularity determination unit 11c of the processing unit 11 performs a regularity determination process to determine the regularity of multiple objects formed on the substrate as captured in the image data, based on the image data acquired in step S41 (step S43). The details of the regularity determination process performed in step S43 are shown in the flowchart of Figure 7 above.

[0053] Next, the defect detection unit 11d of the processing unit 11 generates a grid representing the positions of regularly arranged objects based on the periodic information obtained as a result of the regularity determination process in step S43 (step S44). The grid generated here is a grid obtained by drawing vertical lines at intervals in the x-direction and horizontal lines at intervals in the y-direction, starting from a predetermined position in the image data, as shown in the center right of Figures 2 and 3, and the intersections of the grid represent the positions of the objects. Note that the grids shown in Figures 2 and 3 are the cases where regularity is determined only in the x and y directions, and if regularity is determined in the diagonal direction or curved direction as shown in Figure 8, the grid will have lines drawn at intervals corresponding to these directions. The starting point of the grid is the image data position, for example, the position of the object in the upper left corner among the multiple objects detected in the object detection process (i.e., the position of the object closest to x=0, y=0). Also, for example, if the learning model 110 outputs the reliability of detection along with the position of the object for each object, the starting point may be the position of the object with the highest reliability.

[0054] Furthermore, the information processing device 1 does not need to actually generate such a grid image in step S44; it only needs to calculate the coordinates of the positions where objects should exist, which are represented as intersections of the grid. For example, the information processing device 1 can determine the starting point (x0, y0) from the result of the object detection process, and based on the period Tx in the x direction and the period Ty in the y direction obtained by the regularity determination process, it can calculate the positions where objects should exist as follows: (x0, y0), (x0 + Tx, y0), (x0 + 2Tx, y0), (x0 + 3Tx, y0), ..., (x0, y0 + Ty), (x0 + Tx, y0 + Ty), (x0 + 2Tx, y0 + Ty), ...

[0055] Next, the defect detection unit 11d compares the position of the object obtained as a result of the object detection process in step S42 with the position of the grid intersection obtained in step S44, and detects grid intersections where no corresponding object exists as defects (step S45). The defect detection unit 11d counts the number of defects detected in step S45 (step S46). The defect detection unit 11d stores the positions of the defects detected in step S45 and the number of defects counted in step S46 in the storage unit 12 (step S47).

[0056] The display processing unit 11e of the processing unit 11 displays defect information regarding defects in the substrate on the display unit 14 (step S48) based on the location and number of defects stored in step S47, and then terminates the process. In step S48, the display processing unit 11e superimposes, for example, a bounding box indicating the location of an object obtained in the object detection process of step S42 and a grid generated in step S44 onto the image data acquired in step S41. The display processing unit 11e then highlights the location of the defects detected in step S45 on these images by, for example, surrounding them with a rectangular frame of a different color than the bounding box, thereby presenting the information in a manner that is easy for the user to recognize. For example, the display processing unit 11e can provide the user with information regarding defects detected on the substrate by displaying an image on the display unit 14 similar to the schematic diagrams shown in the lower part of Figures 2 and 3.

[0057] <Summary> In the information processing system according to this embodiment with the above configuration, the information processing device 1 acquires image data captured by the scanning electron microscope 102 of multiple inspection targets that are expected to be regularly arranged on the substrate, detects multiple inspection targets from the acquired image data, and determines the regularity of the arrangement of the multiple inspection targets based on the acquired image data. Based on the detection results of the multiple inspection targets and the determination results of the regularity of the arrangement of the multiple inspection targets, the information processing device 1 detects defects in the substrate. Thus, the information processing system according to this embodiment is expected to accurately detect defects related to multiple inspection targets that are expected to be regularly arranged on the substrate, based on image data obtained by photographing the substrate.

[0058] Furthermore, in the information processing system according to this embodiment, the information processing device 1 receives image data as input and uses a pre-trained learning model 110 to detect multiple objects to be inspected from the image data, thereby enabling the system to accurately detect objects to be inspected from image data.

[0059] Furthermore, in the information processing system according to this embodiment, the information processing device 1 determines regularity based on the autocorrelation of acquired image data. For example, the information processing device 1 calculates a correlation value between the acquired image data and image data obtained by sliding this image data in a predetermined direction, calculates multiple correlation values ​​by changing the amount of sliding in the predetermined direction, and determines the repeating period of the arrangement of multiple objects captured in the image data as regularity based on the multiple calculated correlation values. Alternatively, the information processing device 1 may calculate the distance between adjacent objects based on the detection results of multiple objects to be inspected, and determine the repeating period of the arrangement of multiple objects captured in the image data as regularity based on the calculated distance. As a result, the information processing system according to this embodiment is expected to accurately determine the repeating period of the arrangement of multiple objects captured in the image data.

[0060] Furthermore, in the information processing system according to this embodiment, the information processing device 1 determines the regularity in the two-dimensional direction of image data captured from the surface of the substrate, and determines the regularity in the one-dimensional direction of image data captured from the cross-section of the substrate. As a result, the information processing system according to this embodiment is expected to perform defect detection suitable for the image data obtained by the capture.

[0061] <Embodiment 2> In the information processing system according to Embodiment 2, the information processing device 1 calculates the feature quantities of the object detected from the image data and performs defect detection taking into account the calculated feature quantities. Figure 11 is a schematic diagram illustrating the defect detection process performed by the information processing system according to Embodiment 2. The diagram shown in Figure 11 is an image of a substrate in which elliptical first objects and circular second objects are regularly formed, with a grid based on the regularity obtained by the regularity determination process superimposed on the image data.

[0062] The information processing device 1 according to Embodiment 2 calculates, for example, roundness as a feature quantity for each object detected by the object detection unit 11b from the image data. The information processing device 1 considers objects with a roundness lower than a threshold as elliptical first objects, and objects with a roundness higher than a threshold as circular second objects. In this example, roundness is used as the feature quantity for objects, but the feature quantity is not limited to roundness; various other quantities such as area or shape can be used.

[0063] The information processing device 1 according to Embodiment 2 generates a grid of first objects and a grid of second objects based on the period obtained by the regularity determination unit 11c determining the regularity. In Figure 11, the grid of first objects is shown with solid lines, and the grid of second objects is shown with dashed lines. The information processing device 1 can generate a grid of first objects by selecting one of the first objects as a starting point based on the object detection result and drawing vertical and horizontal lines corresponding to the period from this starting point. Similarly, the information processing device 1 can generate a grid of second objects by selecting one of the second objects as a starting point based on the object detection result and drawing vertical and horizontal lines corresponding to the period from this starting point.

[0064] The information processing device 1 according to Embodiment 2 detects defects in a substrate based on a comparison of the positions and feature quantities of a first object and a second object detected from image data with grids of the first object and the second object generated based on regularity. If no object exists at an intersection of the grid, the information processing device 1 detects the position of this intersection as a defect in the substrate. Furthermore, the information processing device 1 according to Embodiment 2 detects these positions as defects in the substrate if, for example, a second object exists at an intersection of the grid of the first object, or a first object exists at an intersection of the grid of the second object. In the example shown in Figure 11, the object detected in the lower right of the image data is at an intersection of the grid of the first object, but its feature quantity is classified as belonging to the second object, so the information processing device 1 detects it as a defect. The information processing device 1 displays, for example, the grid lines and the location of the defect, indicated by a rectangular frame, superimposed on the original image data.

[0065] In this example, objects are classified into two types, the first object and the second object, based on their features. However, this is not the only way to classify objects; they may be classified into three or more types based on their features.

[0066] In the information processing system according to Embodiment 2 with the above configuration, the information processing device 1 calculates the feature quantities of the object detected from the image data and detects defects in the substrate based on the calculated feature quantities. As a result, the information processing system according to this embodiment can distinguish between multiple objects of different types provided on the substrate and detect defects, thus improving the accuracy of defect detection.

[0067] Since the other components of the information processing system according to Embodiment 2 are the same as those of the information processing system according to Embodiment 1, the same reference numerals are used for the same components, and detailed explanations are omitted.

[0068] <Embodiment 3> Figure 12 is a schematic diagram illustrating the outline of the information processing system according to Embodiment 3. In the information processing system according to Embodiment 3, the information processing device 1 acquires image data of the substrate captured by the scanning electron microscope 102 and detects defects, and also acquires information on the conditions (so-called recipe) when substrate processing was performed on the substrate from the substrate processing device 101, and stores the defect detection results and substrate processing conditions in a database (storage unit 12) in association with each other. The information processing device 1 collects defect detection results and substrate processing conditions for a large number of substrates in the database, and the user can perform analysis on the defects of the substrate based on this information.

[0069] The information processing device 1 displays a list of various substrate processing conditions on the display unit 14 and accepts the user's selection of one condition from among them. The information processing device 1 generates correlation information between the selected condition and information regarding defects detected on the substrate, and displays the generated correlation information on the display unit 14. The lower part of Figure 12 shows an example in which the information processing device 1 generates and displays graphs as correlation information showing the correlation between the amount of gas and the number of defects for each of the three types of gases A, B, and C used in substrate processing. From these displayed graphs, the user can find correlations such as the fact that the number of defects increases with increasing gas amount for all gases, and that increasing the amount of gas A has a significant impact on the number of defects. It is expected that this knowledge can be reflected in the settings for subsequent substrate processing to suppress defects.

[0070] In this example, the correlation between the type and amount of gas used in substrate processing and the number of defects was explained as an example, but the conditions for substrate processing are not limited to these, and various conditions such as gas ratio, total gas amount, pressure, temperature, or voltage can be used. Furthermore, although the information processing device 1 displays correlation information between the substrate processing conditions and the number of defects, it is not limited to this, and may also display correlation information with, for example, the distribution of defects or the characteristic quantities described in Embodiment 2 regarding the inspected object related to the defects.

[0071] In the information processing system according to Embodiment 3 with the above configuration, the information processing device 1 acquires the substrate processing conditions performed by the substrate processing device 101 on the substrate, and outputs correlation information between the acquired substrate processing conditions and the detected defects. As a result, in the information processing system according to this embodiment, it is expected that, based on the correlation information output by the information processing device 1, the user can, for example, analyze the correlation between the substrate processing conditions and defects and make improvements to the substrate processing conditions.

[0072] Since the other components of the information processing system according to Embodiment 3 are the same as those of the information processing systems according to Embodiments 1 and 2, the same reference numerals are used for the same parts, and detailed explanations are omitted.

[0073] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims, not in the sense described above, and all modifications within the meaning and scope equivalent to the claims are intended.

[0074] The matters described in each embodiment can be combined with each other. Furthermore, the independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. In addition, the claims use a form in which claims referencing two or more other claims (multi-claim form), but are not limited to this. A form in which multi-claims referencing at least one multi-claim (multi-multi-claim) may also be used.

[0075] 1 Information Processing Device (Computer) 11 Processing Unit 11a Image Acquisition Unit 11b Object Detection Unit 11c Regularity Determination Unit 11d Defect Detection Unit 11e Display Processing Unit 11f Learning Processing Unit 12 Storage Unit 12a Program (Computer Program) 12b Model Information Storage Unit 13 Communication Unit 14 Display Unit 15 Operation Unit 99 Recording Medium 101 Substrate Processing Device 102 Scanning Electron Microscope 110 Learning Model

Claims

1. An information processing method comprising: an information processing device acquiring image data of multiple objects to be inspected arranged on a substrate; detecting the multiple objects to be inspected from the acquired image data; determining a regularity in the arrangement of the multiple objects to be inspected based on the acquired image data; and detecting a defect in the substrate based on the detection result of the multiple objects to be inspected and the determination result of the regularity in the arrangement of the multiple objects to be inspected.

2. The information processing method according to claim 1, wherein the information processing device detects the plurality of objects to be inspected from the acquired image data using a learning model that has been pre-trained to receive image data as input and output location information of objects to be inspected depicted in the image data.

3. The information processing method according to claim 1 or 2, wherein the information processing device determines the regularity based on the autocorrelation of the acquired image data.

4. The information processing method according to claim 3, wherein the information processing device calculates a correlation value between the acquired image data and image data obtained by sliding the image data in a predetermined direction, calculates multiple correlation values ​​by changing the amount of sliding in the predetermined direction, and determines the repetition period of the arrangement of the multiple objects depicted in the image data based on the multiple correlation values ​​calculated.

5. The information processing method according to claim 1 or 2, wherein the information processing device calculates the distance between adjacent objects based on the detection results of the plurality of objects to be inspected, and determines the repetition period of the arrangement of the plurality of objects captured in the image data based on the calculated distance.

6. The information processing method according to claim 1 or claim 2, comprising: calculating the characteristic quantity of each detected object to be inspected; and detecting a defect in the substrate based on the detection results of the plurality of objects to be inspected, the determination result of the regularity regarding the arrangement of the plurality of objects to be inspected, and the characteristic quantities of the plurality of objects to be inspected.

7. The information processing method according to claim 1 or claim 2, which involves acquiring the conditions for substrate processing performed on the substrate and outputting correlation information between the acquired substrate processing conditions and the detected defects in the substrate.

8. The information processing method according to claim 7, which outputs correlation information between the substrate processing conditions and the number of defects, the distribution of defects, or the characteristic quantities of the inspected object related to the defects of the substrate detected.

9. The information processing method according to claim 1 or claim 2, comprising acquiring image data of the surface of the substrate and determining the regularity in the two-dimensional direction of the plurality of objects to be inspected as captured in the acquired image data.

10. The information processing method according to claim 1 or claim 2, comprising acquiring image data of a cross-section of the substrate and determining the regularity in the one-dimensional direction of the plurality of objects to be inspected as depicted in the acquired image data.

11. A computer program that causes a computer to perform the following processes: acquire image data of multiple objects to be inspected placed on a circuit board; detect the multiple objects to be inspected from the acquired image data; determine the regularity of the arrangement of the multiple objects to be inspected based on the acquired image data; and detect defects in the circuit board based on the detection results of the multiple objects to be inspected and the determination results of the regularity of the arrangement of the multiple objects to be inspected.

12. An information processing device comprising a processing unit, wherein the processing unit acquires image data of a plurality of objects to be inspected arranged on a substrate, detects the plurality of objects to be inspected from the acquired image data, determines the regularity of the arrangement of the plurality of objects to be inspected based on the acquired image data, and detects defects in the substrate based on the detection results of the plurality of objects to be inspected and the determination results of the regularity of the arrangement of the plurality of objects to be inspected.

Images